Skip to content

Conversation

thomaschow
Copy link

@thomaschow thomaschow commented Apr 14, 2025

  • When using Spark35BigQueryTable, the Spark35BigQueryWriteBuilder supports V1Write but BigQueryWriteBuilder is not an instance of V1Write. We need to return a V1Write instance for truncate.

@davidrabinowitz
Copy link
Member

/gcbrun

@davidrabinowitz
Copy link
Member

Thanks for the contribution! It failed on

Step #1 - "init": org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project spark-3.5-bigquery-lib: Compilation failure
Step #1 - "init": /workspace/spark-bigquery-dsv2/spark-3.5-bigquery-lib/src/main/java/com/google/cloud/spark/bigquery/v2/Spark35BigQueryWriteBuilder.java:[44,10] cannot find symbol
Step #1 - "init":   symbol:   class WriteBuilder
Step #1 - "init":   location: class com.google.cloud.spark.bigquery.v2.Spark35BigQueryWriteBuilder

@davidrabinowitz
Copy link
Member

/gcbrun

@davidrabinowitz
Copy link
Member

Please run ./mvnw spotless:apply -Pall and push the results

@davidrabinowitz
Copy link
Member

/gcbrun

@davidrabinowitz davidrabinowitz self-requested a review April 18, 2025 17:19
@isha97
Copy link
Member

isha97 commented Apr 22, 2025

/gcbrun

@thomaschow
Copy link
Author

Hi @isha97, is there anything outstanding I should do for this PR? Please help me understand the failing checks, thanks!

@davidrabinowitz
Copy link
Member

Notice that SupportsOverwrite is not fully implemented, specifically the overwrite(Filter[] filters) is not implemented yet.

@thomaschow
Copy link
Author

Hi @davidrabinowitz thank you for the response!

Notice that SupportsOverwrite is not fully implemented, specifically the overwrite(Filter[] filters) is not implemented yet.

Wasn't this the case prior? Since the implementation of that comes from BigQueryWriteBuilder which just returns null. Do we need it after this change?

hemanthbonthu pushed a commit to hemanthbonthu/spark-bigquery-connector that referenced this pull request Jul 22, 2025
…1353)

* Add new flag 'fs.gs.cloud.logging.enable' to export logs to cloud logging.

* Add LoggingInterceptor to intercept log records sent to flogger and write those log records to Cloud logging.
Also added unit test

* Add LoggingInterceptor to intercept log records sent to flogger and write those log records to Cloud logging.
Also added unit test

* Configure logging configuration and add it to root logger handler.

* Configure logging configuration and add it to root logger handler.

* Merge remote-tracking branch 'origin/exp' into exp

* Use credentials to create logging client. Structure log entry to include class and method name

* Use credentials to create logging client. Structure log entry to include class and method name

* Fix unit tests.

* Add java doc, sort configuration and resolve other review comments.

* Add java doc, sort configuration and resolve other review comments.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants