feat(bigquery): validate Arrow queries before sending them

[ohmayr]

Validates query_and_wait(query_results_format=ARROW) before sending the query, so missing dependencies or unsupported options fail up front instead of after the query runs.

• Raises ValueError for compression_codec without ARROW, and for ARROW with WRITE_APPEND.
• Checks for pyarrow and a google-cloud-bigquery-storage that has the read_rows() option from feat(bigquery-storage): add Arrow options and cancel to read_rows #18643, by looking for ReadRowsStream.cancel().
• Installs the in-repo google-cloud-bigquery-storage in the system nox session until it's released.
• Adds unit tests for each check and a system test for WRITE_APPEND.

[gemini-code-assist]

Code Review

This pull request introduces validation for ARROW query results in query_and_wait to ensure that required dependencies (pyarrow and google-cloud-bigquery-storage) are installed and support the necessary features, and that unsupported configurations like WRITE_APPEND are rejected before sending the query. Feedback on the changes suggests importing the inspect module lazily inside the supports_arrow_serialization_options property to avoid overhead during library startup, and explicitly passing project and credentials when instantiating bigquery.Client in system tests to ensure reliability.

[gemini-code-assist]

Avoid importing inspect at the module level because _versions_helpers is imported during the startup of google-cloud-bigquery (via _arrow_query -> _job_helpers -> client). inspect is a relatively heavy module to import and is only needed when checking Arrow serialization options. Moving the import inside the supports_arrow_serialization_options property will improve the import/startup time of the library for all users who do not use Arrow queries.

Suggested change

	import inspect
	from typing import Any
	from typing import Any

[ohmayr]

Fair point, _versions_helpers is on the import path for everyone. Moved the import into the property in 6e55670.

[gemini-code-assist]

Import inspect lazily inside the property to avoid importing it at the module level during startup.

Suggested change

	def supports_arrow_serialization_options(self) -> bool:
	"""True if ``BigQueryReadClient.read_rows()`` accepts
	``arrow_serialization_options``, which ``ARROW`` query results need.
	Checks the signature instead of a version number, so the result is
	also correct for a build of google-cloud-bigquery-storage that
	hasn't been released yet.
	"""
	from google.cloud import bigquery_storage
	read_rows = bigquery_storage.BigQueryReadClient.read_rows
	return "arrow_serialization_options" in inspect.signature(read_rows).parameters
	def supports_arrow_serialization_options(self) -> bool:
	"""True if BigQueryReadClient.read_rows() accepts
	arrow_serialization_options, which ARROW query results need.
	Checks the signature instead of a version number, so the result is
	also correct for a build of google-cloud-bigquery-storage that
	hasn't been released yet.
	"""
	import inspect
	from google.cloud import bigquery_storage
	read_rows = bigquery_storage.BigQueryReadClient.read_rows
	return "arrow_serialization_options" in inspect.signature(read_rows).parameters

[ohmayr]

Done in the same commit (6e55670).

[gemini-code-assist]

In system tests, instantiating bigquery.Client() without explicit project and credentials arguments relies on ambient environment variables (like GOOGLE_CLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS). To ensure the test runs reliably and targets the correct test project/credentials configured for the test suite, it is safer to pass the project and credentials from the existing bigquery_client fixture.

Suggested change

	with bigquery.Client(default_query_job_config=append) as client:
	with bigquery.Client(
	project=bigquery_client.project,
	credentials=bigquery_client.credentials,
	default_query_job_config=append,
	) as client:

[ohmayr]

The bigquery_client fixture in conftest.py is itself a plain bigquery.Client(), so this picks up the same project and credentials as the rest of the suite. I'd rather match the fixture than special-case this test.

[tswast]

No action required: I'm wary of adding client-side checks like this. What if we add another results format that also takes a compression_codec? Unlikely, but something to ponder.

[ohmayr]

The reason we check this client-side is that compression_codec isn't a top-level jobs.query field. We only serialize it into arrowSerializationOptions.bufferCompression when query_results_format=ARROW. Without this check, passing compression_codec="LZ4_FRAME" on a default JSON query wouldn't be sent on the wire at all and would be silently ignored. If another format supports compression_codec in the future, relaxing this check to allow that format will be backward-compatible.

[ohmayr]

Adding a Do Not Merge so that I can remove the noxfile hack to keep the CI green which installs the bigquery-storage package from source. Since there's a scheduled release tomorrow, I will merge it then. Should not block the review for the remaining PRs.