CARVIEW |
You can manage data profile scans and data quality scans across your project by using the Metadata curation page in the Google Cloud console. For more information, see Profile your data and Scan for data quality issues. This feature is generally available (GA).
Changed
BigQuery ML has improved throughput by more than 100x for the following generative AI functions:
Actual performance varies based on the number of input and output tokens in the request, but a typical 6-hour job can now process millions of rows. For more information, see Generative AI functions.
Changed
BigQuery ML now can automatically detect model quota increases in Vertex AI, and automatically adjusts the quota for any BigQuery ML functions that use those models. You no longer need to email the BigQuery ML team to increase model quota.
Feature
You can now use continuous queries to export BigQuery data to Spanner in real time. This feature is in Preview.
]]>The Gemini for Google Cloud API (cloudaicompanion.googleapis.com) is now enabled by default for most BigQuery projects. Exceptions include projects where customers have opted out, and those linked to accounts based in EMEA regions including BigQuery Europe, Middle East, and Africa regions.
]]>A weekly digest of client library updates from across the Cloud SDK.
Node.js
Changes for @google-cloud/bigquery
8.1.1 (2025-07-23)
Bug Fixes
Python
Changes for google-cloud-bigquery
3.35.1 (2025-07-21)
Documentation
Feature
You can now associate data policies directly on columns. This feature enables direct database administration for controlling access and applying masking and transformation rules at the column level. This feature is in Preview.
]]>You can now use the
VECTOR_INDEX.STATISTICS
function to calculate how much an indexed table's data has drifted between when a
vector index was created and the present. If table data has changed enough
to require a vector index rebuild, you can use the
ALTER VECTOR INDEX REBUILD
statement
to rebuild the vector index. This feature is in Preview.
Feature
Access Transparency supports BigQuery data preparation in the GA stage.
Feature
The CREATE EXTERNAL TABLE
and LOAD DATA
statements now support the following options in Preview:
null_markers
: define the strings that representNULL
values in CSV files.source_column_match
: specify how loaded columns are matched to the schema. You can match columns by position or by name.
Feature
You can now use the MATCH_RECOGNIZE
clause in your SQL queries to filter and aggregate matches across rows in a table. This feature is in Preview.
A weekly digest of client library updates from across the Cloud SDK.
Java
Changes for google-cloud-bigquery
2.53.0 (2025-07-14)
Features
- bigquery: Add OpenTelemetry support to BigQuery rpcs (#3860) (e2d23c1)
- bigquery: Add support for custom timezones and timestamps (#3859) (e5467c9)
- Next release from main branch is 2.53.0 (#3879) (c47a062)
Bug Fixes
Dependencies
- Update dependency com.google.api.grpc:proto-google-cloud-bigqueryconnection-v1 to v2.69.0 (#3870) (a7f1007)
- Update dependency com.google.apis:google-api-services-bigquery to v2-rev20250615-2.0.0 (#3872) (f081589)
- Update dependency com.google.cloud:sdk-platform-java-config to v3.50.1 (#3878) (0e971b8)
Documentation
Python
Changes for google-cloud-bigquery
3.35.0 (2025-07-15)
Features
- Add null_markers property to LoadJobConfig and CSVOptions (#2239) (289446d)
- Add total slot ms to RowIterator (#2233) (d44bf02)
- Add UpdateMode to update_dataset (#2204) (eb9c2af)
- Adds dataset_view parameter to get_dataset method (#2198) (28a5750)
- Adds date_format to load job and external config (#2231) (7d31828)
- Adds datetime_format as an option (#2236) (54d3dc6)
- Adds source_column_match and associated tests (#2227) (6d5d236)
- Adds time_format and timestamp_format and associated tests (#2238) (371ad29)
- Adds time_zone to external config and load job (#2229) (b2300d0)
Bug Fixes
- Adds magics.context.project to eliminate issues with unit tests … (#2228) (27ff3a8)
- Fix rows returned when both start_index and page_size are provided (#2181) (45643a2)
- Make AccessEntry equality consistent with from_api_repr (#2218) (4941de4)
- Update type hints for various BigQuery files (#2206) (b863291)
Documentation
Feature
You can now use the DISTINCT
pipe operator to select distinct rows from a table in your pipe syntax queries. This feature is generally available (GA).
You can now use the WITH
pipe operator to define common table expressions in your pipe syntax queries. This feature is generally available (GA).
Feature
You can now use named windows in your pipe syntax queries. This feature is generally available (GA).
]]>You can now add comments to notebooks, data canvases, data preparation files, or saved queries. You can also reply to existing comments or get a link to them. This feature is in Preview.
Feature
You can now create BigQuery ML models by using the Google Cloud console user interface. This feature is in Preview.
]]>You can now commercialize your BigQuery sharing listings on Google Cloud Marketplace. This feature is generally available (GA).
Feature
You can flatten JSON columns in BigQuery data preparation with a single operation. This feature is generally available (GA).
]]>Starting August 1, 2025, GoogleSQL will become the default dialect for queries run from the command line interface (CLI) or API. To use LegacySQL, you will need to explicitly specify it in your requests or set the configuration setting default_sql_dialect_option
to 'default_legacy_sql'
at the project or organization level.
You can now use your Google Account user credentials to authorize the execution of a data preparation in development. For more information, see Manually run a data preparation in development. This feature is in preview.
]]>You can now update a Cloud KMS encryption key by updating the table with the same key. This feature is generally available (GA).
Feature
You can use the @@location
system variable to set the location in which to run a query. This feature is generally available (GA).
Feature
BigQuery now supports the following Apache Hadoop migration features in Preview:
- Use the
dwh-migration-dumper
tool to migrate the metadata necessary for a Hadoop permissions and data migration. - Migrate permissions from Apache Hadoop, Apache Hive, and Ranger HDFS to BigQuery.
- Migrate tables from a HDFS data lake to Google Cloud.
A weekly digest of client library updates from across the Cloud SDK.
Java
Changes for google-cloud-bigquery
2.52.0 (2025-06-25)
Features
- bigquery: Integrate Otel in client lib (#3747) (6e3e07a)
- bigquery: Integrate Otel into retries, jobs, and more (#3842) (4b28c47)
Bug Fixes
Dependencies
- Remove version declaration of open-telemetry-bom (#3855) (6f9f77d)
- Update dependency com.google.api.grpc:proto-google-cloud-bigqueryconnection-v1 to v2.66.0 (#3835) (69be5e7)
- Update dependency com.google.api.grpc:proto-google-cloud-bigqueryconnection-v1 to v2.68.0 (#3858) (d4ca353)
- Update dependency com.google.cloud:sdk-platform-java-config to v3.49.2 (#3853) (cf864df)
- Update dependency com.google.cloud:sdk-platform-java-config to v3.50.0 (#3861) (eb26dee)
- Update dependency io.opentelemetry:opentelemetry-bom to v1.51.0 (#3840) (51321c2)
- Update ossf/scorecard-action action to v2.4.2 (#3810) (414f61d)
Feature
You can now create and manage scheduled notebooks using the Schedule details pane in BigQuery Studio. This feature is generally available (GA).
]]>You can now use the
use the PARTITION BY
clause of the
CREATE VECTOR INDEX
statement to partition TreeAH vector indexes. Partitioning enables partition pruning and can decrease I/O costs. This feature is in preview.
Feature
BigQuery search indexes provide free index management until your organization reaches the limit in a given region. You can now use the INFORMATION_SCHEMA.SEARCH_INDEXES_BY_ORGANIZATION
view to understand your current consumption towards that limit, broken down by projects and tables. This feature is generally available (GA).
You can now use the Apache Iceberg REST catalog in BigLake metastore to create interoperability between your query engines by allowing your open source engines to access Iceberg data in Cloud Storage. This feature is in Preview.
Feature
Colab Enterprise notebooks in BigQuery let you do the following in Preview:
]]>You can now publish the results of a data quality scan as Dataplex Universal Catalog metadata. Previously, data quality scan results were published only to the Google Cloud console. The latest results are saved to the entry that represents the source table. You can view the results in the Google Cloud console. If you want to enable catalog publishing for an existing data quality scan, you must edit the scan and re-enable the publishing option. This feature is generally available (GA).
Feature
You can now use data insights to have Gemini generate table and column descriptions from table metadata. This feature is in Preview.
]]>In BigQuery ML, you can now forecast multiple time series at once by using the TIME_SERIES_ID_COL
option that is available in ARIMA_PLUS_XREG
multivariate time series models. Try this feature with the Forecast multiple time series with a multivariate model tutorial. This feature is generally available (GA).
Feature
You can now manage IAM tags on BigQuery datasets and tables using SQL. This feature is generally available (GA).
Feature
The BigQuery migration assessment is now available for workflows that use Cloudera and Apache Hadoop. This feature is in Preview.
Feature
The Merchant Center best sellers report supports multi-client accounts (MCAs). If you have an MCA, you can use the aggregator_id
to query the tables. The BestSellersEntityProductMapping
table maps the best-selling entities to the products in the sub-accounts' inventory. This provides a consolidated view of best-selling products, which you can then join with product data for more detailed insights. This feature is generally available (GA).
Feature
BigQuery now offers the following Gemini-enhanced SQL translation features:
- Create Gemini-based configuration YAML files to generate AI suggestions for batch or interactive SQL translations. This feature is now generally available (GA).
- After making a batch SQL translation, review your translation output, including Gemini-based suggestions, using the code tab and configuration tab. This feature is now generally available (GA).
- When making an interactive SQL translation, create and apply Gemini-enhanced translation rules to customize your SQL inputs. This feature is in Preview.
Dark theme is now available for BigQuery in Preview. To enable the dark theme, in the Google Cloud console, click Settings and utilities > Preferences. In the navigation menu, click Appearance, and then select your color theme and click Save.
]]>The following GoogleSQL functions are now available in preview:
- The
ARRAY_FIRST
function returns the first element of the input array. - The
ARRAY_LAST
function returns the last element of the input array. - The
ARRAY_SLICE
function returns an array that contains consecutive elements from the input array.
An updated version of the ODBC driver for BigQuery is now available.
Feature
For supported Gemini models, you can now use Vertex AI Provisioned Throughput with the ML.GENERATE_TEXT
and AI.GENERATE
functions to provide consistent high throughput for requests.
This feature is generally available (GA).
]]>A weekly digest of client library updates from across the Cloud SDK.
Java
Changes for google-cloud-bigquery
2.51.0 (2025-06-06)
Features
- bigquery: Job creation mode GA (#3804) (a21cde8)
- bigquery: Support Fine Grained ACLs for Datasets (#3803) (bebf1c6)
Dependencies
- Rollback netty.version to v4.1.119.Final (#3827) (94c71a0)
- Update dependency com.google.api.grpc:proto-google-cloud-bigqueryconnection-v1 to v2.65.0 (#3787) (0574ecc)
- Update dependency com.google.apis:google-api-services-bigquery to v2-rev20250511-2.0.0 (#3794) (d3bf724)
- Update dependency com.google.cloud:sdk-platform-java-config to v3.49.0 (#3811) (2c5ede4)
Feature
You can reference Iceberg external tables in materialized views instead of migrating that data to BigQuery-managed storage. This feature is generally available (GA).
]]>The organization-level configuration settings for default_sql_dialect_option
and query_runtime
are unsupported.
You can now use the BigQuery advanced runtime to improve query execution time and slot usage. This feature is in Preview.
Feature
BigQuery tables for Apache Iceberg have been renamed BigLake tables for Apache Iceberg in BigQuery. This feature is now generally available (GA).
Feature
BigQuery metastore has been renamed BigLake metastore and is now generally available (GA). The feature formerly known as BigLake metastore has been renamed BigLake metastore (classic).
]]>A weekly digest of client library updates from across the Cloud SDK.
Node.js
Changes for @google-cloud/bigquery
8.1.0 (2025-05-29)
Features
Go
Changes for bigquery/storage/apiv1beta1
1.69.0 (2025-05-27)
Features
- bigquery/analyticshub: Add support for Analytics Hub & Marketplace Integration (2aaada3)
- bigquery/analyticshub: Adding allow_only_metadata_sharing to Listing resource (2aaada3)
- bigquery/analyticshub: Adding CommercialInfo message to the Listing and Subscription resources (2aaada3)
- bigquery/analyticshub: Adding delete_commercial and revoke_commercial to DeleteListingRequest and RevokeSubscriptionRequest (2aaada3)
- bigquery/analyticshub: Adding DestinationDataset to the Subscription resource (2aaada3)
- bigquery/analyticshub: Adding routine field to the SharedResource message (2aaada3)
- bigquery: Add support for dataset view and update modes (#12290) (7c1f961)
- bigquery: Job creation mode GA (#12225) (1d8990d)
Python
Changes for google-cloud-bigquery
3.34.0 (2025-05-27)
Features
Bug Fixes
Documentation
Feature
In the navigation menu, you can now go to Settings and select Configuration settings to customize the BigQuery Studio experience for users within the selected project or organization. This is achieved by showing or hiding user interface elements. This feature is in preview.
Feature
BigQuery now supports using Spanner external datasets with authorized views, authorized routines, and Cloud resource connections. This feature is generally available (GA).
Feature
The CREATE EXTERNAL TABLE
and LOAD DATA
statements now support the following options in preview:
time_zone
: specify a time zone to use when loading datadate_format
,datetime_format
,time_format
, andtimestamp_format
: define how date and time values are formatted in your source files
Feature
In the BigQuery console, in the Welcome tab, you can now try the Apache Spark demo notebook that walks you through the basics of Spark notebook and showcases serverless Spark in BigQuery. This feature is generally available (GA).
]]>You can now use the dbt-bigquery
adapter to run Python code that's defined in BigQuery DataFrames. For more information, see Use BigQuery DataFrames in dbt. This feature is in preview.
Feature
You can now use your Google Account user credentials to authorize the creation, scheduling, and running of pipelines as well as the scheduling of notebooks and data preparations. For more information, see Create a pipeline schedule. This feature is in preview.
Feature
You can now create event-driven transfers when transferring data from Cloud Storage to BigQuery. Event-driven transfers can automatically trigger transfer runs when data in your Cloud Storage bucket has been modified or added. This feature is generally available (GA).
]]>You can now create a serverless Spark session and run PySpark code in a BigQuery notebook. This feature is generally available (GA).
Feature
Column metadata indexing is now available for both BigQuery tables and external tables. This feature is generally available (GA).
]]>You can now share Pub/Sub streaming data through BigQuery sharing with additional client libraries support and provider usage metrics. This feature is generally available (GA).
Feature
BigQuery offers optional job creation mode to speed up small queries that you use in your dashboards, data exploration, and other workflows. This mode automatically optimizes eligible queries and uses a cache to improve latency. This feature is generally available (GA).
]]>A weekly digest of client library updates from across the Cloud SDK.
Java
Changes for google-cloud-bigquery
2.50.1 (2025-05-16)
Dependencies
- Update dependency com.google.cloud:sdk-platform-java-config to v3.48.0 (#3790) (206f06d)
- Update netty.version to v4.2.1.final (#3780) (6dcd858)
Documentation
Python
Changes for google-cloud-bigquery
3.33.0 (2025-05-19)
Features
- Add ability to set autodetect_schema query param in update_table (#2171) (57f940d)
- Add dtype parameters to to_geodataframe functions (#2176) (ebfd0a8)
- Support job reservation (#2186) (cb646ce)
Bug Fixes
When you migrate Teradata data to BigQuery using the BigQuery Data Transfer Service, you can now specify the outputs of the BigQuery translation engine to use as schema mapping. This feature is in preview.
Feature
You can use custom constraints with Organization Policy to provide more granular control over specific fields for some BigQuery resources. This feature is available in Preview.
Changed
Starting September 15 2025, the bigquery.datasets.getIamPolicy
IAM permission is required to view a dataset's access controls and to query the
INFORMATION_SCHEMA.OBJECT_PRIVILEGES
view. The bigquery.datasets.setIamPolicy
permission is required to update a
dataset's access controls or to create a dataset with access controls using the
API. For more information on this change and how to opt into early enforcement, see Changes to dataset-level access controls.
Feature
When you Set up Gemini in BigQuery you are now prompted to grant the BigQuery Studio User and BigQuery Studio Admin roles. These roles now include permission to use Gemini in BigQuery features. This feature is generally available (GA).
Feature
You can select multiple columns and perform data preparation tasks on them, including dropping columns. For more information, see Prepare data with Gemini. This feature is generally available (GA).
]]>You are now able to set access controls on routines. This feature is in Preview.
Changed
You can now perform supervised tuning on a BigQuery ML remote model based on a Vertex AI gemini-2.0-flash-001
or gemini-2.0-flash-lite-001
model.
Continuous queries let you build long-lived, continuously processing SQL statements that can analyze, process, and perform machine learning (ML) inference on incoming data in BigQuery in real time.
- To monitor your continuous queries, you can use a custom job ID prefix to simplify filtering or view metrics specific to continuous queries in Cloud Monitoring.
- Continuous queries can use slot autoscaling to dynamically scale allocated capacity to accommodate your workload.
This feature is generally available (GA).
Feature
Spanner now supports cross regional federated queries from BigQuery which allow BigQuery users to query Spanner tables from regions other than their BigQuery region. Users don't incur Spanner network egress charges during the preview period. This feature is in preview.
Libraries
A weekly digest of client library updates from across the Cloud SDK.
Go
Changes for bigquery/storage/apiv1beta1
1.68.0 (2025-05-12)
Features
- bigquery/analyticshub: Support new feature Sharing Cloud Pubsub Streams via AH (GA) and Subscriber Email logging feature (#11908) (a21d596)
- bigquery/storage: Increased the number of partitions can be written in a single request (43bc515)
- bigquery: Add performance insights (#12101) (aef68ab)
- bigquery: Add some missing fields to BigQuery stats (#12212) (77b08e8)
- bigquery: Add WriteTruncateData write disposition (#12013) (b1126a3)
- bigquery: New client(s) (#12228) (f229bd9)
- bigquery: Support managed iceberg tables (#11931) (35e0774)
- bigquery: Support per-job reservation assignment (#12078) (c9cebcc)
Bug Fixes
- bigquery: Cache total rows count (#12230) (202dce0), refs #11874 #11873
- bigquery: Parse timestamps with timezone info (#11950) (530d522)
- bigquery: Update google.golang.org/api to 0.229.0 (3319672)
- bigquery: Upgrade gRPC service registration func (7c01015)
Documentation
- bigquery/storage: Updated the number of partitions (from 100 to 900) can be inserted, updated and deleted in a single request (43bc515)
Python
Changes for google-cloud-bigquery
3.32.0 (2025-05-12)
Features
- Add dataset access policy version attribute (#2169) (b7656b9)
- Adds preview support for incremental results (#2145) (22b80bb)
- Add WRITE_TRUNCATE_DATA enum (#2166) (4692747)
- Adds condition class and assoc. unit tests (#2159) (a69d6b7)
- Support BigLakeConfiguration (managed Iceberg tables) (#2162) (a1c8e9a)
- Update the AccessEntry class with a new condition attribute and unit tests (#2163) (7301667)