Google BigQuery FAQ
Here you will find a list of common questions regarding data warehouse jobs
Arize can ingest data from BigQuery views or native tables. Use cloud storage to ingest external tables.
change_timestampfield is used for Arize to track which new rows need to be imported as we constantly sync new data. This column type currently must be the TIMESTAMP type. Values in the
change_timestampfield should also be unique.
2023-01-27 17:57:59.932798 UTC
2023-02-15 12:01:00.134899 UTC
2023-02-15 12:15:00.134899 UTC
When adding new data, Arize will only ingest data “newer” (based on
change_timestampcolumn) than what's currently ingested.
Deleted or updated data in your data warehouse will not be reflected by Arize. Only new records added to your data warehouse table will be ingested.
If your query failed due to an issue with a specific row or set of rows, fix the error and append the data to your table again using an updated
Any queries that fail to run for a given table import job will be displayed within the job's details.
Sample Query Failure in Job Details
The QueryID that is displayed corresponds to a BigQuery job id. The BigQuery job id can be used to retrieve the execution of a given query and lookup the specific query to rerun.
There are two ways to look up additional information about a BigQuery job.
- 1.BigQuery CLI
bq show -j --format=prettyjson <project id>:<region>.<job id>
The results of this command will display the query that ran for the given job, query parameters that were applied, error statuses, and information about its execution cost.
- 2.BigQuery UI
Job statuses can be retrieved for a given BigQuery project.
After searching for and selecting the given job, BigQuery will provide information about the query that was run and when it was run. However, it does not fill in the values of any query parameters that are used in the query. This information is provided by the Job Details page in Arize, and can be used to recreate the query.
Last modified 2mo ago