Send events to BigQuery
Stream events directly into a customer's BigQuery dataset, table per event type or single wide table.
What is BigQuery?
BigQuery is Google Cloud's serverless data warehouse. As a destination, it lets your customers analyze your product events in seconds without operating a pipeline, events arrive in their dataset, queryable from any SQL tool, BI dashboard, or notebook they already use. Pushrail's BigQuery adapter streams events row-by-row using the Storage Write API so latency stays low and dedup is exactly-once.
Why deliver events to BigQuery
- Query in seconds, analysts run SQL against your events the same day the integration goes live.
- Customer owns the warehouse, billed to their GCP project, governed by their IAM.
- Pairs with Looker, Looker Studio, Mode, Hex, and every other BI tool that talks BigQuery.
- Schema-aware, Pushrail can fan out one table per event type or write to a single wide table with the payload as JSON, whichever the customer prefers.
How Pushrail delivers events to BigQuery
The BigQuery adapter uses the Storage Write API in committed mode for exactly-once inserts. By default Pushrail creates one table per `eventType` with columns for the canonical envelope (`event_id`, `event_type`, `occurred_at`, `customer_external_id`, `source`, `payload` as JSON, `metadata` as JSON, `received_at`). Customers can also choose a single-table mode where every event lands in one `events` table with the same envelope columns. Schema migrations are append-only, if a new event type appears, Pushrail creates its table; if a payload field is added it lives inside the JSON column.
Auth and credentials
Customers grant Pushrail BigQuery write access via Workload Identity Federation (preferred) or a service-account JSON key. Required IAM roles are `roles/bigquery.dataEditor` on the destination dataset and `roles/bigquery.jobUser` on the project. Service-account keys are stored encrypted at rest; Workload Identity binds Pushrail's GCP identity to a customer-side service account with no exchanged secret. Rotations happen in-place from the dashboard.
Batching, retries, and replay
Streaming Storage Writes are real-time at the row level (no batching window). Transient errors retry with exponential backoff. Permanent errors (schema mismatch, dataset-not-found) move to the DLQ for inspection. Replay re-streams the same rows; the adapter dedupes by `idempotencyKey + eventType` so a re-run of a window does not duplicate rows.
Example payload
Pushrail accepts the canonical event shape on POST /v1/events. Below is the ingestion request your service makes.
{
"eventType": "order.completed",
"occurredAt": "2026-05-26T14:21:08.493Z",
"source": "billing-service",
"customerExternalId": "acct_8K2zRq",
"idempotencyKey": "order_38a91f-completed",
"correlationId": "req_4f30b2",
"payload": {
"orderId": "ord_38a91f",
"amount": 12900,
"currency": "USD",
"items": [
{ "sku": "PR-PRO-MONTHLY", "qty": 1, "price": 12900 }
]
},
"metadata": {
"tier": "pro",
"region": "us-east-1"
}
}Example configuration
The fields your customer fills in to point Pushrail at their BigQuery setup.
{
"type": "BIGQUERY",
"name": "Customer BigQuery warehouse",
"projectId": "acme-analytics-prod",
"datasetId": "pushrail_events",
"tableStrategy": "TABLE_PER_EVENT_TYPE",
"auth": {
"mode": "WORKLOAD_IDENTITY",
"serviceAccountEmail": "pushrail-writer@acme-prod.iam.gserviceaccount.com"
}
}Common use cases
- Productize 'we'll deliver your events to your BigQuery' as an enterprise data-export feature.
- Replace ad-hoc CSV exports, customers run SQL on the data directly instead of waiting for a file.
- Feed a customer's BI tool (Looker, Mode, Hex) with live product event data.
- Power customer-built analytics on top of your raw events.
Related destinations
Continuous ingest into a customer-owned Snowflake table via Snowpipe-style staging.
High-throughput row-shaped event delivery into a customer's ClickHouse cluster, one row per event.
Stream batched NDJSON events into a customer-owned GCS bucket, same delivery model as S3, on GCP.
Frequently asked questions
Do I need to build a BigQuery integration to use this?
No. Pushrail delivers events into your customer's BigQuery dataset for you, you send one canonical event to Pushrail's ingest API, and the BigQuery adapter handles the Storage Write API, table creation, and schema mapping.
Whose credentials are used to write to BigQuery?
The customer's. They grant write access via Workload Identity Federation or a service-account JSON key, scoped to roles/bigquery.dataEditor on the dataset and roles/bigquery.jobUser on the project. Keys are stored encrypted at rest and rotated in-place from the dashboard.
One table per event type or a single table?
Either. By default Pushrail creates one table per eventType with columns for the canonical envelope, but the customer can choose a single-table mode where every event lands in one events table with the payload as JSON.
What happens if a BigQuery write fails?
Transient errors retry with exponential backoff; permanent errors like a schema mismatch or missing dataset move to the dead-letter queue for inspection. Every attempt is recorded in the delivery logs. Replay re-streams the same rows and dedupes by idempotency key plus event type, so re-running a window does not duplicate data.