Send events to Google Cloud Storage
Stream batched NDJSON events into a customer-owned GCS bucket, same delivery model as S3, on GCP.
What is Google Cloud Storage?
Google Cloud Storage is GCP's object storage service, the GCP equivalent of S3. As a destination, it gives a customer the same durable, queryable raw-events archive in their own GCP project, billed to them and governed by their IAM. Pushrail's GCS adapter writes the same NDJSON batches with the same key layout as the S3 adapter, so customers running multi-cloud get a consistent shape on both sides.
Why deliver events to Google Cloud Storage
- Customer owns the data in their own GCP project, no shared bucket, no cross-cloud copy.
- BigQuery loves it, GCS is BigQuery's native staging area for external tables and load jobs.
- Mirrors the S3 model, multi-cloud customers get the same layout in both clouds.
- Pairs naturally with Dataflow, BigQuery, and Looker for downstream analytics.
How Pushrail delivers events to Google Cloud Storage
The GCS adapter writes the same batched NDJSON files as the S3 adapter, partitioned by event type and hour: `gs://<bucket>/<prefix>/event_type=order.completed/dt=2026-05-26/hour=14/part-<sha>.ndjson`. Batches flush on size threshold or time window. Object names are deterministic, so replays are idempotent at the object level.
Auth and credentials
Customers authorize Pushrail using Workload Identity Federation (recommended, no service-account keys exchanged) or a service-account JSON key. Workload Identity binds Pushrail's GCP identity to a customer-side service account; the customer revokes access by removing the binding. Service-account keys, if used, are stored encrypted at rest with AES-256 and can be rotated in-place from the dashboard.
Batching, retries, and replay
Default batch is 1 MB or 1,000 events with a 60-second flush window. Transient GCS errors (5xx, rate-limit) retry with exponential backoff; permanent errors (403, 404) fail straight to the DLQ. Replay re-emits to the same deterministic object names, so re-running a window does not duplicate rows for BigQuery external tables that scan the prefix.
Example payload
Pushrail accepts the canonical event shape on POST /v1/events. Below is the ingestion request your service makes.
{
"eventType": "order.completed",
"occurredAt": "2026-05-26T14:21:08.493Z",
"source": "billing-service",
"customerExternalId": "acct_8K2zRq",
"idempotencyKey": "order_38a91f-completed",
"correlationId": "req_4f30b2",
"payload": {
"orderId": "ord_38a91f",
"amount": 12900,
"currency": "USD",
"items": [
{ "sku": "PR-PRO-MONTHLY", "qty": 1, "price": 12900 }
]
},
"metadata": {
"tier": "pro",
"region": "us-east-1"
}
}Example configuration
The fields your customer fills in to point Pushrail at their Google Cloud Storage setup.
{
"type": "GCS",
"name": "Customer GCS archive",
"bucket": "acme-pushrail-events",
"keyPrefix": "raw/pushrail/",
"auth": {
"mode": "WORKLOAD_IDENTITY",
"serviceAccountEmail": "pushrail-writer@acme-prod.iam.gserviceaccount.com",
"audience": "//iam.googleapis.com/projects/123/locations/global/workloadIdentityPools/pushrail/providers/pushrail-oidc"
},
"batchSizeMb": 1,
"flushIntervalSec": 60
}Common use cases
- Land raw events directly in a customer's GCP project for downstream BigQuery work.
- Provide a GCS-backed audit archive for compliance-sensitive customers.
- Feed a customer's Dataflow pipeline without operating Pub/Sub on their behalf.
- Multi-cloud parity, same shape as the customer's S3 layout in AWS.
Related destinations
Deliver events as NDJSON files into a customer-owned S3 bucket, batched, partitioned, and idempotent.
Batched NDJSON event delivery into a customer-owned Azure Storage container, parity with S3 and GCS.
Stream events directly into a customer's BigQuery dataset, table per event type or single wide table.
Frequently asked questions
Is GCS delivery the same as S3 delivery?
Yes. The GCS adapter writes the same batched NDJSON files with the same key layout as the S3 adapter, so multi-cloud customers get a consistent shape in both clouds. You send one canonical event to Pushrail and it handles the rest.
Whose credentials are used to write to GCS?
The customer's. They authorize Pushrail with Workload Identity Federation (recommended, no key exchanged) or a service-account JSON key. Keys are stored encrypted at rest with AES-256 and can be rotated in-place from the dashboard; revoking the Workload Identity binding cuts off access.
Can I load these files into BigQuery?
Yes. GCS is BigQuery's native staging area for external tables and load jobs, so the NDJSON files Pushrail writes can be read directly by BigQuery, Dataflow, and Looker.
What happens if a write fails?
Transient errors (5xx, rate limits) retry with exponential backoff; permanent errors (403, 404) fail straight to the dead-letter queue. Every attempt is recorded in the delivery logs. Replay re-emits to the same deterministic object names, so re-running a window does not duplicate rows for external tables scanning the prefix.