Send events to Amazon Kinesis
Ordered shard delivery into a customer-owned Kinesis Data Stream, configurable partition keys, native AWS SDK.
What is Amazon Kinesis?
Amazon Kinesis Data Streams is AWS's managed streaming service. For AWS-native customers it is the obvious choice for a real-time event feed: it integrates with Lambda triggers, Kinesis Data Analytics, Firehose, and EventBridge Pipes without operating Kafka. Pushrail's adapter uses the native AWS SDK to write events with explicit ordering controls.
Why deliver events to Amazon Kinesis
- AWS-native, no Kafka cluster to operate, integrates with the rest of the customer's AWS stack.
- Lambda or Kinesis Data Analytics consume the stream directly.
- Shard-level ordering, events for the same partition key land in the same shard and preserve order.
- Firehose can fan-out to S3, Redshift, or OpenSearch from the same stream.
How Pushrail delivers events to Amazon Kinesis
The Kinesis adapter calls `PutRecords` with up to 500 records per batch. Partition keys are configurable, default is `customerExternalId` so events for one customer route to one shard and preserve order. The canonical JSON event is the record `Data` payload; envelope fields are embedded in the JSON (Kinesis does not have native headers).
Auth and credentials
Customers authorize Pushrail via `sts:AssumeRole` (recommended, short-lived credentials, no exchanged secret) or with an IAM access key pair scoped to `kinesis:PutRecord` and `kinesis:PutRecords` on the destination stream ARN. Credentials are encrypted at rest. The customer can lock the trust policy on the role to Pushrail's external ID for added isolation.
Batching, retries, and replay
The adapter batches up to 500 records or 5 MB per `PutRecords` call. Per-record failures within a batch (ProvisionedThroughputExceeded) retry individually with exponential backoff; the rest of the batch lands. Batch-level transient errors retry with exponential backoff. Permanent errors (AccessDenied, ResourceNotFound) land in the DLQ. Replay re-emits with the same partition keys, so ordering is preserved on re-runs.
Example payload
Pushrail accepts the canonical event shape on POST /v1/events. Below is the ingestion request your service makes.
{
"eventType": "order.completed",
"occurredAt": "2026-05-26T14:21:08.493Z",
"source": "billing-service",
"customerExternalId": "acct_8K2zRq",
"idempotencyKey": "order_38a91f-completed",
"correlationId": "req_4f30b2",
"payload": {
"orderId": "ord_38a91f",
"amount": 12900,
"currency": "USD",
"items": [
{ "sku": "PR-PRO-MONTHLY", "qty": 1, "price": 12900 }
]
},
"metadata": {
"tier": "pro",
"region": "us-east-1"
}
}Example configuration
The fields your customer fills in to point Pushrail at their Amazon Kinesis setup.
{
"type": "KINESIS",
"name": "Customer Kinesis stream",
"streamArn": "arn:aws:kinesis:us-east-1:123456789012:stream/pushrail-events",
"region": "us-east-1",
"partitionKey": "customerExternalId",
"auth": {
"mode": "ASSUME_ROLE",
"roleArn": "arn:aws:iam::123456789012:role/PushrailKinesisWriter",
"externalId": "acme-ext-7Pq2"
}
}Common use cases
- AWS-native customers who don't run Kafka but want real-time event delivery.
- Feed Lambda or Kinesis Data Analytics for in-stream processing on the customer's side.
- Pair with Firehose to land events in the customer's S3 + Redshift + OpenSearch.
- Real-time customer-facing analytics over your product events.
Related destinations
Native Kafka producer with SASL/TLS, deliver events into a customer's topic with configurable partitioning.
Fan-out to customer-owned SNS topics, one event to many subscribers.
Deliver events into a customer's EventBridge bus, rule-driven routing inside AWS.
Frequently asked questions
Do I need a Kafka cluster to get real-time delivery?
No. For AWS-native customers, the Kinesis adapter delivers a real-time event feed without operating Kafka. It uses the native AWS SDK and integrates with Lambda triggers, Kinesis Data Analytics, Firehose, and EventBridge Pipes.
How is ordering preserved?
Partition keys are configurable; the default is customerExternalId, so events for one customer route to the same shard and preserve order. Replay re-emits with the same partition keys, so ordering holds on re-runs.
Whose AWS credentials are used?
The customer's. They authorize Pushrail via sts:AssumeRole (recommended, short-lived credentials) or a scoped IAM access key pair for kinesis:PutRecord and kinesis:PutRecords on the destination stream ARN. Credentials are encrypted at rest, and the role's trust policy can be locked to Pushrail's external ID.
What happens when a record fails?
The adapter batches up to 500 records per PutRecords call; per-record failures like ProvisionedThroughputExceeded retry individually while the rest of the batch lands. Batch-level transient errors retry with exponential backoff, and permanent errors like AccessDenied or ResourceNotFound land in the dead-letter queue.