1. Bind the Stream
vars.
Create partner_webhooks with an immutable structured schema. The binding
references this resource; it does not define its schema:
2. Verify and ingest
202 only after send resolves. At that point the Stream has durably
acknowledged the record; lakehouse publication can happen independently.
3. Normalize in a Pipeline
webhook_events to an Iceberg Sink. The Sink and Catalog deduplicate
Pipeline batch identities, so a crash and retry cannot publish the same batch
as a second logical commit.
4. Keep the raw envelope
If audit or replay matters, fan out the same Pipeline input to a second Sink:Production checks
- Verify the signature before writing to the Stream.
- Preserve the source event ID even if deduplication happens later.
- Keep payloads below the 5 MiB Stream request limit.
- Treat schema changes as versioned data contracts.
- Alert on Stream validation errors and Pipeline cursor lag.
