Skip to main content
There is one source contract: every declared source dataset is captured, and read only at a published immutable basis. No live mode, no federated passthrough, no query-time read of a customer system.

​
Three strategies, one engine

A dataset declares exactly one capture strategy. They are three readers into one temporal plane, not three ingestion systems.

Snapshot

Repeated complete captures of the whole relation.

Cursor

A keyset cursor over a declared append-only column.

CDC

PostgreSQL logical replication.
A deployment cadence runs whichever reader each declaration names.

​
What a complete capture does

A complete capture streams bounded Arrow batches into one private ordinal-keyed staging relation, derives stable document identities from the declared key, rejects duplicate keys, reduces that stage to the sparse recorded transition, and applies its puts plus explicit deletions in one transaction. The process therefore retains only O(changes) transition metadata. A first baseline necessarily makes every row a change; the next one usually does not. Private staging is erased rather than retained as source history. Cursor and CDC captures stage only their declared deltas — they arrive as transitions and skip the reduction. Everything downstream is identical.

​
Publication is atomic

After all selected datasets complete, Postgres atomically publishes:
  • the temporal await token and snapshot token
  • the clock time
  • the member set, with each member’s own capture evidence and source position
A reader can never see a half-published capture. Source positions are provider-native and stay distinct from each other:
The source position, the extraction interval, the dataset version, the temporal basis, and the Postgres acceptance time are five distinct pieces of evidence. Collapsing them would destroy the ability to explain a disagreement between two datasets in one capture.

​
Dataset versions are history-sensitive

A dataset version’s identity covers the parent it extended, the exact transition, and the resulting totals. An unchanged observation re-admits its parent version; a state that is returned to is published as its own version, because arriving back somewhere is a real event.

​
Freshness

Freshness is bought with capture frequency. Besides the deployment cadence, one provider-verified occurrence may enqueue a fresh capture of exactly one named declared dataset — taking the same singleton lease, the same staging, the same validation, and the same atomic publication, and firing dataset.changed exactly as a cadence tick does. Resolving a wall-clock instant selects the newest transaction published by that instant. It cannot invent source changes between capture ticks.

​
Running a capture by hand

Serving replicas never contact customer systems as a startup or timer side effect. The deployment-owned job is the only cadence.