Skip to main content
This profile is in beta. Hosted runtime qualification remains separate.
Bijection consumes a customer-operated Debezium PostgreSQL connector through Kafka. It publishes a consistent initial snapshot, then each complete source transaction across all selected synced tables. You operate the Connect workers, brokers, PostgreSQL replication slot and their retention policies.

​
Declare the resource

bijection/warehouse.ts
Bind warehouse.tables.orders and warehouse.tables.lines using defineSyncedTable. Connections, source controls, required views and atomic publication use the same owners as native PostgreSQL CDC. The initial profile requires PostgreSQL 16.14 with pgoutput, one through eight ordinary tables, primary keys, every declared column, and REPLICA IDENTITY FULL. It supports boolean, integer, real, double precision, bytea, text, varchar, uuid, numeric, json, jsonb, date, timestamp and timestamptz columns. Exact integers and decimals are preserved. Source inspection rejects unsupported columns and mismatched validators.

​
Prepare Kafka Connect

Use Debezium 3.7.0.Final, JSON key and value converters with embedded schemas, precise decimals, serializable initial snapshots and transaction metadata. Set skipped.operations=none and event.processing.failure.handling.mode=fail so truncates are captured and processing failures cannot silently discard source changes. Each selected table topic, the transaction topic and the notification topic must have one partition and retain their complete history from offset zero for initial enrollment. Use ordinary nontransactional Connect production; this profile refuses Kafka transaction-control and filtered-offset gaps. Do not compact this history, replace topics, filter columns, override keys, or apply other transforms. Install the debezium-source-offsets JAR from the pipelines repository beside the Debezium plugin on every worker. It preserves each source record’s actual partition and offset in headers, including empty-table snapshot notifications. Configure:
Bijection also requires SQL catalog access to PostgreSQL to independently check its system identity, timeline, publication and table schema. Grant that login EXECUTE on pg_catalog.pg_control_system() and pg_catalog.pg_control_checkpoint() as well as catalog and table access. It does not create or acknowledge the customer’s replication slot.

​
Capture and control the installation

Create a private local installation file:
Use authority: "exclusive" only when this CLI is delegated connector control. The following commands inspect the installation and produce credential-free captures or configuration plans:
Review the saved plan before applying it. Apply checks the connector’s observed configuration against the plan, preserves private provider configuration, sends one PUT, and inspects the outcome. A concurrent provider editor is outside this CLI’s exclusive-control contract; Kafka Connect supplies no configuration CAS. An uncertain response is inspected without automatically repeating the PUT. pause, resume and stop require exclusive authority. Their response proves request acceptance; status reports task state without worker traces or credentials. Bijection source pause and Connect task pause are independent. Install a PostgreSQL connection using the normal connection workflow. The private PostgreSQL JSON adds:
Use caPem in the local installation and connect_ca_pem in the private connection for a private Connect CA. The private connection also accepts paired connect_username/connect_password instead of connect_bearer. The outer credential includes PostgreSQL host, port, database, user, TLS, publication and relations as for native CDC. Logical aliases in relations must match the resource’s tables. broker_credential uses the shared Kafka TLS/SASL credential format. Rotate credentials through the private connection owner, never application exports or capture files.

​
Recovery and progress

Original broker records enter bounded acquisition custody before decoding. Retries replay retained input before source I/O; sealed snapshot and transaction units can finish publication without PostgreSQL, Connect or Kafka. An END, exact per-table counts and complete global event ordering are required before publishing a transaction. Duplicate identical events are tolerated; conflicting duplicates, gaps, changed topics or source identity stop progress. Kafka offsets and PostgreSQL positions remain separate. Bijection commits their capture receipt with a complete unit; it does not use consumer-group commits as a visible-publication checkpoint. A Kafka tombstone is not a row deletion. Empty polls and arbitrary heartbeats do not refresh source freshness. A complete zero-row transaction can prove quiet progress. Source schema changes require a new checked capture and explicit source rebaseline. Coordinate DDL while capture is stopped: Debezium does not provide the native reader’s DDL marker helper. Source inspection includes the captured configuration digest, exact broker position vector, raw-input digest and native capture/publication backlog. The producer checkpoint persists after temporary acquisition custody is released. Use the normal source status, pause, resume and rebaseline commands for governed publication. Rebaseline starts a new generation; it does not delete customer slots or alter Connect offsets. It requires the original complete bootstrap history or a separately prepared new connector installation. Capture custody is bounded by native source budgets and is released by the existing custody owner; this resource does not expose a long-term raw archive or arbitrary historical replay selection.