> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bijection.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Debezium

> Consume a customer-hosted Debezium PostgreSQL connector

> [!WARNING]
>
> This profile is in beta. Hosted runtime qualification remains separate.

Bijection consumes a customer-operated Debezium PostgreSQL connector through
Kafka. It publishes a consistent initial snapshot, then each complete source
transaction across all selected synced tables. You operate the Connect workers,
brokers, PostgreSQL replication slot and their retention policies.

## Declare the resource

```ts bijection/warehouse.ts theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
import { debezium } from "@bijection/connectors/debezium";
import { cdc } from "./_generated/cdc.js";

export const warehouse = debezium.database.cdc({ contract: cdc });
```

Bind `warehouse.tables.orders` and `warehouse.tables.lines` using
[`defineSyncedTable`](/integrations/synced-tables). Connections, source controls,
required views and atomic publication use the same owners as
[native PostgreSQL CDC](/integrations/postgres).

The initial profile requires PostgreSQL 16.14 with pgoutput, one through eight
ordinary tables, primary keys, every declared column, and `REPLICA IDENTITY
FULL`. It supports boolean, integer, real, double precision, bytea, text,
varchar, uuid, numeric, json, jsonb, date, timestamp and timestamptz columns.
Exact integers and decimals are preserved. Source inspection rejects unsupported
columns and changes to the captured schema, type modifiers, nullability or keys.

## Prepare Kafka Connect

Use Debezium **3.7.0.Final** and Kafka Connect **4.1.0**, JSON key and value converters with embedded
schemas, precise decimals, serializable initial snapshots and transaction
metadata. Set `skipped.operations=none` and
`event.processing.failure.handling.mode=fail` so truncates are captured and
processing failures cannot silently discard source changes.
Each selected table topic, the transaction topic and the notification
topic must have one partition and retain their complete history from offset zero
for initial enrollment. Use ordinary nontransactional Connect production; this
profile refuses Kafka transaction-control and filtered-offset gaps. Do not
compact this history, replace topics, filter
columns, override keys, or apply other transforms.

Install the `debezium-source-offsets` JAR from the
[pipelines repository](https://github.com/bijectionhq/pipelines/tree/main/src/debezium)
beside the Debezium plugin on every worker. It preserves each source record's
actual partition and offset in headers, including empty-table snapshot
notifications. Configure:

```properties theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
transforms=bijectionSource
transforms.bijectionSource.type=io.bijection.debezium.SourceOffsets
header.converter=org.apache.kafka.connect.storage.SimpleHeaderConverter
producer.override.enable.idempotence=true
producer.override.acks=all
producer.override.max.in.flight.requests.per.connection=1
```

Grant topic DescribeConfigs access. Effective `cleanup.policy` must be `delete`,
`retention.ms` at least seven days (or `-1`) and `retention.bytes=-1`. Admission
checks these settings and continuous offsets independently. The reported Connect
version and PostgreSQL plugin apply to the responding worker; operators must
install the same admitted artifacts on every eligible worker. Bijection checks
the actual Kafka cluster ID against Connect and the private binding.

Bijection also requires SQL catalog access to PostgreSQL to independently check
its system identity, timeline, publication and table schema. Grant that login
`EXECUTE` on `pg_catalog.pg_control_system()` and
`pg_catalog.pg_control_checkpoint()` as well as catalog and table access. It does not create
or acknowledge the customer's replication slot.

## Capture and control the installation

Create a private local installation file:

```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
{
  "endpoint": "https://connect.example.com",
  "connector": "warehouse",
  "authority": "observe",
  "postgresConnection": "postgres.private.json",
  "bearerToken": "PRIVATE_TOKEN",
  "tables": { "orders": "public.orders", "lines": "public.lines" },
  "topicPrefix": "warehouse",
  "notificationTopic": "warehouse.notifications"
}
```

Use `authority: "exclusive"` only when this CLI is delegated connector control.
The following commands inspect the installation and produce credential-free
captures or configuration plans:

```sh theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
bijection debezium plan --connection connect.private.json --output connect.plan.json
bijection debezium apply --source orders --connection connect.private.json --plan connect.plan.json
bijection debezium capture --connection connect.private.json --output connect.capture.json
bijection debezium check --connection connect.private.json --frozen connect.capture.json
bijection debezium status --connection connect.private.json
```

Capture reads the selected PostgreSQL catalog with schema locks in a read-only
serializable transaction. It writes both `connect.capture.json` and
`connect.capture.ts`; move the TypeScript module to `bijection/_generated/cdc.ts`.
The module contains logical tables, column types, nullability and primary keys.
`check --frozen` checks the catalog, runtime and configuration again.

Review the saved plan before applying it. Connect changes require `--source`
and the usual deployment selector. The deployment records the exact intent in
its existing external-call owner before any PUT. It checks the source epoch,
configuration, credential revision, runtime and explicit installation grant
again before dispatch. Repeating a command after a lost CLI response recovers
the same retained request. An uncertain provider result is reconciled with GET;
it never automatically repeats the PUT.
New request admission expires after 24 hours. A retained operation can still be
recovered; an expired request without a retained operation is refused, so receipt
cleanup cannot turn an old request into another connector change.

The installation example grants observation authority. To delegate
specific operations, replace its `control_authority` with:

```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
{"kind":"exclusive","grants":["configure","pause","resume","stop"]}
```

Grant only the operations this deployment should perform. `configure` repairs
the installation's captured configuration contract; it cannot change database
credentials or physical scope through a plan. Prepare the customer's connector
for initial enrollment before installing its read connection. Private binding
changes use the ordinary connection owner and a new checked capture.

Connect supplies no configuration CAS. Exclusive authority requires that other
editors do not race this control owner. `pause`, `resume` and `stop` establish
request acceptance; `status` owns task completion. Use
`debezium operation-status --operation <key>` to inspect or reconcile a retained
operation. Status never starts a pending send. An original connector whose
identity or credentials changed remains unresolved rather than being resent to
a replacement. Bijection stage pauses and Connect task pause are independent.

Install a PostgreSQL connection using the normal
[connection workflow](/integrations/connections). The private PostgreSQL JSON
adds:

```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
{
  "debezium": {
    "configuration_digest": "CONFIGURATION_DIGEST_FROM_CAPTURE",
    "broker_cluster_id": "BROKER_CLUSTER_ID_FROM_CAPTURE",
    "control_authority": { "kind": "observe" },
    "broker": "broker.example.com:9093",
    "broker_credential": "{\"tls\":{}}",
    "connect_endpoint": "https://connect.example.com",
    "connect_bearer": "PRIVATE_TOKEN",
    "connector": "warehouse",
    "topic_prefix": "warehouse",
    "notification_topic": "warehouse.notifications"
  }
}
```

Use `caPem` in the local installation and `connect_ca_pem` in the private
connection for a private Connect CA. The private connection also accepts paired
`connect_username`/`connect_password` instead of `connect_bearer`.

The outer credential includes PostgreSQL `host`, `port`, `database`, `user`,
TLS, `publication` and `relations` as for native CDC. Logical aliases in
`relations` must match the resource's tables. `broker_credential` uses the
shared Kafka TLS/SASL credential format. Rotate credentials through the private
connection owner, never application exports or capture files.

## Recovery and progress

Original broker records enter bounded acquisition custody before decoding.
Acquisition maintains its own raw cursor ahead of interpretation and publication.
Semantic decoding failures continue raw capture within the existing source and
invocation budgets. Retained original bytes survive crashes between custody and
cursor updates. Sealed snapshot and transaction
units can finish publication without PostgreSQL, Connect or Kafka. An END,
exact per-table counts and complete global event ordering are required before
publishing a transaction. Duplicate identical events are tolerated; conflicting
duplicates, gaps, changed topics or source identity stop progress.

Kafka offsets and PostgreSQL positions remain separate. Bijection commits their
capture receipt with a complete unit; it does not use consumer-group commits as
a visible-publication checkpoint. A Kafka tombstone is not a row deletion.

Empty polls and arbitrary heartbeats do not refresh source freshness. A complete
zero-row transaction can prove quiet progress. Source schema changes require a
new checked capture and explicit source rebaseline. Coordinate DDL while capture
is stopped: Debezium does not provide the native reader's DDL marker helper.

Source inspection includes the captured configuration digest, exact broker
position vector, raw-input digest and native capture/publication backlog. The
producer checkpoint persists after temporary acquisition custody is released.
Status also exposes raw and published offset vectors, sealed and published
sequence numbers, retained batch limits, waiting topics and stage timestamps.
Kafka acquisition time never substitutes for source observation time.

Use `integration postgres-control <destination> --revision <revision>` to set
`--control pause_acquisition`, `resume_acquisition`, `pause_publication` or
`resume_publication`. Acquisition pause still permits interpretation of retained
input. Publication pause is checked again in the atomic publication transaction.
Both are independent of Connect task pause.

After deploying a decoder repair, pause acquisition and inspect
`debezium_archive.replay_plan` in source status. Submit that exact digest with
`--control replay --plan <digest>`. Replay preserves the unpublished input,
generation, contract and published predecessor. It refuses identity, continuity
and schema changes; those need a new capture and explicit source rebaseline.
Rebaseline starts a new generation and leaves the customer slot and Connect
offsets alone. It requires complete bootstrap history or a separately prepared
new connector installation.

The recovery archive shares ordinary acquisition custody and source budgets.
It pins lookahead until publication consumes its offset range, then the existing
custody owner releases it. Capacity exhaustion stops acquisition visibly. This
profile does not retain indefinite history or expose arbitrary historical
rollback.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.