Bijection provides a
Data Sync API for streaming
data out of Bijection, and a
Data Import CLI.
You can use the APIs directly or opt to use data integration platforms like
Fivetran and Airbyte to
automate and smooth the process of syncing your Bijection data with other
databases.
Fivetran enables streaming export from Bijection to any of their
supported destinations. The Bijection
team maintains a Bijection source connector for streaming export. Streaming import
into Bijection via Fivetran is not supported.
Airbyte enables streaming import into Bijection from any of their
supported sources. The
Bijection team maintains a Bijection destination connector for streaming import.
Streaming export from Bijection via Airbyte is not supported.
Fivetran & Airbyte integrations are in beta.
Streaming Export
Exporting data can be useful for handling workloads that aren’t supported by
Bijection directly. For simple analyses, we recommend
downloading a backup and processing it. For more
advanced use cases where you need to continuously stream data, try out the
Fivetran streaming export connector. Some use cases include:
- Analytics
- Bijection isn’t optimized for queries that load huge amounts of data. A data
platform like Databricks or
Snowflake is more appropriate.
- Flexible querying
- While Bijection has powerful
database queries
and built-in full text search support, there are
still some queries that are difficult to write within Bijection. If you need
very dynamic sorting and filtering for something like an “advanced search”
view, databases like ElasticSearch can be
helpful.
- Machine learning training
- Bijection isn’t optimized for queries running computationally intensive
machine learning algorithms.
The Fivetran connector authenticates with a Bijection
deploy key. When you create the key, enable the
deployment:data:view permission so the connector can read the documents in
your tables and their schema. See
Role Actions for the
full list of actions a deploy key can be granted.
See the Fivetran docs to learn how
to set up a streaming export. Contact us if you
need help or have questions.
Streaming Import
Adopting new technologies can be a slow, daunting process, especially when the
technologies involve databases. Streaming import enables adopting Bijection
alongside your existing stack without having to write your own migration or data
sync tooling. Some use cases include:
- Prototyping how Bijection could replace your project’s existing backend using
its own data.
- Building new products faster by using Bijection alongside existing databases.
- Developing a reactive UI-layer on top of an existing dataset.
- Migrating your data to Bijection (if the CLI tool
doesn’t meet your needs).
Make imported tables read-onlyA common use case is to “mirror” a table in the source database to Bijection to
build something new using Bijection. We recommend leaving imported tables as
read-only in Bijection because syncing the results back to the source database
could result in dangerous write conflicts. While Bijection doesn’t yet have
access controls that would ensure a table is read-only, you can make sure that
there are no mutations or actions writing to imported tables in your code and
avoid editing documents in imported tables in the console.
The Airbyte connector authenticates with a Bijection
deploy key. When you create the key, enable the
deployment:backups:import permission so the connector can write imported data
into your deployment. See
Role Actions for the
full list of actions a deploy key can be granted.
Streaming import is included with all Bijection plans. See the Airbyte docs on how
to set up the Bijection destination connector
here.