Skip to main content
Use ordinary queries for native views. Use a mutation to start a run, then ordinary queries to observe it and read its results. External pipelines publish dataset versions which can feed another run or become native rows through a collection sync. A definition is an exact calculation with its argument and output contract: an ordinary function, or a captured SQL statement. A run is one durable invocation of a definition with fixed arguments. A result is the accepted immutable content at one named output of a run. A hold is one consumer’s finite retention interest in a result. A selection chooses which accepted result a data product currently serves.

Application development

Use ordinary queries and mutations for application state. Use Compute.api for an explicit durable invocation of an ordinary query or action. Keep maintained pipelines for automatic refresh, selection and publication. The customer-success example exposes api.customerHealth: start admits the existing portfolio action, get observes its execution and named outputs, and listCustomers / getSummary read the selected run’s exact results. The shared argument and return validators remain the ordinary function contract. Native SQL admission checks the same relational formula and its captured input basis.
start captures the exact invocation before sending it. Mounting, reconnecting and restoration only observe. recover reads the original admission; resume explicitly continues its saved identity, input and implementation. A deliberate new assessment uses a new persisted interaction key. useComputation wraps ordinary generated functions, which you can also call directly: inspect, resume, cancel and retire exist only when access defines them. get takes exactly one locator. A requestKey is a UUIDv7 you persist for one interaction. It fixes its own admission window and is hashed with the caller. A lost reply therefore recovers from the key alone, and holding someone else’s key locates nothing. Reusing a key with changed arguments is refused. After its window ends, a key cannot start new work. A deliberate recalculation uses a new key. An explicitly selected older run is observed with useQueryState; hide its protected presentation when that observation is unavailable. The consolidated query preserves execution, acceptance, cancellation, retirement and cleanup as independent facts. Each output retains its current access and retention assessment. One unavailable output does not rewrite successful execution or hide another available output. Named readers accept { run }, resolve its immutable accepted output inside the query and preserve inherited pagination contracts. A new run resets the page chain; a subsequent page never selects a newer result. Every read rechecks current application scope and native resource authority. Creating an account review consumes that exact inspected output, with current business checks; it does not rerun the assessment. Cancellation requests a stop. retire releases the run owner’s interest; neither proves physical termination or completed cleanup. Independent holds remain independent and outlive the run’s closure, and unmounting changes no server custody.

Declaring result readers

Keep the action’s ordinary return validator as the single output contract. The installed Compute client derives SQL step identity and generates readers:
get includes execution, acceptance and each output’s current availability. Named readers accept the original run and recheck current access. Dataset readers use ordinary pagination; artifact readers return the declared value. Expired results remain unavailable on their historically successful run. read(ctx, run) receives that run, or null when accessing recovery metadata.

Implement an action with SQL

compute.sqlHandler implements an ordinary internalAction with SQL. It returns the usual args, returns and handler declaration and registers no other callable kind. Each named output is one statement, and each argument binds as a :name parameter:
State returns explicitly: each statement’s analyzed output is checked against it when the run accepts. Deriving the contract from native analysis is not implemented. Bounds apply to the complete accepted result; exceeding them refuses the calculation and never returns a prefix. A retry finds the original statement under its output name.

Server responsibilities

Ordinary database reads expose local readiness through ctx.db.checkRead and ctx.db.requireRead, and current serving state through ctx.db.publication. These operations observe the same transaction basis and never start remote work. Retained scalar series evidence is checked with ctx.series.requireAnalysis. An action explicitly enters the installed run owner through runs(ctx) from bijection/analytics. Its start returns the original request and a separate submission; capture, admission and result availability remain distinct. Keep the request reference before interpreting a lost reply. Generic retained datasets and artifacts use results from bijection/results. Physical SQL file exports belong to the typed queryArtifacts resource adapter in bijection/runtime/adapters/exports, which checks the original holder and current source policy on every access.

Vocabulary

Trino is a customer supplied cluster or managed service. Installing an executor binding gives Bijection a way to submit statements; Bijection does not provision or own that cluster. Funrun executes application functions; it does not turn a Trino statement or a customer ETL job into a local transaction. Each arrow has its own acceptance boundary. This graph is not a transaction spanning Bijection, Trino and a customer job. A collection sync prepares one whole version privately and activates it in a single commit, so readers see either the previous version or the new one. A chunked version sync instead applies rows chunk by chunk and has no whole-version boundary.

SQL over ontology relations

SQL is another authoring surface for the same typed relational declarations. sqlRelation accepts a bounded read-only query, explicitly bound logical inputs and bounded scalar or structured parameters. Compose its result with defineRelation, a computation or q.program. The native engine checks types, current permissions, read requirements, work bounds and execution placement.
Parameter values and lexical declaration arguments bind values only. Unquoted identifiers fold to lowercase; double-quoted names are exact. Integer literals remain exact. Native analysis describes result columns. An optional returns validator supplies TypeScript inference and is checked against those analyzed columns. SQL declarations capture their defaults explicitly:
The defaults are UTC and binary collation. Timezone names must use an exact spelling from the pinned IANA database. The supported collations include binary, two Unicode case-folding profiles and eight locale sort profiles. Unknown settings refuse before execution. Calendar extraction, timestamp parsing and SQL casts, DATE_FORMAT, datetime TO_CHAR and timestamp truncation capture their chosen timezone in the ordinary plan. An explicit function timezone overrides the declaration default. Epoch arithmetic and canonical instant transport retain their UTC representation. Civil timestamps retain their wall clock under DATE_FORMAT and time-part extraction. Converting a civil clock to an instant assigns the captured timezone and refuses a skipped or ambiguous local time; TRY_CAST returns null for that conversion failure. Import SQL files through normal source preparation:
The captured file is bounded at 1 MiB. Diagnostics retain its filename and SQL span. Group named relations with q.program. An ordinary action calls QueryWork.run and returns exact accepted output references under its declared runtime validators. For maintained refresh and publication, bind that action through new PipelineAdapters(components.pipelines).bind(reference, options). QueryWork owns the finite analytical read; the pipeline retains the accepted references. Inspect the same captured SQL before choosing an execution binding:
The report includes exact operand and result domains, frames, implementation requirements, runtime pins and actionable refusals from the actual lowerers. Deployment eligibility and named-client qualification require their own owner records; successful static lowering does not attest them. Use sqlFeatureInventory(cases, evidence) from the same inspection entry point to keep a release inventory of functions, clauses, types, client operations and Dataset behavior. A report admits at most 1,024 cases and 16 MiB of output. Each case names a stable id, category, surface, and owner, and compatibility scope (furnace, ontology_sql, or additional), plus either an inspectSQL request in query or an explicit outstanding requirement in gap. Analysis failures remain individual refused rows, so an unsupported case does not hide later results. Failed declarations retain their stable diagnostic code and authored source span. Successful analysis retains the exact definition, requirement IDs, target decisions and runtime pins. The attached coverage matrix matches the existing immutable evidence records; client names, versions and deployment qualification retain their original scope. Cases without a relational plan, including client lifecycle checks, must name their outstanding owner evidence explicitly. The 1,024-case and 16 MiB limits apply to the complete serialized inventory, including its coverage matrix. External lowering remains query-specific. Independent lateral populations use the checked join machinery on Spark, Databricks, Snowflake and BigQuery. Spark also supports bounded correlation over a scanned and filtered population; broader correlated shapes refuse when no exact lowering is available. Retained SQL binds observed clock ticks into the execution identity and deadline, and refuses missing, mismatched or expired observations before executing work. Grouping keys can include scalar subqueries. Repeating an authored key in the projection, HAVING, or ordering reuses its value from the original input population. Nested correlated subqueries can refer to that complete key; nearer aliases still shadow the original source, and ungrouped source columns remain unavailable after grouping. Grouping-set omissions retain their null values rather than evaluating the omitted expression again. The native SQL interface supports projections and stars over closed schemas, wildcard EXCLUDE, EXCEPT, REPLACE, RENAME and ILIKE selection, DISTINCT, deterministic DISTINCT ON, bag joins, three-valued predicates, null tests, grouping, global aggregates, HAVING, CTEs, subqueries in FROM, and UNION, INTERSECT and EXCEPT with distinct or ALL multiplicity. Recursive UNION CTEs use the positive set fixed-point evaluator and its work budget. Recursive UNION ALL preserves every occurrence and evaluates each step over only the previous iteration’s rows until that frontier is empty. Recursive terms preserve the exact seed field domains, including collection element nullability. UNNEST zips multiple arrays, pads shorter arrays with null, exposes one-based WITH ORDINALITY or zero-based WITH OFFSET, and supports left joins with an ON predicate, applied before retaining an unmatched parent. UNION BY NAME aligns the union of both declared schemas, fills missing columns with SQL null, and preserves left column order followed by new right columns. Shared names require compatible domains. ALL BY NAME retains duplicate occurrences; the distinct form removes equal complete rows. Literal and bound integer row counts support LIMIT, OFFSET, TOP and FETCH. Integer GENERATE_SERIES(start, stop [, step]) includes its stop when reachable and defaults to a positive unit step. RANGE([start,] stop [, step]) excludes its stop and defaults to start zero and step one. Explicit negative steps descend; an endpoint in the wrong direction returns no rows. A zero step is a typed failure. Both forms accept one-column aliases, discover and prepare named integer parameters, and use ordinary bounded sequence and unnest kernels. Collection table functions accept arrays and typed analytical maps:
EXPLODE exposes col for an array or key,value for a map. POSEXPLODE adds a zero-based pos column first; map positions follow the native canonical key order. AS item(position, key, value) renames every output column in order. The _OUTER forms emit one all-null row for an empty or null collection. Separate generators preserve every occurrence and form a cross product. INNER JOIN and LEFT JOIN ... ON apply the predicate inside the generated population before preserving an unmatched parent. Use an explicit ORDER BY when output order matters. These forms use the existing metered sequence, map and lateral operators; external execution still requires an exact lowering. Multi-output SELECT aliases and Hive LATERAL VIEW syntax remain unsupported. Static PIVOT lists lower to one filtered group with implicit grouping over input columns not referenced by pivot keys or aggregate expressions. Tuple keys, multiple aliased aggregates, and DEFAULT ON NULL are supported. Text pivot values can name their output directly; other static values require explicit aliases. Quoted labels retain Unicode, spaces and punctuation independently of native row keys. A single aggregate uses the value label as its column name; multiple aggregates require aliases and produce value_aggregate names. Dynamic ANY [ORDER BY key] and subquery pivots freeze their columns at prepare. The warehouse Flight SQL gateway discovers values through bounded, authorized retained runs in one session. The prepared program asserts the complete pivot value set at execution; a changed set, including an empty set, refuses the query and requires a new prepare. Ordinary value changes retain the prepared columns. Schema refresh preserves the frozen values and performs no new discovery. Code-first callers use sqlPivotSyntax(sql, frozenValues) to obtain each next discovery query, execute it through their authorized run owner, and pass its exact pivot_json and pivot_schema strings back as { [id]: [{ json, schema }] }. The final sqlRelation accepts that map as pivots; supplied values grant no read authority and retain the same execution assertion. Discovery retains null fields, exact integers and decimals, binary values and timestamp microseconds. Unsupported JSON round trips refuse the query. A prepare admits at most 16 dynamic pivots, 256 values per pivot, eight key fields and 1 MiB of frozen values. Discovery resolves pivots in CTE definitions and pivot value subqueries before their dependents. A CTE sees its earlier lexical declarations. Discovery refuses empty sets, bind-time parameters in discovery sources and pivots that depend on their own recursive CTE. Single-value UNPIVOT expands declared input columns once per source occurrence, retains other columns, and supports input labels and an output alias. It excludes null values by default; INCLUDE NULLS retains them. Unpivoted columns require a compatible common value domain. Both forms retain ordinary plan, value and work bounds and require an eligible implementation of their lowered operations. The current unpivot input list is limited to 64 columns by the shared scalar constructor. LIMIT 0 and FETCH FIRST 0 ROWS ONLY preserve the analyzed output schema without reading the input population. Projection EXPLODE(array_or_map) expands each parent into its array elements or map entries. Arrays expose col; maps expose key,value in canonical key order. POSEXPLODE adds zero-based pos. Both _OUTER forms retain null or empty parents with NULL generated columns, including the position. Use AS (position, entry), AS (key,value) or AS (position,key,value) to name multiple outputs. Alias counts must match the bound collection domain. Multiple generators form a Cartesian population and preserve duplicate occurrences. Grouped collection results can be expanded in the same projection. Put generators in a separate subquery when using windows or QUALIFY, so expansion and filtering have an explicit order. A positive OFFSET completes authored ordering with the projected scalar or reference values in output-column order. Without ORDER BY, that tuple sorts ascending by the exact logical value order. Null, missing and present values remain distinct ordering cells; identical output rows are interchangeable. Structural or opaque outputs require explicit ordering that distinguishes their rows. An unresolved tie still refuses the complete result. OFFSET 0 is a no-op, and an empty slice needs no default ordering. COUNT(), COUNT(*), COUNT(value), scalar COUNT(DISTINCT value), COUNT_IF, SUM, MIN and MAX use ordinary native measures. COUNT(DISTINCT a, b, ...) accepts up to 64 arguments and excludes a tuple when any argument is null or missing. Structural distinct counts, distinct sums and averages, exact discrete and continuous percentiles and deterministic value-at-minimum/maximum reductions use complete bounded collections. The analytical collection ceiling is 1,000,000 items and 16 MiB; exceeding either bound refuses the complete calculation. Native array carriers also retain their inherited 8,192-element and per-value size limits. Select an eligible external execution route for larger collections. Standard approximate cardinality and percentile SQL calls use complete exact fallbacks. Explicit evidence overloads declare sketch error and confidence. Null and missing values do not contribute to those SQL aggregate populations. Bounded arithmetic, conditional expressions, and existence subqueries in WHERE conjuncts use the same declaration analyzer and complete transactional evaluator. Supported lateral populations use the same analyzer and evaluator. SDK inputs can also bind an installed interface with an explicit properties-only implementation selection. String equality and ordering compare exact UTF-8 values, without locale rules or case folding. The grouped regression family REGR_COUNT, REGR_AVGX, REGR_AVGY, REGR_SXX, REGR_SYY, REGR_SXY, REGR_SLOPE, REGR_INTERCEPT and REGR_R2 uses the same pairwise-present population, with arguments in dependent-y, independent-x order. FILTER applies before statistical evaluation. Counts are integers; other results use exact integer, decimal or binary64 input coordinates, rounding only the final statistic to finite binary64. Empty populations return count zero and null statistics. A constant independent coordinate yields null slope, intercept and R-squared; a constant dependent coordinate with varying independent coordinates yields R-squared one. Both coordinate declarations must have numeric domains. All nine spellings also support windows through the same exact pair-moment kernels. BIT_AND, BIT_OR and BIT_XOR reduce signed integral values without converting them through floating point. They retain Int16, Int32 and Int64 result widths, omit null/missing values and return null for an empty population. Grouped and window calls accept DISTINCT and FILTER; XOR retains the actual duplicate multiplicities, including as rows leave a sliding frame. The SDK exposes g.bitAnd, g.bitOr, g.bitXor and q.window.bitAnd, bitOr, bitXor, with an optional { distinct: true }. Native and portable execution use the shared bounded reducer. Spark groups use integral bitwise aggregates; its ROWS and RANGE windows use the installed exact frame runtime, including a disk index for distinct populations. Other direct SQL adapters refuse this reduction until their exact operation and carriers are admitted. SKEWNESS(number) computes the population standardized third moment over present numeric observations. It keeps integer, decimal and finite binary64 coordinates exact through accumulation, merge and frame removal, then rounds the final result once. Empty and constant populations return null. FILTER and window frames share that population contract. The SDK exposes g.skewness and q.window.skewness; the field must have a declared numeric domain. Spark uses its installed exact moment kernel rather than a provider calculation with different rounding. Univariate statistics, covariance, correlation and all regression calls accept DISTINCT in groups and windows, including FILTER. Pairwise null exclusion precedes tuple distinctness: a pair contributes only when both coordinates are present, and argument boundaries and exact declared domains are retained. Finalization uses the ordinary exact moment kernels after canonical collection deduplication. Empty REGR_COUNT populations return zero; other empty statistics return null. These DISTINCT populations share the bounded collection and native carrier limits described above. Spark uses the installed shared scalar finalizer; other direct adapters refuse that finalizer until its exact kernel is installed. Spark’s ROWS and exact RANGE collection windows prove their bounds over the complete contributing partition before collecting any frame. That conservative fence includes duplicate observations and the 8 MiB guest request limit. A partition that fails it is refused even if individual frames would fit. Collections can share a window with the installed exact-coordinate and retractable statistics runtime. PERCENTILE_APPROX(value, fraction[, accuracy]) accepts a constant fraction or fraction array and a positive constant integer accuracy, defaulting to 10,000. It uses an exact bounded nearest-rank calculation, so its rank error is zero. The catalog marks this as exact_bounded_fallback; it is not a sketch and must retain the complete bounded collection. Grouped and window forms preserve filters, distinctness, null exclusion and fraction-array order. Collection exhaustion refuses the complete result. APPROX_PERCENTILE accepts the same standard interface and retains a separate four-argument explicit error/confidence overload with an accuracy-bearing result. PERCENTILE(number, fraction_or_array[, frequency]) computes exact continuous percentiles in groups and windows. Frequencies have an integral declared domain; positive values contribute rank mass, zero and null observations are omitted, and negative frequencies refuse the calculation. The default frequency is one. The total admitted rank mass is at most 10^18, independent of the number of retained observations. Frequencies never expand into repeated physical rows. Up to 64 constant percentages share one ordered histogram and cumulative index, preserving percentage-array order and duplicates. Decimal and binary64 percentages retain their exact fractions through rank selection and linear interpolation; only the final result rounds to finite binary64. Empty populations and empty percentage arrays return null. FILTER precedes the population and DISTINCT deduplicates complete value/frequency pairs. The collection and Spark partition fences above still apply. BASE64(text_or_binary) encodes exact UTF-8 or binary bytes with the standard alphabet and padding, without line breaks. UNBASE64(text) follows Java MIME decoding: it ignores non-alphabet characters, accepts unpadded two- or three-symbol tails, and refuses malformed padding or one-symbol tails. HEX(text_or_binary_or_integral) emits uppercase UTF-8/binary bytes or the integral value’s Int64 two’s-complement representation. UNHEX(text) accepts either case, prepends a zero nibble for odd lengths, and returns null for malformed hex. The SDK exposes q.base64, q.unbase64, q.hex and q.unhex. Native, portable and installed Spark scalar kernels share these contracts; other direct SQL adapters refuse them until an exact implementation is admitted. COUNT_IF(predicate) counts true values and returns zero for an empty group or window. Distinct counts in windows exclude nulls and maintain exact multiplicities as rows enter and leave the frame. REGR_SLOPE(y, x) and REGR_INTERCEPT(y, x) share the exact pair-moment owner with covariance and correlation. They exclude pairs containing null, round the final result once to binary64, and return null when no slope is defined. Unnamed scalar expressions receive positional column1, column2, etc. names; explicit names remain preferable for reusable queries. Large IN lists, array constructors and CASE expressions lower into bounded groups of shared operators. Parser, expanded-plan, scalar-count and value bounds still apply. SHORT/SMALLINT, INT and LONG retain distinct 16-bit, 32-bit and 64-bit integer domains, including checked casts and arithmetic. FLOAT retains binary32 precision. TIMESTAMP WITHOUT TIME ZONE and TIMESTAMP_NTZ use the microsecond civil timestamp domain; ordinary TIMESTAMP uses instant semantics. CAST(value AS BINARY) accepts binary values, UTF-8 text and signed integral values. Integral encodings use the source width and big-endian two’s complement: SHORT uses two bytes, INT four and LONG eight. Binary-to-STRING casts follow Java 17 UTF-8 replacement boundaries, including malformed sequences. This differs from the strict UTF-8 decoding option of DECODE. Portable execution and the installed exact Spark kernels use the same conversions. CAST(value AS MAP<STRING, type>) preserves admitted text keys and converts each value, including nested lists, maps and positional structs. TRY_CAST can make an invalid scalar map value null. Dataset assignment checks required values and finite numeric domains within the map before a later projection can discard it. NULL and empty composite casts retain their declared SQL IEEE domains. SHOW FUNCTIONS, SHOW FUNCTIONS LIKE 'array_%' and SHOW FUNCTIONS WHERE ... produce ordinary read-only rows with name, canonical_name, kind, signature and evaluation. The catalog describes accepted SQL spellings. Inspect the actual query to establish its typed implementation requirements and eligible installed execution paths. information_schema.functions exposes the same inventory as function_name, canonical_name, function_kind, signature and evaluation. Parser metadata and inspectSQL().language expose it without reading a source or attesting an executor. One UNNEST with several arrays zips by position and pads shorter arrays with null. Separate generators form a cross product. WITH ORDINALITY adds a one-based position; WITH OFFSET adds a zero-based position. A left join on a predicate preserves the input row when every array is empty or null, or no generated row satisfies the predicate. Lateral join predicates run inside the right population before outer preservation. Arrays are evaluated once per input row and use ordinary lexical scopes and work budgets. Spark lexical array bodies use the pinned spark.lambda_bindings runtime. Typed identity boundaries preserve the declared nested carrier without copying values and prevent repeated type/nullability walks through nested bindings. Each schema program is bounded to 64 KiB, 64 levels and 8,192 schema nodes; incompatible types or executor identities refuse before evaluating inputs. Captured row and outer-array expressions retain their analyzed population and conditional reachability. Bodies requiring shared scalar kernels execute as one bounded shared expression outside Spark’s higher-order functions. Their original semantic checks remain part of the compiled validation contract. WITH RECURSIVE ... AS (seed UNION step) uses the existing positive set fixed point. The seed and step must preserve the exact row contract. Duplicates are removed, cycles converge, and resource exhaustion fails the complete result. The analyzer rejects nonmonotone operations on recursive populations. Recursive UNION ALL retains duplicate seed and path occurrences. Its recursive input is the previous frontier, rather than the accumulated result, and it terminates only when that frontier is empty. Repeating equal rows does not establish convergence in this mode. Both quantifiers share the same cumulative work account and refuse the complete result on exhaustion. Bag iteration also retains the shared 10,000-round bound and claims no full-row uniqueness key. SDK relations select it with .fixedPoint(step, { quantifier: "all" }); the default is "distinct". SQL providers still need an eligible implementation of the specific recursive requirement. Spark admits one closed scalar recursive region with at most eight rounds, 100,000 accumulated rows and 64 MiB of encoded keys. Set recursion must prove equality of the final complete sets. Bag recursion retains duplicate seeds and paths, advances only the previous frontier and must prove that frontier empty. Exhaustion refuses the complete result. Ordered and bag-collection consumers require an occurrence-preserving executor and remain outside this Spark profile; other SQL adapters refuse both recursive modes. Spark recursion executes through bounded disk-backed stages in the captured session. The SQL scope owns and releases those stages. Rendering a standalone SQL string cannot execute that runtime contract. Array functions include distinct, intersection, union, difference, removal, position, minimum, maximum, overlap and joining. Set results preserve first occurrence order; minimum and maximum ignore null members. ARRAYS_OVERLAP preserves SQL’s unknown result when only a null member could establish a match. ARRAY_JOIN skips null members unless given a replacement. GET and array subscripts are zero-based; ELEMENT_AT uses signed one-based positions. Lexical array rewrites share the existing native/portable evaluator and require an installed implementation for foreign placement. They never bypass work or value-size limits. CONCAT_WS skips null values. CONTAINS, INSTR, POSITION, LOCATE and text REVERSE count Unicode scalar values. REPEAT preserves those same character boundaries. LPAD and RPAD count Unicode scalars for text and bytes for binary input. Padding accepts an optional pad value and truncates at the requested length; negative lengths produce an empty value. SPLIT_PART accepts positive or negative nonzero part indices. INITCAP uses pinned Unicode whitespace and casing rules. Shared work and value bounds apply. Calendar helpers include quarter, captured-zone hour/minute/second, weekday/day-of-week/day-of-year, ISO week number, last day and the next declared weekday. Two-argument DATE_TRUNC and DATE_FORMAT use the captured SQL timezone; their explicit-timezone forms preserve the pinned timezone contract. DATE_TRUNC retains a TIMESTAMP_NTZ operand’s civil domain and truncates its clock fields without resolving them to an instant, including clocks inside a DST gap or fold. Date operands become midnight instants in the captured zone; instant operands retain their instant domain. Week and quarter truncation produce local midnight. An instant boundary in a local time-zone gap or fold refuses rather than choosing an instant silently. TO_DATE(value) accepts an exact date, timestamp or ISO date text; TO_DATE(timestamp, timezone) selects a local calendar date. MONTHS_BETWEEN(end, start[, round_off[, timezone]]) accepts exact dates or instants, with literal or bound-constant options defaulting to true and the captured SQL timezone. Matching day numbers or two month ends produce whole months. Other pairs use a 31-day fraction and whole elapsed seconds from each zoned day’s first valid instant. The default rounds to eight decimal places, with ties toward positive infinity. The shared kernel evaluates the formula exactly before its final binary64 conversion, so Spark’s intermediate floating-point artifacts are not reproduced. There is no ambient session timezone or implicit string parsing. Analytical maps use MAP_FROM_ARRAYS(keys, values), MAP(key, value, ...), MAP_AT(map, key), MAP_KEYS(map) and MAP_VALUES(map). Their logical carrier is a sorted array of {key,value} entries, retaining exact integer, decimal, temporal, reference, array and closed-struct key domains. Keys are present and unique; floating keys normalize signed zero and NaN identity. Nulls inside composite keys are values. Unicode, empty and $-prefixed text keys are supported. Missing values, null keys, duplicate keys and unequal constructor array lengths refuse. Entries use the declared domain’s logical order, including struct members in name order. The ordinary analytical array and value budgets apply. MAP_AT returns null when no key matches; typed missing and explicit null inputs retain their independent presence. SQL brackets select keys in the same declared domain: m[2], m[ARRAY(1,2)] and m[:key]. Chained brackets retain each intermediate map or array type. A missing key, null map or null key returns null. Empty map values require a declared key and value type, for example CAST(MAP() AS MAP<INT,STRING>). The typed constructor is q.entryMapFromArrays(keys, values). Its result is an EntryMap<K,V> entry sequence. q.mapAt, q.mapKeys and q.mapValues accept both entry maps and native record maps. The inherited q.mapFromArrays record constructor retains its printable ASCII field-name restrictions and 1024-entry bound. These are separate logical domains. Native and portable kernels share map admission and operations. Spark uses installed shared scalar kernels and an Arrow array-of-struct carrier. Source maps normalize before grouping and equality; ordering uses the shared logical binary key. Other SQL adapters refuse entry maps until their exact kernels are installed. Composite CAST supports typed map keys and values and rechecks key uniqueness after conversion. TRY_CAST requires unchanged key domains. FROM_JSON supports analytical maps with text keys, including Unicode names; JSON objects do not supply other key domains. TO_JSON also renders stable scalar key domains using Spark’s JSON conversion. Declare native map fields and installed warehouse targets with v.map(v.int64(), v.union(v.string(), v.null())). Values are typed {key, value} entry arrays; native argument and return validation rejects null, duplicate or noncanonical keys using the shared SQL key identity. Native maps retain sorted keys and normalized floating-point keys; SQL map constructors produce that canonical representation. Generated SQL validators retain map meaning, exact widths, dates and decimals through round trips. v.record continues to declare native object records. Dataset declarations derive date, instant, civil-timestamp and fixed-decimal storage formats from these native validators, including map keys and values. A conflicting explicit Dataset format is refused; the row keeps its original native validator. Dataset entry_map contracts retain separate key and value identities. The structural carrier uses an outer items record and entry/value wrappers, with physical struct members ordered by admitted field ID. Native publication, Python Arrow batches, IPC transfers and row-page delivery use these same entries. Ingress normalizes key order; retained reads reject duplicate, noncanonical or out-of-order keys. Optional maps keep absence separate from a present null and from an empty map. Exact decimals and civil timestamps retain their declared formats through publication and result spooling. ARRAY_DISTINCT(values) keeps the first occurrence of each exact value, including one null. ARRAY_INTERSECT(left, right) returns distinct common values in the left array’s order. Matching uses the declared element domain without coercion. The typed equivalents are q.arrayDistinct(values) and q.arrayIntersect(left, right). Missing arrays propagate before null arrays in the typed API; SQL input fields follow SQL’s existing missing-to-NULL contract. Native and portable evaluation also support nested elements. Spark, Databricks, BigQuery and Snowflake SQL lowerings preserve scalar-element order and null equality. Nested SQL set equality and non-Spark binary64 set equality refuse during implementation inspection. ARRAY_SORT sorts ascending with nulls last. SORT_ARRAY accepts a literal or bound ascending flag, placing nulls first when ascending and last when descending. FLATTEN removes one nesting level and returns null if a subarray is null. Native and portable execution preserve exact values and charge the existing work and allocation budgets. XXHASH64(value, ...) and q.xxhash64(...values) hash 1–64 typed operands with seed 42, chaining each result into the next operand. Null contributes no bytes; all-null inputs return 42. Boolean and date encodings use little-endian 32-bit words; integers and instants use 64-bit words. Finite binary64 values normalize signed zero. Text uses exact UTF-8 bytes; binary values use exact bytes. Decimal encoding follows its declared precision and scale, and typed arrays hash members in order. Missing values, maps, structs without a declared field order, opaque domains and civil-time values refuse. The result is signed Int64, not a concatenated-byte digest. SHA2(value, bits) and q.sha2(value, bits) return lowercase hexadecimal SHA-224, SHA-256, SHA-384 or SHA-512. Text uses its exact UTF-8 bytes without Unicode normalization; binary input hashes its exact bytes. SQL requires a literal digest size, with 0 selecting 256; the typed helper defaults to 256. Native and portable evaluation support all four sizes. Spark and Databricks lower all four sizes; BigQuery lowers 256 and 512; Snowflake lowers text input for all four sizes. Other size or encoding combinations refuse before execution. These are implementation contracts; deployed provider behavior requires separate evidence.
TO_JSON(value [, options]) serializes closed structs, lists and maps. Integer and decimal JSON numbers remain exact; floating numbers use Java’s text and nonfinite values become quoted strings. Null struct fields are omitted by default, while null array elements and map values remain present. Binary values become base64 strings. Scalar map keys use Spark’s internal string conversion, including epoch days or microseconds for temporal keys. Binary and composite map keys are refused. Map entries retain their canonical logical order. The optional constant text map accepts ignoreNullFields, pretty, writeNonAsciiCharacterAsCodePoint, timeZone, dateFormat, timestampFormat and timestampNTZFormat. Temporal patterns use the admitted English locale. Instant formatting uses the captured SQL timezone; civil timestamps keep their local clock. Default timestamp output has millisecond precision. Both native and Spark scalar execution use the shared kernel and its work budget and 1 MiB output ceiling. q.toJson keeps its native JSON contract: explicit null fields are included, missing object fields are omitted, timestamps use exact UTC microseconds, map keys are text and nonfinite numbers refuse. GET_JSON_OBJECT(text, path) prepares a constant JSONPath once; row-dependent paths compile when reached and charge their work and scratch space. Named keys, array indices and array wildcards follow Spark’s projection rules, including single-match array unwrapping and double-wildcard flattening. A wildcard string result retains its JSON quotes. JSON numbers retain exact source lexemes and duplicate members retain source order. Input must be strict JSON; permissive single-quoted JSON is not accepted. Invalid paths, malformed JSON and absent matches return null. Root/index JSON null is the text "null"; a named null member is a nonmatch. Object-wildcard spellings do not select fields. Paths, nesting, visited values, work and complete output size are bounded; resource or host failures remain errors. JSON Pointer operations retain their separate path contract. JSON_PATH_QUERY(text, path) and q.jsonPathQuery return an array of exact JSON source fragments. JSON_PATH_EXISTS and q.jsonPathExists return whether that same selection contains a node, including a JSON null or false value. These use the core selectors and Boolean filters from RFC 9535: names, wildcards, negative indices, selector unions, slices, recursive descent, existence tests, singular-query comparisons, !, && and ||. Filter queries may refer to the document root with $ and the current candidate with @. Function extensions are refused at admission. Paths are public constants of at most 4096 UTF-8 bytes, including captured SQL parameters. Selectors run in authored order, visiting children in source order. Duplicate occurrences are retained; strings keep their JSON quotes and escapes. A missing match returns an empty array or false. Malformed JSON and SQL null return null; SDK missing inputs remain missing. Comparisons use exact decimal mantissas and exponents without binary floating point or exponent expansion. Input exponent text is bounded to 4096 bytes. Objects compare without depending on member order; duplicate members of the same name retain their source order. A singular comparison query that selects duplicate member names refuses rather than selecting one implicitly. Nesting, matches, intermediate selections, work and allocations retain the ordinary semantic bounds. Complete input and fragment output are each bounded to 1 MiB. Resource exhaustion remains an error. Spark executes the pinned shared kernel; other direct SQL adapters refuse these operations. FROM_JSON(text, schema) accepts a literal SQL type such as 'STRUCT<id: BIGINT, amounts: ARRAY<DECIMAL(38,2)>>'. The root is a struct, array or text-keyed map. Values and array elements may be null; struct members must be present, and unknown members or invalid values refuse the complete parse. Exact numbers retain their declared domains. Parsing and nominal value admission share the caller’s work account. TRY parsing converts malformed input to null while resource, program and unsupported-host failures remain fatal. LIKE and ILIKE accept literal, parameter and row-dependent patterns. % matches any sequence and _ matches one Unicode scalar, including line breaks. Backslash is the default escape; ESCAPE '' disables escaping. An unfinished escape in a row pattern fails when that pattern is evaluated. Null operands produce null. Native, portable and installed Spark execution use the same pattern semantics. REGEXP_LIKE, REGEXP_EXTRACT, REGEXP_REPLACE and REGEXP_EXTRACT_ALL share prepared regex kernels in native and portable execution. Patterns may be literal text, bound constants or row-dependent text, at most 4096 UTF-8 bytes. A program admits at most 16 regex operations. Portable execution retains at most four compiled patterns per dynamic operation. Native readers compile dynamic patterns within their evaluation so protected row text does not enter a shared cache. Compilation work consumes the caller’s budget. An invalid dynamic pattern fails when its branch is reached; null patterns propagate null. Installed Spark execution uses the separate spark.java_regex profile for the complete Java 17 grammar. A task-owned Eclipse Temurin 17.0.17+10 process retains the same Unicode 13 populations. Literal patterns are validated there before input scans; dynamic patterns compile only when reached. The process has a 64 MiB heap, a finite startup deadline, a two-second per-call deadline, bounded work and a sixteen-pattern cache, and is destroyed on cancellation or task completion. BIJECTION_SQL_REGEX_JAVA supplies the absolute child executable when the Spark JVM uses another supported JDK. Driver and executor probes check the exact child runtime. The portable subset explicitly refuses repeated empty matches over supplementary characters with external_runtime_required, since Java advances by UTF-16 code units there. Isolated-surrogate results refuse instead of being changed during UTF-8 encoding. REGEXP_EXTRACT(value, pattern[, group]) defaults to the whole match (group 0). No match and unmatched optional groups return an empty string; an invalid group fails after a successful match. REGEXP_REPLACE uses Java replacement references ($1, ${name}) and backslash escapes. Extraction and replacement use Java’s ASCII defaults for \d, \w and \s, Java horizontal and vertical whitespace classes, ASCII POSIX properties with scoped Unicode alternatives, Java character-property aliases, \Q…\E literal quoting, and lexically scoped U flags for Unicode classes. Category, script, binary and Java character predicates use pinned Unicode 13 populations and Java 17 property names. Java case folding uses ASCII by default and pinned Unicode 13 mappings under scoped u, including the scope at each backreference. Control escapes map a complete Unicode scalar with XOR 64. Named escapes such as \N{LATIN CAPITAL LETTER A} use pinned Java 17 character names, including assigned unnamed ranges such as CJK UNIFIED IDEOGRAPHS 4E00. Names ignore ASCII edge whitespace and case while preserving interior spaces. Java 17 Unicode block properties (InGreek, blk=Basic_Latin, and block=) use pinned Unicode 13 ranges and Java block-name aliases. Blocks retain their ranges under case folding, complements and nested intersections. Word boundaries use Java 17 letter/digit and base-mark rules by default; U selects the pinned Unicode word population independently of case-folding u. Native regex operators retain their Unicode class defaults. Dot excludes Java line terminators; scoped s and d flags select all-character and Unix-line dot modes. Anchors use Java’s final-terminator and scoped multiline rules, with CRLF kept as one boundary. \\R matches CRLF or one Unicode line break, and \\e matches the escape character. Lookahead, fixed- or variable-length lookbehind and numeric or named pattern backreferences use a pinned backtracking kernel when the linear automaton cannot represent the pattern. Each search permits at most 256 backtracking attempts; the complete bounded search and capture snapshots consume the caller’s work and byte allowance before matching starts. Exceeding that allowance refuses the statement. \X matches one extended grapheme cluster atomically, and \b{g} asserts a cluster boundary. Both use the pinned Unicode 13.0 rules, including combining marks, CRLF, Hangul, regional pairs and pictographic ZWJ sequences. Java group syntax, group names, hexadecimal escapes and UTF-16 surrogate pairs follow the same bounded profile. Unknown properties and non-Java group syntax refuse. The profile retains its explicit state, work and backtracking bounds; it does not promise admission of every Java Pattern. Installed Spark execution uses the complete pinned Java runtime described above. REGEXP_EXTRACT_ALL(value, pattern[, group]) returns nonoverlapping matches in input order. Group 0 selects the whole match; the default is group 1. An unmatched optional group produces an empty string; no matches produce an empty list. Native SDK patterns advance across Unicode scalar boundaries; SQL uses Java’s UTF-16 progression, with an explicit portable refusal where needed. Input work and complete output allocations consume the existing budget. ARRAY_CONTAINS returns null for a null array or search value. A successful match returns true even when another member is null; a miss returns null when an array member is null, and false otherwise. Native array membership retains its existing total-value equality. sqlSchemaSyntax parses closed analytical schema intent without granting a write. The dataset component’s analytical schema owner admits nullable scalar additions and identity-preserving renames against an exact contract and version. It retains the original version for a replacement build. The accepted schema and version stay readable while the proposed contract is private to rewrite admission. Verified replacement selection publishes that contract and version in the same transaction; cancellation releases the private proposal and its basis hold. The original version remains an exact historical basis. Code-declared schemas change through declaration and deployment. Physical properties stay with the dataset metadata owner. An installed SQLWarehouse target with analytical: true exposes these changes through ordinary client ALTER TABLE updates. It resolves the current catalog definition and selects an exact installed signature from schemaCandidates: sqlWarehouseContract({inputs: {basis: oldRow}, output: newRow}). The ordinary write candidates pool supplies eligible interpreters after the contract changes. DDL runs as one parameter-free autocommit statement against an existing complete current version. The native session owner atomically publishes the verified rewrite, new contract, named selection and original SQL receipt. Retrying after a lost reply recovers the receipt before resolving the newer catalog. Refusal, rollback and expiry cancel the original private schema request. Missing eligible code refuses before schema mutation; explicit data-write transactions keep their separate ownership. Export a named BI configuration for the optional Flight SQL gateway:
Power BI uses the standard Flight SQL ADBC driver through a packaged Power Query extension in Import or DirectQuery mode. SqlView.Generator folds filters, projections, joins, grouping, sorting and row limits using Bijection’s explicit SQL dialect. Unsupported comparer, culture and rounding arguments refuse folding; DirectQuery requires the complete query to fold. Source identity uses the verified endpoint and catalog, with credentials kept in Power Query’s credential owner. This implementation requires a runtime exposing both Adbc.Connection and SqlView.Generator. Tableau uses Other Databases (JDBC), Arrow Flight SQL JDBC 19.0.0 and Generic SQL-92 for extracts. Both use verified TLS and their existing credential dialogs. The exported manifest records configuration and hashes. Actual application, driver and deployment observations establish compatibility separately; a configuration export is not qualification. Native queries also admit right and full bag joins, exact decimal literals, explicit casts and deterministic text operations (LOWER, UPPER, LENGTH, TRIM, CONCAT, SUBSTRING, REPLACE). Decimal arithmetic uses declared precision and scale; overflow, inexact conversions and unsupported domains refuse rather than silently rounding through binary floating point. SQL AVG promotes integral operands to DOUBLE before grouped, distinct and window aggregation. Decimal inputs capture DECIMAL(min(p+4,38), min(s+4,38)) finalization with half-up rounding after exact accumulation. CAST(AVG(value) AS DECIMAL(precision, scale)) overrides that scale. The shared average kernel admits scales through 38. Float64 averages retain their Float64 contract. APPROX_COUNT_DISTINCT(value[, rsd]) returns a present Int64 count with exact error zero, including zero for empty or all-absent populations. The default relative standard deviation is 0.05; an explicit RSD is a non-null literal or bound numeric constant within (0, 0.39], without a decimal precision quantum. This overload shares the existing exact distinct reducer: scalar values use native distinct state, structural values use complete bounded collections, and both retain their ordinary resource limits. It supports grouped and window populations, FILTER, and redundant DISTINCT. APPROX_PERCENTILE(value, fraction[, accuracy]) uses the same exact nearest-rank fallback as PERCENTILE_APPROX, including fraction arrays and grouped, filtered and window populations. Catalog signatures and evaluation metadata state both the standard fallback and explicit evidence forms. APPROX_COUNT_DISTINCT(value, relative_error, confidence) and APPROX_PERCENTILE(value, fraction, rank_error, confidence) use the shared bounded bottom-k implementation. Error and confidence are explicit literal fractions with at most six decimal places; percentile ranks use basis-point precision. Results retain both estimate and accuracy evidence. Unsupported accuracy requests refuse before execution. Approximate percentiles require a numeric operand and omit null and missing values from the population. These explicit evidence overloads apply to grouped populations and retain ordinary executor eligibility. Analytical windows include ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD, FIRST_VALUE, LAST_VALUE, NTH_VALUE, NTILE, COUNT, SUM, AVG, MIN, MAX, population/sample variance and standard deviation, PERCENT_RANK, CUME_DIST, CORR, COVAR_POP and COVAR_SAMP. Value and offset windows support RESPECT NULLS (the default) and IGNORE NULLS. NTH_VALUE(value,n) uses a one-based literal or bound integer position up to 1,000,000. Frame offsets and offset/window bucket arguments infer Int64 prepared parameters, including named windows. Aggregate windows accept FILTER, and COUNT_IF is available as a window. LAG and LEAD accept a literal or bound signed integer offset with magnitude up to 1,000,000. A negative offset reverses the navigation direction; zero selects the current row, including null values. COUNT, SUM and AVG also support distinct frame populations. Implicit frames include all peers, including nullable, textual and multi-key ordering. GROUPS frames count those peer populations. Explicit ROWS frames count occurrences. RANGE frames with only current-row and unbounded endpoints also support arbitrary peer keys; finite-distance RANGE frames require one present numeric/date/timestamp coordinate. Native average and statistics windows retract exact accumulator state as rows leave a sliding frame. SQL positional windows use retained occurrence order within tied peer groups; ranking and RANGE membership preserve the original peers. Native onTie: "error" windows still refuse unequal tied rows. Windows preserve input occurrence identity and share the native intermediate-work budget. Foreign placements still require an installed implementation of each operator and carrier. LAG and LEAD accept an optional third default argument. It applies only when the selected row is outside the partition; an existing row containing null stays null under RESPECT NULLS. IGNORE NULLS counts present values and uses the default when no such value exists. Defaults preserve ordinary lazy conditional evaluation. SQL keeps SHORT (Int16), INT (Int32), LONG (Int64), FLOAT (Float32) and DOUBLE (Float64) as distinct logical domains, including inside arrays and structs and through prepared parameters and Arrow results. Integral SQL literals use Int32 when they fit, otherwise Int64. Integer arithmetic uses the wider operand domain and checks each intermediate result; a later operation cannot cancel an earlier overflow. Float32 operations round each intermediate result to the nearest representable Float32, with ties to even. SQL floating domains keep IEEE NaN, infinities and signed zero, including overflow to infinity. Native finite-number domains keep their finite checks. SUM widens integers to Int64 and floating values to Float64. Native value types can retain these widths with integerBits: 16 or integerBits: 32 on a bigint base, or floatBits: 32 on a number base, in defineValueType constraints. Admission checks the exact domain; it does not round stored values. Use CAST(... AS FLOAT) to convert a value, or Math.fround(value) before admitting a Float32 SDK value. Text casts and TO_JSON format Float32 using its own precision. SQL floating string casts use Java 17 spelling, including signed zero, exponent thresholds and special values. Numeric text accepts Java decimal and hexadecimal floating syntax and its f, F, d and D suffixes. CAST(... AS FLOAT) rounds text, decimal and integer inputs directly to binary32; it does not round through binary64 first. TIMESTAMP_NTZ carries a civil clock with microsecond precision and no timezone. Its canonical value is 2024-03-31T02:30:00.123456, without an offset or Z. That clock remains valid even when a timezone would skip it during a daylight saving transition. SQL string casts accept dates and civil clocks, validate any supplied timezone before discarding it, and truncate fractional microseconds to the six-digit SQL domain. Typed civil SQL literals also truncate to microseconds and reject timezone suffixes. Native casts and native carrier validation keep their exact unzoned contract. CAST(... AS TIMESTAMP) explicitly assigns the query’s captured timezone and refuses a skipped or ambiguous local clock. DATE_FORMAT formats civil fields without offset tokens; CAST(... AS STRING) returns the complete clock with a space between its date and time, for example 2024-03-31 02:30:00.123456. Native declarations use a string base with format: "timestamp_ntz". Prepared parameters and Arrow results preserve the unzoned microsecond domain. The general native unzoned timestamp’s nanosecond carrier remains a separate type. Spark and Databricks retain a native TIMESTAMP_NTZ SQL carrier, canonical clock text across Python transport, and explicit UTC arithmetic for instant conversions. Other SQL adapters refuse this domain until an exact civil timestamp kernel is installed. Numeric functions include ROUND, BROUND, FLOOR, CEIL, SQRT, POWER, LN, LOG, EXP, trigonometric and inverse/hyperbolic functions, CBRT, angle conversion and SIGN. ROUND breaks midpoint ties away from zero; BROUND breaks them to even. The literal rounding scale is between -38 and 38. Exact arithmetic and rounding keep integer and decimal operands in their own domains. SQL / over integers returns DOUBLE, so 3 / 2 is 1.5. Mixed numeric comparisons, arithmetic and conditional results choose a common SQL domain before calling the checked operators. VALUES chooses its column domains across every row before creating physical chunks, preserving integer widths, decimal scales and nullable nested fields at chunk boundaries. Decimal / and % retain integer coefficients and a bounded SQL result domain. Division rounds midpoint ties away from zero at the inferred scale; remainder keeps the dividend’s sign. Zero divisors and results outside the declared precision fail. Prepared queries retain those result domains through CTEs. SQL DOUBLE casts and numeric functions retain IEEE NaN, infinities and signed zero. SQRT returns NaN for negative operands and EXP can overflow to infinity. SQL rounding uses the decimal spelling of a Double, so ROUND(CAST(2.675 AS DOUBLE),2) is 2.68. POWER and transcendental SQL functions convert numeric arguments to DOUBLE and use the pinned numeric kernel. FLOOR and CEIL return LONG for nondecimal inputs; calendar parts, lengths and collection sizes return INT. Invalid domains, overflow and inexact exact-domain results return structured failures with SQL locations. A provider placement without the matching kernel refuses instead of using vendor rounding rules. LN(x) and LOG(x) are natural logarithms; LOG(base,x) uses that base. Logarithms return null for nonpositive operands; a base of one can produce infinity or NaN. Native numeric expressions retain their finite result checks. Trigonometric arguments use radians; ATAN2(y,x) preserves signed-zero quadrants. These functions and EXP convert SQL numeric operands to DOUBLE and use the pinned software math implementation in native execution and the Spark guest. They do not promise correctly rounded transcendental results. SIGN compares within the original numeric domain and returns Double -1, 0 or 1 in SQL. ISNAN(value) returns false for NULL and tests IEEE NaN without treating infinities as NaN. NANVL(value, fallback) evaluates the fallback only for NaN; a NULL first value remains NULL. Both FLOAT arguments preserve binary32, while integral and decimal arguments use DOUBLE coercion. The native and installed Spark paths use the same numeric kernel. FROM_UNIXTIME(seconds [, format]) formats whole epoch seconds in UTC, with yyyy-MM-dd HH:mm:ss as the default. Fractional seconds truncate toward zero before conversion. TRUNC(date, format) returns a DATE at the start of the year, quarter, month or Monday-based week. Its year and month aliases are YYYY/YY and MON/MM; format names ignore case and may vary by row. Invalid date strings, unknown truncation units and NULL inputs return NULL. OCTET_LENGTH counts UTF-8 bytes for text and raw bytes for binary; BIT_LENGTH multiplies that count by eight. BIN renders a LONG in base two, using 64-bit two’s complement for negative values. CHR and its CHAR spelling construct a Unicode scalar value; negative, surrogate and out-of-range code points fail. SUBSTRING_INDEX(text, delimiter, count) returns the prefix for a positive count or suffix for a negative count, counting overlapping occurrences from the corresponding end. Delimiters may vary by row and have no regex meaning. Zero counts and empty delimiters return empty text; too few matches return the original text. LUHN_CHECK checks nonempty ASCII digit strings; invalid digits return false. These functions propagate NULL and use the shared metered kernel in native execution and Spark placement. TYPEOF(expression) returns its declared SQL type name, including nested array, map and struct domains. It resolves scalar, aggregate, window and subquery arguments without running those arguments. ELEMENT_AT(array, index) counts from one; negative indices count backward from the end. An index of zero fails, while an index outside either end returns NULL. ELEMENT_AT(map, key) returns the corresponding value or NULL for a missing key. Both forms propagate NULL; map keys use the map’s declared exact key contract, including analytical entry maps. FROM_UTC_TIMESTAMP(timestamp, timezone) projects an instant’s civil clock in the named zone. TO_UTC_TIMESTAMP(timestamp, timezone) interprets that civil clock in the zone and returns its UTC instant. Both accept row-dependent zones, fixed offsets and Java’s standard short-zone names, using the pinned timezone database. Gaps move forward and folds choose the earlier instant, matching SQL explicit timezone shifts. Invalid zones fail when evaluated; NULL propagates. Literal zones are prepared once, and dynamic zone preparation is metered. UNIX_SECONDS and UNIX_MILLIS floor toward negative infinity, so one microsecond before the epoch is -1 in either result. UNIX_MICROS preserves that exact microsecond value. UNIX_TIMESTAMP([value [, format]]) returns whole epoch seconds, truncating fractional seconds toward zero. With no arguments it uses the statement’s captured admission time. DATE, TIMESTAMP and TIMESTAMP_NTZ inputs ignore the format operand. DATE and TIMESTAMP_NTZ inputs are interpreted in the captured SQL timezone; instant timestamps retain their epoch. Text inputs use that timezone (UTC by default) and the default pattern yyyy-MM-dd HH:mm:ss, or a row-dependent English datetime pattern. Patterns support optional sections, quoted literals, reduced years, numeric and named months, ordinal dates, fractional seconds, offsets and timezone names. Missing date fields default to 1970-01-01 and missing clock fields to zero. Parsing is strict: invalid dates or unmatched input return NULL; unsupported or malformed patterns fail only when evaluated. Literal patterns are prepared once. English zone-name preferences are pinned to Java 17; civil offsets use the pinned IANA database. Explicit pattern zones retain their compatible gap/fold resolution; an unzoned local time in a captured-zone gap or fold returns NULL. FROM_UNIXTIME formats the epoch instant in the captured timezone. RAND([seed]) and RANDOM([seed]) draw finite DOUBLE values in [0,1). An explicit integral seed is constant per request; NULL uses zero. Seeded draws use Spark 3.5.6’s XORShift stream in logical row-occurrence order, including duplicates. Jump-ahead indexing preserves each draw across lazy evaluation and retries. With no seed, admission time and the call site determine a captured seed. SHUFFLE(array) uses that captured context and a per-occurrence stream to permute complete array values, preserving NULLs, duplicate elements and nested domains. Empty arrays stay empty and NULL arrays return NULL. Rejection sampling and swaps consume the enclosing work budget. The shared kernels define these operations across native, portable and installed Spark execution. ST_POINT(x, y) constructs a finite XY Point after SQL numeric conversion. Its coordinates use the explicit planar frame bijection.sql/srid0, with one coordinate unit per metric unit. Values outside longitude and latitude ranges remain valid planar coordinates. NULL and missing operands propagate; nonfinite coordinates refuse and signed zero follows the native geometry normalization to zero. The constructor does not infer WGS84 or implicitly reframe an existing geometry. Native, portable and installed Spark execution share the same constructor. ST_GEOMFROMGEOJSON(text) uses the existing typed JSON parser and geometry admission owner. It accepts the seven native geometry kinds in the same planar SQL frame, preserving collection members and typed empties. NULL text propagates; malformed JSON, invalid coordinates and unsupported geometry profiles refuse. Literal GeoJSON establishes XY or XYZ dimensions without eagerly validating its value; invalid input in an unselected CASE branch remains unevaluated. Dynamic text uses the finite XY profile. Dynamic XYZ, mixed coordinate dimensions, M ordinates and geometries outside native admission remain warehouse gaps. The native geometry work allowance applies to parsing and admission as well as to subsequent operations. ST_H3CELLIDS(geometry, level, fullCover) indexes all seven admitted native geometry kinds, including polygon holes, multipolygons, mixed collections and empty geometries. Both coverage modes produce the same point identities. Line segments use H3 grid paths with Sedona’s bounded safe approximation when the grid cannot connect endpoints directly; full coverage also includes shared neighbors. An unbridgeable full-cover approximation uses its polygon envelope. Polygons select cells by centroid containment, falling back to the polygon’s centroid for tiny shapes. Full coverage expands those cells and covers every ring. Levels are 0–15; coordinates are finite longitude and latitude within their geographic ranges. Native WGS84 geometries and the explicit planar SQL frame are admitted for this geographic interpretation; another Cartesian frame requires an explicit transformation. Native geometry admission still applies, including rejection of duplicate adjacent coordinates and invalid rings. The result is a sorted, duplicate-free array of exact LONG cell IDs with a 100,000-cell ceiling. Cumulative work and allocation charges include the polygon frontier before coverage iteration. Point identities are compared with H3 core 4.5.0 at all sixteen resolutions. Geometry coverage uses 52 reference cases from prebuilt Sedona 1.7.1 and its H3 4.1.1 JNI core, including holes, concave polygons and antimeridian shapes. The SQL admission suite separately checks the native refusal for the four degenerate repeated-coordinate cases. ST_ASTEXT(geometry) formats a declared geometry as WKT using Sedona’s WKTWriter(4) display rules. It preserves supported XYZ coordinates, collection members and typed empties; NULL and missing values propagate. Coordinates use locale-independent decimal text without scientific notation, with the JTS 16-place fractional display precision. The geometry’s declared coordinate frame and admission rules remain in effect. Native, portable and installed Spark execution use the same metered formatter. Arrays support distinct, intersection, union, difference, sorting, flattening, slicing, repetition and bounded integer sequences. Text supports separator-aware concatenation and regex extraction of all matches and match counts. SQL hashing includes SHA-1, SHA-2, MD5, CRC32 and xxHash64 over exact text or binary carriers. These operations use the shared semantic kernels; an executor placement must provide the same operation or retain it as residual computation. Temporal formatting and truncation use the captured timezone when omitted; truncation includes Monday-start weeks and calendar quarters. GROUP BY ALL infers the nonaggregate projection expressions, including expanded stars. Window expressions require explicit grouping expressions. SQL normalizes missing data cells to null. Predicates use three-valued logic: null comparisons are unknown, NOT preserves unknown, and WHERE, HAVING and join conditions accept only true. Restricted data remains restricted. Null join keys do not match. Joins retain duplicate occurrences; unmatched outer join fields are null. An empty global count returns one row containing zero; empty grouped input returns no groups. Other empty SQL aggregates return null. The explicit global empty-population policy does not change ordinary view group declarations. Null-only result columns, null-only arrays and null struct members retain their null domains through SQL and Arrow result decoding. SQL SUBSTR and SUBSTRING count Unicode characters. Positive positions start at one, zero selects the first character, and negative positions count from the end. Their Int32 coordinates clip at string boundaries; negative lengths return an empty string. SQL TRIM, LTRIM and RTRIM remove spaces by default. Standard TRIM(BOTH characters FROM value) syntax selects a Unicode character set; LEADING and TRAILING restrict the side. Null inputs produce null results. Tuple COUNT(DISTINCT a,b,...) uses exact structured argument boundaries and omits a tuple when any argument is null. Grouped and window counts use the same rule, including aggregate filters and empty populations. AVG(DISTINCT ...) uses the same default finalization as ordinary AVG: integral and floating inputs produce DOUBLE; decimal inputs produce DECIMAL(min(p+4,38),min(s+4,38)) with half-up final rounding. A SQL cast converts that result afterward. Decimal casts round half away from zero; numeric casts to integers truncate toward zero and reject overflow. Native declaration casts retain their exact conversion contract. Array casts apply the scalar rules to each element, recursively through nested arrays. They preserve null arrays, null elements and empty arrays. TRY_CAST replaces an invalid element with null. Struct casts still require ordered field metadata. Distinct averages select membership in the original operand domain before converting integral values to Double for accumulation. Adjacent large Int64 values remain separate members even when they share one Double representation. Spark places distinct numeric sums and averages through bounded scalar fragments on the existing customer worker. These fragments use the shared typed row evaluator for exact integers, decimal rounding, null handling and guarded failures. Installation checks the admitted implementation before input scans; each batch owns and disposes its prepared handle. Spark analytical runs can bind foreign geometry kernels through the worker’s BIJECTION_RELATIONAL_ENGINE_CONFIG JSON file. Its optional spatial_runtime is an explicit ExecutionRuntime receipt containing pinned libraries and resources; absent pins allow only shared kernels. Prepare the receipt from the reviewed worker image with ExecutionRuntime.capture(("geometry", "projection")).model_dump() from bijection_worker.spatial.execution_runtime, and retain it with the installed worker configuration. Runtime discovery never substitutes for that receipt. The SQL read identity includes its digest. Each fragment selects only its required pins, so shared predicates, GEOS operations and PROJ transformations can compose in one query. Driver and executor probes reject changed versions or resource hashes before input execution; the original scope owns cleanup. Geometry distances accept integer or finite binary64 metres; use an explicit CAST(... AS DOUBLE) for an SQL decimal distance. Spatial SQL predicates (ST_INTERSECTS, ST_EQUALS, ST_CONTAINS, ST_WITHIN, ST_COVERS, ST_COVEREDBY, ST_DISJOINT, ST_TOUCHES, ST_CROSSES, ST_OVERLAPS, ST_DISTANCE) infer their coordinate frame from declared operand types. They propagate SQL nulls and reject mismatched frames. ST_EQUALS compares topology, including empty geometries, rather than coordinate encoding. Geometry measurements, centroid, boundary, hull, buffer, simplification and overlays bind the existing geometry kernels; distance parameters retain metre units. These bindings do not relabel coordinates or establish external executor eligibility. ST_DWITHIN(a, b, metres) is the inclusive withinDistance test over ST_DISTANCE: an empty operand has no distance and is never within, and null operands propagate. ST_TRANSFORM(geometry, target, pipeline) binds the explicit coordinate-map transform operation with a constant PROJ pipeline and a constant target (WGS84, EPSG:4326, a Cartesian frame name in metres, or a JSON space declaration). Like buffer and simplification it has no native kernel: external compute evaluates it, and the native evaluator refuses it. ST_GEOMFROMWKT(text) (also ST_GEOMFROMTEXT) and ST_GEOMFROMWKB(binary) decode OGC WKT and ISO WKB into the planar SQL frame through the same geometry admission as ST_GEOMFROMGEOJSON. Literal text or bytes establish XY or XYZ; dynamic values use XY. M ordinates, SRID prefixes, EWKB and mixed dimensions refuse. ST_ASBINARY writes little-endian ISO WKB and ST_ASGEOJSON writes RFC 7946 member order; both keep the declared frame. ST_X, ST_Y and ST_Z return a Point’s ordinates and null for an empty Point, another geometry kind or a missing height. ST_MAKEENVELOPE(west, south, east, north) returns an admitted WGS84 box with parallels densified to half a degree of longitude, as native box regions are. West greater than east crosses the antimeridian as one Polygon whose edges take the short arc. Degenerate boxes return their Point, LineString or MultiLineString. The cell scalars read longitude and latitude in WGS84 or the explicit SQL frame, like ST_H3CELLIDS, and return WGS84 geometry: H3_LATLNG_TO_CELL(latitude, longitude, resolution), H3_CELL_TO_PARENT, H3_CELL_TO_CHILDREN, ST_H3KRING(cell, k, exactRing), H3_CELL_TO_BOUNDARY, H3_GET_RESOLUTION and H3_IS_VALID_CELL use LONG cell IDs and the same 100,000-cell ceiling, checked before any cell set is built. ST_GEOHASH(point, precision), ST_GEOMFROMGEOHASH(geohash [, precision]) and ST_POINTFROMGEOHASH(geohash [, precision]) use the geohash-base32-v1 grid at precisions 1–9. Invalid cells, resolutions and coordinates refuse. ST_MGRS(point, precision) writes a WGS84 position as a compact MGRS reference such as 4QFJ1234567889, and ST_USNG(point, precision) writes the same square in USNG’s four spaced parts, 4Q FJ 12345 67889. Precision is 0–5 digits per axis, from the 100 km square to 1 m, and digits truncate, so a reference names the square holding its position. UTM uses the transverse Mercator projection with Krüger’s series to sixth order, with the Norway (32V) and Svalbard (31X to 37X) zones; from 84°N and below 80°S MGRS uses UPS zones Y/Z and A/B. USNG has no polar zones and refuses those latitudes. ST_POINTFROMMGRS(text) returns a reference’s south-west corner and ST_GEOMFROMMGRS(text) its square as a WGS84 Polygon of its four corners. Both read MGRS or USNG text: spaces and case are ignored and the zone number may have a leading zero. Longitudes of a square past the antimeridian wrap. Malformed references, letters that name no square, a precision outside 0–5 and coordinates outside WGS84 refuse. ARRAY_AGG, COLLECT_LIST and COLLECT_SET also operate over windows. An aggregate argument’s ORDER BY sorts the selected members independently of the window ordering that selects the frame. Equal collection keys use source occurrence identity. Without an argument order, lists follow frame order; grouped lists use source occurrence order. Sets use canonical value order; ARRAY_AGG(DISTINCT value ORDER BY value DESC) reverses that order. Null values are omitted and FILTER removes noncontributing rows before collection. Collection bounds refuse oversized complete results. Frame collections currently recompute their selected population and charge that work to the admitted budget. The SDK exposes the same owner through q.window.collectList and collectSet. Native and portable frames accept exclude: "no_others" | "current_row" | "group" | "ties". Exclusion happens after frame selection: current_row removes the anchor, group removes its ordering peers, and ties removes those peers while retaining the anchor only when the base frame already included it. Peers use the complete window ordering. Empty aggregate frames return their ordinary empty result; collections return an empty array. First, last and nth positions use the remaining population, including non-null nth selection. Ranking, lag and lead retain their original partition semantics. Excluded native folds rebuild through the existing charged state; unexcluded folds retain their incremental path. Python uses the same exclude argument. Spark carries original peer identities separately from stable anchor positions through partitioning and spill. It retracts excluded spans from the same exact aggregate state and restores them before advancing the base frame. Positions and non-null positions use the remaining spans. Collection windows use the same exclusion rule before the complete collection reducer. Peer search and membership changes charge the existing work meter. Other adapters refuse frame-consuming exclusions; ranking and lag/lead ignore them. SQL accepts EXCLUDE NO OTHERS, EXCLUDE CURRENT ROW, EXCLUDE GROUP and EXCLUDE TIES after ROWS, RANGE and GROUPS bounds, including shorthand, named and inherited windows. Prepared bounds keep their original count or coordinate-domain constraints. Dataset build queries round-trip the same syntax through their ordinary read compiler. FIRST(value) and LAST(value) retain null payloads by default. Their optional second argument is a bound, non-null Boolean literal controlling null omission; IGNORE NULLS and RESPECT NULLS express the same policy. FILTER and inline WHERE remove noncontributing rows before selection. Grouped calls accept argument ORDER BY, including independent null placement, and otherwise use source occurrence order. They reuse the ordered-collection bounds reported by inspection and refuse a population exceeding those bounds. Window calls use their OVER ordering and frame, retain the existing tie contract, and return null for an empty contributing frame. Spark preserves nullable and nested payloads through the pinned frame-selection runtime. Inline null treatment and WHERE are normalized before binding, preparation and execution. Finite RANGE frames accept nullable numeric, date and timestamp coordinates. NULLS FIRST and NULLS LAST place null peers independently of ASC or DESC; the SQL default is NULLS LAST. A finite bound on a null coordinate selects its null peers, while an unbounded edge can include other populations. Finite numeric distances retain exact reduced fractions with at most 4096 bits in each integer. SQL decimal literals retain their decimal value; approximate literals and binary64 parameters retain their exact binary value. Bound numeric parameters use the same contract, and ROWS/GROUPS still require integer counts. Native relation declarations accept numbers, bigints, or explicit fractions: frame: { kind: "range", start: { numerator: -1n, denominator: 10n }, end: 0 }. Numbers retain their binary64 value; the explicit fraction expresses one tenth exactly. Dates measure days and timestamps measure microseconds. Comparisons do not round endpoints or overflow the coordinate’s input domain. Native relation declarations expose this placement as frame.nullsFirst on a range frame. Spark uses its pinned exact frame runtime for nullable coordinates, fractional distances and distances outside the direct integer range; other adapters continue to report their actual coordinate and offset eligibility. Calendar RANGE distances retain separate signed month, day and elapsed-microsecond components. Month addition clamps to the last valid day, then adds days and microseconds. Date and civil timestamp arithmetic uses the proleptic Gregorian calendar; instant arithmetic uses UTC. Endpoint arithmetic is wider than the input carrier, so a large interval can select a complete or empty population without overflowing a date or timestamp. SQL admits literal or bound quantities, single units, unit sequences, YEAR TO MONTH, and the ordered day/time qualifiers through DAY TO SECOND. Fractional quantities must resolve to whole microseconds; fractional months refuse. Each component has magnitude at most Int64::MAX, and an interval’s text is bounded to 256 bytes. SDK frames accept { months: -1, days: 0, microseconds: 0 }; Python uses CalendarRangeDistance(months=-1). Month-end clamping can move either boundary backwards between adjacent timestamps. Executors search the resulting endpoints exactly and update or rebuild their ordinary aggregate state. Spark’s partition scratch shares a 100,000,000-operation allowance for frame comparisons, preparation and membership changes across all its measures, in addition to its row and spill-byte bounds. Cancellation and the original invocation deadline still apply. Spark collection windows use this same exact comparator inside an executor-side filter, followed by the shared complete collection reducer. This covers fractional, nullable, date, instant and civil/calendar frames, including collections alongside exact statistics. Before allocating a population, checks conservatively fence all contributing members in each partition by the declared item and byte limits and the 8 MiB guest request bound. A separate 100,000,000-comparison allowance is divided across the node’s collection measures; anchors with absent collected values still count toward that work. These partition fences can refuse a partition even when each individual frame would fit. No prefix or approximate collection is returned. Tuple UNPIVOT, such as UNPIVOT ((amount, label) FOR period IN ((jan_amount, jan_label) AS jan, (feb_amount, feb_label) AS feb)), keeps an independent common domain for each output position and preserves input occurrences. The default excludes a tuple only when every output value is null; INCLUDE NULLS retains those tuples too. Expansion uses one input population and a constant ordinal relation, so it does not require a correlated generator. Hive LATERAL VIEW binds EXPLODE, EXPLODE_OUTER, POSEXPLODE and POSEXPLODE_OUTER through the ordinary collection-table-function owner. Views run in declaration order before WHERE and grouping, so a later generator can refer to a preceding view’s columns. Arrays expose col; maps expose key and value in canonical key order; positional forms also expose a zero-based pos. AS column lists rename those outputs and must match their arity. Omitted column lists retain the default names. LATERAL VIEW OUTER retains one row with null generated fields for null or empty collections. Duplicate observations retain separate occurrences. These declarations use the existing native and portable lateral operators and their budgets; external SQL placement retains each adapter’s actual correlated-population eligibility. The complete REGR_* family uses the same pairwise-complete exact moments in groups, windows and native maintained aggregates. A row contributes only when both coordinates are present. REGR_COUNT returns an exact Int64 zero for an empty contributing population; the other results return null. Centered sums, means, slope, intercept and REGR_R2 round their final result once, retaining integer distinctions above binary64’s exact integer range. A zero independent variance makes REGR_R2 null; with nonzero independent variance and constant dependent values it returns one. Spark groups and row frames use the pinned paired aggregate kernel; exact range populations use its retractable frame runtime. Other SQL adapters report their missing exact kernel during placement. #URL_ENCODE, URL_DECODE and TRY_URL_DECODE use the shared UTF-8 application/x-www-form-urlencoded kernel. Space encodes as +; a literal plus encodes as %2B. Encoding preserves ASCII letters, digits, ., -, * and _ and uses uppercase percent bytes for everything else. Decoding follows the pinned Java 21 percent-digit grammar and malformed UTF-8 replacement behavior. Malformed percent escapes fail URL_DECODE and return NULL from TRY_URL_DECODE; work and resource failures still fail either calculation. Null operands return NULL. The SDK equivalents are q.urlEncode, q.urlDecode and q.tryUrlDecode. Spark placement installs the same scalar kernel; other SQL providers refuse until they have an admitted exact implementation. See the Spark URL function contract. TO_BINARY(text [, format]) selects literal hex (the default), base64, utf-8 or utf8, case-insensitively. Hex accepts an odd leading nibble. Its Base64 profile checks the alphabet and final padding before using the shared MIME decoder; the permissive junk skipping of UNBASE64 does not apply. TRY_TO_BINARY returns NULL for invalid conversions or unknown formats. A NULL format returns typed binary NULL. Format declarations are static; work and resource failures still propagate. ENCODE(text, charset) and DECODE(binary, charset) support literal US-ASCII, ISO-8859-1, UTF-8, UTF-16BE, UTF-16LE, UTF-16 and UTF-32, case-insensitively. They pin Spark 4’s non-legacy charset contract. Unmappable characters and malformed byte sequences fail. UTF-16 emits a big-endian BOM for nonempty encoding and recognizes either BOM on decoding; explicit BE/LE forms retain a leading BOM as text. UTF-32 emits big-endian bytes without a BOM and recognizes either decoding BOM. Its Java decoder’s admitted surrogate code units undergo the same final UTF-8 projection: adjacent pairs combine and isolated units become ?. Empty encodings emit no BOM. NULL values or charset literals yield typed NULL. No provider session chooses the charset or error policy. Spark uses the same installed scalar kernel; other adapters refuse. The SDK exposes q.toBinary, q.tryToBinary, q.encode and q.decode with canonical generated format and charset enums. The separate DECODE(value, search, result [, search, result ...] [, default]) SQL form uses ordered lazy CASE branches and null-safe comparison. The first matching result wins; an omitted default is NULL. Invalid expressions in unreached result branches are not evaluated. FORMAT_NUMBER(value, scale) uses comma grouping, a decimal point and exactly scale fractional digits, with half-even rounding. Integral and decimal values remain exact. Floating values round their exact binary value: decimal 2.675 formats as 2.68, while CAST(2.675 AS DOUBLE) formats as 2.67. Negative zero and negative values rounded to zero retain a minus sign. NULL propagates and a negative scale returns NULL. Scale can vary by row; output and bounded numeric scratch are charged before expansion, so an oversized scale fails without a partial string. Formatting uses no provider locale. The SDK exposes q.formatNumber(value, scale); Spark executes the same installed scalar kernel, and other direct SQL providers refuse this operation. Numeric patterns use the separate profile below. TO_CHAR(number, pattern) and its TO_VARCHAR spelling use a fixed numeric pattern. TO_NUMBER(text, pattern) returns the exact decimal precision and scale counted from that pattern; TRY_TO_NUMBER returns NULL for an unmatched input. Patterns admit 0 and 9 digit runs, . / D, , / G, a leading $, leading or trailing S or MI, and trailing PR. Integral runs beginning with 0 require their full input width and format with zero padding; runs beginning with 9 permit fewer digits and format with spaces. Fractional runs permit fewer input digits and format with trailing zeros. Group sizes are checked from the right. Signs and angle brackets preserve the declared pattern rules; unsigned patterns format the magnitude. Parsing accepts Java 21’s BMP decimal digits and whitespace profile. Empty numeric input returns NULL in the try form. Invalid patterns fail admission, including in try forms, and resource failures always refuse the request. Patterns contain at most 128 ASCII bytes and 1–38 digit positions. Formatting does not round: excess integral or nonzero fractional digits produce # in every digit position. Trailing fractional zeros do not cause overflow. Integral and decimal inputs stay exact; floating inputs first use the explicit SQL DECIMAL(30,15) conversion. NULL and missing input propagate. The SDK helpers are q.formatNumeric(value, pattern) and q.parseNumeric(text, pattern, { onInvalid: "null" }). Native, portable and installed Spark execution share the same kernel; other SQL adapters refuse these operations. TO_CHAR and TO_VARCHAR also accept dates, instant timestamps, civil TIMESTAMP_NTZ values and binary values. Datetimes use DATE_FORMAT’s pinned English pattern grammar in the captured SQL timezone; civil fields retain their civil meaning. Their format can vary by row. Binary formats are case-insensitive base64, hex (uppercase output) or utf-8 (strict decoding). Numeric and binary formats must be constant per request. A binary NULL format or malformed UTF-8 refuses the request. Binary selector trimming follows Java’s ASCII whitespace rule for hex and base64; UTF-8 additionally follows the strict charset spelling contract. q.toChar(value, format) selects the same prepared kernel from the declared input domain. Its optional third timeZone argument defaults to UTC and is captured explicitly in the scalar declaration. Native, portable and installed Spark execution use this adapter; other direct SQL providers refuse it. FORMAT_STRING(pattern [, values...]) and PRINTF use an installed spark.format_string kernel on customer Spark compute. It requires Eclipse Temurin 21.0.9+10-LTS on driver and executors and verifies the Java source pin and executor probes before customer scans. Locale is US, %n emits LF, and %t/%T interpret BIGINT arguments as epoch milliseconds in UTC. Text, boolean, Int16, Int32, Int64, Float32, binary64 and exact decimal arguments retain their declared Java primitive or BigDecimal domains; cast other types explicitly. Positional and relative arguments, flags, widths, precision, general, character, numeric and date conversions use that JDK’s Formatter rules. A null pattern returns NULL without evaluating the value arguments. Null value arguments follow Formatter (%s and %h produce null, %b produces false). Invalid patterns, missing arguments and incompatible conversions refuse the complete result. Patterns are limited to 4,096 UTF-8 bytes and 63 value arguments. A prepass refuses excessive width, precision and cumulative expansion before Formatter allocates its intermediates. Output is bounded to 1 MiB with a one-million-step allowance per invocation; exhaustion refuses the result. Native, portable and other direct SQL adapters report the missing JVM kernel instead of substituting a provider formatter. These implementation checks do not qualify a deployment.

Ontology SQL acceptance and warehouse scope

The read-only acceptance boundary is the Ontology column of Palantir’s SQL dialect reference and its Ontology SQL contract, reviewed on November 2, 2026. It contains 62 supported function spellings and 17 operators. The SDK acceptance suite keeps an executable case for each, including null populations, aggregate inputs and window frames. It evaluates the packaged shared semantic guest and checks admission to the installed exact Spark lowering. Correlated EXISTS uses the native executor; the Spark lowerer explicitly refuses that requirement. The application SQL suites separately cover filtered population providers, interface discovery and visible implementations, relationship joins, native series scopes and structured keys, DECLARE and complex parameters, metadata filtering before bounded construction, positional result labels and the 10,000-row result allowance. Prepared plans retain their selected branch or scenario context and re-admit current authority when bound and executed. This is source and host-test evidence. It does not establish observed desktop application compatibility, deployed authorization and lifecycle behavior, or performance. Every external executor still has to admit the exact requirements of a plan; connector availability alone does not imply that admission. Compute remains delegated to external installations. The warehouse acceptance inventory records all 219 supported function spellings in the same published Furnace column. Each has an executable example across the Ontology and warehouse suites, with exact Spark admission checks for query expressions. CAST, TRY_CAST and the Z-order procedure use their statement grammar. The suite also checks captured clock and random occurrences, aggregate populations and window frames. These examples establish coverage of the published inventory; additional overloads and deployed physical execution remain separate acceptance checks. Warehouse writes and durable transaction sessions use the opt-in installed SQLWarehouse described below. The ordinary ontology gateway remains read-only. The warehouse compiler also accepts INLINE and INLINE_OUTER in table, SELECT-generator and LATERAL VIEW positions. STACK(rows, values...) expands row-major cells into col0, col1, and subsequent columns, pads an incomplete row with nulls, and requires one exact declared domain per output column. RANGE exposes Spark’s default id column. The inline forms preserve declared struct field order and expand null elements to a null field population. The outer form produces one null row for empty or null arrays. NAMED_STRUCT declares ordered fields with literal names. These Spark table functions lower to the existing bounded lateral and unnest operators. JSON_OBJECT_KEYS preserves input key order and duplicate key occurrences. JSON_ARRAY_LENGTH counts outer array elements. Both return null for malformed JSON and the wrong root container, accept Hive single quotes and raw control characters, and read the first complete container. They use separate shared SQL kernels; SDK JSON Pointer operations retain their own strict JSON contract. JSON_TUPLE(json, field, ...) returns one positional row of nullable strings in SELECT, table and LATERAL VIEW forms. It retains repeated requested fields, uses the last non-null occurrence of a duplicate member, and renders nested containers as compact JSON. A null or non-object input produces a null row; dynamic field expressions run only after an object start token. JSON floating numbers use Spark’s Java 17 output representation. TIMESTAMP(value) uses the same checked ANSI conversion as CAST(value AS TIMESTAMP), including numeric seconds and date inputs. POSEXPLODE positions use INT; its outer form admits a null position for an absent collection. SQL JSON key arrays retain Spark’s nullable element declaration. FORMAT_NUMBER(value, precision_or_pattern) also accepts dynamic text patterns. Its US pattern grammar supports required and optional digits, grouping, quoted affixes, positive and negative subpatterns, currency, percent, per-mille and scientific or engineering notation. Patterns are bounded to 128 UTF-8 bytes. Decimal coefficients retain exact half-even rounding. Floating operands use Java 17 decimal digits and their original binary value to resolve rounding ties. IEEE NaN and infinities render as NaN, ∞ and -∞ with the applicable affixes. Native, portable and exact Spark scalar execution share this kernel. TO_DATE(value, format) parses a date format that may vary by row, including numeric inputs converted to text. TO_TIMESTAMP(value) uses checked SQL conversion, so a date or a date-only string produces midnight and numeric inputs represent seconds. With a format, it parses the input’s text in the captured SQL timezone, retains microseconds from fractional seconds and respects explicit offsets and zone fields. Null formats produce null; invalid input or patterns refuse under ANSI rules. Prepared date and format parameters infer text without sampling. Explicit timezone conversion uses FROM_UTC_TIMESTAMP or TO_UTC_TIMESTAMP. MONTHS_BETWEEN(end, start [, roundOff]) coerces date and text inputs to timestamps in the captured SQL timezone. Its nullable boolean rounding expression may vary by row. The calendar kernel rounds to eight decimal places by default, ignores time of day for matching month days or two month ends, and otherwise uses a 31-day month. The explicit fourth timezone argument continues to admit typed temporal inputs. TIMESTAMP_SECONDS accepts fractional numeric seconds, including negative microseconds. Decimal inputs must represent exact microseconds; floating inputs truncate submicrosecond fractions, and NaN or infinities produce null. Ordinary timestamp casts retain their own numeric truncation rules. Calendar extraction, LAST_DAY, ADD_MONTHS and time-of-day extraction use checked date or timestamp conversion for text inputs. UTC DATE_TRUNC accepts year/month/day aliases and returns null for an unknown or null unit; its unit expression may vary by row, and an unknown unit does not evaluate the timestamp expression. Its complete UTC lowering also covers week and quarter boundaries. NEXT_DAY accepts a row-varying weekday name, preserves nulls and returns the following week when the input already has that weekday. The explicit non-UTC extension keeps strict timezone disambiguation. Installed Spark execution uses the shared scalar evaluator for hashes, JSON extraction, CONCAT_WS, INITCAP, array flattening and exact array percentile and extremum-by reducers. Provider admission includes their checked failures and nested value carriers. Databricks, Snowflake and BigQuery also lower Unicode and binary LPAD/RPAD directly. The adapter preserves missing/null propagation, negative and zero lengths, empty-pattern truncation and checked Int64 lengths before invoking the provider’s padding operation. It supplies an explicit binary pad and guards the BigQuery empty-pattern error; Databricks and Snowflake retain their own physical scalar carriers. These lowerings have source-level checks; deployment qualification remains separate. Warehouse CREATE retains USING, PARTITIONED BY and TBLPROPERTIES requirements until the Dataset owner checks the actual destination binding. Managed Iceberg targets accept matching identity, year/month/day/hour, bucket and truncate transforms. Partition columns resolve to the owner’s stable field identities. The installed writer supplies write.format.default=parquet. A different layout or unsupported property refuses before candidate admission. Standalone build candidates require the same destination admission for these physical requirements. CREATE TABLE ... USING parquet and USING avro require a matching managed file-backed Dataset destination. Install it with datasets.bind(dataset, files.manifest({ namespace, rowFormat: "parquet" })) or "avro". The existing candidate and publication owners accept immutable file versions; the provider name selects the actual file encoding. Parquet writes GZIP data pages. Avro writers select raw DEFLATE when it reduces the file size. Readers check compressed and decoded sizes against the file profile. These destinations accept neither Iceberg partition clauses nor table properties. Bounded native queries read exact retained versions through the Dataset owner, checking current access on each page. File snapshots have no append change log, so VERSION BETWEEN requires a change-log representation. Interactive Spark runs continue to require retained Iceberg inputs. Dataset-owned managed Iceberg tables admit ALTER TABLE partition add/drop/replace with aliases, multiple transforms on a source column, write ordering with ASC/DESC and per-column null placement, identifier fields, and bounded table properties. Identifier fields require present non-null primitive columns without floating-point domains and remain separate from the Dataset row key. A checked type conversion allocates fresh field identities and remaps the physical policy to those identities. Dropping a column still used by the resulting policy refuses before execution. A pending revision captures the complete storage binding alongside its schema and defaults. Publication selects them together. Writers select the exact captured policy even after a cancelled write changed table defaults; historical snapshots, schemas, specs and orders remain retained. Policy changes rewrite the files and disable snapshot reuse. Spark applies Iceberg’s distribution settings; bounded streaming writers route partitions and sort each file. Exported provider files that need repartitioning, ordering or a different codec stream through that writer. An explicit range distribution sorts the complete bounded streaming write before splitting files, with a 64 MiB sort allowance. Larger range sorts use the external Spark writer. Distributed fragments apply local sort and codec settings but require an unpartitioned destination and refuse uncoordinated range ordering before tasks start. Native publication checks the selected snapshot’s identifiers and each file’s spec, sort metadata, partition values, row ordering and explicitly requested codec, within its scan allowance. Mutable operational properties are write.format.default=parquet, write.parquet.compression-codec (zstd, snappy, gzip or uncompressed), write.distribution-mode (none, hash or range), and write.target-file-size-bytes (1–64 MiB). The file size is a writer target. Comments and user.* metadata retain their bounded text profile. Retention, Dataset ownership and other reserved properties keep their existing owners. These statements stage privately in an explicit SQL session. Later reads and writes in that session use the accepted private schema, defaults and storage policy. COMMIT validates every table’s original selection and binding, then publishes all schema and selection changes in one metadata transaction. Autocommit uses the same staging and commit engine. A failed schema statement restores its preceding private revision; rollback and expiry release every remaining reservation. Allocated field IDs are never reused. CALL zorder('target', 'column1,column2') rewrites the captured complete population through an installed Spark warehouse candidate. It range-shuffles and sorts by a Morton key on external compute, then writes the original rows to the existing run-owned Iceberg branch. The Dataset publication group and transaction conflict checks still decide publication. Duplicates and exact integer/decimal values survive the rewrite; zero logical rows are affected. One to eight distinct primitive columns are supported. Each dimension contributes 128 ordered bits; text and binary use their first sixteen bytes. An installed linear per-file sort policy refuses before tasks start. The ordinary read-only gateway and portable warehouse interpreter refuse the procedure. The native writer keeps the original Spark job group and branch cleanup owner, including when a write response is lost. Managed analytical tables declare schemaOwner: "dataset" on their deployment storage binding. The native binding owner records creation custody before creating the Iceberg table, preserves the Dataset’s exact field IDs, and pins the resulting table UUID before installation. Optional identifierFields names resolve against that initial declaration. Runtime SQL then owns schema, defaults and storage-policy evolution. Replaying the original deployment declaration preserves those revisions; changing that original declaration or moving ownership requires resetting the installation. Existing unmarked tables cannot be adopted as analytical tables. These tables use the existing publisher lifecycle for retention and closure. They never enter the native change-log writer. Native maintenance expires unreferenced snapshots and cleans orphan objects where the catalog declares managed maintenance; analytical compaction uses the installed external SQL writer. SQL targets remain declared application resources. Captured SQL context accepts collation: "unicode_ci" and collation: "unicode_turkic_ci". Both use the pinned ICU4X 1.5 case mapper and normalizer, with NFC before and after case folding and UTF-8 key order. Accents remain significant. The default Unicode fold makes Straße and STRASSE equal. The Turkic fold pairs I with ı and İ with i. Both profiles use byte ordering after the fold. unicode_sv_ci uses Swedish alphabetic order, with Å, Ä and Ö after Z. unicode_de_phonebook_ci uses German phonebook order, with umlauts sorted alongside their letter expansions. The other locale profiles are unicode_es_traditional_ci for traditional Spanish, unicode_fr_ca_ci for Canadian French, unicode_cs_ci for Czech, unicode_pl_ci for Polish, unicode_da_ci for Danish and unicode_zh_pinyin_ci for Chinese pinyin. All eight profiles use pinned ICU4X 2.4.0 data at secondary strength, which ignores case and retains secondary distinctions. The nonbinary profiles apply to SQL comparisons, membership, joins, grouping, distinct and set operations, ordering, window partitions and peers, collections and extrema. Original values remain the delivered payloads. A grouped key uses one complete original tuple, chosen in binary order; equal extrema choose the least original value. References and source/request IDs keep their exact identity. Map member identities stay binary while their values can contain collated text. Native relations supplied as SQL inputs retain their own semantics. Recursive UNION DISTINCT requires binary collation. Recursive UNION ALL is supported. Collections retain their explicit item, byte and work bounds, including before post-collection distinct or extrema selection. A provider must execute the pinned kernel or refuse the placement; its ambient COLLATE or LOWER function cannot supply this profile. The broader warehouse and analytics surface has these remaining boundaries: Analytical request-time and captured relational plans share the 16 MiB logical program ceiling. Their complete hashed manifest has a separate 17 MiB allowance for the program and calling metadata. The worker claim transport allows at most 32 MiB of escaped JSON and checks that bound before decoding. Ordinary Python and captured-engine manifests, scalar arguments and worker messages keep their own smaller bounds. These are admission ceilings, not measured capacity claims; the selected executor must still satisfy the plan’s work and memory requirements. Captured relational jobs and request-time analytical runs on Spark use the same checked SQL scope as ordinary Spark functions. It installs the plan’s exact scalar and Java regex kernels, resolves runtime window and recursion stages, and validates the complete result before sealing it. Original Dataset references retain custody of input snapshots; captured Spark views are derived inputs within that attempt. The compiler rechecks the captured program’s mathematical identities, and the scope rechecks view identity, cancellation and the original deadline. The existing engine owner and SQL scope clean up their views, retained runtime frames and snapshot registrations. Compiler input and output have separate 32 MiB and 64 MiB transport ceilings; an oversized or stalled compiler is terminated before its result is decoded. Analytical results admit complete typed values separately from native records and arrays. Each row is bounded to 16 MiB, each collection to one million items, and logical admission to four million visited values. The complete byte and row limits of the request still apply. A sealed analytical computation uses result files of at most 32 MiB and pages large enough for one admitted row; ordinary results retain their existing page and chunk allowances. Pending builders retain and recheck the selected limits across checkpoints. Exact integer, IEEE, binary and missing-value carriers remain canonical throughout admission and paging. Flight SQL Arrow construction uses the shared collection and nesting ceilings and a separate 256 MiB allocation-charge allowance per page. Gateway result and IPC transfer ceilings remain independently configurable. Its result accounting charges 32 bytes per visited value plus payload bytes; a million-element list therefore needs an appropriate gateway result budget even when its Arrow buffers are much smaller. These admission charges do not assert a measured process-memory or provider-scan bound. The sqlBuildSyntax authoring helper accepts CREATE TABLE target (id BIGINT NOT NULL, label TEXT DEFAULT 'new') as an empty typed candidate. Binary, arrays, text-keyed maps and structs are supported, including nested combinations. Declared column and struct-field order survive the empty candidate. NULL/NOT NULL control its inferred output shape. Within a sqlTransactionSyntax, a schema declaration’s constant defaults fill omitted INSERT columns in later parsed statements; CREATE OR REPLACE resets that private authoring metadata. Installed warehouse targets declare their defaults with SQLWarehouseTable.defaults: maps from target column names to closed constant SQL expressions, such as { amount: "CAST(0 AS DECIMAL(18,2))" }. The expressions are captured in the typed computation; they admit literals, casts, scalar operators and registered pure scalar functions, such as ABS(-22) or UPPER('new'). The shared function registry controls accepted signatures. Defaults cannot refer to rows, parameters, subqueries, aggregate or window functions, clocks, randomness or provider functions. Defaults apply only to omitted columns, so a supplied NULL is checked against the ordinary target contract. SQL candidates assign source values to the declared target domains through the shared checked scalar kernel: numeric widths, strings, nested fields and required values retain their target contracts. Invalid conversions and NULLs for required fields refuse the candidate, including when a later transaction statement replaces that candidate. The direct Spark lowering uses this same kernel; other direct SQL adapters refuse a candidate that needs a conversion they cannot execute exactly. These authoring defaults do not alter a dataset’s installed schema or create provider defaults. A runtime CREATE declaration does not change the installed warehouse’s defaults. Arbitrary provider options and constraints are refused. Dataset-owned analytical targets also accept runtime ALTER TABLE. Additions can declare SHORT, INT, FLOAT, decimal, date, instant, civil timestamp, binary, struct, array and typed map domains, including nested combinations. Required additions declare a closed default. ALTER COLUMN ... TYPE ... USING ... evaluates the conversion against the preceding schema; operations in one statement see earlier renames and conversions. Changed physical types receive fresh field IDs. Row keys and business-time identities retain their existing contracts. Grouped ADD COLUMNS (...) and DROP COLUMNS (...) use the same closed column operations, including nested types with commas. Additions and ALTER COLUMN ... COMMENT preserve Unicode descriptions in physical Iceberg field metadata. Comment-only changes keep field identities, logical contracts and defaults; retained schemas keep their previous descriptions. A later type or nullability change carries the description onto its newly allocated physical field. Dataset FLOAT and DOUBLE columns retain NaN, infinities and signed zero through analytical row validation, program ports and native reads. FLOAT still requires an exactly representable binary32 value. Explicit finite scalar refinements retain their narrower domains. Analytical map keys use the same typed IEEE ordering and duplicate rules as the shared semantic engine. Narrow numeric Dataset contracts retain their logical widths in nested fields and exact C1 Arrow. Iceberg stores SHORT in an INT column with checked 16-bit range admission; INT and FLOAT use native 32-bit storage. Writers refuse out-of-range integers and floating values that are not already exact binary32 values. Explicit SQL casts perform the requested rounding before that boundary. DROP COLUMN retires its field IDs and defaults; a later same-name addition receives fresh IDs. Dropping a row-key or business-time identity refuses. SET NOT NULL checks every rewritten row and refuses a population containing NULL. DROP NOT NULL, SET DEFAULT and DROP DEFAULT use the same original schema reservation. Runtime defaults become visible with the accepted schema and version, so later omitted INSERT columns use the published expression. Cancellation retains the preceding defaults. The original Dataset and SQL session owners decide publication; a rewrite never creates provider defaults. The managed Iceberg writer selects the exact admitted schema and preserves historical schema IDs and Parquet aliases. Python writes stage schema and snapshot changes in one catalog transaction. Native external writers prepare schema metadata before writing their private branch; the Dataset publication still selects the contract and rows together. Physical CREATE clauses bind the Dataset owner’s deployment configuration:
USING must match the installed storage provider. PARTITION BY accepts one to four declared transforms: a column or identity(column), year, month, day, hour, bucket(count,column) and truncate(width,column). Identity-only PARTITIONED BY (column,...) is also accepted. These clauses assert the existing Dataset layout; the binding owner provisions its physical table. Table metadata uses bounded comment and user.* text properties. Preparation seals the exact binding revision. BEGIN captures every target’s physical revision, and the Dataset session owner rechecks written targets at admission, candidate staging and commit. A changed placement refuses publication while rollback and original receipt recovery remain available. Retained historical read bases continue to use their own exact physical identities. CREATE TABLE target (id SHORT NOT NULL, label STRING) AS SELECT x, y FROM incoming also assigns by position. The shared candidate compiler checks each value against the declared columns before assigning it to the installed target contract, so a wider installed column cannot hide overflow or a violated NOT NULL declaration. Prepared CREATE and INSERT statements infer direct parameter domains from these assigned positions without sampling values. Structured and binary cells retain their domains through the same path. INSERT accepts optional INTO with either VALUES or a query source; both forms use the same candidate, prepared-parameter and affected-row semantics. INSERT INTO target (amount, id) SELECT x, y FROM incoming assigns by position, rather than matching source labels. An INSERT without a column list assigns all values in the target’s declared column order. Supplying the closed target columns to the helper orders the candidate by that contract and fills omitted columns with declared defaults or typed NULLs from the target schema. The ordinary output contract rejects missing required values. These are inert build declarations: publication and write authority still come from the existing Dataset owner. The ordinary ontology gateway is read-only; the installed warehouse mode below routes writes through these owners. SQLWarehouse accepts context defaults. Its begin method and exported BEGIN function accept a per-session context; Datasets.beginSQLSession also accepts context. The session stores the normalized settings in sqlContext and includes them in its immutable declaration. Reusing a BEGIN key with different settings refuses. Prepared statements and write candidates use the session’s original settings during recovery, including after warehouse defaults change. An installed warehouse accepts a closed block containing one to sixty-four write or schema statements between BEGIN and COMMIT:
The block owns a private SQL session. Statements run in order against its evolving row contracts; each complete candidate stays private until the final atomic group commit. Repeating the original request resumes its pending child or recovers its accepted statements. The first child binds the complete block, so reusing its identity with changed SQL refuses before altering the original work. A later refusal rolls back the session and reports SQLBlockRolledBack. Blocks admit up to 128 named parameters in writes. Prepare infers each domain from the row contracts at that statement, including earlier schema changes. Use an explicit CAST if the context leaves a parameter ambiguous. Repeated names must agree on the domain. The warehouse validates the whole parameter map before opening the session and fixes the values in the original request identity. Each child receives the parameters it uses, and its write is checked against the current private schema. Schema statements remain closed. Blocks do not nest or return query result sets. Strings and comments do not create statement boundaries, and each child preserves its original source positions. Warehouse targets with managed Dataset retention accept named retained versions:
Tags and branches share one name namespace per warehouse and Dataset. Unquoted names normalize ASCII letters to lower case; quoted names are exact. CREATE supports IF NOT EXISTS or OR REPLACE; REPLACE requires an existing name of the same kind. Omitted versions capture the current complete private candidate or the session’s original selection. References retain the exact accepted version and its row contract, including earlier analytical schemas. A prepared named read freezes that version; moving the name requires no change to the prepared read, provided the version remains retained and current access still permits it. Reference statements need read and write authorization but no Compute candidate. Inside a session, later statements see pending reference changes. Reference holds remain independent of later table writes. COMMIT compares every reference against its initially observed revision and changes all reference and table pointers in the same Dataset transaction. Rollback and expiry release private holds. Names do not grant provider catalog authority or select a provider branch; REPLACE BRANCH explicitly moves a retained Dataset pointer. Provider retention settings and combined reference/column ALTER statements are refused. For application-owned warehouse sessions, Datasets.beginSQLSession freezes the current revisions of up to sixteen named Dataset selections, including absent writable targets. The application authorizes every target and checks current policy on each subsequent operation. The handle is bound to its authenticated principal and owner and expires within one hour. resumeSQLSession recovers the original pending compiled meaning or a terminal acceptance/refusal before the application compiles or dispatches work again. admitSQLSession reserves a stable request, its compiled meaning and its exact session revision before any external execution. Every row ID in an atomic batch is reserved under the principal and warehouse owner, including across sessions. A row ID cannot be reused as part of another batch, and changing an admitted request or program is refused. One pending statement fences new admissions, new statement planning through sqlSession, and COMMIT. stageSQLSession attaches complete accepted Dataset versions to that pending admission and preserves its row count for retries. It never admits a new statement implicitly. Invalid or incomplete outputs leave the admission pending for recovery. refuseSQLSession records an execution failure and releases the statement fence. Both acceptance and refusal advance the session revision and remain recoverable after the session ends. Staging does not move public selections; sqlSession reads the resulting exact version set once the pending statement ends. Accepted versions retain their ordinary Dataset access policies: an unselected version is not an access control boundary. commitSQLSession checks every original selection revision, including read dependencies and absence, and moves the final candidates together through the existing publication and selection owner. A changed selection refuses the commit even if it was subsequently restored to its original version. The accepted commit receipt is recorded in the same transaction and survives a lost response. rollbackSQLSession and expiry release session holds without moving selections. Pending statements receive a durable session-ended refusal; late results cannot become candidates. Resuming an expired pending statement performs this cleanup even before the scheduled expiry runs. Sessions permit at most sixty-four row request IDs, including failed statements and every member of a batch. sqlSessionReadContext retains either the private read-your-writes basis or an explicit set of exact versions from the session’s declared Dataset members. Repeated reads of the same basis reuse its custody, later writes preserve earlier in-flight reads, and commit, rollback or expiry closes every retained basis. A session admits at most sixty-five distinct read bases. Autocommit reads share exact-snapshot contexts within a lease window, scoped to the authenticated principal; repeated queries of the same basis reuse custody. sqlSessionStatus recovers owned metadata after session end or version retirement, and sqlSessionStatements lists the bounded original statement keys for recovery and cleanup. Neither operation reopens version custody. An application can bind a current executionAccess query when beginning a session. A native program frame can then read status only after that query verifies its actual Compute caller against the original session and principal. The status query observes the lease deadline so executor policy cannot cache an open session beyond its expiry. Session mutations and statement locators remain scoped to their authenticated principal. An installed warehouse also accepts historical reads:
The version string names an accepted retained Dataset version. Preparation checks its Dataset and exact row contract, seals its identity with the original SQL and native program, and calls the warehouse’s current read policy with that version. Execution repeats the checks and acquires the existing exact-version read context. A transaction owns historical contexts through commit, rollback or expiry; autocommit uses the ordinary finite snapshot lease. Input and result delivery retain their current Dataset source policies. Retention grants no read authority. One logical table name uses one basis throughout a query; mixing historical versions or current and historical occurrences of that name refuses. CTEs cannot select independent retained versions. The parser removes the historical clause before semantic compilation while preserving the remaining source positions, so providers never reinterpret the requested version. VERSION BETWEEN 'from-version-id' AND 'to-version-id' returns every original insert occurrence with a commit after the first endpoint and through the second. Both endpoints must be retained versions of the same Dataset contract and change-log generation. Preparation and execution authorize both versions. The existing read context and query checkpoint owners retain and release both. The retained basis records the start version and distinguishes the append population from a whole snapshot ending at the same version. Repeated appended values remain separate occurrences; ranges containing updates, deletions or erasures refuse, as do rewrites that replace retained history. The complete interval is validated before a result is sealed. An empty interval returns no rows. The bounded native Dataset reader and installed Iceberg SQL executors on Trino, Presto, Snowflake and ClickHouse support this population. The executor reads the end snapshot and checks the whole interval in the same statement, independent of filters and limits on the result. Range contexts cannot vend a whole snapshot, snapshot credentials or whole-snapshot artifacts. Ordinary Spark functions can query an admitted datasetChanges(row) range with ctx.datasets.spark_changes(value): a distributed relation of original log events, including removals, over the retained endpoint snapshots. This does not extend the standard-client VERSION AS OF grammar. These APIs provide Dataset custody and commit metadata. SQLWarehouse composes them with installed Compute candidate functions and exposes standard-client session routing. sqlWarehouseMeaning compiles one SQL write against authorized closed table contracts and the captured target’s presence. It reuses the existing SQL candidate lowering, including assignments, defaults, and ordered MERGE checks. It reports exact affected counts for CREATE, replacement, INSERT, UPDATE, DELETE and ordered MERGE branches. Matching UPDATE rows count even when their values remain unchanged; MERGE excludes DO NOTHING branches. A batch validates each intermediate candidate and sums the individual affected populations. The returned compiled meaning and exact input names can be bound to a session admission before starting the installed candidate function. The warehouse’s engine selects python (the default) or spark. Prepared writes bind that selection, and the original statement intent retains its compiled placement across retries. The source-captured candidate interpreter is available as bijection_sql from @bijectionhq/pipelines. sqlWarehouseContract() declares one ordinary internal action that serves the warehouse’s current row contracts. Exact input references and the captured typed graph carry each statement’s domains. The Dataset resource owner binds the original destination before allocating I/O. Pass sqlWarehouseLibrary to python(..., { libraries }) to capture the interpreter’s actual source; the Python entrypoint can import and call bijection_sql.materialize(ctx, args). Pass sqlWarehouseRequirements as the Python implementation requirements so admission requires an executor with Node and the compatible reference evaluator. Install that action through the ordinary Compute job and functionResources configuration. Set destinations: {} and resolveDestinations: (ctx, args) => warehouse.resourceDestinations(ctx, args). This binds /rows to the original statement’s exact Dataset contract, including a schema rewrite’s new row definition. The same interpreter handles later schemas without additional predeclared action variants. The action accepts compiled, inputs and the original session context. The compiled JSON has exactly program, identities and encoded scalar arguments; its typed relational graph returns rows and affected, the latter containing one { count } int64 row. Each non-null input is an exact Dataset reference named by that graph; unused or absent declared inputs are null. The interpreter checks the shared semantic identity, reads complete inputs, evaluates all outputs, and stages the complete candidate through its declared function result store. It refuses an input or output above one million rows or 64 MiB; it never uses a preview or the gateway’s read limit as a write candidate. Its result contains rows and affectedRows. It moves no public selection. This is an installed materialization primitive used by SQLWarehouse. For native Spark execution, install the candidate with sqlWarehouseSparkRequirements, set the warehouse’s engine: "spark", and call bijection_sql.materialize(ctx, args, execution={"engine": "spark", "workingBytes": 64 * 1024 * 1024, "stagingBytes": 1024 * 1024 * 1024, "timeoutMs": 300_000}) from its captured Python handler. The ordinary function host must supply spark-dataframes and its finite native allocation on BIJECTION_SHARED_WORKSPACE. Inputs open through their exact admitted Dataset references. Global fragments use the existing checked Spark SQL compiler and required semantic kernels; complete candidates reach the Dataset writer as DataFrames. Only the single affected-row count is collected on the coordinator. No input or complete output is collected there. Small source-free fragments may use bounded portable evaluation. The executor’s own admission and resource limits still apply; the portable interpreter’s one-million-row and 64 MiB complete-population limits apply to Python execution. The Spark profile allows 1–256 MiB of coordinator workspace, 1 MiB–1 TiB of private staging and 1–3,600,000 ms of evaluation time. A mismatched placement or missing native capability refuses before input consumption. Existing Dataset and Compute owners retain custody, staged output acceptance and cleanup. The installed Python callable can select an explicit analytical per-fragment work profile with bijection_sql.materialize(ctx, args, relation_limits={ "profile": "analytical", "maxRowSteps": 30_000_000, "maxProcessedBytes": 256 * 1024 * 1024}). This profile permits up to one billion steps and 2,147,483,647 cumulative logical bytes, within the portable guest’s counter representation. The default remains 100,000 steps and 16 MiB. A profile without the analytical selector retains the native ceilings of one million steps and 64 MiB. The evaluator copies the allowance before yielding and passes it to delegated row-local evaluation as well as complete-population and scalar fragments. Complete input/output limits and the external executor’s own admission still apply. TypeScript execution accepts the same profile as executeProgram(program, inputs, clocks, { relationLimits }). compileRelation({relation, inputs, schemas, limits: relationLimits}) uses the same analytical profile. Native view work policies keep their separate limits. The ceilings describe admission and refusing behavior; they do not establish deployment capacity or performance. Install the interactive owner in an application module, for example warehouse.ts:
totals is a Dataset bound to the same installed Dataset resource as the other warehouse targets. The coordinator supplies every warehouse target name in inputs, including the destination; unused inputs are null. The resource owner binds the output’s exact row contract through the ordinary function result store. Candidate contracts include the original owner, session, statement, target and int64 admissionTimeMicros. The coordinator captures this instant once in its admitting mutation and saves it with the original arguments before dispatch. Current-time expressions and unseeded randomness use this instant; worker execution and retries do not read another clock. The internal authorize query accepts the verified actor, operation (read, write or control), table name, Dataset, contract and exact version (or null when preparation has no historical basis), and returns a boolean. The coordinator derives the actor from authentication for client operations and from the actual retained Compute execution for candidate I/O. An argument alone never proves a background caller. The executor gate and resource policy prove the native Compute caller, match its request digest to the pending original intention, check exact captured inputs, recheck current source permissions and bound its deadline to the session lease. Resource admission runs in the current ordinary owner transaction and refuses use as a private cross-component read rule. The optional resource policy declares root tables and additional child queries used by local authorization and cleanup. The separate release query retains cleanup authority after session end. Rollback records session end and cancellation of the original pending run in the same mutation, including after removal of the current candidate function. Accepted outputs resolve through functionResources.result before staging. Retries use the original Compute installation, admission scope, arguments and implementation; they do not recompile pending work through current code. Start the gateway with:
Prepared statements pin both their program and their native compiler/input digest. Write templates also seal destination resource, selection, writable status, defaults and lowered statement structure. Changes refuse a new binding before dispatch; an already admitted request still resumes its saved meaning. The gateway transports analytical parameters as bounded tagged JSON strings at the application-function boundary. The application decodes them after checking wire bytes, nodes and depth, then binds their declared semantic domains. Large arrays therefore do not pass through inherited native argument arrays. Exact integer, binary and IEEE tags survive that transport. The gateway also checks actual Arrow IPC bytes against its configured batch limit; prepared statement custody retains those values through recovery within the analytical byte budget. The low-level sqlCompile, sqlBind and installed warehouse handlers accept this string in parameters; direct SQLWarehouse authoring methods continue to accept typed values. decodeSQLParameterValues exposes the same bounded decoder to application adapters. Warehouse mode uses its Dataset catalog for metadata, prepared statements, queries, writes and transactions. Each request keeps the client’s business authority. Query compilation describes declared Dataset inputs through the same native binder as execution. Transaction reads retain the session’s exact private versions using ordinary Dataset read contexts; autocommit reads retain a finite read context over their captured selection set. Opening a context never replaces current Dataset disclosure policy. SQL compiler contexts identify their actual invoking owner: a function-store I/O receipt or a retained-resource holder. Each input reference contains its exact Dataset version, declared columns, carrier encoding and engine source. Native custody assessments remain with the invoking owner and are rechecked around operations. Delta sources retain their exact nonnegative table version, including version zero; provider-owned fixed tables retain their original selected version. Runtime Spark stages create private session views without replacing existing views, and refuse a changed captured frame. A prepared write batch admits all parameter rows as one immutable graph and one session statement before dispatch. Rows see the preceding private result; all intermediate candidates are validated, including candidates overwritten by a later replacement. Invalid parameters refuse the batch before Compute starts. The complete batch is limited to 64 rows and 64 KiB of scalar arguments, and its compiled graph must fit 48 KiB. Complete candidate input/output bounds still apply. CREATE cannot repeat in one parameter batch. The owner resumes the original admission and Compute request key before any replacement compilation or dispatch. Explicit transactions stage complete candidates privately; autocommit writes publish through the same atomic selection owner. HTTP polling and reconnects with identical request identities recover that admission. UPDATE and MERGE preserve the explicit unknown affected count. Rollback and expiry prevent late candidates from being selected; they do not assert that external compute has stopped. Native source checks and portable unit histories do not qualify deployed authorization, restart or client behavior.

Query the exposed catalog from the CLI

The application API resolves SQL against the signed-in caller’s existing catalog. Relations unavailable to the caller have the same response as unknown names. Discoverability grants no row, predicate, aggregate or provider disclosure permission. Logical relationship populations appear as "links.<collection>.<relationship>" with typed source_id and target_id columns. They follow the exposed navigation through existing references and link assignments, including many-valued inverses and many-to-many paths. Only live endpoints contribute. Endpoint and link reads remain subject to native authorization. Hidden storage names are never catalog names or part of the returned public program. Catalog entries include the existing public resource identities and typed columns, with optionality separate from data nullability. Compilation returns the same column metadata and the compiler revision. The native input-contract digest pins definition meaning and collection incarnations; the catalog does not invent a separate schema revision. It also pins the semantic profile and native implementation, so a backend upgrade requires preparing the SQL again.
--params uses the CLI’s exact native JSON value codec. The CLI captures the file and values before authentication, compiles them against current catalog metadata, then journals the resulting program and native input-contract digest before asking. Retry uses those retained bytes. It does not re-read the SQL file or recompile against a changed schema. Explain starts no execution. Select an installed finite Spark computation when the query reads retained Iceberg datasets:
The query selects a nonempty subset of the installation’s dataset inputs with matching complete row contracts and bound parameters. Every transferred field passes current read and disclosure checks. Explain reports the pipeline, physical plan digest, binding revisions and required runtime profiles. Native result custody keeps the original request identity, validates complete typed rows and owns paging and recovery. Cancellation reaches the existing computation owner. Sealing the native result revokes intermediate result access and retains its bounded cleanup obligation. Streaming results support up to 10 million rows and 10 GiB under the admitted deployment and caller quotas, within the installed computation’s deadline. Source-free expressions and authorized catalog metadata use the native evaluator. SDK result bounds select the same path with execution: { kind: "binding", binding: "analytics" }. Use execution: { kind: "automatic", bindings: ["small", "analytics"] } to inspect those installed bindings in order. The existing planner checks the resolved operation requirements, exact runtime profiles and transfer policy. Explain reports the selected binding and plan; unknown estimates remain unknown. Admission retains that selection and the original input versions. A lost reply reopens the same evaluation before any selection is attempted. Cancellation, failure or an expired intermediate never causes execution on another binding. Choose { kind: "native" } for the native route or { kind: "source", binding: sourceBinding } for an explicit source executor. The feature catalog separates implementation contracts, current installation requirements and verification evidence. Runtime metadata does not attest a test run. Compilation and SQL explain also expose operation_requirements: resolved operand/result domains, frame kinds and bounds, ordering domains, and null treatment or other operation parameters. Admission still decides whether the current caller, installed executor and selected inputs can execute the query. LIMIT selects the logical result; --max-rows and --max-bytes bound the complete accepted result. Intermediate work retains its separate bounds. A limited population requires explicit ordering or an already ordered input. Delivery uses existing result pages, cancellation and release, with current access checked on every page. Writes, DDL and arbitrary functions are refused. Statements are bounded to 1 MiB and 262,144 tokens. Input addresses, expansion, nesting and output columns have additional shared compiler bounds, reported by the catalog. Analyzed plans admit at most 100,000 relation nodes and 100,000 scalar nodes. Normalized plans and total type metadata each have a 16 MiB bound. These plan bytes also bound grouping-set expansion and parameter literals; they are independent of the authored SQL text limit. Literal parameter data uses the analytical value’s 16 MiB bound, with the existing structure, nesting and collection bounds still enforced. These structural limits work alongside the request’s execution work and byte budgets and the selected executor’s resource limits. Native transaction execution keeps its separate local row and byte limits. SDK and Go host checks cover a complete analytical graph above the former 65,535-address boundary; they do not establish deployed capacity or performance. Closed array parameters use one typed constant per reference, with their element domain inferred by the shared type checker. Arrays of structs become typed input tables through UNNEST(:items); every occurrence contributes to the ordinary relational operators. DECLARE @items ARRAY<...> fixes an explicit element contract, including for empty or null-only arrays. Values remain data throughout binding and do not become SQL expressions. A single-array UNNEST expands its captured value once. Structured SQL constants retain their declared physical casts without repeatedly copying nested constructor text in external SQL. A 20,000-row exact-integer fixture runs through both the shared evaluator and the retained Spark runtime; this is fixture evidence, not a capacity claim.

Standard SQL clients with Flight SQL

Run an optional foreground gateway for ADBC, JDBC or other Flight SQL clients:
Use bijection sql serve --execution '{"kind":"binding","binding":"analytics"}' to select that same installed computation for SQL clients. The gateway captures its configured result and Arrow batch ceilings before issuing tickets, bounded by the selected execution path’s complete-result limits. Deployment quotas and smaller configured ceilings remain binding. The installation and input-contract requirements above also apply to client queries. Every RPC supplies authorization: Bearer <business-token>, or the bearer the gateway returned from a service credential handshake (see BI tools). The application validates the caller on every RPC; the gateway never executes SQL as its process account. Tables and schemas come from that caller’s catalog. The read-only endpoint supports catalog/schema/table metadata, SQL information, schema discovery, statements, prepared statements, Arrow result streams, cancellation, and prepared-statement close. The default mode refuses writes and transactions. --warehouse <module> enables them through an installed SQLWarehouse; Substrait plans refuse. The catalog is bijection. Business relations use schema public; bounded schemata, tables, columns, relationships and interface_implementations views use information_schema. Type discovery includes exact scalars, nanosecond TIME, arrays, structs and maps. Individual clients may still restrict nested values. Native identity keys are exposed through primary-key metadata only when their _id column is present and non-nullable. Asserted keys and logical relationships remain ontology descriptions rather than SQL foreign-key constraints. Included Arrow table schemas retain bijection.ontology_reference and bijection.logical_key metadata. Exact table/schema patterns narrow application discovery before materialization; wildcard patterns retain their full matching population. Logical relation names stay exact; Quote a logical name containing dots as one SQL identifier. Prepared parameters accept named :parameters or positional ? placeholders. The shared parser assigns positional fields p000001, p000002, and so on; named fields appear in sorted parameter-schema order. Mixing styles refuses. Bind one or more Arrow rows whose column order and Arrow types match the negotiated parameter schema. A batch admits at most the gateway’s configured caller request ceiling (32 rows by default), with at most 512 KiB of parameter IPC data or the smaller Arrow batch allowance. Every row is decoded and bound before any request starts. The gateway returns one ordered endpoint per parameter row; every endpoint has its own complete-result bounds, retention and current authorization. A later admission failure closes earlier accepted requests; an uncertain submission retains the original identity reported in the error. An all-null scalar binds to its original nullable domain regardless of the driver’s placeholder carrier. Field labels do not change positional binding. A Decimal128 carrier may use smaller precision at the negotiated scale; each value is checked against the original domain. Binary and nested list, struct and map bindings preserve native integer and byte tags through saved prepared statements and reconnect. Nested nullability, map keys and the shared 64 KiB parameter bound are checked during decoding. Preparation returns both parameter and result schemas before values are bound. Domains are inferred from declared columns and unambiguous expression constraints. Declare an otherwise ambiguous parameter with a cast, for example CAST(? AS BIGINT) or CAST(:price AS DECIMAL(18,2)). Parameters accept null; unconstrained parameters and conflicting domains refuse preparation. Binding validates the original symbolic template against the current catalog and input digest; a changed schema, semantic implementation or authorization refuses stale work. Values bind into that template without recompiling SQL from their values. Spatial parameters use UTF-8 JSON carriers whose field metadata retains the original geometry or position validator, including nested fields. Binding checks coordinate space, geometry kinds and position accuracy through the same native value admission as SDK calls. With sql serve --warehouse, Flight SQL bulk ingestion uses these explicit table options. Every target must already have an installed logical row contract; creation selects its first version rather than inventing a schema from a header. Use the logical table name and Arrow field names as target columns. Column order is preserved and append omissions use the installed defaults. The target’s prepared INSERT determines every logical domain, including nested list, struct, map and binary values. All incoming batches are decoded before one atomic warehouse submission. A later invalid row or stream error submits no write. Transaction handles retain private writes until commit. One UUIDv7 request identity preserves retries of the complete population. Prepared writes also retain the installed target column order, so reordered incoming headers keep their original parameter domains. Bulk ingestion accepts 1..64 columns within the gateway row and 512 KiB parameter ceilings. Each complete population consumes one caller request slot; its rows do not consume separate statement requests. Schema metadata is also bounded. An empty append population submits one typed, zero-row INSERT through the installed owner and returns zero affected rows. This still checks the selected target, current authority and transaction state, and uses one UUIDv7 request identity for retry. The IPC stream must carry its schema, including a zero-row batch for clients that emit the command descriptor only with their first record write. Bulk creation and replacement carry the complete population in one parameterized CREATE TABLE or CREATE OR REPLACE TABLE candidate. It retains all rows across incoming record batches and requires the complete installed target row contract. A captured read of the target enforces fail-if-missing even for an empty existing table. Empty replacement publishes an empty typed candidate and returns zero. The analytical ingestion profile carries its rows in one typed array parameter, within the gateway row and byte ceilings and the installed owner’s argument, plan and work allowances. Native domains without a SQL type spelling use at most 128 scalar parameter cells. Replacement uses private transaction state and the same atomic publication and retry owner. Empty creation uses the prepared owner’s column types and the same zero-row population query; nested analytical arrays, structs and maps keep typed CREATE admission. Civil timestamps retain their clock without acquiring a timezone. A client header cannot replace these domains. Temporary tables, separate catalog/schema selectors and provider options are refused. The ordinary ontology gateway remains read-only. Bulk append and transaction capability metadata are enabled only by the warehouse gateway. Int64, decimal128, microsecond timestamps, binary values, lists and closed structs preserve their declared domains. SQL TIME results use nanosecond TIME64 within one day; excess precision and out-of-range values refuse. Binary parameters retain their byte domain through binding, recovery and dispatch. Optional fields use a presence struct (present, value) with bijection.optional=presence_struct/v1 metadata, keeping absence distinct from null. Unions whose members share the same exact Arrow carrier preserve their full validator and logical metadata; object IDs from several collections can share a string column. Unions requiring different carriers or nested field metadata, and open values, refuse. FlightInfo application metadata identifies the native request and ticket expiry; batch application metadata keeps its result/output/content identity and available row metadata, sliced to the exact batch. Each batch also carries resume_ticket, rows_received and complete. resume_ticket is a base64-encoded Flight ticket for the first unread row after that batch. Defaults admit 256 handles globally, 32 retained requests and 128 prepared statements per caller, a 15-minute lifetime, and a 16 MiB complete result allowance. Native execution admits 65,536 rows; the gateway’s default streaming-compute row allowance is 100,000. Flags on sql serve can lower result limits and configure handle, caller, batch and lifetime limits within the native envelope. --max-transfer-bytes bounds cumulative Flight payload bytes across reconnects and gateway restarts (64 MiB by default, at most 1 TiB). The journal reserves actual Arrow schema, batch metadata, padded buffers and application metadata before release. Uncertain delivery consumes its reservation. Large pages split into batches; an individual row or schema that cannot fit the batch allowance is refused. LIMIT bounds output and does not bound upstream work. The gateway reads at most 256 native rows per page. Requests remain recoverable after stream completion until explicit CancelFlightInfo, CLI closure or finite expiry; network delivery and EOF do not acknowledge durable consumption. The Python SQLAlchemy adapter closes after a full fetch, before a new execution, and when a cursor or connection closes. Generic drivers that expose no explicit result close rely on finite expiry. Retained table results use bounded protected chunks and a sealed index tree. Reading a page fetches its index branch and data chunk; it does not load the complete result. The result becomes visible only after every output is complete, and storage accounting includes all chunks. Failed cleanup reports the native request identity; backend retention remains authoritative after process loss. Analytical SQL workers consume warehouse cursors incrementally into an attempt-owned result spool. Cursor transports fetch at most 1,024 rows per batch; Spark uses its partition iterator with prefetch disabled. The worker checks the admitted result limits while consuming rows, closes the iterator and engine session on refusal, and sends parts only after the complete spool exists. Spark’s driver memory still depends on the largest result partition. Databricks, DuckDB and BigQuery analytical workers read structured Arrow through the shared C1 carrier schema. They validate field names and physical types, preserve nested missing/null markers, and use safe conversions for exact decimals and microsecond timestamps. Retained and expanded batches are bounded to 16 MiB; dictionary expansion and collection lengths are checked before materializing Python rows. The result spool then applies the cumulative row and encoded-byte limits before any parts are sent. This path does not widen the separate buffered SQL-source reader’s scalar projection contract. Streaming compute supports explicit result bounds of up to 10 million rows and 10 GiB. Buffered native and SQL-source execution retain their 65,536-row and 16 MiB envelope. Deployment settings QUESTION_MAX_RESULT_ROWS and QUESTION_MAX_RESULT_BYTES can narrow the supported bounds. Before dispatch, the existing retained-result capacity owner reserves three times the logical byte allowance, 64 bytes per admitted row, and 192 MiB for bounded staging. Deployment, component, and caller reservations are enforced atomically; their default storage ceilings are 8, 4, and 2 GiB. Operators can configure QUESTION_BYTES_PER_DEPLOYMENT, QUESTION_BYTES_PER_COMPONENT, and QUESTION_BYTES_PER_CALLER up to 1 TiB each. Request flags grant no additional quota. Accepted executions retain their reservation after a policy reduction, so they can finish or release storage. Inspect the effective route envelope and exact reservation before starting a gateway:
This command runs without signing in, opening a listener or reserving storage. Its report derives the reservation formula and default ceilings from the native capacity owner, includes the reservation at the gateway’s caller request ceiling, and identifies transfer allowances smaller than the logical result maximum. A 10-million-row, 10-GiB result reserves 33,053,581,312 bytes and exceeds all three default storage ceilings. installed_policy and available_capacity are not_observed: local arithmetic cannot establish installed policy or free quota. Deployment limits, current usage and provider guarantees are checked at admission; preflight never raises them. Startup also prints the effective row/byte envelope and reservation per result. Large Spark results first measure bounded partition counts and digests. A second pass must reproduce them before delivery. Each bounded capsule gets its own shuffle partition, avoiding Spark’s whole-partition driver buffer. Capsules hold at most 4 MiB and 4,096 rows, with at most 65,536 physical partitions; a larger partition inventory refuses explicitly. The owned metadata broadcast and iterator are closed on completion or interruption. The attempt uses bounded memory and a disk spool within the result byte allowance; its shuffle belongs to the customer’s existing compute owner. Preview retirement revokes access immediately and deletes eight parts per transaction through the original preview’s cleanup timer. Choose a distinct canonical lowercase UUIDv7 for each SQL parameter row before its first execution, and send it in x-bijection-request-id on execution and parameter-binding RPCs. Repeat the same IDs, SQL bytes and exact parameter values to recover a lost submission reply. The gateway reuses the original ticket, Arrow schema and expiry before compiling or binding again. A pending request continues its retained allocation, capture and submission. Caller and branch scope, execution selection, effective lease duration, and row, byte, request and transfer limits are bound to that identity. Changed, closed, spent or expired submissions refuse. Missing transport custody after allocation also refuses, so recovery cannot reset transfer charges. Every RPC authenticates the current caller. Without this header, each execution creates a fresh request. The gateway’s local journal permits the same identity to be followed with bijection run follow or closed with bijection run retire <run>. It retains the negotiated Arrow schema and a principal/deployment/branch scoped ticket before dispatch. A restarted gateway can consume that same ticket without preparing or submitting SQL. A finite reader lease and generation fence prevent concurrent consumption. Each batch release rechecks the generation after source IO and stays within its known lease even under client backpressure. Disconnects release the reader. Persist each consumed batch’s data and resume_ticket together; passing that ticket to DoGet resumes at the exact following row, including within the original result page. The gateway authenticates the current caller, verifies the checkpoint signature against scoped journal custody, and rechecks the original result’s current read authority. It never advances a durable consumer position merely because a batch was sent. Reusing the initial ticket deliberately starts at page zero; reusing an earlier checkpoint repeats the rows after that checkpoint. A final checkpoint returns an empty typed stream while the request remains retained. Neither continuation nor reconnect extends the original expiry or cumulative transfer allowance. A process crash may require waiting for the 30-second reader lease. Python consumers can admit with cursor.adbc_execute_partitions(sql), take original_info = cursor.detach_result(), and read ResumableResult(connection, original_info) from bijection.sql_flight. Detaching transfers cleanup ownership to the consumer. Save each returned FlightBatch’s data and resume_ticket in the same application commit. Recover with ResumableResult(connection, original_info, resume_ticket=saved_ticket) and call result.release() after committing the final batch. Closing that reader stops observation; explicit release closes the retained request. Prepared handles and exact parameter bindings also survive a restart in the same private journal, scoped to the deployment, branch generation and principal. Recovery preserves negotiated schemas and validates the original symbolic plan and input digest when binding or executing; changed inputs require a fresh preparation. Prepared custody is bounded, expires with the original handle and is removed on close. Orderly shutdown preserves accepted requests and prepared templates for recovery. The journal retains authored SQL and source maps for delayed runtime diagnostics. A confirmed request close removes its private journal entry; failed cleanup retains the close obligation for recovery.

Connect BI tools

BI servers and desktop tools (Tableau, Power BI, Excel, DBeaver and others) hold one user and password per connection, while business tokens are short lived. Give such a tool a service credential instead:
The first command writes reports.bi.servicekey; store its one line as the tool’s password and delete the file. The service reads only what its grants and role ceiling allow, like any other caller. Run the gateway where the tool can reach it, with TLS, under a process supervisor (systemd or a container restart policy):
Connect with an Arrow Flight SQL driver, using the service name as the user and the credential’s one line as the password: The driver sends the user and password once, on the Flight handshake. The gateway checks that the credential belongs to this deployment and that the deployment accepts a token it signs, then returns a connection bearer that the driver presents on every later RPC. For each RPC the gateway signs a fresh one-minute token, so the deployment checks the credential, its revocation and the service’s grants every time; bijection auth credential revoke takes effect on the next RPC. The private key stays in the gateway’s memory and is never written. A connection idle for 12 hours must reconnect, and a gateway holds at most 256 open service connections. Use live queries that aggregate in SQL. Native results default to at most 65,536 rows and 16 MiB; streaming compute defaults to 100,000 rows and 16 MiB. Larger streaming results require explicit result and transfer allowances and available deployment storage quotas, so large extracts refuse rather than arrive incomplete. Named gateway fixtures cover ADBC 1.12.0 with Arrow 26.0.0, Arrow JDBC 19.0.0, SQLAlchemy 2.0.44 with Arrow 26.0.0, and Apache Superset 6.0.0 with SQLAlchemy 1.4.54 and Arrow 18.1.0. They exercise real drivers against retained-request fixtures. This evidence is separate from deployed provider qualification. DBeaver Community 25.2.5 has a supplied configuration using that JDBC driver; its desktop workflows and ODBC applications remain unobserved. For Superset, install the Bijection Python SDK with its superset extra in the Superset environment. The SDK registers bijection+flightsql and the Bijection engine specification through the applications’ standard SQLAlchemy driver and engine-spec entry points. Enter this SQLAlchemy URL:
Put the service credential in Encrypted extra, outside the URL:
The engine supports authorized metadata, SQL refreshes, UTC time grains and cancellation of the original retained result from another connection. The default gateway is read-only. An installed SQLWarehouse enables Dataset writes and transactions with --warehouse <module>.

Compile through the application API

Compiling and explaining start nothing. bijection run sql starts a run from the exact program and journals its request and inputs before dispatch; an action uses runs(ctx).start. For reusable typed statements, prepare before supplying values. Preparation and binding describe work and start nothing.
Bind a filtered object population as a table with tableProviders. Capture the ordinary object-query request from the caller-visible SQL catalog:
The provider is public composition data, not an executable plan or permission. Preparation and binding both check its current collection contracts and use the same native population, navigation and source-key admission as object queries. Supply the same providers when binding a prepared statement. A changed provider, schema, or current disclosure refuses the old template. Providers may override referenced business inputs; unused names and information_schema overrides refuse. sqlCompile and sqlAnalyze accept the same providers. With a SQL file, the CLI accepts the same provider map through --table-providers @providers.json. It captures that file before authentication or catalog discovery, together with the SQL source and scalar parameters. A table declaration names the logical collection in backticks:
This declares a table variable whose type is the logical collection customers. Without a supplied provider it selects that caller-visible collection; an explicit population must retain the declared collection identity. Scalar declarations such as DECLARE @minimum BIGINT supply a parameter domain without sampling values. Scalar declarations also accept BINARY, ARRAY<DECIMAL(18,2)>, STRUCT<label: STRING, values: MAP<STRING, BIGINT>> and nested combinations. Container elements and fields are nullable; struct fields must be present. Empty arrays/maps and null containers keep their declared domains. Composite and binary declarations validate supplied values without element-wise casts. Exact decimal and timestamp constraints are checked before the request starts. Scalar declarations can also initialize query-local values and share one comma-separated statement:
An initialized value is part of the SQL source and needs no external parameter. Its expression can refer to earlier declared variables, but cannot read rows, use subqueries, aggregates or windows. Each use retains its declared conversion and ordinary expression bounds. Uninitialized declarations still require caller parameters. Arrays and structs use the same positional casts as CAST. SQL array constructors choose a common numeric element type across the entire array, including through nested lists and structs and across constructor chunks, before applying those casts. Ordinary single-quoted strings interpret Spark control, Unicode and octal escapes. Prefix a string with r or R to retain its backslashes, for example r'\d+' for a regex pattern. Both doubled quotes and backslash-escaped quotes work in ordinary strings. Double quotes and backticks delimit identifiers; raw strings may use either single or double quotes. SQL rewrites preserve the decoded string value. Installed interfaces with visible implementations appear as "interfaces.InterfaceName" relations. Their native mappings select only the implementations visible to the current caller and expose the common properties. Select a closed interface population or bounded native-series samples with the same tableProviders argument:
An interface provider admits 1..32 distinct currently exposed implementations. Its common properties support ordinary business-key joins; it grants no object locator or write authority. Binding rechecks the interface mapping and every implementation’s resource identity. A series provider requires 1..64 explicit series IDs, a half-open [from,to) interval with bounds in epoch microseconds, and explicit payload fields. It exposes only series, key, time and selected values, using the native correction, erasure, retention and publication owners. The native reader requires complete coverage before SQL filters or LIMIT, and checks any supplied freshness and settlement requirements. Capture refuses beyond 4096 rows, 4 MiB, 64 pages or its ten-second work deadline. A changed series contract refuses the prepared provider. Larger populations need an eligible external source or retained dataset. The SQL catalog includes native key descriptors and the public composition contracts used by sqlTableProvider. Exact sqlCatalog({schema, table}) filters avoid transferring unrelated catalog entries. information_schema.tables includes descriptions, columns includes numeric precision/radix/scale, and relationships describes logical endpoint collections, link relations and cardinalities. These relationship descriptions do not declare SQL foreign-key constraints. interface_implementations lists each interface’s SQL table and its caller-visible implementation tables. Simple metadata equality predicates are applied before constructing bounded constant relations, so a selected table remains discoverable in a large catalog. Unfiltered metadata queries retain their stated work and program bounds. For a transactional native result, use the sqlRead query. It returns a bounded result of at most 10,000 rows and 4 MiB, including columns and SQL source locations. Queries without a limit use the 10,000-row allowance; OFFSET + LIMIT must fit within it. Smaller requested max_rows and byte ceilings require a complete result within those bounds and refuse when exceeded. Scalar parameters and table providers use the same admission as compilation. External populations require an explicit run or sqlAnalyze.
Branch execution inherits the exact generation selected by the client. Snapshot execution uses the existing immutable query reader and preserves its basis; it does not start a retained action at a historical database. Prepared gateway handles and retained tickets remain scoped to the caller, deployment, branch and generation through reconnect. Bijection uses these existing branch and snapshot semantics for hypothetical and historical SQL reads. Compilation returns the native output validator and input-contract digest. The same native admission owner rechecks that digest when selecting inputs. Schema changes refuse the prepared work. Execution and release continue through the existing result owner, including external outcome uncertainty and revocation. Credentials and provider relation names remain installation details. Compilation returns a source_map keyed by JSON pointers into program_json. It covers relation operators, scalar operands, input references, parameters and generated helpers, including copied CTEs and outer-join lowering. Spans use one-based lines and columns and identify the source SQL declaration. Source maps contain positions, not SQL text or literal payloads. They travel beside the program and do not change semantic identity when SQL formatting changes. Native diagnostics preserve their codes and carry SQL spans as additional context. A span is selected by the failing expression’s path: the exact mapped expression or its nearest mapped ancestor. A failure that names no expression carries no span rather than a guessed one; message text never supplies a location. In-process SDK evaluation preserves these spans on ScalarFailure.source, including failures during scalar evaluation and relational operations such as window ordering. When SQL frontends are composed, source.declaration identifies the authored SQL relation within the enclosing expression. The CLI keeps the source map and SQL beside the compiled program, outside the request and its identity. Compilation failures and later failures naming an expression print SQL at line L, column C with the SQL line. The private request journal retains this metadata after acceptance, when dispatch inputs are retired, so retry and follow can render diagnostics after the SQL file changes. Closing the request removes its local diagnostics with the request record.

Native views and ordinary queries

A view is a closed relation in the schema. Parameters belong to queries. The view’s serving option chooses physical maintenance; application readers keep the same API.
bijection/schema.ts
Install the application’s normal read and write policies on these declarations. Read the view through an ordinary query:
bijection/billing.ts
useQuery(api.billing.balances, {}) subscribes to that transactionally evaluated result. A mutation reading the same view sees its pending writes. A defineRelation({ args, relation }) declares a reusable parameterized formula; an ordinary query or mutation reads it with ctx.db.read(definition, args). Retained analyses have their own internal declaration.

Admit durable ordinary functions

Declare argument and return validators on an ordinary query or action. A Compute instance from @bijectionhq/compute owns its durable admission and accepted result. compute.api(functionReference, { access }) exports ordinary start and get functions. An inspect guard exposes detailed inspection. Optional resume, cancel and retire guards expose those controls. The application owns the current business authorization for every operation. Admission captures the caller, deployment, calling contract and implementation, and commits with its durable continuation atomically. Recovery reads the original receipt; an explicitly retried intention cannot select a newer implementation. An already admitted request keeps its original code across deployment. See React computation results for the checked server and React examples. For a relational q.program with named outputs, use the installed QueryWork owner from @bijectionhq/compute; ordinary application functions call its start, inspect, output, cancel and close methods. These component owners coordinate existing native or external execution.

Fleet sample summaries

For a fleet beyond the bounded native series API, declare the external observations once with defineVirtualSeries and reduce them for each population member before ordering the result:
population is a typed relation source and observations the dataset or connector source that owns the readings. Put ranking in q.program({ rows: ranking }) and admit it through the installed QueryWork owner. Ordinary queries observe its request and read the accepted output with results.paginate(ctx, …). The limit applies after the complete population’s reductions and ordering; it does not limit which tanks participate. Configure customer compute and an input basis supported by that executor. Each observation resolves to its greatest correction order before the interval applies, then each instant keeps its greatest order. A highest-order conflict refuses; a null correction removes an earlier value. A tank with no observations retains count zero and a null maximum. A multi-field key is compared field by field, q.and(q.eq(tank.site, sample.subject.site), …). Clocks are int64 microseconds; values are int64 or float64. m.average returns exact decimal text with explicit half-even rounding. Units are part of the declaration. Generic decimal ordering is not admitted; rank an integer diagnostic or use a customer program for a complete rational metric. Counts do not establish continuous temporal coverage. Validate intervals in public endpoints. The repository’s examples/fleet-assessment app adds complete population accounting, two-window diagnostics, publication, explicit selection, coherent serving, schedules and a React UI on the existing computation owners.

React hooks

useComputation(api.forecastRequests, { interactionKey: "forecast/account" }) restores the original invocation without dispatching work. Call start(args) in an explicit event handler. recover() observes the original admission; resume() explicitly continues the saved intention with its original identity, input and implementation. Observe progress through the hook or with useQuery(api.forecastRequests.get, { run }). The native request journal retains no result cache. Use ordinary useQuery and usePaginatedQuery for application queries that read exact result references. Admission, execution, accepted result availability and current permission remain separate. See React computation results.

Computation access

Compute.api requires start and read guards. Optional cancel and retire guards expose their corresponding mutations. The installed component’s access policy independently checks the actual caller. Every output read checks current source authority and custody; a locator grants no permission.

Serve a maintained result

maintain from @bijectionhq/pipelines/maintained selects and retains an accepted pipeline result. Ordinary application queries call its resolve and inspect methods. A separately authorized mutation calls refresh; observing status or reading rows starts no work. Pages share one selected immutable result and refuse when its selection is replaced or withheld. See keeping a result current.

Remote interface computations

A property-only object interface can combine keyed virtual tables reachable through one executor. Map direct fields with .implements(...), register the interface in the schema, and select its concrete population explicitly:
The result contains properties rather than native operation targets. Interfaces requiring operations or relationships remain native-only. A selected implementation must belong to the installed contract; missing inputs refuse instead of disappearing. Native data requires an explicit published representation readable by the executor. Remote inputs bound to different executors refuse. Equal visible values preserve occurrence multiplicity through UNION ALL; sorting and limiting apply to the complete population. Declared input keys are checked before result limits, including keys in rows outside the result. These key guards require authorization for the complete validation population and disclosure of the key fields; a narrower scoped grant refuses rather than authorizing an implicit full scan. The optional recovery declaration also accepts submission: "identity_before_send". It requires a saved execution identity and a provider mechanism for recovering that exact submission. A correlation ID alone is insufficient. results: "original_statement" requires resumable original results. These are mechanisms, not promises that provider history never expires. A lost provider-generated identity can leave the run terminally unknown; observing or reconnecting never resubmits it. An explicit rerun uses a new request key. bijection computation describe inspects generated declarations offline: selected interface populations, scoped input requirements and recovery requirements. Executor preflight reports the same recovery eligibility as execution, the enforcement owner of each limit and which evidence has not run. Successful provider planning is labelled separately from workload qualification. requestCatalog.inspect(ctx, request) reads the original QueryWork request through its installed owner. Its observations and diagnostics preserve: the admitted executor decision, selected implementations, exact retained versions or observed binding revisions, hashed statement identities, the committed checkpoint digest and partial cleanup inventory. Provider handles and continuation capabilities remain private. An unselected implementation added later does not change an explicit population’s input digest. Completion remains separate from result availability. Available source-path, decoding, result-size and provider measurements are retained with execution custody. Missing stage timings are null, never zero; the report does not invent provider queue time or a transfer duration. Runtime artifact, schema, input and executor revisions identify the evidence; absent provider versions and cold/warm conditions remain unavailable. Provider planning and metadata inspection establish no workload latency or failure-recovery result. See the repository’s examples/remote-interfaces application for complete declarations, inspection and lifecycle endpoints.

Scoped input requirements

Pass basisRequirements in the options to requestCatalog.start(ctx, program, options). Each named input has independent obligations. Relational input names use escaped JSON pointers such as /tables/orders.
The default is an empty set of obligations. fixed requires an owner-attested exact version. Snapshot groups require a common attested source snapshot across their complete ancestry. Freshness evaluates either the selected input or every source in its ancestry; missing evidence stays unproven. Independent executor observations do not establish a coordinated snapshot. On Snowflake and BigQuery a snapshot group of observed tables reads every table at one pinned provider instant (AT(TIMESTAMP => …), FOR SYSTEM_TIME AS OF), which establishes the snapshot. On Databricks, up to sixteen observed inputs can name Unity Catalog managed Delta tables in one metastore. Inputs naming the same table reuse its pin. For multiple distinct tables, the provider clock and catalog mappings are sampled between two matching collections of table identities and latest commits. Each table is read VERSION AS OF its selected commit. The group requires a declared warehouse statement timeout that fits within every member’s retention window. External Delta and Hive metastore tables refuse pinned reads. On PostgreSQL the source gateway holds one read-only Repeatable Read transaction for both metadata and results. It locks up to sixteen permanent local heap tables before the first read and checks the locks again when the stream completes. The lease permits one streamed result and expires after sixty seconds idle or fifteen minutes total. On Trino, a group of observed Iceberg tables can use one exact commit from the same configured Nessie catalog. Other executor profiles and groups with retained inputs are refused. The run’s diagnostics report each pin beside the bound statement’s digest. An expired pin or a changed catalog identity returns input_basis_unavailable. To require provider byte budgets, declare them on the source execution:
The provider-enforced cap must be no greater than that budget. Trino receives the per-statement physical scan limit in its session properties, including for metadata and series archive reads. Large limits are rounded down to the largest integer byte count the session setting can represent exactly. The per-statement limit alone does not establish an aggregate budget. The run limit reserves each statement’s full installed cap before dispatch, including SQL metadata and version-pin probes. It shares one durable native allowance across retries; recovering the original workload does not reserve it again. Unknown provider billing retains the full reservation. Exhausted capacity refuses another dispatch with resource_limit_exceeded; cancellation and timeouts do not refund unknown work. Preflight checks its metadata work and capacity for the result using a transient allowance. It reserves no run work. BigQuery supports both limits through its configured billed-byte cap. Executors without a provider-enforced billed-byte cap refuse the per-run requirement with placement_unsupported. A result carries the canonical evidence graph. Its live basis_assessment reports each predicate, its state, reason and witness at the stated time. Freshness is checked at acceptance and fresh delivery; an explicit historical read preserves the original graph and reports its current assessment. Input retention requires current owner holds through the requested deadline. Result retention remains the lifetime of the accepted result.

Choose the result you need

A complete result contains the complete logical result, or the run refuses. Bounds never silently truncate it. An explicit relation limit is part of its meaning. A small page does not make an unbounded population scan cheap. SDK table readers compare complete page traversals with the sealed row count and reject repeated cursors. A collection bound below that count refuses before a read. Whole-result facets use the same output and content identity; a response for another output or a partial distribution refuses. A distinct-value or bucket bound returns an explicit unavailable field instead of a truncated distribution. Pages address request_ref and output; the server resolves the accepted result from the run. Cursors bind the result and its digest. Supplying another result’s cursor refuses. Retiring one run ends its demand; other holders retain theirs. An uncertain remote outcome remains uncertain until the existing executor owner reconciles it. bijection run cancel <run> requests cancellation from the original owner and reports cancellation and termination separately. A lost reply leaves the original locator available to run get and another explicit cancel. Cancellation does not retire the run or dispose retained results. run retire releases that run’s interest and reports cleanup separately.

Freshness, execution and retention

Declare read.requirements, read.preparation, execution.timeout, execution.limits and delivery.retain_for on the analysis. Consumers may strengthen inherited requirements, never weaken them. Preparation can request source work only after the accepting mutation commits. Analysis preparation uses a finite native read grant and never invokes a guest action handler as the original user. The grant cannot outlive the authentication that admitted it.

Stored rows and external populations

q.from(table) reads stored rows. q.source(table) additionally requires complete external-population evidence for every installed source. Missing evidence does not mean an empty population. Use a source closure policy when every transitive source input must meet the same evidence contract. Admission derives the input set from the relation, including inputs that contribute no rows:
Local application tables retain their transactional read contract. Synced tables and publication inputs require their native source evidence. Virtual tables whose observation owner cannot provide this evidence refuse the policy; they cannot silently become local inputs. Each source needs its own qualified cut: this policy does not assert a shared snapshot across independent sources. Branches that share the same primitive source must agree on its cut and incarnation, including when they arrive through separate publications. An overrides map can choose a different contract for a named source input, for example { archive: { version: "complete" } }. An exception must belong to the defining relation’s transitive source closure. Nested views preserve their own policies, and a consumer’s requirements are added to them. Clock bounds, population completeness and coherence remain separate obligations. The same requirements can accompany a transactional query or a retained computation. ctx.db.checkRead and ctx.db.requireRead can assess an anchored policy for explicit relation roots. A portable formula over supplied arrays requires an input owner to establish these obligations before evaluation.

Require evidence for a business decision

A successful read establishes its result over the selected population under its read requirements. A qualified empty population is different from an installed source whose acquisition never established that population. Source requirements are checked before rows, including empty ranges, joins and aggregates. Declare the table’s minimum contract beside its source:
This example assumes a source contract that supports a qualified snapshot. population: "observed" explicitly selects acquired records without claiming complete external coverage. A consumer can strengthen the declared contract; it cannot weaken it or silently fall back to observed records. Resolve a declared source scope inside the query or mutation that uses it. The following handler fragment assumes application authorization has returned a vendor, and the source declares companyId as its installation scope:
The source handle binds the read to its actual installation and incarnation; it grants no access. Missing installation, missing coverage and expired observation refuse rather than returning null. An arbitrary row filter cannot certify a smaller external population. .visible() deliberately asks about caller-visible rows and cannot establish absence over the full business population. A valid scope with no installation produces a native missing-evidence refusal under that scope’s read policy. Installing a source invalidates that observation even before it publishes any rows. Invalid scope shapes and application parent references remain errors. For readiness, inspect the same evidence without reading rows or starting work:
An unavailable assessment contains no partial basis. Its issues are content-free; inspect detailed source status only under that source’s current read policy. A ready assessment’s validUntilMs is the earliest time boundary, not a promise that later publications or policy changes cannot invalidate it earlier. Passing its basis into another call grants no permission and does not skip reevaluation. ctx.db.requireRead makes the requirements mandatory for the current execution. A caught native refusal still prevents its mutation from committing. Row reads, source metadata and scope-directory dependencies share the transaction basis; evidence expiry also constrains commit and query release. Subscriptions can become unavailable when evidence expires even if no business row changes. checkRead may instead report missing evidence for an explicit fallback. Malformed contracts and authorization failures remain errors. All-input or all-source defaults on a standalone guard need an explicit relation root; an unanchored closure refuses instead of waiting for refresh. A mandatory vendor-closure rule also needs enforced write ownership or a protected transition rule. A guard constrains a read that occurs; it cannot prevent another writer from skipping that read. Ordinary mutation atomicity does not infer a business invariant from this example. Refresh belongs to an authorized mutation using ctx.sources.requestRefresh({ table: "invoices", scope: { companyId } }). Reads start no acquisition. Refresh completion and read readiness are separate: a provider may be unable to establish the requested guarantee, and existing evidence may remain sufficient while refresh is running. The ordinary query observer distinguishes pending, success, unavailable and error; only confirmed success supports displaying an empty business population. Materialized and retained results preserve the source evidence that produced them. A new publication time cannot renew old input observations; retention preserves availability without granting freshness or access. Independent providers keep independent cuts. An age bound tolerates lag and cannot freeze a provider while a local decision commits. A remote condition that must hold when a command executes needs the provider’s supported conditional operation.

Federate virtual tables through an executor

Declare each virtual table’s logical schema, key, access and executor with defineVirtualTable. Its physical relation and column mapping are deployment configuration installed with the executor, so relocating a warehouse table never rewrites the relation:
Each placement maps every declared field exactly once and has a revision. A run records the revisions it admitted; a changed placement refuses work prepared under the previous one, and a table without a placement is refused by name. The computation derives its executor from those tables. All remote inputs must use that same executor; Trino federates across its configured catalogs. Native inputs require explicit authorized exports. Connection credentials and catalog aliases remain deployment configuration. The Trino cluster is operated by the customer or their provider. See the runnable customer-success example. The analysis-to-pipeline example shows a durable workflow from a result through dataset publication and customer compute to the currently served native relation.

Configure a customer-owned executor

Connect an existing customer engine with typed coordinator settings and a private token file:
Add --interactive for the local operator setup page. Private credentials stay in the CLI process. Setup reviews the declaration and connection revisions and compares them atomically on apply; stale reviews must be prepared again. Table placements apply in the same commit: --tables replaces the installed set, and omitting it keeps the placements installed under the reviewed revision. Preflight and execution share one eligibility decision; a refusal names the requirement (binding, adapter, installation, input version or disclosure) and the input, and an admitted plan lists who enforces each limit. Use --credential-ref instead of --token-file to reuse a private protocol credential; coordinator settings remain required, and Trino/Presto users come from that credential rather than --user. Native dataset bindings and freshness belong to deployed query declarations; neither they nor table placements are copied into credentials. The customer operates the analytical engine. The supported engines are Trino, Presto, Snowflake, Databricks, BigQuery and ClickHouse. Snowflake uses a warehouse and optional role; Databricks uses a warehouse ID. BigQuery uses a customer billing project, location and positive billed-byte cap. These three direct warehouse profiles accept observed tables and independently verified retained warehouse datasets. Snowflake reads a retained clone, Delta uses its exact retained table version, and BigQuery reads a retained snapshot. Admission checks the table lifetime, ownership marker and complete physical schema. Query holds retain that accepted dataset version through its existing custody owner. Trino, Presto, Snowflake and ClickHouse read pinned Iceberg Datasets, native exports and series archives. For Snowflake, map the Dataset catalog identity to the database containing the corresponding Iceberg tables. Schema and table names must match the Iceberg names, including case. Enable Iceberg Time Travel. Queries select the retained snapshot ID with AT(VERSION => …). For ClickHouse, map the Dataset catalog identity to a DataLakeCatalog database containing the Iceberg tables. A statement can pin one external table and can include captured native inputs. Its read-only role needs access to the table and its system.iceberg_history entry, and permission to set iceberg_snapshot_id. The same statement checks for the exact snapshot before the result can be accepted. Databricks and BigQuery require an input representation that their installed profile can pin. Trino’s direct statement adapter requests uncompressed json result encoding. It accepts inline pages and segments, and spooled JSON segments through the same coordinator origin’s /v1/spooled/download/ and /v1/spooled/ack/ routes. The protected cursor captures the original manifest before downloading and checkpoints decoded rows with their acknowledgement capabilities before releasing segments. An interrupted download resumes that manifest; an interrupted acknowledgement retries its saved capability without reading rows again. Expired segments make the original result unavailable; they never cause SQL to be resubmitted. Compressed encodings, direct object-storage or worker retrieval, and Presto spooling refuse. Presto retains its inline-page profile. Protocol fixtures do not establish deployed continuation replay, segment lifetime or cancellation behavior. A Databricks connection can also declare the warehouse’s own STATEMENT_TIMEOUT in whole seconds, from 1 to 3600, with --statement-timeout-seconds (statementTimeoutSeconds in the connection settings). The declaration lets Databricks statements run as long statements up to that admitted timeout. Runs may request up to 60 minutes on Snowflake, BigQuery, Trino, PrestoDB, and on Databricks with this setting; other executors stay at 5 minutes. The warehouse administrator must set STATEMENT_TIMEOUT to the declared value or lower, and admission refuses a warehouse whose effective timeout is unset or larger. Without the setting, a Databricks run ends within one scheduler attempt, under 40 seconds. Preparation validates the configuration without contacting the engine. Explain the intended installed analysis to check its admitted physical types and input readiness. bijection connection executor preflight <definition> [args...] provides a structured report using that same native metadata admission path. It never schedules source preparation or reserves a result. A successful report contains advisory binding, credential, schema and program revision evidence; blocking refusals include remediation. Workspace plans display the evidence. Use connection executor probe <connection> explicitly to check one constant-only statement and exact scalar decoding. Run a bounded declared query to check its workload and result decoding. BigQuery configuration keeps the billed-byte cap as exact positive Int64 text; the native connection owner converts it to the provider’s enforced numeric limit. ClickHouse reads observed tables directly over its HTTP interface. Set engine: "clickhouse" in defineExecutor and use --engine clickhouse, --endpoint, --user and --token-file during setup. It does not accept Iceberg catalog aliases or claim a coordinated source snapshot. The customer operates the ClickHouse service. Capture explicitly selected table contracts with connection executor capture-catalog <connection> --table <alias=database.table> --output <path>. Register the reviewed JSON under clickhouse.catalog in bijection.json; preparation generates _generated/clickhouse.ts before module discovery. Pass a generated table’s source and the executor to defineVirtualTable, then declare access policies. Add --bindings-output <path> to capture the physical placements and UUID/DDL contracts for executor setup --tables <path>. Those installed placements keep their existing native revision and authority owner. The generated source and installed placement share the ordinary captured remote contract. Deployment, native preflight and execution admission refuse a mismatched capture, identity, definition, type or nullability; capture-catalog ... --check compares the current observation with the source artifact without writing it. Sorting and primary indexes do not generate logical keys. Unsupported or lossy types require a reviewed scalar projection. Install the executor declaration and capture its tables before deploying the application schema that consumes those captured sources. The example includes a minimal bootstrap application for this sequence. The direct reader retains a complete bounded result and rejects partial responses and late exceptions. A ClickHouse query ID correlates one dispatch and cannot recover a lost result stream. Lost responses remain unknown without resubmission. A checkpointed complete result can finish native publication after a restart. A captured refreshable materialized view exposes a refresh(governingRule) recipe for defineIntegration. Install it with a separate private command credential. It uses single-attempt delivery; acknowledgement and current view progress remain separate. bijection connection executor view-status <connection> <database.view> reports refresh observations, not attribution to one command. Insert-triggered materialized views expose no refresh recipe. Dataset capture uses the existing saved-result and dataset publication owners. The repository’s examples/clickhouse-analytics demonstrates configuration, bindings, retained results, refresh commands and exact dataset publication.

Generate virtual table bindings

Trino, Presto, BigQuery and ClickHouse tables can have captured interfaces. A manifest selects the physical columns the application consumes. Explicit capture writes an approved contract lock; codegen generates typed sources from those bytes. Deployment never updates a lock to accept drift. Generated application schemas contain logical types and a capture digest. Physical addresses and column mappings remain executor connection configuration. Generate a placement proposal from the same lock and install it before checking or deploying the captured table. Native admission requires the installed capture to match the application’s logical expectations and digest exactly. Declare an ordinary executor:
For a first connection, analyze these declarations before registering manifests or importing captured tables. Export the public executor metadata and prepare the connection without activating application code:
The candidate file contains executor metadata, never deployment environment values. Pending setup requires deployment authority and a declaration with no callbacks, acquisition or commands. It does not install executable code. Use --candidate executor-candidate.json for capture and placement setup until the first activation. After activation, ordinary setup resolves the installed declaration and refuses a pending candidate. Existing executors need no candidate file. Register manifests in bijection.json:
Each manifest names one executor and explicitly selected fields:
Presto uses the same address shape. ClickHouse uses catalog: "clickhouse", schema for its database, and table. Its native catalog capture also uses this shared contract, including UUID and DDL checks. BigQuery uses project, dataset, table; the data project is distinct from the executor’s billing project. Manifests and their adjacent .lock.json files contain interfaces and mappings, never credentials or sampled rows. Snowflake uses catalog for the database and schema for its schema. Databricks uses the Unity Catalog catalog and schema. Capture preserves exact NUMBER/DECIMAL precision and distinguishes civil TIMESTAMP_NTZ from instant timestamps. Snowflake uses DESCRIBE TABLE, including its collation specification. Databricks requires DESCRIBE TABLE EXTENDED ... AS JSON (Runtime 16.2 or newer). Nonbinary collations and unsupported structural types require a reviewed scalar projection. Catalog creation observations detect some replacements; they do not establish immutable retained input versions.
For the first activation, append --candidate executor-candidate.json to both capture commands and the placement setup command above. After adding the generated table declarations, ordinary required checks and deployment validate the actual candidate executor against the prepared connection. A changed executor declaration requires preparing the connection again. Each scan inspects at most 64 relations per invocation. Snowflake, Databricks, BigQuery, Trino and Presto traverse bounded catalog pages; BigQuery’s listing is restricted to the installed executor location. The default traversal allows 50,000 relation observations, 100,000 inspection requests and one elapsed hour. --max-relations, --max-requests and --max-duration-ms set explicit bounds. Failed and resumed requests consume inspection capacity. Completed pages have separate private receipts. A failed metadata request keeps the current page and cursor. Resumption binds the project, manifest, approved lock, candidate, deployment, connection, declaration, credential revision, mapping, catalog scope and work bounds. Changed bindings refuse. The result says observed_listing: pagination does not establish a coordinated catalog snapshot. An approved table absent from that traversal is not_observed, not deleted. New relations produce declaration and contract proposals. Existing relations produce compatibility or drift reports. Neither changes application authority, keys, policies or deployed declarations. Repeat finite batches with --resume; omit it to begin a fresh traversal. Adoption uses ordinary source review, capture, generation and deployment. bijection remote capabilities lake --connection analytics reports the installed adapter’s statement contracts and provider byte cap. Its source snapshot and change support stay unproven until the actual execution owner qualifies the selected source. Inspection does not establish deployed behavior. For recurring discovery and change monitoring, Source Health owns the durable cadence, page records and invalidation outbox. Its admitted sample callback must use a current authorized source adapter. defineRemoteDiscovery and defineRemoteMonitor are exported from @bijectionhq/source-health/application. Monitor events create refresh demand in Pipelines; readable input versions and publication checkpoints remain with their existing execution and dataset owners. Use the generated source through the ordinary table declaration: Each manifest table key must match its logical schema table name. Installing a placement proposal replaces that executor’s complete placement map; include all of its tables in the manifest or combine proposals explicitly before installation.
The source owns validators and field mappings. Keys, navigations and access remain application declarations. Catalog keys do not establish uniqueness. Nullable or unknown-nullability columns generate explicit null unions, never optional object fields. Unsupported selected types refuse generation. Trino and Presto use their existing exact scalar carriers, including decimal and temporal string formats with at most nanosecond precision. Capture checks the native adapter’s advertised value support. The BigQuery execution profile supports integer, string, boolean, binary BYTES, finite FLOAT64, exact NUMERIC, DATE, TIME, DATETIME and TIMESTAMP fields. NUMERIC(P,S) preserves its captured bounds; unparameterized NUMERIC uses precision 38 and scale 9. Decimals travel as exact strings. Declared decimal bounds are checked inside the statement before text formatting and again on decoded outputs, including resumed computations. BYTES uses exact binary carriers. Parameterized BIGNUMERIC is admitted when its captured precision fits the 38-digit logical decimal contract; unrestricted wide values still require a scalar projection. DATETIME remains a civil timestamp; TIMESTAMP is normalized to UTC. Temporal values retain all six microsecond digits, padded to the common nine-digit fractional format. Parameters that would lose submicrosecond precision are refused. Captured scalar parameter relations use the selected executor’s exact carriers on all six SQL execution profiles. Preparation checks row widths and parameter types, removes unused input columns, and preserves the original cell positions. Missing fields remain distinct from explicit nulls. A parameter that exceeds a provider’s precision, timestamp resolution or request encoding bound refuses before submission. Databricks does not admit standalone TIME parameters; Presto timestamp parameters must fit millisecond precision. Required Int64 AVG uses exact division and final half-even rounding on Snowflake, Databricks, Trino, Presto and BigQuery for scales through 18; wider scales require exact residual execution. ClickHouse uses Int256 quotient and remainder arithmetic through scale 38, explicitly checks the 38-digit output coefficient over the complete group population, and rescales the already rounded coefficient with divideDecimal. It requires the provider’s divideDecimal function. Empty populations return NULL, and a later filter or result limit cannot hide a coefficient overflow. Placement still checks the complete expression and result contract. A floating sum is not substituted with a provider’s ordinary SUM when that would change the shared exact accumulator. BigQuery decimal sums use exact wide intermediates, reject rounding and coefficients beyond 38 digits, and publish the declared fixed scale. Calendar buckets use UTC and Monday-start weeks. Floating comparisons, sums and keys remain refused. Decimal and date/time keys retain the shared IR’s existing restrictions; timestamp keys use canonical strings. Full-width BIGNUMERIC sources and native nested values require a supported scalar projection. examples/bigquery-iot includes generated bindings, access policies, reading and daily-consumption computations, and connection setup. Collated columns require an explicitly normalized source for the current exact comparison profile. Unselected added columns and column reordering are compatible. Missing or renamed selected columns, weaker required nullability, changed physical types and changed relation kinds refuse checks. Capture requires complete metadata for each selected relation; truncated browsing output cannot become a contract. Direct view definitions are tracked by default. A table’s explicit "viewPolicy": "schema" permits direct definition changes while preserving its selected interface and does not request view-definition access. Captures label name-only identity separately from provider replacement observations; neither claims an immutable incarnation. BigQuery partition-filter requirements are captured and compared. A definition digest does not freeze transitive views, policies, UDF implementations or data.

Deployment targets

Use targetSources instead of source when environments have different physical addresses:
Capture every target through its selected deployment and connection:
All target interfaces must have compatible logical carriers. A group with only targetSources requires an explicit selection before it is a source:
A missing target refuses instead of falling back to another environment. Native checks use the actual installed connection and the exact selected physical address and captured contract.

Candidate and runtime checks

check defaults to required remote metadata validation. Offline mode still analyzes the candidate through ordinary Bijection preparation, but makes no upstream executor request. Activation performs its own native checks even if a client omits or forges a previous report. It validates candidate field carriers and compiles affected candidate computations through the production typer and SQL lowerer with typed symbolic arguments. This establishes structural support; argument values, input readiness and provider acceptance remain invocation checks. Evidence records source declarations, program digests, selected-field counts, metadata coverage, target-binding identity, and explicit catalog, compilation, planning and execution outcomes alongside candidate, contract, connection, credential, capability and metadata identities. The candidate identity includes its source package. The activation transaction rechecks connection authority and refuses expired observations. The deployment audit retains successful native evidence. A source preparation cache cannot substitute for a fresh upstream check. --provider-plan explicitly validates a declared computation’s bound SQL using Trino/Presto EXPLAIN (TYPE VALIDATE) or a BigQuery dry run. It does not execute the workload. BigQuery reports available byte estimates and retains its enforced execution cap; external-source processing estimates can be unknown. Planning evidence identifies the bound statement and argument digests and covers the submitted arguments, not every future invocation. Runtime admission checks captured metadata again. Exact result decoding and complete-result bounds still apply. Deployment cannot atomically freeze an independent upstream catalog. A captured interface is not a retained data snapshot: observed and fixed computations keep their existing input contracts.

Compose with external pipelines

Use the accepted typed output of the installed QueryWork owner as a pipeline argument. Start the declared pipeline through its own component client in an ordinary authorized mutation, with one stable request key:
The pipeline’s declared dataset argument derives its row contract from the source declaration. Admission validates the exact accepted result, current source authority and retention before recording its input and dispatch intent in the same transaction. Aborting the mutation publishes none of them. The native upload owner collects abandoned byte preparations. Inputs exceeding the retained-file or kept-result CSV bounds refuse. Another publication cannot substitute for that input. Closing the report after admission does not drop the pipeline’s retained input; current authority and erasure still apply. An already published exact dataset version can supply the same declared argument. Store the returned run locator in an ordinary table. Queries call inventoryPipeline.get(ctx, pipelineRun) or inventoryPipeline.inspect(ctx, pipelineRun) under current access policy. The locator grants no permission. Customer jobs read exact admitted inputs and submit immutable candidate output versions. Verification, publication, explicit selection and native serving are separate states. See customer compute for worker configuration and provider protocols.

Publish data for native reads

Declare the inventory row contract once as a typed output, in a module that imports neither the schema nor business handlers. The maintained product binds its row output with publish: { inventory: inventoryOutput }, and the schema synchronizes from the same keyed contract:
Maintenance selects one accepted result. The native publication owner prepares all required tables and materialized views privately and activates them with that selection in one transaction. Pending preparation leaves the previous selection in place. currentInventory.inspect(ctx) exposes selection and preparation; currentInventory.resolve(ctx) checks current usability. Ordinary table reads enforce the same current authority, custody and serving policy. Reading either status or rows never starts work.

Know where a query runs

Native views run in Bijection’s transactional engine. Trino analyses run on the customer’s configured Trino service. External ETL jobs run on customer compute. The common boundary is an immutable, authorized result with input evidence, not a claim that these execution environments share a transaction.

Constrain execution

A local index is an access contract. Native reads honor its range and order; executor lowering preserves its predicate. Unsupported placements, incomplete populations and exhausted work or result budgets refuse.

Read a plan and act on a refusal

The lower-level result client remains useful for native retained queries and administrative tooling. Application analyses expose ordinary authorized mutations and queries. A failure or an unknown provider outcome is observable state; it never authorizes an automatic resubmission.