Warehouse ingestion

Updated July 2026

Overview

Sync selected Postgres or Supabase tables into Embrasure Warehouse using a primary key and either a cursor column or PostgreSQL xmin. This is the standard setup for scheduled analytical copies.

Incremental strategies

Embrasure reads each selected table incrementally using a primary key plus a cursor column, or PostgreSQL xmin when a cursor column is not available. Rows with the same primary key update the current warehouse copy.

Cursor mode is preferred for larger tables because the source can use a covering index. xmin mode is a deliberate fallback for PostgreSQL and Supabase tables; it tracks row versions without requiring a user column, but may scan the source table on each run. Preflight blocks xmin for very large tables so a missing cursor does not turn into an unbounded source query.

Each run advances the table checkpoint after the batch is written successfully. This gives you scheduled analytical copies with simple database requirements.

Recommended index

Strongly recommended for production tables. Without this index, incremental sync can be slow and may put unnecessary load on your source database. Large tables without a usable cursor index may be blocked during preflight.

Postgres
create index concurrently if not exists orders_updated_at_id_idx
on public.orders (updated_at, id);

Required setup

Read-only credentials

Use a source user with SELECT access to the schemas and tables you want to sync. Embrasure does not need write access to the customer database.

Primary key

Every synced table needs a stable primary key so Embrasure can update the warehouse copy of each row.

Cursor column

Use an updated_at, modified_at, created_at, or increasing integer column when available. PostgreSQL and Supabase tables without one can use the xmin strategy instead.

Cursor index

Large cursor-mode tables need an index on the cursor and primary key columns. xmin mode does not require this index, but it may scan the source table on each run.

Column types

Embrasure reads structured Postgres catalog metadata before deciding how to copy a column. If a type cannot be represented safely, the column is skipped with a visible reason instead of being coerced into the wrong shape.

  • Common Postgres values stay queryable as native analytical types.
  • Source-specific values preserve their original meaning, even when that means text encoding.
  • Unsupported values are visible and explainable instead of causing unclear warehouse failures.
  • Primary key and cursor columns are stricter than normal columns because they control sync correctness.
Native scalarsIntegers, floating point numbers, booleans, strings, UUIDs, dates, times, and timestamps are copied into matching analytical types.
DecimalsNumeric and decimal values preserve precision and scale when the destination supports them. Values outside destination limits are not silently narrowed.
JSONjson and jsonb columns are copied as JSON text. This keeps the source value intact without guessing a nested warehouse schema.
ArraysArrays of supported scalar values are copied as JSON array text. Arrays of unsupported or composite values are skipped.
Binarybytea values are copied as base64 text so query engines read them consistently.
Enums and domainsEnum, domain, range, network, geometric, and custom scalar-like values are copied as text unless a safer native mapping is explicitly supported.
UnsupportedComposite types, arrays of composite types, pseudo types, trigger types, handler types, and event types are skipped rather than copied incorrectly.

Primary keys and cursors

If a selected primary key or cursor column is missing or unsupported, Embrasure pauses or disables that table until the configuration is corrected. For auto-discovered non-key columns, Embrasure can remove stale or unsupported columns from the selection and continue syncing the rest of the table.

Likely next additions

The next compatibility additions are focused on common production Postgres extensions and application patterns:

  • citext as string for emails and usernames.
  • hstore as JSON text for key-value columns.
  • interval as text with source metadata.
  • PostGIS geometry and geography as WKT or GeoJSON text with SRID metadata.
  • Richer enum and domain metadata so agents can understand allowed values.

Limitations

  • Hard deletes are not captured. Use a soft-delete column if deleted rows matter.
  • Rows can be missed if the cursor column is not updated consistently.
  • Very large cursor-mode tables without a usable cursor index are blocked by preflight; xmin mode trades the index requirement for source scans.
  • Dropped or renamed columns may pause sync until the mapping is reviewed.

What gets stored

Embrasure stores the synced warehouse tables plus operational metadata such as selected tables, checkpoints, row counts, sync status, errors, and preflight results.