Warehouse ingestion
Updated July 2026
Overview
Sync selected Postgres or Supabase tables into Embrasure Warehouse using a primary key and either a cursor column or PostgreSQL xmin. This is the standard setup for scheduled analytical copies.
Incremental strategies
Embrasure reads each selected table incrementally using a primary key plus a cursor column, or PostgreSQL xmin when a cursor column is not available. Rows with the same primary key update the current warehouse copy.
Cursor mode is preferred for larger tables because the source can use a covering index. xmin mode is a deliberate fallback for PostgreSQL and Supabase tables; it tracks row versions without requiring a user column, but may scan the source table on each run. Preflight blocks xmin for very large tables so a missing cursor does not turn into an unbounded source query.
Each run advances the table checkpoint after the batch is written successfully. This gives you scheduled analytical copies with simple database requirements.
Recommended index
Strongly recommended for production tables. Without this index, incremental sync can be slow and may put unnecessary load on your source database. Large tables without a usable cursor index may be blocked during preflight.
create index concurrently if not exists orders_updated_at_id_idx
on public.orders (updated_at, id);Required setup
Read-only credentials
Use a source user with SELECT access to the schemas and tables you want to sync. Embrasure does not need write access to the customer database.
Primary key
Every synced table needs a stable primary key so Embrasure can update the warehouse copy of each row.
Cursor column
Use an updated_at, modified_at, created_at, or increasing integer column when available. PostgreSQL and Supabase tables without one can use the xmin strategy instead.
Cursor index
Large cursor-mode tables need an index on the cursor and primary key columns. xmin mode does not require this index, but it may scan the source table on each run.
Column types
Embrasure reads structured Postgres catalog metadata before deciding how to copy a column. If a type cannot be represented safely, the column is skipped with a visible reason instead of being coerced into the wrong shape.
- Common Postgres values stay queryable as native analytical types.
- Source-specific values preserve their original meaning, even when that means text encoding.
- Unsupported values are visible and explainable instead of causing unclear warehouse failures.
- Primary key and cursor columns are stricter than normal columns because they control sync correctness.
| Native scalars | Integers, floating point numbers, booleans, strings, UUIDs, dates, times, and timestamps are copied into matching analytical types. |
|---|---|
| Decimals | Numeric and decimal values preserve precision and scale when the destination supports them. Values outside destination limits are not silently narrowed. |
| JSON | json and jsonb columns are copied as JSON text. This keeps the source value intact without guessing a nested warehouse schema. |
| Arrays | Arrays of supported scalar values are copied as JSON array text. Arrays of unsupported or composite values are skipped. |
| Binary | bytea values are copied as base64 text so query engines read them consistently. |
| Enums and domains | Enum, domain, range, network, geometric, and custom scalar-like values are copied as text unless a safer native mapping is explicitly supported. |
| Unsupported | Composite types, arrays of composite types, pseudo types, trigger types, handler types, and event types are skipped rather than copied incorrectly. |
Primary keys and cursors
If a selected primary key or cursor column is missing or unsupported, Embrasure pauses or disables that table until the configuration is corrected. For auto-discovered non-key columns, Embrasure can remove stale or unsupported columns from the selection and continue syncing the rest of the table.
Likely next additions
The next compatibility additions are focused on common production Postgres extensions and application patterns:
- citext as string for emails and usernames.
- hstore as JSON text for key-value columns.
- interval as text with source metadata.
- PostGIS geometry and geography as WKT or GeoJSON text with SRID metadata.
- Richer enum and domain metadata so agents can understand allowed values.
Limitations
- Hard deletes are not captured. Use a soft-delete column if deleted rows matter.
- Rows can be missed if the cursor column is not updated consistently.
- Very large cursor-mode tables without a usable cursor index are blocked by preflight; xmin mode trades the index requirement for source scans.
- Dropped or renamed columns may pause sync until the mapping is reviewed.
What gets stored
Embrasure stores the synced warehouse tables plus operational metadata such as selected tables, checkpoints, row counts, sync status, errors, and preflight results.