Lineage should come with receipts

Lineage demos are a bit like magic tricks.

Someone types finance.monthly_revenue into a search bar. The screen blooms: raw events to the left, dbt models in the middle, a Tableau dashboard on the right. Everyone nods. The system seems to know how the company works.

But I always want to ask one impolite question: why should I believe any of those lines?

A lineage edge is not a fact. It is a claim.

Maybe dbt declared it. Maybe the warehouse observed it last night. Maybe Tableau reported it three weeks ago. Maybe an LLM inferred it from a query it couldn’t fully parse. Those are not the same thing.

This distinction mattered less when lineage was a diagram a data engineer clicked twice a month. It matters a lot when an agent uses the graph to decide what broke, who is affected, and whether to propose a production change.

An agent-ready lineage graph needs receipts.

Three hops, three kinds of evidence

A revenue dashboard drops 18 percent overnight. The on-call data engineer gets paged at 2:14 a.m.

The dashboard reads from finance.monthly_revenue, which depends on analytics.orders, which is built from raw.checkout_events. A deploy changed the values in the customer_tier column of checkout_events, and the downstream model still expects the previous categories.

Nothing crashed. The pipeline ran. The dashboard is wrong. To determine what changed, what it affected, and whether a fix is safe, the investigation needs lineage.

The three-hop path looks simple on a screen. The evidence behind it may not be.

One edge may come from a dbt manifest. Another may have been observed in warehouse query history. The final dashboard relationship may have been imported from a Tableau API. The column mapping may be exact for one transformation and partial for another because the query contains a wildcard. Real data platforms are messy.

A receipt keeps those differences attached:

• the provider that reported each relationship
• the environment in which it was seen
• the first and last time it was observed
• the precision of the table and column mappings
• any policy that hid or admitted the edge

Without receipts, the agent can sound certain:

“Root cause identified: the deploy broke the dashboard.”

With the evidence attached, it has to make a narrower claim:

“The changed source field is connected to the affected revenue model by two current, observed dependencies. The final dashboard dependency is declared by the BI provider. Column impact is confirmed through the first transformation and only partial through the second.”

The second answer is less elegant. It is reviewable.

The graph should make the agent less confident

Good lineage should sometimes make an agent say, “I don’t know.”

Isn’t imperfect lineage still better than no lineage? For a human, often yes. A data engineer can notice that an edge looks stale, remember an undocumented dependency, or ask the person sitting nearby.

An agent has none of that ambient skepticism. It will keep reasoning over the graph it receives, including the parts we only guessed.

If a downstream anomaly and an upstream change are connected by confirmed, current lineage, the agent has a reason to test causality. If they are not connected, it should need other evidence before linking them. If coverage is incomplete, the correct answer may be that the relationship cannot yet be established.

Suppose a marketing pipeline also fails at 2:14 a.m. The timing’s suspicious, but it does not sit on the path between checkout_events and monthly_revenue. Without a model of how the systems connect, an agent can invent a plausible explanation between them.

Evidence-backed lineage forces the investigation to show its work. Is there a path? Which direction does data move? When was the relationship active? Does the affected column travel through that path? What evidence contradicts it?

The graph does not make the final causal decision. It defines which hypotheses deserve to be tested.

The graph changes while the incident stays put

Data systems change continuously. A dbt model is renamed. A source is replaced. A dashboard moves to a new semantic layer. A backfill temporarily writes through a path that is not part of the normal pipeline.

That means an investigation sometimes needs lineage as it existed when the alert fired, not lineage as it exists now.

Point-in-time lineage changes the question from:

“What depends on this table?”

to:

“What depended on this table at 2:14 a.m.?”

The distinction matters during a migration, after a hurried fix, and whenever multiple changes overlap. If lineage preserves only the latest snapshot, the investigation trail will be wrong.

An agent-ready lineage system should retain the source evidence even as the active graph changes. Without history, the graph can describe the fix and erase the incident.

Absence is not proof

Before proposing a fix for the revenue incident, the agent has to answer one of the most dangerous questions in data operations: is there any downstream impact?

Sometimes “no impact” means the system performed a complete traversal and found no consumers. Sometimes it means the BI connector lacks access, a query could not be parsed, the request stopped at a node limit, or the supposedly terminal table has never been observed outside the warehouse.

Those are different conclusions.

Every impact claim should come with a coverage claim:

• “No confirmed consumers found within complete coverage” is evidence.
• “No consumers returned because dashboard coverage is unavailable” is a limitation.
• “Traversal stopped after 20 nodes” is a bound.
• “Column mapping is partial” is a warning against exact column-impact claims.

Honest absence is a product feature. It keeps “we could not see it” from quietly becoming “it does not exist.”

The lineage system we want at 2:14 a.m.

The first generation of lineage helped people explore their data systems. The next generation should help people and agents make safer decisions about them.

This is why we’re building lineage differently at Embrasure. We store an edge less like a line on a diagram and more like a citation. Runtime observations, query history, transformation declarations, and BI imports remain distinguishable. Time, coverage, and precision travel with the relationship.

That representation is messier than a beautiful graph. Good.

At 2:14 a.m., I don’t want an agent looking at a beautiful graph. I want it to know which lines it can trust.