Everyone building AI for infrastructure is having the same conversation: Kubernetes incidents, microservice failures, alert noise. Almost nobody in that conversation is talking about data governance. That's strange, because most of what actually breaks a business runs through it.
The layer nobody's modeling
A schema change nobody told the downstream team about. An Airflow DAG that fails silently because the topic it reads from got deleted. A consumer group drifting for three days while every dashboard stays green. PII moving through a pipeline nobody can trace when a regulator asks where it went.
Transactions, user events, inventory, financial decisions: most critical business outcomes travel through topics and pipelines before they reach anything a dashboard can query. When that layer is ungoverned, the blast radius isn't an engineering incident. It's wrong data powering a business decision, and compliance exposure.
Infrastructure tooling and data observability are both maturing, on their own. Neither connects the two causally. No single tool holds the full state of your data layer, ownership, schema versions, consumer health, lineage, in one place, right now.
That's the gap.
Modeling the runtime without the content is half a picture
Here's the structural problem. The data layer is where production state actually lives. The infrastructure is the runtime it moves through. Model the runtime without the content and you have half a picture, and failure doesn't respect that boundary: a schema change breaks a downstream service, a deleted topic kills a pipeline. If your causal model stops at the infrastructure boundary, you're always starting the investigation from the wrong place.
NOFire already tracks microservices, deployments, and cloud resources in the production map: their state, their relationships, how they change over time. Kafka clusters, brokers, and topics are the same kind of entity. A consumer group is deliberately not one of them: it's a coordination label with no lifecycle of its own, often shared across several deployments, so the entity that belongs in the graph is the consuming service, not the group name. State is the operative word. That's what lets you answer hard questions at any moment, not just when the pager fires.
Kafka is in the graph
Kafka support puts your clusters, brokers, and topics into that same graph, read-only, from outside your applications, no code changes, no libraries to ship. NOFire reads the cluster from the broker itself and resolves every consumer group to the actual service, team, and deploy behind it, not to a group name with no owner.
Two failure modes drove this build, because they're the two that actually cost enterprise teams real engineering hours: a broken message a consumer can't process, and a consumer that's stopped or fallen behind. Both get told apart automatically, and both get tied to the service, team, and change behind them, not handed to you as a number with no owner.
A stopped consumer and a slow one look identical on a lag chart. NOFire reads the committed offset against the topic head, sampled over time, so a frozen offset under a growing backlog reads as a stall, not a scaling problem, before it becomes a page.
The case that actually matters
The easy version of a schema failure is a downgrade: someone loosens compatibility mode and an incompatible message gets through. That happens, but it's not the case worth building for, because it's a misconfiguration a reviewer can catch.
The harder case, and the one NOFire is built to catch, is the one where nobody did anything wrong. A producer widens a field from int to long. The Schema Registry approves the change, correctly, under a backward-compatibility policy: a new reader can still read old data, so the registry has no reason to object. Producers adopt the new version happily. Nothing breaks until a consumer still pinned to the old reader schema meets a record written with the new one, and stalls, with no error thrown and no alert fired, because from the registry's point of view nothing went wrong.
That's a poison pill with no villain: not a downgrade, not a bad actor, just backward compatibility guaranteeing nothing in reverse. NOFire connects the schema version change to the exact consumer it broke, because nobody looking at a stalled consumer would think to check the registry, and nobody watching the registry would know which consumer to warn.
The same mechanism catches a malformed payload or a bad producer deploy just as fast, because it doesn't care which team or which cause produced the bad message, only that a message and a change on the graph land at the same moment.
What's next
Poison pills and lag are the two failure modes that cost teams the most, but they're not the only signals that matter once Kafka is in the graph. Cluster-side health, telling a dead broker or an under-replicated partition apart from an application fault, is next.
For what this looks like on one specific stall, followed all the way past the Kafka boundary into a downstream service that never touches a topic, see Following a Kafka Stall Across a Hop.
Where to start
If your Kafka estate is a black box the business runs on and nobody quite governs it, see how Kafka fits into the graph, or get a demo and connect your first cluster.
