Ship It Conversations: The AI Hallucination That Looks Right, With Devansh of Irys on Legal AI, Three-Tier Error Isolation, Models That Change Under You, Overhyped Reinforcement Learning, and Optimizing for Maintenance Over Generation

The thing that stuck with me most from this conversation with Devansh is a trade he's willing to make that a lot of AI teams aren't.

I asked him what the hardest part is about running an AI system like his in production. His answer was being able to track everything.

Even if you could get 20% better performance tomorrow by going fully agentic, he said, the problem is "you never know when they break and you never know how they break until they break." With structure, "even when it's failing you know where it's failing."

He'll take the system he can debug over the one that scores better.

Every on-call engineer understands that in their gut. A clever system that performs a little better and fails in ways nobody can explain is worse to own than a boring one you can reason about at 3 AM.

That trade runs through everything he said. Models are getting really good at doing a lot of things somewhat well, but the last mile to reliable is where they fall short. So his team engineers structure back in instead of hoping the next model fixes it.

At the top of the episode I compared his approach to labeling the breaker panel in your house. Here's why it fits.

His team splits the system into three tiers. Ingestion takes messy input and maps it into a knowledge graph. Reasoning navigates that graph. Delivery applies how a particular firm or lawyer likes to work.

What matters is how they use that split when something goes wrong.

If a lawyer says the answer is missing information, they immediately know where not to look. In his words, "We don't need to care about the personalization there." Then they check whether the data is in the graph at all. If it is, ingestion did its job, and the problem is either the way they mapped the structure or the reasoning that moved through it. Two or three targeted queries, and they know which.

That's incident triage. The first useful question in almost every incident is some version of "which layer is this?"

He also described the tiers by their impact on the user, not as engineering layers. You can always go arbitrarily deep on the engineering, he said, but grounding it in what the user experiences means you can always narrow a problem down.

That's the same reason we write SLOs from the user's side. Nobody reports "the cache hit rate dropped." They report that it's slow, or that something's missing. If your boundaries line up with the complaints, the complaint tells you where to look.

The second idea is about which failures are actually dangerous.

Everybody's heard about AI inventing court cases. Those fall apart the moment somebody looks them up. Devansh says the more interesting hallucination is a real case used the wrong way. The case exists. The citation checks out. It just doesn't apply the way the AI is applying it.

So much of legal work is interpretation: taking what was already argued and deciding how it applies here. Get that wrong and, as he put it, you're "creating bad legal strategy." And because the case is real, it looks fine.

His explanation of why it's so hard is worth sitting with. A model is trained to give you the most probable output, which he called "a kind of reversion to the mean." And then: "Better models mean that you're reverted to a higher mean than before." But legal judgment lives in the outliers. How this firm likes to operate. Which kind of authority this lawyer would rather rely on. A clause that gets modified by another clause somewhere else in the contract.

I see the same pull with AI-generated infrastructure code. Ask for a config and you get a very reasonable, very average config. Your environment isn't average. It has the odd network rule, the legacy service, the account set up differently for a reason nobody wrote down.

In the recap I said a config that's valid but wrong hurts more than one that won't parse. The broken one fails in CI. The valid one deploys clean, passes every check, and waits.

The knowledge graph is part of how his team catches those connections. It exists, as he put it, "to teach our AI layers where to look." Chunking documents and searching for similar text hopes that everything relevant sits close together. Real input doesn't. His example was a PDF with handwritten notes and an email chain that refers to part of that PDF. Two documents, one idea, and a chunker treats them as strangers.

When I asked what happens when the corpus goes from 10 docs to 10 gigabytes, he said size isn't always the problem. For most tasks, "it's still only three, four, five documents that actually matter." The hard part is finding them.

That's most incidents. Terabytes of logs and metrics, and the answer is in three lines somewhere. Volume isn't the problem. Knowing where to look is. Service catalogs, dependency maps and good runbooks are our version of that graph.

The third idea is the one I'd point every platform team to. The model can change under you.

When I asked what his team logs and measures, he said it's still an open discussion. But the number one thing they track is simple: "does it behave as expected?" Can they predict ahead of time what path the system will take? If they can, at least they know it's stable.

That's harder than it sounds, because labs change models behind the scenes. Post-training. Quantization. Tokenizer changes. No new version number, and your output shifts anyway.

What I like about his metric is that it isn't "is it good?" It's "is it predictable?"

That's the right order. Without a stable baseline, you can't tell whether a change made anything better. Normally we'd pin a dependency and move on. You can't pin a model someone else is quietly changing, so you watch it. That's drift detection, and we know how to watch for drift.

His pick for the most overhyped technique in AI was reinforcement learning. RL works when the output is truly verifiable and the constraints are hard. The trouble is people using it for open-ended goals, like making an LLM reason better or write better.

"What does better mean? Better has no signals."

So teams fall back on proxies, like whether code compiles in a certain amount of time or a sentence uses fewer emojis. Every SRE has seen this with metrics. Measure how fast tickets close and you get tickets closed, not problems fixed.

Which brings me to his advice for anyone building with AI.

Devansh thinks too many people are building AI for high performance, because that's what gets you the cool demos. It often leads to overly complicated systems that get re-engineered from scratch once people find where they go wrong. So "people should be engineering for error isolation, first and foremost."

That goes double with coding agents, because "these systems will just churn out heaps and heaps of code." His answer, and the line I said I'd tape to my monitor: "optimize for maintenance, not for generation. Because maintenance is where your system either lives or dies."

Generation is the demo. Maintenance is the pager. Code an agent wrote in an afternoon is code your team owns for years.

His closing thought is one I agree with. None of us really know where AI is headed, and there are no true experts, so don't wait for someone to tell you how to build it. Get in the mud. In his words, "the only way to build taste and judgment is for you to fail repeatedly," and then look hard at where things failed and where they worked.

That's how on-call works too. Nobody gets good at incidents by reading about them. You get good by working a lot of them and doing the postmortem honestly.

So my takeaway from this one is simple.

You can't fix what you can't find.

Label the breakers before you need them. Watch for drift in the things you don't control. And when a tool makes it cheap to generate more, make sure it isn't making it expensive to maintain.

A system that's 20% better and impossible to debug isn't better. It's just harder to fix.