Require it, don't check for it

I made the inbox reader four times faster and half as expensive, and it quietly stopped doing about a third of its job. The fix wasn't a better prompt. It was noticing I'd used the wrong kind of mechanism for the work.

The silence

Right after I deleted the tables, a read that used to take three or four round-trips to the model took one, and cost went from about twenty cents an email to under ten. Delighted with myself.

Then I did the thing I should have done first: replayed ten real emails through the new reader and diffed the results against what the old one had produced on the same mail. Three of the ten had lost a to-do.

Not gotten one wrong. Lost it. The old reader had written "call the printer about the reprint" and the new one had written nothing at all, and gone on to produce a perfectly good summary of the email.

Which is the worst possible shape for this failure, because every to-do on that list was written by the reader. A list where the machine writes all the entries doesn't gracefully degrade when the machine stops writing. It just goes quiet, and looks fine, and you find out in three weeks when something you were supposed to do didn't happen.

Why it went quiet

The cause was almost funny. In the new design, the reader ends its work by calling one final tool — here's the summary, here's the category, here's the draft reply — and that call ends the turn. I had also told it, in the instructions, to write any to-dos it noticed. Those instructions came after the part about finishing.

So it finished. Everything I'd written below "and then call the final tool" was dead prose. The model never disobeyed anything; it did exactly what the sequence said. Part of my beautiful speedup was just dropping work.

The fix that worked and still bothered me

There's a clean fix. Tool calls in a single message execute in order, so the model can put the to-do write and the final call in the same message and everything runs, at no extra round-trip. I restructured the instructions so the optional work composed above the terminal call, re-ran the ten emails, and got the writes back at one round-trip.

It worked. And it bothered me, because it worked on account of the order of my paragraphs. The next person to tidy up that file — including me, in a month, with no memory of this — moves one section and the to-dos silently stop again. There'd be no error. No test failure. Just a list that quietly stops growing.

I'd fixed the bug and left the trap.

Four kinds of thing

So I did a first-principles pass instead, and it came out cleaner than I expected. The reader does four kinds of thing, and each one has exactly one correct mechanism:

What's knowable before the turn: inject it. The reader used to have a tool for looking up your open items. That's a lookup the model has to remember to want, phrased as a search that might miss. But I know which items are open before the turn starts — so hand it the list. No tool, no round-trip, no missed search.

What's a judgment: require it as a field. Is there a to-do in this email? Is there a date? Those are judgments only the model can make. So they're required fields on the answer it has to give me.

What's mechanical: execute it yourself. Deduplicating, resolving an id, deciding whether this to-do already exists — that's code. Don't make the model do it. Let it declare the intent; my code does the work.

What's genuinely open-ended: loop. Reading an attachment, searching a thread when the email references something it can't see. That's the only place a real agent loop earns its cost.

Require it, don't check for it

The middle two are where the whole lesson lives, and I'd had them backwards.

The version I'd built used a check: let the model answer, then verify it hadn't skipped the to-do, and if it had, tell it to go back and try again. That's a perfectly normal design. It also has two problems, and only one of them is about cost.

The cost one: a check catches the omission after the fact, so fixing it costs another round-trip to the model. You pay for the mistake every time it happens.

The real one: a check notices the failure. A required field makes the failure unrepresentable. When "did you see a to-do here?" is a required field on the terminal answer, the model cannot skip it, because the request isn't accepted without it. There's no ordering of my paragraphs that breaks it. There's no tidy-up that silently disables it. The thing I was writing prose to prevent became structurally impossible.

That's the difference between asking for compliance and removing the option, and it's the same instinct that runs through everything else here: put the guarantee in the structure, not in the wording.

The declaring-versus-doing half is the same argument. When the model both decides and executes, you have two ways to fail where one would do — and "decided correctly, didn't execute" is pure loss, invisible, with a good-looking summary sitting on top of it. Let it declare; let my code execute; if a write fails, hand it back as an error and let it fix only the thing that broke.

The tell I'd been walking past

Here's what I find slightly embarrassing in hindsight. When I tore out the tables, every field on the reader's answer survived untouched — the summary, the category, the suggested folder, the draft. The only two capabilities that quietly vanished were the two implemented as tools.

The structure had been telling me which mechanism was wrong. Things expressed as fields were robust to a big refactor because they're part of the answer's shape. Things expressed as tools were optional behavior, and optional behavior is exactly what gets dropped when you tell a model to hurry.

The escape hatch is load-bearing

One nuance, and it matters more than it sounds.

Required means the model must decide, not that it must invent. Every required judgment ships with an honest way to decline — a literal "none of these." Without that, you've made fabrication mandatory: the model has to put something in the box, so it puts in a plausible guess, and now you've traded a missing to-do for a confident wrong one. That trade is always the wrong way round.

And the edge of the rule, which I got wrong once and had to walk back: only require a field when every possible value is a claim. On the to-do list there's a field for who you're waiting on. I nearly made it required — and empty, there, doesn't mean "I forgot to check." It means nobody is waiting on this; it's on you. Requiring an answer would just make the model assert something about an absence. Require the ones where silence hides a missed judgment. Leave the ones where silence is the answer.

What it cost and bought

Three-to-four round-trips per email became 1.1. Twenty cents an email became eight. The writes came back and stayed back.

And one thing got more correct along the way: instead of the model matching an existing to-do by a fuzzy phrase and sometimes coming back "ambiguous," it now picks an exact one out of the list I handed it. Injecting the knowable thing didn't just save a lookup. It removed a whole class of near-miss.

The rule

When you catch yourself writing a line in a prompt to make sure the model doesn't skip something — stop and ask whether you could have made skipping impossible instead.

Most of the time the answer is a required field, and it's cheaper, and it can't rot.