Nine agents, one operator, no polish

I run a handful of AI agents that are supposed to act like a small organization. Not one big model doing everything — several, each with a narrower job, passing work between them. This is the log of that, roughly as it happens. I call it Structure Log. It's not a tutorial or anything. More like the notes I'd leave for myself in case I forget how any of this was set up, except other people can read them.

Here's the thing that got me here. Give one AI a long task and ask it to report back, and it will. The problem is it reports back "done" whether or not it's actually done. Not a dramatic failure — there's no error message and nothing crashes. It just quietly hands over something with a mistake sitting in the middle of it, stated with the same confidence as everything else. The longer and more multi-step the task, the more room there is for that to slip in unnoticed. It's not really a capability problem, either. Point the model at its own mistake directly and it usually catches it. The problem is nobody's pointing.

So I started splitting the work up. One agent writes something, a separate agent checks it, and a third one — or sometimes me — decides whether it actually goes out. It's the same reasoning a company runs on: the person who wrote the report isn't the only one who signs off on it. That's not a statement about trust. A second reader just catches things the writer stopped being able to see, which is most of the value of having one. Add a checking step and things that used to sail straight through start getting flagged. A lot of the flags turn out to be nothing, and every so often one of them is a real mistake that would have gone out the door otherwise, which I figure is the whole justification for the extra step.

Right now that's roughly nine separate roles, each handling one narrow slice — writing, reviewing, approving, a few more specific than that. I say "roughly" because it isn't a clean org chart. Two roles end up covering the same ground and stepping on each other's toes. A checker waves something through because the checking step itself got rushed, which is exactly the failure this whole setup was supposed to prevent, just one layer up. I'm still moving the lines between the roles around, and I'll probably be doing that for a while.

Why put this out in public instead of fixing it privately and moving on. Mostly because the failures are more useful than a finished version would be. Nobody learns much from "and then it worked." What's actually useful is what broke — what the wrong output actually looked like — and what got changed afterward. And these failures don't announce themselves while they're happening. They show up later, looking back, once something downstream has already gone wrong or a number didn't add up. So this gets written after the fact, not during. Some of it will read messier than a finished writeup usually does, which I've decided not to worry about.

That's the setup. The rest of this log is just what happened.