Handling Feedback — Adopt, Hold, or Reject
External audits (checks carried out by an AI from a different vendor, independent from our internal team) often raise issues from angles we cannot see from inside. That kind of feedback has real value. But whether every issue that comes back should be adopted without question is a separate matter.
Whether a piece of feedback is "correct" and whether it should be "applied to the project as-is" are not always the same question. This chapter is about how we handle feedback from external audits.
Read It With the Assumption That Outsiders Don't Know the Full Context
External auditors do not necessarily have access to the operating policies, the history of past discussions, or the project's current priorities that the seven internal subagents (AI agents, each assigned one of seven roles) share. They make their judgment based on the deliverables handed to them and whatever explanatory materials come with them at that point in time.
This is not a shortcoming — it is a basic condition of being an external auditor. Precisely because they do not know too much context, they can catch blind spots that the internal team misses. As mentioned in The Role of the "+1" External Auditor, the seven internal subagents operate within the same design philosophy, which makes it structurally difficult for them to notice problems outside that philosophy. External audits play the role of looking in from outside it.
That said, feedback given without full context sometimes includes points that are "correct as general advice, but don't fit where this project stands right now." For example, we sometimes get feedback asking for production-level robustness on a mechanism that is still in the testing stage. The feedback itself is not off base, but whether it's worth the cost at this phase is a separate judgment call.
Separating "Correct Feedback" From "Feedback to Act On Now"
So we decided to sort feedback into three categories: adopt, hold, and reject.
Adopt covers feedback that is correct and worth acting on at the current phase. We reflect this straightforwardly.
Hold covers feedback where the direction is right, but acting on it immediately would crowd out other priorities. We just record it along with the reasoning and revisit it at the next opportunity. The difference from simply ignoring it is that we're explicitly deciding "we'll deal with this later," not leaving it unattended.
Reject covers feedback that simply doesn't align with the project's current design philosophy or goals. This isn't a fault on the external auditor's side — it's a natural gap that comes from a difference in viewpoint. As described in Why We Run Three Rounds, going through multiple rounds gradually narrows this kind of gap. It's not unusual for feedback raised in round one to already be resolved by round two.
Which of the three categories a piece of feedback falls into depends not only on its content but on what the project is prioritizing right now. The same feedback can be an adopt at one time and a hold at another.
Recording the Reason Even for a Reject
Of the three categories, reject is the hardest to handle. It can look like we simply ignored the feedback.
So whenever we decide on reject, we always record the reason. We leave an explanation along the lines of, "we understood the point of the feedback, and here's why we're not applying it to this project." This is both a matter of courtesy toward the external auditor and a way to check our past judgment if the same feedback comes back in a different form later.
A reject with no reason attached is, in practice, no different from waving off the feedback because it was a hassle. The process of putting the reason into words also sometimes reveals whether reject is really the right call, or whether it should actually be a hold.
In the end, a human decides which category to use. Feedback from external audits is material for a decision, not the decision itself. Having this line drawn lets us take in outside perspectives while still staying on course toward the project's goals.