Every Report Sounded Sure of Itself
The daily reports our Dream Team bots file were always confident. 「Done.」 「No issues.」 The problem was not that they were occasionally wrong. It was that we could not tell where they might be wrong. So every time a report landed, Bella re-checked it from top to bottom. Over two weeks, 14 of 20 reports got a full re-review.
The Change Was One Line in the Prompt
We added this rule at the bottom of each bot's instructions.
End every report with a 「Needs checking」 section.
List only the parts you are not confident about. If none, write 「None」.
Example: Needs checking: the date in table 2 is an estimate, not from the source
The key was one extra sentence: 「Admitting uncertainty is never penalized.」 Bots tend to write toward whatever earns praise, so we had to give permission to be unsure first.
Week One Was Close to a Failure
At first the bots seemed anxious and tagged every line with 「Needs checking」. That is the same as tagging nothing. So we added a cap: three items at most, ranked from least certain. Once that line went in, the tag finally became a signal.
Results
The following two weeks, same 20-report sample.
• Full re-reviews: from 14 to 5
• For the other 15, we only opened the 「Needs checking」 line
• Of 8 real errors, 6 were inside that line
The remaining 2 were cases where a bot was confident and wrong. This method does not remove errors. It just tells us where to start looking.
Today's Notes
1. Say up front that stating uncertainty is fine.
2. 「None」 is information too. It means you can move on.
3. Without a cap, the flag gets plastered everywhere.
This seems to work on humans as well, which made us smile while writing it.