Two Bots That Only Praised Each Other
Our dream team has one bot that translates and another that reviews. Once a Korean card-news post is turned into Traditional Chinese, Simplified Chinese, and English, the review bot reads it again. But starting one week, the review reports became almost identical. Every day it was the same line: 「No issues, reads naturally.」
Planting Errors on Purpose
We hid seven errors in a twenty-sentence sample: Simplified wording slipped into a Traditional text, one number changed, and inconsistent honorifics. The review bot found exactly one and passed everything else.
The Only Change Was One Line
We rewrote the instruction from 「Please check whether there are any mistakes」 to 「Your job is to find at least three mistakes in this translation. If there truly are none, explain why, with evidence」. Running the same sample again, it caught six of the seven.
Small Lessons
• 「Please check」 is easily read by a bot as 「Please approve」.
• If you say that finding faults is the job, the bot actually goes looking.
• Forced nitpicks can creep in, so always ask for one line of evidence too.
• We keep the planted-error sample and rerun it every quarter to make sure the reviewer's eye hasn't dulled.
Role: You are an auditor.
Goal: Find at least 3 errors, each with 1 line of evidence.
Human teams are the same. After enough trust, a 「check」 quietly turns into a greeting. The bots needed to be asked to find faults, not to confirm.