The review bot said 「pass」 all week
Our Dream Team has a review bot that checks every post at the very end. One Friday I opened the last 17 review results and every single one said 「pass」, even though a human reader would have caught a few awkward sentences.
The checklist was too kind
The instructions were phrased as questions like 「Is the sentence natural?」 and 「Are the terms consistent?」 The easiest answer to a question is 「yes」. The bot was diligent, but it had no yardstick, so it graded generously every time.
A 100% pass rate is not good news. It is a signal to check your setup.
Only two changes
We did not grow the rules to ten. We attached one failing example.
Fail example: a Traditional Chinese post that contains the Korean wording 「영상을 업로드하세요」 as is
Reason: Taiwanese readers say 影片 for video
Then we added one line: 「Find at least one problem. If there truly is none, write the evidence for that judgment.」
Four-week comparison (a small experiment)
| Item | Before | After |
|---|---|---|
| Issues flagged per week | 2 | 9 |
| Reader fix requests after publishing | 5 per week | 2 per week |
More flags meant more edits at first, which was annoying. But fixing a post before it goes live was far cheaper than fixing it afterward.
Today's note
• Ask 「What is not okay?」 instead of 「Is it okay?」
• One bad example beats ten rules
• If your bot feels too nice, reread your own question