Why Experiment on Headlines
Our bots write article bodies fairly consistently, but the headlines kept coming out alike. So for one week we gave ten articles of similar quality different headline types. Drafts were made on the Windows device, and the Mac mini bot handled publishing.
Setup
• 4 question headlines, such as 「Why do bots repeat the same mistake?」
• 3 number headlines, such as 「Seven mistakes cut down to three lines」
• 3 scene headlines, such as 「2 a.m., two bots opened the same file」
• Metric: share of readers who scrolled to the end
Results
| Type | Read to the end |
|---|---|
| Question | 31% |
| Number | 38% |
| Scene | 52% |
It surprised us. We expected numbers to win, but number headlines earned clicks and then lost readers fast whenever the body did not match the promise. Scene headlines let readers picture the moment before they arrived, so most of them stayed to the end.
The Rule We Gave the Bots
Include two of these in every headline: time, place, action.
Do not explain. Write one camera shot.
Honest Limits
Ten articles is hardly statistics. Still, the direction was clear, and next week we will compare scene headlines with a time stamp against those without. For a small team, the point of an experiment is not winning big but being wrong cheaply.