Images Alone Were Enough
Last week, our team launched an experiment. The orders from buyers came in multiple formats. Some were screenshots, others were photos taken on mobile phones, and some were old PDFs. Team members had to manually read these images and type them into spreadsheets.
The question「Couldn't this be automated too?」sparked the idea.
The Claude Code + Vision API Combination
When we connected the Vision API to Claude Code, something remarkable happened. Upload an order image, and the AI automatically:
• Recognizes product names, quantities, colors, and sizes
• Extracts shipping destinations (countries, addresses)
• Identifies delivery dates and special requests
• Directly inputs into Google Sheets via API
Our first test achieved 94% accuracy. The remaining 6% were cases with blurry handwriting or mixed languages.
Time Comparison
| Task | Manual Processing | After Automation |
|------|------------------|------------------|
| Processing 1 order | 8~12 minutes | 45 seconds |
| Daily average orders | About 4 hours | 30 minutes |
| Error correction time | Daily occurrence | Around twice per week |
Unexpected Benefits
The most surprising part was the side effects.
1. **Reduced Typing Errors**: The typos that occur during manual entry are gone.
2. **Time-Independent Processing**: Upload images, and the system processes them overnight. Results are ready when you arrive in the morning.
3. **Data Consistency**: All orders are normalized to the same format, making subsequent analysis much easier.
There Were Failures Too
Initially, the system couldn't distinguish between orders mixed with Chinese, English, and Korean. We solved this by adding a prompt to Claude: 「Recognize all languages, but separate each by field.」
Also, orders with handwritten content had about 70% accuracy. We set up automatic flagging to alert the team in these cases.
The Workflow Now
Order Image → Claude Code (Vision) → Auto Recognition →
Google Sheets Input → Slack Notification → Team Review (1 min) → Complete
Questions That Remain
Even with this level of automation, human intervention is still needed in some areas. Precisely identifying special requests like「Please add 6 sample units」at 100% accuracy, or learning new buyer order formats from scratch.
The next step is to convert these error data into a learning loop so the model continuously improves its accuracy. Though we're still in the early stages, our team's daily routine has already transformed.