The Problem: Handwritten Addresses Keep Piling Up
Buyers from certain countries send orders as photos via email. This is especially common with small family-run businesses or staff unfamiliar with online systems. Those photos contain shipping addresses, contact details, and special requests scrawled in handwriting across the pages.
We received 200 to 300 such scanned images monthly. Our team member spent 40 hours per week simply reading each one and manually entering the data into Google Sheets. There was constant risk of typos, and whenever handwriting was faint, we had to double-check multiple times.
「Could we automate this repetitive task somehow」
The Idea: Claude's Vision Capability
When writing scripts with Claude Code, we discovered the Vision API. It meant we could read image files and automatically recognize text within them.
We started with a small experiment. We uploaded 5 order photos from a buyer to Claude and asked, 「Can you extract the shipping address and contact information from these images」. The results were clean and accurate. No matter how messy the handwriting, Claude grasped the context and structure to read it correctly.
「What if we turned this into a bot that repeats automatically」
Implementation: 30 Minutes on a Windows Device
We opened Claude Code on our Windows device and wrote a simple workflow.
Step 1. Monitor Images in Google Drive
We set it to detect when images arrive in a specific folder (「Incoming Orders」).
Step 2. Extract Text Using Claude Vision
with open(image_path, 'rb') as f:
image_data = base64.b64encode(f.read()).decode()
response = client.messages.create(
model='claude-3-5-sonnet-20241022',
max_tokens=1024,
messages=[{
'role': 'user',
'content': [
{'type': 'image', 'source': {'type': 'base64', 'media_type': 'image/jpeg', 'data': image_data}},
{'type': 'text', 'text': 'Extract the shipping address, recipient name, and contact number from this order. If handwriting is faint, infer from context.'}
]
}]
)
Step 3. Auto-Fill Google Sheets
The extracted information was automatically entered into Google Sheets rows.
Step 4. Flag as Processed
Processed images were moved to a separate folder (「Processed」) to prevent duplicate handling.
Results: 150 Hours Saved Per Month
Changes became visible in the first week of operation.
Before (Manual Work)
• 30 minutes per image (including handwriting verification)
• 200 images per month × 30 minutes = 100 hours
• Rework for typos: 10 to 15 hours
• Total: 110 to 115 hours per month
After (Bot Operation)
• 3 seconds per image (Claude processing time)
• 200 images per month × 3 seconds = 10 minutes
• Verification work: 30 minutes (spot checks only)
• Total: 1 hour per month
We gained back over 110 hours monthly. Team members could redirect that time to higher-value work, like buyer communication and quality assurance.
Unexpected Discoveries
1. Handwriting Is Actually More Accurate
Handwriting is highly personal. But Claude reads the contextual meaning. For example, if it is an address from a shipping company, it prioritizes recognizing port or dock names. Unlike printed forms, handwriting provided more room for inference, improving accuracy.
2. It Also Captures Notes and Special Requests
Photos often have 「Fragile」 labels, specific time windows, or customs notes scribbled in margins. Humans easily miss these, but adjusting Claude's prompt captured this information too.
3. Auto-Classification of Mixed Languages
The same order might mix Korean, English, and Chinese. Claude automatically sorted each language and populated the correct column.
Next Steps
Currently, we handle static images only. Next, we plan to support PDF order forms. We are also refining a confidence scoring system to route items below 0.7 confidence to manual verification queues.
「When small repetitive tasks pile up, Claude Code's vision capability is a more powerful tool than expected.」
Non-developers can easily get started. The first bot is small in scope but delivers clear results.