The Problem Started With Images
One day, scanned order forms started piling up in the inbox. Image files in various formats like PDF, JPG, and PNG kept arriving. Manually transcribing each one was simply impossible.
The question "How can I automate this task" came to mind, and my previous experience working with Claude Code seemed like it could help.
Discovering Claude Code's Image Recognition Capability
Claude Code isn't just a code generation tool, it has image analysis capabilities. This means it can directly read image files and extract text and data from them.
The first experiment was simple. I uploaded a sample order form image to Claude Code and asked: "Can you extract the customer name, quantity, and special requests from this form."
**Surprising result**: Claude Code accurately identified all fields in the image in just a few seconds. It even read handwriting quite well.
Workflow Design (Completed in 48 hours)
Step 1: Monitor Input Folder
• Watch a specific folder on the Windows device for new image files
• Use a script (PowerShell) to check the folder every hour
Step 2: Send Images to Claude Code
Automation Prompt:
"Extract the following from this order form:
• Customer Name
• Order Quantity
• Shipping Address
• Special Requests
Return in JSON format"
Step 3: Auto-populate Results to Google Sheets
• Receive the JSON response from Claude Code
• Use Apps Script to insert into the next empty row in Google Sheets
• Also record timestamp and original filename
Step 4: Review Notification
• Send email notification to team after auto-processing completes
• Team only needs to input one line of feedback if review is needed
3 Unexpected Success Points
1. Handwriting was mostly readable too
I initially expected it to work only with printed forms, but handwritten fields were recognized with about 80% accuracy. Cursive was weaker, but block letters were nearly perfect.
2. Multi-language mixed content was handled
Some order forms contained a mix of English, Chinese, and other languages. Claude Code automatically recognized and correctly extracted text in each language.
3. Cost was much lower than expected
Claude Code's image processing uses token-based pricing. Token consumption was lower than anticipated, making monthly automation costs very reasonable.
Pitfalls Encountered During Implementation
Pitfall 1: Poor scan quality
• Solution: Add an image preprocessing stage that automatically adjusts brightness
Pitfall 2: Field order varies between forms
• Solution: Change the prompt to "find specific field names" (shift from position-based to content-based)
Pitfall 3: Some customer names are missing
• Solution: Display a manual input window when auto-detection fails
Results
After launching this automation,
• Order form entry time: 3 hours/week → 30 minutes/week
• Data entry error rate: 4-5% → 0.2%
• Team's trust in automation increased significantly
Claude Code's image recognition proved far more practical than anticipated. This is automation technology that non-developers can fully leverage.
Next Experiment
Next, I plan to build a workflow that automatically generates reply emails based on these extracted data. There should be more discoveries by then.