The Beginning of Repetitive Work
Yesterday morning, one of our team members cried out loud. 「Do we really have to type in these tracking numbers manually every single time?」
Here's what was happening. Running a B2B operation meant receiving 20 to 30 shipping-related documents daily. These came as PDFs and image files, and extracting the tracking numbers by hand to enter them one by one into a spreadsheet was consuming roughly 8 hours per day. Hundreds of mouse clicks.
That moment was the signal to open Claude Code.
First Attempt: Reading Text from Images
Claude Code's vision capability (image recognition) can read text within photos. The idea was straightforward.
1. Point to a folder of tracking number images.
2. Claude Code reads each image.
3. Use regex to find the tracking number pattern.
4. Automatically enter it into the spreadsheet.
The first code looked like this.
import anthropic
import base64
import json
from pathlib import Path
def extract_tracking_numbers(image_folder_path):
client = anthropic.Anthropic()
results = []
for image_file in Path(image_folder_path).glob('*.png'):
with open(image_file, 'rb') as f:
image_data = base64.standard_b64encode(f.read()).decode('utf-8')
message = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
messages=[
{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": image_data,
},
},
{
"type": "text",
"text": "Find the tracking number in this image. Extract only digits and letters."
}
],
}
],
)
tracking_num = message.content[0].text.strip()
results.append({"file": image_file.name, "tracking_number": tracking_num})
return results
The Second Barrier: The Accuracy Wall
Something interesting happened. Claude could read the images, but it occasionally converted 「TRK-12345」 to 「TRK 12345」 or misidentified 「1」 as 「l」 (lowercase letter L).
Each digital document scanned slightly differently. Images from wireless fax machines were particularly problematic.
The solution? Give Claude more detailed instructions.
"text": """Extract the tracking number from this image.
Rules:
1. Find content containing only digits and uppercase letters.
2. Length is typically 10-20 characters.
3. If you see 'l' (lowercase L), double-check if it should be '1' (the digit one).
4. Return result in JSON format: {"tracking_number": "value", "confidence": "high/low"}.
"""
Accuracy jumped from 94% to 98.5%.
The Third Phase: Connecting to the Spreadsheet
We integrated the Google Sheets API to automatically input extracted numbers into the spreadsheet. This part proved trickier than expected because we needed to ensure no duplicate entries with existing data.
from google.oauth2.service_account import Credentials
import gspread
def append_to_sheet(spreadsheet_id, sheet_name, tracking_data):
creds = Credentials.from_service_account_file('credentials.json')
gc = gspread.authorize(creds)
worksheet = gc.open_by_key(spreadsheet_id).worksheet(sheet_name)
existing_numbers = worksheet.col_values(1)
for data in tracking_data:
if data["tracking_number"] not in existing_numbers:
worksheet.append_row([data["tracking_number"], data["timestamp"]])
An Unexpected Discovery: Windows Device Performance
When we first tested on a Mac mini, the speed was sluggish. But running the same code on a team member's Windows device showed it was 3 times faster. Likely a network bandwidth issue.
After that discovery, we scheduled all bulk processing to run in the Windows environment.
Results: 72 Hours of Change
• Before: 8 hours per day, manual entry
• Now: 15 minutes per day, automated with validation
• Error rate: Previous 3-5% → Now 0.5% or less
• Team member feedback: 「Finally free from this task」
What's the best part? Those 8 hours can now be spent on more meaningful work.
How to Get Started: Taking the First Step
If you have similar repetitive work, follow this sequence.
1. Open Claude Code and describe the problem. 「I want to extract text from images」
2. Test with one sample image. Check accuracy.
3. Clarify the pattern using regex.
4. Consider integration with your spreadsheet or database.
5. Start small, with a folder of 5-10 files for the first automation run.
One more tip: if image quality is poor, Claude will struggle too. Use original files or increase scan resolution whenever possible, and accuracy will improve dramatically.