Classify 2,000 support tickets with Claude using 21 API calls
You have a 2,000 row support ticket export and a problem: “can’t log in again ugh” and “authentication failed after password reset” are the same complaint, but your spreadsheet has no signal for that. So you scroll. And scroll. And eventually find the hundred rows that actually need attention.
There’s a better way. This guide walks through a Python script that cleans the CSV with pandas, sends only the unique descriptions to Claude for classification, and flags anything below a confidence threshold for human review. The whole thing runs in about 20 minutes to set up and takes roughly 2 seconds to process 2,004 rows.
You need Python, a Claude API key, and three libraries: anthropic, pandas, and python-dotenv. Install them with:
pip install anthropic pandas python-dotenv️ Step 1: Generate the test data
Most tutorials tell you to bring your own data. That’s frustrating because when results look wrong, you don’t know if the script is broken or the data is messy. Generating a known file means you already know what the output should look like.
Save this as a separate Python file and run it once:
"""Build a realistic messy tickets.CSV to test the classifier against."""
import random
import pandas as pd
random.seed(42)
TEMPLATES = [
# login problems, worded the way people actually word them
"cant login again ugh",
"Unable to authenticate after password reset",
"Login page just spins forever and never loads",
"Two-factor authentication code never arrives",
"It says my account doesn't exist but I've had it for years",
# billing
"charged $49 twice this month??",
"I was billed for the annual plan instead of monthly",
"Invoice #1092 shows the wrong amount",
"Still being charged after I cancelled",
# bugs
"Getting a 500 error when I try to export my report",
"Dashboard shows stale data even after refreshing",
"export button does nothing on Safari",
"app crashes when I upload a photo over 10mb",
"Search results are missing items that clearly exist",
# feature requests
"can you add bulk delete to the dashboard",
"Would love a keyboard shortcut for creating new tasks",
"Please add support for exporting to PDF, not just CSV",
"would be great to have dark mode",
# the genuinely vague ones
"Not sure this is the right place, but the site feels slow today",
"Is there a status page for outages",
"Just wanted to say the new update looks great",
]
# Unique IDs, so any duplicate in the file is one we put there on purpose.
ids = random.sample(range(3000, 5000), 2000)
rows = [
{"ticket_id": f"T{i}", "description": random.choice(TEMPLATES)}
for i in ids
]
# Real exports contain accidental repeats (a webhook fires twice, someone
# re-runs the export). Add four, so the dedup step has something to catch.
rows += random.sample(rows, 4)
df = pd.DataFrame(rows)
df.to_csv("tickets.csv", index=False)
print(f"Wrote {len(df)} rows to tickets.csv")
print(f" {df.ticket_id.nunique()} unique ticket_ids "
f"({len(df) - df.ticket_id.nunique()} duplicates)")
print(f" {df.description.nunique()} unique descriptions")The output tells you everything you need to verify the rest of the script against: 2,004 rows total, 4 duplicates planted on purpose, and only 21 unique descriptions. If you skip the deduplication step and call Claude once per row, you’re paying to ask the same question roughly 100 times over. The script below avoids that entirely.
Why Claude handles classification but not cleanup
Before writing a single line of AI code, it’s worth being clear about where the model actually earns its keep. A lot of AI automation adds cost without adding value.
| Task | Plain Python (pandas) | Claude |
|---|---|---|
| Count tickets per day | Yes | Waste of money |
| Fix dates, trim whitespace | Yes | Waste of money |
| Categorize varied language at scale | No | Yes |
Recognizing that “can’t log in again ugh” and “authentication failed after password reset” express the same complaint requires language understanding. That’s Claude’s job. Everything else is pandas.
Step 2: Set up your API key
Create an API key from the Anthropic console and store it in a file called .env in the same folder as your script:
ANTHROPIC_API_KEY=sk-ant-your-key-hereUsing python-dotenv to load this at runtime keeps the key out of your source code. A leaked API key means unauthorized access to your Claude account and potentially large unexpected charges.
Step 3: Define Claude’s role before you write the prompt
Claude’s job here is narrow on purpose: sort each ticket into exactly one of five categories and attach a confidence score. That’s it. It doesn’t resolve tickets, draft responses, or make judgment calls on priority.
The five categories are Billing, Login Issue, Bug Report, Feature Request, and Other. Anything Claude isn’t confident about gets flagged for human review rather than silently mislabeled.
Keeping the task single-purpose is what makes the output trustworthy. The more you ask a model to do in one call, the more ways it has to go wrong.
Step 4: Clean the CSV before touching the API
Strip whitespace and drop duplicate ticket IDs using pandas before a single API request goes out:
import pandas as pd
df = pd.read_csv("tickets.csv")
df["description"] = df["description"].str.strip()
before = len(df)
df = df.drop_duplicates(subset="ticket_id").reset_index(drop=True)
print(f"Dropped {before - len(df)} duplicate(s), {len(df)} tickets left.")This drops the 4 planted duplicates before they reach Claude. If your export had 400 duplicates instead of 4, skipping this step would cost you 400 unnecessary API calls on data you already processed.
Step 5: Send tickets in batches, not one at a time
Two decisions keep the API bill small. First, group tickets into batches of 20 per request instead of one per request. Every API call has overhead; batching absorbs it. Second, cache results so identical descriptions are never sent twice.
The categorize_tickets function sends a numbered list to claude-haiku-4-5-20251001 and parses the response line by line:
import os
from anthropic import Anthropic
from dotenv import load_dotenv
load_dotenv()
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
CATEGORIES = ["Billing", "Login Issue", "Bug Report", "Feature Request", "Other"]
MODEL = "claude-haiku-4-5-20251001"
SYSTEM_PROMPT = (
"You are a classifier for support tickets. You will receive a numbered "
"list of tickets. Classify each ticket into exactly one of these "
f"categories: {', '.join(CATEGORIES)}. "
"Respond with one line per ticket in the form "
"'<number>. <category>, <confidence>', where confidence is a score from "
"0 to 1. Match the input numbering exactly. No other text."
)
def categorize_tickets(client, descriptions):
ticket_list = "n".join(
f'{i}. "{d}"' for i, d in enumerate(descriptions, start=1)
)
response = client.messages.create(
model=MODEL,
max_tokens=25 * len(descriptions),
temperature=0,
system=SYSTEM_PROMPT,
messages=[{"role": "user", "content": f"Tickets:n{ticket_list}"}],
)
lines = [l.strip() for l in response.content[0].text.strip().splitlines() if l.strip()]
if len(lines) != len(descriptions):
raise ValueError(f"Asked about {len(descriptions)}, got {len(lines)} answers back")
results = []
for i, line in enumerate(lines, start=1):
number, dot, rest = line.partition(".")
if not dot or number.strip() != str(i):
raise ValueError(f"Line {i} is misnumbered: {line!r}")
category, comma, confidence = rest.partition(",")
if not comma:
raise ValueError(f"Line {i} has no confidence score: {line!r}")
category = category.strip()
confidence = float(confidence.strip())
if category not in CATEGORIES:
raise ValueError(f"Made-up category: {category!r}")
if not 0.0 <= confidence <= 1.0:
raise ValueError(f"Confidence out of range: {confidence!r}")
results.append((category, confidence))
return resultsMost of that function is response validation, not API logic. That’s deliberate. Claude is good at classification but can still drop a line or invent a category label. Without strict parsing, a mislabeled row looks identical to a correct one in your output CSV.
Haiku is used instead of Sonnet or Opus because sorting a sentence into five buckets doesn’t require a more expensive model. Model choice is the single biggest lever on your API bill.
Step 6: Cache answers across the full run
The classify_dataframe function identifies which descriptions haven’t been seen yet, groups them into batches, and uses a shared cache dictionary so duplicate descriptions are never sent to Claude twice:
from concurrent.futures import ThreadPoolExecutor, as_completed
def classify_group(client, descriptions):
try:
return categorize_tickets(client, descriptions), 0
except Exception as exc:
print(f" Group of {len(descriptions)} failed ({exc}), retrying one by one")
results, failed = [], 0
for d in descriptions:
try:
results.append(categorize_tickets(client, [d])[0])
except Exception:
results.append(("Needs Review", 0.0))
failed += 1
return results, failed
def classify_dataframe(df, client, cache, max_workers=5, group_size=20):
descriptions = df["description"].tolist()
unseen = [d for d in dict.fromkeys(descriptions) if d not in cache]
groups = [unseen[i:i + group_size] for i in range(0, len(unseen), group_size)]
print(f" {len(df)} rows, but only {len(unseen)} new sentences "
f"= {len(groups)} request(s)")
failed = 0
with ThreadPoolExecutor(max_workers=max_workers) as pool:
futures = {pool.submit(classify_group, client, g): g for g in groups}
for future in as_completed(futures):
group = futures[future]
results, group_failed = future.result()
cache.update(zip(group, results))
failed += group_failed
df[["predicted_category", "confidence"]] = pd.DataFrame(
[cache[d] for d in descriptions], index=df.index
)
return df, failedAgainst the 2,000 ticket test file, this produces 2 requests instead of 2,000. If one batch fails, classify_group retries each ticket in that batch individually rather than marking the entire group as failed.
Step 7: Flag low-confidence results for human review
CONFIDENCE_THRESHOLD = 0.9
df["category"] = df["predicted_category"]
low = df["confidence"] < CONFIDENCE_THRESHOLD
df.loc[low, "category"] = "Needs Review"
print(f"{low.sum()} of {len(df)} flagged for a human.")
df.to_csv("tickets_categorized.csv", index=False)The output file keeps two columns: predicted_category holds Claude’s original guess, and category holds the final label. When confidence falls below 0.9, only category changes to “Needs Review”. The original guess stays intact so you can audit it later.
The threshold is set at 0.9 on purpose. Sending a few extra rows to a human costs nothing. A confidently wrong label in your final report costs considerably more. Run it, see what gets flagged, then adjust the threshold to match your data.
Step 8: Make it production-ready
The script above works, but the filename is hard-coded and loading a large CSV all at once will exhaust memory. Four changes close the gap between demo and real tool.
Accept CLI arguments instead of hard-coding filenames
Use Python’s argparse to pass the input file, output file, worker count, and group size on the command line. The same script then runs on any file without editing source code. Arguments: --input, --output, --workers, --group-size.
Catch duplicates across chunks, not just within them
Do a fast first pass reading only the ticket_id column before loading any descriptions. This identifies every ID that repeats anywhere in the file so deduplication works even when duplicate rows land in different chunks:
id_col = pd.read_csv(args.input, usecols=["ticket_id"])
dupe_ids = set(id_col[id_col.duplicated(keep="first")]["ticket_id"])
kept = set()
# inside the chunk loop:
before = len(chunk)
is_dupe = chunk["ticket_id"].isin(dupe_ids)
already_kept = chunk["ticket_id"].isin(kept)
chunk = chunk[~(is_dupe & already_kept)]
kept.update(chunk.loc[chunk["ticket_id"].isin(dupe_ids), "ticket_id"])
dropped = before - len(chunk)Process in chunks of 500 rows
Use pd.read_csv(chunksize=500) and write each chunk’s results to the output file immediately. If the script crashes at row 40,000, the first 39,999 rows are already saved.
Let the SDK handle rate-limit retries
Pass max_retries when initializing the Anthropic client. Transient rate-limit pauses are retried automatically rather than being marked as failures and labeled “Needs Review.”
The complete script
Save this as task.py with your .env and tickets.csv in the same folder, then run python task.py --input tickets.csv:
import argparse, os, time
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, as_completed
import pandas as pd
from anthropic import Anthropic
from dotenv import load_dotenv
load_dotenv()
API_KEY = os.getenv("ANTHROPIC_API_KEY")
if not API_KEY:
raise RuntimeError("ANTHROPIC_API_KEY is not set. Check your .env file.")
CATEGORIES = ["Billing", "Login Issue", "Bug Report", "Feature Request", "Other"]
MODEL = "claude-haiku-4-5-20251001"
CONFIDENCE_THRESHOLD = 0.9
REQUIRED_COLUMNS = {"ticket_id", "description"}
SYSTEM_PROMPT = (
"You are a classifier for support tickets. You will receive a numbered "
"list of tickets. Classify each ticket into exactly one of these "
f"categories: {', '.join(CATEGORIES)}. "
"Respond with one line per ticket in the form "
"'<number>. <category>, <confidence>', where confidence is a score from "
"0 to 1. Match the input numbering exactly. No other text."
)
def categorize_tickets(client, descriptions):
ticket_list = "n".join(f'{i}. "{d}"' for i, d in enumerate(descriptions, start=1))
response = client.messages.create(
model=MODEL,
max_tokens=25 * len(descriptions),
temperature=0,
system=SYSTEM_PROMPT,
messages=[{"role": "user", "content": f"Tickets:n{ticket_list}"}],
)
lines = [l.strip() for l in response.content[0].text.strip().splitlines() if l.strip()]
if len(lines) != len(descriptions):
raise ValueError(f"Asked about {len(descriptions)}, got {len(lines)} back")
results = []
for i, line in enumerate(lines, start=1):
number, dot, rest = line.partition(".")
if not dot or number.strip() != str(i):
raise ValueError(f"Line {i} is misnumbered: {line!r}")
category, comma, confidence = rest.partition(",")
if not comma:
raise ValueError(f"Line {i} has no confidence score: {line!r}")
category = category.strip()
confidence = float(confidence.strip())
if category not in CATEGORIES:
raise ValueError(f"Made-up category: {category!r}")
if not 0.0 <= confidence <= 1.0:
raise ValueError(f"Confidence out of range: {confidence!r}")
results.append((category, confidence))
return results
def classify_group(client, descriptions):
try:
return categorize_tickets(client, descriptions), 0
except Exception as exc:
print(f" Group of {len(descriptions)} failed ({exc}), retrying one by one")
results, failed = [], 0
for d in descriptions:
try:
results.append(categorize_tickets(client, [d])[0])
except Exception as exc:
print(f" Ticket {d[:40]!r} failed: {exc}")
results.append(("Needs Review", 0.0))
failed += 1
return results, failed
def classify_dataframe(df, client, cache, max_workers=5, group_size=20):
descriptions = df["description"].tolist()
unseen = [d for d in dict.fromkeys(descriptions) if d not in cache]
groups = [unseen[i:i + group_size] for i in range(0, len(unseen), group_size)]
print(f" {len(df)} rows, {len(unseen)} new sentence(s) = {len(groups)} request(s)")
failed = 0
with ThreadPoolExecutor(max_workers=max_workers) as pool:
futures = {pool.submit(classify_group, client, g): g for g in groups}
for future in as_completed(futures):
group = futures[future]
results, group_failed = future.result()
cache.update(zip(group, results))
failed += group_failed
df[["predicted_category", "confidence"]] = pd.DataFrame(
[cache[d] for d in descriptions], index=df.index
)
return df, failed
def find_duplicate_ids(path):
id_col = pd.read_csv(path, usecols=["ticket_id"])
return set(id_col[id_col.duplicated(keep="first")]["ticket_id"])
def load_chunks(path, chunk_size):
for i, chunk in enumerate(pd.read_csv(path, chunksize=chunk_size)):
if i == 0:
missing = REQUIRED_COLUMNS - set(chunk.columns)
if missing:
raise ValueError(f"CSV is missing column(s): {', '.join(sorted(missing))}")
yield chunk.reset_index(drop=True)
def parse_args():
p = argparse.ArgumentParser(description="Sort support tickets with Claude.")
p.add_argument("--input", default="tickets.csv")
p.add_argument("--output", default="tickets_categorized.csv")
p.add_argument("--workers", type=int, default=5)
p.add_argument("--group-size", type=int, default=20)
p.add_argument("--chunk-size", type=int, default=500)
p.add_argument("--max-retries", type=int, default=2)
return p.parse_args()
def main():
args = parse_args()
client = Anthropic(api_key=API_KEY, max_retries=args.max_retries)
dupe_ids = find_duplicate_ids(args.input)
kept_dupe_ids = set()
start = time.time()
cache = {}
counts = Counter()
total = dropped = flagged = failed = 0
first = True
for n, chunk in enumerate(load_chunks(args.input, args.chunk_size), start=1):
print(f"--- Chunk {n} ({len(chunk)} rows) ---")
chunk["description"] = chunk["description"].str.strip()
before = len(chunk)
chunk = chunk.drop_duplicates(subset="ticket_id").reset_index(drop=True)
is_dupe = chunk["ticket_id"].isin(dupe_ids)
already_kept = chunk["ticket_id"].isin(kept_dupe_ids)
chunk = chunk[~(is_dupe & already_kept)].reset_index(drop=True)
kept_dupe_ids.update(chunk.loc[chunk["ticket_id"].isin(dupe_ids), "ticket_id"])
dropped += before - len(chunk)
if len(chunk) == 0:
print(f" 0 rows left after dedup, skipping this chunk")
continue
chunk, chunk_failed = classify_dataframe(
chunk, client, cache,
max_workers=args.workers, group_size=args.group_size,
)
chunk["category"] = chunk["predicted_category"]
low = chunk["confidence"] < CONFIDENCE_THRESHOLD
chunk.loc[low, "category"] = "Needs Review"
chunk.to_csv(args.output, mode="w" if first else "a",
header=first, index=False)
first = False
counts.update(chunk["category"])
total += len(chunk)
flagged += int(low.sum())
failed += chunk_failed
print(f"nDone in {time.time() - start:.1f}s.")
print(f"{total} tickets, {dropped} duplicate(s) dropped, "
f"{flagged} flagged for review ({failed} of those were errors).")
print(f"Only {len(cache)} unique sentences were ever sent to Claude.")
for category, count in counts.most_common():
print(f" {category}: {count}")
if __name__ == "__main__":
main()The final run processes 2,004 rows, drops 4 duplicates, classifies via 21 unique sentences sent to Claude, and completes in about 2 seconds. The results file includes a “Needs Review” filter for anything below the 0.9 confidence threshold. Tickets like “Not sure this is the right place, but the site feels slow today” land there by design: too vague for confident classification, exactly what a human should read.
⚠️ Common pitfalls
- One API call per row. Without batching and caching, 2,004 rows means 2,004 requests to classify 21 sentences. The grouping and caching in Steps 5 and 6 collapse this to 2 requests.
- Skipping response validation. Claude can drop a line or return a label that isn’t in your category list. Without the parsing checks in
categorize_tickets, a wrong label looks identical to a correct one in your output CSV. - Dropping the confidence threshold. Without it, tickets Claude doubted are indistinguishable from tickets Claude was certain about. The only signal pointing you to rows worth checking disappears.
- Using a larger model than the task needs. Haiku classifies these sentences accurately. Opus does too, but at considerably higher cost for the same result.
- Raising worker count past your rate limit. More workers don’t mean more speed if you exceed your API tier’s limit. You get throttling, not throughput.
When one script isn’t enough
This script handles classification from a file. When you outgrow it, here’s what to reach for next:
- Claude Agent SDK: when you want the agent to take actions (read files, run tools, write changes) instead of returning answers for you to handle.
- Model Context Protocol (MCP): when tickets live in a helpdesk or database instead of a CSV. MCP lets Claude query the source directly without a manual export step.
- Message Batches API: for large volumes where turnaround time isn’t urgent. The developer notes it costs roughly half as much as synchronous requests, in exchange for delayed results. This is a separate feature from the request-grouping done in Step 5, though both are sometimes called batching.
- LangChain: when a single prompt isn’t enough and you need to chain multiple steps or pull in additional context before classification.
- n8n: when the person maintaining the workflow doesn’t know Python. It has a visual drag-and-drop interface with a built-in Claude node.
I’m founder of BizStack at Cagri Sarigoz LLC and a passionate advocate for entrepreneurs.
With over 14 years in tech, marketing, and AI, including my role as Head of SEO at CitizenShipper and co-founder of TaleBot at Intale AI, I’m dedicated to sharing genuine, useful product insights and tips.
At BizStack, I aim to cut through the digital noise to provide clear, actionable advice.
And more than all else, I’m a father to a (always) little girl and a husband.
Contact me at [email protected] for assistance.
