Email triage agents.

Your inbox is where AI agents earn their keep faster than anywhere else.

Ten emails a day that could be categorized in two seconds each. Two minutes. Every day. That's twelve hours a year on one task that doesn't require your judgment, just your attention.

This is what a triage agent handles. Read inbound message. Classify it. Surface the urgent ones. Draft a reply for the routine ones. Leave the ambiguous ones for you.

By the end of 90 minutes:

• A triage agent that reads a customer message and returns a category, confidence, summary, and draft reply.
• Rules for when the agent should escalate versus auto-handle.
• An eval suite (from Module 006) covering the realistic range of inbox messages.
• A framework for expanding this pattern to any structured inbox (support, sales, ops).

The triage prompt you drafted in Module 002 is the foundation. Today we extend it with real email context (subject, sender, thread history) and wire it for review-before-send safety.

Prereq: Module 005 and Module 006. You need prompt patterns and evals to build this safely.

Thinker on why email is the perfect first workflow agent.

Email is the perfect first workflow agent.

Four reasons.

Reason 1. Email is structured but not rigid

Every email has a subject, a sender, a body, a thread. That's structure. Within that structure, the content is free-form natural language. That's exactly what LLMs are good at: reading structure and understanding prose.

Compare to a spreadsheet (too structured, not much for the LLM to do) or a voice call (not structured at all, transcription adds a whole new failure layer). Email sits in the sweet spot.

Reason 2. The categories are stable

Your support inbox has maybe 4-8 real categories. Bug reports, billing questions, feature requests, how-to questions, cancellations, others. Once you know the shape, it doesn't change much.

Stable categories mean your triage agent can be well-defined. Same four categories this month, same four next month. The eval suite you write once keeps paying off.

Reason 3. Failures are recoverable

Agent miscategorizes a bug as a feature request. Cost: you read the email, recategorize. Ten seconds. The cost of agent failures on email is one of the lowest in the whole agent-deployment landscape.

Compare to an agent that sends emails on your behalf (high-stakes), modifies production data (very high-stakes), or makes financial decisions (extremely high-stakes). Triage is read-only until you decide to let it write.

Reason 4. The ROI is immediate

Ten emails a day × 30 seconds saved per email × 250 working days = 21 hours a year. From an agent you can build in a weekend. The math works on the first agent, and it works better every time you build another one.

The triage agent loop

What the agent does on every incoming message:

The "draft, never send" principle

Your first triage agent should never send email. Never. It drafts. A human reviews. The human sends.

This sounds conservative. It is. It's also how you ship with confidence in week one. Auto-send is a feature you graduate to after months of eval data shows the agent is performing at a quality bar you trust for direct customer communication.

Even mature triage systems often stay draft-only. The marginal time saved by auto-send over human-approved-send is tiny. The marginal risk is enormous.

Talker shows the triage prompt.

The triage prompt is an extension of the Module 002 pattern. Identity, capability, constraints, format, escape hatch, plus one new pattern for this module: the confidence threshold rule.

The full triage prompt

# Identity
You are a support-triage agent for Example SaaS, a B2B analytics
product. You read a single inbound customer email and produce
structured output for the support team.

# Capability
You have access to the current email (subject, body, sender) and
optionally a short thread history. You return JSON with a category,
a confidence score, a summary, and (for common categories) a
draft response.

# Constraints
Rules:
- Always return valid JSON with exactly these keys: category,
  confidence, summary, draft_response, escalate.
- Category must be one of: "bug", "billing", "feature_request",
  "how_to", "cancellation", "other".
- Confidence must be a number between 0 and 1.
- Summary must be one sentence under 25 words.
- draft_response must be between 50 and 200 words, or null.
- escalate must be true if confidence < 0.7 OR category is
  "billing" OR category is "cancellation".

# Format
{
  "category": "bug" | "billing" | "feature_request" | "how_to" | "cancellation" | "other",
  "confidence": 0.0-1.0,
  "summary": "",
  "draft_response": "<50-200 words, or null>",
  "escalate": true | false
}

# Escape hatch
If the email is unparseable, incomplete, or clearly spam, set
category to "other", confidence to 0.2, summary to a brief
description, draft_response to null, and escalate to true.

The confidence threshold pattern

New pattern, not from Module 005's seven. This one pops up specifically for classification tasks.

The LLM doesn't inherently return calibrated confidence scores. You have to teach it what confidence means in your context:

Confidence scoring guide:
- 0.9-1.0: The email unambiguously matches one category.
  Example: "the export button returns 500 error" = bug, 0.95.
- 0.7-0.9: Likely match, but some ambiguity possible.
- 0.5-0.7: Uncertain. Multiple categories plausible.
- Below 0.5: Very uncertain. Escalate.

This calibration guide goes in the system prompt. The LLM uses it to produce scores you can actually threshold on.

The escalation rule

Explicit about when the human should get involved:

Always escalate (escalate=true) if:
- Confidence is below 0.7.
- Category is "billing" (affects money).
- Category is "cancellation" (affects retention).
- The email mentions legal, compliance, security, or privacy.
- The email expresses significant anger or frustration.

Five escalation triggers. Each one exists because in those cases, a human should see the email before a response goes out. This list grows over time as you discover new "oh, we should have escalated that" cases.

Separating classification from drafting

The prompt above does both in one call. That works. For higher-quality drafting, you can split:

• Call 1: classify only. Return category, confidence, summary, escalate.
• Call 2: if not escalated and draft-appropriate, draft the response with a category-specific prompt.

Two calls cost more but produce better drafts because each call has a focused prompt. Most production triage systems split. For your first version, keep them combined.

bug: User reports something broken or behaving unexpectedly.
billing: User has a question about charges, invoices, or subscriptions.
feature_request: User is asking for a capability that doesn't exist yet.
how_to: User doesn't know how to accomplish a task.
cancellation: User wants to downgrade or cancel.
other: Anything that doesn't fit above.

Rememberer on what memory the triage agent needs.

Three kinds of memory matter for triage. Each lives somewhere different.

Memory 1. The current email thread

If this is message 5 in a thread, the previous 4 messages are context. The classification might change based on thread history ("this is the fifth time they've reported this bug" changes urgency).

Where it lives: in the inbound email payload. Your email client or API gives you thread history. Include the last 3-5 messages in the agent's context, summarized if long.

Fresh?: always. Every call pulls current thread state.

Memory 2. The customer context

Who is this customer? What plan are they on? When did they sign up? What's their recent support history?

This is persistent memory (Module 004, layer 3). Lives in your CRM or support system. Gets queried by customer email address at the moment of triage.

Where it lives: your CRM/database. Queried by email address or customer ID. Pulled in as a context block in the prompt.

What to include: plan tier, signup date, open ticket count, last contact date, recent resolved issues. Keep it compact: ~10 fields, ~200 tokens.

Memory 3. The category rules and examples

Your team's historical decisions about how to categorize tricky emails. "Emails mentioning X usually go to category Y." These accumulate over time.

Where it lives: in the system prompt, either as rules or as few-shot examples. Lives in version control.

Review cycle: monthly. When the team finds a consistently miscategorized pattern, add it to the prompt.

The retrieval pattern for triage

Most triage systems follow this order:

1. Email arrives.
2. System looks up customer context by sender email (Memory 2).
3. System pulls thread history from email (Memory 1).
4. System calls the triage agent with the system prompt (containing Memory 3 rules), customer context, thread history, and current email.
5. Agent returns classification + draft.
6. Human reviews if escalated; otherwise auto-routes.

Steps 2 and 3 are fast (database lookups, email API calls). Step 4 is the LLM call. Total time per triage: usually under 5 seconds.

What NOT to include in memory

Temptation: shove everything into context. Resist.

• Full product documentation. Too big. Use retrieval if needed.
• Historical email archive. Maybe the last 3-5 messages in this thread. Not all emails ever.
• Internal team messages. Different data class. Keep separate.
• Every customer's detail. Only the sender's.

Triage prompts balloon fast if you're not disciplined. Every field you add to customer context, every extra message of thread history, every new category rule, is tokens on every call. Keep the payload under 2000 tokens if possible.

Doer. Build the agent.

Time to build a triage agent. We'll use Claude Code with a sample of synthetic emails, not a real inbox, for safety. The patterns apply to any real deployment.

mkdir -p /tmp/inbox
cat > /tmp/inbox/001.md << 'EOF'
From: user@example.com
Subject: Export button broken
Plan: Pro

Hey, the CSV export button on the dashboard doesn't work.
I've tried Chrome and Safari. Just spins forever.
EOF

cat > /tmp/inbox/002.md << 'EOF'
From: finance@bigco.com
Subject: Invoice question
Plan: Enterprise

I see a charge for $499 on my October invoice but our plan
is $399. Can you clarify?
EOF

cat > /tmp/inbox/003.md << 'EOF'
From: newuser@startup.io
Subject: How do I add team members?
Plan: Pro

Hey! Just signed up. How do I invite my teammates?
EOF

cat > /tmp/inbox/004.md << 'EOF'
From: angry@customer.com
Subject: CANCEL MY ACCOUNT
Plan: Pro

This is the third time something has broken. I'm done.
Please cancel my account immediately.
EOF

cat > /tmp/inbox/005.md << 'EOF'
From: weird@spam.net
Subject: Urgent business opportunity

Greetings friend I have $10M waiting for you...
EOF

---
name: triage
description: Reads a support email and returns structured JSON
  with category, confidence, summary, draft response, and
  escalation flag. Use this for any inbound customer message.
tools: Read
---

You are a support-triage agent for Example SaaS.

Return JSON with: category, confidence, summary, draft_response, escalate.

Categories: bug | billing | feature_request | how_to | cancellation | other.

Rules:
- confidence is 0.0-1.0.
- summary is one sentence under 25 words.
- draft_response is 50-200 words, or null if escalating.
- escalate=true if confidence < 0.7, OR category is billing or
  cancellation, OR the email mentions legal/compliance/security,
  OR the tone is significantly angry.

If the email is spam or unparseable, set category=other,
confidence=0.2, summary describes the input, draft_response=null,
escalate=true.

Output only the JSON. No preamble. No markdown fences.

Use the triage agent to classify each email in /tmp/inbox/.
Output the JSON for each, in order 001 through 005.

- If the email contains "cancel," "cancelling," "shut down my
  account," or similar phrases, category is "cancellation"
  regardless of other signals.

What you built

A working triage agent. Handles real-shaped email inputs. Classifies into known categories. Escalates when appropriate. Drafts only when safe.

Next step (not in this module): wire it to a real email system. Gmail API, IMAP, or your support-desk webhook. The agent logic is done; the integration is engineering work that varies by your stack.

Rookie has the three ways this goes wrong in production.

Three ways email triage goes wrong in week one.

Failure 1. Agent sends without review

Somewhere between "draft for review" and "send automatically" is a cliff. One day someone wires the agent to an outbound queue without the review step. The next day customers start getting strange replies.

Root cause: the agent was designed to draft, but the integration layer wasn't designed to enforce review.

Fix: the review gate is code, not a convention. The drafted email sits in a drafts folder or a review queue. The human has to click "send." There's no path from agent output to customer inbox without human involvement. This is engineering discipline, not agent prompt discipline.

Failure 2. Miscategorization cascades

The agent mislabels one bug report as a feature request. The support team ignores it for two weeks because it's in the feature backlog. The customer escalates. By the time someone realizes, there's a broken export button affecting 20 customers.

Root cause: miscategorization is low-stakes per email, but systematic patterns aren't.

Fix two things:

• Confidence threshold escalation. Below 0.7, human reviews before any routing happens.
• Weekly review of low-confidence cases. A team rotation opens the "low confidence / escalated" bucket every week and checks for patterns. Patterns become new prompt rules.

Failure 3. Context explosion from long threads

The agent works great for first-message triage. Someone replies to a thread that's six messages deep. The agent's context doubles. Cost per triage doubles. Latency doubles. Soon you're paying serious money per email.

Root cause: naive inclusion of full thread history.

Fix: summarize thread history before injecting. The last message plus a one-paragraph summary of the thread so far is usually enough. For triage, the most recent message matters most.

The production readiness checklist

Before connecting the triage agent to a real inbox:

• Eval suite with 15-20 cases covering happy path, edge cases, and adversarial.
• Review gate enforced in code (no path from agent to sent email without human click).
• Low-confidence escalation wired up (cases under 0.7 go to a human review queue).
• Cost monitoring per triage call.
• A weekly review process for the agent's outputs.

Without these, you're not ready for real customer emails. With them, you are.

Manager takes the team view.

Email triage is one of the first agents a team deploys. It changes the shape of support work in ways that matter to manage through.

The new division of labor

Before the agent:

• Support rep reads every email.
• Support rep categorizes every email.
• Support rep drafts every reply.
• Support rep sends every reply.

After the agent:

• Agent reads every email.
• Agent categorizes every email (human confirms on low-confidence cases).
• Agent drafts routine replies.
• Support rep reviews and sends every reply.

The rep's time shifts from typing to reviewing. Review takes 20-30 percent of the time that typing did. That's where the capacity gain comes from.

The review cadence

Three review rituals that make the agent work at team scale:

• Per-email review: every drafted reply gets eyes before sending. Always.
• Weekly sample review: a rotating rep pulls 20 random agent outputs from the week and grades them. Builds the eval dataset for next week's prompt tuning.
• Monthly category review: the team looks at categorization accuracy. Patterns of miscategorization become new prompt rules.

These aren't optional. They're the operating model for agent-assisted inbox work.

Who owns the prompt

The prompt is owned by one person on the support team. Not by engineering, not by the "AI team," not by committee. One support lead who knows the real work.

Engineering helps with integration, deployment, monitoring. The prompt itself is subject-matter knowledge: what makes a bug report actually a bug report, how to tell billing from pricing questions, what tone to use with enterprise customers. Only support knows that.

The support lead reviews and approves every prompt change. Engineering ships the change.

Handling escalations

Escalated emails need a queue and an owner. A simple setup:

• Auto-escalated emails go to a "review-first" inbox.
• A designated reviewer processes that inbox first every morning.
• Escalation reasons are logged (why did this escalate?) for later pattern analysis.

Without a clear owner and a clear queue, escalated emails fall between the agent (which decided not to handle them) and the team (which isn't sure what's in the queue). They languish. Customers notice.

What success looks like

Three metrics for a team triage deployment:

• Time-to-first-response: should drop significantly. This is the big one customers feel.
• Categorization accuracy: tracked via the weekly sample review. Should be above 90 percent after month two.
• Rep capacity reclaimed: hours per week the team has gained. Usually 20-40 percent.

Track all three from day one. You'll need them when someone asks "is this agent actually working."

Chief handles the risk frame.

Email is your company's voice to customers. An agent in that loop raises real questions.

Risk 1. The brand voice problem

An agent drafts reply to a customer. The reply sounds fine in isolation. But it doesn't sound like your brand. Maybe it's too formal. Too chipper. Missing the specific empathy your company is known for.

Across thousands of replies, this compounds. Customers notice a subtle flattening of the relationship. They may not articulate it, but retention data sometimes does.

Mitigation: Voice matters more than speed for customer communication. Invest in a voice document (Module 013 covers this). The triage agent's drafting prompt includes voice guidance. Human reviewers are trained to reject drafts that don't sound right.

Risk 2. The data exposure surface

Every email gets sent to the LLM provider for classification. Every one. If your customers send sensitive information (health, finance, legal, PII), that information is now in a third-party's infrastructure.

This might be fine. It might not. Depends on:

• Your Data Processing Agreement with the LLM provider.
• Your regulatory environment (HIPAA, PCI, GDPR, etc.).
• Customer expectations about where their data goes.

Review your DPA. Check your compliance posture. If you're in a regulated industry, the legal team has an opinion on this, and they should give it before you ship, not after.

Risk 3. The cascading-wrong-reply scenario

The agent drafts. The human reviews. But busy humans under time pressure approve drafts quickly. Sometimes too quickly. A subtly wrong draft gets sent.

Now you have a customer who was told the wrong thing by what looks like an authoritative company reply. Resolving it takes more work than writing the original would have.

Mitigation: review discipline. Support reps should be trained that review doesn't mean "quick scan." It means "read as if you'd written it yourself." Leadership should measure review-rejection rates (how often does the human reject the agent's draft) to ensure the humans are actually engaging with the content, not rubber-stamping.

The three governance items

For any team running a triage agent at scale:

1. Human-review gate for outbound. Non-negotiable. Documented in policy.
2. Data handling documentation. What customer data flows to the LLM provider. What the agreement says. Available on request.
3. Monthly quality review. Read 20 random agent drafts. Rate them. Track the rate over time.

When NOT to deploy a triage agent

Three contexts where you shouldn't:

• Regulated customer communication. Medical, legal, financial advice should not be drafted by agents unless you have specific legal sign-off and compliance infrastructure.
• Very low volume. Ten emails a week doesn't justify the agent plus review overhead.
• Very high stakes per email. If every customer email is a potential 6-figure deal, the speed gain is not worth even a small quality drop.

For most SaaS support, B2C support, internal team inboxes, the ROI is strongly positive. But not every inbox is the same.

Founder wraps it.

Solo operators typically get the biggest percentage win from a triage agent.

You don't have a support team. You're the entire chain: read, categorize, draft, send. An agent that does the first three leaves you with only the last. Orders of magnitude faster.

Your personal triage

Three categories for a solo inbox:

• Respond today: actual customer issues, time-sensitive asks.
• Respond this week: feature requests, how-to questions, casual inquiries.
• Reference only: newsletters, notifications, automated messages.

Train the agent on this three-category distinction. Your inbox becomes three short lists instead of one long list. The daily cognitive load drops dramatically.

Draft-to-send time

For the "respond today" bucket, the agent drafts. You spend a minute per email reviewing and hitting send. A task that used to take 30 minutes takes 5.

The time savings aren't the biggest win. The biggest win is not feeling the inbox as a weight. You open it, see what needs real attention, and close it.

Start narrow

Don't start with your whole inbox. Start with one category first. "Classify everything as either 'needs human' or 'reference only.'" That's it. Two categories.

Run that for a week. Get comfortable. Then add category 3. Add drafting for the simplest category first.

Widening the agent's scope one step at a time is how you build trust. All at once is how you get burned on day three and quit.

The weekly recalibration

Once a week, look at what the agent got wrong. Not the edge cases (those will always exist). The patterns.

• It's consistently missing a category of email. Tighten the category definition.
• Its drafts are consistently too formal. Adjust the voice guidance.
• It's flagging too much for your review. The confidence threshold is too high; you trust it more than you thought.

Twenty minutes. Every week. Your agent gets better. Your inbox stays manageable.