PREDICTIVE TREND INSIGHT
Can I configure a bot to automatically categorize my incoming emails? Illustration

Can I configure a bot to automatically categorize my incoming emails?

Direct Summary:

Yes — categorizing incoming emails is one of the more reliable LLM use cases, because it's a classification problem with a fixed, known set of categories, which plays to a language model's strength (assigning the closest matching label from options you specify) rather than its weakness (generating novel facts). The practical pattern: fetch new emails via the Gmail or Outlook API, classify each one with a structured-output LLM call constrained to your exact category list, then apply the resulting label back through the same API.

"Quality is not an act, it is a habit."

— Aristotle

Key Insights

  • Constrain the output to your exact category set: using structured output (a JSON schema or enum constraint) rather than free-text classification prevents the model from inventing a category that doesn't exist in your labeling system.
  • Classification is inherently more reliable than open-ended generation: picking the best-fit option from a fixed list is a narrower, easier task for an LLM than generating novel, factually-correct content — expect higher accuracy here than on open-ended drafting tasks.
  • Low-confidence cases need a fallback category: include an explicit "uncategorized/needs review" option in your category list so ambiguous emails get flagged for a human rather than forced into the nearest wrong bucket.

Email categorization is one of the safer, more accurate applications of an LLM in a business workflow, precisely because it's a narrow decision (pick one label from a fixed set) rather than open-ended generation. The reliability comes from constraining the model's output — giving it a specific, closed list of categories rather than letting it invent labels freely.

Building the categorization pipeline

1. Fetch new emails via the mailbox provider's API. Gmail and Outlook both expose APIs to read new messages and apply labels/folders programmatically — this is the integration point, not a scraping or forwarding hack.

2. Classify with a structured-output call constrained to your exact categories. Pass the email subject/body and ask the model to return one value from an explicit enum (e.g., "billing", "technical_support", "sales_inquiry", "spam", "needs_review") — constraining the schema prevents invented categories.

3. Apply the resulting label back through the same API, and route low-confidence cases to review. Include a "needs_review" fallback category so ambiguous emails get a human look instead of a forced, possibly-wrong classification.

email_categorizer.py
# pip install anthropic
import anthropic

client = anthropic.Anthropic()
CATEGORIES = ["billing", "technical_support", "sales_inquiry", "spam", "needs_review"]

def categorize_email(subject: str, body: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=20,
        system=f"Classify the email into exactly one of: {', '.join(CATEGORIES)}. "
               f"Reply with only the category name.",
        messages=[{"role": "user", "content": f"Subject: {subject}\n\n{body}"}]
    )
    category = response.content[0].text.strip()
    return category if category in CATEGORIES else "needs_review"
Approach Reliability
Free-text classification, no constraint on output Lower — model may invent labels that don't match your system
Structured output constrained to a fixed category list Higher — output is guaranteed to be one of your actual categories, or falls back to review

The reason this use case works well while others (like autonomous email replies) need more caution is the nature of the task: classification into a known set of options is exactly the kind of narrow, bounded decision LLMs handle reliably, whereas open-ended content generation carries more risk of subtle errors going unnoticed.

Practical Challenge

Build the categorize_email function above with your own category list, run it against 20 real (or sample) emails, and manually check the accuracy — note how many landed in "needs_review."

Concept Check

Why is constraining an email classifier's output to a fixed category list more reliable than free-text classification?
Correct! A closed category list guarantees the output maps to a real label in your system, with a defined fallback for ambiguous cases.
Incorrect. Try again! The benefit is correctness/consistency of the label, not raw speed — and free-text classification is possible, just less reliable for downstream automation.

Sources & Further Reading

Previous Guide Dashboard Next Guide