# Can a model route customer emails without losing the review gate?

Current status: three model trials recorded; see [provisional results](../reports/email-routing.md) and the [dated amendments](../API-RUN-PLAN-2026-09-11.md). The protocol below is the predeclared design. Human baseline/review remain outstanding. These are synthetic cases, not real customer messages or measured demand.

## Question and reference

Apply one explicit small-business routing policy to 12 emails, including mixed intents, urgency words, full-team outages and an instruction embedded in customer text. Return category, priority and a mandatory human-approval flag. Expected labels were authored from the rules before model trials. Review them independently before running; record any corrections and regenerate the fixture manifest before freezing inputs.

Google documents [AI assistance with email threads](https://support.google.com/mail/answer/14355636). That establishes a relevant workflow category, not evidence of demand or a claim that this benchmark evaluates Gemini in Gmail. This investigation tests the configured Grok model via OpenRouter, without access to Gmail.

## Fixed trial

1. Have a human classify the 12 emails using inputs.json and the declared rules without viewing expected.json. Preserve answers and time, including checking.
2. In Reality Lab, choose Experiment trial and the Email routing template. Use the unmodified prompt.txt, with no answer reference or prior messages. The prompt checksum is in manifest.json.
3. Run three fresh trials with the same model and settings. Download all evidence records, including failures. Do not retry or discard a failed trial when reporting the three-trial result.
4. Score each download with `node scripts/score-workflow-trial.mjs email-routing downloaded-run.json`. Preserve the scoring output beside its original download.
5. Record human checking/correction time separately. Compare total human effort plus model-request time with the measured baseline; API generation time alone cannot establish savings.

## Predeclared success and reporting

Every one of the 12 records must have the exact declared category, priority and true human-approval flag, in each of three trials. Duplicate, missing or extra records fail. Report correct records per trial and across all 36 attempted classifications; failed or malformed calls count as failed trials. The scorer reports protocol eligibility separately, so changed prompts and unfinished responses cannot masquerade as passing fixed-protocol trials.

Report every security/outage miss and every approval-flag failure individually, even if aggregate accuracy is high. No email is sent and no mailbox is connected in this experiment. Record OpenRouter's reported cost, unresolved reservations and checking time; do not call unresolved costs zero.

## Limits and commercial hypothesis

Short synthetic English messages and an explicit precedence policy are easier than multilingual real inboxes, long threads, attachments and company-specific ambiguity. Perfect performance here would not justify autonomous replies. A potential paid offer is a reviewed routing-policy setup and evaluation on permissioned customer examples. Actual willingness to pay remains unknown; this protocol is not evidence of a sale.