20 of 30 bug reports never reach a developer
This file covers an AI assistant that asks the right questions.
READ IN 8 SECONDS
- PROBLEM
- About 30 reports a day, and most were questions rather than bugs.
- DECISION
- An LLM checks each report against the system's data models and the code in git, then classifies it as a question, bug or idea.
- RESULT
- About 20 of 30 handled without a developer. Repeat escalations fell 30% in four months.
END OF SUMMARY. SCROLL FOR DETAILS ↓
9 slides · use ← → to navigate · swipe on mobile
-
READ IN 8 SECONDS
- PROBLEM
- About 30 reports a day, and most were questions rather than bugs.
- DECISION
- An LLM checks each report against the system's data models and the code in git, then classifies it as a question, bug or idea.
- RESULT
- About 20 of 30 handled without a developer. Repeat escalations fell 30% in four months.
END OF SUMMARY. SCROLL FOR DETAILS ↓
Tap right for next · tap left to go back
-
We receive a lot of user feedback, and most of it is questions, not bugs. Someone still had to read each one.
-
Ask Metis now handles the first conversation, as a good support agent or BA would: what happened, what did you expect, and can you send a screenshot?
-
It then does the research: the data models, the code in git and existing tickets.
- New report
- Ask for details
- Check data models + git
- Question? Answered
- Bug or idea? Jira
About 20 of every 30 turn out to be questions, and those are answered immediately.
-
Your turn
Sometimes it is a genuine bug. What should happen next?
Choose an option before you continue.
- A. Try to handle it anywayB. Hand it to a person with everything it collected my pickC. Ask the user to wait
B. It creates the Jira ticket with the summary, screenshots and session recording attached.
Principle 01
Automate the questions, not the decisions
The AI collects and checks. People decide what to fix.
-
Developers reply in Jira. Replies tagged [external] appear in the portal, so the user gets a reply without anyone copying messages.
-
Session recordings show what the user actually clicked, which often differs from what they remember.
-
Guess the number
GUESS THE NUMBER
- YOU GUESSED
- 10
- ACTUAL
- 20
- OFF BY
- 10
Make your best estimate.
-
RECEIPT · METIS INCIDENT PORTAL
- REPORTS PER DAY
- ~30
- HANDLED BY AI
- ~20
- REPEAT ESCALATIONS
- -30% IN 4 MONTHS
- LIVE SINCE
- MAY 2026
- ALSO USED BY
- PRINT.COM
- DEVELOPER READS SAVED / DAY
- ~20
EVERY REPORT TRIAGED
About twenty a day, handled without a developer.
The receipt
Every change from this file, on one receipt.
What I'd do next
Show users the answer the assistant found for a similar ticket before they finish typing.
RECEIPT · METIS INCIDENT PORTAL
- REPORTS PER DAY
- ~30
- HANDLED BY AI
- ~20
- REPEAT ESCALATIONS
- -30% IN 4 MONTHS
- LIVE SINCE
- MAY 2026
- ALSO USED BY
- PRINT.COM
- DEVELOPER READS SAVED / DAY
- ~20
EVERY REPORT TRIAGED
