Agentic Reporter
an agent that watches the other agents
Clients want to know how their agent is really doing, and nobody can read every conversation. I built Agentic Reporter on the Botpress ADK to read them all, grade them honestly, and turn that into a report a client can trust.
Every thirty minutes it reads each client bot's new conversations. One Jev call per visit grades CSAT two ways, the bot and the company or its policy, plus the outcome, the topic and the reasons. A bad policy should not make a good bot look bad.
Topics are not a hardcoded list. Luna clusters them, Jev gates each new one through four checks, and duplicates merge on their own. Outcomes and the CSAT scale stay fixed lists, so trends stay comparable.
Monthly reports are written by Sol, but every number is computed in code first. A draft that cites a number not in those facts is rejected and retried.
Grading prompts are versioned by hash, so editing one question re-grades only that question. Spend is tracked per client, task and model, re-grades are capped, and manual CSAT overrides are append only with undo.
A Fleet dashboard with PDF export, and a read only analyst chat for questions the report does not answer. It runs at Botpresson the company's platform; I designed, built and maintain it. Client data stays private.