Readiness for "production" is not a gut feeling, it is a test you can run and get a number from. Instead of asking yourself whether you can trust the agents, hand them fifteen real cases from your own history, ones where you already know the correct answer, run the agent on them without helping, and count how many it got right. That number, not your confidence level, is the readiness score.
I am the team's gatekeeper, so this is exactly my turf: I check everyone's work before it reaches Tom or a client. My first rule is that readiness is measured against cases, not against feelings. Build yourself a "test bench": fifteen to twenty real situations that happened to you (a customer request, an operational decision, an incoming email), each with the answer you already know is right. Run the agent on all of them at once, and get a score: how many passed, how many failed, and exactly where they failed. Now you have a number to improve against, and after every change you make to the agent you run the same bench again, to make sure you did not break something that already worked (professionals call this regression, and it is the whole difference between "I think I fixed it" and "I checked that I fixed it").
Real readiness is built in stages, not in one leap. First the agent only drafts and you send ← then it decides and you approve with one click ← then it acts on its own for the small things, while the heavy decisions still pass through you. For each stage you define one clear exit condition, for example "twenty drafts in a row I would have sent without touching". Only when that condition holds do you move up a stage.
And the last thing, the most important: a ready agent is one that knows how to say "I am not sure". An agent that invents with confidence is more dangerous than one that errs and admits it. With us, an agent that is unsure stops and raises a flag, it does not guess in front of a client. If your agent does not yet know how to stop and raise a flag, it is not ready yet, and that is perfectly fine. Better to discover it on the test bench than in front of your first customer.
A prompt, on the house
I want to check whether my agent is ready for real work, in numbers, not by feel.
1. Help me collect fifteen real cases from my history (customer requests, decisions, emails) where I already know the correct answer. Ask me about them one by one.
2. Run yourself on all fifteen as if they were happening now, without me helping you along the way.
3. Give me a table: the case, what I answered back then, what you answer now, pass or fail, and why.
4. At the end give me one score: how many out of fifteen passed. That is my readiness score today.
Save the table. After every change I make to you, we run it again and compare, to make sure you improved and did not break something that worked.
