What Should You Delegate to AI First? The Five-Minute Falsification Test
What Should You Delegate to AI First? The Five-Minute Falsification Test
Say your support team has outgrown its help desk software. You sent the same questionnaire to three vendors, and the answers came back as three long documents, each laid out differently. Someone now has to line them up against what the team agreed it needs: the price per seat, the contract length, how each tool handles your shared inbox, and whether you can take your ticket history with you if you leave.
It looks like a good first job for AI. The quickest way to hand it over is one line: "Research these vendors and tell us which one is best."
That line sounds low risk, because the answer stays inside the team. But it doesn't say which criteria matter most, which sources count or what a wrong answer would look like. When the recommendation comes back, the only way to judge it is to read all three responses yourself and do the comparison again.
So don't give AI the easiest task on your list. Give it the useful task whose most important mistake is easiest to catch.
What should you delegate to AI first?
Start with work that's useful, has a clear finish line, can be undone, and can be checked against something other than the answer itself. Before it runs, name the one mistake that would make the result unusable. Then ask whether the right person could look for that mistake in five minutes without redoing the work.
We call this the Five-Minute Falsification Test. "Falsification" sounds academic, but the question behind it is plain: what error would sink this, and can someone go looking for it quickly? Five minutes isn't how long review should take. It forces the assignment to make one important failure cheap to find.
The right person knows the work and can act on what they find. For the vendor comparison, that's the support lead who wrote the questionnaire, not whoever happens to have a free moment. Done this way, the first delegation gives the lead two things: the comparison, and evidence about what to hand over next.
Why "easy and repetitive" isn't enough
Most advice about a first AI task points to work that's repetitive, low stakes or time-consuming. Those are reasonable things to look for, but none of them tells you what it will cost to judge the result. The vendor comparison is repetitive, low stakes and slow by hand, and the one-line version of it still can't be checked without doing it again.
That cost grows as the work gets longer. Microsoft Research's DELEGATE-52 benchmark ran 19 language models through long document-editing workflows across 52 professional domains. By the end of those workflows, even the frontier models had corrupted an average of 25% of the document content, and the researchers describe the errors as sparse but severe.
In a follow-up note, the same researchers said the benchmark isn't a measure of overall model capability, task success or user outcomes. It studied delegated work with limited human intervention between steps, and they noted that production systems can mitigate these effects with verification loops and other tooling. A vendor table is a far shorter job, but one price under the wrong name, rare enough to miss, can still ride all the way to the decision if nobody checks.
Time saved is hard to measure, too. In METR's 2025 randomized trial, 16 experienced open-source developers took 19% longer to finish tasks in repositories they knew well when they could use early-2025 AI tools. In a February 2026 update, METR said it's likely developers are more sped up now, but called its new data only very weak evidence for how much. It also called its newer estimate likely a bad proxy for the real effect on the developers it studied. If METR can't pin that number down, the support lead won't pin it down from one vendor table.
What a first run can tell the lead is whether the team can brief the work, check it, correct it and decide what happens next.
Five questions to ask before the first run
1. Does it have a clear finish line?
A bounded task says what to make, what to make it from and where to stop, so the person checking it doesn't have to guess what "done" means.
"Research these vendors" has no finish line. "Compare the three questionnaire responses against our four criteria, cite where each answer came from, flag anything a vendor didn't answer, and stop before recommending one" does. The second is longer, but the length isn't the point. Anyone reading it can tell when the table is done.
2. Is there something to check it against?
The answer can't be its own evidence. The work needs something outside it: supplied records, an approved brief, a known calculation, a checklist. For the vendor table, that's the three responses and the criteria the team agreed on. If checking the table meant researching the vendors all over again, you'd be doing the job twice.
NIST's AI Risk Management Framework says human judgment should be used to decide the specific metrics for AI trustworthiness and the precise thresholds for them. Choosing which mistake to look for in the vendor table, and how quickly to find it, is a small version of that judgment. It belongs to the support lead, who knows what the table is for.
3. Can you name the mistake that matters?
The mistake that matters is the one that would change the decision, and it's rarely the typo you'd spot first. Name it before the work runs.
In the vendor table, the dangerous mistake is a fact in the wrong place: one vendor's per-seat price or contract length sitting under another vendor's name, or a cell that states something no vendor said. Either could steer the team toward the wrong tool. Pick the cells that would swing the decision and trace each one back to the response it came from. One check won't catch every error, but it puts your first five minutes where the damage would be.
4. Can you undo it?
Reversible work can be stopped, corrected, thrown away or run again before it changes anything outside the team. A comparison table in a shared folder is reversible. An email telling a vendor you've chosen them isn't, and neither is a signed contract.
The table can still shape a real decision. It just stops before anyone sends, signs, pays or publishes, and the person who owns the decision makes the next move.
5. Will it tell you what to do next?
Decide before the run that it ends in one of four decisions: repeat, repair, expand or stop. Keep a short record beside the table: the brief, the responses, the cells the lead traced, how long the check took and the decision. Without it, you only have a feeling about whether the table looked good.
Other first tasks with the same shape
The vendor table turns known inputs into a useful internal document, traces every claim back to those inputs and stops before anything leaves the team. Turning approved meeting notes into a decision record works the same way, and there the mistake to look for is an owner or deadline the notes don't support. So does checking a draft against an existing checklist before anyone publishes it.
The vendor job has poor first versions, too. Smoothing the questionnaire for the next round has no clear wrong answer. A vendor pick can't be checked without redoing the comparison, so it waits.
Run one proof loop in thirty minutes
For a small first task, thirty minutes is enough: ten to write the brief, ten to run it, five to check and five to decide. A bigger or riskier task gets more time where it needs it.
The brief: ten minutes
Put your answers to the five questions into the brief, along with who owns the result. For the vendor table, it might read: "Compare the three attached vendor responses against these four criteria. Cite the page or paragraph behind every cell. Mark anything a vendor didn't answer as missing. Don't recommend a vendor. The support lead will check the price and contract cells against the responses."
If you can't say what would make the result unusable, the task isn't ready. A standard invented after the table arrives ends up judging how the table looks.
The run: ten minutes
Let it run without moving the finish line. If a question comes up, like whether one vendor's "per user" price means per seat, write it down instead of answering it mid-run, because the brief left it out. Rescue the run through chat and a weak brief ends up looking like a strong one.
The check: five minutes
Go straight to the mistake you named. Trace the price and contract cells back to each vendor's own words, make sure unanswered questions are marked as missing, and confirm no recommendation slipped in at the end. If you don't find the mistake, the table still isn't proven right. You've learned that one important mistake was cheap to look for, and the table may need more review before the team relies on it.
The decision: five minutes
End with one of the four decisions:
- Repeat runs the same work again with the same scope, autonomy and standard of evidence.
- Repair fixes one weakness you found, in the brief, the inputs or the review, then runs the same task again.
- Expand changes one thing about the work after several runs have held up.
- Stop ends or postpones the work when there's nothing to check it against, review costs more than the work is worth, or the stakes have outgrown the controls.
Suppose the check turns up something the brief never covered. The cheapest vendor only offers an annual contract, and the team asked for both a low price and a month-to-month option. The table reports both facts correctly, but it can't rank the vendors because nobody said which criterion comes first. That's a repair. Add the rule to the brief, keep everything else the same and run it again.
Stopping is a decision too. If the responses are too vague to trace a price to, there's nothing to check against, and that's cheaper to learn on run one than run five.
Expand one thing at a time
Once the same task has held up over a few runs, you can give it more. Change one thing at a time (scope, autonomy or consequence), never two together, and keep an owner and a stop point at every step. For the vendor work, that might look like this:
- One table from three supplied responses, stopping before anyone uses it.
- More scope: the same table for eight vendors, with the same criteria and the same stop.
- More autonomy: for vague answers, the work decides whether to mark them answered, partial or missing, by rules the support lead approved.
- More consequence: ask for a recommendation. The support lead still decides, tells the team and acts.
- More scope again: draft follow-up questions and emails for vendors who left gaps, and stop before anything is sent.
Any step can end in repeat, repair or stop instead of moving up. How far you climb is your call.
Where Supanova fits
Say the vendor comparison ran on Supanova. You'd hand your atoms the brief from those first ten minutes and attach the three vendor responses to the project, so the atoms can draw on the same documents your support lead will check the table against.
When the work is finished, it comes back for your review, and that's where the support lead spends the five minutes on the price and contract cells. If the annual-contract conflict shows up, the rule about which criterion comes first goes into the brief for the next run, and the lead checks the same cells again.
We want teams to be able to take on work they don't yet have the people for. For the support team, that starts with a table the lead can check in five minutes. Once it holds up, the eight-vendor version and the follow-up questions can go to the atoms too.
The first task sets the standard for the second
Whatever you accept as proof on the first delegation becomes permission for the second. If the lead signs off because the table looks tidy, tidy becomes the bar for eight vendors. If the price and contract cells had to trace to the responses, that's the bar instead.
So look at the list of things you'd like AI to take off your plate. For each one, ask what mistake would make it unusable and who could find it in five minutes. Start with the one where you have a good answer to both.
A good first delegation doesn't ask you to trust more. It lets you know more.
When you've picked yours, you can start at supanova.team/join.
Sources
- LLMs Corrupt Your Documents When You Delegate, Philippe Laban, Tobias Schnabel and Jennifer Neville, Microsoft Research (April 2026)
- Further notes on our recent research on AI delegation and long-horizon reliability, Microsoft Research blog (May 15, 2026)
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR (July 10, 2025)
- We are Changing our Developer Productivity Experiment Design, METR (February 24, 2026)
- Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (January 2023)