A multi-agent AI workflow only produces usable results if two things are in place: a written set of rules for every stage, and one coordinating agent (the orchestrator) with the power to enforce them. This post shows how we do that, using a real 16-stage study in which the orchestrator re-ran a missing analysis and threw away 36 finished interviews because they broke the rules.
Most descriptions of multi-agent work start by naming roles: a researcher, a strategist, a critic, a writer. That’s understandable — roles are easy to name and parallel activity is easy to show. But a list of agents is not a managed workflow.
What we design first is the workflow contract: an agreement, fixed before any work starts, about what must happen. It sets the order of the stages, what each stage must produce, what evidence must be carried into the next stage, and when an output is accepted, rejected or run again. The orchestrator agent then manages that contract. It hands tasks to specialist agents and personas (simulated people with defined backgrounds), checks what they return against the rules, records its decisions, and sends work back or elsewhere when the rules are not met.
The animation below is the execution trace, a step-by-step record of the run, from one contextual-targeting study. The question was where a prenatal supplement could appear next to fertility content in Italy without feeling intrusive, and which advertising approaches each placement allowed. Press play to watch the run, pause on any stage, click a bar, or use the step controls. The upper rail is the orchestrator. The lanes are specialist functions. The growing area at the bottom is the work produced; red marks work the orchestrator rejected, keeping it out of the study.
Design the workflow before choosing the agents
A workflow should be strict wherever one mistake leads to more mistakes later. In practice, that means fixing exactly what passes from one stage to the next.
For each step, we define five things:
- What the step may use. Which files and earlier decisions is this step allowed to read? An analysis agent should not quietly read a creative recommendation made later, and a re-run should not take in material that was rejected.
- What the step must hand over. The step must return something that can be checked: a segment map, panel specification, interview guide, set of transcripts, coding file, challenge memo or final report—not a claim that the task is done.
- Pass/fail checks (acceptance criteria). The output must have properties anyone can verify. Six segments means six. Line-by-line coding means every exchange is linked to its source. Three creative routes per segment cannot be three phrasings of the same idea.
- Rules that hold for the whole run. Some decisions must stay in force from start to finish. In this study, content about pregnancy loss was excluded early. It stayed excluded from respondent generation, fieldwork, placement planning and the final report.
- What happens on failure. The workflow says what to do when a check fails: retry the same task with corrected input, pass it to a different specialist, drop that line of work, or stop the run.
This is what makes running agents in parallel safe. Four agents can explore on their own because their outputs meet at a defined step where they are merged. Six simulated specialists can widen a bank of interview questions because a later step has a clear job: remove duplicates and closed (yes/no) questions, keep every topic covered, and keep any mention of the product out of the first three phases of the interview.
What the orchestrator agent actually does
An orchestrator does not simply announce that one task is finished and start the next. It holds the current state of the run and has the authority to make decisions that change its outcome.
At the first stage of this study, four independent context analyses were sent out. Three came back. The missing one was the decision-influence analysis: it covered the partner, doctor, pharmacist, family and online forums involved in the decision. The workflow did not allow the study to continue without that perspective, so the orchestrator issued the task again. The recovered analysis later produced the study’s central finding: the customer was surrounded by people with partial authority and partial information. That left a gap a brand could fill in practical terms, but it could never credibly claim to be a personal relationship.
Retrying a task is trivial in itself. What matters is that the orchestrator knew why the missing output was essential to the structure of the study and refused to quietly shrink the scope.
Later, all 36 interviews were completed. They produced 220,651 words. They also failed the checks: some respondents had seen the test advert (the stimulus) in English and others in Italian. A reaction to the language could no longer be told apart from a reaction to the placement. The output was large and unusable.
The orchestrator discarded the entire set and re-ran the fieldwork with that one variable corrected and everything else held constant. The rejected transcripts stayed outside what any later agent was allowed to read. This is one of an orchestrator’s most valuable jobs: protecting later work by refusing finished work that does not meet the contract.
AI agents and synthetic personas have different jobs
We use agents and personas for different purposes.
Agents carry out a defined method. A behaviour analyst reconstructs the actual customer journey. A sensitivity analyst grades where advertising becomes intrusive. A transcript coder works line by line. A category analyst keeps evidence that contradicts the main findings. A boundary tester pushes a creative route until it fails. Each role is defined by what it does, what it receives and what it hands over.
Personas stand in for people in specific situations. The 36 synthetic respondents in this run were not a cloud of generic “Italian women.” They were spread across six contexts, with deliberate differences between respondents inside every segment, and fixed rules for language and form of address. Personas were research instruments inside the workflow; they did not decide what the workflow should be.
Letting a persona control the method confuses simulated experience with analysis. Giving a method agent a vaguely described persona invites it to invent the population it is supposed to study. The orchestrator keeps both inside their proper boundaries.
Turning vague instructions into checks an agent can pass or fail
Vague instructions push judgement to the end of the run, when errors are hardest to trace. “Find good placements” sounds strategic but gives an orchestrator nothing to test. A version it can check looks more like this:
Select six contexts supported by more than one independent analysis, with named inventory that can actually be bought, deliberate coverage across sensitivity levels, and an explicit exclusion list carried into every later stage.
In the same way, “develop creative ideas” became four parallel creative approaches across all six segments. Each route had to include a headline, body copy, placement, a supporting interview quote and a stated risk. A separate challenger agent then tested the assumptions those creative agents had in common. Only after the challenge was on file did a selection agent choose three genuinely different routes per segment, recording for each challenge whether it was accepted, addressed, or rejected with a reason.
These checks do more than control quality. They shape the order of the work: independent work comes before merging, widening before narrowing, and challenge before selection. Every step can be traced back throughout.
Why the execution trace is part of the deliverable
The final report matters, but so does being able to explain how it came about. This run went through 16 controlled stages in 4 hours and 31 minutes. It produced 983,208 words before rejected work was removed and the findings were condensed into a 13,791-word client report.
Every final claim can be traced backwards: from a recommendation to a category, from a category to coded exchanges, and from those exchanges to a specific respondent and interview condition. Exclusions and limitations stay attached to the findings; they don’t disappear during polishing.
The animation makes this management visible. You can see tasks sent out in parallel, outputs merged, work redirected, rejected and re-run, an adversarial review (an agent whose job is to attack the work), and the final integration. It shows why the orchestrator sits above the lanes: not because it is the most capable agent involved, but because it is responsible for the contract that connects every lane.
Our rule for running multi-agent workflows
Define the workflow strictly. Give every stage a concrete output and a pass/fail check. Keep method agents separate from personas who stand in for real situations. Give the orchestrator enough authority to inspect, reject, redirect and stop. Keep a source trail for what survives and a decision trail for what does not.
Then let the agents be creative inside the boundaries that make their work usable.