I managed to write a configuration for two-agent reviews. I also managed to write one where one agent's feedback is silently discarded. ptygrid v0.5.7

I wrote it. However, halfway through, I started worrying more about whether it "could be written" rather than whether it would work. There are definitely forms that pass the configuration check but do not behave as intended. That was my takeaway for today.

Two agents sprout from a single root and merge at the end.

The setup itself can be written straightforwardly using `pattern: supervisor`. ptygrid's workflows include four types: pipeline, fan-out, supervisor, and handoff. Only supervisor allows you to write the "multiple children sprout from one root and merge at the end" structure as is.

cross-model-review:
  pattern: supervisor
  steps:
    - id: implement
      agent: implementer
      joinOn: reply
      timeoutMs: 1800000
    - id: review-a
      agent: reviewer-a
      dependsOn: [implement]
      joinOn: reply
    - id: review-b
      agent: reviewer-b
      dependsOn: [implement]
      joinOn: reply
    - id: verdict
      agent: judge
      dependsOn: [implement, review-a, review-b]
      joinOn: reply

The kickoff—the initial instruction given to the agent at each step—is long, so I omitted it, but all four are written in the actual file. `joinOn: reply` is rejected at load time if the kickoff is empty. This is because if there is no thread to reply to, that step will never complete.

You must not use fan-out here. `fanOut: 2` simply duplicates the same step twice, and both copies will use the same `agent`, meaning the same `cmd`. If you want to line up different models, you need sibling steps, not duplication. The feature with the name closest to "parallel" was the least useful for this purpose.

I attached worktrees to the two reviewers. Since they run tests simultaneously, if they shared a working tree, it would be impossible to distinguish between "review feedback" and "the partner's mess."

  - name: reviewer-a
    cwd: "."
    env:
      PTYGRID_REVIEW_DIR: "/tmp/ptygrid-cross-model-review"
    worktree:
      enabled: true
      base: HEAD
      # setup: "npm ci"
    autostart: false

The HEAD in `base: HEAD` is the HEAD of the main tree at the moment the reviewer is spawned. That is why I did not attach a worktree to the implementation agent. If I did, the changes would go into a branch with a random name, the main HEAD would not move, and both agents would end up reviewing an empty diff.

If I delete a line that looks redundant, the whole thing fails to load.

What catches the eye in the YAML above is probably the `verdict` dependency. The beginning of `dependsOn: [implement, review-a, review-b]` looks unnecessary. If review-a and review-b are finished, then implement is obviously finished as well.

I removed it and tried `[review-a, review-b]`. It told me: `supervisor step 'verdict' must dependOn root step 'implement'`. It fails at load time. There is a rule that a supervisor must have exactly one root, and everyone other than the root must include the root in their `dependsOn`. It is not redundant; it is mandatory. If I make two roots, it says `supervisor pattern requires exactly one root step (found 2)`, and if I change the same structure to `pipeline`, it says `pipeline step 'verdict' has 3 dependencies; pipeline is linear (max 1 dependsOn per step)`.

All of them fail properly, and since the error message includes the step ID, you know exactly where to fix it. Up to this point, I thought it was a kind configuration language.

As long as it failed, it was still kind.

The problem was the comparison part. There is a `handoffTo` mechanism to carry the body of the previous step to a specified step. Thinking straightforwardly, one would want to write `handoffTo: verdict` in both review-a and review-b.

You can write it. It passes the parse.

It passes, but it doesn't work. `handoff_bodies` only holds one body per target, and it discards the second one with `if bodies.contains_key(target) { continue; }`. It is implemented such that the one written first in the configuration wins. Only the review from review-a reaches the judge, and the feedback from review-b disappears without any warnings or errors. Since there is no rule on the validation side to stop this, there is no way to notice.

The verdict is returned. The run ends in green. The only thing that doesn't appear anywhere is the fact that one of the reviews did not exist.

There was no way to write "proceed if both pass" using `condition` either. Since `condition` only looks at the first dependency, the validation side imposes "exactly one `dependsOn`." If I add it to the 3-dependency verdict, it fails with `step 'verdict' condition requires exactly one dependsOn (found 3)`. This is still better because it at least fails.

The detour was also blocked. `reply_inbox` fixes the reply destination to the sender of the original message, so replies to the workflow return to a run-specific mailbox called `queen:workflow/<name>/<run_id>`. The judge cannot read it because it does not know the run ID.

In the end, for the sample, I had the review body written to a file, and used the reply only as a signal that "writing is finished." The only fact the workflow uses for synchronization is that "both agents replied," and the content is passed via a file. That is why there is not a single `handoffTo` written in this sample.

There is another hole of the same nature. Since it is not configured to reject unknown keys, mistyped keys are silently discarded. My validation YAML, where I wrote `closeOnExit` instead of `close_on_exit`, loaded successfully, but the value was never set. Moreover, the naming convention for spelling differs by location. Inside `agents:`, it is snake_case, while inside `workflows:`, it is camelCase. They coexist in the same file.

YAML that included `onEach: reply`, which I haven't implemented yet, also passed for the same reason. You can write it, it passes, and nothing happens. It turns out that configuration falling through is the cheapest kind of error.

A 200-millisecond tax and a 3.4-second administrative fee

I also measured the cost of the orchestration itself. These are figures measured for the orchestration layer using a synthetic workflow that only contains `sleep`.

A 6-stage serial run takes 31 seconds, with an ideal value of 30 seconds. Since the overhead for 5 dependency edges is exactly 1 second, that is 0.2 seconds per edge. The `DRIVER_TICK_MS` on the implementation side is 200. One tick's worth of time is added directly.

In a version where the same 6 stages were split into 3, the time required per step remained 5.2 / 5.1 seconds, no different from the serial version. Spawning does not become heavier even if 3 are set up simultaneously. However, since the wall-clock time for the entire run was not recorded, I cannot say that '30 seconds became 10 seconds.' All I can say is that the gains from parallelization were not diminished on the orchestration side.

A bigger factor is the startup cost of a single agent. I had the same work done by 1 cold agent and 2 warm agents, resulting in 7.7 seconds / 4.1 seconds / 4.5 seconds. Against a warm average of 4.3 seconds, the cold start is about 3.4 seconds, and the variation between warm agents is 0.4 seconds, which is more than 8 times the noise.

This is the lower bound. The prompt for measurement explicitly forbids 'thinking, researching, or reading files,' so it does not include any CLAUDE.md or re-reading of the repository, which would always be included in a real task. Actual startup will cost more than this.

Another factor is waiting for an available pane. Only 9 can be displayed on the screen at once, and in a measurement where 12 were thrown, the latter 6 were made to wait 8.2 seconds each. The order of what should be trimmed becomes clear. The 200-millisecond dependency delay is an error; what really matters is the startup and waiting time.

Before 'just writing the configuration,' there is something that must be pasted by hand

I will be honest. This setup does not work with just the configuration file.

The kickoff is not typed into the pane but is simply placed in a durable inbox. For an agent to read it, the built-in MCP server (Queen) must be registered to each CLI, and as of today, that registration is manual. You copy the command from the toolbar badge and paste it into each CLI yourself. I haven't implemented delegation yet. If this is not done, the first stage will remain stuck in 'Running' forever.

`joinOn: reply` is also one line on the configuration side, but the corresponding protocol must be taught to the agent. Which mailbox to await, and which tool to call with which arguments. In the sample, I wrote it in both the `cmd` bootstrap and each kickoff. This is to ensure that even if the initial instructions flow away, the same procedure remains in the most recent message.

It is also necessary to ensure there is only one reply. Since the step completes upon the first reply, if the agent replies 'Understood, I will take a look' first, the judge will go to read an empty review file.

Therefore, what I can say is not 'it works just by writing the configuration.' It is 'once you register the MCP and pass on the etiquette for replies, the setup can be written entirely in the configuration file.' If you leave this ambiguous, the first person to try it will get stuck at the first stage.

I limited it to filling in only three places

Despite writing all this, the sample above is 549 lines long. That is because I wrote the reasons for all the design decisions in the comments, and it is not something for a first-time user to read. That is why I placed a short, practical version separately. That is the `example/review-starter/` one.

In the 3-stage process of implementation → review → judge, you only need to rewrite the three kickoff locations. They are enclosed in banners like `▼▼▼ Rewrite this part (1/3) ▼▼▼`. 1 is the instruction for the implementer, 2 is the check perspective for the reviewer, and 3 is the judgment criteria for the judge. You only need to write the content of your own work.

      - id: review
        agent: reviewer
        dependsOn: [implement]
        joinOn: reply
        timeoutMs: 1800000    # 30分
        # ▼▼▼ ここを書き換える (2/3): レビュー役へのチェック観点 ▼▼▼
        kickoff: >-
          TODO: ここに「何をチェックしてほしいか」を書く。観点を箇条書きで
          並べる(例: 仕様どおりか / エラー処理の抜け / 既存テストの破壊)。
        # ▲▲▲ ここまで書き換える (2/3) ▲▲▲

The etiquette for replies is embedded in the `cmd` of each agent, not in the kickoff. Which mailbox to await, which tool to call with which arguments, and only one reply after the work is finished—this is the place where people get stuck the most, so it must not be in a position where it can be accidentally deleted while editing the three locations. That is why I isolated it in a place you don't touch.

All three entities are set to plain `claude`. I decided against differentiating roles by model. Valid model names change over time, and if an old name is written, it will fail hard at startup. It is the exact opposite of 'ready to use.' I assigned roles via prompts, and I wrote the procedure for differentiating models later in the comments.

Passing is not the same as being correct.

Today I pushed 9 commits to the branch. The last one is the template for that, and in the middle, there is `docs: record that two parallel reviews can run but cannot be reconciled`. It is a record stating that while two parallel reviews can be run, they cannot be reconciled. I believe that being able to leave this record is a greater achievement than the commits that added functionality.

What resonated with me the most was that the kindness of a configuration language is not determined by the number of features it can express. What determines it is how quickly and how loudly it tells you what you cannot write. A rule that crashes the entire load with `must dependOn root step` is kind, while a rule that allows you to write two `handoffTo` lines is unkind. The former can be fixed in 3 seconds. The latter is something you cannot even suspect until you have seen "why is one side's feedback not being reflected?" a few times.

I would like to ask those who provide configuration files or schemas to others: Does your configuration crash when written incorrectly? Or does it pass? How did you find the combinations that pass even when they shouldn't? Today, the only way I could think of was to re-read the implementation and squash them one by one. If there is a better way, please let me know in the comments.

#ptygrid #AIAgent #MultiAgentDevelopment #AIDrivenDevelopment #ClaudeCode #YAML #ConfigurationFile #Rust #MCP #IndieDev #DevLog