I used to take AI answers at face value.

Not on purpose. I’d ask AI a question, get a well-structured response, skim it, and move on. It felt thorough. It sounded confident. So I trusted it.

Then I started fact-checking things that mattered. A market size number that turned out to be two years old. A code snippet with a subtle logic error I didn’t catch until production. A “study” that didn’t actually exist.

And the most frustrating part was that when I pushed back on the answer, the same AI would immediately find its own mistake:

“You’re right, that figure was from 2023. Here’s the updated data.”

It knew better. It just didn’t check itself the first time.

I started doing something manual and kind of annoying: asking the same question to multiple models, then comparing answers. When Claude, Gemini and ChatGPT disagreed on something, that disagreement was usually where the real answer lived. The friction between perspectives was doing the work I thought a single smart model should do on its own.

Turns out, that instinct has a name now. And it’s becoming an actual product.

Today’s guest post is from Ilia Karelin, who writes Prosper, the newsletter about AI and software that gives you an unfair knowledge advantage. This is Ilia’s fourth time writing for AI Maker; his first post, on the 3-document system for AI memory is still one of our most-shared pieces, followed by “When NOT to Use AI,” and, most recently, “Claude in Chrome.”

Prosper

Tactical AI workflows and frameworks every week - so you build systems that actually work, not just prompts that disappear.

By Ilia Karelin

Here are Ilia’s latest three posts that you might want to check out:

1. 3 Ways Claude Cowork Dispatch Turns Your Laptop Into a Background Worker

2. Raycast: The App I Use to Connect Claude, Perplexity, and NotebookLM + 35 More Workflows

3. Your CLAUDE.md Is Static. Your AI Forgot Everything Again

In this post, Ilia breaks down Grok 4.20’s multi-agent architecture, where four AI agents argue with each other before giving you an answer, and connects it to Andrej Karpathy’s open-source LLM Council project. He also gives you a prompt you can copy and use today to run your own “council” inside any AI tool you already have.

I think this piece captures something important about where AI is actually heading: not just smarter models, but models that know how to check each other’s work. That’s the same principle behind how I’ve been building my own workflows, using AI as a thinking partner that challenges my assumptions, not just confirms them.

Here’s Ilia.

Hello 👋🏻

Every AI model you’ve used works the same way. You ask a question. One model thinks about it. One model answers. If it’s wrong, you find out later.

Grok 4.20 does something different. When you ask it a hard question, four agents spin up inside the system. They each think independently. Then they argue. They challenge each other’s facts, flag weak reasoning, and only after they reach consensus does one of them write the final answer.

You never see the argument. You just get a better answer.

xAI launched Grok 4.20 in public beta on February 17, 2026, and it’s the most structurally interesting AI release I’ve seen this year. The parameter count and benchmarks aren’t the interesting part in my opinion. What’s interesting is where AI is going: from solo performers to small, coordinated teams - and I love this approach.

And it connects to an idea Andrej Karpathy - former head of AI at Tesla, co-founder of OpenAI - built as a weekend project late last year. More on that in a minute.

Grok 4.20’s four agents: Captain, Harper, Benjamin, and Lucas

Grok 4.20 runs four specialized agents on every complex query. They share the same underlying model weights, but they have different jobs, different incentives, and different system prompts. Think of it as four experts with the same brain but different priorities.

• Grok (the Captain) breaks your question into sub-tasks, assigns them to the other three, resolves conflicts between their outputs, and writes the final answer. It’s the project manager.

• Harper (the Researcher) pulls live data from the web and X’s real-time firehose. Its job is evidence. When someone on the team makes a claim, Harper is the one checking whether it’s actually true right now, not just true as of the training cutoff.

• Benjamin (the Analyst) handles math, code, step-by-step logic, and formal reasoning. If the answer involves a calculation, a proof, or a code path, Benjamin is the one verifying it line by line.

• Lucas (the Contrarian) is the most interesting one. Lucas is trained to disagree. Its entire purpose is to find alternative angles, catch blind spots, and challenge the other three. Every team benefits from one person whose job is to say “wait, are we sure about that?” Lucas is that person.

These are four personas running on the same Grok 4-series weights, activated with different system prompts and objectives. All four run concurrently on xAI’s Colossus infrastructure, sharing the same prefix cache and input context. The marginal cost is reportedly 1.5-2.5x a single pass, not 4x, which is how xAI can afford to run this on every sufficiently complex query.

If you’re a visual person (I am), here’s how it looks like:

How Grok 4.20’s multi-agent debate works

On hard, multi-step questions, here’s what happens:

1. Grok (Captain) reads your prompt and decomposes it into sub-tasks. It decides which agents need to be involved and what each one should focus on.

2. All four agents think in parallel. Harper pulls sources. Benjamin runs calculations. Lucas explores alternative framings. Each generates their own analysis independently.

3. They debate. This is the part that matters. The agents exchange short messages challenging each other’s reasoning. Harper might say “your GDP number is wrong - here’s the latest from X.” Benjamin might say “that code path has an off-by-one error.” Lucas might say “you’re all ignoring the scenario where the user actually wants the opposite.”

4. Grok synthesizes. The Captain takes the best pieces from each agent, discards the weak branches, resolves conflicts, and writes one unified response.

The user sees step 4. Steps 1-3 happen behind the scenes. Some users report briefly seeing glimpses of the internal discussion in the UI before it collapses into the final answer.

Grok 4.20 hallucination rate: down 65% from Grok 4.1

Here’s the headline result. Grok 4.20’s hallucination rate dropped from roughly 12% (Grok 4.1) to about 4.2%. That’s a 65% reduction.

Why? Because when one agent makes something up, another agent catches it. Harper invents a statistic, Benjamin flags the math. Benjamin writes buggy code, Lucas spots the logic error. The internal cross-checking creates a peer-review loop that a single model can’t replicate by itself.

xAI calls this “adversarial consensus.” I’d call it something simpler: built-in fact-checking.

On benchmarks and arena.ai, Grok 4.20-beta1 (beta2 just recently launched on March 2) sits around 4th spot on the leaderboard. With a completely different approach to LLM, I would say that this is a very good result!

But benchmarks are benchmarks. The hallucination reduction is the number that matters for real work. If you’re using an AI for research, analysis, or anything where accuracy matters more than speed, 65% fewer wrong answers changes the equation.

SuperGrok Heavy: 16 agents for research-grade problems

Grok 4.20’s four-agent setup is the default for complex queries. But xAI also offers a heavier configuration.

SuperGrok Heavy ($30/month) scales the system to 16 agents for extreme-complexity tasks - academic research, multi-domain strategy, problems that require maximum depth.

The extra agents aren’t 16 completely different personas with individual names. They’re additional specialized replicas that explore more hypotheses in parallel. More agents means more angles explored, more cross-checking, and a wider search for the best answer. xAI hasn’t published the individual role breakdown for all 16, but the pattern is the same: parallel thinking, internal debate, captain synthesis.

Is 16 agents worth $30/month? Depends on what you’re doing. For most people, the default four-agent setup on SuperGrok or Premium+ is probably enough. Heavy mode is aimed at researchers and enterprise users who need the absolute maximum depth on genuinely hard problems.

Karpathy’s LLM Council and why Grok 4.20 validates it

The engineering is impressive, it always impresses me. But the bigger deal is that Grok 4.20 is the first major product to ship with the LLM Council pattern built in. And that pattern has been gaining momentum independently.

In late November 2025, Andrej Karpathy - the person most AI practitioners trust when he says something matters - posted what he called a ”Saturday vibe code project”:

An open-source app called LLM Council.

It works like this:

1. You ask a question.

2. Your question goes to multiple different models simultaneously - GPT-5.1, Gemini 3 Pro, Claude Sonnet 4.5, Grok 4. Actually different models, from different companies, trained on different data.

3. Each model answers independently. No one sees anyone else’s work.

4. Every model then reviews and rates the other models’ answers - anonymously. Peer review.

5. A “Chairman” model reads all the answers plus all the reviews and writes the final, synthesized response.

Brilliant.

Karpathy’s insight: when you combine multiple independent perspectives - especially from models trained by different teams on different data - errors don’t correlate. One model’s blind spot is another model’s strength. The ensemble catches things any individual model would miss.

You know I like graphs, so here’s how it looks like in visually:

Research on multi-agent AI debate and accuracy gains

The research backs this up too.

MIT researchers found that when you take multiple instances of the same language model and have them debate each other - generate answers, critique each other, revise - arithmetic accuracy jumps from about 70% with a single agent to roughly 95% with three agents over two rounds. (MIT News, 2023)

A separate study on multi-agent debate showed 4-6% higher accuracy and over 30% fewer factual errors compared to standard single-pass methods, specifically on mathematical reasoning and factual questions. The mechanism is straightforward: agents catch each other’s mistakes.

Zhao et al. (2024) tested the LLM Council concept specifically for evaluation tasks and found something worth noting: the council’s agreement with human evaluators matched the rate at which humans agreed with each other. A group of models debating produced human-level judgment reliability. ([Link to the article])

And earlier this year, a working paper from SUNY Empire State University tested the council pattern on a real policy problem - faculty workload modeling. Five frontier models (Claude, GPT, Gemini, Grok, Perplexity) independently analyzed the same problem. Despite never seeing each other’s outputs, all five converged on the same core solution: the same baseline number, the same two-track structure, the same critical assumption needing validation. The researcher used three co-captains instead of one to triangulate the synthesis. ([Chukhlomin, 2025 — SSRN](Link to the article))

Convergence from independent analysis is a strong signal. When five different systems arrive at the same answer without coordinating, that answer is probably right. When they disagree, the disagreement itself is useful - it tells you where the genuine uncertainty lives.

What Grok 4.20’s multi-agent system means for AI’s future

Here’s the reframe.

For the last two years, AI progress has been about making one model smarter. Bigger parameters, better training data, longer context, more reasoning steps. One brain, pushed further.

Grok 4.20 is a bet on a different path: four adequate brains working together beat one exceptional brain working alone.

That’s not a new idea in other fields. Peer review exists because one scientist can’t catch their own errors. Jury systems exist because twelve perspectives reduce bias. Editorial teams exist because one writer misses things.

But it’s a new idea in AI products. Every other major model - Claude, GPT, Gemini - still gives you one response from one system. If it hallucinates, you’re on your own.

Grok 4.20 hallucinates too. But 65% less often, because three other agents are watching.

Share

Three takeaways from Grok 4.20’s multi-agent architecture

1. The council pattern could be the future

Grok shipped it first. Karpathy built an open-source version. Research backs the accuracy gains. Sometimes, you see Claude or other LLM argue with itself during the “thinking” phase before responding to you, but Grok’s approach is different. It would be amazing if Claude, GPT, and Gemini will follow with their own multi-agent architectures within the next 6-12 months. I’d love for it to be a standard feature in the next year.

2. You can build your own council today

You don’t need Grok 4.20 or SuperGrok Heavy to get this benefit. Karpathy’s LLM Council is open source. You can set it up with any models you have API access to through OpenRouter. Ask the same question to Claude, GPT, and Gemini. Have one of them synthesize. It’s more manual than Grok’s native setup, but the accuracy benefit is real.

3. The Lucas role is the one to watch

The dedicated contrarian - the agent whose job is to disagree - is the design choice that matters most. Every team knows that the person who asks “are we sure?” prevents more mistakes than the person who says “looks good.” xAI built that role into the system. If you’re building your own multi-agent workflows, add a Lucas. The agent that’s paid to find holes is the one that makes the whole thing work.

Try it yourself: a council prompt you can use right now

Most of us probably (I am just assuming here) have Claude or ChatGPT subscription, and probably don’t want to go through Karpathy’s steps to set up the Council LLM. No problem! I got you.

I wanted to see if this pattern actually works in practice - not just in benchmarks. So I tested it on a real business problem, in Claude.

I have this website, ProsperInOSS.com, a Netflix-style discovery site for open-source tools, alternatives to your most favorite tools. I will be frank with you - I haven’t been keeping up with it at all. I haven’t gotten any placement requests. Last time I looked at it was months ago. It only has 70+ tools, organized by category, with GitHub stats and alternatives. My monetization strategy: selling promoted placement spots to OSS companies and creators. I don’t know if it’s the right call or not.

So I gave Claude this prompt:

You are a council of 4 experts analyzing my business strategy. Follow this process strictly:

1. RESEARCHER: Gather relevant facts, data, and evidence about this market. State what you know and what’s uncertain.

2. ANALYST: Examine the business logic, identify assumptions, and check the reasoning step by step.

3. CONTRARIAN: Challenge the other two. Find blind spots, alternative explanations, and reasons this might fail.

4. SYNTHESIZER: Read all three perspectives. Resolve conflicts, keep what survived the debate, discard what didn’t. Write the final answer.

Show each agent’s thinking under their name before giving the final synthesized answer.

My question: [describe your business, strategy, and what you want evaluated].

Here’s what happened:

The Researcher flagged that competitors in the OSS directory space have tried paid placement before - and got backlash from the open-source community, which is culturally hostile to pay-for-play.

The Analyst broke down my unit economics and identified three assumptions I hadn’t questioned - including that OSS companies would even pay for a small directory when they can buy Google Ads at scale.

The Contrarian went harder: it argued that AI itself is eating the discovery use case, and that a static directory might not generate enough value to monetize at all.

Then the Synthesizer pulled it together. It kept what survived the debate (the site has real value as a content asset), discarded what didn’t (pure ad monetization is weak for this audience), and suggested alternatives I hadn’t considered - affiliate revenue from tools with commercial tiers, using the directory as a lead magnet for my newsletter, and reframing promotion spots as editorial deep-dives instead of ads.

Pretty interesting, right? One prompt. Four perspectives. Blind spots I would’ve missed on my own.

What I am trying to say is that you don’t need Grok 4.20 to run a council. You just need the right prompt structure: independent thinkers with different jobs, forced to challenge each other before anyone writes the final answer. Copy the prompt above, swap in your own question, and try it.

Grok 4.20 and the LLM Council pattern: the bottom line

Grok 4.20’s four-agent system is the first consumer AI product built on a simple principle: multiple models checking each other’s work beats one model checking its own.

The council pattern - independent thinking, peer review, synthesized consensus - reduces hallucinations, catches reasoning errors, and produces answers you can trust more. Not perfectly. But measurably more.

The shift from “ask one AI” to “ask a team of AIs” is the most important architectural change happening in AI right now. And the people who understand it will use AI differently than the people who don’t.