Claude Concilium: multi-model code review without fake consensus

Two AI reviewers can produce twice as much confidence without producing more evidence.

That is the failure mode to avoid in multi-model code review. A second model can widen the search: one reviewer may trace concurrency while another checks authorization or compatibility. But if both receive the same vague prompt and then vote, shared assumptions can become a polished consensus.

Claude Concilium is useful as a small set of adapters for Claude Code. It should not be treated as an oracle. This guide separates what the repository actually implements from the review method you still need to design.

What the repository contains

I audited the public repository at commit 970eef6, dated 2 March 2026. The source contains three Node.js stdio MCP servers:

Claude Code
├── mcp-openai  → codex exec / codex review
├── mcp-gemini  → gemini -p
└── mcp-qwen    → qwen -p -

Together they expose five tools:

The repository also has setup notes, an example MCP configuration, a Dockerfile, a smoke test, and an ai-concilium skill. The code is short enough to review before installing.

Three distinctions matter.

First, this is not an orchestrator service. Claude Code is expected to call the servers and synthesize their answers. The repository does not implement a durable queue, shared state, or a consensus algorithm. It is narrower than a complete multi-provider LLM architecture.

Second, fallback is not automatic inside the servers. The skill file tells the host agent to call Qwen and then DeepSeek after certain errors. Each server itself knows only its provider.

Third, DeepSeek is not a fourth server in this repository. The example configuration runs a separate npm package with npx -y. That is a separate dependency and trust decision.

What the smoke test proves

I installed the declared dependencies in a temporary clone with lifecycle scripts disabled and ran:

node test/smoke-test.mjs

All three servers completed MCP initialization and returned their tool lists. That verifies the local protocol wiring for the audited commit.

The test source explicitly avoids calling provider CLIs. A pass therefore does not prove that:

• codex, gemini, or qwen is installed;
• authentication is valid;
• a requested model exists on the account;
• quota-error pattern matching still matches current CLI output;
• the target repository is visible to the provider;
• the generated review is correct.

Add a provider-level canary for every server you enable. Use a harmless fixture repository, ask for a deterministic fact about one file, and confirm the working directory, timeout, and error path.

Source-level behavior and risks

The OpenAI adapter sends prompts through stdin to codex exec, requests an ephemeral session, and sets the Codex sandbox to read-only. That is a good default. Current Codex documentation confirms that codex exec supports non-interactive runs and stdin prompts (CLI reference).

The Qwen adapter also sends its prompt through stdin. The Gemini adapter, however, places the complete prompt in a command-line argument. On some operating systems, process arguments are visible to other local processes or diagnostic tooling. Do not put secrets, customer records, access tokens, or raw production data in review prompts.

All three servers inherit the parent environment. Keep credentials out of project .env files that a child CLI may load, pass only the variables the tool needs, and run the adapters under a low-privilege account.

There are two other operational limits:

• Error classification relies on matching fragments of CLI output. Provider wording can change.
• A non-zero child exit code is not universally treated as failure when stdout is non-empty.

Those are reasons to inspect returned status and evidence, not reasons to discard the project. They are also useful hardening targets if you fork it.

As with any production MCP server, the protocol itself does not make a tool trustworthy. The specification says hosts should show tool inputs, obtain user consent, apply timeouts, and validate results before returning them to the model (MCP tools security considerations).

Install it as reviewed code

Do not pipe an installer into a shell or copy an unpinned configuration from a post. Start with a commit you have inspected:

git clone https://github.com/spyrae/claude-concilium.git
cd claude-concilium
git checkout 970eef6bb5c2267f10a16229cba161e154fd6221

git show --stat
find servers -maxdepth 2 -type f -print

Then review:

1. every server.js;
2. every package.json;
3. config/mcp.json.example;
4. the skill that decides when another provider is called;
5. any credential mounts or environment variables.

The repository does not include lockfiles at the audited commit, so npm install can resolve newer transitive versions than the author tested. Generate and review lockfiles in your own fork if you plan to use it repeatedly. Do not add npx -y deepseek-mcp-server merely to complete the sample configuration; pin and inspect that package separately.

Authenticate providers using their current official instructions. Codex supports ChatGPT sign-in and API-key sign-in, with different billing and data policies (OpenAI authentication). Gemini authentication likewise affects quota, pricing, terms, and privacy (Gemini CLI authentication). Check those pages at installation time rather than copying a quota number into permanent configuration.

Use an absolute working directory. Give a reviewer read-only access to the smallest repository scope it needs. Keep .env, production dumps, private keys, and customer exports outside that scope.

A review protocol that does not vote

A reliable workflow uses models to generate competing hypotheses, then uses engineering evidence to decide.

1. Establish the baseline

Before any model call, run the repository’s deterministic checks:

targeted tests
type checking
lint/static analysis
dependency or secret scan, when relevant

Record existing failures. A reviewer should not attribute a red baseline to the diff.

2. Define the contract and risk

Provide:

• the intended behavior;
• the exact diff or commit range;
• relevant interfaces and invariants;
• supported platforms and compatibility constraints;
• commands already run;
• files that are out of scope.

Do not dump an entire monorepo merely because a model accepts a large context. More context can add irrelevant anchors and expose more data.

3. Keep the first passes independent

Do not show reviewer B the answer from reviewer A. Give each a different, concrete lens:

Reviewer A — correctness and concurrency:
Trace changed control flow. Find a reachable failure involving state,
ordering, cancellation, retries, or cleanup.

Reviewer B — security and boundary contracts:
Check authorization, input trust boundaries, data exposure, dependency
behavior, and backward compatibility.

Different model vendors may add diversity, but vendor count is not a substitute for different tasks and evidence.

Research on multi-agent debate is mixed. An ICML 2024 benchmark found that debate protocols did not reliably beat simpler prompting strategies without careful tuning (Smit et al.). This is why the goal is not consensus.

4. Require a review record

Ask each reviewer to return only actionable findings:

For each finding:
- severity;
- file and line;
- violated contract or threat;
- concrete execution path;
- smallest reproduction or test;
- uncertainty and missing context.

Do not return APPROVE based only on plausibility.
Do not suggest style changes unless they hide a defect.

A finding without a code path or checkable claim goes into “question,” not “bug.”

5. Deduplicate, then verify

Merge findings by root cause, not by wording. Two models may paraphrase the same mistaken claim.

For every material finding, obtain at least one of:

• a failing regression test;
• a static-analysis result;
• a minimal reproduction;
• a protocol or framework requirement from primary documentation;
• a trace through the actual code and state transition.

If the claim cannot be verified within the review budget, label it unresolved. Do not convert agreement into severity.

6. Let a human own the decision

The author or reviewer decides whether to fix, reject, defer, or investigate. Record why. An AI synthesis can organize evidence, but should not silently approve a merge or weaken a test.

Use the existing AI code-review checklist for the final pass, and apply the controls from the MCP security guide when adapters can see private repositories.

A prompt that asks for evidence

Review commit <sha> against this contract:
<approved behavior and invariants>

Scope:
<files and diff>

Your lens:
correctness, concurrency, cancellation, and cleanup only.

Baseline:
<commands and results>

Return findings only when you can provide:
1. severity;
2. file:line;
3. reachable execution path;
4. expected vs actual behavior;
5. a minimal test or reproduction.

List missing context separately. Do not infer confidence from another
reviewer's opinion. Do not edit files.

After both independent passes, give the deduplicated claims—not the persuasive prose—to a verification pass.

Measure whether the second model helps

Do not publish a catch-rate percentage from a handful of memorable reviews. Build a small evaluation set from your own work:

• previously fixed defects with known root causes;
• clean diffs that should not produce findings;
• seeded variants of boundary, authorization, cleanup, and concurrency bugs;
• changes large enough to test context handling.

Compare single-reviewer and multi-reviewer runs on:

• verified defects found;
• unique verified defects added by the second reviewer;
• false-positive findings;
• time to verify;
• latency and provider usage;
• sensitive-data policy violations.

Freeze the prompts and repository commits while comparing. Re-run after a model, CLI, or adapter update. If the second reviewer adds mostly duplicate or unverified findings, remove it from that class of change.

Multi-model review is most useful for risky diffs with several independent failure surfaces. For a typo, a mechanical rename, or a well-covered one-line change, targeted tests and one disciplined review are usually cheaper and clearer.

Bottom line

Claude Concilium is a compact, inspectable bridge from Claude Code to three provider CLIs. At the audited commit, it is not an automatic fallback system and it does not prove correctness through consensus.

Use it to widen the search. Keep reviewers independent, constrain what they can read, demand reproducible evidence, and let tests and human judgment—not a vote—close the review.