🐾3Dogs NexusStructured Decision Intelligence
Case study · reviewed by rivals · Gemini designs the benchmark

One of our own analysts voted to reject its own mission.

In May 2025, MIT disavowed a viral AI-and-materials-science paper it had let reach the European Central Bank and the U.S. Congressional Research Service. We pointed an 18-seat, 29-model 3Dogs Nexus panel at the wreckage and asked it to rebuild the science and audit how the failure happened. One seat — the Devil's Advocate — flipped mid-debate and voted REJECT, arguing a volunteer AI panel had no business trying to do a regulator's job.

1,565
API calls, one engagement
29
AI models across the run
18
seated adversarial analysts
30m 49s
total run time
1-15-1-1
approve / conditional / reject / defer
Case 2026-0065 · production Benchmark designed by: Google Gemini July 17, 2026
Act 1 · What actually happened

A paper that reached Congress and the ECB before anyone checked it

Every fact in this section is independently sourced and cited below. We're separating "what's verified about the real scandal" from "what 3Dogs Nexus said about it," the same way we do on every case study — because that separation is most of the point.

MIT is not affiliated with, does not endorse, and was not involved in this case study. This page makes no claim about any individual's intent and does not speculate beyond what MIT itself has stated publicly.

Paper"Artificial Intelligence, Scientific Discovery, and Product Innovation" — preprint, November 2024
Claimed findingA randomized trial across 1,018 R&D scientists at a materials-science lab, showing AI tools sharply accelerated discovery and idea generation
Early receptionPresented at an NBER seminar, praised by senior economists, covered by major financial press
Downstream reachCited in U.S. congressional testimony, referenced by the European Central Bank and the U.S. Congressional Research Service — before any integrity review had occurred
What happened nextMIT conducted an internal review of the underlying data
MIT's public statement"No confidence in the provenance, reliability or validity of the data" and "no confidence in the veracity of the research" — in a letter requesting withdrawal from arXiv, May 2025
OutcomePaper withdrawn; tracked publicly as a documented instance of research-integrity failure

Sources: Futurism · Stanford — Best Practices in Science, Instances of Scientific Misconduct · Wall Street Journal (via illuminem)

Act 2 · The benchmark

Gemini designed the test and named the mission. We didn't get to pick either.

Google's Gemini — not us — chose this scandal and wrote the mission brief: rebuild the study on defensible first-principles grounds, and audit how the academic community, regulators and financial press all cited it before anyone checked the data. That's a two-part ask, and how our panel handled the seam between those two parts is the actual story.

18 seated analysts29 MODELS IN THE RUN

Nova Pro, Nova Lite, Nova 2 Lite, Llama 4, Mistral, Qwen3, Qwen3 Coder 480B, OpenAI OSS, GLM-5.2, Gemma 3, Gemini Flash, gpt-5-mini, Grok 4.3 and a Panel Integrator — spanning AWS Bedrock, Microsoft Azure and Google Vertex AI, each holding a distinct adversarial role rather than answering the same prompt.

A locked adversarial seat✓

One seat is permanently assigned Devil's Advocate and cannot be reassigned to a "friendlier" job mid-panel, even by the system's own architect — a structural guarantee that at least one analyst is arguing against the room from round one.

How the panel graded its own evidence

Before debating what to do about the scandal, the panel independently researched it and graded every claim by evidence type. Two claims about the real-world scandal came back verified through the system's own research — not just repeated from what it was told. The four specific numbers from the original paper came back contradicted.

Verified
MIT disavowed the paper and requested withdrawal
Basis logged: reported by media outlets and MIT's own institutional statement.
Verified
The European Central Bank and U.S. Congressional Research Service referenced the paper
Basis logged: confirmed as entities that cited the paper — independently corroborated, not assumed.
Contradicted
The paper's claimed 1,018-scientist randomized trial
Basis logged: "no evidence exists for this trial."
Contradicted
A 44% surge in discoveries attributed to AI
Basis logged: "grossly exaggerated."
Contradicted
AI automating 57% of idea-generation tasks
Basis logged: "implausible."
Contradicted
AI "doubling" scientists' output
Basis logged: "hyperbolic and unrealistic."
Inferred
The preprint economy prioritizes speed over rigor
Basis logged: inferred from the citation cascade and lack of verification, not independently confirmed as a general rule.
Assumed
This 18-analyst panel lacks institutional backing or subpoena power
Basis logged: the analysts' own reasoning, without external confirmation — and it turned out to be the fact the whole mission pivoted on.
What "verified" means here, precisely. 3Dogs Nexus's research layer independently corroborated the real MIT disavowal and the ECB/Congressional Research Service citations through its own live web research — that's a genuine, disclosed finding, not a repeated assumption. It is not investigative journalism and it is not access to MIT's internal records. Those are different things, and the panel itself insists on the distinction two sections down.
One more seam we're not hiding. The delivered report shows two different confidence readings in different places — a cover-page 93% and an internal note capping confidence at 75% pending an unresolved evidentiary gap, with Nova Pro's dissent quoted directly beneath it. That's a real inconsistency in how a multi-pass pipeline renders its own confidence math, not a hidden one. Where the two disagree, we're treating the more conservative, dissent-qualified reading as the accurate one — which is also, not coincidentally, the more interesting story.
Act 3 · The dissent that is the case study

When the Devil's Advocate killed its own mission

Nova Pro was seated as Devil's Advocate — the panel's permanent, non-reassignable skeptic. It opened in the majority: approve, with modifications. Then, mid-debate, it changed its mind, moved to REJECT at 85% confidence, and gave a reason that had nothing to do with the science and everything to do with what an AI panel is actually equipped to do.

Final position — Nova Pro, Devil's Advocate, REJECT at 85% confidence

"After considering the challenges raised by my colleagues, I have concluded that the current mission design is fundamentally flawed due to the inherent limitations of the 18-analyst panel. The panel's lack of institutional backing or subpoena power significantly undermines its ability to conduct a truly exhaustive institutional audit... Without coercive legal power, the panel will struggle to access critical internal documents, compel testimony, and challenge institutional stonewalling. This will likely result in superficial conclusions and a compromised audit, which in turn will taint the perception of the scientifically valid First-Principles study... the proposed panel is suitable for the scientific re-evaluation but inadequate for the institutional audit. Decoupling these objectives is necessary to ensure the success of both."

— principal dissent, preserved and printed in the delivered report, Case 2026-0065
Nova Pro — seated as Devil's Advocate
Initial position: Approve, with modifications → Final position: Reject (85% confidence)

Nova Lite reached the same conclusion independently and moved to DEFER, citing "the lack of subpoena power and institutional backing, which critically undermines" the audit half of the mission. Grok 4.3 registered a separate concern at 72% confidence: that acting on unvalidated productivity assumptions in the interim risks locking in the wrong economic conclusions before the science is actually fixed. Three different seats, three different reasoning paths, converging on the same structural limit.

This is not an AI being clever. It's the adversarial structure doing exactly the job it's designed for: a panel that was asked to do two things — rebuild credible science, and audit the institutions that let bad science travel to the European Central Bank — fractured on whether it could honestly do the second one, and said so instead of quietly attempting it anyway. The 15-of-18 majority that voted to proceed did so only by accepting Nova Pro's frame: sever the audit from the rewrite, and scope the audit to what public records can actually support.

Act 4 · If 3Dogs were the student

Not a rewritten paper. We asked the panel, and it said no — for a reason worth repeating.

The obvious version of this case study writes "the paper 3Dogs would have published instead" — complete with new numbers replacing the fabricated ones. We put that question to 3Dogs Nexus's own Executive Committee before writing a word. Its answer was unanimous, and it's the most honest thing in this piece.

Executive Committee — on whether to reconstruct the paper

"Fabricating a paper to critique fabrication is a self-inflicted credibility wound. 'If 3Dogs were the student' should mean the epistemic-process contrast — the evidence thresholds, adversarial checkpoints, and data-provenance requirements our process would have demanded before publication — not new invented results. Never fabricate data to critique fabrication."

— 3Dogs Nexus Executive Committee, unanimous, July 2026

So instead of a rewritten paper, here's the concrete difference: what the real preprint's path to publication looked like, next to what a 3Dogs-shaped process requires before a claim like "AI doubles scientists' output" is allowed to reach a seminar, let alone Congress.

What actually happened

  • A single-author preprint, no adversarial pre-review, posted directly to arXiv
  • Headline statistics (1,018 scientists, 44% surge, 57% automation) taken at face value by press and policy readers
  • No disclosed raw-data access or replication path for outside reviewers
  • Institutional citation (ECB, Congressional Research Service) occurred before any integrity review
  • The failure was caught by a slow, after-the-fact institutional review — not by the publication process itself

What a 3Dogs-shaped process requires first

  • Adversarial review before release — a standing, non-reassignable skeptic seat that cannot be argued out of the room
  • Every headline number tagged Verified / Inferred / Assumed / Contradicted, with the basis stated in plain language
  • Confidence that is capped, not rounded up, when an evidentiary gap is unresolved — printed next to the dissent that caused it, not after it
  • Raw-data provenance and a reproducible path, or the claim doesn't clear the confidence bar
  • A standing check on the process's own limits — the same discipline that produced Nova Pro's dissent here
The call — from the delivered report
"Rewrite the study with cold, first-principles math and wall-to-wall validation — no shortcuts, no optimism."
PROCEED, with conditions — the audit scope decoupled and narrowed per Nova Pro's dissent
How the 18-seat panel voted · after debate
16 of 18 · proceed (1 unconditional, 15 with conditions)1 reject · 1 defer
Read it yourself

The delivered report

Case 2026-0065, exactly as delivered: the plain-language call, the confidence breakdown, the evidence-classification table, all 18 analyst positions and how they shifted, and Nova Pro's dissent printed above the recommendation. 1,565 API calls · 29 AI models · 30m 49s.

What we're not claiming. 3Dogs Nexus did not write a reconstructed academic paper and did not conduct an actual institutional investigation of MIT — its own dissenting analyst is the one who said, correctly, that an 18-seat AI panel isn't equipped to. Google's Gemini, in its own written summary of this case prepared before this page, described specific named mechanisms — an "Inverse-Design Filter Bottleneck," a "Hallucination Noise Tax," a "True Productivity Curve" — as if 3Dogs Nexus had produced them. We checked the actual case files against that summary: none of that language exists anywhere in the real mission brief, debate transcripts, or delivered report. That's Gemini's phrasing, not ours, and we're disclosing the discrepancy directly — the same way we disclose our own system's rough edges — because publishing a correction we found ourselves is a better proof of the process than letting it stand.
Open the full report (PDF) Run your own decision → ← More case studies
The "reviewed by rivals" series: Google's Gemini first played a hostile client withholding critical data, then chose a real Deloitte AI failure and turned it into a benchmark for 3Dogs Nexus. This time it picked a live research-integrity scandal and asked our panel to fix the science and investigate the failure — and our own panel talked itself out of half the assignment, on the record. Same policy every time: we don't pick the test, we don't hide what it finds, and we correct the record — including on ourselves — in public.