Share this article and save a life!
The FDA just funded a jury of AI judges for radiology reports. 🧠
Not one AI. A panel of them. Cross-examining each other.
Last week, the FDA awarded Cognita Imaging a $1.29 million research contract to build what they call an “LLMs-as-a-jury” evaluation framework for autonomous, generative AI radiology reports. According to HIT Consultant, the 18-month contract was funded under the FDA’s Broad Agency Announcement program for advanced research and development in regulatory science.
Here is why this matters to anyone building in this space.
🔹 Traditional human reader studies evaluate only a few hundred cases.
🔹 They miss rare edge cases, subtle equipment discrepancies, and regional workflow variations.
🔹 Cognita’s framework will be stress-tested across approximately 1 million patient imaging exams from a diverse U.S. clinical cohort.
🔹 The system analyzes performance variance across demographics, care settings, imaging hardware manufacturers, and low-prevalence pathologies.
That gap between a few hundred cases and 1 million is not a footnote. It is the entire problem.
The architecture itself is worth understanding. Rather than trusting a single LLM to score diagnostic drafts, Cognita aggregates outputs across an ensemble of distinct models acting as a virtual panel of expert reviewers. The platform then isolates high-discrepancy outliers for expert human radiologist review. Researchers can see whether errors came from the generative drafting model, disagreement within the jury, or ambiguity in the original human radiologist’s ground truth.
This builds on Cognita’s open-source GREEN benchmark, Generative Radiologist Evaluation and Error Notation, which was designed to capture clinically meaningful discrepancies between ground-truth radiologist reports and AI-generated text.
The project is led by Dr. Akshay Chaudhari, Cognita co-founder and Associate Professor of Radiology and Biomedical Data Science at Stanford University, alongside co-investigator and CEO Dr. Louis Blankemeier.
⚡ What most people are missing here: this is the FDA funding the infrastructure to evaluate generative AI in radiology, not just approve individual products. That is a structural shift in how the agency approaches this category.
At Oatmeal Health, we are building AI for lung cancer screening. Every day I think about what it means to deploy AI at scale across diverse patient populations and imaging environments. The question of how you validate performance across millions of real-world cases, not just curated test sets, is one of the hardest problems in this field.
The FDA is starting to take it seriously. That should matter to all of us.
👉 Follow Jonathan Govette, CEO of Oatmeal Health, for daily healthcare insights on LinkedIn. Deeper dives in The Oatmeal Bite on Substack: https://news.oatmealhealth.com
Share this article and save a life!
Author:
CEO/Co-Founder @ Oatmeal Health | AI Lung Cancer Screening | Almost Became a Doctor | Engineer | Follow to Share What I’ve Learned Along the Way
I help patients get the care they need earlier, preventing late-stage cancer.
That’s been the throughline across three companies and almost 20 years in healthcare. At ReferralMD, we fixed broken referral networks so patients didn’t fall through the cracks. At Oatmeal Health, it’s lung cancer: building the diagnostic and screening infrastructure so the 85% of cases caught too late get caught early instead.
Today as CEO of Oatmeal Health, I lead a team embedding AI into radiology workflows to turn routine lung CT scans into reimbursable cancer risk assessments. We partner with FQHCs to reach underserved communities, and with health systems and payers to make early detection economically sustainable. Think HeartFlow or Cleerly, but for lungs.
Between companies, I advised at Techstars and Plug and Play, mentoring founders building in digital health. That experience shaped how I think about what separates companies that ship from companies that stall: distribution, reimbursement, and clinical trust, not just technology.
I’m a CancerX alumnus, a 3x healthcare founder, and someone who believes the biggest problems in cancer aren’t scientific. They’re operational.
We’re hiring mission-driven builders at Oatmeal Health. If you want to work on something that matters, reach out.
When I’m not working, I’m traveling, mentoring, and keeping up with one very energetic husky. 🐾
Substack – The Oatmeal Bite:
Millions of patients get less care because of who they are, where they live, or how they look. I’m fighting to change that. CEO @OatmealHealth, a startup built for the underserved. The Oatmeal Bite: intel for clinicians, investors, and advocates.
Jonathan Govette
CEO of Oatmeal Health
Substack:
https://oatmealhealthjonathangovette.substack.com/
