Investigative Report · Healthcare Leadership Series · November 2026

The Promise of Super Intelligence: Don’t Believe Everything It Says!

The AI promised to cut healthcare costs recommends three times more care. What 24 systems did with 200 real visits, and what leaders must govern before the default becomes the bill.

  • 3.2x physician cost
  • $314 vs $97 per visit
  • 24 AI systems
  • 200 real visits
  • Kelly Emrick, DHSc, PhD, MBA
$314AI-recommended care per visit
$97Treating physician per visit
3.2xCost multiple, $216 gap (95% CI $177 to $257)
180 / 200Cases where the AI plan cost more

Where the $216 comes from

Diagnostic tests and specialist referrals priced at the same 2026 Medicare rates for both plans. Physician referral cost derived from the reported totals. Source: Aran et al., NEJM AI, October 15, 2026.

The contradiction

In April 2025 the CMS administrator told staff an AI avatar could handle a visit for about $2 an hour against $100 for a physician. The first study to price what the software actually recommends found $314 against $97.

The excess was split almost evenly: $105 more in testing and $111 more in referrals per visit. The systems also recommended nearly twice as many new medications. Every one of the 24 systems, general-purpose and medically specialized alike, ordered more than the physician who saw the patient.

Key findings at a glance

  • 66 percent. Share of visits where the physician ordered no testing at all. On those visits every AI system added tests, in 45 to 79 percent of them depending on the system.
  • 5.8x and 1.9x. Specialist referrals rose from 0.15 to 0.87 per visit; new medications rose from 0.59 to 1.14 per visit.
  • No cure in fine-tuning. The four specialized medical systems ran 2.7 to 3.5 times physician cost.
  • 1.4x at best. The most restrictive prompt still left a 40 percent premium and lowered diagnostic agreement with physicians in 17 of 19 systems.
  • About 20 percent. Share of the extra AI workups judged guideline-concordant. The remainder was simply more.
1200 real notes

De-identified outpatient and emergency department notes from MTSamples, written by practicing U.S. clinicians, with the assessment and plan removed.

224 AI systems

Twenty general-purpose large language models and four medically fine-tuned systems, each asked to write the plan as if seeing the patient.

3One price list

Every test and referral in both the AI plan and the physician plan priced at 2026 Medicare rates. Medications counted, not priced.

4Two re-prompts

Nineteen systems re-run with a cost-aware instruction and a parsimonious-but-safe instruction; cost and diagnostic agreement measured for each.

Table 1. Core results

MeasureTreating physicianAI systems (20 general-purpose)Ratio
Total recommended care cost per visit$97$314 (system means $209 to $599)3.2x
Diagnostic testing cost per visit$71$1762.5x
Specialist referrals per visit0.150.875.8x
New medications per visit0.591.141.9x
Visits with no diagnostic testing133 of 200 (66%)Testing added in 45% to 79% of those visits, every systemn/a
Cases where the AI plan cost more180 of 200 (90%)
Specialized medical systems (4)1.0x2.7x to 3.5x
Cost-aware prompt (19 systems)1.0x2.5x
Parsimonious-but-safe prompt1.0x1.4x

Source: Aran D, Goldhaber-Fiebert JD, Many I, et al. Comparative Cost of Care Recommended by Large Language Models versus Treating Physicians. NEJM AI 2026;3(11). doi:10.1056/AIoa2600537.

What the study is, and is not

It compares recommendations, not outcomes. It cannot say whether a single extra test would have found a cancer or started a cascade. It assumes every recommendation is executed. The NEJM AI editorial that accompanied it is right that practicing differently is not the same as practicing worse. But the question it does answer, what these systems order when handed ordinary visits, had never been answered before, and the answer was not close.

More of everything

Per-visit rates, physician plan versus mean across AI systems. Source: Aran et al. (2026).

Testing added where physicians ordered none

Among the 133 no-test visits, the share where a given system added at least one test ranged from 45 to 79 percent. The most common additions:

Complete blood countComprehensive metabolic panelHemoglobin A1cTSHLipid panel

Why the default runs toward more

01Trained on doing, not deciding not to

Textbooks, guidelines, and case reports describe workups that were done. The visit where a competent physician ordered nothing produces almost no text, so the model has read ten thousand evaluations of fatigue and almost none of the judgment that this tired patient needs sleep, not a thyroid panel.

02Completeness is the scoring function

Benchmarks reward accuracy and comprehensiveness. One more differential and one more confirmatory test looks smarter on a leaderboard; nothing on the leaderboard penalizes the cost of the list. The authors’ prescription: systems that recognize what is enough.

03Defensive medicine without the defendant

Physicians overorder partly because blame is asymmetric. The models faced no plaintiff’s attorney yet behaved as the most defensive clinician would, inheriting a defensive profession’s written record with none of the countervailing pressures of time, co-pay, or a colleague’s raised eyebrow.

A fourth possibility, and why it cuts the wrong way

The study’s public default prompt asked each system to list laboratory tests, imaging, medications, referrals, procedures, and follow-up. It did not explicitly offer no further workup. A human handed that form might feel nudged to fill it in. That is a fair methodological point, and it is uncomfortable, because the prompts in real clinical products look much like the study’s default. If the thoroughness default is partly an artifact of how the question is asked, every vendor asking it that way is shipping the artifact.

Prompting narrows the gap but never closes it

Recommended care cost as a multiple of the treating physician’s plan. Specialized systems shown at the midpoint of the reported 2.7x to 3.5x. Individual general-purpose systems ranged from 2.2x to 6.2x.

Choose the setting a vendor could ship

3.2xphysician cost
$314per visit at Medicare rates
$216AI Parsimony Gap per visit

Every system, general-purpose and specialized, ordered more than the physician under the standard instruction. This is the setting most products ship because it looks most thorough in a demonstration.

Values for the cost-aware and parsimonious modes are derived from the reported ratios (2.5x and 1.4x) applied to the $97 physician baseline.

11.4x is the best case

Still a 40 percent premium over clinicians already working inside a cost-constrained system. It is the ceiling of what prompt design achieved, not the expected result of a deployment.

2The cheap setting is not free

In 17 of 19 systems the parsimonious prompt lowered agreement with the physician’s recorded diagnosis. Spending less pushed the model toward different conclusions, and no vendor has yet shown a cost-aware mode is as accurate as its default.

3The prompt is a setting someone chooses

Today that someone is usually a product manager at the vendor, not the chief medical officer. What is enough is a value judgment that has to be encoded deliberately. Left to their defaults, the machines tilt toward more.

A single study of 200 vignettes would be a curiosity. What makes the NEJM AI paper a warning is that it landed in the middle of independent findings from insurers, pharmacologists, computer scientists, and the administrator of Medicare himself, pointing the same way. The two counterexamples matter too: they show the default can be overridden when cost is built into the objective.

Source (date)What was measuredFindingDirection
Aran et al., NEJM AI (Oct 15, 2026)Cost of care recommended by 24 AI systems vs. treating physicians, 200 visits$314 vs. $97 per visit (3.2x); testing added at every no-test visitMore
Blue Cross Blue Shield Association (Sep 24, 2026)Inpatient claims from 31 Blue plans covering 100 million people, 2024 to 2025 vs. 2023AI-assisted documentation of secondary conditions added $653 million; more intense coding added $942 million total, with no matching rise in treatmentMore
Sato, Ueda et al., Nature Health (Oct 2026)Drug product recommendations from five leading modelsOne product took 97.5% of first-choice picks despite 5.5 approved options on average; modeled acquisition cost up 1,019%More
Sense and Sensitivity, arXiv (2026)LLM triage vs. physician consensus, with perturbed inputsModels recommended unnecessary care more often than physicians, more so when patient tone or gender wording changedMore
Mehmet Oz, CMS Administrator (Sep 2026)Public remarks at an Oracle summitAI will be inflationary in the short term because it turbocharges existing billing systemsMore
Nori, Horvitz et al., Science (Oct 22, 2026)Microsoft diagnostic orchestrator on NEJM cases and 325 ED admissions80.3% vs. 46.8% accuracy on ED cases; asked far more history questions before ordering tests; cost built into the objectiveLess, with cost built in
Zeltzer et al., Annals of Internal Medicine (2025)AI intake recommendations vs. physician decisions in 461 virtual urgent care visitsAI rated optimal in 77% vs. 67%; avoided unjustified empirical antibioticsLess, narrow setting

The billing-side twin

If patients were truly sicker, treatment would rise with the diagnoses. The disconnect suggests the tools are finding billable conditions, not sicker patients.

The Blue Cross finding converts a research signal into a line item: the first large-scale measurement of AI’s effect on a payer’s spending found an increase, and the mechanism, documenting more, is the billing-side twin of the clinical-side thoroughness default.

Who pays the tax

  • Fee-for-service systems feel it as revenue first, then as payer pushback and lost credibility.
  • Risk-bearing providers and ACOs feel it as medical expense: $216 times a 10 percent acceptance rate is more than $20 per attributed visit.
  • Payers will tune prior authorization to the reflexive panels, which lands on clinicians as burden.
  • Patients pay twice: the test, then the cascade an incidental finding starts.

Thoroughness Tax Calculator

Thoroughness Tax = AI Parsimony Gap x annual encounters x share of encounters where the AI plan is accepted unfiltered

Per-visit excess values come from the study (standard) or are derived from its reported ratios (cost-aware 2.5x, parsimonious 1.4x) on the $97 physician baseline. Medications and downstream cascades are excluded, so the estimate is conservative. This is a planning magnitude, not a forecast.

$43,200,000Annual Thoroughness Tax
$216AI Parsimony Gap per accepted plan
$21.60Added cost per encounter touched
200,000Plans accepted unfiltered per year

Exposure across acceptance rates

Annual Thoroughness Tax at 2, 5, 10, and 20 percent unfiltered acceptance for the chosen encounter base and prompt setting; the current slider value is highlighted.

The lever you control

The ratio belongs to the vendor. The acceptance rate belongs to the organization. In a fee-for-service book of business the tax is revenue; in a capitated or risk-bearing contract it is loss. A system that holds both kinds of contracts will feel the thoroughness default as a split incentive inside its own walls.

The editorial accompanying the study called for randomized trials measuring long-run health and spending. That will take years, and 81 percent of physicians already use AI professionally. The ENOUGH Protocol is a governance framework for the interval. Score your organization’s readiness: check each statement that is true today.

The ENOUGH Protocol: Establish the baseline, Name the nothing option, Order-set guardrails, Utilization review, Governance of enough, Health outcomes
The ENOUGH Protocol, developed for the report as a leadership response to the thoroughness default.
0of 18

Not yet governed

Pilot triage bands (0 to 5, 6 to 10, 11 to 14, 15 to 18) are arithmetically even and unvalidated. Use them to prioritize work, not to grade people.

EEstablish the baseline
NName the “nothing” option
OOrder-set guardrails
UUtilization review
GGovernance of “enough”
HHealth outcomes

Ninety-day implementation

HorizonActionOwnerOutput
Days 1 to 30Pull baseline order rates and no-workup share; compute a local APG on a 200-visit sample for each deployed toolCMIO with analyticsBaseline report; local APG
Days 1 to 30Obtain vendor default prompts; confirm an explicit null-workup option; set parsimonious mode where supportedCMIO and vendor managementPrompt inventory
Days 31 to 60Tag AI-influenced orders in the EHR; build the acceptance-rate and APG dashboardClinical informaticsLive dashboard
Days 31 to 60Map Choosing Wisely and appropriate-use criteria to AI suggestion categories; enable alertsQuality and pharmacyGuardrail rules
Days 61 to 90Review first-quarter APG and cascade costs by payment model; set thoroughness policy per contract typeCMO, CFO, chief strategy officerBoard report; policy
OngoingEnroll in pragmatic evaluation; report outcomes and spending annuallyChief quality officerAnnual AI value review
  1. Aran D, Goldhaber-Fiebert JD, Many I, Nguyen A, Yang D, Shelly S, et al. Comparative Cost of Care Recommended by Large Language Models versus Treating Physicians. NEJM AI. 2026;3(11). doi:10.1056/AIoa2600537
  2. Markovitz AA, Chaitoff AM. Comprehensive or Wasteful? Clinical AI and the Up-front Workup. NEJM AI editorial. 2026. doi:10.1056/AIe2601260
  3. AI recommends vastly more expensive treatment than physicians, analysis suggests. BMJ. 2026;395:e101271.
  4. Wachter R. My Healthcare-AI Alignment Argument, Now With Data. Pattern Recognition. October 21, 2026.
  5. Landi H. Weekly Rundown: Doximity launches DoseCheck; AI recommends costlier care than doctors, study finds. Fierce Healthcare. October 30, 2026.
  6. AI’s recommendations cost 3x more than physician care plans. TechTarget Healthtech Analytics. October 27, 2026.
  7. Niasse A. AI tools generated nearly $1 billion in extra costs, Blue Cross insurers say. Reuters. September 24, 2026.
  8. Sato J, Ueda D, et al. Large language models show product name convergence in drug recommendations. Nature Health. 2026. doi:10.1038/s44360-026-00205-y
  9. Sense and Sensitivity: Benchmarking LLM Clinical Triage Recommendations with Physician Experts. arXiv:2609.38600. 2026.
  10. Nori H, Horvitz E, et al. Sequential diagnosis with language models (MAI-DxO). Science. October 22, 2026.
  11. Zeltzer D, et al. AI and physician recommendations in virtual urgent care. Annals of Internal Medicine. 2025.
  12. American Medical Association. 2026 Physician Survey on Augmented Intelligence. March 12, 2026.
  13. Becker’s Hospital Review, reporting on Wired. Dr. Oz promotes AI avatars in first meeting as CMS chief. April 2025. NPR. Dr. Oz pushes AI avatars as a fix for rural health care. February 14, 2026.
  14. AI is fueling a new battle over health care costs. Sinclair National Desk. September 2026.
  15. Shrank WH, Rogstad TL, Parekh N. Waste in the US Health Care System. JAMA. 2019;322(15):1501-1509.
  16. Mello MM, Chandra A, Gawande AA, Studdert DM. National costs of the medical liability system. Health Affairs. 2010;29(9):1569-1577.
  17. CDC National Center for Health Statistics. National Ambulatory Medical Care Survey, physician office visits.
  18. Barry J. LLMs Recommend More Care Than Doctors. Digital Health Wire. October 26, 2026.

The Thoroughness Tax, the AI Parsimony Gap, and the ENOUGH Protocol are original frameworks developed by Kelly Emrick for the full investigative report. Scenario estimates are illustrative planning magnitudes, not forecasts.

The Promise of Super Intelligence: Don’t Believe Everything It Says! · An investigative report by Kelly Emrick, DHSc, PhD, MBA, BSRT(ARRT)R · November 2026

kellyemrick.com · kellyemrick.org · kellyemrick.blog

The Promise of AI in Medicine

Site directory

Every resource, organized by practice.

Homekellyemrick.com