Microsoft's AI Diagnostic Tool: Claims and Cost Questions

Microsoft's new AI diagnostic system, MAI-DxO, claims to significantly outperform human doctors on complex medical cases, according to a study published in the journal Science. The tool is designed to coordinate multiple AI agents to simulate a panel of specialists debating a diagnosis. While the company frames this as a step toward 'medical superintelligence,' the study's own details raise questions about cost and its applicability in common medical settings like emergency rooms.

The study's numbers show the system achieving an 83% accuracy rate on complex cases from the New England Journal of Medicine. In comparison, primary care doctors and general internists working individually with reference materials were correct 36% of the time. A key caveat noted in the report is that such difficult cases are typically handled by specialists or teams, who were not part of this comparison. Furthermore, the cost per case remains a significant factor. For example, running the system with OpenAI's GPT-5 on a complex case costs an estimated $8,400, and using the MAI-DxO framework reduces that cost by only about $3,000 while providing a modest accuracy increase.

Mustafa Suleyman, Microsoft AI's CEO, called the results a step toward 'medical superintelligence.' He stated, 'We’re just true believers,' and announced a partnership with the Mayo Clinic to train a foundation model for healthcare, with release planned for next year. He believes this model will enable the company to 'make really enormous leaps here,' and says DxO is part of the foundational infrastructure Microsoft is building to see real breakthroughs in healthcare. Suleyman predicts a 'coding style breakthrough for medical records' in 2027, allowing for accurate predictions on how a record might unfold.

The limitations in the data are notable, particularly for emergency department cases. The study shows little to no improvement in accuracy or cost when using the DxO system compared to using a standard AI model in that setting. With GPT-5.5, for instance, both methods scored about 85% accuracy at a similar cost for emergency cases. The system's utility appears to diminish with more capable AI models, as the performance gap between weaker and stronger models is larger than the boost provided by the orchestration layer.

To test the technology in a real-world environment, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston to evaluate DxO's clinical reasoning capabilities across different hospital settings. Patients will receive standard treatment, while consenting participants will also engage with DxO. The company and its partners will then compare the AI's recommendations against those of human doctors. Microsoft's vice president of Health at Microsoft AI, Dominic King, stated, 'This technology is ready for real-world evaluations.'

The publication of this study places Microsoft in direct competition with other tech giants making similar claims. Google and OpenAI have also published studies suggesting their AI can match or outperform doctors in medical contexts. The push to develop medical superintelligence is defined by Microsoft as 'the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care,' a goal that remains unproven in real-world deployment.