Microsoft revealed a slew of new artificial intelligence capabilities for healthcare organizations Thursday, including a product to help companies to build their own AI agents.
The technology giant also announced foundation models for medical imaging and a healthcare data analysis platform, as well as details about its plans to build an AI documentation product geared toward nurses.
Healthcare organizations have shown increased interest in adopting AI tools, even as some experts and lawmakers raise questions about its safe and equitable use. Tech companies say the products have potential to help providers manage their workloads and alleviate burnout.
“We’re at an inflection point. AI breakthroughs are changing, augmenting how we work and live,” Kees Hertogh, vice president for healthcare and life sciences product marketing, said during a press briefing. “The integration of AI into healthcare has significantly enhanced patient care and is rekindling the joy of practicing medicine for clinicians.”
Healthcare AI agents
Microsoft’s agent service would allow companies to create AI tools with pre-built templates and data sources that could be used for appointment scheduling, clinical trial matching and patient triage, Hertogh said.
The service is currently in public preview, which allows wider access to the tools and lets organizations give feedback on the product, according to a spokesperson.
In one example, a doctor could ask an AI agent to find clinical trials for a 55-year-old patient with diabetes and interstitial lung disease.
The tools could also be geared toward patients. Cleveland Clinic used the agent service to build tools that allow patients to ask health questions and navigate the health system’s services, according to a spokesperson.
Microsoft’s Copilot Studio for building AI agents
Courtesy of Microsoft
The product allows organizations to build agents with healthcare-specific features using intelligence from credible sources, which aims to improve safety, said Hadas Bitran, partner general manager of health AI at Microsoft Health and Life Sciences.
The tech company is also releasing additional application programming interfaces, or APIs, in private preview that can help verify the model’s output, like features for detecting omissions or linking answers to grounding data, she said.
Imaging models
Microsoft also revealed foundation models, or systems built on broad datasets that can be used for a number of tasks, focused on medical imaging.
The models, developed with partners like health system Providence and digital pathology company Paige.ai, would allow healthcare organizations to build their own AI tools without the hefty data and computing resources needed to build them from scratch.
The foundation models include MedImageInsight, which allows image analysis that can be used for automatically sending scans to specialists or flagging abnormalities for review.
MedImageParse is aimed at image segmentation, which could be used for segmenting tumors or outlining organs at risk before radiotherapy for cancer patients.
The third model, CXRReportGen, creates reports based on chest X-rays, which Microsoft said could speed image analysis and improve radiologists’ diagnostic accuracy.
Healthcare data analysis platform
The tech giant said healthcare-specific data tools are now generally available in Microsoft Fabric, the company’s analytics product. The platform allows organizations to ingest, store and analyze health data.
“The analytical enrichments can help them enhance reporting with details like patients’ geographic distribution, age, gender and more, as well as the state of patient outcomes and satisfaction,” Hertogh said.
Organizations will also be able to use other data types in public preview, like conversational information from Microsoft’s DAX Copilot AI documentation tool, public social determinants of health information and claims data from the CMS. Also in public preview, the tech giant released healthcare security templates in Microsoft Purview, the company’s data security and governance offering.
Nursing documentation
Microsoft is working with electronic health record vendor Epic and several health systems — like Advocate Health, Northwestern Medicine, Duke Health and Stanford Healthcare — to develop an AI documentation tool for nurses.
Microsoft-owned Nuance Communications made its automated clinical documentation tool DAX Copilot generally available last year. The tool records conversations between clinicians and patients and drafts clinical summaries.
The nurse-focused product has been deployed at multiple customer sites, according to Mary Varghese Presti, vice president of portfolio evolution and incubation at Microsoft Health and Life Sciences. The tool uses ambient voice technology to automatically draft flowsheets, or forms that collect patient data, for nurse review.
“For nurses, documentation is really more data entry. Their back is to the patient, their faces to the computer,” Presti said. “[…] We aspire to enable nurses to be eyes-free and hands-free in their documentation.”
