Google’s medical chatbot Med-PaLM 2 generates accurate and safe answers to medical questions. According to a Wall Street Journal report, the Mayo Clinic is among the hospitals that have begun testing the large language model. 

Google Health researcher Dr. Alan Karthikesalingam talked about Med-PaLM 2 earlier this year
Google Health researcher Dr. Alan Karthikesalingam talked about Med-PaLM 2 earlier this year; Photo Credit: YouTube/Google

Google’s Med-PaLM 2

Med-PaLM 2 has been trained on questions and answers from medical licensing exams and a curated set of medical expert demonstrations. In addition to answering health-related questions, the AI chatbot can perform healthcare tasks like summarizing documents or organizing reams of health data. 

The platform scored at an expert test-taker level performance on the MedQA dataset of US Medical Licensing Examination example questions, and it passed the MedMCQA dataset comprising Indian AIIMS and NEET medical examination questions. Additionally, according to research that Google made public in May, physicians who reviewed Med-PaLM 2’s answers to over 1,000 consumer medical questions preferred the system’s responses to those produced by doctors along eight out of nine categories for evaluation.

While this performance is impressive, Google executives note that the chatbot is not ready to be brought to the public yet. For example, in the research study, physicians found more inaccuracies and irrelevant information in answers provided by Google’s Med-PaLM and Med-PalM 2 than those of other doctors.

As a result, Med-PaLM 2 is continually being assessed against multiple criteria, including scientific consensus, medical reasoning, bias, and likelihood of possible harm, which were evaluated by clinicians and non-clinicians from a range of backgrounds and countries. 

Example that reflects early exploration of Med-PaLM 2's future capabilities

Example that reflects early exploration of Med-PaLM 2’s future capabilities; Photo Credit: GoogleAccording to a statement from the company, “In building Med-PaLM 2, we’ve been focused on safety, equity, and evaluations of unfair bias…By working with a handful of trusted healthcare organizations early on, we’ll learn more about what can be achieved, and how this technology can safely advance.”

AI in Healthcare

Google has been integrating AI into other facets of the healthcare industry in the past few years. For example, the AI-enabled Claims Acceleration Suite is designed to streamline health insurance processes. The platform creates operational efficiencies and reduces administrative burdens by converting unstructured data into structured data that helps experts make faster decisions.

On the clinical side, Google’s Medical Imaging Suite is an AI-assisted diagnosis technology that improves cervical cancer diagnoses and predicts metastasis in patients with prostate cancer. Google Health also recently partnered with iCAD to improve breast cancer screening with AI.

Other companies are similarly working to integrate their own AI technology into medicine. Notably, Microsoft, the largest investor and closest business partner in OpenAI, has partnered with health software company Epic. Using the algorithms behind ChatGPT, the three companies are working to build tools that can automatically draft messages to patients. 

Hospitals have begun testing OpenAI’s GPT algorithms through Microsoft’s cloud service with tasks like summarizing doctors’ notes or generating reminders. Microsoft hosts and controls the AI systems in these cases.