John Snow Labs Surpasses GPT-4 and Med-PaLM2 in Medical LLM Accuracy

John Snow Labs achieves new state-of-the-art medical LLM accuracy benchmarks, surpassing GPT-4 and Med-PaLM2, and commits to delivering novel, responsible, production-ready models for healthcare use cases.

AI Unveiled: Beyond Boundaries of Code and Consciousness

John Snow Labs Achieves New State-of-the-Art Medical LLM Accuracy Benchmarks

In a groundbreaking achievement, John Snow Labs has surpassed hundreds of other high-performing models, including GPT-4 and Med-PaLM2, on the Open Medical LLM leaderboard. This milestone reflects the company’s commitment to delivering novel, responsible, production-ready models.

New Milestones in Medical LLM Accuracy

John Snow Labs has achieved three new milestones in medical LLM accuracy:

A Medical LLM which achieves 87.35 on the same reproducible test harness of the leaderboard, outperforming models such as Med-PaLM2, GPT-4, OpenBioLLMLlama, MedLlama, Orpo-Med, and all others.

A Medical LLM with just 7 billion parameters which outperforms all previous models of that size and is the first 7B model to outperform GPT-4 on PubMedQA (78.4 vs. 75.2).

A Medical LLM with just 3 billion parameters which outperforms all current models of that size by more than 12 points, while still being able to run on a mobile device.

Medical LLM Accuracy

The Need for Accuracy in Medical LLMs

Recent research shows that lack of accuracy was the most concerning roadblock to Generative AI adoption. Despite this, a majority of GenAI projects have not yet been tested for LLM requirements. The same survey indicated a strong preference for small, task-specific language models, with 54% of respondents from large companies using healthcare-specific task-specific language models.

John Snow Labs’ Commitment to Responsible AI

John Snow Labs addresses the need for top accuracy and targeted models optimized for healthcare use cases. The company’s Healthcare NLP & LLM subscription provides access to these models for production use, while also providing continuous updates and new releases, guaranteeing customers remain state-of-the-art over time.

“It’s a great responsibility and honor to provide novel, state-of-the-art, production-ready models to the global healthcare AI community,” said David Talby, CTO, John Snow Labs. “We didn’t give these new models fancy names because we’ll have better ones next week. That’s been the essence of our work for the past seven years, and it’s what makes John Snow Labs the most comprehensive medical language understanding solution on the market.”

![John Snow Labs](_search_image John Snow Labs) John Snow Labs

John Snow Labs will continue releasing new software every two weeks. Coming soon are larger models, larger context windows, new medical text summarization models (currently beating GPT-4 2:1 on blind tests by clinicians), medical speech-to-text models (for both layman and clinician speak), and medical text translation models.