Researchers Introduce OpenBioLLM-Llama3-70B & 8B: Groundbreaking Medical-Domain LLMs

Researchers @Ankit Pal (Aaditya Ura) from Saama AI Labs Introduce OpenBioLLM-Llama3-70B & 8B: Groundbreaking Medical-Domain LLMs

These new open-source LLMs set the bar for medical language models, outperforming commercial giants like GPT-4, Gemini, Meditron-70B, Med-PaLM-1, and Med-PaLM-2.

Key Achievements:

• OpenBioLLM-70B delivers record-breaking performance, surpassing the capabilities of similarly-sized models.
• OpenBioLLM-8B surpasses GPT-3.5, Gemini, and Meditron-70B!

What's Next:

• Expanded medical domain coverage
• Longer context support
• Improved benchmarks
• Multimodal capabilities

Check out the Medical-LLM Leaderboard: https://huggingface.co/spaces/openlifescienceai/open_medical_llm_leaderboard

Fine-tuning Details ???

The fine-tuning process was conducted in two phases to optimize the model's performance:

• Phase 1: Fine-tuned using the LLama-3 70B & 8B models as the base ??
• Phase 2: Utilized the Direct Preference Optimization: Your Language Model is Secretly a Reward Model (DPO) ??

For more in-depth information on the training and QLora parameters, please refer to the model page. ??

Dataset ????

??Curating the custom dataset was a time-consuming process that spanned over ~4 months.

??they diligently collected data, collaborated with medical experts to review its quality ??, and filtered out subpar examples.

??To enhance the dataset's diversity,

1. they incorporated 3k healthcare topics and
2. more than 10+ medical subjects. ????
3. they are currently working on releasing a subset of the dataset alongside our upcoming official paper. ????

Results ????

OpenBioLLM-70B showcases remarkable performance, surpassing larger models such as GPT-4, Gemini, Meditron-70B, Med-PaLM-1, and Med-PaLM-2 across 9 diverse biomedical datasets.

Despite its smaller parameter count compared to GPT-4 & Med-PaLM, it achieves state-of-the-art results with an impressive average score of 86.06%. ??

To gain a deeper understanding of the results, we also evaluated the top subject-wise accuracy of 70B.

Models :

You can download the models directly from Huggingface today. -

70B : https://huggingface.co/aaditya/OpenBioLLM-Llama3-70B…

Use Cases & Examples

???**Below results are from the quantized version of OpenBioLLM-70B

Summarize Clinical Notes

OpenBioLLM-70B can efficiently analyze and summarize complex clinical notes, EHR data, and discharge summaries, extracting key information and generating concise, structured summaries

Answer Medical Questions

OpenBioLLM-70B can provide answers to a wide range of medical questions.

Clinical Entity Recognition

OpenBioLLM-70B can perform advanced clinical entity recognition by identifying and extracting key medical concepts, such as diseases, symptoms, medications, procedures, and anatomical structures, from unstructured clinical text. By leveraging its deep understanding of medical terminology and context, the model can accurately annotate and categorize clinical entities, enabling more efficient information retrieval, data analysis, and knowledge discovery from electronic health records, research articles, and other biomedical text sources. This capability can support various downstream applications, such as clinical decision support, pharmacovigilance, and medical research.

Biomarkers Extraction

Classification

OpenBioLLM-70B can perform various biomedical classification tasks, such as disease prediction, sentiment analysis, medical document categorization

De-Identification

OpenBioLLM-70B can detect and remove personally identifiable information (PII) from medical records, ensuring patient privacy and compliance with data protection regulations like HIPAA.

OpenBioLLM-8B and various models in Hugging Face