aviici4cs/MedLam

Model Card for Model ID

This Llama 3.1 8B Instruct Model was obtained using the Unsloth library. It is fine-tuned using the LoRA PEFT approach to give better responses to medical questions.

Model Details

Model Description

MedLam explores the fine-tuning of LLMs using a Medical QA dataset to improve their performance for domain-specific tasks. By combining state-of-the-art natural language processing techniques and medical data, MedLam aims to deliver an effective and intuitive medical assistant.

• Developed by: Avishek Choudhury
• Model type: Transformer based auto-regressive language model
• Language(s) (NLP): English
• License: MIT
• Finetuned from model Meta Llama 3.1 8B Instruct: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

Model Sources

• Repository: https://github.com/avishek04/MedLam/tree/main
• Paper: https://arxiv.org/abs/2106.09685

Uses

This model was developed only for research and learning purposes and is not meant for commercialization as a final product.

• Assisting medical students in understanding complex topics.
• Supporting professors in teaching and curriculum design.
• Helping doctors with quick and accurate responses to medical questions.

How to Get Started with the Model

Use the code below to get started with the model.

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "aviici4cs/MedLam")

Training Procedure

1. Data Preparation:
   • Preprocessed Medical QA datasets for training and evaluation.
2. Model Fine-Tuning:
   • Applied LoRA PEFT (Parameter-Efficient Fine-Tuning) to train LLMs on domain-specific data.
3. Performance Validation:
   • Analyzed and compared models across multiple configurations to determine optimal hyperparameters.
4. Deployment (Future Scope):
   • Aimed at integrating the fine-tuned model into a user-friendly medical assistant platform.

Hardware

Google Colab

Software

• Programming Languages: Python
• Frameworks: Unsloth Library, PyTorch, Hugging Face Transformers
• Machine Learning Techniques: LoRA PEFT, Deep Learning
• Tools: NumPy, Pandas, Sci-kit Learn

Model Card Contact

• https://www.linkedin.com/in/avishekchoudhury/
• https://github.com/avishek04
• https://avishek04.github.io/

Framework versions

• PEFT 0.13.2