CodeLlama Fine-Tuning for RTL Code Generation
This repository contains scripts, datasets, and documentation for fine-tuning CodeLlama-7B-Instruct model for Verilog/SystemVerilog RTL code generation.
📋 Overview
This project fine-tunes CodeLlama-7B-Instruct to generate synthesizable Verilog/SystemVerilog code for hardware design tasks, specifically focusing on FIFO implementations.
🎯 Features
- CodeLlama-7B-Instruct Fine-tuning with LoRA
- Chat Template Format support
- Dataset Processing and validation scripts
- Training Scripts with checkpoint resume capability
- Inference Scripts for testing fine-tuned models
- Comprehensive Documentation and guides
📁 Repository Structure
codellama-migration/
├── datasets/ # Training datasets
│ ├── raw/ # Original datasets
│ └── processed/ # Processed and formatted datasets
│ ├── split/ # Train/val/test splits (original format)
│ └── split_chat_format/ # Train/val/test splits (chat format)
├── scripts/
│ ├── training/ # Training scripts
│ ├── inference/ # Inference scripts
│ └── dataset_split.py # Dataset splitting utility
├── Documentation/ # All .md documentation files
└── Scripts/ # Utility scripts
🚀 Quick Start
Prerequisites
- Python 3.8+
- CUDA-capable GPU (recommended)
- HuggingFace transformers library
- PyTorch
Installation
pip install transformers torch peft accelerate bitsandbytes
Training
bash start_training_chat_format.sh
Inference
python3 scripts/inference/inference_codellama.py \
--mode local \
--model-path training-outputs/codellama-fifo-v2-chat \
--base-model-path models/base-models/CodeLlama-7B-Instruct \
--prompt "Your prompt here"
📊 Dataset
The dataset contains 94 samples of FIFO implementations in Verilog format. It's split into:
- Training: 70 samples (75%)
- Validation: 9 samples (10%)
- Test: 15 samples (15%)
📚 Documentation
- MIGRATION_PROGRESS.md - Overall migration tracking
- TRAINING_COMPLETE.md - Training completion details
- COMPARISON_REPORT.md - Expected vs Generated comparison
- FILE_INVENTORY.md - Complete file listing
🤖 Model Information
Base Model: CodeLlama-7B-Instruct
Fine-tuning Method: LoRA (Low-Rank Adaptation)
LoRA Rank: 48
LoRA Alpha: 96
Trainable Parameters: ~120M (3.31% of total)
📝 License
This project is for internal use by Elinnos Systems Pvt Limited.
👥 Contributors
Elinnos Systems Pvt Limited
🔗 Links
- Organization: https://huggingface.co/Elinnos
- Base Model: https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf
Note: Model weights are not included in this repository. Fine-tuned models are stored separately.