Instructions to use Miki-T/Mistral-7B-MK-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Miki-T/Mistral-7B-MK-Instruct with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1 (local merge: base + Miki-T/JARVIS-Mistral-Phase1a + Miki-T/JARVIS-Mistral-Phase1b adapters)")
model = PeftModel.from_pretrained(base_model, "Miki-T/Mistral-7B-MK-Instruct")

Transformers

How to use Miki-T/Mistral-7B-MK-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Miki-T/Mistral-7B-MK-Instruct")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Miki-T/Mistral-7B-MK-Instruct", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Miki-T/Mistral-7B-MK-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Miki-T/Mistral-7B-MK-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Miki-T/Mistral-7B-MK-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Miki-T/Mistral-7B-MK-Instruct

SGLang

How to use Miki-T/Mistral-7B-MK-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Miki-T/Mistral-7B-MK-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Miki-T/Mistral-7B-MK-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Miki-T/Mistral-7B-MK-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Miki-T/Mistral-7B-MK-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Miki-T/Mistral-7B-MK-Instruct with Docker Model Runner:
```
docker model run hf.co/Miki-T/Mistral-7B-MK-Instruct
```

Mistral-7B-MK-Instruct

Model ID: Miki-T/Mistral-7B-MK-Instruct

A QLoRA fine-tuned Mistral 7B instruction-following model for Macedonian and English. This is the final, complete release of a three-phase fine-tune: language foundation (500k MK corpus rows) → instruction following (134k MK instruction rows) → reasoning + bilingual robustness (~100k mixed MK/EN rows, this adapter). No further training is planned on this lineage.

This model is the language-reasoning component ("the thinking unit") of JARVIS, a locally-hosted AI assistant project — released under its own name here because it is a general-purpose Macedonian/English instruct model on its own merits, independent of JARVIS's voice I/O (Whisper STT, XTTS TTS), memory, and retrieval layers, which live in the surrounding application rather than in these weights.

Model Details

Model Description

Developed by: Miki Trajkovski
Model type: Causal Language Model (fine-tuned via QLoRA)
Base model: mistralai/Mistral-7B-v0.1
Language(s): Macedonian (mk) and English (en)
License: MIT
Finetuned from model: Miki-T/JARVIS-Mistral-Phase1b (Phase 1b merged adapter)
Adapter type: LoRA (Low-Rank Adaptation)

Model Architecture

Base: Mistral 7B (7 billion parameters)
Fine-tuning method: QLoRA (4-bit quantization + LoRA adapters)
LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.05
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Max sequence length: 1536 tokens

Model Sources

Repository: https://github.com/MikiTrajkovski/JARVIS
HuggingFace Model Card: https://huggingface.co/Miki-T/Mistral-7B-MK-Instruct
Previous phase: https://huggingface.co/Miki-T/JARVIS-Mistral-Phase1b

Uses

Direct Use

This model is designed for:

Macedonian and English reasoning — multiple-choice, commonsense, factoid QA, and grounded (passage-based) question answering
Bilingual response consistency — answers in the language the question was asked in, rather than defaulting to Macedonian
Multi-turn conversation — maintains context across dialogue turns
Final release — no further training is planned on this lineage; in the JARVIS project, downstream capabilities (law RAG, personality, memory) are added on top via prompting and retrieval, not further training

Example usage:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import AutoPeftModelForCausalLM

model = AutoPeftModelForCausalLM.from_pretrained(
    "Miki-T/Mistral-7B-MK-Instruct",
    device_map="auto",
    torch_dtype="auto",
)

model = model.merge_and_unload()

tokenizer = AutoTokenizer.from_pretrained("Miki-T/Mistral-7B-MK-Instruct")

prompt = "[INST] Одговори на следново прашање на македонски јазик.\nШто е вештачка интелигенција? [/INST]"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Out-of-Scope Use

Not for production: This is a research/learning model
Not domain-specific: Not trained on legal, medical, or technical Macedonian (the JARVIS project handles Macedonian law via a separate RAG layer, rather than further fine-tuning)
Not a chat assistant on its own: No personality layer is trained in — in the JARVIS project, personality is achieved through prompt engineering on top of this model, not training

Limitations and Bias

Known Limitations

Complex multi-step reasoning may still be limited at 7B scale
Training data reflects biases present in source instruction/benchmark datasets
Context window: 1536 tokens max during training (served at 4096 in production for RAG headroom)
English capability is a minority share of training (~16%) — sufficient to fix language drift, not equivalent to a native English instruct model
Evaluated on held-out splits from the same benchmark families used in training, not fully independent benchmarks

Recommendations

Always validate outputs for factual accuracy
For domain-specific legal/factual queries, pair with retrieval (RAG) rather than relying on parametric knowledge alone
Not suitable for specialized domain tasks without further fine-tuning or retrieval augmentation

Training Details

Training Data

Dataset	Rows	Source	Purpose
LVSTCK/nq_open-mk	25,000 (capped)	HuggingFace	MK factoid QA
LVSTCK/hellaswag-mk	10,000	HuggingFace	MK commonsense completion
LVSTCK/boolq-mk	3,230	HuggingFace	MK grounded yes/no QA (with passage)
LVSTCK/arc-easy-mk	2,336	HuggingFace	MK science reasoning
LVSTCK/arc-challenge-mk	1,132	HuggingFace	MK science reasoning (harder)
LVSTCK/winogrande-mk	1,227	HuggingFace	MK pronoun/commonsense resolution
LVSTCK/piqa-mk	1,798	HuggingFace	MK physical commonsense
LVSTCK/openbookqa-mk	960 (×2 repeat)	HuggingFace	MK science QA
nq_open, hellaswag, boolq, ai2_arc, winogrande, piqa, openbookqa (EN originals)	8,500	HuggingFace	English contrast pairs — fixes language drift
LVSTCK/sft-mk	12,000	HuggingFace	MK chat replay from Phase 1b
LVSTCK/ultrachat-sft-mk	6,000	HuggingFace	MK multi-turn chat replay
HuggingFaceH4/ultrachat_200k (sample)	5,000	HuggingFace	English chat
LVSTCK/macedonian-corpus-cleaned-dedup (continuation)	20,000	HuggingFace	MK long-form fluency anchor
Total	99,683		83.9% MK / 16.1% EN

Data format: Instruction-response pairs (JSONL), globally shuffled across all sources
Language: Macedonian (Cyrillic script) and English, mixed by design as contrast pairs
Grounded QA: boolq rows include the source passage, teaching the model to answer from provided context — relevant to downstream RAG use
Training mode: completion_only_loss=True — model trained on response tokens only

Hyperparameters

Parameter	Value
Learning rate	8e-5
Warmup ratio	10%
Learning rate scheduler	Cosine decay
Batch size	2
Gradient accumulation	8
Epochs	2
Optimization	AdamW (8-bit, paged)
Gradient checkpointing	Disabled

Training Regime

Hardware: NVIDIA RTX 5070 (12GB VRAM)
Framework: PyTorch + Hugging Face Transformers
Fine-tuning framework: TRL 1.7.0 SFTTrainer + PEFT LoRA
Precision: 4-bit quantization (NF4) + bfloat16 math

Speeds, Sizes, Times

Metric	Value
Training duration	2 days, 8 hours, 23 minutes
Total steps	12,462 (2 epochs)
Throughput	~153 tokens/second
Adapter size	~84 MB
Total VRAM used	~4.8 GB / 12 GB
Total tokens processed	31.1M tokens

Evaluation

Metrics

Metric	Epoch 1	Epoch 2 (final)
Loss	0.937	0.855
Perplexity	2.55	2.35
Gradient norm (avg)	—	1.621
Gradient norm (max)	—	4.406

Starting loss: 0.9061
Final loss: 0.6551
Best loss: 0.5772
Total improvement: 27.7%

Held-out evaluation (rows never seen in training — reserved before any repetition/shuffling, since the full LVSTCK/macedonian-llm-eval suite turned out to overlap the training benchmarks themselves) is tracked separately per language, comparing this checkpoint against the pre-Phase-1c baseline for MK reasoning accuracy, EN reasoning accuracy, and the EN Cyrillic-response drift rate.

Sample Output

Prompt: [INST] Answer the following question in English.\nWhat is water made of? [/INST]

Output: "Water is made of hydrogen and oxygen atoms, Sir — two hydrogen atoms bonded to one oxygen atom."

Interpretation: Responds in English for an English question (the Phase 1b baseline answered this same prompt in Macedonian) — the core behavior this phase was designed to fix.

Model Card Details

Environmental Impact

Factor	Value
Hardware	NVIDIA RTX 5070 (12GB VRAM)
Training duration	2 days, 8 hours
Power consumption (estimated)	~150W × 56 hours ≈ 8.4 kWh
Carbon emitted (estimated)	~4-6 kg CO2e
Cloud provider	None (local desktop GPU)

Compute Infrastructure

CPU: AMD Ryzen 7 7800X3D (8-core)
GPU: NVIDIA RTX 5070 (12GB GDDR6X VRAM)
RAM: 32GB DDR5
OS: Windows 11
CUDA: 12.x

Software

PyTorch: 2.11.0+cu128
Transformers: 5.12.1
PEFT: 0.19.1
TRL: 1.7.0
Bitsandbytes: latest (paged AdamW 8-bit)

How to Use

Load the Model

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
import torch

model = AutoPeftModelForCausalLM.from_pretrained(
    "Miki-T/Mistral-7B-MK-Instruct",
    device_map="auto",
    torch_dtype=torch.float16,
)

model = model.merge_and_unload()

tokenizer = AutoTokenizer.from_pretrained("Miki-T/Mistral-7B-MK-Instruct")

Generate Text

prompt = "[INST] Одговори на следново прашање на македонски јазик.\nЗошто небото е сино? [/INST]"
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512)
input_ids = inputs["input_ids"].to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=200,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

Citation

@misc{trajkovski2026mistral7bmkinstruct,
  author = {Trajkovski, Miki},
  title = {Mistral-7B-MK-Instruct: A Macedonian and English Instruction-Following Model},
  year = {2026},
  publisher = {Hugging Face Hub},
  howpublished = {\url{https://huggingface.co/Miki-T/Mistral-7B-MK-Instruct}},
}

Acknowledgments

Base model: Mistral AI (Mistral 7B v0.1)
Fine-tuning: Hugging Face TRL + PEFT libraries
Data: LVSTCK (Nikola Dobrota) — Macedonian NLP datasets, plus their original English benchmark sources (Google Research, AllenAI, Rowan Zellers, Yonatan Bisk, HuggingFaceH4)
Inspiration: Tony Stark's JARVIS from Marvel

License

This model is provided under the MIT License, same as the JARVIS project.

Model Card Contact

Author: Miki Trajkovski
GitHub: https://github.com/MikiTrajkovski/JARVIS
HuggingFace: https://huggingface.co/Miki-T

Framework Versions

PEFT: 0.19.1
Transformers: 5.12.1
PyTorch: 2.11.0+cu128
TRL: 1.7.0
CUDA: 12.x

Downloads last month: 23

Model tree for Miki-T/Mistral-7B-MK-Instruct

Base model

mistralai/Mistral-7B-v0.1

Adapter

(2459)

this model

Miki-T
/

Mistral-7B-MK-Instruct