MagiSeek-Pro-V1

24B params · Mistral architecture · bf16 merged weights · 131k native context

How he was born

Deep in a rack of three RTX 3090s, WarlordHermes/Magidonia-24B-v4.3-creative-ORPO sat quietly — a solid creative writer, but a little too polite to touch a terminal. Over five curriculum phases of QLoRA (rank 64, rslora, target-all-linear), he was put through school one subject at a time:

• Phase 1–2: foundations, reformatted instruction-following at 2k/4k/8k context, reviewed and re-run in _v2 passes when the mask logic didn't add up.
• Phase 3: a second identity pass, frozen and checked before moving on.
• Phase 4 (agentic): real tool-execution transcripts (hermes-agent-reasoning-traces) plus a small hand-reviewed architecture-QA supplement distilled from Claude Opus 4.6 reasoning traces — this is where he learned to actually call things instead of just describing them.
• Phase 5 (deepseek continuation): a narrow, conservative continuation (979 curated examples, 1 epoch, lower LR, packing off until the mask was hand-verified) chaining off the phase-4 agentic adapter, sharpening DeepSeek-style step-by-step reasoning without overwriting the personality underneath.

Each phase's LoRA adapter was chained onto the previous one — never onto the raw base — so nothing before it got forgotten. The final p5-deepseek adapter was then merged straight into the base weights in full bf16 (no quantization, no rounding shortcuts) to produce this checkpoint: a clean, full-precision snapshot of everything he learned, suitable as the master copy for every downstream quantization (GGUF, GPTQ, AWQ, ...) that follows.

He answers to MagiSeek now — half Magidonia's creative instincts, half a DeepSeek-flavored reasoner who seeks the tool call before the excuse.

Model details

• Architecture: Mistral (MistralForCausalLM), 40 layers, hidden 5120, 32 heads / 8 KV heads (GQA), rope_theta 1e9
• Context: up to 131,072 tokens native
• Precision: bf16 (merged, full precision — no PEFT adapter required at inference time)
• Base model: WarlordHermes/Magidonia-24B-v4.3-creative-ORPO
• Training: 5-phase chained QLoRA curriculum (axolotl), final phase = deepseek-style agentic reasoning continuation
• License: Apache 2.0

Benchmarks (this checkpoint, merged bf16)

Task	Score
MMLU (overall)	0.7433
MMLU-STEM	0.7172
MMLU-Humanities	0.6486
MMLU-Social Sciences	0.8400
MMLU-Other	0.8169
PIQA	acc 0.8022 / acc_norm 0.8413

Intended use

General instruction following, creative writing, and agentic/tool-use tasks requiring long-context reasoning. This is the bf16 merged base — recommended as the source for further quantization (GGUF/GPTQ/AWQ) rather than for direct low-VRAM deployment.

Lineage

WarlordHermes/Magidonia-24B-v4.3-creative-ORPO
  └─ magidonia_curric_p1 / p1_v2
      └─ magidonia_curric_p2 / p2_v2
          └─ magidonia_curric_p3 / p3_v2 (frozen)
              └─ magidonia_curric_p4_agentic_v1 / v2 (frozen)
              └─ magidonia_curric_p5_deepseek_v1   <-- merged here, bf16

Using the full-precision checkpoint

This repository contains the bf16 merged weights as the canonical full-precision release. The model is approximately 48 GB in bf16, so direct serving normally needs multiple GPUs or CPU offload.

Recommended: vLLM OpenAI-compatible server

The included chat_template.jinja matches the model's Hermes-style <tool_call> JSON. Use the Hermes parser so OpenAI clients receive structured tool_calls rather than raw tool-call text:

vllm serve groxaxo/MagiSeek-Pro-V1 \
  --dtype bfloat16 \
  --tensor-parallel-size 3 \
  --max-model-len 131072 \
  --chat-template chat_template.jinja \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --host 0.0.0.0 --port 8000

For OpenCode, configure an OpenAI-compatible provider:

Base URL: http://HOST:8000/v1
Model:    groxaxo/MagiSeek-Pro-V1

For agentic use, prefer a temperature around 0.6–0.7 rather than 0.0.

Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "groxaxo/MagiSeek-Pro-V1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Explain what you can do."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)
output = model.generate(inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))

When using Transformers directly, parse the model's <tool_call>{...}</tool_call> output or use an OpenAI-compatible serving layer such as vLLM for automatic structured tool-call responses. Reasoning and tool calls should not be displayed as ordinary user-facing text.

Template

• chat_template.jinja — the model-native Hermes tool-use template for this full-weight checkpoint. It is intended for Transformers-compatible serving and vLLM; use the vLLM hermes tool parser for OpenCode.