Instructions to use texdata/Sokol-4B-SLO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use texdata/Sokol-4B-SLO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="texdata/Sokol-4B-SLO")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("texdata/Sokol-4B-SLO")
model = AutoModelForMultimodalLM.from_pretrained("texdata/Sokol-4B-SLO", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use texdata/Sokol-4B-SLO with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf texdata/Sokol-4B-SLO:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf texdata/Sokol-4B-SLO:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf texdata/Sokol-4B-SLO:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf texdata/Sokol-4B-SLO:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf texdata/Sokol-4B-SLO:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf texdata/Sokol-4B-SLO:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf texdata/Sokol-4B-SLO:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf texdata/Sokol-4B-SLO:Q4_K_M

Use Docker

docker model run hf.co/texdata/Sokol-4B-SLO:Q4_K_M

LM Studio
Jan

vLLM

How to use texdata/Sokol-4B-SLO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "texdata/Sokol-4B-SLO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "texdata/Sokol-4B-SLO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/texdata/Sokol-4B-SLO:Q4_K_M

SGLang

How to use texdata/Sokol-4B-SLO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "texdata/Sokol-4B-SLO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "texdata/Sokol-4B-SLO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "texdata/Sokol-4B-SLO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "texdata/Sokol-4B-SLO",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use texdata/Sokol-4B-SLO with Ollama:
```
ollama run hf.co/texdata/Sokol-4B-SLO:Q4_K_M
```

Unsloth Studio

How to use texdata/Sokol-4B-SLO with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for texdata/Sokol-4B-SLO to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for texdata/Sokol-4B-SLO to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for texdata/Sokol-4B-SLO to start chatting

How to use texdata/Sokol-4B-SLO with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf texdata/Sokol-4B-SLO:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "texdata/Sokol-4B-SLO:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use texdata/Sokol-4B-SLO with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf texdata/Sokol-4B-SLO:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default texdata/Sokol-4B-SLO:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use texdata/Sokol-4B-SLO with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf texdata/Sokol-4B-SLO:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "texdata/Sokol-4B-SLO:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use texdata/Sokol-4B-SLO with Docker Model Runner:
```
docker model run hf.co/texdata/Sokol-4B-SLO:Q4_K_M
```

Lemonade

How to use texdata/Sokol-4B-SLO with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull texdata/Sokol-4B-SLO:Q4_K_M

Run and chat with the model

lemonade run user.Sokol-4B-SLO-Q4_K_M

List all available models

lemonade list

Sokol — Slovenian Assistant LLM (4B, on-device)

Sokol (falcon) is a small 4B Slovenian assistant built for fully-local, on-device use (desktop via GGUF / llama.cpp / LM Studio). It chats in fluent Slovenian, translates EN↔SL, reasons through tasks (<think>), and makes native tool / function calls (Qwen3.6 XML format).

It is a text LLM, not a speech model — pair it with your own ASR + TTS to build a local voice assistant (it routes <think> to reasoning_content, so reasoning is never spoken).

This single repo ships both the full-precision weights (safetensors) and quantized GGUFs (under gguf/), so you can run it with transformers, or fully local with llama.cpp / LM Studio.

What it is

Base: SicariusSicariiStuff/Qwen3.5-4B_Abliterated (dense, qwen3_5 arch, natively supported in transformers ≥ 5.6; multimodal-capable).
Phase 2 — CPT (full-parameter): continued pretraining on ~1.78 B tokens of Slovenian text for language adaptation (full-FT, DeepSpeed ZeRO-2, 1 epoch).
Phase 3 — SFT (LoRA): chat + translation + tool-calling + reasoning + identity (~250 k examples).
Deployment: GGUF (desktop, llama.cpp / LM Studio) and full safetensors (transformers).

Files in this repo

path	format	use
`*.safetensors` (+ `config.json`, tokenizer, `chat_template.jinja`)	full BF16	transformers, further fine-tuning, requantization
`gguf/sokol-4b-slo-Q8_0.gguf`	GGUF Q8_0	highest-quality local (largest)
`gguf/sokol-4b-slo-Q6_K.gguf`	GGUF Q6_K	near-lossless, smaller
`gguf/sokol-4b-slo-Q5_K_M.gguf`	GGUF Q5_K_M	balanced
`gguf/sokol-4b-slo-Q4_K_M.gguf`	GGUF Q4_K_M	smallest, fastest

GGUFs are exported with --no-mtp (the multi-token-prediction head is dropped), which is required for the qwen3_5 GDN-hybrid architecture to load cleanly in current llama.cpp.

Run it (GGUF, llama.cpp)

# thinking + native tool calls; --reasoning-budget routes <think> to reasoning_content
llama-server -m gguf/sokol-4b-slo-Q5_K_M.gguf --jinja --reasoning-budget 8192 \
             -c 8192 --temp 0.4 --top-p 0.9

Or open the .gguf directly in LM Studio. For clean, non-thinking output pass --reasoning-budget 0 (useful when driving TTS in a voice stack, so <think> is never spoken).

Capabilities

Slovenian chat — fluent, formal Slovenian conversation.
Translation — bidirectional EN↔SL.
Reasoning — emits <think> traces (distilled from a large Qwen3.8-Max teacher); pass enable_thinking=False for direct answers.
Tool / function calling — native Qwen3.6 XML, e.g. <tool_call><function=get_weather><parameter=city>Ljubljana</parameter></function></tool_call>.
Identity — presents as Sokol, a local Slovenian assistant.

Training data (provenance)

CPT — Slovenian text (~1.68 M docs / ~1.78 B tokens)

dataset	docs	source	license
`sl_wiki`	160,539	Slovenian Wikipedia	CC BY-SA
`sl_fineweb`	1,523,000	FineWeb2 (sl subset)	ODC-BY

SFT — ~250 k examples

dataset	examples	what	source
`sft_chat_sl`	79,324	Slovenian conversation	`cjvt/GaMS-Nemotron-Chat`
`sft_translate`	120,000	bidirectional EN↔SL translation	OPUS-derived
`sft_toolcall_hermes`	5,086	tool calling	Hermes function-calling
`sft_toolcall_apigen_sokol`	4,829	tool calling (length-filtered)	APIGen-MT
`sft_toolcall_toolace`	9,171	tool calling	ToolACE
`qwen38max_reasoning_sl`	31,255	reasoning `<think>`	distilled from Qwen3.8-Max
`identity_sokol_sl`	56	Sokol identity	hand-written

Nothing is truncated: all SFT examples are ≤ 7942 content tokens (cutoff 8192).

Evaluation

Slovenian-LLM-Eval — acc_norm (`cjvt/slovenian-llm-eval`, 200/task)

Sokol (CPT + SFT) vs the abliterated base, same eval, via flywheel-sl:

task	base Qwen3.5-4B	Sokol	Δ
arc_easy	0.565	0.610	+0.045
arc_challenge	0.425	0.450	+0.025
hellaswag	0.460	0.550	+0.090
piqa	0.605	0.665	+0.060
openbookqa	0.400	0.445	+0.045
winogrande	0.525	0.590	+0.065
boolq	0.825	0.825	+0.000
AVG	0.544	0.591	+0.047

Wins 6/7 tasks, ties 1, no regressions. CPT also halved Slovenian perplexity (6.996 → 3.438 on held-out SL text, −50.9%).

Judge track — chat quality (1–5)

Open-ended answers to 32 general Slovenian questions (8 categories), judged by a stronger 27B Slovenian judge model on correctness / helpfulness / language:

metric	base Qwen3.5-4B	Sokol	Δ
jezik (fluency)	4.88	4.97	+0.09
koristnost (helpfulness)	4.84	4.81	−0.03
pravilnost (correctness)	4.78	4.63	−0.16

Sokol clearly wins language fluency and tool-calling (per-category +0.67) and the objective acc_norm; the small aggregate correctness dip comes almost entirely from two n=4 categories (grammar, instruction-format), i.e. 1–2 questions each — noise at this eval size, not a systematic regression. (v1 is 32 questions; per-category deltas need a larger v2 to be reliable.)

⚠️ Disclaimers & license

General assistant. Outputs may be incorrect or fabricated; not professional advice. Verify anything important.
Mixed data-license provenance (Wikipedia CC BY-SA, FineWeb2 ODC-BY, GaMS-Nemotron-Chat, OPUS, Hermes/APIGen/ToolACE, Qwen3.8-Max distillation). Marked license: other; resolve data terms before public/commercial use. Derivative of Qwen3.5-4B — the upstream Qwen license also applies.
Provided AS IS, no warranty. See LICENSES.md.

Citation

@misc{sokol_4b_slovenian_assistant,
  title  = {Sokol: a small on-device Slovenian assistant LLM (4B)},
  author = {Tadej Fius},
  year   = {2026},
  note   = {Derivative of Qwen3.5-4B. Slovenian CPT+SFT. General Slovenian assistant LLM.}
}

Downloads last month: 35

Safetensors

Model size

5B params

Tensor type

BF16

Model tree for texdata/Sokol-4B-SLO

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

SicariusSicariiStuff/Qwen3.5-4B_Abliterated

Quantized

(5)

this model

Sokol — Slovenian Assistant LLM (4B, on-device)

What it is

Files in this repo

Run it (GGUF, llama.cpp)

Capabilities

Training data (provenance)

CPT — Slovenian text (~1.68 M docs / ~1.78 B tokens)

SFT — ~250 k examples

Evaluation

Slovenian-LLM-Eval — acc_norm (cjvt/slovenian-llm-eval, 200/task)

Judge track — chat quality (1–5)

⚠️ Disclaimers & license

Citation

Model tree for texdata/Sokol-4B-SLO

Slovenian-LLM-Eval — acc_norm (`cjvt/slovenian-llm-eval`, 200/task)