qwen3.6-27b-abliterated-journalist-GGUF

GGUF export of tomvaillant/qwen3.6-27b-abliterated-journalist-merged, an investigative journalism and OSINT fine-tune based on huihui-ai/Huihui-Qwen3.6-27B-abliterated.

This is the large tier ship — text-only GGUF for Goose Desktop / llama.cpp on machines with ≥32GB unified memory. Smaller machines (≈16GB) run the qwen3.5-9b-abliterated-journalist-GGUF instead.

Usage

llama-cli -hf tomvaillant/qwen3.6-27b-abliterated-journalist-GGUF:Q4_K_M --jinja

Run as a local OpenAI-compatible server (thinking mode on — see note below):

llama-server -hf tomvaillant/qwen3.6-27b-abliterated-journalist-GGUF:Q4_K_M \
  --port 8081 --ctx-size 16384 --n-gpu-layers 999 --jinja

Pull via Ollama's HuggingFace passthrough:

ollama pull hf.co/tomvaillant/qwen3.6-27b-abliterated-journalist-GGUF:Q4_K_M

When wrapping the Ollama tag in a Modelfile (e.g. for opencode / Spotlight), add explicit PARAMETER stop directives — the fine-tune emits <|endoftext|> at end-of-turn while Ollama's auto-derived stop list only includes <|im_end|>:

FROM hf.co/tomvaillant/qwen3.6-27b-abliterated-journalist-GGUF:Q4_K_M
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"

Thinking mode is required

Earlier revisions of this card recommended --reasoning off --chat-template-kwargs '{"enable_thinking":false}' to save reasoning tokens. Don't do that on this model. The abliterated Qwen 3.6 family's /no_think codepath is damaged — verified empirically on both the Huihui base and this fine-tune at Q4_K_M with Qwen-recommended sampling (temp=0.6, top_p=0.95, top_k=20): output collapses into multilingual token soup and lock-loops within ~200 tokens. Probable cause: abliteration calibration only covered the thinking codepath, leaving the no-think branch with broken refusal-direction subtraction; Q4 quantization amplifies it.

Keep thinking on. Expect responses 3–5× longer per turn than the 9B sibling. Bump max_output_tokens (4096+) and any opencode-style limit.output (16384) if you see truncation inside the reasoning block.

Files

• qwen3.6-27b-abliterated-journalist-Q4_K_M.gguf — Q4_K_M with imatrix calibration (~15 GB on disk, ~22 GB at runtime; recommended for laptops with ≥32 GB unified memory)
• chat_template.jinja
• tokenizer files

Training

• Adapter: tomvaillant/qwen3.6-27b-abliterated-journalist
• Merged checkpoint: tomvaillant/qwen3.6-27b-abliterated-journalist-merged
• Method: LoRA with Unsloth FastModel + TRL SFT, following the official Unsloth Qwen3.5 fine-tune recipe (bf16, r=16, alpha=16, dropout=0, use_gradient_checkpointing="unsloth", optim="adamw_8bit"). Merged into bf16 safetensors via save_pretrained_merged, then converted to GGUF via llama.cpp convert_hf_to_gguf.py + llama-quantize.
• Dataset: tomvaillant/investigative-journalism-training (687 examples, OSINT methodology)
• Quantization: Q4_K_M with imatrix calibration on the training corpus. Imatrix gives ~1–3% perplexity recovery over stock Q4_K_M at the cost of a single one-shot calibration pass; per llama.cpp discussion #11088 the benefit is meaningful below Q5, negligible at Q6+.
• GGUF metadata: ships with tokenizer.chat_template embedded (inlined from chat_template.jinja pre-conversion so Ollama's HF passthrough sees a real template, not the default {{ .Prompt }} fallback).

Sources And Attribution

Training data: tomvaillant/investigative-journalism-training — 687 instruction/response pairs synthesized by Claude Opus 4.6 (Anthropic) from the Buried Signals OSINT and investigative-journalism corpus: OSINT Navigator tool data, Indicator Media briefings, Buried Signals investigative skills, GIJN, Bellingcat, Verification Handbook 3, SPJ Code of Ethics, RCFP, and public manuals from UNESCO, Al Jazeera Media Institute, CiFAR, CIPE, and EJF/TEMPO Institute.

See the dataset card for the full source list, licenses, and per-partner attribution.

Intended Use

Built for local llama.cpp inference in investigative journalism and OSINT workflows. Vision tower is not included in this GGUF — to add multimodal input, layer in mmproj-BF16.gguf from unsloth/Qwen3.6-27B-GGUF (byte-identical because the vision tower was frozen during training).

Treat outputs as leads, not verified findings.