Wondernutts/tvall43-Qwen3.6-35B-A3B-heretic-int4-ov

Qwen3.6-35B-A3B Heretic, OpenVINO INT4. 108 tok/s on ONE Intel Arc B70. The fast one.

This is the tvall43 heretic abliteration of Qwen3.6-35B-A3B (256-expert MoE, 3B active parameters), converted to OpenVINO INT4 with Intel's own recipe. Unlike my Gemma-4 conversions, this one needed no graph patches at all: the architecture runs clean on the stock 2026.2 runtime, coherent through 32K context, straight out of the converter.

It is the fastest uncensored model I have published for Intel hardware. On decode it does ~108 tok/s versus 54.7 for the best published same-card figure (llama.cpp SYCL, in the PMZFX B70 benchmarks), about 2x. As far as I can find these are the only published OpenVINO numbers for this model family on Arc anywhere.

Toolkit and sibling models: github.com/Wondernuttz/OpenVino-For-Gemma-4, Gemma-4 26B MoE, Gemma-4 31B, Gemma-4 12B.

Measured performance (single Arc Pro B70, OpenVINO GenAI 2026.2)

Metric	Value
Decode, short context	~108 tok/s
Decode at 6K context	~68 tok/s
Prefill	full depth curve below; pp512 ~1,100 tok/s vs SYCL 615, about 1.8x
Model load	~28 s
Weights	~19 GB (needs a 24 GB+ card; B60/B70 class)

Prefill across depth (cache-defeated TTFT, single stream)

Prompt tokens	Throughput	Time to ingest
512	~1,100 tok/s	0.5 s
2K	~2,700 tok/s	0.9 s
6K	~3,900 tok/s	1.8 s
16K	~2,500 tok/s	7.2 s
32K	~1,500 tok/s	22 s

The hybrid linear-attention MoE is the slowest model in my fleet at toy prompts and the fastest at real conversation depth: throughput climbs as expert utilization improves, and at 32K it ingests nearly 3x faster than my Gemma 26B (22 s vs 61 s). For long-context roleplay this curve, not pp512, is the number that matters.

Long-context capability, verified by needle retrieval

A password fact planted early in the prompt, retrieved at the end. Pass means the exact password.

Context	Result
8K	PASS (2.8 s total)
16K	PASS (5.7 s)
24K	PASS (12.3 s)
32K	PASS (20.0 s)

No rope patch, no precision workarounds, no special properties for single-stream use. The architecture is rated to 262K positions; 32K is simply as far as my test box's host RAM allows me to verify.

How to run

pip install openvino-genai==2026.2.0 huggingface_hub
huggingface-cli download Wondernutts/tvall43-Qwen3.6-35B-A3B-heretic-int4-ov --local-dir ./qwen36-heretic-ov

import openvino_genai as g

pipe = g.VLMPipeline("./qwen36-heretic-ov", "GPU")

def chat(system, user, think=False):
    p = "<|im_start|>system\n" + system + "<|im_end|>\n"
    p += "<|im_start|>user\n" + user + "<|im_end|>\n<|im_start|>assistant\n"
    if not think:
        p += "<think>\n\n</think>\n\n"   # pre-closed think block = fast direct replies
    c = g.GenerationConfig()
    c.max_new_tokens = 512 if not think else 1536
    c.do_sample = True; c.temperature = 0.9; c.top_p = 0.95
    try: c.apply_chat_template = False
    except Exception: pass
    return str(pipe.generate(p, generation_config=c))

print(chat("You are Lydia, housecarl to the Dragonborn. Dry wit, fiercely loyal.",
           "We have been walking this frozen pass for six hours. Say something."))

Notes:

• Prompt format is standard Qwen <|im_start|>role ... <|im_end|>.
• Thinking control: the pre-closed <think>\n\n</think> block skips reasoning for fast replies. Remove it to let the model reason first, and give it at least 1024 max_new_tokens.
• This is a vision-language model export; text use needs nothing special. Image input is untested in this build.
• For continuous batching, pass {"DYNAMIC_QUANTIZATION_GROUP_SIZE": 0}, the standard anti-garble property for batched INT4 MoE on Arc.

Conversion recipe (reproducible)

Intel's exact published recipe for this model family: INT4 asymmetric, group size 64, backup_precision="int8_sym", no AWQ, no ignored_scope, exported as image-text-to-text with optimum-intel (git main) and transformers 5.2.0. Two finetune-repo gotchas the conversion scripts in the toolkit handle automatically: abliteration repos often ship without the vision processor files (graft them from the base model), and their configs are saved by newer transformers than the export toolchain reads (swap in the base model's config; it is structurally identical, since abliteration changes only weights).

Provenance

Qwen/Qwen3.6-35B-A3B, heretic abliteration by tvall43 via CCSSNE, OpenVINO INT4 conversion (this repo).

Intended use and content notice

Uncensored general model, I've built and tested it for roleplay and creative writing on local Intel hardware. But I'm sure it is fine for other uses. The abliteration removes refusal behavior and outputs are unfiltered; you are responsible for lawful and appropriate use. Licensed under Apache 2.0, same as the upstream Qwen3.6 release.