Super lightweight model for fitting into 16gb of VRAM with 100k+ context
Used a modified version of this for newer Qwen Models https://github.com/KaihuaTang/Qwen-Tokenizer-Pruner https://github.com/CartridgeStack/Qwen-Tokenizer-Pruner-v2/tree/main
Run this command:
python main.py --old_model_path ".\Qwen3.6-27B" --new_model_path ".\Qwen3.6-27B-pruned-v3" --support_data ".\sample_data" --support_lang en de fr es it pt nl pl cs sv da fi no ro hu sk hr sl et lv lt tr id vi ca af
Then quantized pruned model w/ mradermacher's iamatrix: https://huggingface.co/mradermacher/Qwen3.6-27B-i1-GGUF
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for dgaretz/Qwen3.6-27B-pruned-IQ4_XS.gguf
Base model
Qwen/Qwen3.6-27B