🔄 In a Training Loop

Joseph Jones PRO

KlondikeDev

6 1 6

https://oc.kunix.org/

AI & ML interests

Restoring the Digital Millennium. https://kunix.org/digital-millennium.html

Recent Activity

reacted to JonathanColetti's post with 🔥 about 17 hours ago

I just uncensored qwen 3.8 27B. You can check out the demo here: https://huggingface.co/spaces/JonathanColetti/Qwen3.8-27B-Uncensored-Demo or the full repo: https://huggingface.co/JonathanColetti/Qwen3.8-27B-Uncensored-GGUF What I did was used an H100 NVL and use https://github.com/p-e-w/heretic to uncensor it. I measured the KL divergence and the refusal rate (Refusal rate is the count of refusals over 100 hold out prompts from mlabonne/harmful_behaviors). Check out the full repo and give it a like if you think its cool Thanks!

liked a model 1 day ago

Qwen/Qwen3.8-27B

commentedon an article 2 days ago

Extreme Overtraining in Tiny Language Models

View all activity

Organizations

reacted to JonathanColetti's post with 🔥 about 17 hours ago

Post

1185

I just uncensored qwen 3.8 27B. You can check out the demo here: JonathanColetti/Qwen3.8-27B-Uncensored-Demo or the full repo: JonathanColetti/Qwen3.8-27B-Uncensored-GGUF

What I did was used an H100 NVL and use https://github.com/p-e-w/heretic to uncensor it. I measured the KL divergence and the refusal rate (Refusal rate is the count of refusals over 100 hold out prompts from mlabonne/harmful_behaviors). Check out the full repo and give it a like if you think its cool

Thanks!

posted an update 2 days ago

Post

Lucy out now!

Lucy is a series of Image Generation models created by OpenCerebral.

Lucy: Prototype, faces only -- unconditional, no prompts.
Lucy-2: Text-to-image, general purpose (though still small)

More Lucy models will be arriving in the future!

opencerebral/Lucy
opencerebral/Lucy-2

replied to their post 6 days ago

That's not a bad idea -- I was planning on 4096 context, but I might scale up, there's not a reason not to

replied to their post 6 days ago

That's not too far off from our original plan, which was FineWeb-Edu, Cosmopedia V2, and DCLM, then some FineMath, but I will definitely look into those other ones, I've been wanting to find some better datasets

posted an update 6 days ago

Post

2147

Boris-2 coming soon!

The Models:

Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.

Why does 125M get more tokens than 250M?

Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.

Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!

We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview

Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!

4 replies

replied to appvoid's post 6 days ago

Making SLMs is so fun! I really enjoy it, personally. Given how little compute is needed, I think everybody should try it!

replied to Banaxi-Tech's post 7 days ago

Is this from scratch or does it use something like a minimal Linux backend? Very cool either way!

Joseph Jones PRO

AI & ML interests

Recent Activity

Organizations

KlondikeDev's activity