README
🚀
AI & ML interests
None defined yet.
Recent Activity
View all activity
Hoglet-33
updated a
Space about 4 hours ago
Post
241
Today, we planned to release Pebble-50M and Pebble-50M-Chat to the world. Unfortunately, due to a few issues, that didn't go quite as planned.
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
basically-ai
basically-experimental
What happened:
- Some data and benchmark results were lost or corrupted
- The models performed worse on benchmarks than our other Pebble models
Despite that, you can still find both models here:
Pebble-50M-beta: basically-experimental/Pebble-50M-beta
Pebble-50M-Chat-beta: basically-experimental/Pebble-50M-Chat-beta
There are still some interesting improvements in these models:
- Compatible with non-CUDA devices
- Vocabulary increased to 16K tokens
- Context length increased to 16K tokens
For now, there won't be any more Pebble releases for a while. We're going to take some time to experiment with other approaches and hopefully make the next generation a monumental leap over this one.
Follow for updates:
@Hoglet-33
- 1 reply
Hoglet-33
published 2
models about 4 hours ago
ContextReq
updated 4
models 2 days ago
basically-experimental/Pebble-10M-Chat-GGUF
Text Generation • 10.3M • Updated • 407 • 3
basically-experimental/Pebble-25M-GGUF
Text Generation • 24.5M • Updated • 205 • 1
basically-experimental/Pebble-25M-Chat-GGUF
Text Generation • 24.5M • Updated • 226 • 1
basically-experimental/Pebble-10M-GGUF
Text Generation • 10.3M • Updated • 244 • 3
i64systems
posted an update 2 days ago
Post
60
NOW UPDATED ❗❗
openbob 5.0.1🐭🔎
An integer execution method for reproducible inference from publicly available model weights, demonstrated on Qwen3-4B. Journaled bytes and all.
Keep an eye out for the gpt-oss-120B on the 24gb GPU- deterministically.
We make AI models do the same things every time!⛓️😈
i64systems/Qwen3-4B-openbob-i8
openbob 5.0.1🐭🔎
An integer execution method for reproducible inference from publicly available model weights, demonstrated on Qwen3-4B. Journaled bytes and all.
Keep an eye out for the gpt-oss-120B on the 24gb GPU- deterministically.
We make AI models do the same things every time!⛓️😈
i64systems/Qwen3-4B-openbob-i8
Post
3415
Pebble-25M and Pebble-25M-Chat are out now!
We’re excited to release Pebble-25M and Pebble-25M-Chat!
Both models use our 3:1 Mamba2/Transformer hybrid architecture and were pretrained on 25B tokens. Pebble-25M-Chat was then further fine-tuned on an additional 250M tokens from smol-smoltalk, following the same approach used for the Pebble-10M models.
We hope you enjoy experimenting with them!
Pebble-50M is coming in a few days.
Models
Pebble-25M: basically-ai/Pebble-25M
Pebble-25M-Chat: basically-ai/Pebble-25M-Chat
Pebble-10M GGUFs
In case you missed it, our friend @ContextReq made GGUF versions of the Pebble-10M models:
https://huggingface.co/ContextReq/Pebble-10M-GGUF
https://huggingface.co/ContextReq/Pebble-10M-Chat-GGUF
Follow us if you don’t want to miss future releases and updates!
@Hoglet-33
basically-ai
basically-experimental
We’re excited to release Pebble-25M and Pebble-25M-Chat!
Both models use our 3:1 Mamba2/Transformer hybrid architecture and were pretrained on 25B tokens. Pebble-25M-Chat was then further fine-tuned on an additional 250M tokens from smol-smoltalk, following the same approach used for the Pebble-10M models.
We hope you enjoy experimenting with them!
Pebble-50M is coming in a few days.
Models
Pebble-25M: basically-ai/Pebble-25M
Pebble-25M-Chat: basically-ai/Pebble-25M-Chat
Pebble-10M GGUFs
In case you missed it, our friend @ContextReq made GGUF versions of the Pebble-10M models:
https://huggingface.co/ContextReq/Pebble-10M-GGUF
https://huggingface.co/ContextReq/Pebble-10M-Chat-GGUF
Follow us if you don’t want to miss future releases and updates!
@Hoglet-33
- 7 replies
i64systems
posted an update 4 days ago
Post
92
remat is no longer a one-model claim!!proved it on Qwen3-30B-A3B, K=32 of 128 experts resident, output task byte-identical to the full reference, zero bytes different *in bf16*🥰🥰 GPU comes next😈
- 6 replies
Post
3284
Pebble 10M and Pebble 10M Chat are now released!
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- basically-ai/Pebble-10M
- basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
basically-ai
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- basically-ai/Pebble-10M
- basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
- 32 replies
Post
3047
We are announcing the first generation of the Pebble model family!
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
basically-ai
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
- 7 replies