The ATOM Report: Measuring the Open Language Model Ecosystem Paper • 2604.07190 • Published Apr 8 • 5
Running Featured 80 Distilling 100B+ Models 40x Faster with TRL 📝 80 TRL distillation for 100B+ teachers, 40x faster
view article Article Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries +7 aminediroHF, qgallouedec, kashif, lewtun, edbeeching, albertvillanova, nouamanetazi, lvwerra, sergiopaniego • Mar 10 • 148
view article Article Ulysses Sequence Parallelism: Training with Million-Token Contexts kashif, stas • Mar 9 • 28
Running on CPU Upgrade 233 The Synthetic Data Playbook: Generating Trillions of the Finest Tokens 📝 233 Explore synthetic data experiments on a virtual bookshelf
view article Article Compute and Competition in AI: Different FlOPs for Different Folks sasha • Feb 12 • 15