Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling Paper • 2502.06703 • Published Feb 10 • 153
view article Article Sparse Mixture of Experts Language Model from Scratch: Extending makeMoE with Expert Capacity Mar 18, 2024 • 13
view article Article makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch May 7, 2024 • 111