Pythia 160M, Mamba 130M, and RWKV 169M models trained on OpenWebText for 4000 steps (context window: 1024; effective batch size: 512). 6 seeds each.
James Michaelov
jmichaelov
AI & ML interests
None yet
Recent Activity
updated
a model
about 12 hours ago
jmichaelov/pythia31m_b128_constant_seed0
published
a model
about 12 hours ago
jmichaelov/pythia31m_b128_constant_seed0
updated
a model
about 12 hours ago
jmichaelov/pythia31m_b16_constant_seed0