Adds TinyMoE-100m-2x8-retrained and correcting the results for TinyMoE-100m-2x8
Browse files`TinyMoE-100m-2x8-retrained` is trained on a mix of Cosmopedia-v2 and
FineWeb-Edu-dedup (60/40) from the smollm-corpus. Also re-evaluated
`TinyMoE-100m-2x8` with corrected metrics.
For context, I initially misread the leaderboard's terms and used plain
`acc` and bits-per-byte. Both entries now correctly use `acc_norm` for
ARC-Easy and byte perplexity for WikiText-2, so the original model's
scores are lower than what I first reported elsewhere.
Both models have been evaluated using `lm-eval`.
Hope this is fine — let me know if anything else needs adjusting. I do not want to fraudate the result of the model.
- index.html +16 -3
index.html
CHANGED
|
@@ -1254,14 +1254,27 @@ const models = [
|
|
| 1254 |
org: "flamef0x",
|
| 1255 |
params: "99.8M",
|
| 1256 |
blimp: 61.13,
|
| 1257 |
-
arc:
|
| 1258 |
-
wiki:
|
| 1259 |
-
tokens: "~625M",
|
| 1260 |
releaseDate: "2026-06-15",
|
| 1261 |
links: {
|
| 1262 |
card: "https://huggingface.co/FlameF0X/TinyMoE-100m-2x8"
|
| 1263 |
}
|
| 1264 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1265 |
{
|
| 1266 |
name: "SRLM-1M",
|
| 1267 |
org: "martico2432",
|
|
|
|
| 1254 |
org: "flamef0x",
|
| 1255 |
params: "99.8M",
|
| 1256 |
blimp: 61.13,
|
| 1257 |
+
arc: 25.88,
|
| 1258 |
+
wiki: 3.878,
|
| 1259 |
+
tokens: "~625M",
|
| 1260 |
releaseDate: "2026-06-15",
|
| 1261 |
links: {
|
| 1262 |
card: "https://huggingface.co/FlameF0X/TinyMoE-100m-2x8"
|
| 1263 |
}
|
| 1264 |
},
|
| 1265 |
+
{
|
| 1266 |
+
name: "TinyMoE-100m-2x8-retrained",
|
| 1267 |
+
org: "flamef0x",
|
| 1268 |
+
params: "99.8M",
|
| 1269 |
+
blimp: 66.01,
|
| 1270 |
+
arc: 33.88,
|
| 1271 |
+
wiki: 2.879,
|
| 1272 |
+
tokens: "—",
|
| 1273 |
+
releaseDate: "2026-07-05",
|
| 1274 |
+
links: {
|
| 1275 |
+
card: "https://huggingface.co/FlameF0X/TinyMoE-100m-2x8-retrained"
|
| 1276 |
+
}
|
| 1277 |
+
},
|
| 1278 |
{
|
| 1279 |
name: "SRLM-1M",
|
| 1280 |
org: "martico2432",
|