Post
2446
Nobody knows what is doing, when you train a model, you are experimenting to advance the frontier, so keep failing 🫵
Join the conversation
Join the community of Machine Learners and AI enthusiasts.
Sign Up
Saying "Nobody knows what they are doing" is just a convenient excuse to justify terrible engineering. There is a fine line between scientific trial-and-error and proud, brute-force ignorance.Let's be clear about your "frontier":Blind Gambling: When independent labs don't understand the underlying mathematics or hardware physics, they just throw data at a wall and pray to the loss curve. That is digital alchemy, not science.The Loop: Instead of fixing structural bottlenecks or learning non-linear dynamics, people just brute-force configs. It’s the engineering equivalent of a cat grooming itself because it has nothing else to do.Zero Legacy: This unscientific approach is why the ecosystem is flooded with overfitted, hollow checkpoints that break down outside of their strict test sets.You aren't advancing the frontier; you are just polluting the platform because you refuse to open a textbook. Brute force has hit a physical wall. True innovation requires cognitive architecture and actual engineering, not just romanticizing failure. 🫵🤡
What is this guy even talking about
First, real indepedent labs aren't brute-forcing anything; we don't have the compute to do that. I mean, look at AxiomicLabs/GPT-2.5-135M, it gets within 2 points (via II score) of SmolLM2 with 60x less data. That is the exact opposite of brute force.
Second, in frontier deep learning, formal theory has always lagged results. Saying "no body knows what they are doing" simply ackolegdes (fuck i cant spell) that we are exploring uncharted territory with informed hypotheses rather than pretending a textbook already has all the answers.
100% true
I consider AI models to be effectively be lazy programming; Throwing slop at the wall and some stuff might stick, brute forcing it taking hundreds of thousands of iterations before it is anywhere near useful. Then to get a better model you do it again... Which for larger and larger models costs millions or billions of dollars in compute power.
Add to that it's very slow compared to something hand-coded using an interpreted language or compiled. Though there's a lot of things a trained model can do faster/better than a human, or at least have the endless patience to reroll something until you get something useful.
Personally I'm on the fence of if AI is a good thing or not: Depends on if it offers more good use to us (lowering the bar to make movies books and remove busywork) or if it's more a hindrance (Youtube censoring elbows and closing hundreds of channels daily with no recourse to fix it, surveillance and Flock cameras flagging people who end up getting arrested because it read a plate wrong, or Sony planning to monitor voice chat and then banning your account if you say a naughty word)