Thoughts on the scaling hypothesis

Last updated Jul 14, 2026 Edit Source

Rough thoughts on scaling post-training and pre-training LLM resources (data/compute etc) as a catalyst of progress.

Instead of hiring ten thousand geniuses, owning a million GPUs that search for patterns non-stop is simply a more predictable path to general intelligence.

POSITION: I think neither of these options are accurate, so this might be a false dichotomy. One easy way to see how this is true is number of humans entering a field. Each new human entering and being trained on a field can be analogized to the empty context RL rollout of an LLM. If the ‘10000 geniuses in a datacenter’ framing of compute is right, then arguably yes, more and more smart humans entering say the field of string theory would lead to more progress in it, or more ML researchers entering the field after the invention of Transformers would find some architecture that improved on it.

What we see though is more of the opposite. In the first few years of existence of a field, the low hanging fruit are picked by the founders of it, and every subsequent discovery is harder and harder to make, no matter how many humans (proxy for ‘compute’) you throw at it. At the same time, one foundational belief that is wrong can cripple progress for decades, since anyone else coming in builds on top of it. Concentration of human (or compute) resources in one field draws oxygen from other nascent fields that may be promising. One can almost track progress (or lack of it) mostly to how ’nascent’ or ’novel’ or ‘different but right’ a field or an idea is rather than how many humans (or AIs) are working on it, leading to a huge dichotomy between exponential resource growth and perceived linear progress (some documentation of this trend here).

The real question then is what kind of process allows new hills to be identified and scaled at arbitrary time horizons? Its not obvious that applying arbitrary amounts of human or AI resources does this (deep learning was invented and most discoveries happened in canada even though research budgets were a magnitude higher almost everywhere else). Point here is that intelligence (whether human or AI) is not a fungible function of a bunch of inputs - where its possible to just buy progress without any sort of path dependence (example i think about: what if Einstein was born after the quantum nature of the atom was discovered instead of before? Would he have theorized the exact same inconsistent formulation of GR? Or gone down other paths?)

If we pour more compute into the same data, it’s possible to expand the collection of patterns we’re able to extract from it.

POSITION: So far it seems LLMs follow a log-linear scaling trend (also claimed its creators). As long as scaling capabilities linearly requires an exponential increase in data/compute, a human-like learner with linear/sub-linear economics can come in, adapt quickly to a domain and do it faster. One way to think about the qualitative nature of the ‘pattern search problem’ is difference between try to find a lost kid by a) sending a search party out to larger and larger swathes of the forest vs b) having a detective piece together clues into a model of the world and then following the inconsistencies. Both might find the lost kid, but the process followed by the latter is more optimal/faster and requires many orders of magnitude less resources. The former is more fungible (you can create an arbitarily large search posse consisting of all the villagers), but the latter is not (requires years of careful training and specialization). The former requires more and more resources for linear gains, vs the latter gets non-linear gains for only tiny increments in data exposure. There’s a qualitative difference in the search process of the latter that is path-dependent in a way that ’throwing compute at it’ doesnt capture. The argument can be made that machines are fungible, so we can just create large search parties for any problem and throw infinite resources at it. The problem with this is the same as the problem with using search parties - for a lot of problems it may just work, out of pure luck, but what if you’re looking in the wrong forest? how do you quickly adapt and change course? Throwing the same untrained search party member at the same set of clues a 1000 times has diminishing marginal returns over letting a trained detective examine them for 10 mins.

Extra compute is advantageous because you can simply work harder and longer at cracking difficult problems

POSITION: This seems to one those assertions that seems locally true but insufficient to explain how truly hard problems are approached. In practice, a ‘6 year long horizon task’ doesnt mean trying continuous tiny improvements every day for 6 years, but instead collecting a library of priors (tools/frameworks/skills) for the first 5 years to then be able to have the fundmentals in place to be able explore a shorter solution space quickly in the final year or so - otherwise the set of possible solutions via incremental minmaxing can be infinite. Most PhD programs are structured this way, a 2-year course roadmap to equip a wide range of fundamentals in order to have the vocabulary to tackle harder problems. For a PhD supervisor, getting an ‘aligned result’ means building up the student’s foundational ‘vocabulary’ ladder bit by bit so the the results is within the set of desirable outcomes, instead of error-compounding into local minimas and not having the tools to recover on their own. A faulty result means changing / updating parts of this learnt prior ladder and redoing just the last few months of progress, not re-doing a 6 year re-run rollout of the phd program. This entire process is entirely path dependent, depending on how good/bad the path ordering of prior acquisition is, you might get a widely different outcome. Machines might shrink that 6 years to 1 year, but the path dependence is a pure information limit that cannot be wished away by arbitrarily applying ‘extra compute’. Even today, the usage of GPT 5.5 Pro which theoretically ‘thinks more’ is patchy because of the uncertain nature of results and cost of verification. The ‘slot machine’ outcomes might be good enough when the dollar/opportunity costs are cheap and low stakes, but spending a lot of money and time on extra thinking for uncertain final results becomes costlier with more compute / time applied (specially if it comes at the opportunity cost of other pursuing other approaches to the problem)

To do a good job of solving problems generally, you need to store all the knowledge you can. Hence more parameters the better

POSITION: Agree partially that more parameters does help with better problem solving, but this is might be because it leads to more ‘generalizable understanding’ - not explicitly because more knowledge is stored is across fields, but because compression rate is better with more parameters (i.e. a concept is ‘juiced’ across more layers of understanding, resulting in more ‘understanding’ or ‘compression’ compared to shallower models)

Empirically it seems specialists in one domain are rarely specialists in every domain. While they may have some base understanding of how the world works across domains, what separates them is not raw storage of knowledge but rapid acquistion of knowledge / skills required to solve a problem. Theres a path dependence here - if you’re a physics specialist its likely easier for you to pick up knowledge skills in geometry rather than abstract number theory, because of the priors a physics training imparts. A lot of Einstein’s gedanken thought experiments around speed of light and time can be traced back to playing around with clockwork mechanisms in the patent office and wondering about how they’d mechanically work at relativistic speeds.

Scaling is rational given the opportunity costs of not scaling

POSITION: I think it’s as (ir)rational as any human pursuit that promises guaranteed utopian results (Pascal’s wager et al).

I think there are strong incentives not to acknowledge/recognize any of the path dependency above and for smart people to continue rationalizing the ‘10000 geniuses in a datacenter’ narrative (i.e. to focus on quantitative parts rather than the qualitative). For one its predictable - one can raise money, gather human beings around, and make policies for predictable trends, even though they may be wrong. Its hard to do that for something unpredictable like ‘if we make our research funding less military-driven, maybe Geoff Hinton will choose to relocate and invent deep learning here.’ Dots dont connect forward.

“The socioeconomic value of linearly increasing intelligence is super-exponential in nature. A consequence of this is that we see no reason for exponentially increasing investment to stop in the near future.” - Sam Altman, 2025

Without even judging them right or wrong, it might be important to realize the self-serving incentives at play - the more resources that go into following a quantitative scaling trend, the higher the stakes if it doesn’t deliver as promised, and the higher the incentive to demonstrate its continued success at whatever cost, and attribute any success to it (‘we couldnt do this one tiny thing with LLM v5.5 but because of the scaling hypothesis being correct we can now do this magically in v5.6’), as opposed to more path dependent factors (like better training data, better reward functions, more efficient training pipelines, consolidation of use cases etc.)

This sort of quantitative trend-following is new in the ML world, but has already been seen out in say the physics world, where a perverse set of incentives entered the fray around justifying the building of larger and larger particle colliders to taxpayers. Instead of aiming for falsifiable results/experiments that would prove / disprove theoretical claims, the focus turned to ‘promising guaranteeing results’ to the tax payer. https://www.nature.com/articles/s41567-020-01054-6

When it comes to the question what the new mega collider could do for science, the CERN director explains:

“A good example of a guaranteed result is dark matter. A proton collider operating at energies around 100 TeV [that’s the energy of the planned larger collider] will conclusively probe the existence of weakly interacting dark-matter particles of thermal origin. This will lead either to a sensational discovery or to an experimental exclusion that will profoundly influence both particle physics and astrophysics.”

Implicit in this statement is not a Popper-like pursuit of experiments that help to reject bad theories and reject this line of experimentation completely, but to tweak bad theories to fit whatever experimental results in order to ensure the range of particle colliders continue to be sustainable funded. This sounds familiar!

Case in point : GLM 5.2 is a model that performs on par with a lot of Opus 4.x models even though it was trained on OOM less compute and also has less parameters. What explains this? Blind distillation alone is insufficient to explain this fast-catching up, given for instance Anthropic’s own models like Sonnet 5, which have access to unlimited signals from a superior model like Mythos 5 is unable to land on the pareto curve.

Another case in point: GPT-5.5 xhigh which was released a month after ARC-AGI-3 scores around 0.43% . But GPT-5.6 which uses the same base scores a 7.8% four months after ARC-AGI-3 was released. The only difference that explains the inability of a model upon benchmark release to be capable of a task but suddenly be capable in 4 months is not some magic interdeterminate scaling of data and compute, but explicit hillclimbing on tasks of that type. Its worth wondering why gpt-5.5 wasnt able to solve even a single ARC-AGI-3 problem upon release (meanwhile most literate humans you could pick up off a street could stumble their way to solve atleast one of the games, without any specialist training beforehand).

This few-month lag is itself the continual improvement loop, where each model helps train the next model faster. If this lag period reduced to a week where a new model releases every week, it would count as a form of continual recursive improvement.

POSITION: This is not as clearcut as it sounds and needs more critical probing. The employees at Apple have been using the hardware/software they make to improve the next generation of apple hardware for the last 40 years. Cranes near construction buildings have a bootstrapping process where they use the crane arm to build themselves up. Most would agree that these are fairly harmless examples of so called ‘recursive self-improvemrnt’ at play. What makes this different from truly ‘dangerous’ RSI is that they are narrow and scoped to fairly innocuous goal function (i.e Apple yearly sales or crane height). I would slot today’s scaling regime within the same set of innocuous global goal functions. If the goal function is approximately tied to maximizing cost per token while maximizing gpu utilization, there might be many ways to do this. For instance, make it excellent at one shotting fancy game demos that get the maximum engagement online and that people want to share. Or make it friendly and sycophantic, so that a user may continue spending tokens pursuing dead ends. Or even something completely unrelated like presenting a nice concise summary with a table and ascii diagram that makes things easier to read. All of these are a form of ‘improvement’ that have nothing to do with increases in sample efficency or skill acquisition efficiency or generalizating out of distribution (i.e. measures of intelligence).

The steelman: The publically stated claim of many AI labs is that automating AI research will be the path to RSI. But what exactly is automated AI research? If its in the automating the post-training experiments to cater to more use cases and pleasant output formats - this seems largely a product engineering goal to maximize usefulness to more users in service of the revenue goal, not too different to Apple’s form of recursive improvement (and similarly useful to the topline). If it’s pre-training experiments to scale to more parameters, so as to improve the knowledge base, this too is largely an engineering problem with the same limits as the rate of progress of Apple’s efforts to fit more compute into its unified chips. If its in the supposed discovery of new architectures that increase data/energy efficiency of the type a senior researcher like Ilya Sutskever would make - these suffer from the same hard exponential limits I stated back in my position in Assertion 1 - not something that has been proven to be solved by scale, i.e. throwing more bodies (biological or otherwise) at the problem within status quoist incentive structures. If - a big IF - one were to assume that such superhuman AI researcher were indeed possible by scaling up today’s LLMs - why is it a given that what it may propose be aligned with the labs’ primarily commercial interests? If its suggestion were to abandon LLMs altogether and that deep learning and GPU programming was a dead end to achieve human-level sample efficiency, would the lab’s sacrifice their top line for that uncertain risky bet? Indeed even today, human senior researchers like Jerry Tworak and Yann LeCun find themselves unable to marshall resources to pursue alternate unorthodox research directions even within highly GPU-rich research environments. Arguments of this sort usually end up in an argument-to-authority to an all-knowing wise God figure that will exist in the future.

This sort of non-conclusive framing of recursive self improvement doesnt mean truly transformative RSI is not possible. But it will look very different - closer to the rapid adaptation/mutation (or modification-via-descent) of a polymorphic computer program in response to adversarial host conditions rather than a static entity whose weights are fixed in a time in a datacenter. Human counter responses to viruses (biological, cyber or otherwise) have been largely successful till date because our industrial processes and response patterns have been able to adapt faster than the virus can mutate. The thing to be cautious about is hence a digital entity that can adapt, morph and self-replicate in unpredictable non-linear ways, climbing arbitrary abstraction ladders in short time spans before any human systems can respond. There is no reason such an entity needs to be trained on the entire internet or even occupy large datacenters, or even needs to be a single entity static in time, or need to be particularly good at making frontend websites. In short, recursive self improvement might be transformative only when it improves ‘generality’ itself, or the virtuous skill of ’learning to learn’, where sumitting each subsequent rung of the abstraction ladder, makes the next rung exponetially easier to scale (the opposite of a log-linear scaling trend). The rapid progress of such an entity I think is closely modelled say by the rapid, specialized progress of one of the Eastern Tiger economies, like China or South Korea, which having started behind, marshalled all their resources via central planning to reverse engineer their way to a large industrial base and manufacturing exports - enough to rapidly catch up to the rest of Western society, and now in a position to overtake it in many respects - each decade of progress building upon the previous in an gradual, continual, snowballing effect which to an outside observer would appear as a fairly normal state of affairs until the very last decade when it became impossible to ignore.

A new future scaling law could involve creating its own RL envs to rehearse skills / or it’s own reward functions to train on.

POSITION: I think this is a fair argument for a new scaling regime (most recently on Dwarkesh’s channel). My contention here if that it were possible for an LLM to model the transition rule of any arbitrary state machine it were to encounter at exactly the right level of abstraction to model all its most important phenomenon - why would it need to do RL rollouts to train on it? It could just use this model itself to make predictions and update it on the fly as evidence changed without ever updating its weights via training. In which case bottleneck again becomes path dependent and not a function of indeterminate scaling of data/compute i.e. what code priors it has built up via interaction to know how to model arbitrary state machines at the right level of abstraction (i.e. if you want to drive a car, you dont need to necessarily understand how the engine and cooling unit work, you merely need to understand how roads and obstacles work and have a causal model of the steering wheel, brake clutch and accelerator in your head that you can uldate with each edge case encountered on the fly while driving, without having to do a thousands of ‘negative signal’ car crashes to gain that intuition). This brings back my prior path dependence point - if you have never built ground up intuition of objects and their physics, it is not possible to sample efficiently learn how to drive a car no matter how much compute you throw at the problem (and hence even orangutans can pick up how to drive faster than a Tesla).

Frontier exploration is compute-hungry. Discovery of frontier knowledge requires massive/expensive search efforts

POSITION : The nature of machine intelligence is unlike any other pursuit - the aim to find a system that can do more and more with less and less, not something which might take a country worth of resources to barely outperform the 85 percentile human at certain select tasks which are over represented in internet data. It’s challenging precisely because the future is a lightcone of events never seen in any dataset - a unique engineering problem where you cannot anticipate more than 90% in-field design requirements beforehand! (see for instance this blog which claims that making LLMs good at biology is hard because programmatic API access to biology data DBs is hard, which is a tough claim for something that claims to generalize well across domains. imagine Watson & Crick complaining about DNA being hard to discover because their library allowed them to borrow only two books a week).

if we imagine what an end game for our technological future looks like, we might think of fleets of self replicating von neumann probes that terraform the galaxy in their wake. To travel those large relativistic distances, these will need to be shrunk to their smallest possible forms, and they will need to work in situations not present in any human training set, nor in any earth-bound prior. In that sense, space exploration is the ultimate test of generalizing out-of-distribution, and of the bitter lesson - if it takes two light years to communicate with a probe, human supervision or intervention of any sort just does not exist within the set of engineering tradeoffs. The machines will need to program themselves on the fly, adapt to novelty without a 4 month long hill climbing training run, and the ‘just hill climb on the test set’ regime of today where we prepare ML models in a factory and then release them into the world will be dead on arrival. These entities will need to learn on the fly from sparse experience, and a ‘bad rollout’ is not a reward signal to improve from but is simply a fatality event. They must be highly autonomous even at low energies, able to robustly recover from arbitrary hardware and software failure.

I dont see a straight line joining the more-data-and-compute scaling regimes of today and that autonomous space faring future. The LLM scaling hypothesis is not just inadequate because it claims to offer a magic path to ‘scale the frontier’ and falls short, but also because it just doesn’t see far enough.