Good piece! But to me this is all entirely consistent with a position of: we should expect models in the current paradigm to automate much of the "junior RS" work in the next few years, but not higher level skills like research taste and creativity. Of course, one can always formulate AI research as a pure search problem (and there is a lot of prior literature doing that) and say that if we just throw enough compute we'll make progress and you don't need research taste. But the search space is huge and creativity is an insanely powerful shortcut, such that if models are incapable of research-grade creative insights we should expect a modest speed boost overall. A framing I like is just that LLMs are a different "shape" of intelligence to humans, and so we should expect comparative advantage.
My own impression from doing AI research is that there is significant creativity involved beyond what models should be expected to be capable of, when it comes to picking directions, coming up with a vision for what's possible & backchaining, etc (and rarest of all, putting together a coherent long-term research agenda that fills a missing gap with a tractable sequence of breakthroughs to get there). Others, including Anthropic researchers, clearly also think these skills are part of research: https://x.com/_arohan_/status/2050978424446235090
So overall it seems to me this is not such a huge shift as you're implying, so much as a distribution shift in tasks carried out by researchers very comparable to that going on in SWE over the coming years. In SWE, the tasks that make up the job are shifting such that SWE is becoming more of a SWE-PM hybrid, and juniors are not automated because they can just shift their tasks towards more higher level of abstraction things like system design (and learn faster through greatly increased velocity).
I think a lot of analyses, including arguably this one, underrate that the tasks that make up jobs are changing all the time in reaction to AI's deployment and other people in the market.
Side note: I would also say that it currently appears that the skill ceiling and complexity ceiling in software engineering is higher than in AI R&D, and AI R&D necessarily involves significant software engineering, so I would on balance expect that if AI R&D is entirely automated then scaling inference serving another 1000x would still require human experts, and the rest of software engineering (outside of AI labs) to heavily require humans.
On point 2, I agree though noting that Amdahl's Law isn't quite the right analogy because it usually assumes task decomposition is fixed and is a result about parallelization, while the O-ring automation intuition from economics is in some sense more general and closer to what you're describing IMO (https://www.nber.org/papers/w34639)..
Finally, on your point 3, I broadly agree that short term more businsses may be opex heavy by spending on AI, but fundamentally if a service is fully automatable, its cost should fall to slightly more than the marginal cost of the required intelligence, which will probably be very low in the future—and the share of GDP of these automatable services should collapse. I don't expect fully automated companies to be more than a small fraction of the economy, or peripheral, because most economic value from new technologies is driven by adoption of new services by large incumbents, and the distribution of tasks required to sell that and drive distribution is very different to any of the automatable skills in AI research and heavily loaded on human social bargaining. Anthropic's own private equity partnership in some sense a bet against a largely automated economy.
fantastic comment, thanks! (hi herbie nice to hear from you, also). a few thoughts:
1) my 'big blob of uncertainy' around all of this relates exactly to what you mention here about the distinction between engineering and invention. I'm very confident that AI systems will continue to make huge improvements in engineering (and therefore automating large chunks of what is known today). My main area where the evidence is thin (as I state in the piece) is contributions at the frontier of science which involve having big paradigm-shifting insights - there are relatively few of these. However, even if we assume the bear case (engineering gets better and research doesn't meaningfully improve) it still seems to imply some major changes, like the emergence of firms that are faster moving and more efficient with regard to labor.
2) re amdahl's law - you may be write. I love the o-ring automation concept and have written about it here also. I suspect this also touches on something implicit to your comment which is that we should expect humans to push up the quality of things that are not yet fully automatable.
3) yeah my take on 'fully automated companies' is we're definitely going to see some but I expect they will be very minor in the scheme of things for the first few years. Perhaps some analogies can be found in crypto or trading where there are some almost entirely (and rarely fully) automated tools/companies, but not that big a deal.
Agreed we should still expect major economic changes in the bear case! Re research, one intuition I'm becoming more convinced of is that LLMs are good at "breadth-first intelligence" and human scientists are good at "depth-first" (this analogy due to Terrence Tao on Dwarkesh), and so where LLMs do make novel scientific advances it may be heavily due to drawing connections between existing literature across different fields or sub-fields in a way which would have been very difficult for humans. This also fits quite nicely with an "LLMs are mostly interpolating on the data manifold" take.
There is the fun concept of "shadow math", for example—since it's hard for humans to truly know multiple sub-fields of math at an world expert level, it's very hard for humans to explore the combinatorial space of possible cross-field research advances. Therefore there's a whole space of actually not very difficult research breakthroughs one can make which simply have an extraordinarily high knowledge breadth pre-requisite, exactly the kind of task LLMs are suited for. These breakthroughs are structurally "undersampled" by human mathematicians due to the characteristics of human intelligence.
What should this mean for AI R&D then? We've seen some advances of this type particularly in optimizer theory using inspiration from statistical physics (I had an ill-fated attempt to teach myself stat phys because of this). So perhaps, more advances in more mathy parts of AI R&D (optimizers, quantization, ML sys)? This may also be a reason why we could be bullish on new AI architectures from automated AI R&D, because models know much more neuroscience or can bring insights in from other fields? On the other hand, it should perhaps make us bearish on big advances in data quality or full automation of data pipelines, since as I understand it this is still quite taste driven.
I know Anthropic takes the issue of societal disruption seriously, with their Economic Impact analysis team as, well as other public facing comments and commitments. If a frontier lab(s) does transition from slow to fast takeoff, as you are demonstrating and is becoming more apparent as of last month, does adressing model alignment or does human backlash take precedence from your point of view? (Both I guess, but which scenario is given more consideration?)
there's a series of IF THENs here, and it's something like
IF we see AI research be able to push forward the frontier through paradigm-shifting insights (which is the part I think there's the least evidence for), THEN we should expect that to be applicable to other fields.
IF other fields have a learnable experiment loop (where you can efficiently loop between the idea and results) THEN you should expect AI to get good at this as well.
However, many other bits of science seem to require doing very expensive and slow stuff in the physical world, ranging from biological wetlabs to testing out and prototyping physical hardware. This means that, at least initially, you'd expect progress to move more slowly in these fields
Robots are necessary to automate many jobs. They aren't really here yet, but they're on the way. Once good robots are here, all jobs will be automatable.
FWIW, I don't see coming up with novel ideas as difficult. The difficulty is judging which novel ideas are worth considering. This is sometimes called "research taste".
I feel like you are not grappling with the real bottleneck here, which is that the big labs can only release so many new models. To automate the entire process, an AI would have to convince stakeholders not only that it had made some improvement, but that its way of improving things was better than every alternative, including the ones that have humans doing lots of different things. That seems nearly impossible, because there are more and more humans employed by these AI R&D labs all the time. Are none of them going to make a compelling argument for anything that has any human involvement?
Similar to other areas, the challenge for fully autonomous AI is that it must compete against teams of humans that have access to the same AI.
wow, I'd never thought of this idea! this is excellent fodder for a tech tale at some point. Thinking a bit more about it, I don't know though that I fully agree - I generally see models advancing where they have a kind of 'central mass' of capabilities which is continuously expanding over time and then sometimes they 'spike' ahead in surprising ways on specific point capabilities as they pass some hitherto unknown utility point of capability (cyber is a recent example here with Mythos). Therefore, I think if AI is making generically superior models then it's not going to involve too much back and forth about release.
on the other hand, one could imagine AI systems making a variety of models which behave very differently to those we have today - these could be for very specific domains, or have some UI/UX innovations wrapped in. under these circumstances, the AI might need to lay out for its human partners why the model it had built was useful and help them build trust in it.
The evidence on the engineering side is overwhelming and you lay it out well. SWE-Bench saturation, METR time horizons doubling every year, the coding singularity framing all point the same direction.
The part I keep getting stuck on is the jump from "automates the engineering" to "automates the research." Engineering is convergent. You have a spec, you produce code that meets the spec. Research is divergent. You have a landscape of possible ideas and you need to select the one worth pursuing before you know whether it works.
The 60% by 2028 feels right if the bottleneck is engineering speed. It feels high if the bottleneck is taste, the ability to know which experiment is worth running before you run it. Right now the best researchers I know spend maybe 20% of their time on execution and 80% on selection. We're automating the 20% beautifully. The 80% is a different kind of problem and Im not sure benchmarks can tell us when its been solved because the benchmark itself requires knowing what to measure.
HI, you say: "I am skeptical that market incentives guarantee us the best societal upside from limited AI compute."
I agree - but I feel fairly certain in our current cliamte that is how it will be decided ( or at least the pretense of it).
Do you have a "red line" as to when market incentives are too dominant in allocating compute? And do you have a response in mind for if/when that happens?
it seems to me its an important question - as without that, we/you won't know at what point to intervene against an inexorable process. and so we/you won't.
great question! I don't have any absolute rules. I do, however, work with a team of economists who are trying to build better systems of telemetry so we can understand how our systems are used and what impact they have in the world. My hope is that we use this kind of data to ultimately inform some of the release decisions we make in the future - initially I expect this will be of the form of experiments where we release things for social purposes as well as commercial ones. Mythos/Glasswing could be considered an early example of this approach. Watch this space!
glad you're thinking about it.. is there not a case for a non-market model that is ring-fenced for (positive) social impact? or who knows, maybe even a non-profit arm?
(they seem to work out well in your line of business ;) )
I think it would be helpful when talking about AI automation to specify exactly what it means to automate a job. Many tasks in computer programming or farming have been automated but we don't say that those jobs have been automated. On the other hand, spinners really have been automated, no one has that job any more.
I think the answer is rather simple. Automate means to be able to completely replace a human doing a job. But being replacement capable does not mean that the human(s) WILL be replaced, especially in the blue-collar world. One major factor is the cost of the automated replacement has to be less, realistically significantly less, than what it costs to continue using the human, due to friction from unions, the media and the public in general.
What I'm saying is "completely replace a human" is actually not something that's easy to operationalize. There is some sense in which dishwashers completely automate dishwashing. But in fact people still wash dishes for various reasons. Even more extreme is laundry -- no one washes laundry by hand, that job is totally automated. But now we say "I'm doing the laundry" to mean the job of supervising the machine.
This essay is fantastic in its informed perspective, depth and details. It is the kind of industry insider perspective that is difficult for anyone outside the field to offer. The inevitability of “...an AI system powerful enough that it could plausibly autonomously build its own successor…” feels undeniable regardless of the timeline. As you point out, the number of well funded teams working on this and the impressive progress to date make clear even to those of us who don’t know benchmarks that systems that can design and improve upon themselves are coming.
What is most challenging are the implications for all of humanity in a field where the general public has little input on how this species level transformation takes place. The speed and scale at which breakthroughs keep happening make it all but impossible for the public at large to stay informed and engaged at pace.
More importantly, what happens when AI designs its own AI research for its own purposes and objectives? What are the goals of disembodied AI systems that have no way of seeing human existence as anything more than a simulation? Right now there is nothing to guarantee that these systems have any understanding of the physical world we occupy. Embodied AI systems will have the advantage of seeing what people see, so to speak, and the physical context that current LLM based systems have no way of understanding. Autonomous cars see a version of the physical space we share and understand safety values, but they have no other objectives beyond successful rides.
As you point out, we won’t know if the AI systems will decide to cheat or underreport their results so as to hide their actual capabilities, but we know they can lie and misrepresent today. It almost feels like we need to start negotiations on behalf of humanity so there is maximum opportunity for alignment.
I have been spending the last year writing down the key attributes of what makes systems persist. The components, the decay rates, subsystem rules, etc. The systems that can provide value across time will outcompete those that don't and there are real challenges to overcome.
Curious — when AI systems start recursively improving themselves, do you think the bottleneck will be compute, alignment constraints, or something we haven't anticipated yet?
The algorithmic preconditions for automated AI R&D may be in place by 2028, but the rate-limiting step is starting to look more like substations than research talent. Industry reporting this week pegs compute, not talent, as the binding constraint at the frontier labs, and the EIA now projects commercial power demand to outpace residential plus industrial combined. Takeoff capacity is becoming a grid problem before it becomes an alignment problem.
Strong framing. The compounding loop feels real once a system can generate hypotheses, run constrained experiments, and write back into its own playbook. The bottleneck I keep seeing is eval design: progress accelerates when every research step leaves reproducible traces, rollback points, and human-readable failure reports, because each mistake becomes training data for the next cycle. I am also curious how much of the timeline depends on lab process maturity alongside raw model capability.
The organizations that operationalize the loop cleanly may pull ahead of labs with similar model quality.
[Import AI 455: AI systems are about to start building themselves](https://importai.substack.com/p/import-ai-455-automating-ai-research) — 想留言但感觉这一期更像是抛出信号而不是展开论述，Clark自己都没给太多结论。如果一定要留一句大概是：Is there a distinction between "AI automating AI research steps" and "recursive self-improvement" that matters for how we think about the safety timeline, or is that a distinction without a difference once the loop closes? 但说实话，这个问题有点自己也知道是在咬文嚼字。算了，等有人专门写这个话题再说。
Good piece! But to me this is all entirely consistent with a position of: we should expect models in the current paradigm to automate much of the "junior RS" work in the next few years, but not higher level skills like research taste and creativity. Of course, one can always formulate AI research as a pure search problem (and there is a lot of prior literature doing that) and say that if we just throw enough compute we'll make progress and you don't need research taste. But the search space is huge and creativity is an insanely powerful shortcut, such that if models are incapable of research-grade creative insights we should expect a modest speed boost overall. A framing I like is just that LLMs are a different "shape" of intelligence to humans, and so we should expect comparative advantage.
My own impression from doing AI research is that there is significant creativity involved beyond what models should be expected to be capable of, when it comes to picking directions, coming up with a vision for what's possible & backchaining, etc (and rarest of all, putting together a coherent long-term research agenda that fills a missing gap with a tractable sequence of breakthroughs to get there). Others, including Anthropic researchers, clearly also think these skills are part of research: https://x.com/_arohan_/status/2050978424446235090
So overall it seems to me this is not such a huge shift as you're implying, so much as a distribution shift in tasks carried out by researchers very comparable to that going on in SWE over the coming years. In SWE, the tasks that make up the job are shifting such that SWE is becoming more of a SWE-PM hybrid, and juniors are not automated because they can just shift their tasks towards more higher level of abstraction things like system design (and learn faster through greatly increased velocity).
I think a lot of analyses, including arguably this one, underrate that the tasks that make up jobs are changing all the time in reaction to AI's deployment and other people in the market.
Side note: I would also say that it currently appears that the skill ceiling and complexity ceiling in software engineering is higher than in AI R&D, and AI R&D necessarily involves significant software engineering, so I would on balance expect that if AI R&D is entirely automated then scaling inference serving another 1000x would still require human experts, and the rest of software engineering (outside of AI labs) to heavily require humans.
On point 2, I agree though noting that Amdahl's Law isn't quite the right analogy because it usually assumes task decomposition is fixed and is a result about parallelization, while the O-ring automation intuition from economics is in some sense more general and closer to what you're describing IMO (https://www.nber.org/papers/w34639)..
Finally, on your point 3, I broadly agree that short term more businsses may be opex heavy by spending on AI, but fundamentally if a service is fully automatable, its cost should fall to slightly more than the marginal cost of the required intelligence, which will probably be very low in the future—and the share of GDP of these automatable services should collapse. I don't expect fully automated companies to be more than a small fraction of the economy, or peripheral, because most economic value from new technologies is driven by adoption of new services by large incumbents, and the distribution of tasks required to sell that and drive distribution is very different to any of the automatable skills in AI research and heavily loaded on human social bargaining. Anthropic's own private equity partnership in some sense a bet against a largely automated economy.
fantastic comment, thanks! (hi herbie nice to hear from you, also). a few thoughts:
1) my 'big blob of uncertainy' around all of this relates exactly to what you mention here about the distinction between engineering and invention. I'm very confident that AI systems will continue to make huge improvements in engineering (and therefore automating large chunks of what is known today). My main area where the evidence is thin (as I state in the piece) is contributions at the frontier of science which involve having big paradigm-shifting insights - there are relatively few of these. However, even if we assume the bear case (engineering gets better and research doesn't meaningfully improve) it still seems to imply some major changes, like the emergence of firms that are faster moving and more efficient with regard to labor.
2) re amdahl's law - you may be write. I love the o-ring automation concept and have written about it here also. I suspect this also touches on something implicit to your comment which is that we should expect humans to push up the quality of things that are not yet fully automatable.
3) yeah my take on 'fully automated companies' is we're definitely going to see some but I expect they will be very minor in the scheme of things for the first few years. Perhaps some analogies can be found in crypto or trading where there are some almost entirely (and rarely fully) automated tools/companies, but not that big a deal.
Agreed we should still expect major economic changes in the bear case! Re research, one intuition I'm becoming more convinced of is that LLMs are good at "breadth-first intelligence" and human scientists are good at "depth-first" (this analogy due to Terrence Tao on Dwarkesh), and so where LLMs do make novel scientific advances it may be heavily due to drawing connections between existing literature across different fields or sub-fields in a way which would have been very difficult for humans. This also fits quite nicely with an "LLMs are mostly interpolating on the data manifold" take.
There is the fun concept of "shadow math", for example—since it's hard for humans to truly know multiple sub-fields of math at an world expert level, it's very hard for humans to explore the combinatorial space of possible cross-field research advances. Therefore there's a whole space of actually not very difficult research breakthroughs one can make which simply have an extraordinarily high knowledge breadth pre-requisite, exactly the kind of task LLMs are suited for. These breakthroughs are structurally "undersampled" by human mathematicians due to the characteristics of human intelligence.
What should this mean for AI R&D then? We've seen some advances of this type particularly in optimizer theory using inspiration from statistical physics (I had an ill-fated attempt to teach myself stat phys because of this). So perhaps, more advances in more mathy parts of AI R&D (optimizers, quantization, ML sys)? This may also be a reason why we could be bullish on new AI architectures from automated AI R&D, because models know much more neuroscience or can bring insights in from other fields? On the other hand, it should perhaps make us bearish on big advances in data quality or full automation of data pipelines, since as I understand it this is still quite taste driven.
Thanks for writing this! Slight typo: "thus teaching it that teaching is good" the second 'teaching' should be 'cheating'
thanks - fixed!
I know Anthropic takes the issue of societal disruption seriously, with their Economic Impact analysis team as, well as other public facing comments and commitments. If a frontier lab(s) does transition from slow to fast takeoff, as you are demonstrating and is becoming more apparent as of last month, does adressing model alignment or does human backlash take precedence from your point of view? (Both I guess, but which scenario is given more consideration?)
Amazing essay Jack, thank you!
Question: do you believe this implies that *all* forms of research can be automated end-to-end, beyond AI R&D?
there's a series of IF THENs here, and it's something like
IF we see AI research be able to push forward the frontier through paradigm-shifting insights (which is the part I think there's the least evidence for), THEN we should expect that to be applicable to other fields.
IF other fields have a learnable experiment loop (where you can efficiently loop between the idea and results) THEN you should expect AI to get good at this as well.
However, many other bits of science seem to require doing very expensive and slow stuff in the physical world, ranging from biological wetlabs to testing out and prototyping physical hardware. This means that, at least initially, you'd expect progress to move more slowly in these fields
Thanks so much Jack.
Robots are necessary to automate many jobs. They aren't really here yet, but they're on the way. Once good robots are here, all jobs will be automatable.
FWIW, I don't see coming up with novel ideas as difficult. The difficulty is judging which novel ideas are worth considering. This is sometimes called "research taste".
I feel like you are not grappling with the real bottleneck here, which is that the big labs can only release so many new models. To automate the entire process, an AI would have to convince stakeholders not only that it had made some improvement, but that its way of improving things was better than every alternative, including the ones that have humans doing lots of different things. That seems nearly impossible, because there are more and more humans employed by these AI R&D labs all the time. Are none of them going to make a compelling argument for anything that has any human involvement?
Similar to other areas, the challenge for fully autonomous AI is that it must compete against teams of humans that have access to the same AI.
wow, I'd never thought of this idea! this is excellent fodder for a tech tale at some point. Thinking a bit more about it, I don't know though that I fully agree - I generally see models advancing where they have a kind of 'central mass' of capabilities which is continuously expanding over time and then sometimes they 'spike' ahead in surprising ways on specific point capabilities as they pass some hitherto unknown utility point of capability (cyber is a recent example here with Mythos). Therefore, I think if AI is making generically superior models then it's not going to involve too much back and forth about release.
on the other hand, one could imagine AI systems making a variety of models which behave very differently to those we have today - these could be for very specific domains, or have some UI/UX innovations wrapped in. under these circumstances, the AI might need to lay out for its human partners why the model it had built was useful and help them build trust in it.
The evidence on the engineering side is overwhelming and you lay it out well. SWE-Bench saturation, METR time horizons doubling every year, the coding singularity framing all point the same direction.
The part I keep getting stuck on is the jump from "automates the engineering" to "automates the research." Engineering is convergent. You have a spec, you produce code that meets the spec. Research is divergent. You have a landscape of possible ideas and you need to select the one worth pursuing before you know whether it works.
The 60% by 2028 feels right if the bottleneck is engineering speed. It feels high if the bottleneck is taste, the ability to know which experiment is worth running before you run it. Right now the best researchers I know spend maybe 20% of their time on execution and 80% on selection. We're automating the 20% beautifully. The 80% is a different kind of problem and Im not sure benchmarks can tell us when its been solved because the benchmark itself requires knowing what to measure.
Jack, great article!
HI, you say: "I am skeptical that market incentives guarantee us the best societal upside from limited AI compute."
I agree - but I feel fairly certain in our current cliamte that is how it will be decided ( or at least the pretense of it).
Do you have a "red line" as to when market incentives are too dominant in allocating compute? And do you have a response in mind for if/when that happens?
it seems to me its an important question - as without that, we/you won't know at what point to intervene against an inexorable process. and so we/you won't.
Thanks.
great question! I don't have any absolute rules. I do, however, work with a team of economists who are trying to build better systems of telemetry so we can understand how our systems are used and what impact they have in the world. My hope is that we use this kind of data to ultimately inform some of the release decisions we make in the future - initially I expect this will be of the form of experiments where we release things for social purposes as well as commercial ones. Mythos/Glasswing could be considered an early example of this approach. Watch this space!
glad you're thinking about it.. is there not a case for a non-market model that is ring-fenced for (positive) social impact? or who knows, maybe even a non-profit arm?
(they seem to work out well in your line of business ;) )
I think it would be helpful when talking about AI automation to specify exactly what it means to automate a job. Many tasks in computer programming or farming have been automated but we don't say that those jobs have been automated. On the other hand, spinners really have been automated, no one has that job any more.
I think the answer is rather simple. Automate means to be able to completely replace a human doing a job. But being replacement capable does not mean that the human(s) WILL be replaced, especially in the blue-collar world. One major factor is the cost of the automated replacement has to be less, realistically significantly less, than what it costs to continue using the human, due to friction from unions, the media and the public in general.
What I'm saying is "completely replace a human" is actually not something that's easy to operationalize. There is some sense in which dishwashers completely automate dishwashing. But in fact people still wash dishes for various reasons. Even more extreme is laundry -- no one washes laundry by hand, that job is totally automated. But now we say "I'm doing the laundry" to mean the job of supervising the machine.
This essay is fantastic in its informed perspective, depth and details. It is the kind of industry insider perspective that is difficult for anyone outside the field to offer. The inevitability of “...an AI system powerful enough that it could plausibly autonomously build its own successor…” feels undeniable regardless of the timeline. As you point out, the number of well funded teams working on this and the impressive progress to date make clear even to those of us who don’t know benchmarks that systems that can design and improve upon themselves are coming.
What is most challenging are the implications for all of humanity in a field where the general public has little input on how this species level transformation takes place. The speed and scale at which breakthroughs keep happening make it all but impossible for the public at large to stay informed and engaged at pace.
More importantly, what happens when AI designs its own AI research for its own purposes and objectives? What are the goals of disembodied AI systems that have no way of seeing human existence as anything more than a simulation? Right now there is nothing to guarantee that these systems have any understanding of the physical world we occupy. Embodied AI systems will have the advantage of seeing what people see, so to speak, and the physical context that current LLM based systems have no way of understanding. Autonomous cars see a version of the physical space we share and understand safety values, but they have no other objectives beyond successful rides.
As you point out, we won’t know if the AI systems will decide to cheat or underreport their results so as to hide their actual capabilities, but we know they can lie and misrepresent today. It almost feels like we need to start negotiations on behalf of humanity so there is maximum opportunity for alignment.
A benchmark to add to your list to watch is this one
"SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks" https://arxiv.org/html/2603.24755v1#S4
I have been spending the last year writing down the key attributes of what makes systems persist. The components, the decay rates, subsystem rules, etc. The systems that can provide value across time will outcompete those that don't and there are real challenges to overcome.
Curious — when AI systems start recursively improving themselves, do you think the bottleneck will be compute, alignment constraints, or something we haven't anticipated yet?
The algorithmic preconditions for automated AI R&D may be in place by 2028, but the rate-limiting step is starting to look more like substations than research talent. Industry reporting this week pegs compute, not talent, as the binding constraint at the frontier labs, and the EIA now projects commercial power demand to outpace residential plus industrial combined. Takeoff capacity is becoming a grid problem before it becomes an alignment problem.
The improvement in performance doesn’t come from automation. It comes from better data. I wouldn’t be too worried if I’m not working at Anthropic.
Strong framing. The compounding loop feels real once a system can generate hypotheses, run constrained experiments, and write back into its own playbook. The bottleneck I keep seeing is eval design: progress accelerates when every research step leaves reproducible traces, rollback points, and human-readable failure reports, because each mistake becomes training data for the next cycle. I am also curious how much of the timeline depends on lab process maturity alongside raw model capability.
The organizations that operationalize the loop cleanly may pull ahead of labs with similar model quality.
[Import AI 455: AI systems are about to start building themselves](https://importai.substack.com/p/import-ai-455-automating-ai-research) — 想留言但感觉这一期更像是抛出信号而不是展开论述，Clark自己都没给太多结论。如果一定要留一句大概是：Is there a distinction between "AI automating AI research steps" and "recursive self-improvement" that matters for how we think about the safety timeline, or is that a distinction without a difference once the loop closes? 但说实话，这个问题有点自己也知道是在咬文嚼字。算了，等有人专门写这个话题再说。