By Donato Technologies · 5 min read
TL;DR
- AI is already helping build the next AI. That feedback loop is what people mean by an “intelligence explosion.”
- A recent Oxford paper by Toby Ord argues the “infinite vertical line” version is harder to reach than most models assume. Fast but finite is the more likely shape.
- Fast but finite is still brutal. A plain 10x speed-up means a decade of progress every year.
- The real cost for IT teams isn’t learning new tools. It’s unlearning old beliefs, on a deadline.
- What to do: re-test your assumptions monthly, write constraints instead of steps, measure your own build loop, and hire for how fast people update.
If you use AI every day, you’ve already lived this:
A limit you built a workaround for in spring is gone by summer. Something you told your team “AI can’t do” is now a Tuesday-afternoon task. Your best prompt from last quarter already looks a little embarrassing.
That’s what a steep curve feels like from the inside.
What’s actually “exploding”?
The loop.
AI helps design, code, and test better AI. That better AI helps build the next one. Researchers call this recursive self-improvement (RSI), and the idea goes back to I. J. Good in 1965.
It’s not just theory anymore, at least in small form. Ord’s paper points to AI that has already:
- Found better algorithms for training neural networks
- Designed a more efficient way to multiply matrices
- Helped write code for new AI systems
- Run experiments to improve other AI systems
Small examples. Same loop.
Is the “singularity” coming?
Probably not the vertical-line version. Here’s why, in plain English.
Ord, from Oxford’s Martin AI Governance Initiative, focuses on one number most forecasts skip: generation time. That’s how long it takes to go once around the loop. Design a chip. Train a model. Fine-tune it. Ship it.
For growth to hit infinity in a finite time, that loop has to shrink toward zero. Not “get shorter.” Zero.
Here’s the toy version. Say the first doubling in computing speed takes 2 years. The next takes 1 year. Then 6 months. Keep halving and, on paper, you hit infinity by year 4.
In the real world, you can improve the recipe forever. The oven still needs 40 minutes. Chips still have to be built. Models still have to be trained. Ord’s read is that loop time almost certainly has a floor.
So what shape does it take instead?
Ord sketches one plausible path. Our short version:
| Phase | What it looks like | What it feels like on your team |
|---|---|---|
| 0. Human-speed | Steady exponential growth, humans doing the research | Where things were a couple of years ago |
| 1. Loop speeds up | AI takes over more of the R&D, loop time drops, growth rate climbs | New capabilities show up faster than your review cycles |
| 2. Machine-speed | Fully automated loop, still compounding, but at a much higher rate | Your playbooks expire in weeks |
| 3. Plateau | Limits kick in (physics, data, hardware) and the curve flattens | Things slow down, but at a level that’s hard to imagine today |
Ord is clear that this is one plausible path, not a forecast. Nobody knows which phase we’re in.
“Finite” doesn’t mean “slow”
Ord makes a point that’s easy to miss: you don’t need a singularity for this to be disruptive.
If AI simply sped up research 10x, you’d get a decade of human-paced progress every year. No vertical line required.
And one more thing worth saying out loud: even tracking this is hard. One widely used yardstick, how long a task AI can finish on its own (METR’s time horizon), has grown exponentially, maybe faster. But that measures task length, not general intelligence. And Ord’s broader warning is that the data we have is too noisy to tell us, from the inside, whether a true explosion is underway.
So stop arguing about the forecast. Build for the shape you can see: fast, uneven, and hard to time.
The real cost: the tax of unlearning
In most fields, experience compounds. In AI, it expires.
Your expertise has a short shelf life. Last year’s hard limit (context length, reasoning, handling images and audio together) turns into a footnote in months. Your baseline has to include some skepticism about your own expertise. That’s uncomfortable if you’ve spent a decade earning it.
You’re moving from builder to orchestrator. You used to write the code or run the workflow. Now you direct agents that do it. That means thinking less in steps and more in systems and constraints: What does “done” look like? What’s off-limits? How do we check the work?
Science fiction keeps turning into Tuesday. Benchmarks that sounded like sci-fi a year ago are routine utilities now, often before the analysis of them gets published.
The people who handle this well aren’t the ones with the best forecast. They’re the ones who can throw out a mental framework on a Tuesday afternoon because the ground moved, and not have a crisis about it.
Call it cognitive elasticity. It’s trainable.
What to build (starting Monday)
- Keep a “can’t do” list. Write down 5 to 10 things you believe AI can’t do. Re-test them monthly. Cross off what’s wrong.
- Put an expiry date on your playbooks. Prompts, workflows, internal guides: add a “re-check by” date. We’d start at 90 days.
- Write constraints, not steps. Give the agent a goal, limits, and acceptance tests. Let it pick the steps.
- Measure your own loop. Ord’s key number is loop time. Yours matters too. Track how long it takes to go from idea to shipped, then shrink it. You won’t get to zero, and that’s fine.
- Practice reviewing. Producing is cheap now. Judging is scarce. Treat reviewing agent output as a skill you train, not a chore.
- Rebuild one workflow a month from scratch. Pretend the old way never existed. See what falls out.
What doesn’t work:
- Waiting for the tools to settle. They won’t.
- A one-time “AI literacy” workshop. It expires in a quarter.
- Debating the forecast. The data can’t settle it.
If you’re hiring
This part’s for TA leaders and hiring managers.
- Hire for update speed, not tool history. “5+ years of experience with [AI tool]” is a trap when the tool turns over in months.
- Ask the unlearning question. “What did you believe about AI six months ago that you’ve since dropped?”Strong answers are specific, a little embarrassed, and recent.
- Test orchestration. Hand over a task, an AI tool, and a few constraints. Watch how they scope it, check it, and fix it.
- Write job posts around outcomes and judgment. Tool lists go stale before the posting does.
At Donato, we help companies across the U.S. staff IT roles, and these are the traits we look for in people. If you’re building a team that works this way, we’re happy to compare notes.
FAQ
What is an intelligence explosion?
A runaway feedback loop where AI systems help build better AI systems, which are then better at building the next one. The term comes from I. J. Good in 1965.
What is recursive self-improvement (RSI)?
It’s the process of AI being used to improve AI. It can mean anything from AI helping humans write code to AI doing most of the research work itself.
What is “generation time” in AI?
The time it takes to go once around the improvement loop, such as designing and training the next model. In Toby Ord’s analysis, it’s the key variable: growth can only reach a true “singularity” if generation time shrinks toward zero.
Will AI progress go infinite?
Probably not. Ord’s analysis suggests the vertical-line version is harder to reach than common models assume, and real-world limits like hardware, data, and physics are likely to cap growth. But fast, finite growth can still outrun most planning cycles.
How should IT teams prepare for fast AI progress?
Re-test assumptions on a schedule, write constraints and acceptance tests instead of step-by-step instructions, measure your own idea-to-shipped time, and build review skills for agent output.
What skills should IT hiring managers look for in the AI era?
Learning speed, comfort changing their mind, and the judgment to direct and check AI work. These tend to age better than experience with any single tool.
Source: Toby Ord, “The Dynamics of Intelligence Explosions,” Oxford Martin AI Governance Initiative, University of Oxford. link
