What is recursive self-improvement?

Recursive self-improvement (RSI) is a theoretical process by which an AI agent could enter a feedback loop that results in it improving itself very quickly. For example, an AI capable of human-level AI research might make adjustments to its own architecture (software or hardware) that lead to improved performance, which in turn enables it to make further adjustments that lead to even more improved performance.This would create a cycle of increasingly advanced capabilities that could quickly far surpass humans.

AI agents may be able to recursively self-improve much faster than humans because they can be copied, edited and run on new hardware, while humans are limited in our ability to alter more fundamental aspects of ourselves like our biology. Methods for doing this, like neurosurgery, nootropics, and cognitive implants, are new and/or risky. Each cycle of self-improvement could reach ever-higher levels of capabilities, and even a slow AI takeoff could leave us with AI agents that are superior to humans. However, few AI systems are designed to pursue recursive self-improvement as an explicit goal^[1].

Not everybody agrees that RSI is possible. François Chollet argued in 2017 that recursively self-improving systems cannot achieve exponential progress in practice; Eliezer Yudkowsky wrote a rebuttal shortly after. If RSI is possible, it seemsis unlikely that an AI agent would choose not to pursue this strategy given the benefits it could provide, since it may be an instrumentally convergent path.