Isn't the real concern technological unemployment?
Technological unemployment through AI, while not the kind of life-and-death risk for humanity that we focus on, is a real concern.
In the past, technology has often been blamed for causing unemployment within certain sectors. Weavers, buggy whip makers, and bank tellers lost their jobs when they were automated, but technology also produced new kinds of jobs or transformed the jobs that were made obsolete. These new and transformed jobs have required workers to learn new skills, which was possible when changes were somewhat slow.
Some fear that the rate of job automation through AI could be much higher than in the past, and could affect a much larger proportion of jobs.1 In particular, AGI might replace humans in most white-collar jobs quickly without offering many replacement jobs, which would lead to a brutal transition and possibly social unrest.
Nevertheless, soon after AGI could automate these jobs, assuming humanity is not wiped out, we might enter a post-scarcity economy which would eliminate the need to work for subsistence. We might still expect people to want to work for personal fulfillment or to do tasks that we prefer be done by humans (e.g., tasks related to caring for other humans), but not needing to work to survive would probably help the transition.
We do not concentrate on technological unemployment because we fear that a superintelligence with the power to outcompete many of us in the labor market might also be able to kill us all, and we concentrate on the latter risk.
Others argue that this could be slower, for instance that a large portion of the work in tech jobs is not automatable by large language models (LLMs) even if they can program as well as a human ↩︎