How promising is compute usage regulation for AI governance?

Recent AI progress has been characterized as relying on the "AI triad": compute, data, and algorithms. AI development cannot proceed without all three, making regulation of these components a candidate strategy for AI governance. Regulating compute may be especially viable, because:

• Compute cannot be used simultaneously by more than one actor, making it possible to restrict it more effectively than data and algorithms, which can be copied and used by anyone once they have been made public.

• Compute takes up substantial physical space and requires physical materials to build and maintain.

• Amounts of compute are easily quantifiable (e.g., in terms of FLOPS or number of CPUs).

It’s also easier to govern compute because of some related features of the computing hardware industry today:

• Training the best AI models today requires especially large amounts of compute. This means they require a lot of physical infrastructure. This can involve “football-field-sized supercomputer centers'', which have extremely high energy and water demands, making them easy to track and detect.

ML training runs are more expensive than they used to be, so much so that Sevilla et al. call the current era of ML the “Large-Scale Era”¹

• The compute industry is dominated by cloud service providers like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud — which makes it easy to track and govern.

• The semiconductor industry, which produces the computing hardware, is also highly concentrated, making it easier to trace. For example:

  • The world’s largest contract manufacturer of semiconductor chips, TSMC, has a near monopoly on commercial production of industry leading chips².

  • The world’s largest semiconductor supplier, ASML, is the sole supplier of EUV photolithography machines, required to produce the most powerful chips.

• The production is so highly concentrated because chip manufacturing requires immense startup costs, and involves tacit knowledge about how to make the process work.

However, there are a number of limitations to compute governance. For example:

• Improvements in algorithms might mean that less compute will be necessary to make dangerous models.

• Improvements in hardware efficiency might make it possible to train dangerous models more cheaply and in a way that is more difficult to track.

• Scaffolding can allow AI systems to accomplish a broader range of tasks, for de facto increased capabilities, even without new training runs. Post-training optimization such as RLHF and supervised fine-tuning also increase capabilities of base models without substantially increasing the number of FLOPs used in their training.

• Compared to other interventions like supporting alignment research, compute usage regulation is more potentially adversarial and poses the risk of increasing political or geopolitical tensions. For example, some have identified this as an effect of the early 2023 US export controls for advanced microchips to China.