What can't AI do yet?

There are lots of things that current AI can't do. It can’t serve as a drop-in replacement for a car-mechanic, a nurse in a hospital or a mathematician. Certainly, an LLM such as GPT-4 hooked up to some simple tools can help do some of the tasks in pretty much any job. But not all of tasks.

Moravec’s paradox: Things AI can’t do are often things humans find ridiculously easy e.g. walking around. Things humans find hard, often AI finds easy. E.g. learning how to fold towls? That’s a peace of cake for us, but only recently possible for AI. Beating the world champion in chess? Hard for us (except you, Magnus!) but easy for robots. But this isn’t a perfect match.

Humans don't need to see 1 million examples of pedalling a bike before they learn how to do it. They just try a few times, and succeed.

So what's the secret sauce that separates man from machine? We don't know. If we knew the answer, it'd help a great deal with making more powerful AI, or predicting when AI will gain what capabilities.

Researchers have proposed that "AI can't plan", "AI sucks at composing abstractions, but is great at comparing them", "AI is just mixing known capabilities together, so it can't do anything novel", "Single pass LLMs aren't Turing complete", "there's no special sauce, the AI just isn't big enough yet" and more. Unfortunately, none of these really explain the existing successes, and failures, of current models very well.

However, the fact remains that the graveyard of tasks that were thought impossible for AI is quite crowded. In particular, tasks that were thought impossible by LLMs are falling one by one. For instance, creating beautiful pictures autonomously, inventing interesting new mathematical concepts, or speaking English fluently. And indeed, for decades, AI couldn't do these things. Now it can.

In fact, the sheer number of tasks which have fallen to AI in recent years is part of what makes it difficult to say what current systems can't do. Now, we're scrambling to find some new way to separate modern machines from man. Don't be confident anything we find will last.