Max Headroom 
UNCLASSIFIED
Quantum computing (QC) and AI have one thing in common: They make mistakes.
There are two keys to handling mistakes in QC: We’ve made tremendous progress in error correction in the last year. And QC focuses on problems where generating a solution is extremely difficult, but verifying it is easy. Think about factoring 2048-bit prime numbers (around 600 decimal digits). That’s a problem that would take years on a classical computer, but a quantum computer can solve it quickly—with a significant chance of an incorrect answer. So you have to test the result by multiplying the factors to see if you get the original number. Multiply two 1024-bit numbers? Easy, very easy for a modern classical computer. And if the answer’s wrong, the quantum computer tries again.
One of the problems with AI is that we often shoehorn it into applications where verification is difficult. Tim Bray recently read his AI-generated biography on Grokipedia. There were some big errors, but there were also many subtle errors that no one but him would detect. We’ve all done the same, with one chat service or another, and all had similar results. Worse, some of the sources referenced in the biography purporting to verify claims actually “entirely fail to support the text,”—a well-known problem with LLMs.
Andrej Karpathy recently proposed a definition for Software 2.0 (AI) that places verification at the center. He writes: “In this new programming paradigm then, the new most predictive feature to look at is verifiability. If a task/job is verifiable, then it is optimizable directly or via reinforcement learning, and a neural net can be trained to work extremely well.” This formulation is conceptually similar to quantum computing, though in most cases verification for AI will be much more difficult than verification for quantum computers. The minor facts of Tim Bray’s life are verifiable, but what does that mean? That a verification system has to contact Tim to verify the details before authorizing a bio? Or does it mean that this kind of work should not be done by AI? Although the European Union’s AI Act has laid a foundation for what AI applications should and shouldn’t do, we’ve never had anything that’s easily, well, “computable.” Furthermore: In quantum computing it’s clear that if a machine fails to produce correct output, it’s OK to try again. The same will be true for AI; we already know that all interesting models produce different output if you ask the question again. We shouldn’t underestimate the difficulty of verification, which might prove to be more difficult than training LLMs.
Regardless of the difficulty of verification, Karpathy’s focus on verifiability is a huge step forward. Again from Karpathy: “The more a task/job is verifiable, the more amenable it is to automation…. This is what’s driving the ‘jagged’ frontier of progress in LLMs.”
What differentiates this from Software 1.0 is simple:
Software 1.0 easily automates what you can specify.
Software 2.0 easily automates what you can verify.
That’s the challenge Karpathy lays down for AI developers: determine what is verifiable and how to verify it. Quantum computing gets off easily because we only have a small number of algorithms that solve straightforward problems, like factoring large numbers. Verification for AI won’t be easy, but it will be necessary as we move into the future.
AI & ML, Commentary
Radar
