“Everything is going to be touched by AI”

Peter: I love the Google Duplex example and I think it shows how powerful AI can be. But how big is it? Andrew Ng, your PhD advisor once said that “AI is the new electricity”. Do you follow him in that?

Pieter: It's a very interesting metaphor. It means that every industry is going to be revolutionized by AI. Just like every industry – not merely the power generation industry – was revolutionized by electricity. It's not just IT companies like Google, Facebook, … that are going to be transformed by AI. It's going to be everyone: mining, manufacturing, healthcare companies, … everything is going to be touched by AI. There's a lot of truth to that because AI is the ability to automate information processing for some tasks which makes everything more productive and faster moving and that will give certain companies an advantage over others.

Peter: You’ve worked at OpenAI which Elon Musk basically founded because he wanted to make sure AI didn’t fall into the wrong hands. He talked about summoning the demon. You’re in this every single day: how potentially dark is this? Or is it just an exaggeration?

Pieter: There's a lot of uncertainty about how AI will progress over time.  But anybody I talk to seems to think that it is possible. Ultimately, what does it take to be intelligent? It takes some storage, some computation, some sensing, some interaction with the world and if you put enough of it into the right program, it should be able to be intelligent. It might require a very big computer and a lot of interaction with the world but there is no reason why it couldn't be built.  

What we are seeing is that people have very different estimates of how long this evolution is going to take. But everybody can see that AI progress has accelerated a lot. The big breakthrough happened in 2012 when Geoffrey Hinton’s lab at the University of Toronto showed that you can train a big neural network to recognise, with pretty good accuracy, what was in an image. They showed us that you can just collect a lot of data, put a big neural net in place and then train it to recognise the pattern in that data. From there big progress was made on image recognition, on machine translation, on speech recognition.

Of course, that doesn't give you full intelligence. Intelligence is not just about capturing input output patterns. Besides capturing input output patterns, you need AI that achieves goal-oriented behavior.  This is studied under "reinforcement learning" where big breakthroughs have happened too. But there’s a lot of progress to report already. DeepMind, for instance, is teaching AI systems to solve Atari games or play the very complex game of GO. Then in Berkeley – with OpenAI – we have robots learning to run and to manipulate objects.

That's what people have been seeing and that's why they've been wondering: for the first 50 years progress was so slow but, in the last few years, so many things became possible so quickly. What does that mean for the future? How soon are we going to build something that is a fully-fledged AI system? People like Bill Gates and Elon Musk, they think this could be very fast, like within five years. And they believe that even if there’s only a 0.1% chance that it’s within the next 5 years then we SHOULD be thinking about it now. And so that's where they're coming from. They’re not saying it’s going to happen now, but they’re saying there's a 0.1% chance or maybe a 1% chance (somewhere in that range) that it’s going to happen in the next 5 years. Given that the impact of AI will be so great, let’s think about it and take it very seriously.

Peter: Do you have any advice for companies out there who are trying to understand what AI could do for them? Is there any type of preparation that could help them be more successful?

Pieter: There's a few things to think about. One is that pretty much all the currently successful AI systems are based on having collected the right data. So your success greatly depends on understanding what data you should be collecting so that you can train your AI to understand certain patterns. My company Gradescope, for example – which built an AI for the automatic grading of homework and exams – collected a lot of data from instructors and teaching assistants grading exams and homework and, from that, we trained an AI to then help them in the future.

Another thing is that it's very hard to make progress without in-house AI experts unless you completely outsource things, of course. Often companies reach out to me and ask for a consultation about what AI could mean for them. It's very easy for people like me to give a little bit of high-level advice and tell them what they should be paying attention to. But then, ultimately, I feel it’s important to have a team of AI experts that can really execute on that. It's not enough just to get some high-level advice on what the important things are to look at.

Of course, hiring in AI is very competitive because there's more demand than there are experts. That's why we also see other types of companies emerging: like Element AI in Montreal with Yoshua Bengio, of course, who is one of the main icons there. They present themselves as a consulting shop for AI: big companies that don’t have their own AI team ask them to build the systems that they feel they need. We will start seeing more of that because it takes time to build an AI team. You're competing with very popular companies like Google, Facebook, … where people love to work because, among other things, those companies have very deep pockets. So, even though I believe it’s beneficial to have your own AI team, it's often very hard to build up one.

Peter: This is a pivotal time for AI. It is nearing a state where it's considered practical and will have a big impact. What are you still going to see in your lifetime, if you look back at this period? Will it indeed be a pivotal time or do we run the risk of another winter before we have our big break in AI?

Pieter: We might have another winter. It's definitely possible. But I don't see one anytime soon and the reason is twofold. Firstly, the current AI systems are generating significant revenue and anything that generates revenue keeps people working on it to generate more and more revenue. That's on the industry side of things. It’s very easy right now to justify hiring AI experts, to have them improve all kinds of things, opening up new opportunities at the company and so forth. That's 1 reason.

Reason 2 is that a lot of AI breakthroughs depend on three things: data, compute and new insights. Now we will continue to have smart people with new insights but these new insights will be a lot more powerful if they are paired with enabling factors of more data and more compute. As the rate at which data is being generated and collected keeps going up, that enabling factor keeps opening up new opportunities.

On the compute side there is, of course, Moore's law but that's a pretty slow-paced progression right now. In addition to that, there's a lot of specialised compute being developed for neural networks. Like chips that are designed to be really good at neural network computing which would give us between a tenfold and a thousand-fold speed-up for AI compute in the next few years. That's a very big change in compute capabilities. Because of those 2 enabling factors – more data and more compute –  smart people will keep coming out with great ideas. So I don't see any slow down in the next few years.