In an AI world the biggest challenge will be the interaction with humans
A lot safer
AI run vehicles are ready to take over. They are far better and more trustworthy than human beings. In the US only, people kill 40.000 people in traffic every year. On average all of us kill another person in traffic every 1.8 million kilometre. So the very moment a robot does a fraction better, we save 1000s of lives. If all cars would be AI powered, the number of fatal accidents would be close to zero.
The problem with autonomous vehicles are not the rules, not the sensors not the computing capacity nor their experience. The interaction with us, human creatures makes it extra challenging. Algorithms have to learn that in between what they know is best and performing that, there are humans and humans are pretty unpredictable. Just imagine autonomous vehicles merging (*) in between human driven ones or crossing a road without traffic lights. Either the robot is too polite and immobilized, or it is to aggressive and causes accidents anyhow. When all cars are autonomous, the problem would be solved. Not in a hybrid world.
Training robots to understand that humans are not a barrier
As a result, in between that fully autonomous Day After Tomorrow and Today, there is a Tomorrow that is most challenging. In the future robots will have to interact with humans, and not consider us a barrier in between them and their goal. Computers need to learn to interact. The most difficult element to overcome in this learning curve is the fact that computers need to 'realise’ that their actions can influence human behaviour. Algorithms have to understand their own behaviour and 'read’ the effects it has on humans to really interact. In order to make sure computers understand us, we have to be able to explain why we act the way we act and why certain actions are 'good’ and others are 'bad’ and that good and bad are not absolute but related to circumstances. It forces us to deep-dive into essential human issues that we never really had to explain to other humans, but will be essential when talking to those alien brains.
The lessons learned are these: in an AI world the biggest challenges will be the training of the AI in a safe environment and the interaction with human beings. Training will always involve big data and the interaction with humans will make us question ethics, sociology, psychology, morality and even philosophical issues.
(* In traffic engineering, the late merge or zipper method is a convention for merging traffic into a reduced number of lanes. Drivers in merging lanes are expected to use both lanes to advance to the lane reduction point and merge at that location, alternating turns.)