by Tim Menke
figures by Neal Akatsuka

Imagine getting into your car in the morning, sipping your coffee and sitting back to relax while your car drives you to work. Then you remember to call a friend who you have not spoken to in a while, or you have a look at the amazing photos from your latest vacation. It is, of course, not a safe idea to do any of these things if you had to navigate your car through the busy morning commuter traffic. But if our cars could drive us autonomously, we could spend the freed-up time in more pleasant or productive ways. Given the current rate of technological development, the advent of self-driving cars is inevitable. Is this a blessing or a curse? While there are safety and policy concerns that need to be addressed in the development phase, the benefits to road safety and quality of life will prevail in the long run.

The technology explained

The players in the self-driving car market are diverse: traditional car manufacturers like Nissan, Audi and Mercedes, and new companies such a Tesla, Google’s Waymo and Uber, are all competing to develop the first fully autonomous self-driving car. The approach they are taking is similar across the board. Essentially, a self-driving car needs to perform three actions to be able to replace a human driver: to perceive, to think and to act (Figure 1). These tasks are made possible by a network of high-tech devices such as cameras, computers and controllers.

Figure 1: Like a human driver, a self-driving car executes a cycle of perceiving the situation, thinking about it and performing an action depending on the assessment. High-tech devices take over each of these tasks.
Figure 1: Like a human driver, a self-driving car executes a cycle of perceiving the situation, thinking about it and performing an action depending on the assessment. High-tech devices take over each of these tasks.

A self-driving car’s “sight” or perception is achieved via several mechanisms. Cameras are installed on the roof and in other places around the car for 360º vision (Figure 2). While cameras are good for identifying shapes and colors in the environment, they are not helpful in finding the distance to an object. Fortunately, self-driving cars also have high-tech sensors, called LIDARs, that are much better at determining the location of people and objects around the car. LIDAR (or “Light Detection and Ranging”) is a device with a constantly rotating laser beam. It sends invisible light pulses all around it. By measuring the travel time and location of the reflected laser light, it can discriminate the shape and location of surrounding objects. In addition to cameras and LIDARs, self-driving cars often have conventional radars. Although radars can only see very fuzzily, they are great at detecting moving objects and are much cheaper than the other devices.


Figure 2: A side-by-side comparison with a conventional, human-driven car illustrates where the components of a self-driving car are located. (Photo of Waymo self-driving car by Grendelkhan on Wikimedia Commons. Photo of human-driven car by Mick Tinbergen on Unsplash.)
Figure 2: A side-by-side comparison with a conventional, human-driven car illustrates where the components of a self-driving car are located. (Photo of Waymo self-driving car by Grendelkhan on Wikimedia Commons. Photo of human-driven car by Mick Tinbergen on Unsplash.)

Once a self-driving car “sees” its surroundings, the cameras, LIDARs and radars all send the information they perceive to the “brain” of the car. The “brain” of a self-driving car is a powerful computer that often sits in the trunk and controls the car’s thoughts and actions. By use of GPS, it initially has a rough idea of the route it must follow to get to a destination. This is not enough, however, to navigate along a busy street, to change lanes or to stop at a traffic light when necessary. Therefore, the computer takes in camera, LIDAR, radar and GPS data and determines the precise location of the car as well as the objects around it. Programmers have developed very smart algorithms that can be “trained” to identify objects’ in the car’s surroundings. For example, these algorithms classify a bulky, fast-moving object with two wheels as a motorcycle, rather than a bicycle. Other objects would be identified as cars, pedestrians, traffic lights or obstacles, accordingly.

Once the computer has determined which objects surround the car and how far away they are, it must decide how to act and then execute its decision. For example, if the lane is closed ahead due to construction, the computer will identify the options: it could either change lanes immediately or slow down to let the vehicles in the next lane pass first. But how does the computer know which option is better? Programmers have developed algorithms that behave like the neurons in a brain. These algorithms can “learn” from the actions taken in previous situations and infer what to do in a new, similar situation. This type of machine learning is like speech recognition on your smartphone or facial recognition for photos on social media. Once the car’s computer decides what action to take, it sends electronic commands to controllers that turn the steering wheel and control the throttle and brakes.

It is important to note that this cycle of gathering information and deciding and executing an action is repeated several times per second. All this is by enabled high-speed computers and very smart algorithms, which allows the computer to constantly assess the situation and make quick decisions.

 What is needed in technology and policy to get self-driving cars into the market?

Once they are fully developed, self-driving cars will potentially have a huge positive impact on our safety and lifestyle. The high-tech vision system has the potential to outperform humans in detecting dangerous situations. Unlike distracted or drunk drivers, self-driving cars always operate at their maximum ability. Therefore, they are likely to reduce accidents and lower the huge death toll on our roads. Additionally, the smart algorithms will find faster routes to our destinations, drive more efficiently and consume less fuel. We will have the added benefit of spending our time on other tasks while our car is driving for us.

Given these advantages and the fact that the technology required to build self-driving cars is basically available, why haven’t they entered the market yet? It turns out that present tests of self-driving cars have been limited either to specific roads or to specific tasks. In order to navigate down a road, the self-driving car requires very precise maps of the street and surroundings. These need to be much more detailed than those found on common online map services and will require more time to be created for every road in the country or the world. Moreover, more work needs to be done on the computer algorithms that constitute the car’s “brain.” Software engineers do not yet trust their programs to correctly assess all the possible situations that can occur in traffic. In addition, more data needs to be collected to reliably train the machine learning algorithms. Therefore, testing of self-driving prototypes is still closely supervised. A safety driver is present at all times, ready to intervene if the car makes a mistake.

An example from Tesla illustrates that the technology is not yet ready for fully autonomous driving: a Tesla driver died when his car with partial self-driving features crashed into a turning truck that it did not recognize as an obstacle. It is important to note that the self-driving feature involved in the Tesla crash was explicitly not meant for fully autonomous driving. The driver had been warned several times to put his hands back on the wheel. However, the accident illustrates that more programming and engineering is required to make cars fully autonomous. Despite these challenges, the rate of progress on self-driving car technology suggests that the goal is within reach.

Recognizing that companies are working diligently and rapidly to release the first self-driving car, policymakers have acted to ensure the public’s safety in the development phase of self-driving cars. Last year, the US Department of Transportation issued federal guidelines for the safety and testing of autonomous vehicles. For the moment, the guidelines do a good job at balancing safety concerns for the public and freedom for the manufacturers to drive the technology forward. They require companies to report to the government on how they handle self-driving car safety. At the same time, the guidelines ensure that American companies are mostly unrestricted and can stay competitive. This is important, as the first company to release a fully self-driving car will gain a huge competitive advantage in the global market.

Self-driving cars are an exciting new technology that has the potential to deeply transform transportation, making it safer and improving our quality of life. In order to operate, self-driving cars rely on high-tech devices, powerful computers and advanced algorithms. In the next few years, we should expect a tight race in the development of the first fully autonomous car.

Tim Menke is a graduate student in the Harvard Physics PhD program. He works in the field of quantum information processing at Harvard and MIT. On the side, Tim is interested in the interplay between technology and public policy.

For more information:

Cover Image citation: Photo by Redd Angelo on Unsplash.

14 thoughts on “Self-driving Cars: The technology, risks and possibilities

  1. Superb site you have here but I was wondering if you knew of any community forums that cover the same topics talked about here? I’d really like to be a part of online community where I can get feedback from other experienced individuals that share the same interest. If you have any suggestions, please let me know. Thanks a lot!

  2. Level 3 self driving autonomy is still in debate. We can see the example of Audi A8 ambitions for total self driving, that means no hands on steering wheel at all. Unfortunately they had to back off, the reason? State laws
    Thus, we need proper laws in place first and foremost.

  3. hey Tim,
    Totally agree with you, applications are not yet entirely self-sufficient. Still the pace of development on self-driving vehicle technologies, amid these obstacles, clearly shows that the target should be within grasp.

Leave a Reply

Your email address will not be published. Required fields are marked *