By Dr Nick Reed
Why you don’t have a self-driving car (yet)
The act of driving appears deceptively easy. With only four main actions – steer left, steer right, speed up, slow down – it seems that it should be almost trivial to develop electronic systems capable of managing these four operations. Google first revealed their work on self-driving cars over a decade ago and billions have subsequently been invested by vehicle manufacturers, technology companies, investors and governments in their development. So why are we still waiting for self-driving cars to make a global impact on the mobility of people, goods and services? Of course, whilst the actions might be simple, the real challenge lies in deciding which action to take and when.
Why does this feel easy to us?
Driving is not a task for which evolution directly has shaped our human abilities. However, our transport infrastructure, the vehicles that use it and the rules that govern the system have been shaped by those abilities. For example, steering, accelerator and brake controls became standardised to suit how we could mechanically manipulate vehicle behaviour. Once vehicles capable of sustaining high speeds were available, motorways were created to facilitate rapid journeys – these needed to have lane widths, road curvature and gradients that made it possible for humans to operate cars, trucks, buses or motorcycles at appropriate speeds; road signs needed to have text and pictograms that could be read from sufficient distance and so on. Faster vehicles necessitated more efficient braking systems to slow our vehicles sufficiently rapidly in response to emerging hazards. Motorway service areas are suitably spaced to enable us to meet our physiological needs to rest, refuel and refresh when completing longer journeys. With due consideration for safety and efficiency, the road system also accommodates individuals with a wide range of needs and capabilities and vehicles of different sizes, configurations and characteristics.
How should we consider introducing a self-driving vehicle to this environment that has been adapted to our human capabilities?
At first glance, it seems very similar. A self-driving vehicle has sensors to collect information about the environment surrounding it. Computing power substitutes for our brain, processing this information and deciding what to do next in the context of stored knowledge of the road system and the rules of the road. It then sends a message to actuators that cause the vehicle to follow the required instructions for steering, acceleration and / or braking. However, there are some crucial differences.
Firstly, the process by which electronic systems gather information about the driving scene is very different. Although other senses are relevant (particularly hearing and proprioception), driving is primarily a visual task. The central region of human vision has a resolution that measures in the hundreds of megapixels and connects to neural processing systems that are finely tuned to detect and resolve movements in 3D. It can do this readily, compensating for movement of the observer and any bumps or vibrations they experience (within limits!) and has a wide dynamic range, capable of dealing with highly contrasting and rapidly changing lighting conditions. This system is far from foolproof, as any book of visual illusions will demonstrate, and can be affected by fatigue, intoxication or any number of diseases or degenerative conditions. It only covers a forward horizontal field of view of around 200 degrees (with a much lower spatial resolution in the peripheral regions) and a driver must process reversed images presented in the mirrors to build a more complete picture of the surrounding environment. However, it is a system that has evolved over hundreds of millions of years to help us avoid predators or catch prey and the task of driving has developed accounting for its extraordinary capabilities.
By contrast, a self-driving vehicle may use cameras, lasers, radar or ultrasound systems for gathering information about its local environment. Each generates data about the scene with varying degrees of resolution and resilience to environmental and dynamic conditions it experiences. Sensors can be placed at locations all around the vehicle such that the system has a continuous 360 degree perspective on the outside world. This data then must be processed for the vehicle to build its own representation of its surroundings. Whilst self-driving vehicles with limited capability have been around for decades, recent developments in computer processing and machine learning techniques have dramatically increased the capabilities of electronic systems in synthesising and analysing multiple sources of data to extract key information and determine what to do next. As well as exceeding human capabilities in terms of visual field, different sensors can provide more precise information about the movement of other objects – radar gives very accurate information about relative speeds and locations of other objects, which can only be estimated by a human observer.
By contrast, a self-driving vehicle may use cameras, lasers, radar or ultrasound systems for gathering information about its local environment. Each generates data about the scene with varying degrees of resolution and resilience to environmental and dynamic conditions it experiences. Sensors can be placed at locations all around the vehicle such that the system has a continuous 360 degree perspective on the outside world. This data then must be processed for the vehicle to build its own representation of its surroundings. Whilst self-driving vehicles with limited capability have been around for decades, recent developments in computer processing and machine learning techniques have dramatically increased the capabilities of electronic systems in synthesising and analysing multiple sources of data to extract key information and determine what to do next. As well as exceeding human capabilities in terms of visual field, different sensors can provide more precise information about the movement of other objects – radar gives very accurate information about relative speeds and locations of other objects, which can only be estimated by a human observer.
Secondly, memory plays distinct roles for humans and self-driving vehicles. A human driver driving on a familiar road can use their memory to help position the vehicle appropriately for upcoming manoeuvres or to manage their speed more effectively based on their stored knowledge of upcoming road configurations or hazards. They also must use their memory of the rules of the road and the expected behaviours to navigate safely and in a predictable manner. When driving in a new environment, a human driver has the flexibility to use their skills, senses, memory and judgement to manage the situation, driving in a more cautious manner to compensate for their unfamiliarity with the road and its features. Specific memories can fade but the skill of driving is very durable. Once learnt it is not easily forgotten.
Memory has different characteristics in the context of computers. Large quantities of stored data can usually be accessed very accurately and rapidly. Some developers use stored high-definition maps to enable the vehicle to understand where it is, to plan its route from its origin to its destination and to detect any differences when the map is cross-referenced against the information received by its sensors. In effect, the vehicle has a perfect memory of the configuration of the world around it and can use that to help with positioning and navigation. However, this means that the vehicle is limited to regions that have been mapped to the required quality. Other developers are taking a different approach and use systems that are effectively capable of applying the principles of driving in a wide variety of situations. This means they are not limited to specific environments but may necessarily have to drive more cautiously to make up for their more limited foreknowledge of the road configuration.
Another notable difference is our appreciation that contravening the rules can result in punishment and that poor driving behaviours may result in harm to ourselves, our passengers or to others. Computers can be made to operate in full compliance with the rules of the road and this should lead to reduced risk. However, they cannot be punished in the same way as a human driver and do not have an innate sense of self-preservation or desire to avoid harming others. We trust human drivers will do all that they can to prevent collisions that could result in casualties (and this might very occasionally mean breaking the rules of the road). Whilst we can trust the developers of self-driving vehicles to have done the same, programming errors or some unusual combination of stimuli might result in a self-driving vehicle causing harm in what would be considered a very safe situation for human drivers.
However, the final piece of the puzzle is perhaps the one that is the greatest challenge for the developers of this technology.
Humans are social creatures with an innate ability developed over millions of years to observe, understand and predict the behaviour of others and to take reasonable actions in response – even in situations that are incredibly rare or bizarre. Tiny details, such as a pedestrian taking a quick glance over their shoulder, that would be barely perceptible to a self-driving vehicle’s sensors would be immediately recognised by a human observer and used to guide future actions – in this example, appreciating that the pedestrian may want to cross the road and so preparing to take avoiding action if needed. With improving sensors and exposure to more and more human interactions, computers are beginning to develop these skills but they are far behind in the sensitivity and subtlety of situations that we humans find trivially easy to comprehend.
Recent events in San Francisco highlight such challenges for this technology. Self-driving vehicles deployed in the city have been observed stopping in intersections, driving into roadworks, obstructing emergency vehicles and ignoring guidance from police officers. The resulting publicity surrounding such incidents has provoked activists into placing traffic cones on the bonnet of self-driving vehicles. This disables the vehicles and, in their eyes, highlights them as the hazard they perceive self-driving vehicles to be. The city transport authority has also urged greater caution and more transparency from self-driving vehicle operators to build confidence in the safety of their services.
Make no mistake, it is incredibly impressive to see self-driving vehicles operating in public on busy city streets. This is a phenomenal software and hardware engineering accomplishment. However, the difficulties observed with these early deployments highlight how far we still need to go for self-driving vehicles to reach the capabilities we human drivers display – and it is our innate human ability to understand the world around us that is the most challenging for them to match.
By Dr Nick Reed, Founder – Reed Mobility
