Rand's AV Study: Not Feeling Its Calculation of Unknown Unkowns

Agreed - Perfect is not all it's cracked up to be

I love how polling and models and other predictive tools are put together and often manage to get results so wrong. There are just too many variables, including the idiosyncrasies and preferences of many different humans, and the personalities of thought leaders - whether elected, in business, or in the media - that are popular or ignored at any given point in time.

That is my skeptical frame of mind when reading the Rand study, entitled The Enemy of Good: Estimating the Cost of Waiting for Nearly Perfect Automated Vehicles, which is nicely summarized in this video on the report's summary page.


The study and the video remind me of the classic Donald Rumsfeld quote, uttered sometime around the start of the Iraq War.

Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.

Huge disclaimer

I have not now, nor have I ever been, an expert in statistics or modeling. The language used to summarize conclusions based on scenario modeling is such that I end up re-reading it several times without the benefit of increased comprehension. Therefore, the following could be completely wrong. 

AVs present a whole bunch of unknowns

Giving the author's credit, they do articular a whole bunch of known unknowns, including:

• The safety/crash equation between human driver risks and new, as yet unknown, highly autonomous vehicle (HAV) risks, such as cyberattacks.
• Percent of HAV market penetration when we begin to see safety improvements in terms of lives saved and injuries avoided.

My big BUT

The study makes some possibly dubious assumptions:

• Someone out there who is credible AND will be heeded will be able to calculate and recognize at what point in time it is when we have reached the break-even safety point for HAVs, that is when they become exactly as safe and then surpass the safety of conventional, human-operated vehicles. 
• We are several decades from the full diffusion of HAVs. What? Talk about an unexplored assumption.
• The pace of HAV learning and operational capability will NOT be fast or accelerate in the future. Same What? as above.

And the winner is ...

In a nutshell, the Rand study authors' conclusion is stated very succinctly:

From a utilitarian standpoint, it seems sensible that HAVs should be allowed on U.S. roads once they are judged safer than the average human driver so that the number of lives lost to road fatalities can begin to be reduced as soon as possible. Yet, under such a policy, HAVs would still cause many crashes, injuries, and fatalities—albeit fewer than their human counterparts. This may not be acceptable to society, and some argue that the technology should be significantly safer or even nearly perfect before HAVs are allowed on the road. Yet waiting for HAVs that are many times safer than human drivers misses opportunities to save lives. It is the very definition of allowing perfect to be the enemy of good. [Emphasis added.]

The conclusion makes perfect sense, as do the numbers calculated to support the opinion, but the assumptions are so massive that the numbers could be wildly off. Of course, only time will tell.

My guesses 

I guess - predict - that HAVs will be (a) safer and (b) be adopted quickly.

(a) Increased safety: HAVs will mean fewer and then zero jerks who will drive when sleepy, in a rush, distracted by children, radio, podcasts, or pretty trees, or in any way under the influence of drugs, alcohol, or indigestion or pain. Jerks who over-estimate their driving skills will not be speeding or weaving in and out of traffic or erring in their predictions of being able to make a lane change or a turn before another vehicle comes barreling along. Even if HAVs are only as safe as human drivers in terms of their optimal driving skill, they will all learn from each other and they will mostly be driving at that optimal point, instead of doing so only at noon when one is alone in the car.

(b) Fast adoption: For the most part, in terms of early HAV adoption, we are talking about humans in Western and modern Asian countries. These are people who went underground before any large-scale testing to embrace subways when they were first built; who readily relinquished control and got on planes even though crashes that result in many lost lives happen occasionally; who quickly bought expensive smartphones even though this meant giving up privacy and being, at times, unreachable; and who, despite the safety risks, while driving, use those smartphones, unwrap and eat food, scramble with other devices, and deal with other people - sometimes demanding tiny people. Plus, there is no one who would prefer to pay attention to rush hour traffic than nap, play games, read, text friends, read entertainment news, or even work. Maybe there's an exception for a few people who get to drive each day on pretty two-lane roads with little traffic.

Those are guesses resulting from zero modeling, scenario planning, or any calculations. Many assumptions have been made and they are mine.