Turns out that safety experts, who are engineers, actually agree on a safety rubric and on the notion that one does not have to sacrifice safety to achieve innovation. One can require both safety processes and be technology neutral. I have also found, at not quite halfway through the comments submitted in response to the proposed Framework for Automated Driving System Safety, that safety engineering comments should be read as a primer on autonomous vehicle (AV) safety. In case anyone's memory is short, the draft framework was issued just before the previous Administration left office.
 |
| Painting of anxious face. |
The second comment to be examined was submitted by Miles Thompson, at MITRE Corporation, on behalf of MITRE and written by Thompson and other staff. MITRE is a not-for-profit corporation that operates federally funded research and development centers across several sectors and does consulting. MITRE's work extends to cybersecurity, aviation, health care, and defense and intelligence.
Both Koopman and MITRE are in the business of AV safety. That is not to challenge their comments, but to be up front about their safety expertise being a part of their AV professional activities. This would be the same as a career public school teacher commenting on education or a prosecutor discussing the criminal justice system.
My reading of the safety expert comments is that the whole either/or challenge of safety versus innovation is BS - okay, these engineers don't say that directly because they are polite and they serve in roles where such language would be detrimental to their careers and to their employers' reputations. These educated briefs push against the either/or myth of safety versus innovation and they reject the idea that safety must be sacrificed at the altar of innovation.
The average person is wrong
The average person is someone like me, someone with no engineering education, familiarity with safety protocols outside of the kitchen and basic driving, or training beyond a CPR class well over 10 years ago that I barely remember. I once caused a major flood in my house, when I was an adult homeowner, by fiddling with a toilet that was having some hiccups. A law degree trains you to ask questions and to read well; if you are lucky it begins your path to writing well. Nothing on safety or fixing toilets.
I officially admit that my idea for AV safety was completely wrong. I kept thinking that some sort of driving test, perhaps done remotely, and on a schedule (akin to current car inspections) would ensure that only safe AVs would be operating on our roads. Nope. What the two experts who are featured in this post provide is a not so much primer on AV safety issues - for dummies, like me - but a well-written argument to shift our perspective from particular safety tests administered by or at the direction of NHTSA to enabling, encouraging, and maintaining a safety culture at the companies that are developing AVs.
Not safety defined, but safety integrated at each step of the way
Koopman expressly declares that a test is not a panacea, rather:
NHTSA should not spend massive resources attempting to define a comprehensive ADS “driver test.” While a minimalistic road competence test could potentially keep unsophisticated design teams that are not capable of fielding safe vehicles off the roads, a very extensive “driver test” would consume huge NHTSA resources and would be unlikely to provide strong evidence of operational safety at even the level of average human driver ability. At best such tests would only be likely to identify ADS designs so bad that they can’t pass a predefined test. (That goal is not a bad one, but could and should be achieved in an economical manner.) Such a test might, however, provide protective cover for an organization that is motivated to cut corners on safety by building to the test (even a randomized test can be expected to be gamed if it is the only safety measure required) instead of actually building a vehicle that will be safe in the real world.
... ... ...
Rather than attempting to assess functionality beyond basic tests in the vein of current FMVSS and NCAP approaches, NHTSA should instead emphasize assessing whether an organization has created a viable safety case, has performed self-determined tests responsive to that safety case adequately, and whether organizations are indeed paying attention to and taking action upon emergent safety issues of their own accord. [Editor's note: FMVSS are US Federal Motor Safety Standards set forth by NHTSA, the National Highway Transportation Safety Administration; and NCAP is the European New Car Assessment Program.]
Safety integrated and transparent
What Koopman advises is a way to integrating safety, rather a set of specifications or specific equipment. In contrast, MITRE argues for performance standards: "NHTSA should adopt a system engineering approach driven by testable performance-based standards derived from implementation-independent safety requirements instead of prescriptive regulation." Both agree, however, that UL4600 or an equivalent safety protocol is necessary.
One of Koopman's main concerns is transparency, not allowing the companies that produce AVs to avoid demonstrating safety.
NHTSA could simply ask to check whether the data supporting the manufacturer claims can be replicated on their own terms. Such an approach should be coupled by field engineering feedback so that a company that performs overly simplistic tests will be confronted with realworld data of any inadequacies of their product before significant losses in real world operations are likely to have occurred. While there are obvious limitations to such an approach, it is clearly better than allowing manufacturers to use completely opaque processes to design and deploy systems without any checks and balances on computer-based system and software safety at the time of deployment.
Airline model and new suggestions
Both Koopman and MITRE recommend something like what the airline industry has arrived at, a cooperative approach. MITRE points to the required data collection and Koopman praises the cooperation among all stakeholders. Koopman sees NHTSA's role as a facilitator whereas MITRE wants NHTSA to create a framework. These aren't opposites. I hope that I am understanding what both of these comments say and I fully admit that I am not well educated about safety engineering.
 |
| T-shirt with cicada image and message to get out of the way because they have waited 17 years. |
Traditional vehicles that meet the FMVSS are released to the public and considered safe until enough evidence is gathered to say otherwise, which usually ends in the developer recalling the unsafe products to avoid regulatory punishment. With continuous validation, the certificate of safe operation can be provided on a time-limited period, based on the quality and frequency of the metric data. If the metric data stops, that may be enough justification for NHTSA to say it can no longer determine the safety of the ADS and it should be removed from public roads. [Emphasis added.]
MITRE also suggests a confidential employee reporting system so that errors are reported instead of hidden until disaster strikes.
NHTSA needs the capacity to assess and regulate
Koopman calls on the companies to tell their safety stories and not having NHTSA dictate from the outset. But he also calls NHTSA - and by extension Congress - on the carpet for not developing and growing computer engineering expertise at the agency.
NHTSA has historically under-staffed in the area of computer-based system safety, and especially software safety. However, in recent years automobiles have transformed from electromechanical systems to computers-on-wheels. Especially in electric vehicles, there is simply no way to understand whether a vehicle is acceptably safe without understanding computer technology.
 |
| Painting - What Is Your Must? |
