Essential properties for safe behaviour of a perception function in automated driving

Gauerhof, Lydia (2023): Essential properties for safe behaviour of a perception function in automated driving. Dissertation, LMU München: Fakultät für Mathematik, Informatik und Statistik

Abstract

The recent success of Machine Learning (ML) has led to the widespread application of ML in various domains, such as household products, natural language processing, and recommendation programs. Technological and computational advances and the increasing availability of data have fueled this trend. In this context, machine learning is expected to enhance the safety of autonomous safety-critical systems due to its compelling performance. In a safety case, it shall be shown that a function is sufficiently safe, i.e., an acceptable residual risk is not exceeded. To this end, it should be demonstrated that certain properties that ensure safety are satisfied and that causes of failures are adequately mitigated. For this reason, the goal of the work is to extract and confirm properties of the ML-based function and the data that are essential to the safety argument. Thus, the work addresses the research question, what properties are necessary for a safe behaviour of an ML-based perception function, and how can we acquire them. Thereby, the use of ML in safety-critical systems, such as perception functions in automated driving, comes with the challenge of providing a convincing safety argument. For example, ML-based perception functions process sensor data and extract information about objects and drivable areas. When perception fails, major damage and fatalities can result. Therefore, in complex environments that evolve over time, the risk of ML functions failing must be reduced to an acceptable level. Established standards in the automotive industry such as functional safety ISO 26262 and safety of intended functionality ISO 21448 do not explicitly state how to ensure the safety of ML-based functions. In addition, established approaches recommended in the standards cannot be directly applied to ML components. For this reason, we elaborate the challenges in validating perception functions. On the one hand, the input space of an automated system and the intended functionality is very complex. On the other hand, formulating the specification of a perception function is a challenge in itself. Moreover, it is difficult to ensure the required properties hold over the entire input space and to verify and validate the implemented function with respect to the intended functionality. Our focus is on the specification of the essential properties of a perception function for automated safety-critical systems and its realization, starting with design up to verification and validation. Only the two together, specification and realization, make it possible to develop a comprehensive safety argument. Specifically, the use case of a safety-critical pedestrian detection function for automated driving is investigated. For this purpose, a Deep Neural Network (DNN) for pedestrian detection is trained and its properties are investigated. In addition, novel methods are developed to satisfy its essential properties. First, a set of safety requirements is derived and examined for their impact on the activities of the ML lifecycle. In addition, functional insufficiencies are investigated as they might lead to hazards~\cite{ISSREW2020}. To this end, relevant data characteristics are extracted. Error categories are identified and remedial actions are proposed, focusing on the suitability of the training data. In addition to the approach to improve the training data, other measures are taken. When input data strongly differs from that used in training and test, its impact on the performance should also be analysed. In this case, we propose to complement data suitability with online anomaly detection that monitors the behaviour of the DNN. To this end, we present two recent publications on anomaly detection. While FACER is trained to detect different types of noise that can distort the data, ReverseVAE is able to detect anomalies outside the distribution of training data. Both of these anomalies can have a large impact on the safe behaviour of an ML-based function. Another capability of ReverseVAE is the ability to manipulate the data with certain visual attributes. Thus, data can be generated, with defined visual attributes, which could later be used for training or testing of an ML-based function. Since testing of an ML function cannot be guided only by the specification alone, we present different approaches to specifying a test oracle and testing approaches from different domains and application areas and novel test setups developed. In order to address the challenges outlined above, an iterative and continuous specification of requirements in interaction with the development is proposed. To compensate for the traceability that is missing between the requirements and the lines of code in an ML-based function, we propose explicit artefact links and illustrate this with examples. All in all, this work provides a holistic view on the research question of what properties are required for a safe behaviour of a ML-based perceptual function and how we can acquire them. This is intended to bridge the gap between already established safety practices applied to non ML-based systems and scientific knowedge in ML development.

• Dokumententyp: Dissertationen (Dissertation, LMU München)
• Keywords: Machine Learning, Automated driving, perception, safety
• Themengebiete: 000 Allgemeines, Informatik, Informationswissenschaft ; 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik
• Fakultäten: Fakultät für Mathematik, Informatik und Statistik
• Sprache der Hochschulschrift: Englisch
• Datum der mündlichen Prüfung: 19. September 2023
• 1. Bericht­erstatter:in: Beyer, Dirk