IEEE P2817

New IEEE Standard - Active - Draft.
Autonomous systems operate in complex, dynamic environments and make runtime decisions that control their behavior, often without human intervention. Ensuring the safety, reliability, and performance of such systems requires rigorous verification processes that account for these complexities. This Guide for Verification of Autonomous Systems enables the user to define a customized process for verification of their autonomous system at the component and system levels based on their needs and available resources. It documents challenges and best practices for specification development, verification across all levels of abstraction, integration with fault management and runtime assurance components, and provision of evidence for licensure, certification, and assurance.

This Guide for Verification of Autonomous Systems enables the user to define a customized process for verification of their autonomous system based on their available resources. It documents best practices across all levels of abstraction within a given system from the lowest level components and software to the highest level learning or decision-making elements (specifically including verification of the inputs to any learning algorithms, such as training data). Integration guidance for developing a verification process using various techniques and methodologies, with corresponding different tool types, is provided. Verification process templates are provided as examples based on the availability of verification techniques and/or methodologies. The criteria for selecting a particular existing template are outlined. Different types of verification techniques, methodologies, and processes are enumerated and described, and a list of different tool types that should be part of any autonomous system verification toolbox is provided. The tool types captured in this Guide encompass techniques and/or methodologies that can be used for verification processes (such as those that quantify the completeness of coverage-guided test suites) and include both theoretical and software tools.
The purpose of this Guide is to identify existing challenges and best practices and provide guidance that supports the definition of valid verification processes for a range of autonomous system configurations. The guidance is intended to include both mobile and immobile robots, singly and in groups, focusing primarily on systems that can operate autonomously rather than on automated or supervised robots. Guidance is not primarily aimed at but may also be applicable to autonomous systems that do not physically act on the external world (e.g. autonomous agents operating in the context of financial tools, social networks, or city surveillance networks).