Artificial Intelligence and Machine Learning (AI/ML) rapidly transform systems by providing autonomous capabilities. This new class of systems can become a constituent system in a System of Systems (SoS) to evolve it into a System of Autonomous Systems (SoAS). SoAS is fraught with new Systems Engineering (SE) challenges for architecture development, integration, testing, and evaluation that originate from the Level of Autonomy (LoA). The LoA refers to the level of autonomous capabilities of a system depending on its AI/ML technology. Architecture and integration challenges include interface compatibility, safety, and security considerations. Test and evaluation challenges are concerned with identifying the suitable LoA in constituent systems while considering uncertainty and undesired emergent behaviors. We proposed a comprehensive SE methodology to address these LoA-imposed challenges. This methodology is an implementation of the core architecture processes in ISO/IEC/IEEE 42020 (i.e., Architecture Conceptualization, Elaboration, and Evaluation) and is organized into two main parts.
The first part focuses on architecture conceptualization and elaboration processes by applying systems engineering concepts and principles for developing SoAS architectures. This falls under the System Engineering for AI (SE4AI) umbrella, where SE principles are tailored to accommodate challenges posed by the integration of autonomy. In this part, we proposed a Model-Based Systems Engineering (MBSE) method that builds upon the Object-Oriented Systems Engineering Method (OOSEM) and modifies it to facilitate autonomy integration by leveraging the SWOT analysis (i.e., Strength, Weakness, Opportunity, Threat) and SoAS taxonomy. Moreover, the proposed method tailors the Unified Architecture Framework (UAF) viewpoints and views to provide necessary information for different groups of stakeholders at each step of the SoAS architecture development according to the modified OOSEM. Implementing the proposed method results in SoAS executable models that will be simulated to generate the required evaluation data for the next part of our proposed SE methodology. The proposed method provides a step-by-step guide to perform the autonomy integration, including how to identify LoA impacts, how to model varying LoAs in the SoAS architecture, and what UAF views to build. The model provided here (i.e., SoAS_Archtecture.mdzip) demonstrates the implementation of the proposed architecture development method on a hypothetical search-and-rescue example. This model was developed in Magic Systems of Systems Architect 2022x.
Then, the second part focuses on the architecture evaluation process of the SoAS alternatives by proposing a test and evaluation method for SoAS. The proposed method is a step towards AI for Systems Engineering (AI4SE). It applies the emerging AI/ML techniques and the Bayesian Network (BN) approach to identify the most suitable SoAS alternative while taking into account uncertainty and undesired emergent behaviors. This method provides stakeholders with a decision analysis tool that enables them to effectively explore the SoAS design space while considering uncertainty, compare different SoAS architectures, select the architecture that better addresses the mission needs, and provide explainability of SoAS-level emergent behaviors. The model provided here (i.e., SoAS_Analysis.R) demonstrates the steps to achieve the data (i.e., BN_Data.csv) required for the SoAS's BN. This model was developed in RStudio software tool. Then, the data are used to create the BN model (i.e., BN_Model.xdsl), provided in the Releases section of this repository. This model was developed in GeNIe Academic software tool. The final BN provides two types of analyses: predictive analysis to select the most suitable LoAs in constituent systems that yield the desired SoAS performance; and prescriptive analysis to identify the root causes of an observed SoAS undesired emergent behavior and define preventive strategies for future operations.
Together with SE4AI and AI4SE parts, the complete SE methodology provides a step-by-step guide to develop different feasible SoAS architectures with varying LoAs, evaluate each architecture, and finally, select the most suitable one for a mission.
We acknowledge Dassault Systemes as a contributor to this research.