Publications

Applied ontologies have been used more and more frequently to enhance systems engineering. In this paper, we argue that adopting principles of ontological realism can increase the benefits that ontologies have already been shown to provide to the systems engineering process. Moreover, adopting Basic Formal Ontology (BFO), an ISO standard for top-level ontologies from which more domain specific ontologies are constructed, can lead to benefits in four distinct areas of systems engineering: (1) interoperability, (2) standardization, (3) testing, and (4) data exploitation. Reaping these benefits in a model-based systems engineering (MBSE) context requires utilizing an ontology's vocabulary when modeling systems and entities within those systems. If the chosen ontology abides by the principles of ontological realism, a semantic standard capable of uniting distinct domains, using BFO as a hub, can be leveraged to promote greater interoperability among systems. As interoperability and standardization increase, so does the ability to collect data during the testing and implementation of systems. These data can then be reasoned over by computational reasoners using the logical axioms within the ontology. This, in turn, generates new data that would have been impossible or too inefficient to generate without the aid of computational reasoners.

The US Army envisions heterogeneous teams of advanced machines and humans that will collaborate together to achieve a common mission goal. It is essential for commanders to quickly and effectively respond to dynamic mission environments with agile re-tasking and computerized aids for plan definition/redefinition, and to perform some tasks with bounded autonomy. Workload constraints limit an individual's ability to concurrently control many platforms, so some mission segments many need to be autonomous or to be quickly selected via a tactics playbook. Denied environments also dictate the need for machine participants in some mission segments to be autonomous (or semi-autonomous). A Multi-Agent System (MAS) provides a natural paradigm for describing a system of agents that work together in such environments. An agent can be a human or machine, but is generally a machine. Creating MAS systems and requirements has proved to be a formidable task due to mission complexities, the necessity to deal with unforeseen circumstances, and the general difficulty of defining autonomous behaviors. We define a process called Multi-Agent Systems Collaborative Teaming (MASCOT) Definition Process that starts with a Subject Matter Experts (SME), produces a set of agent specifications, and derives system requirements in sufficient detail to define a MAS that can be modeled in a test-bed, used for facilitation of a safety analysis, and produced into an actual system. The MASCOT process also enables concurrent development of an effects based ontology. We demonstrate the MASCOT process on an example case study to show the efficacy of our process.

Cyber-enabled and cyber-physical systems connect and engage virtually every mission-critical military capability today. And as more warfighting technologies become integrated and connected, both the risks and opportunities from a cyberwarfare continue to grow--motivating sweeping requirements and investments in cybersecurity assessment capabilities to evaluate technology vulnerabilities, operational impacts, and operator effectiveness. Operational testing of cyber capabilities, often in conjunction with major military exercises, provides valuable connections to and feedback from the operational warfighter community. These connections can help validate capability impact on the mission and, when necessary, provide course-correcting feedback to the technology development process and its stakeholders. However, these tests are often constrained in scope, duration, and resources and require a thorough and holistic approach, especially with respect to cyber technology assessments, where additional safety and security constraints are often levied. This report presents a summary of the state of the art in cyber assessment technologies and methodologies and prescribes an approach to the employment of cyber range operational exercises (OPEXs). Numerous recommendations on general cyber assessment methodologies and cyber range design are included, the most significant of which are summarized below. -Perform bottom-up and top-down assessment formulation methodologies to robustly link mission and assessment objectives to metrics, success criteria, and system observables. -Include threat-based assessment formulation methodologies that define risk and security metrics within the context of mission-relevant adversarial threats and mission-critical system assets. -Follow a set of cyber range design mantras to guide and grade the design of cyber range components. -Call for future work in live-to-virtual exercise integration and cross-domain modeling and simulation technologies. - Call for continued integration of developmental and operational cyber assessment events, development of reusable cyber assessment test tools and processes, and integration of a threat-based assessment approach across the cyber technology acquisition cycle. Finally, this recommendations report was driven by observations made by the MIT Lincoln Laboratory (MIT LL) Cyber Measurement Campaign (CMC) team during an operational demonstration event for the DoD Enterprise Cyber Range Environment (DECRE) Command and Control Information Systems (C2IS). This report also incorporates a prior CMC report based on Pacific Command (PACOM) exercise observations, as well as MIT LL's expertise in cyber range development and cyber systems assessment.