Detect Dynamic Errors and Conflicts in MBSE Models

By Yves Génevaux & Ken Nathan

Abstract

Model-Based Systems Engineering models can contain errors which should be detected by MBSE tools and proper MBSE formalism. While most of the static mistakes can be detected easily by the majority of the available tools on the market today, an entire category of errors is frequently missed. These are the dynamic errors. Errors in the actual behaviour of the model, errors that require simulation to be detected. There are two key reasons for this. Firstly, most MBSE models, although they can be animated, cannot be truly executed and secondly, a lot of the information pertaining to the behaviour remains external to the model, in textual requirements. Here we present how the MBSE tool, STIMULUS, helps to detect the problems in the dynamics of the model by using Requirements-In-the-Loop simulation. We will focus on the automatic detection of contradictions in the definitions of the expected behaviour, often called “conflicting requirements”.

 Introduction

 An MBSE environment is composed of a number of elements. The formalism you use, the tools supporting it, the skills and experience of the users, a proper engineering process definition, etc… All of them matter, however, we will concentrate on the first two, the formalism used to model the system, and the capabilities of the tools supporting the modelling activities and finding errors in the model. For instance, if, in one diagram, a signal is sent to an object of the system, and, in the view defining the object interface, the signal is not present, there is obviously a mistake.

 Most of the MBSE formalisms provide decent capabilities in static error detection. By static, we mean that it is not necessary to perform a simulation of the model to observe a problem. A very different challenge is the detection of dynamic mistakes, when the model does not behave as expected, when it does not provide the expected services. And there are many of this type of error in the specification: 70% of the mistakes made in a project are made at specification time:

 What conditions must be met in order to detect dynamic mistakes?

1-    The model must include all the behavioural information, the functional information with regards to the object of interest (a function, a sub-system, or the entire model). This means that all the functional requirements must be part of the model.

2-    The formalism used must support simulation.

3-    The tool must support a powerful simulation meaning two things:

• You can simulate locally (a given object) or globally (the interactions between several objects)
• A simple “animation” of the model, managed by the user who interactively runs one given scenario, is not good enough. Many errors only happen “sometimes”, so the tool must be able to generate numerous combinations of inputs automatically so as to explore the envelope of the possible behaviours effectively.

 STIMULUS and its Requirements-In-the-Loop technology fulfils all of these conditions. It can be summarised through the following three key points:

 1.     The STIMULUS model allows the description of the system architecture and the interfaces between the architecture’s components with a hierarchy of Block Diagrams, and the expected behaviour of the architecture’s components is described using textual requirements, hierarchical state-machines or a free combination of both. All the functional information, the dynamics, is included in the STIMULUS model.

 2.     State-machines and textual requirements have executable semantics; therefore, simulation is possible. System architects can now debug a system specification in the same way a programmer would debug a piece of code.

 3.     Use cases can be described using constraints on the inputs and between the inputs and STIMULUS can generate numerous combinations of inputs automatically to produce a large number of model executions for one given use case. 

 Let’s see in practice how you find errors in a STIMULUS model. For the simplicity of the paper we’ll use a rather small model, composed of one object only, but you can apply the same approach on much larger models made of numerous objects interacting together. Please see the video: https://argosim.com/product-videos/#architecture

 We will consider a headlight controller with an automatic mode:

• When it’s dark, the headlights are set to ON,
• When it’s light, the headlights are set to OFF.

Like many systems, it is necessary to define a hysteresis to avoid behavioural issues if there is only one limit between “ON” and “OFF”. In that instance, we will define that the headlights will be:

• ON if the ambient light intensity is under 60% for more than one second,
• OFF if the ambient light intensity is above 70% for more than one second.

Below is the functional specification of a headlight controller that is typically what we get supplied with from embedded systems development organisations. Once written in STIMULUS, it looks like this:

• For the ON and OFF modes:

• For the AUTO mode:

The 3rd and 4th requirement of the AUTO mode define the behaviour in the hysteresis zone whereas the 5th requirement defines the initialisation when the driver sets the switch to AUTO.

 Without simulation, the classic verification mode is based on manual reviews. As the requirements work in parallel, this technique very quickly becomes ineffective. Understanding how 2-3 requirements work in parallel is not easy, when there are dozens, hundreds… it becomes impossible. How can you assess there is no problem as the number of requirements increases? We at Argosim faced numerous examples where beyond 3 to 5 functional requirements with timing properties, errors exist that had not been detected by a review.

Use Cases description

Let’s define a Use Case to generate interesting test vectors for that model. There are two inputs, the switch and the light intensity.

For the light intensity, let’s define a Use Case where:

• It is a percentage,
• It starts at 90% then goes up and down between 75% and 55%. It will cross the limits of the hysteresis which should lead to interesting test cases.
• We do not want a deterministic generation of inputs so we simply bind the speed at which light intensity evolves (its derivative).

Here is the corresponding STIMULUS Use Case:

Note that Use Cases can be defined using a free combination of state machines and textual constraints.

 Simulation 1: AUTO Mode

 Let’s start with a first simulation mode, keeping the switch in the AUTO position. We ask STIMULUS to generate one execution. Here is the simulation view:

We can observe the reaction of the system according to the inputs generated by STIMULUS as well as according to the requirements. Many similar executions can be displayed by STIMULUS as the test vector (the lightIntensity values) at each cycle is bound by a set of constraints but is not deterministic. This allows us to explore the envelope of the possible behaviours of the specification automatically.

Simulation 2: The switch can change from time to time

 Let’s make a second simulation. This time we allow STIMULUS to change the switch from time to time, for instance every three seconds. We define this other Use Case with this constraint:

This time, STIMULUS has more freedom to explore the possible behaviours of the specification. Let’s ask to STIMULUS to display an execution of the requirements:

The system architect in charge of the specification can look at this simulation and assess that the behaviour of the specification is correct for this execution. As many different combinations of inputs are possible, it is important ask STIMULUS to generate other executions and to check if they are correct as well. STIMULUS can do this with the simple click of a button. Another execution is run and we get this new simulation:

STIMULUS has interrupted the simulation at Time = 9 seconds and a red vertical bar indicates that a contradiction, a conflict in the specification, has occurred at that point. An error message is displayed by STIMULUS, and clicking on the message leads to the identification of the conflicting requirements: 

The capability of detecting contradictions (conflicts) in specification models automatically, whatever the number of pieces of conflicting information is very valuable. Without STIMULUS these errors, often nigh impossible to find just through reviews, tend to be uncovered late in the development process, which leads to costly rework and project delays.

 A quick correction in the set of requirements can be made such that when we move the switch to AUTO, at the first cycle, the system should do the “Initially” requirement(s), and at the next cycles (if the switch is still AUTO), the “afterwards” requirements will apply.

This should prevent the conflict to happen, let’s verify if the correction works. The first execution displayed by STIMULUS is this one:

Note that STIMULUS has generated exactly the same inputs as before, and the specification behaviour is, in that instance, the same and it is correct.

 Let’s perform a second simulation. This time, instead of observing a conflict at 9 seconds, we get:

The change made in the requirements seems to have resolved the conflict. Running more simulations will provide a much higher level of confidence that the conflict has disappeared. Note that this works with larger models, mixing state-machines and textual requirements as well.

 A video presenting the detection of contradictions in the specification can be seen here:

https://argosim.com/product-videos/#conflicting

 Conclusion

 STIMULUS from Argosim provides a very powerful and easy-to-use solution to detect the contradictions and errors in specification models automatically, which include all the necessary functional information: architecture, interfaces between the architecture’s components and the behaviour of all components, described with a free combination of state machines and textual requirements.

We very much encourage you to ask us any questions you like on the applicability of STIMULUS for your projects. STIMULUS is available via a free download at: https://argosim.com/free-trial/