Functional Safety of Low Opacity Systems

Classical functional safety analysis is based on the concept that the behaviours of a system may be adequately determined understood and controlled under all operational conditions.? This concept is referred to as determinism.? To date only deterministic systems have been seen as fit for use in safety systems.? However, the opacity of systems is rapidly decreasing with the rise in complexity and volume of processing being undertaken and the sheer amount of data which a system may have to now process to reach an end goal.? This trend ultimately leads to Artificial Intelligence (AI) and Machine learning (ML) algorithms where the code to a lesser or greater extent may change to suit the task being undertaken.

This opacity is a problem to the Functional safety assessor trying to determine if a system is safe based on a fixed input and output model and the relationship between the two is not always well understood.

This is not to say though that there are steps which cannot be taken to assess and determine safety in these systems.

Operational Domain

Where a general solution cannot be determined the fist stage is to understand the operational domain and limit the use case strictly to this domain.? This reduced the number of use cases and variable to be used as inputs to the hazard analysis and therefore limits the number of hazardous responses the system may give.? This restriction is similar to that provided in IEC 61508 route 2 and this is no coincidence as the basis of route 2 analysis is not to determine the failure mechanism, rather the rate in service and ultimately the safety of a system.? With high opacity systems likewise, it may not be possible to determine the mechanism, therefore the domain must be bounded in the assessment.

Boundary Behaviour

With the domain bounded the behaviour must now be bounded.? This is perhaps the biggest single change in methodology in that when the hazard analysis is done the requirements generated are not functions, rather limiting system behaviours or boundaries.? This is essential as the functional behaviour within the boundary cannot be well defined and may change with time.

With the limiting behaviours defined there needs to be mechanisms in pace to ensure that the system does not violate these behaviours.? These functions may be deterministic and quite simple in nature relying on simple dynamic checks to determine that key parameters or metrics are met.? A key factor in this is the independence of data.? The behavioural check should rely on independent data to that generated by the system.? This is to ensure that the checks are truly measuring the system behaviour and not the expected behaviour.

System Stability

Where a control system is used it must be stable within the operational domain limits.? This rule applies and has a second dimension when elements of machine learning are applied.? In this case the learning must trend to a stable point and the system must retain stability at all points in the learning process.?

This ensures that the boundary measures are only used in extremis and are not relied on as system checks.? The impact of this is that the system may learn to rely on the boundary checks and not reach a stable point inside the boundary.? To this end the boundary functions should have a duty of a low demand function.?