Co-Simulation Performance Analysis

Every Saturday, Thomas G. z and Michael Bambula are excited to bring the developers on stage and provide you an extended read into various simulation topics.

by?Klaus Schuch

In general, the simulation of a model should be as fast as possible and as accurate as needed. This is not only the case for pure simulations where we want to get the results as fast as possible to for example perform extensive parameter studies. Fast simulations are even more important for simulations that interact with the real world. To perform a useful co-simulation of a simulation model with a real time system (e.g., an engine on a testbed), the simulation speed of the simulated model must be (at least on average) faster than real time - in other words - the average real time factor (over time) of the simulation model must be less than 1. The simulation will be slowed down to real time by the synchronization with the real time system. The simulation itself is not performed on a real time system. Hence we have to expect that the real-time factors jitter around the mean value. One reason for the jitter is typically that process/thread scheduler of the operating system might schedule other tasks (threads, processes) in between. Of course also the real time factors of the individual models might vary based on operating points.

As a tool to analyze the performance of a co-simulation model (and to find hints on how to change the model to speed it up), we developed the Performance Analysis app. It can be started for any Model.CONNECT? model and will analyze the performance of the active case.

Consider a simple co-simulation model consisting of 3 FMUs (A, B, and C). To keep it simple, the individual real time factor of each FMU is ~1.0, and the used time step size for each FMU is 1 ms.

The result of such an app run is an html report in the active case directory. (The report will be opened automatically after the app run.)

The first part of the report gives a short overview with the main results:

The co-simulation model is here configured to run sequentially (coupling mechanism Sequential) with the per step sequence: first A then B and at last C. With almost constant individual real time factors, the overall real time factor is (approximately) the sum of the individual real time factors plus some overhead (for data exchange, result writing, online-monitoring, etc.).

The scheduling plot of the performance report shows when the DoStep function calls of the FMUs take place and how long they last.

Switching to the coupling mechanism Parallel improves the real time factor of the co-simulation.

The corresponding scheduling plot shows that the 3 threads (1 per FMU) calculate nicely in parallel:

Even though the 3 FMUs are calculated in parallel, the overall real time factor is bigger than 1. There is still some overhead. At least some parts of the overhead can be attributed to scheduler of the operating system, i.e., the individual threads do not always get immediately scheduled by the scheduler when Model.CONNECT requests it. The white area around the sync time points in the "zoomed in" scheduling plot indicate this overhead. This sync overhead is the reason why we want to use time steps which are as long as possible.

What happens if the time step sizes of the individual models are not equal?

Let's increase the time step size of FMU C to 10 ms but keep its real time factor (~1.0). The time step sizes of individual models typically correspond to the time constants of its dynamics, e.g., short step sizes for electrical models and longer steps sizes for thermal models.

The real time factor of this co-simulation model with sequential scheduling is roughly the same as for the equal time step case. It gets slightly better (3.07). The tiny drop can be partly attributed to decreased interaction with FMU C. Note that this also means that C will calculate with outdated values coming from A and B for some time intervals of its calculation.

Switching to parallel scheduling (coupling mechanism = Parallel) now only decreases the real time factor to 2.02. It doesn't improve the speed of the simulation as drastically as for the the equal time step case. The strategy of the coupling mechanism Parallel can be simply described: If no FMU is calculating, start the calculation of the FMU that is furthest behind in time. If several FMUs are at the same point in time and furthest behind all other FMUs, calculate these FMUs in parallel.

We can see in the scheduling plot that this parallel strategy degenerates to a partly sequential scheduling.

To improve the performance of models with different time step sizes, we developed the so called Parallel Fast coupling mechanism.

With the Parallel Fast scheduling we can now again improve the real time factor to 1.11.

We used the performance analysis app on a given real world example. The step sizes of all the models were originally set to 1 ms. The corresponding scheduling plot shows that overall real time factor was mainly driven by the VTMS model.

The given simulation time interval shown in the scheduling plot can be simulated in ~154 ms.

Without a significant loss in accuracy, the time step size of the VTMS model could be increased to 10 ms. (The step sizes of other models were increased as well just to not waste CPU time which might be valuable for other threads). With just this simple adaption of the model, the same simulation time interval can now be simulated in ~103 ms which is a decrease by about 33%.

In the scheduling plot of the adapted model, we can see that the ENGINE model has now become the limiting factor of the overall simulation and should be the target for further modification to improve the simulation speed.?

As much as we love to nerd out about simulation and read lengthy articles about it, we have to cut it short at this point.

We want to thank Klaus Schuch for the insights and the impressive work that is performed day to day behind the scenes.

Real-world activities and their real-time limitations bring this Simulation Saturday to an end, but stay tuned for another one soon!

Cheers, Thomas and Michael

What’s next:

You tell us!

• Which simulation topic are you particularly interested?
• What features would you love to see?
• What do you love or hate most about the tool(s) you use?

We love to hear all of it in the comments and encourage you to: - learn more on our product sites:?AVL Advanced Simulation Technologies Tools - try it:?Rescale - simulate for free via?AVL University Partnership Program as a student and academic researcher. - get in contact via [email protected] and [email protected]

?