A (holistic) decision support tool for optimal lockdown release

A little introduction

Two weeks ago, we posted some work we did on predictive modeling and control of Covid-19 for the case of Belgium (see here). We realized that if we wanted to give the controller more "degrees of freedom" and, hence, more control authority, we had to extend the model with some extra ingredients. In this article we discuss which features we added to the model, we demonstrate these and we run our controller under different settings to illustrate how an optimal sequence of measures for lockdown release could look like. We emphasize that the model is uncertain and that results need to be treated with caution. However, it would be even worse NOT to use such powerful engineering tools at hand to base our decisions on.

So, what do we have in store for you today?

• We present our updated SEIR dynamics, which include quarantining certain individuals after testing.
• We apply our controller to the current situation in Belgium and discuss how a testing and tracking strategy of the population can influence the safe release of lockdown measures.
• We discuss the limitations of the current model and how we could improve them in the future.
• We discuss the limitations of our controller and discuss what other factors are needed to obtain a truly holistic decision-making tool.

Improving the model dynamics

In our previous work, we extended the classical SEIR model in order to allow the prediction of the disease spread with a higher resolution. To this end, the infected pool is split into four types of infectiousness: 1) supermild (SM): people who show little to no symptoms at all, 2) mild (M): people with noticeable, mild symptoms, 3) heavy (H): people hospitalized but not in need of intensive care and 4) critical (C): people hospitalized and in need of intensive care. The removed pool was split into a pool of immunes and deaths. It takes several days before a heavy or critical infection becomes so severe that hospitalization is needed. To include this latency, the pool of heavily and critically infected patients is further split in two: not yet hospitalized (H, C) and hospitalized (CH,CC). The new model dynamics without testing and quarantining is shown on the left flowchart below.

However, we want to investigate the effect that testing and specific quarantine can have on the virus containment. Therefore, some further adjustments to the model structure are needed. The dynamics are shown on the right flowchart above. In this model we introduced testing as follows: people from the susceptible, exposed, supermild (asymptotic) infected, mildly infected and immune pool can be quarantined after having tested positive for Covid-19. Quarantine is denoted by the Q-suffix. For individuals in the susceptible and immune pools, this corresponds to receiving a false positive test. If a person has Covid-19, the disease progresses in the same way, irrespective of the individual is being quarantined or not.

The transmission rate of the disease in this deterministic model depends on the product of three factors. The first factor is the probability of encountering a contagious individual (testing and quarantine will reduce this effect). The second factor, Nc, is the average number of human-on-human interactions per day (social distancing will reduce this factor). The third factor is the chance that a human-on-human contact with an exposed or asymptotic person will result in exposure, β. In our previous work, we did not explicitly split Nc and β and exerted control on the product of these factors. The main advantage of splitting Nc and β is that β is now a characteristic of the disease. We note that wearing cloth face coverings would actually lower β. The advantage is that we can now exert control over 2 separate control handles to control the virus outbreak: the number of social interactions through different lockdown measures and a factor of ‘random’ testing of the population. To make the concept of 'random' testing a bit more comprehensible, you could compare it to the strategy of South-Korea or the current strategy of China. Testing does not necessarily refer to a serological test but could also be checking employees' temperature three times every morning.

A note on data quality

In our previous work, we calibrated our model to the number of reported cases in 5 European countries during the initial days of the outbreak. During this calibration procedure, we assumed all reported cases were either heavy or critical cases and found that SARS-CoV-2 is spreading at a similar rate in different countries. As new data has become available, it turns out this assumption resulted in an overestimation of the number of critical cases with a factor 2. On top of this, the number of reported deaths in Belgium also includes deaths in elderly homes and ‘suspected’ cases [3]. This most likely results in a number of ‘false positives’, which makes it hard to use the number of deaths for modeling. From now on, we use the number of patients occupying intensive care beds as the most reliable measure for virus spread. We recalibrated β to the available Belgian ICU data until March 20th and found that a six-fold decrease in the number of 'random' daily contacts (from 11.2 to 1.8) since lockdown measures are in place (from March 16th onwards) resulted in a good fit to the current Belgian ICU data. Note that we assume a step-wise change in the number of random contacts Nc on March 16th. It is needless to state that this is a poor representation of actual human behavior. The transition to the lockdown was, in fact, a gradual one, as can be deduced from the Google COVID-19 Community Mobility Reports [4]. However, we use 1.8 random daily contacts because it brings the system to a ball-park accurate representation of the actual situation which we will use as an initial condition for demonstrating the controller.

Process control for the layman

As we have the impression that the control part, which we see as our main addition to the problem, is more difficult to grasp for the layman, here is a short intro to process control. Experts in control are welcome to skip this section.

A predictive model consists of a set of equations and aims to predict how the system will behave in the future given a certain input. Process control flips this around and aims at determining what input is needed to achieve a desired system behavior (= goal). It is a tool that helps us in “controlling” how we want a system to behave. It is commonly applied in many industries, but also in our homes (e.g. central heating, washing machine). It's basically everywhere. Here's how it works. An algorithm monitors the deviation between the goal and the true system value and then computes the necessary action to "drive" the system to its goal by means of an actuator (in industry this is typically a pump or a valve). Applying this to Covid-19, the government wants to "control" the spread of the virus in the population by imposing measures (necessary control actions) on the public (which is the actuator here) and achieve the goal that the number of severely sick people does not become larger than can be handled by the health care system. However, the way the population behaves is a lot more complex compared to the heating control in our homes since not only epidemiology (virus spread) but also different aspects of human behavior on both the individual and the societal level (sociology, psychology, economy) are involved. This leads to multiple criteria we want to ideally control simultaneously and we want to use the "smartest" algorithm we can get our hands on.

Model-based predictive control

A controller thus monitors the deviation between our goal and the current condition and calculates an action to be taken. Many types of controllers exist depending on how they calculate this action. For example, it can be computed based on very simple rules of thumb, simple mathematical actions, or in a more advanced setting, the controller can use the knowledge from a predictive model in its calculations. The latter is called model-based predictive control. Especially for a complex problem with multiple objectives (health, economy, sociology) and many possible actions (different lockdown measures, testing, tracking) such advanced model-based predictive controllers (MBPC) are very powerful. The algorithm uses a predictive model (e.g. a SEIR model) to compute the best possible set of measures. Moreover, it can do this simultaneously for multiple variables we are interested in. The optimal sequence of input (over a limited control horizon) is calculated by solving an optimization problem. This means that the algorithm virtually tries many potential solutions and determines the "optimal" one by minimizing a "cost function". It does this over a so-called prediction horizon. In the case at hand, the possible control actions can be the different levels of the social distancing parameter or introducing testing&tracking. Potential goals one could think of for which our cost function could be optimized are not exceeding ICUs, keeping R0 below one and minimize the impact on psychological and social well-being. The concept of such a decision-making tool is well established but the challenge lies in the way to define and quantify the different variables for this particular system. This is beyond the sole engineering expertise and requires the combined expertise of other scientific disciplines. Or in other words, a holistic approach. We have not yet done that, but we describe some ideas under perspectives and want to pursue this in the coming months.

Demonstration of the MBPC controller

We now apply the controller to the Belgian case starting on April 8th. We need to define a couple of things:

1. A cost function, i.e. a criterion that the controller will impose on the system. Here we choose to keep the number of occupied ICU beds constant in the near future. This resembles controlling R0 around the value of 1 ('the dance'). We choose this to not put additional strain on the health care workers. For now, we don't yet consider any "economic" terms here. But these can easily be added in the future (see perspectives) and given a weight to prefer one policy over the other.
2. The frequency with which we allow altering the measures: we allow the controller to change policy on a weekly or bi-weekly basis and demonstrate the importance of swift policy changes in terms of 'controllability'.
3. The degrees of freedom or inputs the controller can play with. The improvements in the model dynamics, allow us to define two separate control handles on which our controller can act: i.e. the number of 'random' tests to be performed each day and the degree of social distancing/ lockdown. For the latter, the number of random daily contacts Nc must be discretized. We adopted three discrete levels. The discrete levels are:

• Business-as-usual: a 2008 study determined that the average Belgian has 11.2 'random' contacts on a daily basis. This figure includes all ages, all genders and all forms of contact (physical and non-physical). This number was obtained using the Social Contact Rates Data Tool (SOCRATES) by L. Willem.
• Lockdown: we use 1.8 random contacts per day as the minimum in a Belgian lockdown scenario. This value resulted in the best fit to the number of occupied ICU beds during the period of March 13th until April 8th.
• Intermediate: we use 6 random contacts per day as an intermediate value and could assume this corresponds to a state of 'heightened alertness', similar to our previous post. A next step will be to link these (and more detailed) discretization levels to specific government policies (see perspectives on how we plan to achieve this).

Some results now: the figure below shows the proposed control sequence in order to keep the number of ICU beds constant (i.e. well below the maximum capacity over the next weeks) using 2 control handles and update frequency of 7 days. The black line represents the model fit to data from the previous 3 weeks. The red line represents the model predictions (taking into consideration the suggested measures by the controller). The greyscales in the background represent the different discretization levels of Nc (with dark grey: total lockdown, light grey: intermediate lockdown measures and white: business as usual). The black dotted line indicates an amount of individuals that are either tested or tested and traced (the model does not distinguish between these cases at the moment).

 A number of important observations and reflections can be made from these results.

• Through optimization of the two control handles, the controller is able to stabilize the number of ICU patients with fairly limited fluctuations. This is an important strength of the MBPC algorithm since the controller does not only consider the immediate effect of the imposed actions but uses model predictions for up to 63 days in the future to optimize the sequence of the 2 inputs. As such, the risk of inducing a second peak by releasing measures is controlled.
• Testing, tracking and targeted quarantine shows to be a very promising course of action for long term Covid-19 containment. Creating sufficient testing capacity can significantly speed up the safe release of lockdown measures. However, continuous high capacity testing (or testing and tracking) will most likely be needed as a decrease in the number of tests almost immediately forces the controller to take stricter lockdown measures again to maintain its objective (stable ICU occupation).

It is important to note that for now, the controller does not include any weighing between the use of testing and lockdown measures. However, one could easily think of a 'simple' inclusion of economics of measures in the cost function, in which the sum of the policy costs must be considered as well in the minimization. One could, for instance, assume that each test costs an average of 1 euro while one day of lockdown costs millions of euros in economic damage. Such cost function would most certainly change our optimal control path. The inclusion of economic aspects is under development. Read our perspective section to learn more about this.

• By reducing the frequency (see fig below) at which measures can be taken (from 7 to 3 days), the controller’s ability to maintain its target becomes even more robust as it becomes less sensitive to uncertainties in the model. The risk of “runaway” virus behavior will thus be further reduced. In reality, such swift adaptations in policy can be interpreted in many different ways. It may not be feasible to allow lockdown release one week and impose a complete lockdown again in the next but a situation where shops/schools/businesses reopen 3 days a week, can very well be considered. 

• Finally, we present a scenario where no significant testing and tracking is available. It is clear that severe lockdown measures remain necessary for a longer period before a 'safe' release can be guaranteed. Furthermore, it is clear that the controller is less stable (oscillatory behavior), it is struggling to maintain a constant number of ICU beds occupied. This is due to the fact the effect of lockdown measures has a much bigger delay than the effect of testing.

Perspectives – grasping heterogeneous human behavior in the model

The main drawback of the current model is its deterministic nature, which inherently assumes that all contacts we make every day are completely random. In other words, the model assumes that the population is 'homogeneously' mixed.

A first important challenge when using a deterministic model is to link the discrete levels of the control handle Nc (number of contacts) to specific government policies. A model extension that could be used to facilitate this is age-structuring. In this approach, all population pools are split in age-bins and the interactions between the age-bins are governed by a so-called interaction matrix. This modeling approach was recently used by a team of the London School of Hygiene and details can be found here [1]. We have already extended our model with the Belgian interaction matrix shown below and we will run the controller using this model in the very near future. Such a controller is able to propose more discrete policies but an 'abstract' parameter for social distancing will remain due to the deterministic nature of the model.

Moreover, because all interactions are random in the deterministic model, it is nearly impossible to couple testing to back-tracking of confirmed cases. In the real-world, an efficient strategy to limit Covid-19 spread would be to test everyone with mild symptoms and then make use of an (anonymous) app to notify everyone the infected person has come into contact with. Such a strategy would have an impact far greater than the absolute number of tests performed. However, in our current deterministic implementation, the absolute number of tests should not be taken literally. Our model inherently assumes random testing of the population and the absolute number should be interpreted qualitatively as the 'testing and back-tracking intensity'. For this reason, we have implemented ‘random’ testing of the population as a “control handle” in the current work.

A more realistic implementation of social networks can be accomplished by distinguishing between people’s inner circles and random contacts. A modeling framework that incorporates such social heterogeneity exists and simulates the disease spread on a dynamic stochastic network. Such a stochastic model has significant advantages over the current approach, for starters, it is possible to implement a scenario where mild cases are tested and back-tracked. The main drawback is the increased amount of computational resources required to simulate this model. Running a controller implies optimizing a cost-function which in turn implies thousands of function evaluations. This may render the coupling of stochastic networks with controllers unwieldy. However, we will use the Flemish Super Computer (VSC), supported by the Flemish Government, which kindly makes computational power available for heavy Covid-simulations these days. We already implemented our additional Covid-19 dynamics (shown earlier in the flowchart above) in an existing stochastic implementation of the SEIR model by Ryan S. McGee [2] and will asses the development of a controller for this model in the future. Stay tuned!

Perspectives – going truly holistic

Given that our economy is under pressure, the next required addition is an economic model that should include the effects of measures on consumption, labor supply, teleworking and economic activity more generally. Such models do not have to be developed from scratch as they have been reported in the literature. More precisely, our idea is to link the SEIR model with a Dynamic Stochastic General Equilibrium (DSGE) model of the Belgian economy. Such models are state-of-the-art in macroeconomic analysis and allow to conduct welfare analysis of policy measures based on the utility function of households, as well as optimal policy analysis to minimize economic losses.

Furthermore, a motivation factor should be included to indicate "how well" the population is executing the enforced behavior (e.g. quarantine fatigue). The latter requires input from social scientists who can run questionnaires in populations of both patients and health care workers dedicated to producing this type of information. Questionnaires are already being performed but not in view of providing specific information for a model/controller. Additionally, more objective parameters of psychological functioning can be taken into account such as evolutions in suicide rates, use of psychotropic drugs, sick leave due to mental problems, and health care costs due to psychological disorders (available from the intermutualistic agency).

With these model extensions, we can now also extend the cost function of the MBPC and have it include additional constraints when calculating the optimal lockdown release. We envision this to be feasible in the next couple of months, but obviously this will be a stepwise effort of adding ingredients and rerunning the MBPC. We have reached out to experts in the domain of economy and psychology who are eager to throw in their expertise in order to support the transformation of the current controller into a truly holistic multi-objective decision support tool. We have written a couple of proposals to obtain funding for this work as up to now our efforts were merely voluntary as we believe it is our duty to share our knowledge and ideas with society.

References

[1] https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(20)30073-6/fulltext 

[2] https://github.com/ryansmcgee/seirsplus

[3] https://www.standaard.be/cnt/dmf20200411_04920528

[4] https://www.google.com/covid19/mobility/