COVID-19 Pandemic, How Does Your State Rank?

At the start of 2020 none of us ever thought our lives would be drastically changed by a small sub-micron size virus. Suddenly we were thrown into a world with a new vocabulary, talking about Severe Acute Respiratory Syndrome Coronavirus 2 or SARS-CoV-2; the cause of the Corona Virus Disease 2019 (COVID-19) pandemic. The news media bombarded us with unfamiliar words such as morbidity, mortality, herd immunity, asymptomatic, viral reproduction number or R number, etc.

Then there are the numbers or metrics reported daily on the various news broadcasts.

• Total cases
• Tests Taken
• Percent Positive Tests
• Total Deaths
• Number Recovered
• Patients Hospitalized

States developed online dashboards to display many of these metrics and talked about flattening the curve to phase out of lock-downs and begin returning to some form of normalcy. North Carolina's dashboard displays many of these metrics as seen in Figure 1.

Unfortunately, North Carolina’s and the other states’ dashboards are only effective at communicating the numbers and don’t demonstrate performance against a clearly defined goal. As a result, performance is open to interpretation and in some cases confusion. This lack of performance clarity at a state level compounds the problem when looking across the country and making state to state comparisons.

Covid-19 Key Performance Indicators (KPIs)

Early in the pandemic, with family members in two of the key hot spots (Washington State and New York City), an attempt was made to develop a small set of KPIs to better understand the magnitude of the problem and whether concern was warranted. At the beginning, it was clear the total number of cases was directly linked to the ability to test, and initially with the limited testing this metric was underestimated. As testing increased, so did the number of cases.

After reviewing the availability and completeness of state information, measures, and metrics; two KPIs were developed around the total deaths to answer the following questions.

• Is the Rate of deaths increasing or decreasing?
• Is the magnitude of deaths at an acceptable limit?

Obviously, there is no acceptable limit concerning deaths and each death is a tragedy; however, our way of life continues with influenza taking tens of thousands of lives each year in the United States.

So how bad is the flu? The CDC tracks Influenza/Pneumonia Mortality by State and for the last reported flu season, 2018 -2019, there were 59,041 deaths in the United States. Pneumonia typically results from a flu infection so the numbers are reported together. Compared to the 197,636 COVID-19 related deaths at the time of this article (September 19), it is clear the mortality rate for COVID-19 is 3 to 4 times greater than that for influenza.

KPI for Rate of Deaths looks at the rate of deaths over the last 3 weeks and determines if the rate decreases over time. As seen in Figure 2, the Wisconsin daily death rate has been decreasing over the last three weeks; whereas, South Dakota's rate has increased considerably.

Scoring for this KPI is between 0 - 10, with 10 being the best score. A score of 10 is given when the death rate continues to decline over the 3 week period as with Wisconsin. South Dakota's KPI score is a 0 since the death rate has been increasing over the last 3 weeks.

A reduced score is given when there is not a consecutive decline in the overall death rate as in Figure 3. The dotted line shows the overall trend and a score is given based on the rate of decline (slope of the dotted line).

KPI for Magnitude of Deaths compares the current daily death rate averaged over 7 days to the daily Influenza/Pneumonia Mortality rate for the state as depicted by the red Flu/Pneum line in Figures 2 & 3.

Again, the scoring for this metric is 0 - 10, with 10 being the best or a relatively lower death rate compared to the Influenza/Pneumonia rate. The term relative is very important in determining the scaling range for this metric. With a COVID-19 estimated mortality rate being nearly 4 times that of the flu, anticipating a rate equivalent to the flu was not realistic, so a range of 200% - 300% of the flu rate was used for scaling the KPI. If over the last 7 days, the average death rate due to COVID-19 was 200% or less than the flu rate, a KPI score of 10 was given. If the death rate was 300% or more, a KPI score of 0 was given. Between 200% and 300% a proportional score was given.

As seen in Figure 4, the current death rates for some states are extremely high (i.e., Arkansas is at 2173% of the flu rate); whereas, states such as Vermont, New Hampshire, New York, etc. are relatively low at this time.

The Magnitude of Deaths KPI is important since it takes into account state specifics related to the spread of the disease such as population, population density, time indoors due to climate, healthcare availability, demographics, etc.

Overall State Ranking

By design, both KPIs were scaled form 0 - 10 so an overall state score and ranking could be developed. The Magnitude of Death KPI was determined to be more important and was weighted 60% of the total score where the Death Rate KPI was weighted 40%.

Texas is a good example why understanding the magnitude is important relative to the declining rate. As seen in Figure 5, the Texas COVID-19 death rate has been declining over the last three weeks; however, the number or magnitude of deaths is approximately 1195% of the Influenza/Pneumonia Mortality rate. With such a high mortality rate, Texas does not appear to be in a satisfactory position.

Utilizing these two KPIs and an overall state score, a dashboard was developed to rank the 50 states from best to worst as well as to provide a RAG (Red, Amber, Green) analysis of the overall state COVID-19 performance. Figure 6 is a list of the Top 10 Best Ranked performing states as of September 19, 2020 from this dashboard.

At the time of this article, Vermont is the best ranked state with no COVID-19 deaths over the last 3 weeks followed by Alaska. As seen in Figure 7, the Alaska COVID-19 death rate has been declining over the last 3 weeks and is currently under the state's Influenza/Pneumonia Mortality rate.

Figure 8 is an extract from the dashboard showing the 10 Worst Covid-19 performing states with Arkansas being the worst performing state.

Figure 9 shows how the Arkansas death rate has dramatically increased over the last seven days and is approximately 2,173% of the state's Influenza/Pneumonia Mortality rate.

Dashboard: The COVID-19 State Dashboard provides state ranking, trends, county specific information as well as total death projection to the end of the year based on current state rates and percent of population infected (see Figure 10).

As of September 19, this simple projection estimates approximately 274,949 deaths in the United States by the end of the year. Realizing herd immunity is achieved with approximately 70% of the population already having the disease, a rough estimate of total infected in each state was developed to determine how close we are to herd immunity.  See Figure 11 for these state estimates. These values were calculated assuming a mortality rate of 0.4% or 4 deaths per 1000 infected individuals. New Jersey has the highest projected infected rate at 46% indicating the state is more than halfway through the pandemic in terms of herd immunity. Unfortunately, true herd immunity is impacted by the entire community and is not bounded by state or country political lines. When looking at the entire United States it is estimated 21% of the population will have been infected by the end of the year, indicating the country is less than a third through the pandemic.

Conclusion

Unfortunately, the reporting of the various numbers and metrics related to COVID-19 without clearly defined goals as well as an effective means to quantify progress towards those goals has propagated a confused understanding of the situation. This use of two simple Key Performance Indicators has been successful on a personal level to understand the severity of this disease and the overall progress towards returning to normalcy. Based on these KPIs, it is clear this disease is considerably worse than the seasonable flu and we are still in the first half of this pandemic.