Develop Covid Dashboard from scratch and host it on Heroku [Project]

Hello there,?If you want to develop data dashboard websites but don’t know much or anything about web development. Then you are in the right place. In this blog, we will be developing a simple and dynamic covid data dashboard website in python using Streamlit from scratch. And at the end of this blog, we will host our covid dashboard website on Heroku for free. What are you waiting for? let’s get started.

Demo

Before jumping into python, let me show what we are going to build in this tutorial. We are going to build this dynamic data dashboard in this blog. There is a slider in our dashboard. Using this slider, the user can choose no. of countries to show in the below bar charts. If the user chooses 10, our covid dashboard will be showing bar charts for the top 10 countries. In this dashboard, we will be having 3 bar charts. The first bar chat is used to show total confirmed cases, the second one for total deaths, and the last one for recoveries. Data for this dashboard is obtained from Johns Hopkins University. That is for now. Don’t worry about data processing. I will be teaching everything from scratch. And finally, please let me know your thought about this simple and dynamic covid dashboard in the comment section. Check out the live demo here

Setting up the environment

1. Open your terminal and create a folder using mkdir command.
2. Now create a python virtual environment inside the folder.
3. Activate the Python virtual environment and install streamlit and plotly using pip.

Run these commands in your terminal to set up the environment. (For Windows)

mkdir covid-dashboard
cd covid-dashboard
pip install virtualenv
virtualenv .
Scripts\activate
pip install streamlit plotly        

4. Now create a python file called app.py inside covid-dashboard (the folder that was created in the earlier step).

5. In the app.py python file, Enter the following code. This will display Hello world on your browser.

import?streamlit?as?st
import?plotly


st.header("Covid?Dashboard")
        

Save the python file and get back to the CMD where you activated the python virtual environment in previous steps. Now enter the following command to run the streamlit app.

streamlit run app.py        

Now streamlit will send your app URL

Output

If you see this output on your browser, You successfully set up the environment.

Develop the app

1. Now open the app.py in your favorite Python IDE.
2. Clear the program that was created in the setup environment step.

Import the libraries

We have to import

1. Streamlit
2. Pandas for fetching and processing the data
3. Go module from graph_objects, Plotly for the bar chart.

import?streamlit?as?st
import pandas as pd
import plotly.graph_objects as go        

• You can set icon, title, layout, and initial sidebar state with set_page_config method.
• To set the header, use the header method.
• To set the subheader, use the subheader method.
• To provide a hyper link, define the text inside square brackets and define the link inside Parentheses.

Check out the code for reference.

st.set_page_config(page_title="Covid?Dashboard",?page_icon="??",?layout='wide')
st.header("Covid?Dashboard")
st.subheader("Developed?with???by?[Heflin?Stephen?Raj](https://www.heflin.dev/)")
        

Fetching the data using streamlit cache

We are going to the covid data from Johns Hopkins University GitHub repo.

Why cache?

Our covid data is quite big. So whenever the user opens or reloads our website or modifies the value using the slider, Our streamlit app will fetch the data again from the source. This will make our website very slow. So we will fetch the data for the first time. Then we will save the data in the user browser cache. So that wherever the user opens or reloads our website or modifies the value using the slider, our streamlit app will load the data from the cache instead of the source. Thus we can improve our website performance.

Streamlit made the caching process very easy to use for developers with the inbuild Decorators feature of python.

1. Define @st.cache below the function. You can use this function to load and process the data.
2. Whenever you call the function, the result will be loaded from the cache.

@st.cache
def?fetch_data(url):
????data?=?pd.read_csv(url)
????return?data


#get?the?data
covid_data?=?fetch_data("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv")

covid_death_data?= fetch_data("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_deaths_global.csv")

covid_recovered_data?= fetch_data("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_recovered_global.csv")
        

Data processing

Have a look at our raw data

Each row represents the number of covid cases of the state or province of a country. Some countries don't have province/state-wise info. Instead, It has the whole country's covid cases. Look at Albania and China.

Solution: We have to add total covid cases by Country/Region column and finally, we will reorder the data in descending order.

Step 1: Select the last column value and use pandas inbuilt group_by method to group the Country/Region column and sum the value of the last date.

Step 2: Now your data will look like this. Country/Region column is assigned to the index of our data. We will reset the index using reset_index method and sort the data in descending order using sort_values method.

#data?processing

last_updated__death_date=covid_death_data.columns[-1]
counrty_wise_death_cases?=?covid_death_data.groupby('Country/Region').agg({?last_updated__death_date:'sum'}).reset_index().sort_values(last_updated__death_date,ascending=False)

last_updated_date=covid_data.columns[-1]
counrty_wise_confirmed_cases?=?covid_data.groupby('Country/Region').agg({?last_updated_date:'sum'}).reset_index().sort_values(last_updated_date,ascending=False)

last_updated_date_recovered=covid_recovered_data.columns[-1]
counrty_wise_recovered_cases?=?covid_recovered_data.groupby('Country/Region').agg({?last_updated_date_recovered:'sum'}).reset_index().sort_values(last_updated_date_recovered,ascending=False)
        

Final data will look like this

Slider

Let's define the slider in our streamlit app, for that you can use Streamlit's inbuilt method slider method. This method takes the name of the slider, minimum value, maximum value, and default value.

no_of_coutires?=?st.slider("No.?of?countires",?min_value=2,?max_value=len(set(covid_data["Country/Region"])),?value=50)        

Data visualization

For data visualization, We will bar chart module from plotly. In this dashboard, We have three bar charts. Instead of writing code for all three bar charts, I am going to define a function to create the bar chart. Parameters for this Function are data, the title of the graph, x-axis label, and y-axis label. This function will return the bar graph.

#bar?chart?function

def?bar_chart_countries(data,title,xaxis,yaxis):
????last_updated_date = data.columns[-1]
????fig?=?go.Figure(data=[
????go.Bar(
?????????x=data['Country/Region'][:no_of_coutires],
?????????y=data[last_updated_date][:no_of_coutires]
????????)])
????fig.update_layout(
????title=title,
????xaxis_tickfont_size=12,
????xaxis=dict(title=xaxis,titlefont_size=16),
????yaxis=dict(title=yaxis,titlefont_size=16))
????return?fig        

We are going to display two bar charts in a row. So let's define columns in streamlit using beta_columns method. Finally, Call our bar chart function and place it in the columns.

#plot?the?bar?chart
col1,col2?=?st.beta_columns(2)

col1.plotly_chart(bar_chart_countries(counrty_wise_confirmed_cases,title="Total?Confirmed?Cases",xaxis="Countries",yaxis="No.?of?people"))
col2.plotly_chart(bar_chart_countries(counrty_wise_death_cases,title="Total?Deaths",xaxis="Countries",yaxis="No.?of?people"))
col1.plotly_chart(bar_chart_countries(counrty_wise_recovered_cases,title="Total?Recoveries",xaxis="Countries",yaxis="No.?of?people"))        

We completed our streamlit. If you are looking for source code, please visit the repo.

The final app.py

import?streamlit?as?st
import?pandas?as?pd
import?plotly.graph_objects?as?go


st.set_page_config(page_title="Covid?Dashboard",?page_icon="??",?layout='wide')


st.header("Covid?Dashboard")
st.subheader("Developed?with???by?[Heflin?Stephen?Raj](https://www.heflin.dev/)")


@st.cache
def?fetch_data(url):
????data?=?pd.read_csv(url)
????return?data


#get?the?data
covid_data?=?fetch_data("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv")
covid_death_data?=fetch_data("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_deaths_global.csv")
covid_recovered_data?=fetch_data("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_recovered_global.csv")


#data?processing
last_updated__death_date=covid_death_data.columns[-1]
counrty_wise_death_cases?=?covid_death_data.groupby('Country/Region').agg({?last_updated__death_date:'sum'}).reset_index().sort_values(last_updated__death_date,ascending=False)
last_updated_date=covid_data.columns[-1]
counrty_wise_confirmed_cases?=?covid_data.groupby('Country/Region').agg({?last_updated_date:'sum'}).reset_index().sort_values(last_updated_date,ascending=False)
last_updated_date_recovered=covid_recovered_data.columns[-1]
counrty_wise_recovered_cases?=?covid_recovered_data.groupby('Country/Region').agg({?last_updated_date_recovered:'sum'}).reset_index().sort_values(last_updated_date_recovered,ascending=False)


#data?info
st.write(f"This?data?is?obtained?from?[Johns?Hopkins?University.](https://github.com/CSSEGISandData/COVID-19)?Last?updated?on?{last_updated_date}")


#slider?to?choose?no.?of?countries
no_of_coutires?=?st.slider("No.?of?countires",?min_value=2,?max_value=len(set(covid_data["Country/Region"])),?value=50)


#bar?chart?function
def?bar_chart_countries(data,title,xaxis,yaxis):
????last_updated_date=data.columns[-1]
????fig?=?go.Figure(data=[
????go.Bar(
?????????x=data['Country/Region'][:no_of_coutires],
?????????y=data[last_updated_date][:no_of_coutires]
????????)])
????fig.update_layout(
????title=title,
????xaxis_tickfont_size=12,
????xaxis=dict(title=xaxis,titlefont_size=16),
????yaxis=dict(title=yaxis,titlefont_size=16))
????return?fig


#plot?the?bar?chart
col1,col2?=?st.beta_columns(2)
col1.plotly_chart(bar_chart_countries(counrty_wise_confirmed_cases,title="Total?Confirmed?Cases",xaxis="Countries",yaxis="No.?of?people"))
col2.plotly_chart(bar_chart_countries(counrty_wise_death_cases,title="Total?Deaths",xaxis="Countries",yaxis="No.?of?people"))
col1.plotly_chart(bar_chart_countries(counrty_wise_recovered_cases,title="Total?Recoveries",xaxis="Countries",yaxis="No.?of?people"))        

Host your dashboard on Heroku

We need requirements file, setup file, and Procfile to host our app on Heroku. open your terminal and activate the python virtual environment and enter the code to create the requirements file.

pip freeze > requirements.txt        

Note: If you are using a Windows machine, your requirements file will contain windows libraries. Our app will be hosted on Linux Heroku Server, So delete pywin32 and pywinpty from your requirements file.

Create a file called setup.sh and enter the code in the setup file

mkdir -p ~/.streamlit

echo "[server]
headless = true
port = $PORT
enableCORS = false
" > ~/.streamlit/config.toml        

This will create a folder called .streamlit and configure the server in config.toml

Finally, create Procfile and enter the following in the Procfile.

web: sh setup.sh && streamlit run app.py        

Note: Procfile has no extension.

Create a GitHub repo and upload all the files to the GitHub repo. If you don't want to create a public repo, You can go with a private repo.

1. Go to https://dashboard.heroku.com/apps
2. Create a new app. Name your app and choose the region.
3. Under the deploy tab, connect your GitHub account.
4. Choose the repo and Choose the branch. Finally, Click on the deploy button.

If you successfully deployed your app on Heroku, your build log will be like this.

Congratulations ????

You have successfully built a covid dashboard using Streamlit. Check out my other Streamlit Dashboard developed for the in-depth analysis of global & Indian Covid status and covid vaccination centers info here. If you have any doubts, please let me know in the comment section.