Deep learning (computer vision) & satellite imagery for estimating hurricane intensity

1. Problem statement

Tropical cyclones (i.e. hurricanes and tropical storms) are among the costliest natural hazards which can cause thousands of deaths and billions of dollars of damages each year. The risk assessment models for estimating storm risk and damages generally utilize an exponential or power function of wind speed, and therefore, accurate estimation of storm wind speed is imperative for forecasting storm intensity as well as for predicting the losses. Nonetheless, the current method for predicting the intensity of storms is based on visual inspection of complex cloud features (a subjective technique that relies on human perception) that was developed back in the 1980s. Computer vision (i.e. the technology behind self-driving vehicles, which has also been vastly employed for medical diagnosis in recent years) can be a promising tool for developing an automated, objective, and reliable forecasting technique.

In this article, I utilized over 70,000 satellite images and developed a deep learning model to estimate the intensity of tropical cyclones (TCs). My overarching goals for this exercise are as follows:

• Develop a deep learning model for estimating the wind speed of tropical storms ? The model should outperform a benchmark predictor
• Conduct feature engineering and restructure the inputs so that all the analyses can be performed only using free resources with limited time spent
• Raise awareness about natural disasters and how AI and novel data science tools can be useful in various perils

2. Background

According to a recent report by MunichRe, in 2020, natural disasters in the US caused $95bn of losses, $67bn of which were insured (substantially higher than 2019 losses: $51bn and $26bn, respectively). 2020 was the fourth costliest year (from natural hazards) on record for the US, but it shattered the previous record for frequency of billion dollar disasters with 22 events (compared to the previous record of 16 billion dollar disasters that happened in 2017). The following figure (from NOAA) shows the location and timing of the 22 billion dollar disasters that happened in 2020, 9 of which were caused by tropical cyclones.

The 2020 Atlantic hurricane season was hyperactive, with a record-setting 30 storms, 13 of which reached hurricane status (i.e. maximum sustained wind speed above 74mph). The overall losses from the hurricane season in North America came to $43bn, of which $26bn was insured. The category 4 Hurricane Laura, which hit Louisiana in August, was the most destructive weather event across the US in 2020, costing $13bn (watch a short visualization I made for it a few months ago here). The following animation (by ClimaCell) illustrates how busy the 2020 hurricane season was!

3. Data

The data for this exercise is provided by Radiant Earth Foundation in collaboration with the NASA IMPACT team through a data science competition on DrivenData. The training data consists of 70,257 images from 494 different storms. The images were captured by the Geostationary Operational Environmental Satellites (GOES) long-wave infrared frequency band #13 (10.3 microns), and they all have a consistent shape of 366x366 pixels. For each image in the training set, a corresponding wind speed is provided, and the goal is to develop a model that can estimate the wind speed of any given similar image from a test period. One specific criteria that is considered is that the validation data should only consider the images that succeed (temporally) the training images (i.e. the model may not use images captured later in a storm to estimate the wind speeds of images captured earlier in a storm).

The following figure shows the distribution of training images according to their wind speed, which clearly shows that the number of training images substantially decreases for higher wind speeds.

I split the training data into 80% and 20% for training and validation, respectively. For each storm event, the first 80% of images are used as training data and the latter 20% are used as validation data.

3.1. Preprocessing and Data Augmentation:

Stacking the training and validation data leads to data size of 70,257x366x366, which exceeds the 12.7GB free RAM that is provided by Google Colab. Therefore, the input data should be resized to decrease the required memory. Additionally, as it can be inferred from the previous figure, the number of training images is substantially reduced with increasing wind speed. Abundant training data generally leads to more robust machine learning models, and therefore, it is advised to implement data augmentation techniques to generate synthetic training data. Various methods (e.g. Generative Adversarial Networks, GANs) can be employed to generate synthetic data, and this link provides a list of relatively simple techniques for such (e.g. rotation, adding gaussian noise, flipping, scaling, cropping, etc.). For this exercise, I resized the images to 64x64 pixels by first zooming into a high cloud brightness area of 256x256 pixels and then passing them through an AveragePooling2D layer in Tensorflow (you may also use OpenCV to resize the images to any shape). Then, I rotated the training images by 90, 180, and 270 degrees (due to memory limitations, the rotation was applied only on the training images that corresponded to wind speeds above 40 knots). The training data size after reshaping and rotation is 150,646x64x64. The following figure shows an example of one original training image and the four pre-processed images that are used as input for training the model.

4. Methodology

4.1. Model structure

Computer vision is a powerful tool for developing predictive models, which is built on the long legacy of neural network modeling with a key feature of allowing neural connections to multiple layers of nonlinear operations. New hardware innovations (particularly leveraging graphical processing units, GPUs) and ease of access to massive data have enabled convolutional neural network (CNN) algorithms to become powerful enough to rival human performance in various complicated tasks.

The following figure shows the steps taken in this exercise to estimate the intensity (i.e. wind speed) of a tropical cyclone.

The pre-processing step was explained in section 3.1, which then feeds the pre-processed training data to a Convolutional Neural Network (CNN) model. For this step, I developed and trained multiple CNN models with different structures (that will be further explained in the following), all of which generate an ensemble of forecasts. Then, these forecast ensemble members are utilized as input to a post-processing statistical layer to generate a most likely wind speed forecast. For this exercise, I used a simple neural network model for post-processing, and therefore, I am not benefiting from previous time steps to forecast the succeeding time steps (i.e. the CNN model is examined independently to assess if such a mediocre system can outperform traditional methods and if so by how much). However, the modular structure allows one to conveniently scale the model up in case more resources are available.

The deep learning model of this article has a sequential structure similar to the following figure. The model receives the pre-processed training data as input (with a data size of 150646x64x64x1) which is passed through several set of layers that is fed to a flatten layer and is then fed to a fully connected layer that generates a deterministic forecast. Each set of layer consists of a combination of convolutional layer, batch normalization, MaxPool2D, and/or droupout. The Leaky ReLU (a fairly efficient and streamlined activation function) is used as the activation function, and the CNN model is compiled using MSE (mean squared error) as the loss function and SGD (stochastic gradient descent) or Adam as the optimizer. The model is trained for up to 40 epochs (with an early stop callback to avoid overfitting) using batch sizes up to 1500. The models examined ended up having between ~3,000,000 to ~9,000,000 trainable parameters.

4.2. Computing resources

All the analyses of this exercise have been performed in Python on Google Colab notebooks (this was partly out of personal curiosity to understand how much one can achieve using only the relatively limited but free resources that are now available). The deep learning model was developed using Tensorflow, and the data analysis and visualization were primarily performed using pandas, numpy, and matplotlib.

Google Colab offers GPU processing via the Nvidia K80 GPU which has 12.7GB RAM, which I have used here to train the CNN models. Comparing the processing times, training the model on GPU was about 40 times faster than training the same model using CPU.

5. Results and validation

The model forecasts are evaluated for validation images (a total of 13,865 images). The figure shown here indicates the mean absolute error of forecasts for 10 ensemble forecasts across different wind speeds (a rolling average of 10knots is applied here to reduce the noise and better reveal patterns). As it can be seen, the forecast bias increases for high wind speeds (which had fewer training images). Notably, there is no "best model" that can consistently result in the lowest bias across all wind speeds, which highlights that overlooking the outputs with a single objective function value can be misleading. The following figure shows a density-type scatter plot of the model predictions for the validation period. The objective functions of the final model forecasts (after post-processing individual ensemble members) for validation images are as follows:

• Coefficient of determination (R^2): 0.81
• Mean Absolute Error (MAE): 6.8 knots
• Root Mean Squared Error (RMSE): 9.3 knots

The following animation shows the model forecasts of an event for validation images. The chosen event was among the prolonged storms (with high number of images) that resulted in very promising forecasts where model predictions are quite similar to the "observed" wind speeds. It is to be noted that the "Observed" wind speed denoted in this animation is in fact the labels provided in the dataset, which is not necessarily measured every half an hour but can rather be expert estimates for such). For simplicity, I used this terminology here.

Is the model any good?

To validate the model, I pursued two approaches: compare with a benchmark model and compare with published studies. The DrivenData competition published a benchmark model which fine tuned a pretrained model (ResNet152) for transfer learning. That model had an RMSE of 12.7 knots for an out of sample test period. A study published by Wimmers et al. (2019) investigated the same research question using ~56,000 images (~250,000 augmented images) that were captured by satellite passive microwave imagery. Their model led to an RMSE of 14.3 knots, which is higher than what is presented here (a notable discussion by the authors is that there are no consistent "observed" data for wind speed of TCs, which makes the model performance subjective to the quality and frequency of aircraft reconnaissance observations). Another recent study by Maskey et al. (2020) used over 97,000 GOES infrared images and employed CNN to perform a similar assessment. They achieved an RMSE of 13.62 knots on their test images. Therefore, it can be concluded that the model developed here outperforms the aforementioned approaches.

6. Summary and final notes

An estimated $10 trillion insured coastal property along the Atlantic and Gulf Coasts in the U.S. is vulnerable to landfalling tropical storms. The monetary losses caused by natural hazards have substantially increased over the past few years (due to increased exposure of assets, aggravating vulnerability of aging infrastructure, and climate change impacts), which can have significant financial impacts on insurance and re-insurance companies. Therefore, improving the accuracy and efficiency of risk assessment models can be quite beneficial, and deep learning is a promising tool for that goal.

In this article, my objective was to spend minimal time and solely use free resources (Google Colab) to develop a deep learning model for estimating wind speed of tropical cyclones. The dataset provided through the DrivenData competition made this analysis more convenient. Overall, the results indicated that the model developed and presented in this article is about 15-20% more accurate than a benchmark model or similar published studies. It is to be noted that the model developed here is not necessarily the best model and I have no intention for claiming such. In fact, the most accurate forecasts for the data science competition have over 100 submissions (compared to only 7 iterations that I had) and they benefit from the temporal sequence of data, which led them to be over 20% more accurate. Here, a model was developed that solely relies on the current condition and it does not take into account the preceding conditions (i.e. similar to computer vision applications in radiology, where a diagnosis is to be made based on a single image). In addition, a production model can leverage other datasets (e.g. air pressure from reanalysis and numerical weather prediction models), and probabilistic methods can be incorporated to generate maximum likely range of forecast (instead of a deterministic forecast). Finally, the analyses presented here merely focused on nowcasting and monitoring the current condition. However, an LSTM layer can be employed at the last step of the model structure to forecast the wind speed for succeeding timesteps.