AI for weather forecasting
?Context
The recent landslides in Wayanad are a human tragedy with 300 deaths reported, but it also led to a political slugfest between the state and the center. MPs from Kerala are asking why an early warning was not issued as we are living in the age of AI. That leads us to the question of what is the state of art of AI related to weather forecasting. This prompted me to research the topic and report my findings in this article. But first a few facts related to early waning for Wayanad landslides and early warning systems in India and worldwide.
Early warning for Wayanad landslides
The center claims that a warning was issued to the Kerala government of heavy rain and the possibility of landslides on July 23, 24 as early as seven days before the Wayanad landslides occurred. Further On July 26, a warning was issued that there would be heavy rains of more than 20 cm,?a possibility of landslides, mud could rush and that people could be trapped and die in this too.
However the Kerala CMs version is that between July 23 and July 28, the India Meteorological Department (IMD) did not issue any orange alerts for heavy rain in Kerala. Only on July 29 at 1 pm, an orange alert was issued, warning that rainfall would be between 115 and 204 mm, but the actual rainfall was much higher. The area received 200 mm of rain in the first 24 hours and 372 mm in the next 24 hours, totaling 572 mm in 48 hours. This far exceeded the initial warning.? The red alert and the possibility of heavy rain for Wayanad were announced only at 6 am on July 30, after the landslide.
Whatever may be actual situation, the fact is that an effective early warning system depends on timely coordination between the central, state and local authorities and clearly the system failed. I do hope we will learn from the failure and improve.
Early warning systems worldwide
At the twenty-seventh Conference of the Parties to the United Nations Framework Convention on Climate Change (COP 27) held in Sharm el-Sheikh in 2022, the United Nations Secretary-General António Guterres launched the Early Warnings for All (EW4All) initiative, stating its ambitious goal of expanding early warning systems to protect every person everywhere by the year 2027. Current only half the world is covered. The initiative is led by the World Meteorological Organization (WMO).
According to WMO from 1970 to 2021, the world witnessed nearly 12,000 weather, climate or water related disasters, resulting in more than two million deaths and economic losses of USD 4.3 trillion. The number of people affected by disasters has been rising, with more than 130 million affected globally every year. Due to climate change, by 2030, the world could face 560 medium to large scale disasters each year.
At COP 28 in 2023 a session was organized on AI for EWS in relation to the EW4All Initiative. This session discussed the relevant best practices in the application of, Artificial Intelligence, to make EWS more accessible, valuable, efficient, and actionable. You can access the recording here.
Early warning system in India
In India the India Meteorological Department (IMD) manages the multi hazard early warning under following three categories via a network of agencies.
- Hydro meteorological hazards
- Geological hazards
- Environmental impacts
This is how the system looks
EWS is important for India because we are highly vulnerable to extreme climate events such as floods, droughts, and cyclones. A study released by the Council on Energy, Environment and Water (CEEW) in 2021 found that 27 of 35 Indian states and union territories (UTs) are vulnerable to extreme hydro-met disasters and eighty per cent of India’s population resides in these vulnerable regions.
AI for weather forecasting
Traditionally, weather forecasts are worked out using the laws of physics and powerful supercomputers making complex calculations based on observations from weather stations, satellites and buoys. These models are known as numerical weather prediction (NWP) models. Whilst large language models such as ChatGPT have been dominating the headlines, a quieter revolution has been occurring in the background. AI models for weather prediction are becoming competitive with numerical weather prediction models.?AI powered forecasting models are trained on historical weather data that goes back decades. They are open source and anyone can run them from a laptop to predicts weather conditions up to 10 days in advance more accurately and much faster than the industry gold standard weather simulation system. Here are the top four AI weather forecasting models.
All these models use an encode-process-decode transformer framework but with differing architectures.
领英推荐
Just a few days back Chinese researchers from the Shanghai Academy of Artificial Intelligence for Science (SAIS) at Fudan University have announced the release of a new AI weather forecasting model called FuXI Subseasonal (FuXi-S2S). This model can provide global daily mean forecasts for up to 42 days, covering 5 upper-air atmospheric variables at 13 pressure levels and 11 surface variables.
All these AI models have used historical data from ERA5 historical weather dataset provided by the European Centre for Medium Range Weather Forecasts (ECMWF)?for training. They also publish 10 day global forecasts from these models on a daily basis. For example here is the forecast for 13th August 24 for India generated on 3rd August 24 from the FourCastNet model.
The experience of? ECMWF with AI models was well summarized by Dr. Florence Rabier , Director-General in her presentation at the COP 28 session mentioned above.?
Experiment with FourCastNet Model
I was able to run the FourCastNet model from my laptop. Runtime was only a few minutes. The output was a GRIB file with Global 6 hourly forecast data with the following variables
10?m u wind component
10?m v wind component
mean sea-level pressure
Surface pressure
Total column water vapor
2?m temperature
So essentially the output data dimension was 40*6*721*1440. As can be seen this is a large dataset of about 2 GB with about 250 million float32 values. Here is a sample plot of 2?m temperature from my output.
Conclusion
I would like to acknowledge Dr. Anima Anandkumar ,one of the developers of FourCastNet whose work I follow. I also hope this article will motivate some hydro met researchers to investigate if an AI model could have provided early warning for the Wayanad landslide event. I can help. My coordinates are as follows.
-??????? Mobile/WhatsApp 9910995649
-??????? Mail ID [email protected]
-??????? LinkedIn https://www.dhirubhai.net/in/sudhir2016/
-??????? GitHub https://github.com/sudhir2016
-??????? Hugging Face https://huggingface.co/sudhir2016
?
CEO and Co-Founder
3 个月Dear Sir, thanks for sharing such an informative and important article. Apart from weather forecast, even the early warning of landslides and floods need integration with AI and generative AI. We have developed a Flood Early Warning System along with our partner Jomiso for Tripura’s two river basins. Additionally, at Excel we are developing a Web based Analytical Tool for Hazard Zonation and Vulnerability Analysis for multiple types of hazard, but this article has given us the idea of doing realtime analysis using the data, that we can receive through IoT based sensors installed in the field. But at the core of everything is accurate and early forecast of weather anomalies.