Case Study: How Big Data is Helping the World's Farmers Adapt to Climate Change
Mats Uddenfeldt
Helping you master mindset, habits, and skills needed in your career.
With the United Nations Climate Change Conference in Paris coming up, I thought it would be relevant to shed some light on how a MapR customer is taking on the planet’s toughest Big Data challenge: analyzing the weather’s complex and multi-layered behavior to help the world’s farmers adapt to climate change.
Increased volatility in weather – the source of over 90% of crop loss – has in recent years become a major concern for farmers and our food supply. By combining modern Big Data techniques, climatology and agronomics, The Climate Corporation protects the $3 trillion global agriculture industry with automated hyper-local weather insurance.
TCC built a system using Hadoop that creates weather projections over the next two years in a dense grid across the United States. Their algorithms analyze millions of weather measurements, billions of soil observations, and trillions of simulation data points to quantify weather risk and price insurance policies carefully tailored to specific situations.
The system uses thousands of cloud servers to process historical and forecast data and generate 10,000 weather scenarios, going out several years, over each square in the grid. The resulting trillions of scenario data points are used to quantify risk to crop yield and build corresponding insurance policies. Weather-related data is acquired multiple times a day directly from major climate models and incorporated into a real-time pricing engine.
By combining all this weather related data with more than 10 trillion data points collected from over 2.5 million sensors at their customers, they are able to make qualified calculations and decisions to help their customers make the right decisions regarding their crops, and help them protect and improve their farming operations.
The quantity of data processed has grown an average of 10x every year as the company adds more granular geographic data, requiring highly scalable processes for rapid data acquisition and ingestion. This is where MapR's world-record performance and lowest Total Cost of Ownership becomes a critical success factor for TCC to deliver their solution in an efficient way.
We wish the nations of the world best of luck in the upcoming negotiations, but we're also happy to see the power of technology coming to the rescue in helping secure the produce required to feed the world's populations in the years to come. Or in the words of Andy Mutz, director of engineering at The Climate Corporation:
“We are proud of using Hadoop to provide a class of weather insurance for farmers never before available. Since 85 percent of farmers’ risks are weather related, this is our impact on the world.”
You can read more about The Climate Corporation in this feature in the New Yorker: Climate By Numbers by Michael Specter.
Further Reading
Want to learn more about how Internet of Things as an enabler to your business? Here are some assets for further reading: