DO for DSX : Why?
In a recent post, I introduced the major areas of functionality of our recent delivery of Decision Optimization for Data Science Experience. The DSX Local platform now additionally includes the ability to develop, validate and deploy Decision Optimization models. But, without background on Decision Optimization, you might wonder: “Why is it so important to have Decision Optimization in my Data Science Platform”. This is the topic of this post.
Descriptive, Predictive and Prescriptive Analytics
The definition of Data Science from wikipedia is: “Data Science is an interdisciplinary field of scientific methods, processes, algorithms and systems to extract knowledge or insights from data.”
In other words, using scientific methods, and based on data, data scientists want to:
1. visualize data in different ways in order to understand what is happening now,
2. find out the relationships and behaviors inside data in order to understand what may happen next,
3. use this information and insights in order to decide what actions should be done next.
These intentions are usually classified as Descriptive, Predictive and Prescriptive analytics, as represented in this chart:
For each of these objectives, different scientific techniques can be used, and sometimes the same type of algorithm can be used in different areas.
Another nice example to illustrate this classification is the following:
In your car:
- The car, using sensors, shows the current speed of the car on the speedometer: this is descriptive analytics,
- the car, based on recent driving habits, and oil level sensor, estimates and shows the predicted distance before refuel: this is predictive analytics,
- the car, based on the destination you have indicated, data on the car and traffic, suggests the path to follow: this is prescriptive analytics.
Why Predictive is not enough?
The news suggest machine learning is all what you need to take the right decisions. With enough historical data, algorithms will train to find the right decisions. This is currently true only on specific and limited cases.
As an example, let’s consider the marketing campaign optimization. Imagine a company can promote some products to some customers.
Using a trained predictive algorithm, it is possible to predict the expected revenue for each product/customer combination according to product and customer properties. If there is no additional regulation, then all best combinations can be selected. Easy.
But what if there is some additional regulations?
Consider the simplistic case of 3 products and 10 customers, with a given limit of 3 customers per campaign and 1 product per customer. Because of the constraints, it might be better not to always select the highest individual expected revenue. The optimal solution is yet not trivial, and these are just thousands of combinations. Imagine the real application with dozens of products and millions of customers!
Where does Decision Optimization applies?
If I would be sarcastic, I would say Machine Learning applications reduces to recognizing cats and dogs in pictures. This is an area where it is really performing well. If correctly trained, the model might even perform better than humans, as he might extract new features and better use these features.There are lots of potential (future?) applications of Machine Learning, and you read about them everyday in the news.
Decision Optimization easily outperforms humans after the model has been formulated. Applications of Decision Optimization are just everywhere (already!), although we don’t read about them every day in the news.
When you go to the supermarket, the prices have been optimized using DO, the positioning of products on the shelfs have been optimized using DO, the replenishment of the store has been optimized using DO, the localization of warehouses has been optimized using DO, the transportation of the items from the plants to the warehouses has been optimized using DO, the production lines have been scheduled using DO, the maintenance plan of the production lines has been optimized using DO, etc.
In the hospital, nurses are scheduled using DO, in the post office, mails delivery is optimized using DO.
At home, the water and electricity distribution are planned using DO. The generation of electricity is planned using DO, and if you have a failure and call the customer service, the personal there have their planning optimized using DO.
Traveling? the planes construction and maintenance is scheduled using DO, the different flights are optimized using DO, the types of fleets are assigned using DO, the crew and pilots are scheduled using DO, the price of individual tickets is optimized using DO. The assignment of planes to tails and gates in the airport is optimized using DO, and your luggage is handled optimally using DO.
You understand I could continue, and that Decision Optimization even if not very present on the newspapers is really used everywhere.
A data scientist willing to extract insights to take action from data should really consider DO.
Why Prescriptive requires Predictive?
Decision Optimization solves complex problems but requires data. In order to optimally schedule which goods should be produced and transported, one need to take into account the expected demand for these goods at different periods and locations. Decision Optimization is fed with future data which has been predicted from historical data. This is just an example, but any Decision Optimization application requires at some point the outcome of some kind of predictive model.
Conclusion
With the addition of Decision Optimization to Data Science Experience, data scientists now have access to all technologies required to implement complete decision applications within the same platform with common tools to develop, share, validate and deploy models.
With an example, such as Marketing Campaign Optimization, I will illustrate how the complete set of features can be used.