The Intelligence Buffet !

An enormous spread, reasonable price, and “all you can eat” are key attributes of a buffet system. It’s not going to be much different in the world of AI/ML where the vast choices of ML algorithms and trained models can serve almost unlimited-use cases at the click of few buttons without going through the hassle of complex programing. Welcome to the world of “Intelligence Buffet,” where we have the choice to consume the trained machine learning models with ability to work with large data sets using cloud computing platforms.

Let’s compare how preparing a good quality buffet and good quality ML model have lots of similarities…. for the foodie in all of us!

Good quality buffet always starts with quality produce, the “Intelligence Buffet (IB)” is no different, it starts with Data as produce that plays a significant role in the quality of ML models. Good produce has to be mixed at the right time with specific spices at the perfect temperature as stated in the recipes. Likewise, choosing the correct ML Algorithms (Recipe) for given industry and purpose will create the magic of effective ML model.

Next in the hierarchy is the quality and treatment of our spices. Many Indian dishes (especially muglai) require special treatment of spices like soaking, roasting and timely infusion in the dish. The famous “Daal Bukhara” (Creamy Lentil) takes over eight hours to cook with spices being mixed at predefined intervals. In the world of AI/ML, the training plan for ML model will require meticulous planning about what and when to train with right data and context information. It requires patience to train for good quality results much like the patience of preparing “Daal Bukhara.”

A good tasting dish is only part of the overall user experience; presentation plays an equally important role. Defining an easy to use training interface for humans to train the machine learning model is critical to create quality machine-consumable knowledge-base.

Finally, it’s time to serve: how to layout the buffet and choosing the appropriate plate to serve our dish. Certain type of food requires heavy heated plates to keep it warm, such as holding a steaming hot Fajita skillet in a Mexican restaurant. Serving the ML model in the right application architecture where it’s called out at the right time with right data leading to perfect recommendation/prediction.

Here are 6 step comparison to make a great tasting dish and deploying an effective ML model.

Here are some examples of technology available to prepare this great tasting Intelligence Buffet

1. Machine consumable Data ..Fresh produce

Its time consuming and difficult to get good quality data. While organizations may deploy data scientists and data cleaning mechanisms to get clean data, there are variety of sources for good quality data. Here are some of the data sources

a.    MS MARCO, which stands for Microsoft MAchine Reading COmprehension open machine learning datasets for training systems in reading comprehension and question answering.

b.    Kaggle Datasets: Each dataset is a small community, discuss data, find public code or create your own kernels. Numerous amounts of real-life data set of all shapes and sizes and in many different formats.   

c.     UCI Machine Learning Repository Another great repository of 100s of datasets from University of California. It classifies the datasets by type of machine learning problem. You can find datasets for univariate and multivariate time-series datasets, classification, regression or recommendation systems.

2. ML Algorithms for purpose ..The recipes

There are plenty of open source ML Algorithms to solve specific challenges. The popular TensorFlow is an end-to-end open source platform for machine learning with comprehensive, flexible ecosystem of tools, libraries and community resources allowing developers to easily build and deploy ML powered applications.

Here are few examples of "fit for purpose" ML Algorithms

a. Accord.net: Processing audio signals and image streams.

b. SparkML.lib: High performance algorithm, its 100 times faster than Map reduce.

c.  H2O: geared for business processes, fraud and trend prediction.

d.  Cloudera Oryx: Run machine learning models on real time data, decisions in a moment, recommendations, live anomaly detection.

3. ML Training and integration ..the spice and garnishing

Apple’s CreateML & CoreML combination is one of simplest ways to train and deploy ML models on a Mac. Create ML lets you create and train custom machine learning models on your Mac. Once the model is trained it can be integrated into Apps using CoreML.

4. Cloud based seamless ML platforms …the restaurant

Amazon Sagemaker, Goggle ML, Azure ML are platforms to build, train and deploy the machine learning models right into the apps. It’s like having the large kitchen in a restaurant with ability to serve customized dishes apart from regular menu items. They have pre-defined set of ML algorithms for different use-cases and also provide flexibility to bring your own algorithms to be trained and converted into a ML model and integrated with Apps. With the ease of creating these “fit for purpose” ML models, the time isn’t far when we will see the buffet of these models available on App store to download and integrate with different applications on the go on your mobile phone.

Welcome to the world of "Intelligence Buffet" with all you can consume intelligence at your finger-tips.

Let my Foodie and Techie Friends provide their own comparisons and comments to simplify AI and make it more humane !