Machine Learning: An Attempt to introduce with ML.NET

What is Machine learning ? There are hundreds of definitions in google but the best one as a programmer I found is "Programming the Programmable".

Let’s try to understand types of ML: People have categorized into many types but to make it simple I would generalize ML as two mostly common used learning methods as:

1. Supervised : Presented with sample inputs and their desired outputs and ultimate goal is to learn a general rule that maps input to outputs. Few Supporting algorithms are Regressions (Linear, Polynomial), decision tree, classifications like KNN, Trees, SVM etc.
2. Unsupervised : Here there are no desired outputs given to computer, but leaving it on its own to find structure in the input and here goal is to discover hidden patterns or groups in the data. Supporting algorithms are Clustering  SVD, PCZ, K-Means, Hidden Markov Models etc.

And here comes another exciting learning methods in ML called as Reinforcement learning, it’s a machine learning where an agent learn how to behave by performing actions and watching results. Supporting algorithms are Q learning, SARSA, DQN, DDPG.

These learning methods are more likely to achieve accurate, reliable and generalize data and these can be revolved more around how the data is used and with this here comes the importance of models, how these models are helpful to train data to get desired results through these ML algorithms.

Basically in a nutshell overall Machine learning is based on the concept of creating models using the data and for creating these models, There are many ML tasks such as:

1. Classifications - which defines whether it’s A or B.
2. Regressions - How much and How Many.
3. Clustering - How is this organized.

There are many existing solutions for Machine Learning like

• Alteryx - offers a unified platform that addresses data, analytics and machine learning.
• AWS SageMaker –managed service that provides h the ability to build, train, and deploy machine learning (ML) models quickly.
• IBM Watson Studio, Rapid Minner, SAP Leonardo… etc..

And Supporting Languages are Phython, R on Matlab. But I am Microsoft lover so I would like to support and talk more on Machine learning entries from Microsoft.

Entries from Microsoft are:

1. Azure Cognitive services which are pre-built and pretrained.
2. ML.NET – Framework to create and train Custom models

I will talk more on solutions provided by Microsoft. Let’s start with Pre-built ML Models before I dive into ML.NET

When we talk about Pre-built ML Models these are basically Azure Cognitive services like Speech, vision, language, knowledge search. All can be found in Azure Cloud Blades. There are some limitations with these pre-built ML Models and Microsoft is still coming up with more mature models. But at same time Microsoft came up with ML.NET.

What is ML.NET: Before I start thanks to Ankit Asthana for his contribution in ML.Net

It’s a open source cross platform .net framework for building custom models. Few highlights and selling points are its free, flexible and available offline as well.

Microsoft has already initiated and tested ML.NET in their own products like Azure Stream analytics (for anomaly detection), power point (design ideas), Bing (ad predictions) etc.

Now lets start how ML.NET works , how ML.NET helps us to create our own custom Models.

Before that Being a Microsoft lover and .net Programmer Why I am endorsing ML.NET? I feel its more of developer focused, open source and proved with few successful products though yet it has not good commodity support like Python and R but I am sure coming months ML.Net will tops in charts.

For now ML.NET we have availability to create and train our own models, which makes ML.net flexible and adaptable to our specific data and business scenarios. I would say it is made for .Net Developers to use their existing skills to easily integrate machine learning into almost any .Net application, which means that if your are comfortable with C# or F#, you no longer has to learn new language like R or Python. And ML Framework offer more tool , nugets, features to help you easily build , train and deploy high quality custom machine learning models locally on your system without prior machine learning experience. And since it is free and open source, we can run it anywhere for free.

ML.Net also offers support with python as binding named as NimbusML. Any data scientist more skilled in python can also create models in python with NimbusML, which can then be consumed in our native .NET application.I will write about NimbusML in detail in other blog.

Now coming back, Overall how it works with context to  Microsoft tool stack,  There are basically 3 blocks of processes that can be used.

• Prepare Data:
• Build and Train:
• Evaluate and Run:

Below Diagram represents the Steps from loading the data till running the models for final output.

Below Diagram and steps represents the application iterative process of model development using ML.Net.

Various step are as:

• Collect and load training data into an IDataView object.
• Specify a pipeline of operations to extract features and apply a machine learning algorithm.
• Train a model by calling Fit() on the pipeline.
• Evaluate the model and iterate to improve.
• Save the model into binary format, for use in an application.
• Load the model back into an ITransformer object.
• Make predictions by calling CreatePredictionEngine.Predict()

To be Continued... for more updates.. Sample code, integration with Tensorflow, NimbusML