AI 101: Getting started with AI
Fahad Mahboob (Shah)
Product Strategy | Digital Transformation | Cloud & Building Tech
There are countless courses, articles and posts about how to learn AI, I am documenting how I started learning about AI with the aim of building an AI app. Since my experience is around Commerce Platforms for B2B tech companies, the easiest use case was to build a "Product Recommendation" application. I will mention the steps and examples of how I built my first AI app on my AI learning journey.
Understand the #AI Big Picture
Starting with a general idea of #AI #ML #DL, these are just fancy ways of saying Algorithm, Technique or Statistical model. From now on, when we say "Model", we mean "a mathematical/statistical technique that predicts an outcome without specifically listing the conditions for each outcome".
When building an AI app, start with "What are we trying to solve"? "Do we have enough data to train a ML model", "if we run the data through the ML Model, are the results good enough to deploy it to a production system"? Lets break it down:
AI Application Building Blocks
Step 1: Define the Problem
Given that I have data on what customers purchased from an eCommerce store.
I want to recommend a Product to another customer
So that, it has a high likelihood that it will be purchased?
Step 2: Gather Data and run Analysis
For my product recommendation App, I gathered 104k entries of anonymized data consisting of Transaction IDs, items, users, date/time stamp, qty, product category, geography, user reviews. Ran data analysis to find missing entries, outliers (an indication of incorrect data) and other techniques to cleanup the dataset. I used Dataiku , you can use any tool including MS Excel, to make this analysis.
Step 3: Build ML (Machine Learning) Model
The Data Analysis helps us to figure out which data to include in the ML model for processing with the end result of getting a product recommendation as the output. Machine Learning is essentially a fancy way of using a Mathematical or Statistical model that will predict an outcome based on available data. I am using these 'existing' Models to see which model has more accurate predictions. This means for most applications you don't have to build a new model, you can use existing models and fine tune them for your application.
领英推荐
Step 4: Evaluate Model
Using existing datasets, divide the data set into 80% Training and 20% Testing data. Run the Model and compare the accuracy of predictions from each model against what users eventually bought. From the three ML techniques, pick the one that performs best. in my case Item-Item Collaborative Filtering worked best and is less computationally intensive. This can be an issue when scaling on hyperscalers (AWS, GCP or Azure) as it increases cost.
Looking at the error numbers, Item-Item Collaborative Filtering performs better compared to other Models. Its prediction are closer to the mean (RMSE and MAE).
Step 5: Deploy Model
Once the ML Model is selected, deploy it into production, keep testing the model and improve it where needed by adding more data, more features OR or trying a mix of multiple ML techniques to improve prediction accuracy.
Now you build your first AI App
Using tools like AWS SageMaker, Dataiku, RapidMiner, KNIME and many others, you can start building your AI App. Data stores such as Kaggle or DriveData can be used for feeding your first AI/ML models and testing the results.
Sample AI Applications
Not an exhaustive list by any measure, but gives you an idea of what you can pick and work on to get started.
AI Hyperscaler Landscape