“Smarter” Phones

Smartphones are not just utility items these days. They have become an integral part of our lives and in many ways guide the way we perceive the world. In this post, I will briefly reflect on the history of SmartPhones, rise of Data and Computer Power, and how companies such as Google are leveraging Machine Learning to make SmartPhones Smarter and more relevant. I will present a simple Android App I built to demonstrate how easy it is to build and deploy Machine Learning models on a device using Google TensorFlow lite library and do real-time inference. Finally, I will end this post by sharing my views on the future of AI and way it will transform the world as we know it. Happy reading!

SmartPhones

In the year 1992 IBM introduced the world to Smart Phones by unveiling the Simon Personal Communicator. Although laughable by today’s standards, the features provided by Simon were truly ahead of its time. But Simon was clunky, painfully costly ($1,100 launch price), and had poor battery life. No surprise, despite its hype and brand value, IBM managed to sell only 50,000 of them.

Fast forward to 2007, with the launch of first iPhone, Steve jobs made smartphones a reality. At price points of $499 and $599, Apple sold 1.4 million iPhone during the first year of its launch and a whopping 11.6 million in the following year, practically killing the market for an already successful product — iPod.

But Apple was not the only one trying to ride the smartphone wave. Around 2008 another giant, with deceivingly innocent version names, was about to take the smartphone industry by storm. Starting with version 1.5 with a cute name cupcake, Android OS grew at breakneck speed both in terms of volume of devices running on it and number of feature it supports.

The latest android version — Pie, with features such as cloud storage, Adaptive Battery, gesture navigation, and an AI-powered assistant that can call a salon and book an appointment on user’s behalf, makes the poor Simon look like some forgotten artifact from Stone Age.

Compute & Data

It’s incredible how much compute power we carry in our pocket without paying any mind to it. The Apollo Guidance Computer (ACG) which was critical for the first man moon mission in 1969, had 64Kb memory and operated at 0.043 MHz. Not to mention it was the size of a car and cost millions of dollars. The iPhone 6, which is quickly going out of fashion, operates at 1.6 GHz (almost 32,600 times faster than ACG), has minimum 16gb of storage, cost only $199, and fits in your pocket.

But amazing as it is, rise in compute power is only half the story. Several decades ago a Saudi oil minister Sheikh Yamani said: “The Stone Age did not end for the lack of stones and the Oil Age will end long before the world runs out of oil.” The age we live in today is aptly called the Information Age and currency of the day is data which we are in no danger of running out of in any foreseeable future. In fact, it’s mind-boggling how much data is being generated today and I don’t mean just the data generated by traditional sources such as business transactions. The sensor data generated by a plethora of all-pervasive edge devices is breaking all barriers in terms of volume, variety, and velocity. You move a muscle and that Smart Band on your wrist generates a data point.

Machine Learning

This accelerated growth in data and compute power brought a rather arcane field in computer science under the limelight. Since 1950 when Alan Turing coined the famous Turing Test, many great minds in research are actively engaged in the field of Machine Learning. Over the years great breakthroughs have been made by researchers in this field. But it’s only in last decade or so, when businesses all of a sudden found themselves sitting on piles of data and access to affordable compute power, did Machine Learning gained popularity as a tool to generate new revenue streams.

The traditional computer programming involves writing a deterministic set of rules for achieving an objective. Depending on what the objective is, such programs can be either super simple or overwhelmingly complicated and extremely specific. This limits the application of computer programming in a number of domains. In Machine Learning, however, we don’t write any rules. Instead, we write the program in such a way that it “learns” those rules from the data. Since learning depends on data, with enough quantity of accurate and relevant data Machine Learning is able to make headway in domains which were tough to crack in using traditional programming techniques.

Deep Learning

Although traditional Machine Learning programs or models are “learned” from the data, they are not built completely from scratch using the data. Instead, the machine learning expert performs something called Feature Engineering, which is sort of handcrafting certain key aspects of the model called features based expert’s understanding of the problem domain.

Once Features are crafted, data is molded into these predefined features and predictions about the objective are made. The mapping between the features and predictions is governed by something called model parameters. By gauging how off the prediction is from the truth, expert tweaks the model parameters in an iterative fashion. This iterative process is where the actual “Learning” happens.

As you would have guessed, Feature Engineering is very time consuming and requires a deeper understanding of the problem domain. With the deluge of data, however, experts started exploring whether instead of handcrafting the features can we not “learn” the features the same way we learn the model parameter.

Let me explain this concept using a bit of math. Let x represent factors which explain the objective y we are interested in. We can represent the machine learning model as a function f that maps x to y.

y = f(x)

Now, if the machine learning expert believes that the relation between x and yis linear, meaning movement in y relates monotonically to the movement in x, then f can be represented simply as:

f(x;w) = (x)? w

Here w represents the model parameters that are learned from the data. Above equation is just a compact way of representing product between x and w. Now, it's very rare that the relation between x and y is actually linear. In a situation where this relation is not linear, the model f(x;w) is always going to make a biased prediction not matter how good a job we do with learning wusing the data.

One way to fix this issue is transforming x such that the product of transformed x and w is no longer linear. let’s take g to be one such transformation function and add it to the definition for function f.

f(x;w) = (g(x))? w

In traditional Machine Learning, the expert handcraft the function g as part of the Feature Engineering step.

Now if we want to learn g from data instead then we will introduce an additional parameter in f.

f(x; θ,w) = (g(x;θ))? w

Here θ represent the parameters for function g which are also learned from data. function f, in this form, is known as feed-forward neural network and function g(x;θ) represents the hidden layer of this network.

We can apply more functions like g before we multiply their output with w.

f(x;θ?,θ?…θ?,w) = (g ?(….g?(g?(x;θ?);θ?);….θ?))? w

This operation is equivalent to adding more hidden layers to the network and such a network, with many hidden layers, is called a Deep Network. Since all these hidden layers are learned from the data, Deep Networks mitigates the need for domain-specific Feature Engineering. Such Deep Networks, however, have their own quirks and idiosyncrasies and the branch of Machine Learning which deals with them is called Deep Learning.

TensorFlow

Having understood, at a very high level, what Deep Learning is, we will now turn to some of the tools available for building Deep Networks. Turns out there are many to choose from. Here I would give my highly biased recommendation — Google’s TensorFlow. TensorFlow provides APIs which abstract out the implementation details and enables the expert to focus on the designs decision critical for solving the problem.

TensorFlow works with computation graphs. The Machine Learning Model is represented as a graph with variable parameters. During the learning phase, the graph is fed with the data, the error is calculated by comparing the graph output with the ground truth, and variable values are modified to reduce the error. After multiple such iterations, we achieve parameter values for which the error is minimized.

The model thus trained is ready to be used for making predictions on new data. TensorFlow provides an easy way to freeze the modes (converts all variable to constants with the value equal to their learned value) and use it for inference or prediction on data model has not seen before.

The frozen TF models can be easily integrated with software developed in most of the popular languages. This is good news for business as it really shortens the time between conception and product launch.

TensorFlow Lite

The amount of memory and compute a TF model needs in order to run inference are a bit too much for a device. That is why the ML engineer deploys the model on a server and call it from the device every time the model needs to make inference. But these back and forth calls between server and device introduces latency and increases the risk of user sensitive data falling in wrong hands.

To address these issues TensorFlow came up with a lightweight solution called TensorFlow Lite. By using some voodoo magic (Flatbuffers, tight dependencies, pre-fusing activation & biases), TF lite makes it possible to run inference on the device.

I will not delve into the TF lite implementation details here. Instead, I will share the Andriod App I created which does digit Classification using a pre-trained TF lite model. The complete code can be cloned from my GitHub repo.

You can go through the code on GitHub to understand the whole implementation. Here let me highlight the piece of code that generates a TFLite model from a frozen graph.

Once the TFlite is created I simply include it in the Asset folder in my Andriod project, instantiate it, and invoke it on the button click event.

Below video shows the app in action

Road Ahead

The point I want to drive home with this very long and probably dry post is that although since the industrial age technology landscape has not stopped evolving, the pace at which things move from the realm of fantasy to the field of active research to an engineering discipline is unprecedented.

Smartphones are just one type of devices. With the advent of IoT, everything around us is getting smarter. Who knows what the future would look like. In 1989 Hollywood movie “Back To The Future” one of the “cool” future thing shown was Nike’s self-lacing shoes. With modern day smart shoes which not only track your motion but also assist you with improving your performance, who cares about silly self-lacing. My point is that our imagination is severely limited by our past experience and the extrapolation we do when we imagine the future grossly underfit the reality.

Andrew Ng, one of the leading mind in AI, recently quoted: “AI is the new electricity.” Decades ago when electricity was discovered no one could have imagined how it will become the main driving factor in our lives in so many ways. Same goes for AI and ML. It’s going to redefine so many fields in so many ways that it would be impossible to imagine how life would be without it.