Why AI should Earn it's place in your product, from a CEO's Perspective

Disclaimer: This was written entirely by a human. You might be able to tell from the human tone and possible errors, but stick with me.

Imagine this – you finished reading yet another article about the impressive high-tech CEO who’s using AI/ML (artificial intelligence and machine learning) for his clever “why-didn’t-I-think-of-that” widget. It’s still in early-beta, but the preliminary results are astounding. It shows off his huge market cap and packed talk-show schedule. Now, your FOMO on AI/ML has officially reached its boiling point.??

After, you head back to the office where you sit in a dev working session, and one of your teammates blurts out, “Dude, you could totally use AI to do that”. They are always saying that, but this time it’s feasible. REALLY feasible. Together, you all have what feels like one of those “defining-moment” whiteboard sessions — everything is falling into place. You know which engine you’ll use, which dataset you’ll train it with, and you’ve even identified an API to pull from. However, the kicker will be how you use your internally-generated special proprietary data that nobody else in the entire world could ever get their hands on, so now your secret ingredient is baked into the recipe.? It’ll be one of those fabled days in the company's lore that they’ll talk about when they’re on-boarding the GEN-XYZAlphas’s someday.

As professionals in the product development industry, days like this are happening regularly, and instances of growth and change will continue.?

It’s exciting and invigorating, but unfortunately, most AI brainstorms end up on the cutting room floor. Why? Because they should. Successful embedded AI solutions — the truly useful ones that are safe — can make a product more effective and lasting. Those are elusive.?

Seasoned product development professionals understand important principles that demand a healthy skepticism about using AI/ML:

• An AI’s effectiveness depends on the principle that “more works better”. More training data, more input vector, more transforming… just, more. That also means more testing, more configuration management, more storage, more hosting, more corner cases, more hallucination risk, and more uncertainty of safety risk.
• An AI is most useful when it helps you process complex interactions in data that are hard to analyze conventionally.
• Most of your data really is understandable if you do the diligence on it, and when you do, there’s usually a more reliable algorithmic approach that you should try first.?

If you’re using AI/ML for a human-interfacing product with embedded processing, then we recommend you really challenge your AI solution to earn its place in your code. Remember, you’re trading off size and complexity for the job fulfillment of your algorithm.

Why do I NEED AI for this application?

• Am I really unable to model the physics or phenomenon??
• Is my data that complex, or should I invest more time in understanding it first?

This will sound like heresy in 2024, but you should always challenge yourself to find a lighter-weight solution first, then turn to an AI if you just can’t get there.

Crossplot your data, and be honest with yourself. You may just need a linear regression (a fancy term for curve-fit), or a smoothing filter with some simple logic to deal with specific cases. We’ve seen cases where an AI was used to solve a job that really just boiled down to basic histogramming and sorting. Remember, neural nets existed as a concept for decades but weren’t generally useful until computing power and data storage grew powerful enough to make machine learning feasible. In the meantime, algorithm designers developed a myriad of robust solutions that are well-posed in theory and did most of the heavy lifting to solve complex autonomous problems.?

Will my AI/ML solution introduce potentially dangerous unknown corner cases, and how will I be able to tell?? ?

The embodiment of your AI solution will be an array of weights that will be used to “infer” the answer from an array of inputs. A huge array of weighting factors is difficult to inspect for cliffs and valleys, let alone implicit logical errors or ambiguities. More importantly, ask yourself if you’ll be able to PROVE your solution is safe? If you plan to implement an AI that will control a piece of hardware that moves or affects something, you could be introducing an unsafe condition. Although there are constantly new tools being developed to examine the inference engine (What is Explainable AI (XAI)?),? there is still no comprehensive way to inspect a complex inference engine.??

Ways to Ensure Safety:?

1. Constrain the range of inputs and/or output states.??
2. Decouple the AI from physical effectors or direct action by making it an observer-only solution, although in this case you still need to ensure that the output never causes unsafe actions on the control side.??
3. The simplest way is to keep it small enough to be able to truly test all possible combinations (and usually if you can do that, then you can write a simpler algorithm).

Will I be able to deliver a quality product with this AI ????

Remember safety’s close cousin, quality? Quality is delivered by instilling predictability, repeatability, testability, sustainability, and maintainability into the product design. This needs to hold true throughout the product life cycle, so ask yourself how much care and feeding your embedded AI solution is going to need as the product wears, or as the training data changes. More importantly, understand what it will take to keep your model under proper configuration control.?

Many product architectures segregate the ML processing from the product by letting the cloud do the heavy lifting. This is a good way to go for several applications because this can expand the data storage and then retrain the neural net without impacting the hardware or software of the physical product. However, it is vital to? remember the potential for changing the product’s behavior with unforeseen interactions.??

Am I introducing any privacy or cyber security risks?? ?

While using ML to act on personal data,? there is potential for unwittingly providing access to that data. If your product will handle Personal Identifiable Information (PII), your process needs to have a clear understanding of how that data can become exposed in unforeseen circumstances. For example, we suggest to never allow the PII to appear in the training data, even though it might seem to be encoded though the ML process because it can often be extracted from the inference engine when using the right sequence of inputs.???

Another thing to consider is that you will probably find yourself subject to a regulatory data security standard such as GDPR, CCPA, SOX, DORA, PCI, or HIPAA. It will be important to determine your compliance requirements early-on during product definition because it could have a significant impact on development costs. Unless you really know what you’re doing, avoid using it for access control, learning personal attributes of its users, or using it as a product that could be deemed software as a medical device under the purview of the FDA or its international equivalent. If you’re planning to use it in a medical device….. PLEASE know what you’re doing. Better yet, call Realtime.

Is my training corpus up to the task?

If you’re using AI, then you should be confident that you own or can acquire a dataset that incorporates the information needed for successful results. Data science can be ironic — despite relying on the data to lead you to answers, you must understand enough about it so that you can truly trust the results.? This can present different challenges depending upon whether you’re transferring a pre-trained engine, or if the goal is to train a new network from scratch.???

Either way, we STRONGLY recommend that you work with an experienced data scientist who understands the potential pitfalls and challenges with complex data structures and how they apply to machine learning. It is far too easy to fall victim to problems such as selection bias, information bias, sampling bias, quantization error, aliasing (just to name a few). If you have ever tackled one or more of these challenges you know how dangerous data can be, even if you go at it with the best intentions.

The strange thing about ML is that actual code is relatively simple, and easy to test and verify - it is the data that essentially provides the logic, and if you’re uncertain of its veracity, then you may end up with choices that aren’t the best for product outcomes. Further, machine learning is designed to change. So, today’s verified design might become tomorrow’s product reliability nightmare.??

The product development community is producing embedded AI/ML solutions daily, with disorienting results. Some of those results are frequently overstated, but there’s no doubt that truly world-changing innovations can be done. Here at Realtime, we love helping our customers create some of them. In product design, elegant simplicity is always the goal wherever possible, however, the reality is AI is usually in contrast with that goal. If you let that contrast play itself out, if the process truly challenges itself toward ingenuity, then the benefits will far outweigh the headaches that come with it.

For example, having an embedded AI detect which human is using a device to display a custom user interface that maximizes that user’s ergonomics is a really cool idea. But designing the perfect UI that is satisfying to use for >80% of humans…that’s the true beauty of innovation.?

As a successful product endures in the market, it will inevitably go through a phase where the providers search for ways to make it more affordable to manufacture and easier to support. I would challenge us all to think about what embedded AI will look like during those phases. While we’re in the AI swoon, it’s hard to imagine a day when people are “de-platforming” embedded AI solutions in order to achieve affordability. That is not to say that AI is a fad or tech-bubble that will somehow fade away. It’s absolutely here to stay. However, as new trends go, there is always an adjustment after the nuances start to reveal themselves. Designers of enduring products should endeavor to see into that future.

This article was written by Dave Felio, CEO of The Realtime Group

One Hundred Year Study on Artificial Intelligence (AI100) (stanford.edu)