Analyzing the Outliers: How TensorIoT is using Amazon Lookout for Metrics to improve our Solutions

How TensorIoT is using Amazon Lookout for Metrics to improve our Solutions

By: Nicholas Burden,? Senior Technical Evangelist, TensorIoT

In an increasingly connected world, the connectivity and flow of data and information between sensors and devices creates a tremendous amount of available data. This presents a major challenge for businesses — how can we process these vast amounts of available data in order to extract valuable information? In the sphere of data science, anomaly detection is one of the newest buzzwords, but understanding why anomaly detection matters can be challenging.

Amazon Lookout for Metrics is a new machine learning (ML) service that processes your business and operational time series data to automatically detect and diagnose anomalies, such as an unusual rise in product sales or an unexpected drop in throughput. As official launch partners for Amazon Lookout for Metrics, TensorIoT wants to share how we’re using this innovative technology in our solutions. In this blog, we’ll take a closer look at the tools Amazon Lookout for Metrics provides and how we’ve incorporated this technology into our own solutions.

What does Lookout for Metrics Do?

It sends you an alert each time a measurement is displayed in the Metric System instead of the Imperial System!

Okay, not really. The actual Amazon Lookout for Metrics makes it simple for users to diagnose detected data anomalies by grouping related anomalies together and automatically sending an alert warning that helps determine the potential root cause. Lookout for Metrics also ranks anomalies in order of severity, making certain that critical issues don’t go unnoticed in a high signal-to-noise environment. To understand why this service is important, let’s dive deeper into what constitutes an anomaly, and why knowing when anomalies occur is valuable information.

Anatomy of an Anomaly

In the world of data, an anomaly is any point of data that has differences or deviations from the normal data in a dataset. Anomalies are sometimes referred to as outliers, since these points of data typically “lie outside” of data patterns. There are different types of anomalies, but the common element they share is that they behave much differently than other recorded data. To better understand the different types of anomalies, let’s look at examples from the three major types: Point Anomalies, Collective Anomalies, and Contextual Anomalies.

Point Anomaly

Point anomalies are the most basic type of anomaly. Point anomalies occur when data points fall outside the anticipated range, norm, or pattern, resulting in an unexpected data point.

When we examine a graph showing door open cycles times over time, we can see the point anomalies where operators are leaving the machine doors open for more time than the normal duration. These outlying points give insight into potential issues in the manufacturing process.

Collective Anomaly

Collective Anomalies occur when individual data points fall within the expected normal range, but examining the group of data points shows unusual results, atypical patterns, and unexpected behavior.

In this monitoring data from a hypothetical stamping press, having a 1.5 stroke length is well within the normal data point range. However, the repeating of the same stroke length over an extended period of time compared to the normal stroke patterns is a collective anomaly that indicates something is amiss.

Contextual Anomaly

Contextual Anomalies don’t focus on specific data points or data groups, but instead look at anomalies in the overall context of data. For these anomalies, the question becomes “Is there something anomalous in the context of the overall data?”

In this graph, we see that the electricity use fluctuates within a range over the course of a day. But based on the context of the data, there’s clearly something going on that’s causing electricity use to spike at normal daily levels at different points overnight, demonstrating a contextual anomaly.

What do Anomalies Signify?

Monitoring and analyzing a business’s data patterns in real-time can help detect subtle or unexpected anomalies with a root cause that requires investigation. For example, if you’ve ever received a call from your bank or credit card asking you to verify a transaction, then you’ve got firsthand experience with the world of anomaly detection. By using anomaly detection algorithms to process their customer data, financial institutions can prevent fraudulent transactions from occurring in the first place. If a credit card user’s data places their home location in Los Angeles and shows that they don’t purchase gift cards, then an attempt to use that card in Florida to buy gift cards will be flagged as an anomaly and the system will generate an automatic verification requirement. This example demonstrates the clear value of identifying anomalies in the financial sector. Anomalies also provide critical information for other industries, like detecting mechanical failures early, an area of anomaly detection covered by another AWS service, Amazon Lookout for Equipment.

How does TensorIoT use Amazon Lookout for Metrics?

To gain value from data, you have to separate the signal from the noise. The key enabler is then being able to use information, not just gather data. With connected technology, the new goal isn’t obtaining information, it’s figuring out the relevancy of information and how data can be interpreted in a meaningful way. That’s where TensorIoT comes in.

For businesses seeking to gain deeper intelligence into their accumulated data, TensorIoT is already skilled at creating user-friendly visualization tools (like our SmartInsights product) that act as a clear glass pane for your decision making purposes. We build the connectivity to get the data ingestion into Amazon Lookout for Metrics for analysis, and then provide the tools for you to action the outcome.

TensorIoT gives you the tools to keep your business running at peak performance and provide a manageable and organized solution for even the most complex hierarchies. We help companies rapidly and precisely detect abnormalities, diagnose issues, and take immediate action to take advantage of trends and adapt to economic shifts.

Why do Anomalies Matter?

Each anomaly discovered could be an opportunity to save money, reduce waste, or even create new business opportunities. Detecting anomalies in pricing or revenue can tell you if you’re experiencing some sort of technical glitch in your online store, protecting your company from unintentional errors in pricing or shipping costs. As previously mentioned, anomalies can also signal fraudulent activities, enabling your business to prevent malicious actors before their actions have deep impacts.

In addition to protecting businesses, anomaly detection can help drive business outcomes and improve sales. Anomaly detection can be used in tandem with social media monitoring to predict demand shifts and improve product innovation. By using a baseline level of user activity and engagement on social media, digital marketing, and advertising, anomaly detection can identify spikes in searches for a particular topic, giving businesses a chance to capitalize on increased interest and target product and campaigns to get an early jump on trends. Another use for anomaly detection in the retail sphere is determining when demand spikes at certain times of the year, giving advertisers and marketers a chance to prepare demand generation ahead of schedule.

Even before the release of Amazon Lookout for Metrics, TensorIoT helped companies implement modern architectures, leveraging machine learning and IoT connectivity to make data intelligible and actionable. By using Lookout for Metrics along with TensorIoT solutions, we help make your business smarter by giving you warnings when critical anomalies take place, ensuring that nothing slides under the radar.

Our ML team leverages AWS tools like Amazon Lookout for Metrics to build and deploy custom models for forecasting and predictive maintenance, and our development team gives you the tools to leverage the collected data and insights from cutting-edge solutions, freeing you to take advantage of the full benefits of collated data.

As always, TensorIoT will keep using the latest in AWS technology to build our solutions and keep making things smarter. Contact us at?contact@tensoriot.com?to learn how we can help your business leverage machine learning.