Case study: Deploying AI to transform a physical operation

Background:

The world of physical operations is starting to change but is still largely labor intensive, with limited technology adoption. As a result, operational scalability becomes a big challenge as the business grows. The industry is ripe for disruption with the recent advancements in AI as well as various IoT & Edge technologies. I was fortunate to lead VMware’s physical security organization (prior to its acquisition by Broadcom) through a multi-year digital transformation as we took on this challenge, leveraging Artificial Intelligence (AI) as our cornerstone technology.

This write-up is a business focused case study of our journey. It starts with a quick look at our transformation vision and then goes deeper into our AI deployment. It includes the business case for AI, an overview of the solution, the benefits realized, challenges faced and the resulting best practices and lessons learned.

Let's take a quick step back for additional context, before we dive into the case study. Early in my career, I spent over a decade at Honeywell, one of the world’s leading industrial companies, working in various manufacturing and supply chain roles. Then I moved into the high-tech world of Silicon Valley and have now spent over a decade working in the software business. This combined experience working in the physical and digital worlds gave me a unique vantage point as we undertook this transformation, ushering a physical operation into the modern digital era. It is also important to mention that although this case study is focused on the transformation of a physical security operation, much of the learnings are equally applicable to the broader world of physical operations including logistics, retail and manufacturing.

Our transformation vision:

We established a clear and compelling transformation vision to help us align the team and our stakeholders on this multi-year journey. We set out to reimagine our operation and use cutting edge technology combined with operational excellence, to become the best security operation in the industry. We decided to pursue the transformation along 2 vectors – customer experience & internal operations.

Let’s look at the customer experience transformation first. This is worth spending a few minutes on, since it isn’t that common for security teams to focus on customer experience. The tagline we came up with which proved to be quite effective, was to transform from a 'TSA like’ security experience to a 'Disneyland like’ experience. Everyone who has flown in the US knows what the TSA experience is like. They do a good job keeping airline passengers safe but going through airport security isn’t usually a pleasant experience. Contrast that with the experience at a Disney theme park. You rarely fee unsafe and hardly ever even think about security once you pass through the screening at the gates. You don’t see security guards despite the hundreds of thousands of people from all over the world that visit these parks every day! There is security but it isn't visible and seems to just work like most things in that happiest place on earth! While we weren’t aspiring to replicate the happy Disneyland experience, that was our North star. Our focus was on providing a largely frictionless security experience. In addition to using technology, we also invested quite a bit in retraining our staff to show up differently, to accomplish this goal.

The other transformation vector was our internal operation. Our focus was on infusing the right enabling technologies into our operation while simultaneously driving operational excellence. In addition to AI, we also implemented an Edge / IoT management platform to provide real time visibility into the health of all our cameras, panels and other assets in the field. We even had a robot assisting our security guards with patrol activities but realized that it wasn't quite ready for primetime yet. We also looked at deploying drones, but the regulatory hurdles weren’t worth navigating at the time. As important as the technology adoption was our laser-like focus on operational excellence including driving strong fiscal discipline, resource utilization and ongoing productivity improvements.

It is important to mention that being part of an innovative tech company like VMware helped in driving this transformation since our charter was very well aligned with the company’s mission of helping customers transform their operations using technology. Doing this at another company with a different culture and mission would have been more challenging.

AI deployment:

Let’s double click on our AI deployment, the cornerstone of our transformation, starting with the problem statement.

Problem statement:

There are two classic problems in the security industry, which I like to call ‘finding a needle in a haystack’ problems. First, in a typical business establishment, there are 100’s to 1000’s of cameras constantly capturing video footage. But rarely does anyone monitor the feeds from all these cameras since it’s a huge undertaking given the volume and thus highly unproductive to do so. And even if you wanted to, you cannot justify hiring enough people to monitor all the cameras and watch everything everywhere all the time. All you can do is have staff spend hours poring over humongous amounts of video data after an incident happens, hoping to try and figure out the details of the incident. It is like finding a needle in a haystack! As we will discuss soon, this turns out to be an ideal use case for AI.

The second common problem is related to alarms. Like cameras, a typical business also has many entry/exit doors, each with alarms that go off frequently for everything ranging from doors not closing fully due to an airgap to an actual intrusion or break-in. In most cases, the alarms have a very low ‘signal to noise ratio’, which means most of the alarms are just noise and don’t really need much of a response. But someone still has to clear every alarm, so it becomes a tremendous amount of wasted time and effort to sort through all the false alarms. This turns out to be another great use case for AI!

In addition to these two, there were a few other problem areas we wanted to address with AI, like tailgating. In the security world, tailgating means someone entering an office without badging in. Tailgating can be problematic on several fronts – if intruders get in, they can pose a physical as well as data / cyber security threat; if you don’t know exactly who is in the office, it is difficult to confirm if everyone has safely evacuated the office if there’s an emergency or conduct contact tracing like we had to do during the pandemic. It was impossible for our staff to keep track of tailgating in a vast campus with multiple buildings, each with multiple entry/exits doors.

AI solution:

As illustrated above, AI helps address most of the problems mentioned earlier by doing the heavy lifting of monitoring all cameras, sensors and alarms, and promptly triggering the needed response, round the clock! The AI models become more accurate the more you use and train them. Like humans they get better with practice. But unlike humans they don’t get tired, don’t need breaks and consistently perform at the same high level. There is an ever-increasing library of trained AI models / threat signatures to recognize various types of activities with a high degree of precision (tailgating, door held open, person falling down, person carrying a weapon, etc). Activities that need a response are sent to the GSOC or directly to the security guards on patrol, for prompt action. The rare events that the AI model can’t ascertain (low confidence events) are sent to an AI analyst for further verification and disposition.

Equally important, the AI architecture does all this while complying with the highest privacy & cyber security standards. Data is always fully encrypted, confidential data is stored on-prem, and only anonymized meta data is sent to the cloud. At VMware, we adopted a ‘privacy by design’ philosophy and our privacy team would be involved in every project right from the beginning. It was a great partnership!

Business impact:

We did a couple of small Proof of Concept (POC) projects, reviewed the business impact, made needed adjustments and then deployed AI across our entire headquarters campus, with great results! Here are the key ones:

• A 98% reduction in time taken to investigate incidents, which not only improved security but also staff productivity – time saved that they could use elsewhere.
• An 82% improvement in near real-time visibility to incidents including tailgating, which led to more prompt responses and improved security.
• A 90% reduction in false alarms and the effort to review these alarms, resulting in significant labor productivity improvements.
• Making our staff more productive, not having them work on mundane activities like sorting through tons of video footage data and false alarms, and instead giving them the opportunity to work on innovative projects and learn about hot new technologies like AI, led to an improvement in employee morale.
• The AI deployment also presented an opportunity to upgrade our underlying video management system including full data encryption, which our legacy system wasn’t capable of supporting. This was a huge win on the privacy & cybersecurity risk front!

Challenges, lessons learned, and best practices:

While the AI deployment was very successful, getting there wasn’t all smooth sailing. We faced many challenges along the way and had to find creative ways to overcome them.

First, the technology was still in the early days when we started. There were a lot of vendors and noise in the market, but it was hard to find the ones to bet on and partner with. To address this, we ran multiple, tightly scoped Proof of Concept (POC) projects to quickly sort out what worked and what didn’t. Many times, we had POC’s with different vendors running simultaneously.

A related challenge was justifying the ROI on this technology. The price point of this cutting-edge technology was quite high. Combined with a general lack of appetite in the industry to invest in transformation programs in physical operations, this proved to be a formidable challenge. We used multiple strategies to address this. Here are some of them.

• We had to be very selective about which offices to prioritize for the AI deployment, based on risk, business impact, ROI & ease of implementation. As an example, we chose our global headquarters in the United States as a priority given the expected higher business impact, lower regulatory hurdles and a more attractive ROI given the higher labor costs, compared to some other geographies.
• Another important way to address this challenge was self-funding much of the investment using savings from the simultaneous operational excellence initiatives we were driving. We were always looking for more efficient ways of doing things and taking costs out of our operation. Working at an operationally savvy company like Honeywell early in my career had ingrained this skill and mindset in me and I was fortunate to get great support from my team members in this sometimes challenging endeavor. In addition to the obvious direct benefits, this operational excellence focus also helped us establish strong credibility with our Finance partners and other key stakeholders.
• We always worked in close collaboration with our privacy and cybersecurity teams and strived to also accomplish their objectives in addition to our own. This helped make our solutions more robust, improve the ROI and also made it a little easier to get support for our programs since privacy and cybersecurity were always top of mind at the C level!??

Conclusion:

All in all, the AI deployment met most of our success criteria and played a pivotal role in our transformation from a ‘reactive and best effort’ to a ‘proactive and best in class’ security operation. It helped us scale more efficiently and make a significant positive impact on the business!