Basic Video Content Analytics (VCA)

CCTVs are firmly established as a valuable security system, mainly in post-incident/event investigations. Video Content Analysis (VCA) is the automatic analysis of CCTV images to create useful information about the content. Video analytics uses software & services for monitoring video surveillance feed & providing alerts in case of temporal events. Video Analytics offers efficient surveillance system while reducing human errors. The demand for video analytics has increased over the years owing to growing security concerns. Video analytics is used for various applications such as motion detection, facial recognition, license plate detection, crowd counting & traffic monitoring among others. Video analytics finds applications in diverse industries such as transportation, hospitality, retail, & education among others.

VCA can be used to increase the effectiveness & return on investment in CCTV systems by adding enhanced or increased capabilities to detect events & analyse post-event video. It is a software algorithm that analyzes all the live video footage from all the cameras & tell a human operator if something needed attention. This advanced technology allowed human operators to analyze, check & manage large volumes of videos. VCA has been successfully used in a variety of applications:

? External & internal intruder detection

? Monitoring of plant or buildings for health & safety

? People counting

? Automatic traffic event & incident detection

? Smoke & fire detection

? Camera failure or sabotage detection

VCA is typically associated with analysis of video streams captured by surveillance systems. The live video stream is processed by the video analytics embedded in a video camera /encoder. The video motion is classified into objects based on speed & size. The algorithms can be implemented as software on general purpose machines, or as hardware in specialized video processing units. The term “embedded” describes when the VA software is designed into the camera, DVR or other unit, either as a dedicated part of its design or as an add-on card. CCTV images are broken down into their component parts, covering both static background shapes & moving foreground objects or ‘blobs’.

Information on each object is distilled by the software into its position, size, direction of motion, time in view etc. The exact data – metadata – is tied to the individual VA product. In some cases, parallel methods of extracting metadata operate at the same time because some perform better than others depending on the application & the scene. The process of capturing metadata is frequently separate from the alert rules. The image processing algorithms are unaware of what the user is looking for. This independence allows recorded metadata to be searched using different rules at a later date for forensic purposes.

This technical capability is used in a wide range of domains including entertainment, healthcare, retail, automotive, transport, home automation, flame & smoke detection, safety & security. VCA can easily identify people strolling or entering a secured area & promptly inform security.

Key Features of VCA

Camera Tamper Recognition: It identifies any effort made for tampering with the camera. Partially or entirely obstructing its range of view, or severely altering the camera angle.

Intrusion Identification: It gives automatic outline monitoring & safe area security.

Video Motion Identification: It recognizes valid motion, straining out noise like lighting variations & movements of trees & animals.

Line Crossing: It discovers a moving target which intersects the specified line. Entry or exit route can be determined & can form the line in any way.

Object Left: It identifies the ignored object for a longer duration. Looks for things that are not part of a general picture.

Object Eliminated: When a particular project displaces from the picture, it can identify the situation.

Wrong Direction: It can identify when anyone traverses a line in the incorrect direction.

Duress/ Fall Detection: It can detect when a person is under duress has under fallen.

Person Running: It identifies when anyone is running.

Video Counter: It helps in counting vehicles, people & other objects. It produces complete reports on vehicle traffic people patterns.

Loitering: It recognizes when anyone has been in a particularized regions for more than a defined time.

Video Summary: It decreases a long archived video into a flexible video report with certain events.

In theory, any ‘behaviour’ that can be both seen & accurately defined on a video image can be automatically identified & an alert raised.

VCA platforms are typically offered as:

? Central or core based – All analysis carried out by single/common units i.e. one device performs analytics on multiple video streams - typically rack mounted for use in CCTV control rooms.

? Edge based/distributed analytics – a system where the processing is not performed centrally – often integrated within or placed close to the camera.

VCA is available on edge-based & server-based devices. Edge-based VCA is capable of utilizing the bandwidth, as video images are processed on the camera itself. Server based VCA allows for more advanced calculations since it has a larger processor. Though all VCA are prone to false alarms, advances in algorithms & the increasing use of high-resolution cameras has reduced these rates.

In a CCTV control room, an operator monitors several screens. By drastically reducing the need to view hours of empty camera images, VCA is able to realise the full potential of the equipment installed & the staff operating it. This software has the ability to generate an automatic alert as well as ease the forensic analysis of historical data to identify trends & patterns.

There is a misconception that VCA will one day replace the human operator. However, by significantly reducing the multiple live camera viewing requirement VCA leaves operators free to concentrate on managing real incidents one-to-one, confident in the knowledge that the VCA system is relentlessly watching out for routine events such as detection of intruders or wrongly parked vehicles.

Relying on VCA to monitor cameras for events of interest is in many cases much more effective than reliance on human operators, which is costly with limited alertness & attention. Various research studies & real-life incidents indicate that an average human operator of a surveillance system, tasked with observing video screens, cannot remain alert & attentive for more than 18-20 minutes at a stretch. Moreover, the operator’s ability to monitor the video & effectively respond to events is significantly compromised as time goes by.

VCA software that incorporates AI & deep learning technology is valuable for much more than after-the-fact investigations. Due to the incredibly fast speed & intelligent capabilities, new ways are found to capitalize on it. AI-driven video content analytics software can be applied in:

• Retail businesses leveraging it for proactive, strategic planning to enhance the shopping experience, yielding higher sales & greater customer loyalty
• Healthcare organisations addressing operational challenges like finding unauthorised people in restricted facility areas
• Transportation hubs improving passenger flow & municipalities proactively keeping city streets safer

AI is making it possible for businesses & law enforcement to make proactive, data-driven decisions to increase operational efficiencies, as well as giving them the ability to recognise & prevent potential issues before they arise.

VCA has important impact on organisations. Using VCA in video search activity can virtually eliminate the workload in the task & at the same time improve its accuracy. Applying VCA to a working site can very accurately reveal patterns of activity & trends not previously apparent & this insight can result in substantial secondary benefits in terms of operational efficiency gains. Often there is a need to go through recorded video & extract specific video segments containing an event of interest. In some cases, time is of the essence & such reviews must be undertaken in an efficient & rapid manner.

Surveillance system users are also looking for additional ways to leverage their recorded video, including by extracting statistical data for business intelligence purposes. Analyzing recorded video cannot be performed effectively by human operators, due to the lengthy process of manually going through & observing the recorded video & the associated manpower cost for this task.

These advancements in VCA & system integration are creating opportunities for business insight that can yield greater ROI for the entire organization. According to a study by the Loss Prevention Research Council, 93% of retailers who leverage IP video for retail applications outside of security reported a positive impact on operations. The report also noted that retailers use intelligent video in many insightful ways: to conduct dwell time analysis, produce heat maps that reveal hot & cold merchandizing zones, count queues & detect point-of-sale (POS) fraud. By integrating video with POS technology & people counting analytics, retailers can even determine the percentage of people entering a store who also made a purchase. 

VCA can map out shopper density in different areas of a store at different times of the day. It would be impossible &/or cost prohibitive to manually gather this kind of data on customer behaviour. Airports which use video analytics for security purposes now also use them to enhance operations. Heat maps are used to determine commercial hot spots where people tend to congregate. In this manner, airports can demonstrate to occupants or potential tenants where the prime locations are & charge premium lease rates accordingly.

Video monitoring has grown thanks to the dropping prices of CCTV cameras. This has resulted in a jump in recorded footage, to an extent where human operators cannot cope. CCTV without video analytics is only as effective as the people doing the monitoring. A CCTV operator has to maintain a high level of concentration & divide that attention to monitor for multiple types of occurrences at a single location. Research has found that the prevalence of human error is much higher than perceived. Humans are simply not that great at monitoring for rare events across multiple video streams, with an error rate that fluctuates depending on many different, unpredictable circumstances.

In 1999, Harvard did a famous study, “Gorillas in the Midst” that exhibited a phenomenon called “inattentional blindness”. When someone’s attention is on an object or task, they often fail to perceive an unexpected object even if it is in the middle of their field of sight. In this well known study, observers were shown a video recording of people passing a basketball amongst themselves. They were asked to count the number of passes. One minute into the video, someone in a gorilla suit walks into the middle of the group & beats its chest. Less than half of the observers noticed that there was even a gorilla at all.

Experience appears to improve performance though the effect is not pronounced. A 2014 study directly testing the accuracy of both experienced & inexperienced CCTV human operators in work-like settings found that 66% of the entire cohort failed to register a unique event in simulated video footage - a pirate.

Facial recognition is made possible by deep learning & AI technology. Going forward, more law enforcement organisations will be using video analytics with facial recognition to solve incidents much faster & for retailers to immediately identify shoplifters.

Edge processing & cloud computing will drive acceleration in adoption of advanced VCA. As video continues to gain popularity, the need to conserve bandwidth is driving a surge in cloud migration & edge computing which opens up the possibility for advanced VCA that process data collected from cameras & devices.

Video Analytics is a type of software in AI which is initially used to check video streams in real time. It is no longer a niche technology available only to large companies with big budgets. It is a valuable tool that makes any business versatile & effective.

The global VCA market contributed $3.107 billion in 2017 & is estimated to garner $14.443 billion by 2025, growing at a compound annual growth rate of 21.4% from 2018 to 2025 - according to a report published by Allied Market Research. The video analytics market is driven by increasing demand for IP-based security systems, growing concern over public safety & security, & increasing volume of unstructured information.

References

Davies, Graham, and Sonya Thasen. “Closed‐circuit television: How effective an identification aid?.” British Journal of Psychology 91.3 (2000): 411–426.

Donald, Fiona, Craig Donald, and Andrew Thatcher. “Work exposure and vigilance decrements in closed circuit television surveillance.” Applied ergonomics 47 (2015): 220–228.

N?sholm, Erika, Sarah Rohlfing, and James D. Sauer. “Pirate stealth or inattentional blindness? The effects of target relevance and sustained attention on security monitoring for experienced and na?ve operators.” PloS one 9.1 (2014): e86157.

Simons, Daniel J., and Christopher F. Chabris. “Gorillas in our midst: Sustained inattentional blindness for dynamic events.” Perception 28.9 (1999): 1059–1074.

Wolfe, Jeremy M., Todd S. Horowitz, and Naomi M. Kenner. “Rare targets are often missed in visual search.” Nature 435.7041 (2005): 439.

--------------------------

Endro Sunarso is an expert in Security Management, Physical Security & Counter Terrorism. He is regularly consulted on matters pertaining to transportation security, off-shore security, critical infrastructure protection, security & threat assessments, & blast mitigation.

Endro has spent about 2 decades in Corporate Security (executive protection, crisis management, business continuity, due diligence, counter corporate espionage, etc). He also has more than a decade of experience in Security & Blast Consultancy work, initially in the Gulf Region & later in SE Asia.