Storage in the age of video surveillance

The presence of a ring of video surveillance cameras clinging to a vantage spot like a cluster of digital coconuts has long been a familiar sight in public spaces. And for many years, in both Hollywood and on television, countless storylines have turned on whether the detectives or investigators could access CCTV footage and solve mystery by reviewing the tale of the tape.

But although the idea of cameras and surveillance has become an accepted feature of society (like it or not), what is less obvious perhaps is how much the market for video surveillance equipment is growing and how much the cameras themselves have changed. Both of these factors have profound implications for digital storage.

You had better be ready for your close up

First, the market. A 2020 report from IDC entitled “Worldwide Video Surveillance Camera Forecast, 2020-2025” (#US46230720) estimates that by 2025, the worldwide video surveillance camera market will grow to $44 billion, up from $23.6 billion in 2019, with a five-year compound annual growth rate (CAGR) of nearly 13%. This is largely due to the increasing adoption of smart camera systems and analytical software that enables them to be utilized in a variety of roles — beyond simple surveillance. Another report, by research firm IHS Markit, predicts that by the end of 2021 alone, there will be 1 billion surveillance cameras installed globally, with over 50% of those in a single country: China.

The growth of 4K

In the past, video surveillance cameras have sometimes been criticised both for their ubiquity and their usefulness: critics pointed out that although the cameras seemed to be proliferating in many public spaces, their benefit was undermined by poor image quality and resolution. Not any more. The next gen cameras that are driving the growth to 2025 will increasingly deliver HD and Ultra HD (4K) images of astonishing detail and clarity. In turn, this is opening up a wealth of new applications that can be managed by artificial intelligence systems: for example, monitoring industrial equipment, providing security and (more controversially) real-time facial recognition.

Why are cameras being deployed?

Many of today’s larger organizations such as hospitals, airports, university campuses, and casinos find themselves needing a video surveillance system as either a replacement for an aging CCTV installation or as a brand-new installation. The ability to quickly and easily provide high resolution video evidence of a security incident can be very relevant in narrowing down suspects in case of a crime. And the same video evidence can also limit the liability of an organization in case of a lawsuit. So there are clearly business benefits in upgrading to the latest surveillance technology.

The storage challenge

But if the number of cameras is increasing rapidly, and if the quality of the images they produce is becoming more refined and detailed, then all of this can only mean one thing: we’re going to need a lot more storage. Gone are the days when weeks of footage could be kept on a handful of old video tapes that could be wiped and reused at the end of the month. In the first instance, today’s surveillance cameras record primarily record to disk. And a single hour of RAW 4K video footage produced by just one unit consumes something in the region of 110GB of disk capacity. Multiply this by millions of hours, and hundreds of millions of cameras, and it’s clear that video surveillance applications will require colossal amounts of storage, not just for the primary purpose of storing the original footage, but also for backing up and archiving that material.

What is your strategy for surveillance storage?

This puts us right in the heart of some fairly strategic questions about storage provision for video surveillance, business value and risk management. For example:

• How much footage do I need to keep and for how long?
• Do I face legal obligations that require a minimum retention period?
• Do I need the footage to be kept offsite if it’s to be archived for a longer period?
• Who will have access to the footage, both now and in the future? 
• Will the footage need to be searchable?
• Are there privacy and other regulatory concerns about retaining this information?
• And how much budget do I have to meet my storage capacity needs?

Managing data in a surveillance workflow

There are lots of questions to be addressed in the above, but for now, I just want to focus on that final question. Clearly, even a modest surveillance installation with just a handful of Ultra HD 4K cameras, might generate petabytes of data in a typical year. In the near future, this will become the norm. And although this footage will typically be written to disk as it’s originated, the caches available to cameras are relatively small, which means that it needs to be offloaded - presently, and most often, to other disk-based storage systems - very quickly.

But if your surveillance system is producing terabytes of data on a daily basis, then you may find that the cost of storing all this information on-prem, either on a disk array or object storage device, or in the cloud, can soon become prohibitively expensive. At some point, you may face the same traditional consideration of when to begin deleting information to free up storage capacity for new footage. And in some cases, this “pressure to erase” may conflict with either business goals (creating value from the archive footage) or regulatory goals (being obliged to retain footage for a pre-determined period).

Why disk-based storage may not be the complete answer

Today, large video surveillance systems typically use large RAID systems to store the camera output for the requisite time. The size of the disk infrastructure largely depends on three factors: retention time, camera resolution and video quality (frame rate and compression). And, of course, different RAID approaches (RAID 1, RAID 5 etc) can constrain disk capacity even further. As mentioned above, in the ideal world, businesses would prefer to keep the highest resolution and best quality images they can, but as video quality and proliferation starts to surpass disk capacity growth and the economics of disk storage become stretched, some difficult choices emerge. Should they begin deleting data to free up space or sacrifice quality (and usefulness) because of escalating cost?

The benefits of LTO technology in a tiered storage system

To address this dilemma, however, there is a clear alternative: LTO Ultrium tape. Although the initial setup or expansion costs of tape are sometimes seen as a deterrent, if the objective is to store very large quantities of surveillance data, for a long period of time, and to be able to access it easily and relatively quickly, tape is the ideal solution for a number of reasons:

1. Cost - LTO tape is far less costly to own and operate (it has a lower long term TCO) than either disk or cloud. 
2. Scalability - LTO tape can easily scale to petabytes and even exabytes very cost effectively.
3. Open Standard - LTO is an open tape format with a proven track record of innovation and compatibility, making it easy to store your footage on LTO media today and migrate it to newer, denser, cartridges in the future.
4. Durability and Reliability - LTO has a 30-year shelf life so you can depend on being able to recover data for decades to come.
5. Chain of Custody - Thanks to LTO encryption and WORM technology, LTO data cartridges provide excellent evidential integrity for the legal process. And because tape can be kept offline, away from connected networks, it means that footage is secure against the threat of ransomware and cyber extortion.
6. Reliability - LTO cartridges are extremely reliable and possess a better ‘bit error rate’ than hard disks, both Enterprise-class drives as well as the cheaper disks often used for object storage devices.
7. Convenience - a multi-tiered video retention system provides an easy to use and operate solution. Modern surveillance software makes it seamless to combine flash, disk and tape in a hierarchical system that allows you to implement LTO tape for the most economical long-term retention. Specialised Video Management Software (VMS) like Cozaint’s askAlice solution allows video operators to easily access all of their recorded video with most recently recorded video on hard-disk based storage and all of the surveillance video stored on LTO storage. No additional I.T. staff involvement or extra steps needed for the operator.
8. High Performance - there is a general misconception that tape is slow when it comes to data access, but in reality, this depends on the application and the data that is being sought. For accessing small files at random, tape not an optimal solution. But for accessing large, contiguous data sets - e.g. all the footage from last Wednesday afternoon between 1500 and 1700 - a tape drive will recover data much more quickly than a typical, 7200 rpm hard drive. In fact, LTO-8, with its 360 MB/sec native transfer speed offers comparable read transfer speeds to a consumer grade SSD drive.
9. Environmentally Friendly - tape systems require far less power and cooling than disk equivalents because cartridges at rest are effectively inert and do not need a highly managed environment for long term storage.
10. Future Proofed - tape capacity road map surpasses disk in terms of being able to match the storage challenges created by higher resolution video technology.

Calculating the cost of different approaches

To help customers evaluate LTO tape for the specific application of storing video surveillance, the analyst Brad Johns has developed a Video Surveillance Total Cost of Ownership tool. This excellent resource allows users to enter variables such as the number of cameras, the amount of footage and its growth rate, the frame rate being used, as well as the resolution being captured in order to calculate a detailed economic comparison between different storage technologies like tape and disk.  

From this analysis, it is clear that LTO technology can play a critical role in helping businesses manage the challenge of finding enough economic storage capacity to be able to maximise their investments in higher resolution and more sophisticated video surveillance technology.