As video surveillance systems assume an evermore-prominent role in the protection of citizens, assets and critical infrastructure, the market growth alone poses an institutional quandary. Simply, the number of security cameras being deployed is exceeding the capacity for humans to monitor them economically.
Intelligent video surveillance (IVS) offers a high-tech solution to this mushrooming dilemma by using complex mathematical software algorithms that continuously monitor real-time surveillance information gathered by CCTV cameras.
The algorithms analyze video data for changes in pixels to trigger alerts that enable security personnel to identify potential threats. When IVS detects a suspicious event, an alert can prompt a security professional to investigate the video footage and validate the alarm.
Proponents of IVS say the technology reduces the need and cost for personnel to sit monotonously staring at banks of monitors for long stretches of time … a tedious undertaking that can rapidly render a person ineffective. With IVS, security professionals can instead be used more effectively to evaluate and respond to verified events.
In the past few years, the burgeoning surveillance technology has stimulated a convergence of video surveillance systems with IP-based networks and enterprise systems.
“The ability to derive actionable intelligence — mission-critical information delivered to the right people in time for an immediate and effective response — is the promise of IP-based video management solutions,” says Mariann McDonagh, vice president of Global Marketing for Denver-based Verint Systems Inc., a provider of IP-based video management and analytics software. “Video analytics is the application that delivers on this promise by autonomously sifting through the vast amounts of video data and identifying the most imminent threats before they escalate.”
While the potential benefits of IVS are numerous, systems integrators must negotiate misconceptions in the marketplace about deploying video analytics on IP networks, and other nagging challenges. Keeping pace with technological advances in cameras and other devices used on the edge of networks will be mandatory for security professionals to remain a viable resource for expertise and business applications.
IT, Physical Security Departments Must Form Working Partnership
The migration from conventional video management systems to performance surveillance and IVS can be accomplished smoothly. Nevertheless, it does require ample planning between heretofore disparate departmental groups. A key component found among successful early adopters of IVS is the mutual understanding between IT and physical security departments that surveillance applications need to be network-based.
Having a clearly stated goal of moving all surveillance equipment onto the network allows new infrastructure to be prewired with Cat-5e or Cat-6 cabling, thereby permitting adequate bandwidth for high-quality surveillance images.
Because IP-based intelligent video can be a huge burden on network bandwidth, the issue has received serious consideration during the past year. Many intelligent video vendors have been working with IT companies such as Microsoft, Cisco, IBM and Motorola to optimize bandwidth usage.
Among the improvements, some vendors have developed algorithms to send an XML stream of data during times when no event occurs. Hence, video is streamed across the network only when an alert is triggered. This reduces the bandwidth usage and eliminates the problem of setting up a new network to integrate an intelligent video solution.
The introduction of intelligent edge devices has also enabled the migration of video analytics software from centralized locations to devices such as cameras, DVRs and encoders/video servers.
“One of the key barriers to widespread adoption of IP-based video management technologies is the impact that the solutions have on IT network bandwidth and server processing power,” McDonagh says. “By pushing analytics to the network edge with intelligent IP cameras and video encoders, vendors are reducing the amount of server cycles and bandwidth consumption required to deliver actionable intelligence, without compromising accuracy.”
The Pursuit of Image Accuracy Is Pushing the Engineering Envelope
Although simple motion detection is widely deployed, the hype surrounding more advanced IVS has not been met with widespread commercial success. The main challenge facing this promising technology centers on conventional surveillance systems’ inability to provide the degree of image resolution necessary to perform detailed analysis.
Working with poor image quality dictates that current systems be extremely constrained in their application in order to avoid false positives, missed criminal identification and unreliable trigger events.
To achieve the superior image quality needed to support enhanced analytics, the industry has had to completely rethink how surveillance systems are designed. The biggest technical improvements to date have been on the front end, where almost every aspect of image gathering has improved. Following below, several of these important developments are detailed.
Megapixel cameras — It is now feasible to deploy multimegapixel cameras that use progressive scan, scientific-grade CCD (charge-coupled device) sensors that deliver true 14-bit dynamic range up to 1,100 nanometers (nm). Extremely sensitive across the entire visible spectrum and into near infrared wavelengths, these latest surveillance cameras see beyond what the human eye can.
One dimension for measuring the performance of a surveillance camera is its number of pixels. In the past, to monitor large public spaces it was necessary to use conventional mechanical pan/tilt/zoom (p/t/z) cameras to sweep the area under surveillance. The problem with this method is that when cameras move, the background in the image moves as well, making it difficult to apply analytics.
Multimegapixel cameras, however, can remain fixed. Details are extracted by using digital p/t/z, making real-time analysis by IVS much more accurate. Digital p/t/z offers another significant benefit: post incident surveillance footage can be searched with IVS, which is extremely difficult on conventional p/t/z footage.
For applications such as facial or license plate recognition, the number of pixels directly affects the performance of the system.
Visual resolution vs. spatial resolution — When comparing surveillance cameras it is important to understand the difference between spatial resolution, which measures the number of pixels, and visual resolution, which measures the sharpness of details in the image. Two images that have the same number of pixels can have widely varying visual resolution.
Performance surveillance cameras deliver a sharp image; images from conventional surveillance cameras appear “soft” by comparison. Sharp images are much more useful to ensure effective IVS.
Better dynamic range — The human eye can produce a dynamic range close to 8 bits and sees only to 700nm. A 14-bit performance surveillance camera sees well beyond human vision (it can actually see 64 times more dynamic range) and is far more sensitive than the best NTSC-based traditional camera, which provides about 8-bit dynamic range.
Information of such high quality reduces many of the lighting constraints that currently restrict IVS applications. The images on page 76 demonstrate the added information contained in high dynamic range images provided by performance surveillance
cameras.
Higher quantum efficiency – Scientific- grade sensors can also provide higher quantum efficiencies (QE) than their consumer-grade counterparts. The consumer-grade sensor found in a typical surveillance system has a QE of about 15 percent, while scientific-grade sensors have a QE of up to 60 percent. This means that scientific sensors capture up to three times more information than conventional surveillance cameras that use consumer-grade sensors. This characteristic of performance surveillance cameras allows IVS to run efficiently in a much broader range of lighting conditions.
Progressive Scanning Improves Detection of Moving Objects
Scientific-grade sensors use progressive scanning to expose all the pixels on a sensor at the same time. This allows the camera to capture very sharp images even when an object in a scene is moving.
However, most sensors currently used in surveillance are interlaced. This means the images are actually composed of two image fields taken at different times. As a result, any moving objects in a scene captured with a camera using interlacing will have edges that resemble the teeth of a comb.
When an IVS system is processing images produced by sensors that use interlacing, edge detection becomes very difficult on moving objects. As a result of the compromised edge detection, problems can occur in not being able to properly detect object motion direction. Plus, it is difficult to discern between overlapped objects.
Another problem that arises from interlacing is a basic inability to provide the detail necessary to perform biometric recognition analytics without severe constraints placed on the motion permitted by the subjects.
Compression Techniques, Edge Ware Used to Combat False Alerts
Since the introduction of the technology, intelligent video solutions have shown a high incidence of false positives. The main culprit for the high rate: analytics performed on lossy compressed video.
Video compression usually refers to the use of software to remove adjacent pixels in an image, reducing the color space to the most common colors. When such a method is used, the change in pixilation is dramatic from one frame to another.
Most lossy compression algorithms (for example, MPEG-1, MPEG-2, MPEG- 4 and H.263) were designed for the entertainment and video conferencing industries. Since analytics is performed on pixel changes, lossy compression significantly reduces the usefulness of surveillance images for highly advanced IVS systems.
Outdoor environments have proven especially difficult to apply video analytics because of inherent changes in pixilation due to wind, rain and other factors, says Dieter Kondek, president and CEO of New York-based Agent
Video Intelligence Inc., a provider of enterprise-grade video analytics. “Vendors are learning how to apply multiple algorithms and sophisticated filters to compensate for changing weather conditions,” he says.
In the battle to reduce false positives, video intelligence vendors have been experimenting with lossless compression techniques and moving analytics to the edge of the network. Although lossless compression will preserve original image data, the caveat is that it requires higher bandwidth and more hard drive space. But the superior images make the job of IVS easier by providing more accurate information to analyze.
“The trade-off in pushing image processing to the edge is a sacrifice in functionality when the processing power can’t handle the heavy load,” Kondek says. “There are advantages to both edge processing and a centralized architecture. The trick is to find a way to tap the best of both worlds.”
As example of the latest compression techniques, some vendors are moving to the H.264 or MPEG-10 format and working toward establishing it as the standard in the video surveillance market. Expected to be relatively lossless, MPEG-10 is projected to reduce video bit rates to nearly half of other formats, such as MPEG-4 and MPEG-2.
‘Storytelling’Vs. Identification Cameras Can Work in Tandem
In deciding which specific surveyed areas to upgrade first, systems integrators are differentiating between “storytelling” cameras and identification cameras.
Storytelling cameras are generally analog or Internet protocol (IP)-based with lossy compression, which are able to deliver image quality sufficient for security personnel to distinguish what is happening in a scene.
Identification cameras, in contrast, are generally part of a performance surveillance system with megapixel cameras and a recorder that uses lossless compression. Such a solution has the ability to capture high-quality images that make identification of people possible.
In the retail environment, identification cameras are being deployed at entrances and exits, while storytelling cameras remain throughout the rest of the store. This allows a suspicious person to be tracked all through the store and then identified at the exit.
In corporate environments, facial recognition is being applied to the high-quality images captured by identification cameras deployed in lobby areas.
Multimegapixel Cameras Offer a Range of Possible Applications
Just as the number of pixels in consumer cameras increases, the resolution in surveillance cameras is rising as well. Most recently, 11-megapixel surveillance cameras have emerged, which have the equivalent resolution of 35 conventional cameras. For pubic safety applications, or in any setting where a large area needs to be monitored, these high megapixel cameras provide unparalleled image detail.
One or two carefully placed cameras can provide excellent coverage of several city blocks. Another effective use of performance surveillance and IVS is the adoption of rapid deployment kits (RDKs). These are portable systems that consist of a high-capacity recorder and a few megapixel cameras. Uses for such systems range from covert operations to monitoring suspects, to transit authorities who can temporarily deploy RDKs in trouble areas until the incident is resolved.
Integrating IVS With Conventional Surveillance Systems Is Possible
The higher image quality required for IVS usually cannot be supported by conventional surveillance solutions.
This necessitates the ability to integrate legacy systems with the new performance systems.
The simplest and most effective way of integration is to have the performance system record high-quality images locally and at the same time serve a surveillance analog stream to the legacy system. This approach allows the high performance cameras to record at high resolution, but also to be monitored by the existing video management system.
If an incident occurs, the high quality images can then be retrieved from the performance recorder either directly or downloaded to the network via a Web interface. With this approach, the benefits of performance surveillance can be deployed to complement any existing surveillance system in an incremental manner.
A video analytics solution must be “future ready” and scale to meet an organization’s changing security needs over time, without significant incremental costs, says McDonagh. By deploying analytics at the network edge, organizations can simply and cost-effectively extend analytics across virtually every camera in the enterprise, allowing them to intelligently scale infrastructure.
