Video Mining Is Blowing Up

April 30, 2009 SSI Staff

One of the early benefits that became apparent at the dawn of the DVR age was the ability to apply some sort of (limited) intelligence to the video system itself.

Previously, at the front end, recording was done either continuously or via alarm contact, as that’s all a VCR could do. When it was time for that video to be watched, there wasn’t much you could do other than hit the fast-forward button, sit and wait.

When the DVR hit the streets, it was like sliced bread had just been invented. You could schedule recording, you could jump immediately to a date and time for playback; life was good.

Things got even better as the technology matured. You now had things like thumbnail searching and bump-on-alarm. You could even select an area on a camera view and tell the machine to search for any motion in that area. We were in heaven.

Even in the short time between then and now, IP video has come a long way. Advances are being seen on a regular basis that make video systems more manageable, more valuable. The improvements and increased value we see now can be summed up in two words: intelligent video. Let’s see just how far we’ve come, and take a look at some areas that hold even more promise for the future.

Going Beyond Motion Detection

One of those early benefits was the ability to detect motion from within the system itself, without any outside sensor. This simple motion, or activity, detection was a big deal. It solved a major problem of wasting finite hard drive resources.

Video motion detection is a fairly simple process, simple being a relative term of course. At its most basic, motion detection involves looking at one frame of video, and then a second, and comparing the pixels in each image to see what has changed. If a sufficient amount of pixels change, you can trigger an alarm or start recording, etc.

When you compare straight alarm contact recording with recording based on internal motion detection, it seems like a very intelligent process. However, if you want to apply more intelligence, such as discerning between different types of motion, simple motion detection soon shows its weaknesses.

Fortunately for us, the optical science types were not resting after they developed video motion detection. They immediately set out to create more intelligent detection processes, or algorithms. These advanced motion detection algorithms have actually been around since the very first days of digital video. Unfortunately, back then, they existed only in the realm of the large government contractor or science lab and were rarely seen in the commercial marketplace.

Even back then the benefit of being able to detect specific behaviors, not just plain motion, was evident. Early behavior analytics (as they came to be known) included object counting, loitering, object left behind and even spill detection. Each of these behaviors offered a new level of smarts in a video system.

For example, if you’re watching a piece of property on a desk, such as a laptop, plain motion detection could indeed alarm if someone walked up to that desk. But what if that person never touched the laptop? That would be a wasted false alarm. More advanced analytics, however, would ignore the person simply walking past the desk, and only alarm if the laptop itself was moved. Again, the system itself is intelligent, making decisions without the input of people.

Video behavior analytics have drastically increased the intelligence and value we can offer to the customer with video systems these days. Newer analytics, such as a virtual tripwire and speed/direction algorithms, can in some cases replace physical sensors. If trenching or cabling is an obstacle to mounting an outdoor PIR for instance, a camera and digital video system can be used instead. Although there are some instances where cameras may fall short in these situations, such as when obstacles or inclement weather is present, it can certainly provide a valid alternative.

[IMAGE]11867[/IMAGE]

Analytics: the Next Generation

The future of video analytic algorithms is certainly bright. Imagine doing a search for an incident caught on video. Imagine being able to enter criteria like, “show me every time a blue van traveled westbound through the lot at less than 30 mph,” and getting a list of video clips with just that activity in them. Or how about being able to tell a video system to find every 4-foot-tall boy wearing a red shirt with blond hair, and have your system pull up every camera with someone fitting that description in its view? This is what tomorrow holds.

These are the kinds of things that, just a few years ago, we had to say were the realm of TV shows and movies. I like to call it the “CSI factor.” Technology is changing so much and so fast, that we have just about reached that CSI factor in real life.

Two main things have happened to make these situations a reality. One is the more advanced video analytic algorithms that can now detect colors, shapes and sizes (as well as speed and direction). The second is a little thing called metadata.

Metadata Makes Systems Smarter

Metadata is essentially data about other data. It is information, normally not read by people, locked inside digital files, like images. Metadata is used all the time in consumer digital photo cameras. Simple metadata can contain information such as date and time, camera manufacturer, and in some more high-end cameras, F-stop and shutter speeds.

In our world, metadata can hold time and date, manufacturer name, and things like the colors, shapes, speed and direction of objects in the scene. When each image is captured, the video management system (VMS) can extract this data from each frame of video and store it in a database. This database is then used to provide the search results for the situations I mentioned above. Each entry in the database is tagged (associated) with a certain clip or frame of video. It’s all very slick stuff indeed.

If you enjoyed this article and want to receive more valuable industry content like this, click here to sign up for our FREE digital newsletters!

Pages 1 of 2 Next