Defining video analytics
Go on; what is it? What is video analytics? It’s a trickier question to answer than you might first think.
Technically; video analytics solutions enable us to programmatically detect predefined objects and actions within frames of video footage. These software systems – that leverage computer vision technology, which is powered by a subset of Artificial Intelligence (AI) known as machine learning – generate metadata about those detections. That metadata can trigger near-real-time alerts and communications, or be aggregated and analyzed to reveal patterns and trends.
So, think of automatic people counting, as an example. You might set a zone or virtual line, then configure an alert to notify you whenever someone enters that area or crosses that line. You may then wish to analyze the number of people entering or existing that zone over the course of a week.
Sciencedirect.com offers-up two referenced definitions of video analytics technology:
- “Video analytics extend security by turning all the cameras designated for the area of coverage into trained observers”
(Digital Video Surveillance and Security (Second Edition), 2014) - “Video analytics is a technology that processes a digital video signal using a special algorithm to perform a security-related function”
(Effective Physical Security (Fifth Edition), 2017)
However, the concept of extracting analyzable information from video content isn’t something that’s suddenly emerged with the advent of AI. So what’s the difference between traditional technology-enabled video analysis and the modern AI-powered video analytics that’s emerging today?
Luckily, John Bigelow, Editor at Security Solutions Media, interviewed a panel about this very topic recently:
Defining video analytics: Don’t kick the robot
John Bigelow, Editor at Security Solutions Media
So when we talk about Artificial Intelligence, we’re obviously not talking about Artificial General Intelligence because there’s a couple of different classifications of AI here.
John Bigelow
Artificial General Intelligence is the all omnipotent intelligence that can sort out and solve pretty much any problem. Point of fact, which, by the way, don’t go kicking robots, because in 20 years time when they’re our overlords, and we’re all wired-up as ever-ready batteries, that’s going to come back really badly to bite us in the arse.
John Bigelow
But when we’re talking about Artificial Intelligence, we’re talking about specific Artificial Intelligence. And as we all rush into the world of Artificial Intelligence, there’s a lot of AI washing that goes on. And we saw this a number of years ago with environmental or ‘green washing’ around environmentally friendly stuff. So I guess, Michael, I’m going to start with you — because you seem the most logical place to start, because you’re building the chips that do this stuff — how do we determine what genuinely is AI, versus just really advanced video analytics?
Michael Lang, Solutions Architect Manager at NVIDIA
Yeah; sometimes those two things can be very hard to distinguish.
Michael Lang
So I would say that the proper AI play is fundamentally one that’s in something that you don’t really think a machine can do. So again, whether that be, trying to identify something really unique… it could be in the medical space [in regards to] looking for cancer, or [identifying] invasive species after a forest fire. It’s something that traditionally wouldn’t have signed the capacity for machines to be able to do: That’s when you know you fall into the world of Artificial Intelligence.
Michael Lang
And there’s been a lot of change in that space quite, quite rapidly. Sometimes due to just the scale of the past couple of years. As of last year, the new big thing is Generative AI, which is not so relevant here today, but there’s a lot of new things that are coming. But I said, I think something that we would not have traditionally ascribed a machine to be able to do, that seems even a little bit magical… [is where AI comes in]. But, in fact, there’s a lot of maths behind that. And we work with the software as well, making that happen. And those things are again, things that make a difference, because it frees people at the end of the day.
Defining video analytics: From pixels to object recognition
John Bigelow
Now, Aaron, you’ve spent a significant part of your life in the last couple of years developing analytics for video systems. And so you’ve probably got a very interesting point of view, from the perspective of facial recognition as an analytics tool, [as to] what’s going on now [regarding] where that line lies between analytics and artificially intelligent video.
Aaron Terrey, Director of Vixles Pty Ltd
Yep; sure do. I was with a retailer yesterday, in the exhibition hall, and we were joking that six years ago we were trying to understand dwell-time inside a retail environment. So customer engagement: Understanding what product they were engaged in, how long they were there. And then obviously, the ultimate goal is to translate that back into transaction amounts. So it’s a combination of people count, dwell-time and transactional [data]. So are you converting the people that are coming in into a sale? Anyway, we joked that six years ago, the dwell part of it was actually quite difficult. And the partner we were working with — Pat, this may horrify you — we had laser lights out, and we were trying to measure the floor. And it was just incredibly complex to try and get right. And a lot of that was because it was using older technologies. So it was trying to understand pixels and pixels that were changing within the scene.
Aaron Terrey
You look at the people count side of things, and you know, we’ve really moved from what were originally top-down cameras trying to cross the line, and just seeing how somebody would move through the scene and sort of effectively trigger a line. Now, we’re actually looking to understand that there’s a human being within the scene. So you start avoiding counting prams, and shopping trolleys, and things coming through the line, and actually understand that it’s a lot more accurate.
Aaron Terrey
So fast forward from six years ago to certainly today, to put this technology in by using the AI component, rather than effectively pixels changing within a scene, your accuracy is just significantly higher. And with that, you know, we can start relying a lot more on this technology to help us with business decisions. So, if we’re trying to — Pat may talk to it soon — but we’ve been working on a project where there’s significant business decisions in terms of monetary value, especially with councils, where you’re using computer vision and analytics to really understand the value of whether a road should be changed. More cycle lanes put in, car parks taken in or out. And if you’re making significant decisions around that, you really need to understand the accuracy. And, really be confident that the accuracy is high. And the change between the pixels moving around the screen, and AI with the object detection being highly accurate, is significant.
Aaron Terrey
So just quickly on the face rec side of things, the change again over six years… We put one of our first facial recognition systems into a retailer about six years ago. Nobody really cared so much back then about the biometrics because the press wasn’t really reporting on it in a negative way like they are now. But the technology now — between what effectively was six years ago, with the generation of AI coming into facial recognition — has just improved the accuracy significantly. [In the past], we would consider — in an unstructured environment, in a retail environment — painting the floor white, so the reflection would come up and actually give a nice, clear view of your face. Bearing in mind, in a retail environment, you can’t stand at a Customs Hall and have the light shine in your face and [direct people to] take your hat off. We’re talking about a really unstructured environment.
Aaron Terrey
So fast forward through to what we’re doing today, the accuracy is just so much higher — so we can trust the results.
Defining video analytics: Maybe it’s simply ‘accelerated intelligence’
John Bigelow
Yep. And, Patrick, I’ll go to you with the same question. Because obviously, you’re not seeing it here at the show downstairs. But I was at ISC West earlier this year in Vegas and, walking around the show floor there, the number of — you can only refer to it as ‘AI BS’ — because the number of solutions that people [sighs]… I saw one guy trying to flog an Artificially Intelligent doorbell. I’m pretty sure if you’re buying that then the AI doesn’t belong in the doorbell, it needs to be somewhere else.
John Bigelow
You spend a significant amount of time developing all sorts of things, facial recognition not being one of them, so how do you determine the difference between — in your opinion — just really advanced analytics and genuine AI?
Patrick Elliott, CEO and Cofounder at VisualCortex
Yeah, and there’s really a fine line, right? I think it’s actually progressive in the sense that.. you become smarter by taking in information and you analyze it, and then you change the way you think about things, and then you do something different.
Patrick Elliott
When we started our company, I had the premise that we would never expect a computer vision model to do better than what a human could see. And then over the last year and a bit, it’s totally changed. Because we were able to see people that were 300 meters away, and see that it was a lady with a pram. You can never see [that] with the naked eye. So it’s something that you couldn’t do as a human, right? But it’s kind of accelerating humanity or extending humanity.
Patrick Elliott
The flip side of that, what I found extremely strange with that video with the robot soldiers… It just blew my mind. Because if you had a chance from zero, to build the ultimate droid [to be a] robotic warrior, they probably wouldn’t look like a human, right? So I think we’re still limited by our own intelligence and our own limits that we put on ourselves. And I think when we embrace, and we open up, we really take those learnings, a lot of it’s around ground truth, video is great, because it’s a ground truth, you always go back to empirical, that’s exactly what happened. But when we take that, and then we extend that with analytics, that’s when we start really getting into the intelligence side — whether it’s artificial or accelerated. Maybe it’s more accelerated intelligence [as opposed to Artificial Intelligence, when it comes to computer vision]. That’s when we start really getting into new paradigms and seeing new opportunities.
Patrick Elliott
And just on facial recognition … as a stakeholder in the [VisualCortex] business, we’ve elected not to do facial recognition. But as a parent, and as somebody who employs people, there are very many good cases where you would want to do it to protect your people and your assets, right? I know it’s super controversial, and I’m split as a human on it myself.
Where to next?
Keen to hear more? Check out the full panel discussion – Using Artificial Intelligence to solve real-world customer challenges – here: https://visualcortex.com/2023/11/29/solving-real-world-challenges-with-video-analytics-software/
About VisualCortex
VisualCortex is making video data actionable in the enterprise. Its Video Intelligence Platform provides the stability and flexibility to productionize computer vision technology at scale. Able to be used for any video analytics use case in any industry, VisualCortex’s production-ready cloud-based environment transforms video assets into analyzable streams of data.
The VisualCortex platform delivers the artificial intelligence smarts, governance and usability, enabling organizations to connect any number of video streams, repositories and use existing commodity hardware. An intuitive user interface, out-of-the-box reporting, range of configurations and integrations empower non-technical people to produce, analyze and act on insights derived from computer vision throughout the enterprise. Organizations can easily combine these AI-generated video insights with other data sources and systems to facilitate both real-time operations and strategic analysis. The VisualCortex Model Store also provides a secure marketplace for customers, partners and independent machine learning experts to share quality controlled computer vision models.
For more information, visit www.visualcortex.com
For regular updates, follow VisualCortex on Twitter (@VisualCortexApp), LinkedIn (VisualCortex), YouTube (VisualCortex) and Facebook (@VisualCortexApp).
For regular industry news and analysis, subscribe to VisualCortex’s mailing list here: visualcortex.com/contact-us
