Video Analytics Re-imagined: The Impact of Powerful Edge AI

City Surveillance and Smart Cities

City surveillance is one of the largest end-user applications for video surveillance equipment globally, worth an estimated $4 billion in 2021. The market includes all public space within the remit of a city surveillance or public safety project.

Security cameras are used by police departments, and other government agencies, to monitor these open spaces. This can be through spot checks of key locations to ensure that there is no activity that would require a police response, or through regular “guard tours” to supplement police officers. In specific occasions, real-time monitoring of live events can also be done.

In recent years, the city surveillance market has evolved. The first integration was with communications technology and command and control (C&C) rooms to create safe city solutions. These systems connect video surveillance, analytics, entrance control, gunshot detection, and other sensor inputs into a command-and-control infrastructure. Data analytics can be layered into the system to create predictive crime centers capable of identifying patterns in behavior.

The next evolution is the Smart City. Building on the public safety aspect of safe city solutions, smart cities support mobility and transportation, governance, physical infrastructure, and energy efficiency. Video analytics solutions are well placed to provide traffic management solutions and parking violation identification as part of a broader Smart City.

Challenges with Current Solutions

Video analytics have been used in the city surveillance market to support the identification of vehicles or people, protect virtual perimeters and alert to loitering or other anti-social behavior. A key challenge for end-users has been ensuring that the alerts are accurate, timely and not repeated for the same event.

There is an example from an early installation in the UK where the local council disabled its city surveillance video analytics system because it was creating so many alerts that were false positives. It required more operators to manage these alerts than were initially staffed in the control room. The moral responsibility to respond to an alert meant that the city leaders could not continue to work with a solution where the accuracy to real events was not high.

Another challenge with existing analytics systems is the large geographic spread of security cameras around a city. In some cases, the cameras are networked using wireless infrastructure as cabling cannot be installed. Bandwidth is a real challenge, especially with the need to deploy higher megapixel cameras to meet future image quality requirements. High-end public safety analytics are often run on the server due to processing requirements, which means these high-resolution images must be sent over the network, adding cost and complexity.

Benefits of High-performance Edge AI Solutions

A critical benefit of high-performance edge AI analytics is that accurate analytics algorithms can be run on the camera, in real-time. This does two things in a city surveillance project. First, it means that any alerts generated to the control room are accurate and timely. Each network camera has the processing power required to provide strong meta-data points (such as alert type or object type) and generate alerts that are real and trusted. Trustworthy alerts provide more flexibility in operations and can save cost through more efficient use of resources or people. Analytics solutions that are not accurate will eventually be turned off by the end-user. Edge analytics are also timely as the processing capability is dedicated on the device.

Second, powerful edge AI solutions support better bandwidth management and reduce storage costs. City surveillance projects are characterized by security cameras located in remote locations around parks and streets.

Traffic management

Video analytics systems can be installed around a city to generate data on traffic flow, congestion, and vehicle numbers. This meta-data can be used to manage the roads more efficiently, ensuring traffic is re-routed where possible, and that related traffic controls consider the current road conditions. These types of solutions need accurate edge analytics on the camera.

Due to the remote locations of traffic management systems, there are bandwidth challenges in sending images back to a central location. A better solution is to run algorithms on the device and send back the information rather than the video images. However, scenes are often busy with hundreds of vehicles moving in and out of the field of view. High performance analytics are needed to manage the complexity of objects and activity alerts. Given the importance of an efficient traffic system, the metadata needs to be extremely accurate and low latency, so will benefit from increased processing power.

Traffic enforcement is another Smart City application well suited to using video analytics. Security cameras can monitor the movement of traffic for vehicles that go through a red light. Automatic number plate recognition (license plate recognition) can then identify the vehicle so the system can generate a ticket for the violation. The video surveillance images can be used to prove the event in the case of any legal dispute. Similar solutions can be used to manage city parking ensuring that vehicles that spend too long in a parking space are ticketed. As with traffic management solutions, there are significant bandwidth benefits by running these AI analytics on an edge device.

Commercial

Video surveillance has been deployed in commercial buildings for decades. Security cameras protect perimeters and internal locations in hotels, offices, and restaurants around the world. The market is an important revenue stream for the video surveillance industry, accounting for over 10% of annual equipment revenues.

Commercial buildings typically have many employees, contractors and visitors accessing the site daily. They also have perimeters that need to be secured but may not be so high risk to require full manned guarding or fencing. Virtual tripwires, loitering alerts, people tracking, and object identification analytics have all been used to try to meet the physical security challenges of a commercial security manager.

Challenges with Current Solutions

The traditional analytics market has some challenges in the commercial market. In some use-cases, basic analytics are deployed in relatively complex scenes. This could include hotel and office lobbies where large numbers of people are moving around. For less powerful analytics, the alerts generated can be difficult to interpret for the physical security personnel as they may be inaccurate or delayed due to processing issues. Like the city surveillance market, inaccurate analytics will eventually be turned off.

Benefits of High-performance Edge AI Solutions

In the commercial market, one of the key benefits of adopting high-performance edge analytics is the improvement in network taxation. This means there is less cost associated with networking, servers, storage, and cloud because the analytics are done on the camera. That said, the analytics need to be powerful enough to generate good metadata. This needs processing capability.

Commercial is another market where security requirements can change. Increased processing power, aligned with higher image quality, means security cameras can do more things at once. This could mean alerting to a perimeter breach while at the same time counting the number of people that are entering a building. This additional functionality could be used to create more efficient security solutions in the future.

Airport

Transportation is an important market for the video surveillance and video analytics industry. It is also considered one of the faster growing vertical markets. Airports were one of the earliest video analytics market applications. They have higher security requirements than most industries, as well as large perimeters that need to be protected, but are difficult to protect efficiently.

They also pose a challenge in the sense that the volume of people passing through airports is high and the system must be able to handle that within accuracy and cost constraints. The security system should be able to generate a real-time alert when necessary. This can be challenging as it is common to loiter in areas while waiting for a flight departure.

Challenges with Current Solutions

Large, remote perimeters and high risk, airside locations mean airports are difficult sites to protect. The addition of thousands of passengers, retail locations, the airside-land-side border and the number of employees, contractors, and flight staff with access adds to the complexity.

Some of the early video analytics solutions deployed in airports to protect the perimeter were not that successful. In one US airport, the analytics were turned off due to the regularity of alerts. In this scenario, the alerts were not false, they were alerting to the same animal that was just beyond the perimeter fence. However, the result was that the control room operators were constantly turning the alert off which became labor intensive and not labor saving. This highlights that accuracy and flexibility of the algorithm are extremely important.

Benefits of High-performance Edge AI Solutions

Airports need high quality analytics that are accurate and reliable. Terminal buildings have thousands of passengers passing through them daily. High performance analytics are needed to ensure that the volume of objects, such as passengers and luggage, can be detected and tracked when required.

There are also extremely complex analytics needed for specific applications, such as left luggage applications. Here, the algorithm must identify that the object has been left for a predefined period, generating an alert, while removing all other activity from the scene. This is complex and requires powerful processing capability. The cost of getting this wrong is either a dangerous object is left unchecked, or a terminal building is evacuated based on a false alarm. Neither outcome is positive for the airport security operations team.

In more traditional analytics applications, such as tripwire alerts, there are benefits from improving the accuracy of the algorithm. Airports are high security environments and any threat to a border is taken seriously. False alarms can generate armed responses which are expensive. Missed alerts are significant security breaches and there can be implications for national security.

As AI processing power increases on cameras, the solution accuracy for analytics, such as tripwire, will increase. These applications have been shown to have utility in the market. Improvements in the accuracy and latency of alerts will only be of benefit to end-users. Outside of the terminal, there are huge benefits in bandwidth management by running powerful analytics at the edge.

Retail (Business intelligence)

The retail industry has been using video surveillance for a long time. Originally, these systems fed video to a control room where security personnel watched for suspicious activity. Now, retail end users have embraced video analytics technology to support their loss prevention activities. These include monitoring point-of-sale (POS) activity for fraudulent or unauthorized activity or tracking potential perpetrators through a store.

While loss prevention is the primary use-case for video analytics in the retail market, there is an emerging technology opportunity for business intelligence and operational improvement solutions. Much of the same infrastructure can be used to support these business intelligence applications. The benefit for the retailer is in understanding shopping traffic patterns and customer demographics, measuring retail store trends, and increasing conversion and overall sales for the store.

Challenges with Current Solutions

One of the biggest challenges for current analytics solutions is the ability to run the multiple algorithms required to provide both loss prevention and business intelligence data. Most of the security cameras in retail were installed for loss prevention purposes. Therefore, the priority is this application. Retailers have a challenge in using existing cameras and infrastructure to support these potentially revenue generating applications in people counting, queue length management, customer heatmapping and stock management.

Benefits of High-performance Edge AI Solutions

Increasing the processing power on the camera is an important trend in the retail market. It means that better algorithms can be processed on the edge device and that more applications can be done on the same camera. This could be a mixture of loss prevention applications and business intelligence and optimization algorithms.

Existing solutions must prioritize an algorithm to run on the camera – which in retail means loss prevention. High performance edge AI solutions allow the camera to provide the same loss prevention capability, but with the addition of business intelligence analytics. Not only does this re use the existing infrastructure, but the higher processing capability means more accurate algorithms will generate better data across the different applications.

Furthermore, analytics can be used for different purposes at the same time. One example is where dwell time analytics have been deployed to both alert that a person is loitering near a product and to identify a loss prevention risk. In this solution, an alert can be generated for a member of staff to attend to the potential customer. In real deployments, this type of solution has increased sales while reducing theft. Many existing video analytics cameras in the retail market would not be able to take advantage of this approach because the camera has maximized its processing capability on a more traditional loss prevention algorithm.

This approach also supports the development of hybrid solutions. Here, powerful edge analytics can do some of the heavy lifting, such as analyzing faces or activity, before sending specific information or partial video images back to a central analytics system. This can make the overall solution quicker, more accurate or simply more efficient due to the distribution of processing capability.

Fundamentally, edge AI will take the retail video analytics market into the mainstream. It will ensure high performance analytics are on every camera, providing both loss prevention and business intelligence applications, concurrently, and in real time. It will simplify the installation and operation of these systems, ensuring loss prevention professionals can make best use of the technology. Finally, it will support the cost of deployment through the operational improvements possible from a single high performance camera providing security and business intelligence applications.