The Million-Dollar Blind Spot AI is Helping Toll Agencies See: Convolutional Neural Networks & Applications to Tolling

A picture may be worth a thousand words, but a CNN can make it worth a million insights – or a million dollars when it comes to electronic toll collection. ?As toll agencies across the country explore innovative ways to reduce revenue leakage and improve operational efficiency, one promising technology is the Convolutional Neural Network (CNN), an artificial intelligence system that excels at identifying patterns and objects in visual data.

Introduction to CNNs

A Convolutional Neural Network is an artificial intelligence system inspired by how the human brain's visual cortex processes information. ?Unlike traditional programming methods that require manual definition of object characteristics, CNNs "learn" to recognize patterns from numerous examples.

Imagine teaching a computer to recognize different types of vehicles in photographs or video footage. With traditional methods, you'd have to manually define what each vehicle type looks like by specifying rules about their shapes, sizes, and features – an extremely difficult and tedious task.

CNNs take a different approach. They "learn" to recognize vehicle types from being shown many labeled examples of cars, trucks, and other vehicles, as well as non-vehicle objects. This learning process involves a series of filtering operations that allow the CNN to automatically detect low-level visual features like edges, curves, and shapes. These are then combined into higher-level concepts like wheels and windshields, until the CNN can reliably identify full vehicle patterns.

The "convolutional" part refers to how the CNN slides these filters across the entire image, blending or 'convolving' the filter values at every position to build its understanding of the image features and contents.? What makes CNNs so powerful is that once trained on enough visual examples, they can recognize vehicles in completely new images or video frames, accounting for variations in angle, lighting, or partial obstructions – tasks that are simple for humans but extremely difficult to codify into rules.

Historical Context

The concept of CNNs emerged from pioneering work on neural networks in the late 1980s and early 1990s. Key developments include:

• 1979: Kunihiko Fukushima proposes the "neocognitron," a hierarchical, multilayered artificial neural network used for pattern recognition tasks.
• 1989: Yann LeCun and colleagues publish "Handwritten Digit Recognition with a Back-Propagation Network," considered one of the earliest working applications of convolutional nets.
• Late 1990s: LeCun's team publishes "Gradient-Based Learning Applied to Document Recognition," which helped popularize CNN architecture and training procedures.
• Early 2000s: The term "convolutional neural network" gains widespread use as neural network architecture gains traction in computer vision and pattern recognition tasks.

AI as an Accelerant

Artificial intelligence has played a crucial role in advancing CNN capabilities, acting as a powerful accelerant in several key areas.

1. In the realm of network architecture, researchers have made significant strides in developing increasingly sophisticated models. These advancements have led to the creation of extremely deep CNN models with over 100 layers, enabling the learning of more complex visual concepts and patterns. AI methods such as reinforcement learning and evolutionary algorithms have been instrumental in this process, automatically searching through vast possibilities to discover optimal CNN designs. This AI-driven approach has resulted in more efficient and scalable models, pushing the boundaries of what's possible in computer vision.
2. The training process for CNNs has also been revolutionized by AI techniques. Self-supervised learning, for instance, has opened new possibilities by leveraging the vast amounts of unlabeled data available in the world. This approach allows for effective pre-training of powerful CNN backbones, giving them a strong foundation before fine-tuning for specific tasks. Transfer learning, another AI concept, has proven invaluable in boosting performance across various computer vision tasks by allowing models trained for one purpose to be repurposed for another, saving time and computational resources.
3. Perhaps one of the most significant contributions of AI has been in improving the reliability and interpretability of CNNs. Techniques from the field of explainable AI now allow us to peer into the inner workings of these complex systems. For example, saliency maps can highlight specific parts of an image that the CNN focused on when making a prediction, providing insight into its decision-making process.
4. Bayesian neural network methods have introduced a crucial element of uncertainty quantification. Unlike traditional neural networks that provide a single, definitive output, Bayesian approaches acknowledge the inherent uncertainty in predictions. Named after Thomas Bayes, an 18th-century statistician, these methods apply probabilistic reasoning to neural networks. In practice, this means that instead of a CNN simply identifying a vehicle as a "truck," for instance, a Bayesian CNN might say it's 80% certain it's a truck, 15% certain it's a van, and 5% uncertain. This nuanced approach is particularly valuable in critical applications like automated toll systems or self-driving cars, where understanding the model's confidence level can inform decision-making processes and trigger human intervention when necessary.

Applications in Toll Collection

CNNs offer several compelling benefits for toll agencies, which can be broadly categorized into two groups: applications that address revenue leakage and those that facilitate traffic management.

Applications Addressing Revenue Leakage:

1. Improved Automatic License Plate Recognition (ALPR): CNNs significantly enhance ALPR accuracy, especially in poor visibility conditions, reducing those missed toll transactions. For instance, during heavy rainfall or when glare from the sun obscures part of a license plate, traditional ALPR systems might fail. However, CNNs can be trained to recognize partial plate information or adjust for various lighting conditions, greatly improving read rates.
2. Automatic Vehicle Classification: By accurately categorizing vehicles, CNNs ensure proper toll rates are applied and HOV lane rules are enforced. For example, a CNN could distinguish between a standard passenger car, a small commercial van, and a large semi-truck, applying different toll rates accordingly. It could also identify motorcycles or high-occupancy vehicles for special lane access or discounted rates.
3. Anomaly Detection: CNNs can identify suspicious behavior, such as vehicles attempting to evade toll booths, enhancing enforcement efforts. For example, a CNN might detect a vehicle suddenly changing lanes to avoid a toll gantry or identify a truck trying to pass through a car-only lane to pay a lower toll. These anomalies can be flagged for further investigation or immediate action.

?Applications Improving Traffic Management:

1. Real-time Traffic Monitoring: By analyzing video streams, CNNs can provide valuable data on traffic patterns, helping optimize toll pricing and lane management. For instance, a CNN could detect the onset of congestion by observing changes in vehicle spacing and speeds. This information could be used to dynamically adjust toll rates or open additional lanes during peak hours.
2. Integration with Existing Technologies: CNNs can complement RFID, GPS, or V2X systems, providing visual verification and enhancing overall system reliability. For example, if an RFID tag fails to read, the CNN-powered ALPR system could serve as a backup, ensuring the vehicle is still correctly identified and charged. Or, in a free-flow tolling system, CNNs could verify that the number of axles detected by in-road sensors matches the visual classification of the vehicle, reducing errors in toll calculations.
3. Safety Enhancement: CNNs can be used to detect and alert operators to potential safety hazards on toll roads. For instance, a CNN could identify debris on the road, a stopped vehicle in a dangerous location, or a driver traveling in the wrong direction. This real-time information allows for quick response to potential accidents or traffic disruptions.
4. Customer Service Improvement: By providing more accurate and detailed information about vehicles as they pass toll points, CNNs can help resolve customer disputes more efficiently. For example, if a customer contests a toll charge, the system could quickly retrieve a clear image of the vehicle and its license plate, along with a classification of the vehicle type, providing solid evidence for the charge.

These applications show how CNNs can significantly advance a variety of ETC operational objectives beyond revenue assurance.? By leveraging this AI-powered solution, CNN’s can help agencies operate more efficient, accurate and customer-friendly toll systems.

Moving Forward: Implementing CNN Solutions

"The difficulty lies not so much in developing new ideas as in escaping from old ones."

?- John Maynard Keynes

This observation by the renowned economist aptly describes the challenge many toll agencies face when considering deploying a cutting-edge technology like CNN. ?While the potential benefits look clear, the path to adoption can be complex, especially in a traditionally conservative organization like a toll agency. Implementing CNN solutions in toll operations thus requires a thoughtful, strategic approach.

The process begins with comprehensive data collection and preparation. This crucial first step involves gathering diverse datasets that cover a wide range of environmental conditions and vehicle types. The quality and variety of this data will directly impact the robustness and effectiveness of the CNN models. Agencies should consider partnering with other organizations or using synthetic data generation techniques to ensure they have a sufficiently large and diverse dataset.

Next, a thorough infrastructure assessment is necessary. This involves evaluating existing camera systems and other hardware to determine if upgrades are needed to support CNN-based solutions. High-resolution cameras with good low-light performance may be required for optimal results. Additionally, agencies should consider the computational resources needed to run CNN models in real-time, which might necessitate investments in edge computing devices or cloud infrastructure.

Integration planning is another critical step. This involves developing a comprehensive strategy for how CNN outputs will be incorporated into existing tolling and traffic management systems. It's not just about technical integration; it also involves retraining staff, updating operational procedures, and possibly redesigning user interfaces to make the best use of the new AI-driven insights.

Privacy and security considerations must be at the forefront of any CNN implementation. This goes beyond just securing the systems against potential cyber threats. It involves carefully considering what data is collected, how it's stored, and how it's used. Agencies should work with legal experts to ensure compliance with relevant privacy laws and to develop transparent policies that maintain public trust.

Finally, agencies must establish a robust system for performance monitoring and continuous improvement. This involves defining clear metrics to track the effectiveness of CNN implementations, such as improvements in the precision of toll rates or reductions in congestion or incidents. ?You should conduct regular audits of the CNN models' performance, with processes in place to retrain or fine-tune the models based on real-world performance data. ?This iterative approach ensures that your CNN solutions continue to deliver value over time, while adapting to changing conditions and improving with more data.

Conclusion

As CNN capabilities continue to advance, they will become an essential tool for toll agencies, helping operators 'see' everything they need to offer financially sustainable, efficient, and user-friendly toll roads and managed lanes. From enhancing ALPR accuracy to enabling vehicle classification and anomaly detection, CNNs can create more intelligent, data-driven systems that optimize pricing, reduce congestion, and improve overall performance.?By embracing this technology, agencies can significantly reduce revenue leakage, improve operational efficiency, and enhance the overall user experience. The future of tolling is here, and it's powered by artificial intelligence. Are you ready to open your eyes to the possibilities?

Author’s FYI: I was reading the latest ITS International magazine and Rafael Hernandez, MBA article, “What’s Right with this Picture” caught my eye.? In it, Rafael provides a balanced look at applying AI to improve license plate image review and he astutely summarized the key elements as being technology, policy, and people. ?Like the legs of a sturdy stool, each is essential for stability and function, and effective AI-assisted image review requires the balanced integration of cutting-edge tech, well-crafted policies, and skilled human oversight. ?I wrote my article some months ago, so thank you, Rafael – yours inspired me to post it.