A technology company specializing in building digital twins and safer cities in Japan and South Korea is focused on making driving safer for both drivers and pedestrians. As a leading innovator in the automotive technology space, they specialize in designing advanced dash cameras with smart image capture and networking capabilities. These devices translate driver experience behind the wheel to aggregated, searchable knowledge describing up-to-date characteristics of roadway conditions and behaviors. They also collaborate with insurance companies to let drivers benefit from safer driving.
To do that, the company needs to create a network of alerts and insights as to what drivers are seeing in order to provide the information needed to create a real-time view of the conditions within and around a city.
The Challenge
The company’s goal is to develop a cloud service that aggregates information from individual dashcams to provide a holistic view of an environment, enabling drivers to coordinate their movements, resulting in safer roads. To make that happen, they needed to enable their dashcam products to identify a variety of traffic situations.
These situations included the full range of things a driver might see, from a traffic sign to an open parking space to traffic blocking their path. In addition, their goal was to provide this information in an easy-to-consume way, such as an overlay for Google Maps or OpenStreetMap's.
And all that information needed to be processed in real-time.
The Solution
Real-time object detection from dashcams, like those in autonomous vehicles, requires a chain of data and applications. CloudGeometry’s experience with complex environments, combined with its experience in AI and machine learning as well as mobile applications, made it perfect for this project.
Starting at the edge, CloudGeometry used frameworks such as Tensorflow Lite and Flutter to capture real-time video from dashcams. They created an app that performs object detection at the edge. This app then returns bounding boxes for detected objects. These objects included:
- Traffic signs
- Parking lots and spaces
- Accidents
- Traffic slowdowns
- Cones, barrels, and other signs of road construction
CloudGeometry built the software that transmitted these results over Bluetooth to the user’s device, integrating with the company’s main app for the full user experience.
All of this is important to users, but it is only part of the company’s mission. This data is also sent to centralized servers, where a new robust data pipeline enables it to be digested and aggregated so that the company can create a digital twin of a city. From there, they can advise drivers of conditions before they get there, enabling them not just to arrive sooner, but to drive safer. They can also provide additional services to the city and to third parties.
Key technologies that made the solution possible included GeoJSON, H3, OSM, Snowflake, ElasticSearch, ScyllaDB, Kafka, Databricks (Spark), and S3.
The Benefits
The client’s business is based on providing digital twins of cities based on the feeds received from their dashcams. By providing real-time analysis of the video these cameras receive, CloudGeometry enabled the company to give customers the full experience promised by the technology. It also enabled additional lines of business, such as providing information to insurance companies to enable driver benefits for safer driving, and aggregate driver behavior information for cities trying to optimize traffic patterns.
Key Features
<div class="case__txt--cols"><div><h4>Realtime AI/ML video analysis</h4><p>Machine learning model building, training, and optimization to enable object detection.</p></div><div><h4>Mobile application development</h4><p>Integration of dashcam data with Android and iOS apps for full user experience</p></div><div><h4>Full edge computing architecture</h4><p>Data pre-processing at edge nodes with data aggregation at regional and central servers to provide overall benefit for all users</p></div></div>
