3D environment reconstruction close to serial production
Porting of a Structure-from-Motion environment to an embedded system with graphics acceleration (OEM & Lieferant 1/2018)
The use of more advanced sensor technologies is the prerequisite for autonomous or semi-autonomous driving. All vehicle manufacturers face the same challenges, regardless of the area of use of the vehicles - on or off the roads. Function, safety and cost factors, along with sensor fusion, play a decisive role in the selection of suitable sensor technologies. ServiceXpert Gesellschaft für Service-Informationssysteme mbH and ESG Elektroniksystem- und Logistik GmbH have jointly developed the current software solution based on structure-from-motion (SfM) technology that enables 3D optical environment reconstructions using motion pictures from a mono-camera. In this case, a 3D point cloud of the environment, as well as the relative movement of the camera (six degrees of freedom), is calculated from the tracking of individual image features over several frames and geometric equations. This means significantly lower hardware expenditure in terms of camera selection compared to stereo imaging. This image processing software has now been integrated by ServiceXpert engineers on an embedded one-chip system.
Until now, the SfM solution has been used on PC architectures such as x86/x64 using Windows or Linux, so implementing this solution on dedicated hardware as an embedded system is the next step in the evolution of this technology. This implementation makes this first step into the vehicle possible - of course, taking into account operational safety, costs, and installation space, amongst other things.
The NVIDIA Jetson TX2 serves as a platform for the integration of the software. The hardware is based on NVIDIA Tegra technology and features a dual-core Denver2 processor and a quad-core ARM Cortex-A57 processor. It uses ARM64 technology, an NVIDIA Pascal GPU with 256 CUDA graphics cores and 8GB of memory. It has been developed as a credit card sized, high-performance embedded chip for the automotive industry. The chip is provided as a development kit on a convenient board with comprehensive peripherals and interfaces. The development kit has almost unlimited digital inputs and outputs, HDMI, USB, Ethernet and CAN interfaces. Similarly, the board is flashable via a host PC by means of SSH via Ethernet with a new operating system. In this way, proprietary drivers and libraries from NVIDIA can be installed.
Since the development kit comes with a preinstalled Linux-4-Tegra distribution on Ubuntu 16.04, ServiceXpert was able to successfully port the code in a short time. The effort required for this porting was also small due to the extensive operating system on the board. The software could be compiled directly on the board. Error-prone cross-compilation of the source code via a host PC or laborious flashing or migrating of the final program was not necessary. Optimisations and architecture-specific compilation parameters could be partly adopted. A demo application was already executable after a first integration cycle without hardware-specific optimisations.
To fully exploit the potential of the hardware, the powerful GPU of the TX2 board was used. NVIDIA's proprietary software libraries for the CUDA technology are available for this purpose. This provides significant performance benefits for a large number of simple, parallel computational operations, such as linear matrix-vector operations. In a further development step, the corresponding code sections and associated operations were quickly identified by ServiceXpert engineers, optimised for image feature recognition, and integrated with GPU-accelerated features.
Using the NVIDIA technology, the ServiceXpert development team has been able to extend its broad experience in close-to-production environmental detection and will incorporate this into existing projects with customers and partners. As a hardware-neutral service provider, the ServiceXpert engineers repeatedly draw on numerous project experiences in the evaluation and selection of hardware partners and regularly evaluate existing offers and services. The requirements of the customer can thus be made available and incorporated into an application-specific parameterisation and hardware recommendation.
ServiceXpert, an ESG-Group company, employs over 85 staff in Hamburg and Munich. ServiceXpert is a Europe-wide operating system and software house with a focused service portfolio for Lifecycle management of EE information for leading manufacturers of commercial vehicle and their supplier industry.