An End-to-End solution to Autonomous Driving based on Xilinx FPGA

EasyChair Preprint 1680

Abstract

Nowadays, the autonomous driving topic is very hot, many people are trying to provide a solution to this problem. This time we build our own auto-driving car based on Xilinx Pynq-Z2, it provides an end-to-end solution to autonomous driving and it uses the power of DPU to accelerate the computing. We find that in previous FPT competitions, many competitors use traditional computer vision methods or machine learning methods to detect objects and then make decisions based on the results. Our solution is quite different with theirs since we use a simple CNN model as our AI network and feed the model with pictures taken by the camera in the car and the model returns the control orders. The platform we use is Xilinx’s Pynq-Z2 board, there is a DPU IP in its FPGA, with DPU’s help, we can accelerate the inference process of the model and make it possible to run AI inference task in limited resource platform like Pynq-Z2. Our project process: 1. We control the car through the keyboard to finish specified tasks and the car will generate training data automatically. 2. Copy the generated data to host pc which can be any platform that can run Tensorflow, on the host pc we do the network training and finally get a trained model. 3. Use Xilinx DNNDK to do some optimization to the model and compile it into files that can be run in DPU. 4. Copy the files to Pynq-Z2 and then the car can run and make decisions itself. In most tasks, our solution can completely rely on machine learning methods, such as left-side driving, obstacle avoidance and crossroad driving. When it comes to other scenes, we use traditional computer vision algorithms and state machine to help us finish the task. The car only uses a camera as the sensor now, we may add some more sensors if a camera is not enough.

Keyphrases: Field Programmable Gate Arrays, PYNQ-Z2, autonomous driving, computer vision, machine learning

@booklet{EasyChair:1680,
  author    = {Tianze Wu and Weiyi Liu and Yongwei Jin},
  title     = {An End-to-End solution to Autonomous Driving based on Xilinx FPGA},
  howpublished = {EasyChair Preprint 1680},
  year      = {EasyChair, 2019}}