An end-to-end approach to autonomous vehicle control using deep learning

Abstract

The objective of this work is to develop an autonomous vehicle controller inside Grand Theft Auto V game, used as a simulation environment. It is used an end-to-end approach, in which the model maps directly the inputs from the image of a car hood camera and a sequence of speed values to three driving commands: steering wheel angle, accelerator pedal pressure and brake pedal pressure. The developed model is composed of a convolutional neural network and a recurring neural network. The convolutional network processes the images and the recurrent network processes the speed data. The model learns from data generated by a human driver´s commands. Two interfaces are developed: one for collecting in-game training data and another to verify the performance of the model for the autonomous vehicle control. The results show that the model after training is capable to drive the vehicle as well as a human driver. This proves that a combination of a convolutional network with a recurrent network, using an end-to-end approach, is capable of obtaining a good driving performance even using only images and speed velocity as sensory data.