Behavioral Cloning

Goals

The goals of this project are as follows:

• Use the simulator to collect data of good driving behavior.
• Create datasets for training and validation.
• Build, a convolution neural network using Keras that predicts steering angles from images.
• Train and validate the model.
• Test that the model successfully drives the vehicle and completes a loop around track at least once without leaving the road.

Results

My project includes the following files:

• model.py containing the script to create and train the model.
• drive.py for driving the car in autonomous mode.
• model.h5 containing a trained convolution neural network.
• model_FINAL.h5 containing a further trained convolution neural network.

I started out with a more traditional grouping of convolutional layers (which involve many trainable parameters) and a max pooling layers (to reduce number of parameters). There was some evidence of over-fitting as the training accuracy would be high while the validation accuracy would not change much over many epochs. To work around that I added dropout layers to provide regularization. So I added dropout regularization while making some other architectural changes, stacking convolutional layers together prior to a max pooling, and the result was rewarding. On the cons side, the CNN with regularization needs more resources. In my setup, it takes the whole 8GB of GPU RAM and uses 50% of an RTX2070. There is still have some over-fitting but now I have run out of time and need to submit this project.

I also included some preprocessing of images as layers into the model: a cropping layer to remove portions of the scene on top and bottom of image which did not add to the determination of steering angle. My final model consisted of the following layers:

Layer	Description
Input	160x320x3 RGB image
Cropping 2D	65x320x3 RGB image
Lambda (scale/shift)	65x320x3 RGB image
Convolution 5x5	3x3 stride, same padding, outputs 22x107x8
Convolution 5x5	3x3 stride, same padding, outputs 8x36x16
Max pooling	2x2 stride, outputs 4x18x16
Convolution 5x5	3x3 stride, same padding, outputs 2x6x16
Convolution 5x5	3x3 stride, same padding, outputs 1x2x32
Max pooling	2x2 stride, outputs 1x1x32
Dropout	rate = 0.4
Flatten
Fully connected	16 units
Dropout	rate = 0.4
Fully connected	8 units
Fully connected	outputs 1

Data Collection

I recorded my own data (dcg_data) and used the data sample fully along with the data sample provided by Udacity. I remove some of the zero steering angle data since it is prevalent in rows (which would otherwise incur in bias to drive straight in autonomous mode). I remove inconsistent data recorded while car was not moving during training. I smoothed out all steering angles because the data recorded seemed to cluster on some values that seemed to be set by the UI while using keyboard controls. I also augmented data by adding flipped versions of the training and validation datasets so to account for bias to turn left. Of course, the training data was chosen to keep the vehicle driving on the road. And I did use a combination of center lane driving, recovering from the left and right sides of the road from my own dataset (dcg_data).

Model Architecture

Here is a visualization of the architecture:

Training and Predictions Evaluation

The picture below shows the final steering angle (measurements) distributions used for training, validation and testing. demonstrates that the processing used to mitigate bias in the data was able to smooth out the influence to steer left or drive straight that the data without processing would cause.

A sample of 50 images was selected from the test dataset for the simple purpose of providing a more direct way for me to troublehoot model development and make progress even without using the simulation framework.

This shows the predictions for 50 randomly selected images from the test dataset:

At the end of the process, the vehicle is able to drive autonomously around the track without leaving the road as desired.