Deep-Driving

Drive in SpeedDreams

Preparation

First you need to install SpeedDreams for DeepDriving as described in this wiki.

Start SpeedDreams with the DeepDriver Car

• Start the SpeedDreams application.

cd <speeddreams-install-path>

bin/speed-dreams-2

• Start a new race by selecting the option "Race".

• Chose "Quick Race" on the next screen.

• Click on "Configure" to edit the race settings.

• Chose a track where you want to start the race. For deep-driving, you need to chose a track from the category "US Lanes" to get a track with lane markings. An overview over all tracks can be found here. Press "Next", if you have selected a track.

• Remove all cars from the track by pressing on the waste bin symbol.

• Chose the car category "All car categories" and chose the driver type "LaneDriver".

• Select as many Lane-Driver cars as you want and add them to the race with the plus symbol. 12 Lane Drivers is a good number on fast computers. The sorting of lane drivers is not important. You can chose them in any order you want (this is different to the original project).

• Now chose the driver type "DeepDriver" and add the car "DeepDriver" as last car to the race.

• Press "Next" to apply the changes.

• On the next screen simply press "Next" to take the default values.

• Now the track overview should look like this:

• Press "Start" to start the race.

Start the Driving-App

• Start the script drive.py in the python/scripts directory of the repository. This script uses the GUI, thus you need to install the optional GUI packages.

cd <repository-path>/python/scripts

python drive.py

• As default checkpoint this app uses the pre-trained model from the repository. But you can select a different checkpoint in the file inference.cfg (see this page for more details):

{
  "Data": {
    "ImageHeight": 210,
    "ImageWidth": 280
  },
  "Inference": {
    "CheckpointPath": "<your-checkpoint-path>",
    "Epoch": null
  },
  "PreProcessing": {
    "MeanFile": "image-mean.tfrecord"
  }
}

• The App-Window should look like this:

• On the right side you can see the current frame from the game. On the left side the current game situation is displayed. The ground-truth data from the game is displayed by filled rectangles (red/yellow), while the estimated car position is displayed by green and lila rectangles without filling.

• Wait some seconds until all LaneDriver cars leave their position.

• Afterwards press the "Drive" Button to start the DeepDriver.

• By pressing the "AI" Button you can enable or disable the neural network.

• On a GTX 1080 Ti graphic card the inference should run within 5ms, thus the inference does not slow down the frame rate of the game. However if you run the inference on CPU only, it may take around 100ms, which reduces the frame rate of the game enormously. In this case you can also slow-down the physic-engine of the game by pressing "-" in the game window. Thus the DeepDriver is still driving correctly, even if the game is running with very view frames per second.

Next Step

• Deep-Driving: Record your own Training-Data