Playing for Data
This source code release accompanies the paper
Playing for Data: Ground Truth from Computer Games
Stephan Richter, Vibhav Vineet, Stefan Roth and Vladlen Koltun. In ECCV 2016.
git clone --recursive https://bitbucket.org/visinf/projects-2016-playing-for-data ./playing-for-data.
This code is released under the MIT license, see LICENSE.md for full text as well as 3rd party library acknowledgements.
- CMake (for making OpenEXR and zlib)
- Visual Studio (for compiling renderdoc) - tested on VS 2013
- Python - tested with Anaconda
- MATLAB (for annotating frames) - tested on ML 2013a
Let <PFD_DIR> be the Playing for Data directory with the following structure:
- renderdoc | Graphic Debugger - dump frames to disk
- scripts | Python scripts for extracting data from games using renderdoc
- README.md | this file.
- Use CMake to build zlib <PFD_DIR>/renderdoc/renderdoc/3rdparty/zlib
- Use CMake to build IlmBase (located at <PFD_DIR>/renderdoc/renderdoc/3rdparty/openexr/IlmBase).
CMAKE_INSTALL_PREFIXto your preferred directory.
- Compile IlmBase by building the
- Use CMake to build OpenEXR (located at <PFD_DIR>/renderdoc/renderdoc/3rdparty/openexr/OpenEXR).
Make sure that
ILMBASE_PACKAGE_PREFIXin CMake is set to the install directory of IlmBase (the one you specified as
CMAKE_INSTALL_PREFIXin the 2nd step).
If you built zlib in a separate build directory, copy the zconf.h from that directory to the
- Compile OpenEXR. Make sure that the ilmbase libraries (Half, IlmThread) are on your library search path
(for setting the path under windows see Preparation 2. below),
otherwise compiling the IlmImf project may fail with some cryptic error (Visual Studio returns
"cmd.exe" exited with code -1073741515).
- Compile the complete renderdoc solution.
- Make sure the OpenEXR dlls are in a directory pointed to by your path variable. (Setting the path under windows is done at Control Panel/System and Security/System/Advanced System Settings/Environment Variables)
Setup for capturing:
- Run renderdocui.exe with admin privileges.
- Specify the path where dumped frames are stored: Tools/Options/Directory for temporary capture files
- Enable global process hooking in Tools/Options/Allow global process hooking (May not be needed by all games)
- Open the Capture Executable Tab.
- Point the Executable Path to your game
- If you are using the global hook, the other settings don't matter as they are set in a different process.
If you need to modify them, check <PFD_DIR>/renderdoc/renderdoccmd/renderdoccmd_win32.cpp, starting at line 610.
- Enable Global Hook
- Start the game
- If you see an overlay on the top left of the screen, it works. Otherwise check if your path points to the OpenEXR libraries. F12 will enable the capture mode and dump a frame every 40 frames. Stop capturing by pressing F12 again.
Setup for processing frames (single frame):
- Start renderdocui.exe
- Load a capture file via Open Log
- Open the python shell
- Run the script we provided via Run scripts or use the interactive shell.
You may need to configure the python script we provided to your environment.
Setup for processing frames (automated):
renderdoc saves UI settings in
If it does not exist, run renderdoc once and close it again.
Open the file and edit the following values:
LoadScriptFileto the script you want to run for each capture file.
ExecuteScriptOnLoadto true. This will load the script you specified above and execute it right after a capture file has been loaded.
- You can start renderdoc from the command line:
renderdocui.exe <path to capture file>
- To close renderdoc after processing is done, add
renderdoc.AppWindow.Close()to your script
Annotating the data