HTTPS SSH

VideoParsing

Reference CPU implementation for semantic video segmentation project.

@inproceedings{KunduCVPR16,
 author = {Abhijit Kundu and Vibhav Vineet and Vladlen Koltun},
 title={Feature Space Optimization for Semantic Video Segmentation},
 booktitle={CVPR},
 year={2016},
}

Dependencies

  • CMake - a cross-platform, open-source build system
  • Eigen 3 - a C++ template library for linear algebra (Requires Eigen 3.3.x)
  • Boost - C++ Libraries
  • OpenCV - Open Computer Vision Library
  • AMGCL - C++ library for solving large sparse linear systems (Automatically downloaded if not already installed)
  • Doxygen - tool for generating documentation [optional]

Installation

If you are in Ubuntu 16.04, all the dependencies can be installed by the following command:

$ sudo apt-get install cmake libeigen3-dev libboost-all-dev libopencv-dev doxygen

Once the required dependencies listed above are installed, we can compile and build using the cmake build system. For a Unix system, that may look like the following steps:

  1. Clone/download the repo
  2. cd /path/to/repo
  3. mkdir build
  4. cd build
  5. cmake ..
  6. make -j12
  7. make doc [Optional] - makes documentation of the library API
  8. sudo make install [Optional] - For using VideoParsing in external projects

Demo Sequence

This is a small demo for the semantic video segmentation pipeline.

Download and unzip the demo sequence data (2 GB).

$ cd DataFolder
$ wget http://webshare.ipat.gatech.edu/coc-rim-wall-lab/web/abhijit/DemoSeq.zip
$ unzip DemoSeq.zip

Edit the root_dir in DemoSeq.cfg to point to the unzipped DemoSeq folder (e.g., DataFolder/DemoSeq)

$ cd VideoParsing/build
$ cp ../config-files/DemoSeq.cfg .
$ vi/subl/gedit DemoSeq.cfg
Edit `root_dir` in DemoSeq.cfg to make sure it points to the unzipped DemoSeq folder

Run feature space optimization

$ ../scripts/runFSO_On_DemoSeq.sh

Run dense blocked video CRF with optimized features

$ ../scripts/runBlockVideoGMwFeatures_On_DemoSeq.sh

Running On Custom Datasets

This implementation only has code for the feature optimization and the structured prediction layer. This code requires unaries, optical flow, tracks, and edge-maps to be provided by the user. Formats.md explains the file formats expected by this library. See DemoSeq data for an example. You then need to have a dataset config file which lists the paths to the required data. There are example config files in folder config-files. If you have Doxygen installed, make doc builds a reference documentation of the API, which is useful in case you plan to understand/modify the code.

License

The code is released under the MIT license. You can use the code for any purpose with proper attribution. If you use the code, please cite our paper:

@inproceedings{KunduCVPR16,
 author = {Abhijit Kundu and Vibhav Vineet and Vladlen Koltun},
 title={Feature Space Optimization for Semantic Video Segmentation},
 booktitle={CVPR},
 year={2016},
}