Overview
Atlassian Sourcetree is a free Git and Mercurial client for Windows.
Atlassian Sourcetree is a free Git and Mercurial client for Mac.
Appearance-based methods for visual localisation
This is a MATLAB library to extract visual descriptors and implement a bag-of- visual-words pipeline from video sequences taken by multiple users in order to provide localisation.
The code is customised and ready to be used with the RSM dataset (http://rsm.bicv.org) but can be used on any sort of image sequences if the directory paths are correctly specified.
Current implemented descriptor extraction methods (description below):
LW_COLOR
, SIFT
, DSIFT
, SF_GABOR
, ST_GABOR
, ST_GAUSS
Current supported format of the sequences: jpg
Authors:
- Jose Rivera (jose.rivera@imperial.ac.uk)
- Ioannis Alexiou (i.alexiou@qmul.ac.uk)
- Anil A. Bharath (a.bharath@imperial.ac.uk)
Web: http://www.bicv.org
Date: v4.1 11/2015
Requirements:
SIFT
, DSIFT
, VLAD
and kernel implementations require VLFEAT
Clustering requires INRIA's Yael K-means
Running Instructions:
Rename initialize.m.template
to initialize.m
cp initialize.m.template initialize.m
Run main.m
main
Detailed Instructions:
Parameter selection
- Dependency paths: include the paths to the dependencies.
YAEL
: https://gforge.inria.fr/projects/yael/VLFEAT
: http://www.vlfeat.org/
A version of these libraries is included in the Downloads section of the repository
- Parameter selection.
Select your choice from the following parameters in the params structure before continuing:
params = struct(... 'descriptor', 'ST_GAUSS',... % SIFT, DSIFT, SF_GABOR, ST_GABOR, ST_GAUSS, 'corridors', 1:6,... % Corridors to run [1:6] (RSM v6.0) 'passes', 1:10,... % Passes to run [1:10] (RSM v6.0) 'trainingSet', [1:3,5], ... 'datasetDir', '/data/datasets/RSM/visual_paths/v6.0',... % The root path of the RSM dataset 'frameDir', 'frames_resized_w208p',... % Folder name where all the frames have been extracted. 'descrDir', ... '/data/datasets/RSM/descriptors', ... 'dictionarySize', 400, ... 'dictPath', '/data/datasets/RSM/dictionaries', ... 'encoding', 'HA', ... % 'HA', 'VLAD', 'LLC' 'kernel', 'chi2', ... % 'chi2', 'Hellinger' 'kernelPath', '/data/datasets/RSM/kernels', ... 'metric', 'max', ... 'groundTruthPath', './ground_truth', ... 'debug', 1 ... % 1 shows waitbars, 0 does not. );
These parameters are the following
- datasetDir: The root path of the RSM dataset
- corridors: Corridors to run [1:6] (RSM v6.0)
- passes: Passes to run [1:10] (RSM v6.0)
- trainingSet: training set to use for dictionary construction
- frameDir: Folder name where all the frames have been extracted.
- descrDir
- descriptor: Type of descriptors to be calculated. To choose from
- LW_COLOR: Lightweight spatio-temporal colour descriptor
- SIFT: keypoint based SIFT descriptors
- DSIFT: Dense SIFT
- SF_GABOR: Frame-based DAISY-like descriptors
- ST_GABOR: Spatio-temporal Gabors.
- ST_GAUSS: Spatio-temporal, Spatial Derivative, Temporal Gaussian
- dictionarySize: number of visual words (parameter k in k-means)
- dictPath: directory where to store the created dictionaries.
- encoding: encoding method
- kernel:
- kernelPath:
- metric:
- groundTruthPath:
- debug:
Descriptor generation
computeDescriptors(params);
Bag of Words pipeline
- create_dictionaries (k-means vector quantization)
clusterDescriptors % OR clusterDescriptorsSparse (for Keypoint-SIFT)
- BOVW encoding (Hard assigment, VLAD, or LLC)
hovwEncoding % Remember to modify the parameters of the encoding, which will automatically call % encode_hovw_METHOD/encode_hovw_METHOD_sparse (for Keypoint-SIFT), % where METHOD = % HA, "Visual categorization with bags of keypoints", Dance et al., 2004, % VLAD, "Aggregating local descriptors into a compact image representation", Jegou et al., 2010, % LLC, "Locality-constrained Linear Coding for Image Classification", Wang et al., 2010.
- Kernels for histograms
runKernelHA % OR runKernelHellinger
- Run evaluation routine to add the error measurement to the kernels.
run_evaluation_nn_VW
- [Optional: Move kernels to one folder]
#!/bin/bash cd /folder/to/kernels mkdir all_chi2 find . -name *chi2*.mat -exec cp -vf {} all_chi2/ \; # if chi2 kernel mkdir all_Hellinger find . -name *Hellinger*.mat -exec cp -vf {} all_chi2/ \;
- Generate PDF results and plots with
results_generation.m