1. Suren Kumar
  2. bimodal_sparse


This software accompanies the paper titled "S. Kumar, V. Dhiman, and J. J. Corso. Learning compositional sparse models of bimodal percepts. In Proceedings of AAAI Conference on Artificial Intelligence, 2014."

Tested on Ubuntu 12.04 64 bit

sudo apt-get install python-opencv2 python-sklearn curl
sudo pip install sh munkres

cd external/spams-python
python setup.py build
python setup.py install --prefix ../spams/
cd ../..
export PYTHONPATH=`pwd`/external/spams/lib/python2.7/site-packages/:${PYTHONPATH}

Reproducible experiments
All our experiments are reproducible with the following instructions:

What do individual pieces of code do?
code/gen_machine_audio.py - Generates hindi or english pronounciations of shapes
and colors.
code/align_bimodal_features.py - Allocated the modalities of vision to audio
code/aud_feat.py - Given a audio MFCC feature .csv file, generates a audio 
feature by selecting top components of the audio file
code/comp_dict_learn.py - Learns a compositional dictionary
code/eval_quantitative.py - Code written to display results from single and
compositional model
code/vis_feat.py - Get visual features
code/utils.py - Evaluation and other support files
code/paper_test.py - Generating Quantitative results for the paper

To generate the quantitative results in the paper, use 
cd data && python wgetdata.py && cd ..
python code/paper_test.py