1. HBPNeurorobotics
  2. Neurorobotics Platform
  3. Neurorobotics Platform

Overview

HTTPS SSH

README

The Neurorobotics Platform (NRP) is a subproject of the Human Brain Project (HBP), funded by the European Commission (EC). It is distributed as open source under the terms of the GNU Public License version 2, which you can find a copy of in each of our repositories.

Visit our website at http://www.neurorobotics.net

What is this repository for?

This is the meta-repository for the Neurorobotics Platform. It hosts the issue tracker for users to report bug reports and feature requests. It also provides the guide to installation of the Platform from source code.

Who do I talk to?

To report a bug or request a new feature, please use this issue tracker under the "Issues" menu item.

To get support from the developer team, post on our forum at https://forum.humanbrainproject.eu/c/neurorobotics

In case you need to contact us for another purpose, use the contact form at http://www.neurorobotics.net/contact.html

Source code full NRP installation

This installation doc has been tested on Ubuntu 14.04 and Ubuntu 16.04. If using a different distribution, you will have to adapt the dependencies without support from us.


WARNING

Apt and pip are not good friends. If you installed many dist-packages using sudo pip install, you should consider uninstalling them and using the deb packages provided by Ubuntu instead, even if the versions are late. Indeed, the deb packages are tested by Ubuntu and work together. They are the reference. Anything else is installed locally at build time. So, practically, you should have a $HOME/.local/lib/python2.7/dist-packages containing ONLY Nest related stuff: nest, PyNest, Topology and ConnPlotter. If you have other packages, that exist as deb packages, you are strongly advised to pip uninstall them and apt-get install the deb versions instead. PLEASE: see known issues and troubleshooting and the end of the page!


This documentation helps you install a fully local neurorobotics platform. You can of course choose to install all the backend stuff on one computer and the frontend on your laptop for example also, but this is not described here. Very important: in this setup, NRP is installed in your user space (no root install).

NRP Installation

1. Initial set up

  1. Create a folder in you home which will contain required 3rd-party software

    mkdir $HOME/.local
    
  2. Create a new directory where you will install all the source code for NRP. We suggest $HOME/Documents/NRP. This directory will be referred to in the rest of this document as $HBP. Add the $HBP environment variable to your bashrc. Add the $NRP_INSTALL_MODE variable to your bashrc which will make sure you use the publicly accessible code and build process.

    # add the following to your $HOME/.bashrc file
    export HBP=$HOME/Documents/NRP  # Or whatever installation path you chose
    export NRP_INSTALL_MODE=user  # this has to be set to 'user', else the cloning and builds will fail
    

    Source your .bashrc

    source $HOME/.bashrc
    
  3. Clone the user-scripts repository. This repository includes some helper scripts to ease the installation.

    cd $HBP
    git clone https://<your_username>@bitbucket.org/hbpneurorobotics/user-scripts.git
    
  4. Use the clone-all-repos script from user-scripts to clone all necessary repos at once. The $HBP variable need to be set for this script to work successfully!

    cd $HBP/user-scripts
    ./clone-all-repos
    

    Note: on the clone of Models and Experiments repos, you will be asked for a password because they are still private repos. They contain demo experiments and the defaults models. If you want them, contact the team over http://neurorobotics.net. Else you can ignore them for now.

  5. Set up your environment automatically by adding the following two lines at the Bottom of your .bashrc

    . $HBP/user-scripts/nrp_variables
    . $HBP/user-scripts/nrp_aliases
    

    Note: Generally, if you want to change any of the variables for any reason, then override the variables in your .bashrc and do not alter the nrp_variables file.

    Then source you bashrc. Note that you will get some expected errors after sourcing the bashrc if you're doing a fresh install. This is due to some missing files that will be instantiated during the installation process.

    source $HOME/.bashrc
    
  6. Apt prerequisites

    sudo apt-get install python-dev python-h5py python-lxml autogen automake libtool build-essential autoconf libltdl7-dev libreadline6-dev libncurses5-dev libgsl0-dev python-all-dev python-numpy python-scipy python-matplotlib ipython libxslt1-dev zlib1g-dev python-opencv ruby libtar-dev libprotoc-dev protobuf-compiler imagemagick libtinyxml2-dev git python-virtualenv libffi-dev uwsgi-plugin-python
    

2. Install ROS

Note: if you already installed ROS for your Ubuntu from apt-get, then you can skip this section.

  1. Add GPG key of ROS to apt

    sudo wget -O - https://raw.githubusercontent.com/ros/rosdistro/master/ros.key | sudo apt-key add -
    
  2. Add apt repository for ROS

    # On Ubuntu 14.04 trusty
    sudo apt-add-repository "deb http://packages.ros.org/ros/ubuntu trusty main"
    # OR
    # On Ubuntu 16.04 xenial
    sudo apt-add-repository "deb http://packages.ros.org/ros/ubuntu xenial main"
    
  3. Update repositories

    sudo apt-get update
    
  4. Install ROS Indigo for Ubuntu 14 or ROS Kinetic for Ubuntu 16. Indigo has issues on Ubuntu 16 for our platform. Kinetic has not been tested on Ubuntu 14.

    # On Ubuntu 14.04 trusty
    sudo apt-get install ros-indigo-desktop-full
    sudo apt-get install ros-indigo-web-video-server
    sudo apt-get install ros-indigo-control-toolbox ros-indigo-controller-manager ros-indigo-transmission-interface ros-indigo-joint-limits-interface ros-indigo-rosauth
    # OR
    # On Ubuntu 16.04 xenial
    sudo apt-get install ros-kinetic-desktop-full
    sudo apt-get install ros-kinetic-web-video-server
    sudo apt-get install ros-kinetic-control-toolbox ros-kinetic-controller-manager ros-kinetic-transmission-interface ros-kinetic-joint-limits-interface ros-kinetic-rosauth ros-kinetic-smach-ros python-rospkg
    

    Possible bug: If you upgraded from Ubuntu 14.04 to 16.04 you might experience the following error when installing ROS:

    dpkg: error processing package ros-kinetic-desktop-full (--configure):
    dependency problems - leaving unconfigured
    Processing triggers for libc-bin (2.23-0ubuntu3) ...
    Errors were encountered while processing:
      libopenni0
      libpcl-io1.7:amd64
      libpcl-visualization1.7:amd64
      libpcl1.7
      libopenni-sensor-pointclouds0
      ros-kinetic-pcl-conversions
      ros-kinetic-pcl-ros
      ros-kinetic-perception-pcl
      openni-utils
      libopenni-dev
      libpcl-dev
      ros-kinetic-perception
      ros-kinetic-desktop-full
    

    This issue can be solved by purging all libopenni packages:

    $ sudo apt-get purge libopenni*
    

    And then try installing ROS again. The libopenni packages will be automatically reinstalled.


3. Install OpenSim

  1. Build Simbody

    cd $HBP # your source directory for all NRP repos
    cd simbody
    mkdir build     # if not exist
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local ..
    make -j8        # adapt 8 to the number of cores of your computer
    make install
    
  2. Build Opensim

    cd $HBP # your source directory for all NRP repos
    cd opensim
    mkdir build     # if not exist
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local ..
    make -j8        # adapt 8 to the number of cores of your computer
    make install
    
  3. Test that Opensim is installed correctly

    cd $HBP/opensim
    ./runTest
    

4. Install Gazebo

  1. First, if you have Gazebo 6 already installed, remove it first. (Remove any other version of Gazebo that you would have installed by yourself)

    # With ROS Indigo (Ubuntu 14.04 trusty)
    sudo apt-get remove --purge gazebo6* ros-indigo-gazebo6-msgs ros-indigo-gazebo6-plugins ros-indigo-gazebo6-ros ros-indigo-gazebo6-ros-control ros-indigo-gazebo6-ros-pkgs
    # OR
    # With ROS Kinetic (Ubuntu 16.04 xenial)
    sudo apt-get remove --purge gazebo6* ros-kinetic-gazebo6-msgs ros-kinetic-gazebo6-plugins ros-kinetic-gazebo6-ros ros-kinetic-gazebo6-ros-control ros-kinetic-gazebo6-ros-pkgs
    

    Test that roscore is still functioning. If not, consider reinstalling ROS Indigo by first removing it with apt-get remove --purge ros-indigo* and following the above paragraph again.

  2. Add GPG key of Gazebo to apt

    sudo wget -O - http://packages.osrfoundation.org/gazebo.key | sudo apt-key add -
    
  3. Add apt repository for Gazebo

    # On Ubuntu 14.04 trusty
    sudo apt-add-repository "deb http://packages.osrfoundation.org/gazebo/ubuntu trusty main"
    # OR
    # On Ubuntu 16.04 xenial
    sudo apt-add-repository "deb http://packages.osrfoundation.org/gazebo/ubuntu xenial main"
    
  4. Update repositories

    sudo apt-get update
    
  5. Install build dependencies

    sudo apt-get install libignition-math2-dev libignition-transport-dev libignition-transport0-dev
    
  6. Install sdformat. Remove all previous sdformat packages with apt-get remove (libsdformat, sdformat-sdf, ...).

    Warning: Check that $HOME/.local/lib/x86_64-linux-gnu is in your $LD_LIBRARY_PATH. If not, you might have forgotten to add nrp_variables in your .bashrc.

    Warning: Check that $HOME/.local/lib/x86_64-linux-gnu/cmake/sdformat is in your $CMAKE_PREFIX_PATH. If not, you might have forgotten to add nrp_variables in your .bashrc.

    cd $HBP         # your NRP source code directory
    cd sdformat
    mkdir build     # if not exist
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local ..
    make -j8        # adapt 8 to the number of cores of your computer
    make install
    

    If cmake complains about missing libs, install them using apt-get and re-run cmake.

  7. Install bullet physics engine

    Remove previous bullet packages with apt-get remove (libbullet, ...).

    Check that $HOME/.local/lib/pkgconfig is in your $PKG_CONFIG_PATH, so Gazebo's cmake can find the bullet you just installed. If not, you might have forgotten to add nrp_variables in your .bashrc.

    cd $HBP
    cd bulletphysics
    mkdir build     # if not exist
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local ..
    make -j8        # adapt 8 to the number of cores of your computer
    make install
    
  8. Install Gazebo

    You may require additional libraries to compile gazebo with all its options:

    sudo apt-get install libsimbody-dev libgts-dev libgdal-dev ruby-ronn xsltproc graphviz-dev
    

    Then proceed to the gazebo configuration and compilation:

    Warning: Check that $HOME/.local/lib/x86_64-linux-gnu/cmake/gazebo is in your $CMAKE_PREFIX_PATH. If not, you might have forgotten to add nrp_variables in your .bashrc. Further, make sure that $HOME/.local/lib/x86_64-linux-gnu/pkgconfig is in your $PKG_CONFIG_PATH. Otherwise, you will see errors that later builds will claim they did not find the HBP Gazebo features.

    Warning: If you are updating from a previous Ubuntu 14 install, delete old libraries that will not be overwritten and can accidentally be linked to: rm -rf $HOME/.local/lib/x86_64-linux-gnu

    cd $HBP
    cd gazebo
    mkdir build      # if not exist
    cd build
    cmake -DCMAKE_INSTALL_PREFIX=$HOME/.local -DENABLE_TESTS_COMPILATION:BOOL=False ..
    make -j8         # adapt 8 to the number of cores of your computer
    make install
    

    Make takes quite a while, be patient.

    Warning: Here you may miss a number of libs (see end summary from cmake output). Some are optional (like other physics engines) and you can safely ignore them. Just apt-get the required ones, from the "BUILD_ERRORS" section when running cmake.

    Note: Check that $GAZEBO_MODEL_PATH for example exists. If not, you might have forgotten to add nrp_variables in your .bashrc.

    If you want to run Gazebo straight away, you have to source $HOME/.local/share/gazebo/setup.sh Recompile the GazeboRosPackages if you're not doing a fresh install.

  9. Troubleshoot

    • gzserver segfault on start

      gzserver: /usr/include/boost/thread/pthread/recursive_mutex.hpp:101: boost::recursive_mutex::~recursive_mutex(): Assertion `!pthread_mutex_destroy(&m)' failed.
      

      Solution: Try to launch the gzserver binary with no argument. If it runs, it probably means that you made something wrong when compiling GazeboRosPackages. Make sure that you didn't source another catkin workspace before you built GazeboRosPackages.

    • gazebo cmake error (happened with AMD GPU):

      CMake Error at cmake/CheckDRIDisplay.cmake:54 (STRING):
      string sub-command FIND requires 3 or 4 parameters.
      Call Stack (most recent call first):
      CMakeLists.txt:154 (include)
      
    • valid dri display not found (no glxinfo or pyopengl)

      Solution: install mesa-utils (glxinfo)

    • gazebo cmake error (missing tinyxml2): install libtinyxml2-dev.

    • gazebo libccd link error (/usr/bin/ld: cannot find -lccd):

      You likely have a conflicting version of ros-indigo-libccd installed (from other ros packages such as MoveIt). Solution: edit cmake/SearchForStuff.cmake and replace "pkg_check_modules(CCD ccd>=1.4)" with "SET(CCD_FOUND FALSE)" delete your build directory, rerun cmake, and build as usual

5. Install MVAPICH2

MVAPICH2 is an implementation of the Message Passing Interface (MPI) standard.

  1. Check your current autotools versions, mvapich2 is very specific about autotools versions so we will have to build locally if your OS versions are out of date:

    autoconf --version
    automake --version
    libtool --version
    
  2. If your autoconf version is < 2.67:

    mkdir -p $HBP/mvapich2/autotools
    cd $HBP/mvapich2/autotools
    wget https://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
    tar xvzf autoconf-2.69.tar.gz
    cd autoconf-2.69
    ./configure --prefix=$HOME/.local
    make -j8 # adapt this to your system cores
    make install
    cd ../..
    
  3. If your automake version is < 1.15:

    mkdir -p $HBP/mvapich2/autotools
    cd $HBP/mvapich2/autotools
    wget https://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz
    tar xvzf automake-1.15.tar.gz
    cd automake-1.15
    ./configure --prefix=$HOME/.local
    make -j8 # adapt this to your system cores
    make install
    cd ../..
    
  4. If your libtool version is < 2.4.3:

    mkdir -p $HBP/mvapich2/autotools
    cd $HBP/mvapich2/autotools
    wget https://ftp.gnu.org/gnu/libtool/libtool-2.4.6.tar.gz
    tar xvzf libtool-2.4.6.tar.gz
    cd libtool-2.4.6
    ./configure --prefix=$HOME/.local
    make -j8 # adapt this to your system cores
    make install
    cd ../..
    
  5. If you updated at least one of autoconf, automake, libtool, open a fresh terminal and rerun step 2 (to be sure to use the correct path to the new versions).

  6. Install prerequisites:

    sudo apt-get install bison byacc
    
  7. Build and install mvapich2:

    cd $HBP/mvapich2
    ./autogen.sh # this may take a few minutes
    ./configure --prefix=$HOME/.local --with-device=ch3:nemesis
    make -j8 # adapt this to your system cores
    make install
    
  8. Check if your LD_LIBRARY_PATH contains $HOME/.local/lib (should if you correctly include $HBP/user-scripts/nrp_variables in your .bashrc).

  9. Check if your PATH contains $HOME/.local/bin (should if you correctly include $HBP/user-scripts/nrp_variables in your .bashrc).

  10. Verify that mvapich2 is properly installed:

    which mpirun
    

    The expected output is: /home/<user>/.local/bin/mpirun

6. Install MUSIC

The MUltiSimulation Coordinator. MUSIC is an API allowing large scale neuron simulators using MPI internally to exchange data during runtime.

  1. Create a symbolic link that will be needed for MUSIC to work properly as it doesn't explicitly support MPIACH2. This link only needs to exist during configure/build time:

    cd $HBP # your NRP source code directory
    cd MUSIC
    ln -sf $HOME/.local/bin/mpichversion mpich2version
    export PATH=$PATH:$HBP/MUSIC
    
  2. Create a Python virtual env and install build dependencies:

    virtualenv build_venv
    source build_venv/bin/activate
    pip install Cython==0.23.4 mpi4py==2.0.0
    deactivate
    PYTHONPATH=$HBP/MUSIC/build_venv/lib/python2.7/site-packages:$PYTHONPATH
    
  3. Build and install MUSIC:

    ./autogen.sh
    ./configure --prefix=$HOME/.local MPI_CXX=mpicxx
    make -j8
    make install
    # remove the build_env directoy as it might cause problems later
    rm -rf build_venv/
    rm mpich2version
    
  4. Check if your LD_LIBRARY_PATH contains $HOME/.local/lib (should if you correctly include $HBP/user-scripts/nrp_variables in your .bashrc).

7. Install NEST

  1. Remove any previous system level installations of PyNN:

    sudo apt-get remove --purge python-pynn
    
  2. Install the GSL developer package:

    sudo apt-get install libgsl0-dev
    
  3. Create a Python virtual env and install build dependencies:

    cd $HBP # your NRP source code directory
    cd nest-simulator
    virtualenv build_venv
    source build_venv/bin/activate
    pip install Cython==0.23.4 mpi4py==2.0.0
    deactivate
    PYTHONPATH=$HBP/nest-simulator/build_venv/lib/python2.7/site-packages:$PYTHONPATH
    
  4. Build and install NEST:

    ./bootstrap.sh
    ./configure --prefix=$HOME/.local --with-gsl --with-mpi --with-music
    make -j8
    make install
    # remove the build_env directoy as it might cause problems later
    rm -rf build_venv/
    

    Possible Bug

    NEST creates broken symbolic links in $HOME/.local/lib/python2.7/dist-packages/PyNEST.egg-info and $HOME/.local/lib/python2.7/dist-packages/ConnPlotter.egg-info Either delete the symlinks or fix them. In case you delete them, make sure to escape the * in the delete command or you'll delete the actual library. Ex: $ sudo rm PyNEST\*.egg-info ConnPlotter\*.egg-info



    Warning: If you see an error like: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 and then further up is : libhdf5.so: cannot open shared object file: No such file or directory

    Check if you have libhdf5-dev installed. If not, install it.

    Then check that /usr/lib/x86_64-linux-gnu is in your $LD_LIBRARY_PATH (should if you correctly include $HBP/user-scripts/nrp_variables in your .bashrc).


8. Install Gzweb

  1. First make sure that build dependencies are installed.

    sudo apt-get install libgts-dev libjansson-dev
    
  2. Install nvm (Node Version Manager) and nodejs v0.10 and v7

    Install nvm (steps below taken from https://github.com/creationix/nvm)

    curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash
    source ~/.bashrc # or reopen your shells
    

    Install node versions

    nvm install 0.10
    nvm install 7
    nvm alias default 7
    
  3. Install bower

    npm install -g bower
    
  4. Build gzweb

    cd $HBP # your NRP source code directory
    cd gzweb
    npm install
    ./deploy.sh -m local
    

    While performing ./deploy.sh, you will encounter this error in a loop: Couldn't find an AF_INET address for []

    Just press CTRL+C once each time it happens, and the script should continue.

9. Install NRP

  1. Configure NRP

    Go to user-scripts and run the configure scipts. This will set up Makefiles and default config files.

    cd $HBP/user-scripts
    ./configure_nrp
    
  2. Install dependencies

    Important: Make sure your pip version is updated, otherwise some builds will fail. To upgrade pip run: $ pip install --upgrade pip

    Install dependencies

    sudo apt-get install python-pip python-virtualenv
    # remove standard swagger packages (we use custom ones from NRP dependencies)
    # remove the apt versions of these swagger packages too
    sudo pip uninstall swagger flask-swagger flask-restful-swagger
    
    # install some deps needed for then backend (scipy deps)
    sudo apt-get install libblas-dev liblapack-dev libhdf5-dev gfortran
    

    Very important: in CLE and ExDBackend directory, run ubuntu_fix_cv2.sh to update the platform_venv dir and install all the NRP dependencies. The path to the python packages from the platform_venv dir (will be built later in ~/.opt) is included in your PYTHONPATH. Else, you would have for example authentication problems. The script ubuntu_fix_cv2.sh copies the binary files from the package python-opencv to the platform_venv dir.

    cd $HBP/CLE
    ./ubuntu_fix_cv2.sh
    cd $HBP/ExDBackend
    ../CLE/ubuntu_fix_cv2.sh
    
  3. Compile the patched Gazebo Plugin

    Warning: To avoid compiling against gazebo2 embedded with ros-indigo-desktop-full, make sure you prepend CMAKE_PREFIX_PATH so it points to NRP's gazebo-config.cmake in $HOME/.local/... (check that $CMAKE_PREFIX_PATH contains $HOME/.local/lib/x86_64-linux-gnu/cmake/gazebo/, which should be the case if you included nrp_variables in your bashrc).

    As Ubuntu 14 and 16 use different version of ROS, GazeboRosPackages are required to be the corresponding one. GazeboRosPackages has been updated to work with both the versions.

    cd $HBP/GazeboRosPackages
    # make
    catkin_make
    

    Warning: If it fails, it might be that the ROS setup file has not be sourced. If you added nrp_variables correctly in your bashrc, this should not be the case. Else, open a fresh terminal, and check that $ROS_MASTER_URI is set. If not you can manually source the setup and redo the catkin_make.

    source /opt/ros/indigo/setup.bash # for Ubuntu 14
    source /opt/ros/kinetic/setup.bash # for Ubuntu 16
    catkin_make
    

    Note: On Ubuntu 16 building may complain about some missing ROS modules. Just install the missing modules with apt-get install and re-build. i.e

    sudo apt-get install ros-<version(kinetic or indigo)>-<missing package name>
    
  4. Create links to your models in the gazebo models directory:

    mkdir -p $HOME/.gazebo/models
    cd $HBP/Models
    ./create-symlinks.sh
    
  5. Troubleshooting: Problems compiling gzweb with deploy script

    To make sure that gzweb is compiled with proper Gazebo version, prepend the correct path to package configuration:

    export PKG_CONFIG_PATH=$HOME/.local/lib/x86_64-linux-gnu/pkgconfig:$PKG_CONFIG_PATH
    

    If you get node-gyp errors ("File not found"), try deleting $HOME/.npm and re-running deploy script.

    You may encounter: /usr/bin/env: node: No such file or directory There are node-gyp build errors, exiting.

    This is fixed by a symbolic link: ln -s `which nodejs` $HOME/.local/bin/node

    You may also encounter core dump or segmentation faults depending on your graphics card and configuration. Install the proprietary drivers (not the X.org default driver) on your system and reboot.

    You can see more possible problems and solutions here: http://gazebosim.org/tutorials?tut=gzweb_install&ver=1.9%2B&cat=gzweb#Troubleshooting

10. Install Nginx

Nginx is a web server, just like apache, that is very modular and in our case serves all the traffic, adding an authentication layer. It acts as a port forwarder. The backend services that run on 5000 will be mapped on port 8080 to the outside, the assets will be served on 8040, the rosbridge websocket mapped on 9091, as well as the gzbridge websocket.

  1. list existing nginx packages (apt list --installed | grep nginx) and remove if any

  2. Install nginx-extras and lua-cjson

    sudo apt-get install nginx-extras lua-cjson
    

    Your setup should be finished with this step. The configuration of nginx is done automatically in a later step of the 100% install. Though, if you experience issues after you finish all the steps, come back here and check the text below.

  3. Troubleshooting: If you are having issues with this configuration that could mean that your nginx version is too recent. Try running

    nginx -t -c $HOME/.local/etc/nginx/nginx.conf
    

    In case the output of '' is something like 'unknown directive "more_set_headers"', try replacing the following lines in gzweb.conf

    location / {
    ...
    #more_set_headers 'Access-Control-Allow-Origin: $cors_origin';
    #more_set_headers 'Access-Control-Allow-Methods: $cors_methods';
    #more_set_headers 'Access-Control-Allow-Headers: $cors_headers';
    #more_set_headers 'Access-Control-Expose-Headers: Content-Length';
    
    add_header 'Access-Control-Allow-Origin' '$cors_origin' 'always';
    add_header 'Access-Control-Allow-Methods' '$cors_methods' 'always';
    add_header 'Access-Control-Allow-Headers' '$cors_headers' 'always';
    add_header 'Access-Control-Expose-Headers' 'Content-Length' 'always';
    ...
    }
    

    and in nrp-services.conf

    location / {
            ...
            #more_set_headers 'Access-Control-Allow-Origin: $cors_origin';
            #more_set_headers 'Access-Control-Allow-Methods: $cors_methods';
            #more_set_headers 'Access-Control-Allow-Headers: $cors_headers';
    
            add_header 'Access-Control-Allow-Origin' '$cors_origin' 'always';
            add_header 'Access-Control-Allow-Methods' '$cors_methods' 'always';
            add_header 'Access-Control-Allow-Headers' '$cors_headers' 'always';
            ...
    }
    
    location /api {
            ...
            #more_set_headers 'Access-Control-Allow-Origin: $cors_origin';
            #more_set_headers 'Access-Control-Allow-Methods: $cors_methods';
            #more_set_headers 'Access-Control-Allow-Headers: $cors_headers';
    
            add_header 'Access-Control-Allow-Origin' '$cors_origin' 'always';
            add_header 'Access-Control-Allow-Methods' '$cors_methods' 'always';
            add_header 'Access-Control-Allow-Headers' '$cors_headers' 'always';
            ...
    }
    

    nginx may complain that /<assets_root>/sdk does not exist. If so, just create an empty directory. No idea what is it for...

    Use nginx -t -c $HOME/.local/etc/nginx/nginx.conf to get debug info if $HOME/.local/etc/init.d/nginx restart" returns [fail].

11. Install Frontend

If you run into problems, see the Troubleshooting section at the end of the page.

  1. Install dependencies:

    cd $HBP # your NRP directory
    cd ExDFrontend
    sudo apt-get install ruby-compass
    sudo gem install compass
    
  2. Do a npm install (in the root folder of the ExDFrontend project!), which is needed in order to install a local grunt (also see: http://stackoverflow.com/questions/13925916/fatal-error-unable-to-find-local-grunt):

    cd $HBP/ExDFrontend
    npm install
    
  3. Install grunt:

    npm install -g grunt-cli
    npm install -g grunt
    
  4. Troubleshooting

    • Error when running npm install

      If you see the following error: sh: node command not found

      you could try installing the package nodejs-legacy instead of nodejs:

      sudo apt-get install nodejs-legacy
      
    • Error when running grunt serve

      On Ubuntu 14.04.1 LTS it may be the case that the following error appears after having executed the steps above:

      user@machine:~/src/ExDFrontend$ grunt serve
      Loading "autoprefixer.js" tasks...ERROR
      >> Error: Cannot find module 'postcss'
      Loading "connect.js" tasks...ERROR
      >> Error: Cannot find module 'cookie-signature'
      Loading "imagemin.js" tasks...ERROR
      >> Error: Cannot find module 'imagemin-pngquant'
      Loading "grunt-karma.js" tasks...ERROR
      >> Error: Cannot find module 'socket.io'
      Loading "publish.js" tasks...ERROR
      >> Error: Cannot find module 'npmi'
      
      Running "serve" task
      Warning: Task "autoprefixer" not found. Use --force to continue.
      

      This can be solved by installing some packages (again) manually:

      npm install npmi
      npm install socket.io
      npm install di
      npm install log4js
      npm install lodash
      npm install postcss
      npm install cookie-signature
      npm install useragent
      npm install chokidar
      npm install connect
      npm install bytes
      npm install debug
      npm install send
      npm install imagemin-pngquant
      npm install caniuse-db
      npm install batch
      npm install negotiator
      npm install express
      npm install q
      
    • If you get this kind of JavaScript error, as reported in this post on the HBP forum, you may want to deactivate AdBlocker.

      Failed to instantiate module exdFrontendApp
      Error: [$injector:modulerr] Failed to instantiate module exdFrontendApp
      due to:
      [$injector:modulerr] Failed to instantiate module
      angulartics.google.analytics due to:
      [$injector:nomod] Module 'angulartics.google.analytics' is not
      available! You either misspelled the module name or forgot to load it.
      If registering a module ensure that you specify the dependencies as the
      second argument.
      http://errors.angularjs.org/1.4.8/$injector/nomod?p0=angulartics.google.analytics
      minErr/<@http://10.162.242.194:9000/bower_components/angular/angular.js:68:12
      module/<@h ttp://10.162.242.194:9000/bower_components/angular/angular.js:2005:
      

12. Install Proxy

  1. Install the dependencies

    cd $HBP
    cd $HBP/nrpBackendProxy
    npm install
    
  2. OPTIONAL: To run the proxy right away

    cd $HBP/nrpBackendProxy
    node app.js
    

13. Install Retina

The retina framework implemented at the University of Granada by Pablo Martínez-Cañada et al. [1][2] has been recently integrated in the NRP [3].

As of the date of this writing, the retina plugin has to be included in this build process. This will change in the future though.

  1. Building the retina framework

    cd $HBP
    cd retina
    mkdir build
    cd build
    qmake-qt4 ../retina.pro INSTALL_PREFIX=$HBP/retina/build
    # Make sure to adapt -j8 to the number of cores you have on your machines,
    # and according on the available memory (g++ processes may be extremely memory expensive)
    make -j8 && make install
    

    After the compilation you should have two .so files in the retina/build/lib folder: libretina.so and pyretina.so, the first one containing the core functionality of the retina framework, and the second one being a python wrapper to a subset of the classes available in the framework.

    As we are interested in using the retina framework from python transfer functions in the NRP, we should make sure that pyretina.so is available as a python library. The PYTHONPATH variable is set in $HBP/user-scripts/nrp_variables like in the following snippet (don't change it):

    export PYTHONPATH=$PYTHONPATH:$HBP/retina/build/lib
    
  2. You can test if everything went right by trying whether python is able to load the library or not. If you are a one line lover you can use the following command:

    python -c "import pyretina" 2>/dev/null; if [ $? -eq 0 ]; then echo "OK"; else echo "Something was wrong"; fi
    

14. Configure NRP

  1. Run the configure_nrp script in user-scripts

    cd $HBP/user-scripts
    ./configure_nrp
    

This script copies the configuration files of nginx, ExDFrontend, CLE as well as the gzserver & gzbridge scripts to the correct locations. It also modifies the raw config files to include the correct paths and usernames.

  1. Open a fresh terminal to be certain to have the proper environment and build the Platform at once:

    ./update_nrp build
    

    This will take time since all the python dependencies will be downloaded and numpy takes a while to build. If the script does not fail, then you are done.

Running the Platform

Make sure nginx is running fine (restart it : it should output [ OK ] ) and watch the error_log.

    cle-nginx

Then start the NRP backend by using either

    cle-start

or running the individual cle-<service> in separate consoles (but mind the order of commands in cle-start).

If it fails with 502, possibly the pyretina is missing. Go back to section "Install Retina".

Start the frontend server by opening another terminal and typing

    cle-frontend

You can then connect with your browser:

open http://localhost:9000/#/esv-web in your browser. In case of CORS related errors in your javascript console, beware to activate CORS. Depending on your browser, this is different. On chrome, there is an extension to install called Allow-Control-Allow-Origin that you have to reactivate at each chrome start (even if it is green already).

open http://localhost:9000/#/esv-web?dev in your browser to be able to see a complete list of experiments including non-production ready ones. This also gives you the option to select a specific server to run an experiment on.

open http://localhost:8080/api/spec.html to access the backend through the swagger interface.

After launching an experiment, it might fail saying cannot import name config.

    pip install config

Troubleshooting

Nginx is not starting during system boot? Add the following lines to /etc/rc.local:

if [ -f $HOME/.local/etc/init.d/nginx ]; then
    $HOME/.local/etc/init.d/nginx restart
fi

general access problems

-> file not accessible? 404? Check that the whole path to the file is readable and/or writable to you.

not loading experiment list

-> cors problems?

-> nginx: sudo killall and service restart

python DictError about some dictionary word missing

-> you probably run a local version of Flask. Check in /usr/local/lib/python2.7/dist-packages. There should be ONLY Nest related stuff. If there are pip pacakges, please remove them, especially Flask, SQLAlchemy, Werkzeug, itsdangerous and rerun $HOME/user-scripts/update_nrp build.

problems with gzweb deploy.sh

-> webify_models_v2.py: splits with mesh, do not put/use mesh in model names...

-> server crashes with error "Unable to find local grunt" or other nodejs package related error

cd $HBP/ExDFrontend
npm install

'cle-start' / 'cle-rosbridge' complains about missing rosauth.srv

sudo apt-get install ros-indigo-rosauth

'cle-start' / 'cle-rosbridge' can't find package rosbridge_server. Add the following to .bashrc

export ROS_PACKAGE_PATH=$ROS_PACKAGE_PATH:</path/to/GazeboRosPackages>/src

'cle-start': backend can't find modules (e.g. ImportError: No module named flask_restful_swagger)

There was an error during make devinstall when running $HOME/user-scripts/update_nrp build. Then run "make devinstall" in ExDBackend as follows:

make devinstall > devinstall-log.txt 2>&1

to see what happened. In the case of flask_restful_swagger, it might help to

  • rename ExDBackend/hbp-flask-restful-swagger-master temporarily
  • rerun make devinstall

Don't forget to restore ExDBackend/hbp-flask-restful-swagger-master to its original name afterwards.

'cle-start': Failed to execute request http://localhost:8080/experiment. ERROR: Error: Error: connect ECONNREFUSED

Nginx is not running or not running correctly. Double check nginx is running and that there is no errors in the nginx logs.

Gazebo6 general problems or compiling GazeboRosPackages

to make sure there are no dependencies problems:

sudo apt-get purge gazebo4 # or 2 or 6
sudo apt-get remove gazebo4 # or 2 or 6
sudo apt-get install ros-indigo-desktop-full

then clean build directories and rebuild:

cd $HBP/GazeboRosPackages/
rm -rf build
catkin_make

it might be necessary to do

catkin_make --force-cmake

Building GazeboRosPackages failed for me with Gazebo installed from package repositories. (gazebo.physics.Joint::SetMappedRotationAxis not found) If that happens, remove gazebo packages and compile from EPFL sources as above.

If building Gazebo fails with the message

Linking CXX shared library libgazebo_common.so
/usr/bin/ld: /usr/local/lib/libavcodec.a(allcodecs.o): relocation R_X86_64_32 against `ff_h263_vaapi_hwaccel' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libavcodec.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

You can prevent it from building with ffmpeg support by editing $HBP/gazebo/cmake/SearchForStuff.cmake. In line 391 change HAVE_FFMPEG TRUE to HAVE_FFMPEG FALSE. This will cause it to always assume that ffmpeg is not installed. Alternatively, you could compile ffmpeg from sources and create dynamic libraries instead of the static ones hat are installed from the package management.

If a similar error comes up later during the compile referencing BulletPhysics, try recompiling Bullet: First, delete the old Bullet files: Bullet does not provide a make uninstall target, so we need to do it manually. In the build folder, run

sudo xargs rm < install_manifest.txt

This will remove all files installed by make install. Then recompile and give the flag -DBUILD_SHARED_LIBS=true to cmake. This will build dynamically linked libraries. If you do this, make sure both $HOME/.local/lib and $HOME/.local/lib/x86_64-linux-gnu is on your LD_LIBRARY_PATH. The recommended way of adding it is creating the file /etc/ld.so.conf.d/gazebo.conf with the content

$HOME/.local/lib/x86_64-linux-gnu
$HOME/.local/lib

and running sudo ldconfig

If an error similar to this comes up

gazebo/physics/libgazebo_physics.so.6.0.6.hbp.1.0.0: undefined reference
to `gazebo::sensors::SensorManager::SensorsInitialized()

check your version of cmake, if it is version 3, try and downgrade it to version 2 and see if it helps

e.g.

apt list --installed | grep cmake # check the version of cmake
sudo apt-get install cmake=2.8.12.2-0ubuntu3 cmake-data=2.8.12.2-0ubuntu3 # or just downgrade to a previous version

ImportError: No module named pyretina

Follow the tutorial here: Install Retina.

Toolbar buttons icons (play/stop) show up as strange characters

This issue is due to missing fonts and outdated dependencies. (see NRRPLT-2472)

  • Clear browser cache and close the browser
  • Update ruby (tested with 1.9.3p484)
  • run "sudo gem install compass"

Robot is not moving even though spike monitor works, no errors

Run catkin_make in GazeboRosPackages.

Updating your local install

This is very easy! Just go to user-scripts and run

    ./update_nrp update all

If you don't provide the optional all argument, only the core python repos will be updated, which is a much shorter build, but might miss changes in 3rd party software like gazebo. So we would recommend that you just use this argument and get a lunch break while it builds.

Troubleshooting

The update_nrp script will exit on error if you have local changes in the repos. So if you do not want to risk a failure after one hour of build, we recommend that you make sure your repos in $HBP are clean or changes are stashed or committed.