Issue #53 resolved
Former user created an issue

Hi all --

I am trying to record some data from the Multisense-SL, i.e. stereo, lidar, IMU, RGB data, in high frame rate, i.e. 15fps.

The issue I am facing is the amount of hiccups due to buffering, even if I use 2Gb of tmpfs space for the recording.

Any idea what is the best way to record organized colored stereo point cloud data of 2MP, in 15fps for 1min with the minimum amount of hiccups?

Thanks a lot!

Comments (7)

  1. Matt Alvarado

    Hi Dimitrios,

    There are a few things which you can do to improve the recording of high bandwidth topics via rosbag.

    1. Use the "-b 0" argument when executing rosbag record (http://wiki.ros.org/rosbag/Commandline). This will use a infinite buffer when queuing up messages to write to disk. The downside is you will consume a large amount of memory while recording.

    2. Use a SSD for recording. In my personal experience this significantly improves the logging speed and reduces the number of dropped messages.

    3. Increase the amount of RAM on you logging machine. I have had success logging MultiSense image streams on machines with 16 GB of RAM with a SSD. If you are using a spinning disk you will probably want more RAM so you can buffer more messages.

    4. Increase the kernel's UDP buffer sizes on your logging machine. You can do this with the following commands:

        sudo sh -c “echo 16777215 > /proc/sys/net/core/rmem_max”
    
        sudo sh -c “echo 16777215 > /proc/sys/net/core/wmem_max”
    

    This will reduce the number of dropped packets between the MulitSense and the logging machine improving your overall acquisition frame rate.

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  2. Chris Osterwood

    Hi Dimitrios,

    One other suggestion (but you may be doing this already) is to record disk-efficient topics of the information you are interested in. For example, disparity images (e.g. /multisense/left/disparity or /multisense/calibration/raw_cam_data) are more compact than the PointCloud2 representations of the 3D data.

    If you record disparity images you'll also need to record the calibration information, and upon playback re-project the disparity image into a PointCloud2 topic (or topic).

    We've also found that turning on PNG compression of disparity images can reduce disk space and disk bandwidth by 80% -- albeit with higher CPU use during recording. Left rectified data can go through the same PNG compression process, but bandwidth gains are much lower.

    Thanks,

    Chris Osterwood Carnegie Robotics

  3. Dimitrios Kanoulas

    Dear Matt and Chris --

    thanks a lot for the suggestions. It seems that recording the following data can be done without buffer hiccups in my system: 1) /multisense/depth/camera_info 2) /multisense/depth/compressedDepth, 1MP (5 fps) 3) /multisense/left/camera_info 4) /multisense/left/image_color/compressed, 1MP (5 fps) 5) /multisense/lidar_scan, (40Hz, 05 rad/sec rotation) 6) /tf

    I guess these are enough for regenerating the colored organized point clouds, if I am not wrong.

    For those that are OK of not using ROS (rosbag), Maurice Fallon told me that for the DRC they implemented a standalone driver for the Multisense SL (https://github.com/openhumanoids/multisense-lcm.git) that communicates via lcm (lightweight communication and marshalling) and depends on libmultisense (https://bitbucket.org/crl/libmultisense), lcm (https://github.com/lcm-proj/lcm), and opencv (http://opencv.org/), and is able to achieve similar results as I explained above. I have not test it yet, but it may worth using it.

    The question for me is if I can record stereo (30Hz), lidar (40Hz), rgb (30Hz), and imu (100Hz) data without hiccups for 1min or so. For now it seems impossible.

  4. Matt Alvarado

    Hi Dimitrios,

    I have never been able to log every image with rosbag, so my resulting frame rate for logging images is about 28Hz. That being said, I see no reason why the left disparity, LIDAR, left color, and IMU streams should be causing you issues when you are logging. One additional thing to try is remove all miscellaneous networking hardware between the logging machine and the MultiSense and use a direct connection.

    You can telnet into the MultiSense (telnet 10.66.171.21) to check if there are any transmission failures while logging is occurring.

    Could you also provide more information on the "hiccups" you are seeing while logging. Are there regions of data which are missing, or is the overall frame rate very low?

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  5. Dimitrios Kanoulas

    I could check and give details about the hiccups soon. An extra advice for those trying to rosbag data is to use tmpfs:

    mkdir /tmp/rosbag_space
    
    sudo mount -t tmpfs -o size=6G,nr_inodes=10k,mode=0700,uid=vision tmpfs /tmp/rosbag_space/
    
    cd /tmp/rosbag_space
    
    rosbag record -O <BAG_NAME>.bag -b 1024 <TOPICS_LIST>
    
  6. Log in to comment