Stereo cloud with a lot of outliers

Issue #26 resolved
Dimitrios Kanoulas created an issue

Hi all --

I am trying to understand if the outliers I am getting from the stereo cloud is because of stereo camera calibration, or because of the type of the environment. The truth is that I am getting these big number of outliers no matter the texture of the environment. Attached is a typical .pcd example, where from the camera point of view all the corners are occupied with lots of outliers far away from the real surface. Any idea? Thanks!

  • if recalibration of stereo camera is needed, what is the preferable way to do it for the Multisense-SL head?

Comments (12)

  1. Matt Alvarado

    Hi Dimitrios,

    The outliers in the attached pcd file are normal for MultiSense units.  At the boundry of the projection circle the stereo matching algorithm has less image data to work with, and the lens distortion model is least accurate in this area. One way this issue can be avoided is by setting the MULTISENSE_ROS_PC_BORDER_CLIP environment variable. This will exclude 3-D points whose disparity location is within the user specified number of pixels from the edge of the image. Additionally the /multisense/left/cost image can be used to exclude 3-D points which have a low confidence from the SGM stereo matching process.

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  2. Dimitrios Kanoulas reporter

    Hi Matt --

    1) setting the MULTISENSE_ROS_PC_BORDER_CLIP to set the border clip value does not affect too much the result except if you set it in a big value, eg for a 1024x1024 image toMULTISENSE_ROS_PC_BORDER_CLIP=150. Why I believe is that and correct me if I am wrong: because in the code (multisense_ros/src/camera.cpp) you consider the border of the rectangular region and not the boundary of the circular one. This is one of the reason that even if you set the variable you get outliers in the 4 corners of the image. If I am correct is it possible to solve this issue?

    2) The /multisense/left/cost image looks interesting. What is the range of the value it takes? The bigger the worse?

    Thanks for the help!

  3. Matt Alvarado

    Hi Dimitrios,

    You are correct that the MULTISENSE_ROS_PC_BORDER_CLIP removes a rectangular boundary. This is acceptable behavior for 2MP imagers, but as you pointed out is not ideal for 4MP imagers where the vignetting is more extreme. I can change the computation of the region to exclude a circular region.

    The cost image is stored as an 8 bit image where higher values indicate low confidence. More info about its comparability with MultiSense firmware can be found at .

    The MultiSense ROS driver also offers a means to adjust the strength of the FPGA stereo post filter. The dynamic reconfigure parameter stereo_post_filtering gives you access to this.

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  4. Dimitrios Kanoulas reporter

    Hi Matt --

    thanks for the answer! Yes having the circular region exclusion would be more than helpful! Please let us know when it is available!

    So 0-255 is the values we are getting from the cost image, thanks! And I will try the stereo_post_filtering asap! Thanks a lot again!

  5. Matt Alvarado

    Hi Dimitrios,

    The latest ROS driver, version 3.3.9, offers both rectangular and circular border clipping. These options can be adjusted using the dynamic reconfigure “border_clip_type” and “border_clip_value” parameters.

    Please let me know if you find any issues with the implementation.

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  6. Dimitrios Kanoulas reporter

    Matt --

    I am not sure that the radius of the CIRCULAR border_clip_type is set correctly. You can realize it when setting the border_clip_value to a fixed value: e.g. 100 and then switch from RECTANGULAR to CIRCULAR type. The CIRCULAR should have at least the same effect as the RECTANGULAR, if not be better, since you eliminate more space. Thus this is not true to the resulting cloud (check in rviz).

    My best guess: when you compute the radius (line 2105):

    double radius = sqrt( pow( halfWidth, 2) + pow( halfHeight, 2) );

    is not really the radius you are looking for. How about trying:

     double radius = (sqrt( pow( halfWidth, 2) ) + sqrt( pow( halfHeight, 2) )) ) / 2.0;
  7. Matt Alvarado

    Hi Dimitrios,

    I interpreted the circular clipping type slightly differently than you. My one constraint for both clipping types was when the border_clip_value is 0 no clipping should occur. This means in the 0 case the circular clip should be the circle which circumscribes the entire image ie:

    double radius = sqrt( pow( halfWidth, 2) + pow( halfHeight, 2) );

    This radius will shrink as the value of border_clip_value increases.

    The method you proposed does not preserve this property and appears to be just an average of halfHeight and halfWidth. I also think this approach would exhibit strange behavior for resolutions where the image height is not equal to the image width.

    You can easily visualize the border clip by adding the following code to the end of the Camera:generateBorderClip

    cv::imshow("Debug", border_clip_mask_);

    Whenever the border clipping parameters are changed via dynamic reconfigure, this will display the border clipping mask in the Debug image window.

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  8. Dimitrios Kanoulas reporter

    Aha! Now I see your approach. In that case what I actually need (e.g. in a 1024x1024 image) is a border_clip_value >200 to have an effect. If you would like to increase the rqt reconf and put it to 400 instead of 200 it would be great, because of this different interpretation of the circle.

  9. Dimitrios Kanoulas reporter

    Yeah yeah, I tried it already. It works great! Circ-300 has way better results than Rect-100! Thanks a lot!

  10. Matt Alvarado

    Hi Dimitrios,

    The most recent ROS driver, version 3.4.0, has a max border clipping value of 400 for 4MP imagers.

    Thanks, Matt Alvarado Engineer Carnegie Robotics

  11. Log in to comment