1. jbalzer Avatar jbalzer
  2. Untitled project
  3. YAS

Wiki

Clone wiki

YAS / Documentation

View History

Home | Documentation | Tutorials

Overview

  1. Home screen
  2. Preferences
  3. Viewer

Home screen

  • Record Start streaming sensor data to disk. Clicking this button triggers a dialogue which lets the user specify the file to stream to. The file should have the extension .oni. Note that these files are uncompressed and gain size pretty quickly, several GBs in just a few minutes. ONI files can be opened with YAS through the File menu. They emulate the behavior of a real sensor, looping back to the beginning of the stream once its end is reached.
  • Pause Stop displaying the live stream.
  • Step Record and display a single frame. Clicking this button only has an effect when the live stream is paused.
  • Run Start displaying the live RGB and depth stream.
  • Min/Max This sets a minimum/maximum clipping depth which is applied whenever the stored disparity data is exported or converted to depths for alignment.
  • Store This pushes the current live image to a stack of stored images. These can be aligned with each other, combined into a common Poisson reconstruction, or exported separately to various formats. Note that the overlap between view is not inferred automatically. Alignment proceeds pairwise starting from the first image which is chosen to be the world reference frame. Capture can also be triggered by hitting the enter key.
  • File
    • Open... Import EXR files. Make sure that the alpha channel contains disparities and not depths (because it is possible to export depths). Can also be used to open an ONI file stream. This disables the physical sensor for the remaining execution time of the program.
    • Save... Save the current stored image to file. If the file name does not end with a suitable extension, the format selected in the options combo box in the dialogue window is used. The following formats are supported:
      • PNG Only contains the RGB image.
      • EXR RGBA image in half float precision. The alpha channel will hold raw disparities unless specified otherwise in the I/O tab of the preference menu. The homogeneous view point matrix is also embedded and is restored when using Open...
      • PLY Saves a triangle mesh. Empty if Poisson reconstruction has not been performed, yet. The raw point clouds can be written to disk with Save all...
      • TXT Textual representation of homogeneous vantage point matrices as well as intrinsics of the RGB and depth camera.
      • PGM Contains disparity image. Can be used as input to Herrera's calibration toolbox.
      • R4R Normal map computed by finite-differencing applied to the depth image in R4R format.
    • Save all... Save all stored images. If the file name does not end with a suitable extension, the format selected in the options combo box in the dialogue window is used. The image number is automatically appended to the file name. If the PLY format is chosen, the depth images are saved as oriented and colored point clouds w.r.t. to the world reference frame.
    • Exit Quit the program.
  • View
    • 3D View... Open the OpenGL viewer which displays the currently selected stored depth image as a point cloud along with a visualization of the local camera reference frame.
    • Preferences... Open the preferences menu.
  • Tools
    • Poisson Transform local point clouds to world reference frame and merge them. Then perform Poisson surface reconstruction. The output is rendered in the OpenGL viewer window and can be saved to disk by choosing the PLY format in the File:Save... menu.
    • Align
      • Current Triggers alignment of the currently selected stores view with the next in storage. Has no effect if the last image in the stack is selected.
      • All Align all stored views with each other. Note that the overlap between view is not inferred automatically. Alignment proceeds pairwise starting from the first image which is chosen to be the world reference frame.

Preferences

Sensor

This tab is for setting the calibration parameters of the RGBD sensor.

  • RGB
    • Focal lengths In pixels.
    • Principal point In pixels.
    • Distortion
      • Radial These correspond to the parameters k0, k1, and k4 as defined in the documentation of Bouget's camera calibration toolbox.
      • Tangential Distortion parameters k2 and k3 according to Bouget.
    • Translation The location of the depth camera origin in RGB camera coordinates. Used to build the transformation from RGB to depth camera coordinates.
    • Rotation Rotation matrix used in the transformation from RGB to depth camera coordinates.
  • Depth
    • Focal lengths In pixels.
    • Principal point In pixels.
    • Distortion
      • Radial These correspond to the parameters k0, k1, and k4 as defined in the documentation of Bouget's camera calibration toolbox.
      • Tangential Distortion parameters k2 and k3 according to Bouget.
    • Range Operating range of the depth sensor in meters.
    • Disparity map Two parameters describing the relationship between disparity and depth. Can be obtained with Herrera's calibration toolbox.
    • Error pattern Two parameters specifying the influence of the spatial rectification pattern of Herrera's error model. The pattern can be loaded from a binary file storing it in single precision and row-major ordering by clicking on Load. So far it could not be observed, however, that its application has significant influence on the reconstruction quality.
  • Apply Apply changes. If this button is not pressed the changes will not take effect.
  • Save Save calibration parameters to file.
  • Load Load calibration parameters from file. Make sure that Apply is pressed after loading the parameters from file.

Alignment

This tab contains all parameters related to view alignment. More infos on the SIFT parameters can be found in the OpenCV documentation.

  • Initialization
    • Number of SIFT features The number of best features to retain. The features are ranked by their scores (measured in SIFT algorithm as the local contrast).
    • Number of SIFT octaves The number of layers in each octave. 3 is the value used in D. Lowe paper. The number of octaves is computed automatically from the image resolution.
    • SIFT point threshold The contrast threshold used to filter out weak features in semi-uniform (low-contrast) regions. The larger the threshold, the less features are produced by the detector.
    • SIFT edge threshold The threshold used to filter out edge-like features. Note that the its meaning is different from the point threshold, i.e. the larger the edge threshold, the less features are filtered out (more features are retained).
    • Feature rejection threshold Parameter of Lowe's ratio test (relative distance between first and second nearest neighbor such that the match is accepted).
    • RANSAC sample size Number of random correspondence triplets drawn for consensus.
    • Acceptance threshold Euclidean distance between points in meters such that they are accepted as inliers.
  • Refinement
    • Number of ICP steps Number of iterative closest-points iterations carried out for alignment refinement. Set to 0 to disable refinement.
  • Misc
  • Center CS If this box is checked, following the execution of Align all..., a ground plane (if existent) is estimated by a RANSAC-type procedure. The world reference frame is moved from the camera frame of the first view to the barycenter of the measured subset of this plane with its z-axis aligned with the ground plane normal. Selecting this option is never harmful as the relative poses of all views is maintained. Aligning the world coordinates system with the ground plane is beneficial for volume estimation.

Reconstruction

Parameters of screened Poisson surface reconstruction.

  • Depth This is integer the maximum depth of the tree that will be used for surface reconstruction. Running at depth d corresponds to solving on a voxel grid whose resolution is no larger than 2^d x 2^d x 2^d. Note that since the reconstructor adapts the octree to the sampling density, the specified reconstruction depth is only an upper bound.
  • Point weight This floating point value specifies the importance that interpolation of the point samples is given in the formulation of the screened Poisson equation. The results of the original (unscreened) Poisson Reconstruction can be obtained by setting this value to 0.
  • Samples per node This floating point value specifies the minimum number of sample points that should fall within an octree node as the octree construction is adapted to sampling density. For noise-free samples, small values in the range [1.0 - 5.0] can be used. For more noisy samples, larger values in the range [15.0 - 20.0] may be needed to provide a smoother, noise-reduced, reconstruction.
  • Scale This floating point value specifies the ratio between the diameter of the cube used for reconstruction and the diameter of the samples' bounding cube.
  • Solver iterations Maximum number of iterations of the linear solver.
  • Solver accuracy Value of the residual below which to terminate the iterative linear solver.
  • Confidence Enabling this flag tells the reconstructor to use the size of the normals as confidence information. When the flag is not enabled, all normals are normalized to have unit-length prior to reconstruction.
  • Polygon mesh Enabling this flag tells the reconstructor to output a polygon mesh (rather than triangulating the results of Marching Cubes).

Visualization

  • Occlusion threshold Minimum depth variance of four neighboring pixels before it is marked as an occlusion (red) in the live stream.

I/O

File format options.

  • PLY
    • Binary Save in binary or ASCII format.
  • EXR
    • Save depth Convert disparities to depths before saving. Only select this option for processing the data in third-party applications. YAS expects disparities when reading EXR files.
    • Align with RGB Pull back the depth image to the domain of the RGB image. This brings the color and depth values at a pixel in correspondence.

OpenGL viewer

  • Press Z key Reset view to stored view point.
  • Wheel Translate camera in z direction.
  • Left mouse button Pan/tilt view.
  • Middle mouse button Translate in x/y direction.
  • Right mouse button Translate in z direction.

Updated