##Background ##

Calibrating cameras to extract intrinsic parameters (such as focal length, distortion and principle point) and extrinsic parameters (such as relative translation and rotation) can be a complicated process.

The most oft-used method is capturing multiple images of a known checkerboard with a defined square size and number. Most people use Bouget's Matlab toolbox, but this requires user interaction in the form of corner picking. Anything more than 20 images can be extremely laborious. For a solid calibration, 80 or more images may be required to achieve accuracy.

The RADDOC Toolbox is a modification of Bouget's toolbox that automatically detects checkerboard corners, avoiding the most labour intensive user interaction. However, it still requires user interaction at a number of levels, and is not capable of automatically calibrating stereo or multi-camera systems.

##The Automatic Multi-Camera Calibration Toolbox##

The AMCC Toolbox is a branch of Bouget's and the RADDOC toolbox that includes a number of changes and improvements to both automate the process and perform stereo/multi-camera calibration. It also includes select files from the VGG toolbox.

Most original files are the same as they were in Bouget's toolbox. The biggest difference is that there are a number of extra files that automate a number of these tasks. Most of these files have been copied with a _auto.m extension so scripts like go_calib.m will have a twin go_calib_auto.m that removes the lines that require user interaction.

In addition, the stereo calibration GUI (stereo_gui.m) has some additional functions including an epipolar geometry checker and the ability to properly save and load the modified calibration results from this toolbox.

###What it's good for###

• Auto calibrating fixed multi-camera rigs for monocular, stereo and multicamera configurations where there is a strong overlap between adjacent cameras

###What it's not good for###

• Calibrating catadioptric and other lenses. See https://sites.google.com/site/scarabotix/ocamcalib-toolbox for an alternative

• Calibrating cameras without a high degree of overlap

• Calibrating some extremely fish-eye lenses. This requires some more investigation.

##Setting up the Toolbox##

Establish that you have the Image Processing Toolbox installed and working, this is a required dependency.

Grab the toolbox using git: $ git clone https://bitbucket.org/michaeldwarren/amcctoolbox.git.

Make sure to add the top-level amcctoolbox directory (and all sub-folders) to your Matlab PATH. Done!

Note: if you get frequent "Failed due to exception." errors, it's likely you haven't added all the subfolders to the path or you haven't installed the Image Processing Toolbox. Check the error message for more details.

##Using the Toolbox##

The modified Bouget's/RADDOC Toolbox is located in:

amcctoolbox/amcctoolbox

The 'high-level' automated calibration tools are located in:

amcctoolbox/calibration

There are three main scripts (that you will call directly) in this directory: auto_mono_calibrator_efficient.m, auto_stereo_calibrator_efficient.m and auto_multi_calibrator_efficient.m, plus a batch version of the stereo calibrator: auto_stereo_calibrator_efficient_batch.m. These use the 'efficient' versions of the RADDOCToolbox scripts because we often use more than 100 images, which can bog down processing if all images are in memory at once. The other files in this directory are support functions that you normally shouldn't have to call directly.

You will need a directory of checkerboard images to perform the calibration. * For all calibrations the naming convention follows a certain default (that can be overridden, see the examples below): "camX_imageYYYYY.FMT" where X is the camera number (starting from 0), YYYYY is the image number and FMT is the relevent image format (jpg, bmp, tif etc.).

• Note that there shouldn't be any spaces in the filenames.

• There is no need to remove images that don't have a checkerboard or remove images from other cameras that won't be used in the calibration, the calibrator will automatically suppress or ignore them.

• Note that there is a limitation where the checkerboard does not maintain a single orientation during capture in versions 1.2 and earlier. Depending on which nx_crns/ny_crnrs setup you choose (see the example scripts below), images with the opposite orientation won't work, but it will still calibrate using the correctly oriented images. If you find that a high number of the correct number of corners is being detected but they are flipped in X and Y, swap the numbers over and try the calibration again.

##Using auto_mono_calibrator_efficient.m##

auto_mono_calibrator_efficient.m automates monocular calibrations in the following steps: 1. Load image

1. Extract corners using the auto detector

2. Check if the checkerboard was detected properly and suppress or include image as appropriate

3. Repeat steps 1-3 until all images processed

4. Perform a calibration on the image set

5. Find the image with highest reprojection error

6. Suppress the image and recalibrate

7. Repeat steps 6-7 until the minimum threshold reprojection is reached

8. Saves the output to a file

An example script is shown below, showing the required variables that must be defined before calling auto_mono_calibrator_efficient.m. Copy and run this script to perform an automated monocular calibration, changing the example variables as appropriate.

#!matlab

%%%%%%%%%%%%%%%%%%%% Calibrate Monocular Camera %%%%%%%%%%%%%%%%%%%%%%%%
% Default image name
calib_name = 'cam0_image';

% The name to save the results under
save_name = 'Calib_Results_left';

% Where the images are (forward slashes only, and must include a trailing slash)
input_dir = '/media/Datasets/101215/Calibrations/101215_151454_MultiCamera0/';

% Where the data will be saved (forward slashes only, and must include a trailing slash). This folder should already exist.
output_dir = '/media/Datasets/101215/Calibrations/101215_151454_calibration/';

% Image format: jpeg, bmp, tiff, png etc.
format_image = 'bmp';

% Length of each square of the checkerboard in the X direction (mm)
dX = 100.0;
% Length of each square of the checkerboard in the Y direction (mm)
dY = 100.0;
% X,Y direction is according to image coordinates. So X is in vertical direction and Y is in horizontal direction.

% number of *internal* square corners of the checkerboard in the X direction (do not include corners that are on the edge of the checkerboard)
nx_crnrs = 5;
% number of *internal* square corners of the checkerboard in the Y direction (do not include corners that are on the edge of the checkerboard)
ny_crnrs = 8;

% tolerance in pixels of reprojection of checkerboard corners. 2.0 is a reasonable number. By decreasing this number you are causing the calibrator to be more strict by throwing out more poorly estimated checkerboards, but potentially a more accurate calibration
proj_tol = 2.0;

% indicate whether or not to rotate the image by 180 degrees (0 no, 1 yes)
rot_flag = 0;

% indicate whether or not to use the fisheye calibration routine.
fisheye = false;

% indicate whether or not to use the third radial distortion term when doing a projective calibration
k3_enable = false;

% Calibrate the monocular camera automatically
% and reject those images with error greater than proj_tol
%auto_mono_calibrator_efficient; % version 1.3a and before
auto_mono_calibrator_efficient(calib_name, save_name, input_dir, output_dir, format_image, dX, dY, nx_crnrs, ny_crnrs, proj_tol, rot_flag, fisheye, k3_enable); % version 1.3b and after (current svn version)

% If desired, show the final reprojection error to the user by uncommenting the following line
%analyse_error;

###Using the output### The output for this script will be a mat file in the form defined by save_name in the script, placed in output_dir. This will be a monocular calibration that you can open with calib_gui in Matlab for further examination.

##Using auto_stereo_calibrator_efficient.m## auto_stereo_calibrator_efficient.m automates stereo calibrations in the following steps:

1. Sets up image names according to the grab_dv and Orca image naming convention.

2. Calls auto_mono_calibrator_efficient for the left camera (including checkerboard detection and calibration)

3. Keeps a record of the suppression list

4. Calls auto_mono_calibrator_efficient for the right camera (including checkerboard detection and calibration)

5. Keeps a record of the suppression list

6. Makes sure that the suppression lists match

7. Performs a recalibration based on these suppression lists

8. Reloads both monocular calibrations in the stereo framework

9. Calibrates the stereo geometry based on the monocular calibrations

10. Saves the stereo results

An example script is shown below, showing the required variables that must be defined before calling auto_stereo_calibrator_efficient.m. * Note that you can use the 'batch' version to run two monocular calibrations in parallel, if you have two CPU cores and Matlab's Parallel Computing Toolbox, to increase speed quite dramatically. This will suppress the command window output of the monocular calibrations until both are complete, so it might be worth trying the standard version on a small subset to check for errors before committing to the full calibration.

#!matlab

% Where the images are (forward slashes only, and must include a trailing slash)
input_dir = 'E:/Data/GoPro_stereo_calibration_May_22nd/images/';

% Where the data will be saved (forward slashes only, and must include a trailing slash). This folder should already exist
output_dir = 'E:/Data/GoPro_stereo_calibration_May_22nd/calibration/';

% Image format
format_image = 'jpg';

% Length of each square in the X direction
dX = 100.0;
% Length of each square in the Y direction
dY = 100.0;

% number of *internal* square corners in the X direction (do not include corners that are on the edge of the checkerboard)
nx_crnrs = 5;
% number of *internal* square corners in the Y direction (do not include corners that are on the edge of the checkerboard)
ny_crnrs = 8;

% tolerance in pixels of reprojection of checkerboard corners. 2.0 is a reasonable number. By decreasing this number you are causing the calibrator to be more strict by throwing out more poorly estimated checkerboards, but potentially a more accurate calibration
proj_tol = 2.0;

% Whether a particular cameras images should be rotated 180 degrees (0 for normal, 1 for 180 degree rotation)
rotcam = [0 0];

% The numbering system of the cameras you want to calibrate. This will affect the input default image names and the name of the output files
cam_vec = [0 1];

% the naming convention (these are not necessary if using the standard camX_imageYYYYY format)
cam0_names = '3D_L';
cam1_names = '3D_R';

% indicate whether or not to use the fisheye calibration routine (not strictly required).
fisheye = false;

% indicate whether or not to use the third radial distortion term when doing a projective calibration (not strictly required)
k3_enable = false;

% Perform the calibration (using the standard, single thread calibrator)
auto_stereo_calibrator_efficient(cam_vec, input_dir, output_dir, format_image, dX, dY, nx_crnrs, ny_crnrs, proj_tol, rotcam, cam0_names, cam1_names, fisheye, k3_enable);

% Perform a batch stereo calibration (the monocular calibrations are run in parallel, speeding up computation)
%auto_stereo_calibrator_efficient_batch(cam_vec, input_dir, output_dir, format_image, dX, dY, nx_crnrs, ny_crnrs, proj_tol, rotcam, cam0_names, cam1_names, fisheye, k3_enable);

###Using the output### The output for this script will be three files in the form 'Calib_Results_N.mat', 'Calib_Results_M.mat', 'Calib_Results_stereo_N_M.mat' (placed in output_dir) which correspond to the two monocular calibrations and the stereo calibration respectively. The numbers 'N' and 'M' will be defined according to the cam_vec variable where cam_vec = [N M]. You can open the monocular calibrations with calib_gui and the stereo calibration with stereo_gui in Matlab for further examination.

##Using auto_multi_calibrator_efficient.m##

auto_multi_calibrator_efficient.m automates multi-camera calibrations in the following steps:

1. Sanity checks the cameras that it will be calibrating.
2. v1.2 and earlier: Using the number with the smallest index as the 0'th camera, calibrates each other camera against it using the stereo calibrator. v1.3a and later: Calibrates in pairs defined by the camera_vec vector

An example script is shown below, showing the required variables that must be defined before calling auto_multi_calibrator_efficient.m.

#!matlab

% Where the images are (forward slashes only, and must include a trailing slash)
input_dir = '/media/Datasets/101215/Calibrations/101215_151454_MultiCamera0/';

% Where the data will be saved (forward slashes only, and must include a trailing slash). This folder should already exist
output_dir = '/media/Datasets/101215/Calibrations/101215_151454_calibration/';

% Image format: jpeg, bmp, tiff, png etc.
format_image = 'bmp';

% Length of each square of the checkerboard in the X direction (mm)
dX = 100.0;
% Length of each square of the checkerboard in the Y direction (mm)
dY = 100.0;

% number of *internal* square corners of the checkerboard in the X direction (do not include corners that are on the edge of the checkerboard)
nx_crnrs = 5;
% number of *internal* square corners of the checkerboard in the Y direction (do not include corners that are on the edge of the checkerboard)
ny_crnrs = 8;

% tolerance in pixels of reprojection of checkerboard corners. 2.0 is a reasonable number. By decreasing this number you are causing the calibrator to be more strict by throwing out more poorly estimated checkerboards, but potentially a more accurate calibration
proj_tol = 2.0;

% The index of the cameras to calibrate. In this example we are calibrating four cameras with sequential naming.
% camera_vec = [0 1 2 3]; % version 1.2 and before
camera_vec = [0 1; 0 2; 0 3]'; % version 1.3a and later

% The index of the cameras to be rotated (1 for rotating 180 degrees). Rotcam must be the same dimensions as camera_vec
% rotcam = [0 0 0 0]; % version 1.2 and before
rotcam = [0 0; 0 0; 0 0]'; % version 1.3a and later. 

% indicate whether or not to use the fisheye calibration routine (not strictly required).
fisheye = false;

% indicate whether or not to use the third radial distortion term when doing a projective calibration (not strictly required)
k3_enable = false;

% the base naming convention for the calibration images (not strictly required), will default to the 'camX_image' convention if not used.
% cam_names = ['cam0_image', 'cam1_image', 'cam2_image', 'cam3_image']; % version 1.2 and before
cam_names = ['cam0_image'; 'cam1_image'; 'cam2_image'; 'cam3_image']; % version 1.3a and later

% indicate whether or not to use the batch mode of the stereo calibrator (not strictly required)
batch = false;

% Perform the calibration
auto_multi_calibrator_efficient(camera_vec, input_dir, output_dir, format_image, dX, dY, nx_crnrs, ny_crnrs, proj_tol, rotcam, cam_names, fisheye, k3_enable, batch);

##Deficiencies## * The calibrator is currently quite slow. If you are calibrating over 200 stereo images without parallel processing, be prepared to wait 2 or 3 hours for a result. It will, however, be a completely hands off process! Most of the processing is the automated checkerboard detection and then the continual recalibration and checking against re-projection tolerance. Since the calibrator saves data at key steps, you can go back and modify calibrations after the script has finished to improve the result if you wish.

##Need Help?##

Please email michaeldwarren AT gmail DOT com. Nine times out of ten your setup script will have a misspelled string or other minor setup error. Double check your script before looking for further help. If you really need it, please send your verbatim script and a few example images from your calibration dataset. Specify the toolbox version number and describe in as much detail as possible the issue you are having.