The camera_calibration package of ROS provides us with a convenient tool to calibrate a stereo camera. With camera_calibration, we could calibrate a stereo camera in an online fashion. And camera_calibration could tell you, while in the process of sampling data for calibration, whether you have collected enough images that could possibly result in a good calibration. After the calibration, you could choose to save the data into your file system as a zip file. In this zip file, you will find two YAML files that contain the calibration results for the individual cameras that form the stereo system. Later in the 3D reconstruction process based on OpenCV, you will need to convert the above-mentioned YAML files into some format that OpenCV could handle easily. To be specific, you need to convert and generate files that feed essential information to the OpenCV APIs to make OpenCV work properly to give you a disparity map. The files need to be generated by converting the YAML files include:

• The camera matrix, K.
• The distortion coefficient vector, D.
• The Projection matrix, P. And,
• The Rotation matrix, R. (Called rectification_matrix in the YAML file.)

K, D, P and, R need to be generated for each camera of the stereo system. It is very straightforward to convert data from the YAML file to formats that could be used by OpenCV. In particular, for the current project I am working on, I would like to convert the YAML files into plain text files that could be directly loaded by numpy’s loadtxt() function in Python. Later, the 3D reconstruction will be performed in Python with the help of cv2 and numpy packages.

For K, P, and R, it is really simple. In python, you just read in the YAML files and then export the matrices by numpy’s savetxt() function. For D, things are a little bit more complex. From the documentation of cv2, we know that there should be 8 parameters in the distortion coefficient vector. Namely, there are designated as

where $k_1$ to $k_6$ are for radial distortion, and the ps are for tangential distortion. In the documentation of cv2, you could find the detailed definitions and descriptions of these parameters.

However, it seems to me that camera_calibration package only gives you 5 parameters. These parameters in a YAML file look like the following figure.

We have to figure out what is the relationship between these parameters in the YAML file and the parameters defined in an OpenCV distortion vector. The clue lies in the “distortion_model” entry in the YAML file. It says that camera_calibration package uses a distortion model with the name of “plumb_bob”. It turns out that, after a quick Googling, plumb_bob is just a simple version of the distortion model which is utilized by OpenCV. I found a web page documenting the Plumb Bob model. The result for our converting is so simple that we just copy the 5 values that in the YAML file into a numpy vector, one by one without any modification of the ordering of these values.

Another point that I would like to mention is that for my current cv2 (version 3.3.1-dev), the calibration API function produces a vector of a length 14 to hold the distortion vector. I am not sure why it has to be 14. However, I just allocate a vector of length of 14 with all its elements being zero. Then this 14-element long vector gets filled up by the 5 values from the YAML file, leaving other elements of the vector remaining zero and not touched.

I write a simple python script to do the job. This script requires at least two arguments to work. These arguments define the folder where you keep the input YAML files and define the output directory. There are a couple of optional arguments that control the naming of the input and output files. Using - -help to get the explanation of each argument. The script as follows. The use of sensor_msgs.msg.CameraInfo from ROS just has no relevance to the converting task. Please ignore any confusion brought by sensor_msgs.msg.CameraInfo.

Image courtesy: Cover image is copied from OpenCV documentation.