Why adding an IMU to camera-based odometry? An IMU improves accuracy and robustness of camera-based odometry.

My setup

• monochrome global shutter camera: mvBlueFox-MLC200wG (ON Semiconductor MT9V034 digital image sensor)
  important: time-synchronized to IMU
  
• 132 degree fisheye lense
• IMU: GY-88 (MPU6050)
• rovio for visual inertial odometry
• kalibr for camera-to-IMU calibration

Checkerboard

We will need to print out a checkerboard pattern for camera calibration:

Camera intrinsics

Using the checkerboard pattern, find out the intrinsics of your camera. See ‘How to Calibrate a Monocular Camera‘.

rosrun camera_calibration cameracalibrator.py –size 9×6 –square 0.028 image:=/cam0/image_raw camera:=/cam0

Pinhole camera model:
fu, fv: focal-length (e.g. 410, 412)
pu, pv: principal point (e.g. 374, 243)

Distortion model: radial-tangential (radtan):
k1, k2, r1, r2: distortion coeffs (e.g. -0.2, 0.07, 0, 0)

camchain.yaml

cam0:
  camera_model: pinhole
  intrinsics: [fu fv pu pv]
  distortion_model: radtan
  distortion_coeffs: [k1 k2 r1 r2]
  rostopic: /cam0/image_raw
  resolution: [752, 480]

AprilTags

We will need to print out 6×6 AprilTag patterns (DINA0) for the camera-to-IMU calibration:

Camera-to-IMU calibration

This will calculate the camera extrinsics (rotations, translations for the camera with respect to the IMU).

Using 6×6 AprilTag patterns, record slow translations/rotations of the fixed camera+IMU sensor into a bagfile.

Run the camera-to-IMU calibration:

kalibr_calibrate_imu_camera –bag recording.bag –cam camchain.yaml –imu imu.yaml –target target.yaml

Verify the calibration errors, they all should be below 0:

Reprojection error (cam0) [px]:     mean 2.7, median 2.31432653786, std: 1.8
Gyroscope error (imu0) [rad/s]:     mean 0.007, median 0.007, std: 0.003
Accelerometer error (imu0) [m/s^2]: mean 0.02, median 0.01, std: 0.02

Convert the calibration results to rovio config file:
kalibr_rovio_config –cam camchain.yaml

rovio

Run rovio for a static scene (not moving). If the position drifts, increase initial covariances and prediction noise.

Test on a recording with faster moving. I experience drift now that I will try to solve.

to be continued/refined…