ToF/RGB Sensor Fusion for 3-D Endoscopy

Acquisition of 3-D anatomical structure in minimally invasive surgery is an important step towards intraoperative guidance. In this context, the first prototype of a Time-of-Flight/RGB endoscope was engineered for simultaneous range and color data acquisition. Intrinsic and stereo camera calibration are essential to achieve an intuitive visualization of colored surfaces. Due to the early prototype stage, inhomogeneous illumination and low resolution (64x50 px) complicate the calibration significantly. To overcome these challenges, we propose a fully automatic multiscale calibration framework using a self-encoded marker for checkerboard detection. A first application demonstrates the feasibility of intra-operative measurement. Using our calibration scheme, we achieved a reprojection error of less than 0.7 px for the Time-of-Flight camera and 0.5 px of the RGB camera. Our framework eases calibration and enables future applications to use combined range and colored data.