Novel experimental and software methods for image reconstruction and localization in capsule endoscopy

Abstract

Background and study aims Capsule endoscopy (CE) is invaluable for minimally invasive endoscopy of the gastrointestinal tract; however, several technological limitations remain including lack of reliable lesion localization. We present an approach to 3D reconstruction and localization using visual information from 2D CE images.

Patients and methods Colored thumbtacks were secured in rows to the internal wall of a LifeLike bowel model. A PillCam SB3 was calibrated and navigated linearly through the lumen by a high-precision robotic arm. The motion estimation algorithm used data (light falling on the object, fraction of reflected light and surface geometry) from 2D CE images in the video sequence to achieve 3D reconstruction of the bowel model at various frames. The ORB-SLAM technique was used for 3D reconstruction and CE localization within the reconstructed model. This algorithm compared pairs of points between images for reconstruction and localization.

Results As the capsule moved through the model bowel 42 to 66 video frames were obtained per pass. Mean absolute error in the estimated distance travelled by the CE was 4.1 ± 3.9 cm. Our algorithm was able to reconstruct the cylindrical shape of the model bowel with details of the attached thumbtacks. ORB-SLAM successfully reconstructed the bowel wall from simultaneous frames of the CE video. The “track” in the reconstruction corresponded well with the linear forwards-backwards movement of the capsule through the model lumen.

Conclusion The reconstruction methods, detailed above, were able to achieve good quality reconstruction of the bowel model and localization of the capsule trajectory using information from the CE video and images alone.

• References

• 1 Iakovidis DK, Koulaouzidis A. Software for enhanced video capsule endoscopy: challenges for essential progress. Nat Rev Gastroenterol Hepatol 2015; 12: 172-186
• 2 Grove PM. The Psychophysics of binocular vision. In: 3D-TV System with Depth-Image-Based Rendering. New York, NY: Springer New York; 2013: 347-373
• 3 Van Rijn JC, Reitsma JB, Stoker J. et al. Polyp miss rate determined by tandem colonoscopy: A systematic review. Am J Gastroenterol 2006; 101: 343-350
• 4 Baptista V, Marya N, Singh A. et al. Continuing challenges in the diagnosis and management of obscure gastrointestinal bleeding. World J Gastrointest Pathophysiol 2014; 5: 523-533
• 5 Spyrou E, Iakovidis DK. Video-based Measurements for Wireless Capsule Endoscope Tracking. Meas Sci Technol 2014; 25: 14
• 6 Spyrou E, Iakovidis DK, Niafas S. et al. Comparative assessment of feature extraction methods for visual odometry in wireless capsule endoscopy. Comput Biol Med. 2015; 65: 297-307
• 7 Kannala J, Brandt SS. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans Pattern Anal Mach Intell 2006; 28: 1335-1340
• 8 Cool S, Pieters JG, Mertens KC. et al. Development of a high irradiance LED configuration for small field of view motion estimation of fertilizer particles. Sensors 2015; 15: 28627-28645
• 9 Ciuti G, Visentini-Scarzanella M, Dore A. et al. Intra-operative monocular 3D reconstruction for image-guided navigation in active locomotion capsule endoscopy. InBiomedical Robotics And Biomechatronics (Biorob), 2012 4th IEEE Ras & Embs International Conference. 2012: 768-774
• 10 Visentini-Scarzanella M, Stoyanov D, Yang G-Z. Metric depth recovery from monocular images using Shape-from-Shading and specularities. In: 2012 19th IEEE International Conference on Image Processing. 2012: 25-28
• 11 Mur-Artal R, Montiel JM, Tardos JD. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Transactions on Robotics 2015; 31: 1147-1163
• 12 Rosten E, Drummond T. Machine learning for high-speed corner detection. Computer Vision–ECCV 2006 2006; 430-443
• 13 Calonder M, Lepetit V, Strecha C. et al. Brief: Binary robust independent elementary features. Computer Vision–ECCV 2010 2010; 778-792
• 14 Lo SK, Mehdizadeh S, Ross A. et al. How Should We Do Capsule Reading?. Tech Gastrointest Endosc 2006; 8: 146-148
• 15 Keuchel M, Kurniawan N, Baltes P. et al. Quantitative measurements in capsule endoscopy. Comput Biol Med 2015; 65: 333-347
• 16 Than TD, Alici G, Zhou H. et al. A review of localization systems for robotic endoscopic capsules. IEEE Trans Biomed Eng 2012; 59: 2387-2399
• 17 Sakata S, Grove PM, Stevenson ARL. et al. The impact of three-dimensional imaging on polyp detection during colonoscopy: a proof of concept study. Gut 2016; 65: 730-731