End-to-End Artificial Intelligence-Based System for Automatic Stereo Camera Self-Calibration

Abstract

Stereo camera self-calibration is a complex challenge in computer vision applications such as robotics, object tracking, surveillance and 3D reconstruction. To address this, we propose an efficient, fully automated End-To-End AI-Based system for automatic stereo camera self-calibration with varying intrinsic parameters, using only two images of any 3D scene. Our system combines deep convolutional neural networks (CNNs) with transfer learning techniques and fine-tuning. First, our end-to-end convolutional neural network optimized model begins by extracting matching points between a pair of stereo images. These matching points are then used, along with their 3D scene correspondences, to formulate a non-linear cost function. Direct optimization is subsequently performed to estimate the intrinsic camera parameters by minimizing this non-linear cost function. Following this initial optimization, a fine-tuning layer refines the intrinsic parameters for increased accuracy. Our hybrid approach is characterized by a special optimized architecture that leverages the strengths of end-to-end CNNs for image feature extraction and processing, as well as the pillars of our nonlinear cost function formulation and fine-tuning, to offer a robust and accurate method for stereo camera self-calibration. Extensive experiments on synthetic and real data demonstrate the superior performance of the proposed technique compared to traditional camera self-calibration methods in terms of precision and faster convergence.

Keywords

• Camera self-calibration
• varying intrinsic parameters
• nonlinear direct optimization
• convolutional neural network
• transfer learning
• fine tuning