Visual Localization in Underground Mines and Indoor Environments using PnP

Abstract

This thesis presents a visual technique for localization in underground mines and indoor environments that exploits the use of a calibrated monocular camera. The objective of this work is to provide 6 degrees-of-freedom (6 DoF) localization in the presence of a 3D map made up of 3D point clouds co-registered with intensity information. An efficient data structure, referred to as the feature database, is used to store the image features appearing in the 3D map, and to efficiently retrieve a match for a query image during localization. The Perspective-n-Point (PnP) problem is then used to compute the 6 DoF position of the calibrated camera at the time of image capture given the location of the 2D query image features and their 3D coordinates.

Two experiments were performed using this technique: ground truth experiments and localization experiments. The ground truth experiments, performed on two datasets, compare the localization output against a reference position. The results of the ground truth experiments indicate that the calculated camera position using the proposed technique approximates the reference position, with small errors on the order of millimetres. It was also found that the accuracy of localization increases as more inliers become available in the query image. The localization experiments were performed on datasets collected in underground mines and indoor environments to test the performance of the technique in different environments. The query images used in the localization experiments were captured with two different cameras, demonstrating that any type of monocular camera may be used during localization, as long as a sufficient number of environmental features can be extracted from the query images.