Blog

DOCTORAL THESIS: Automated 3D object recognition in underwater scenarios for manipulation

By Khadidja Himri

Supervised by Dr.Pere Ridao / Dr.Nuno Gracias

 

Abstract

In recent decades, the rapid development of intelligent vehicle and 3D scanning technologies has led to a growing interest in applications based on 3D point data processing, with many applications such as augmented reality or robot manipulation and obstacle avoidance, scene understanding, robot navigation, tracking and assistive technology among others, requiring an accurate solution for the 3D pose of the recognized objects. Thus object recognition is becoming an important topic in computer vision, where machine vision and robotics techniques are becoming key players.

 

In this thesis work, the main objective is to develop a semantic mapping method by integrating a 3D object recognition pipeline with a feature-based SLAM system, in order to assist autonomous underwater interventions in the near future.

 

To this end, the work proposed in this paper targets three axes. First, it aims to compare the performance of 3D global descriptors within the state of the art, focusing on those based on point clouds and targeted at real-time object recognition applications. For this purpose, we selected a set of test objects representative of Inspection, Maintenance and Repair (IMR) applications and whose shape is usually known a priori. Their CAD models were used to: 1) create a data base of synthetic object views used as a priori knowledge, and 2) simulate the point clouds that would be gathered during the scanning under realistic conditions, with added noise and varying resolution. Extensive experiments were performed with both virtual scans and real data collected with an AUV equipped with a fast laser scanner developed at our research centre.

 

The second goal of our work was to use a real-time laser scanner mounted on an AUV to detect, identify, and locate objects in the robot’s environment, with the aim of allowing an intervention Autonomous Underwater Vehicle (I-AUV) to know what manipulation actions could be performed on each object. This goal was tackled by the design and development of a 3D object recognition method for uncolored point clouds (laser scans) using point features. The algorithm uses a database of partial views of the objects stored as point clouds. The recognition pipeline includes 5 stages: 1) Plane segmentation, 2) Pipe detection, 3) Semantic Object-segmentation, 4) Feature-based Object Recognition and 5) Bayesian estimation. To apply Bayesian estimation, it is necessary to track objects across scans. For this purpose, the Inter-distance Joint Compatibility Branch and Bound (IJCBB) data association algorithm was proposed based on the distances between objects. The performance of the method was tested using a dataset of the inspection of a pipe infrastructure made of PVC objects connected by pipes. The structure is representative of those commonly used by the offshore industry. Experimental results show that Bayesian estimation improves the recognition performance with respect to the case where only the 1 descriptor is used. The inclusion of semantic information about object pipe connectivity further improves recognition performance.

 

The final goal of the thesis, consists of integrating the 3D object recognition system with a feature-based SLAM system to implement a semantic map providing the robot with information about the location and the type of objects in its surroundings. The SLAM improved both the accuracy and reliability of pose estimates of the robot and the objects. This is especially important in challenging scenarios where significant changes in viewpoint and appearance arise.