Intra-object segmentation using depth information

Abstract

One of the emerging challenges in 3D technologies, such as augmented reality, is the ability to thoroughly select an object in a given scene. Most approaches deal with the selection of a 2D object making it difficult for the computing device to efficiently process the end result and to render a newly blended object into the scene without perceivable artefacts in the effected region. This paper presents a solution that takes advantage of the texture and depth information in the process of representing and therefore accurately selecting an object in a scene. Moreover, the technique allows for further segmentation of the selected object, referred to as `intra-object segmentation', based on the depth information. The result is an object that is split in layers which facilitates the subsequent editing of the scene. Results show the efficacy of the proposed solution. The intra-object segmented output can be used as input to blending methods, such as inpainting, where these can be applied on each layer or just a selection of layers separately, allowing more control on the process.