Improving forward mapping and disocclusion inpainting algorithms for depth-image-based rendering and geomatics applications

Abstract

Depth-image-based Rendering (DIBR) is a promising technique to generate appealing and immersive contents for virtual reality image mapping. When mapping a reference image to a virtual image, two challenges exist: (1) the most widely used forward mapping algorithm is prone to generating synthetic artifacts; and (2) the existing disocclusion inpainting algorithms are not able to fill big holes in a visually plausible way. This research conducted a comprehensive investigation on the forward mapping algorithm and developed a new algorithm that can eliminate all the reported artifacts of small cracks. This research also developed a novel directional and adaptive hole-filling algorithm using an exemplar-based approach to fill big holes. The performance of the developed algorithms was validated using images from a selected benchmark database. Quantitative and qualitative assessments proved that the developed algorithms have clearly outperformed the state-of-the-art algorithms. The developed algorithms were applied to outdoor/indoor mapping applications to prove the concept. It validated the feasibility of using the developed algorithms to improve user experiences on existing Street View technologies.