A fully automatic depth estimation algorithm for multi-focus plenoptic cameras: coarse and dense aproaches

Abstract

Light field cameras capture a scene’s multi-directional light field with one image, allowing the estimation of depth of the captured scene and focus the image after it has been taken. In this thesis, we introduce a fully automatic method for depth estimation from a single plenoptic image running a RANSAC-like algorithm for feature matching. We filter the estimated depth points on a global and fine scale, allowing a more accurate depth estimation. The novelty about our approach is the global method to back project correspondences found using photometric similarity to obtain a 3D virtual point cloud. We use a smart mixture of lenses with different focal-lengths in a multiple depth map refining phase, generating a dense depth map. This depth map is then used to generate very high quality all-in-focus renders. We also introduce a new method for detection and correction of highly blurred areas, which greatly improves the depth estimation of the scene and subsequently the all-in-focus as well. As far as the author knows, our algorithm is the first fully automatic (zero intervention) method to process multi-focus plenoptic images. On the previous work a plenoptic data simulator was introduced which allows us to create plenoptic datasets with specific parameters. Knowing the depth ground truth of these datasets we are able to test and improve our algorithm and provide guidelines for future work. Tests with simulated datasets and real images are presented and show very good accuracy of the method presented. We also compare our results with other methods, being able to achieve comparable results to the state of the art with substantial less processing time. A short paper was submitted and accepted to Eurographics 2016, the 37th Annual Conference of the European Association for Computer Graphics and a full paper was also submitted to ICCP 2016, an International Conference on Computer Photography.