Faculty of Engineering and Natural Sciences
Hallucinating Faces in Low-Resolution Videos
Carnegie Mellon University
"Face Hallucination" aims to recover high quality, high resolution images of human faces from low-resolution, blurred, and degraded images or video. We approach this ill-posed problem with domain-specific models and priors that can generate plausible high-frequency image details. In this talk, I will present some of our recent formulations and algorithms.
I will first introduce a parametric model-fitting approach that uses Active Appearance Models (AAM). After exposing the shortcomings of the traditional AAM fitting in low-resolution scenarios, we develop a "resolution-aware" fitting criterion. By explicitly accounting for the finite size sensing elements of digital cameras, our fitting method simultaneously models the processes of object appearance variation, geometric deformation, and image formation. In low-resolution this results in significantly more accurate tracking compared to state-of-the-art algorithms.
In the second part, I will present a data-driven approach with a non-parametric video model. By treating a video as a composition of space-time patches, we are able to efficiently represent and reason about complex visual phenomena such as eye-blinks and the occlusion or appearance of teeth. We also exploit this space-time representation to define a data-driven prior on a 3-dimensional Markov Random Field. We augment this setup with a global illumination model and pose Face Hallucination as a probabilistic inference problem.
Publications and demonstration videos can be found at
Göksel Dedeoğlu is a PhD candidate in the Robotics Institute at Carnegie Mellon University. He received a B.S. in Control and Computer Engineering from Istanbul Technical University in 1997, and an M.S. in Computer Science from the University of Southern California in 2000. His current research focuses on enhancing low-resolution face videos by means of space-time models and priors.
December 27, 14:40, FENS G035