Faculty of Engineering and Natural Sciences
CONTENT CREATION FOR 3D-TV APPLICATIONS USING VIDEO OBJECT TRACKING
Dr. Çiğdem Eroğlu Erdem
Momentum A. Ş.
The task of building 3D models of a time-varying scene, using 2D views recorded by uncalibrated cameras is an important but unsolved task to provide content for the newly emerging 3D TV. One approach to this problem is to segment the objects in the scene and order their video object planes (VOPs) with respect to their inferred relative depths. This approach gives a satisfactory sense of three dimensions when the scene is viewed in stereo. However, one of the most important requirements is the temporal stability of the video object planes. The changes in video due to occlusions, camera motion, changing background and noise should not cause sudden changes (temporal instabilities) in the shape and color composition of the video object planes as they cause very disturbing flickering effects when the scene is viewed in stereo in 3D TV applications. However, existing object tracking algorithms inevitably loose temporal stability under difficult conditions, e.g. when the colors of the object and the background are similar causing missing object boundaries or when the motion can not be estimated with sufficient accuracy.
In this work, we try to answer the question: ``If making object segmentation errors are inevitable, how can we conceal them in our application?" Our approach is based on the hypothesis that, if making segmentation errors are inevitable, they should be done in a temporally consistent way to increase the viewing comfort in 3D TV applications. To this effect, we propose a pseudo-3D curve evolution technique, which distributes the existing segmentation errors such that they will be less visible when the scene is rendered and viewed in stereo. It has been shown by experiments that the proposed algorithm significantly improves the temporal stability in terms of two quantitative objective measures based on histogram and shape differences. Subjective evaluation tests indicate that there is an improvement in the perceived quality of the scene when viewed in 3D, which also validates the effectiveness of the proposed quantitative measures. Hence, it is possible to increase the object segmentation quality without increasing the segmentation accuracy. An active contour based object segmentation algorithm, which directly optimizes the temporal stability measures is also developed and some experimental results will be presented.
Another way to provide content for 3D-TV is to use semi-synthetically generated 3D models of the real world. To this effect, research and development activities are carried out at Momentum A.S. on 3D photo-realistic human face modeling and animation. The last part of the talk will briefly introduce these technologies and their applications including a 3D lip reading tool for hard of hearing people and a 3D PC adventure game.
June 29, 2006, 10:40, FENS G032