Posts Tagged ‘3d video’

K. Müller, A. Smolic, K. Dix, P. Merkle, P. Kauff, and T. Wiegand, “View Synthesis for Advanced 3D Video Systems”, EURASIP Journal on Image and Video Processing, 2009.

Abstract: The paper is about 3D video applications and systems based on multiview autostereoscopic displays. These are expected to play an important role in home user environments, since they support multi-user 3D sensation and motion parallax impression. The tremendous data rate cannot be handled efficiently by representation and coding formats such as MVC or MPEG-C Part 3. Multiview video plus depth is a new format that efficiently supports such advanced 3DV systems, but this requires high quality intermediate view synthesis. For this, a new approach is presented that separates unreliable image regions along depth discontinuities from reliable image regions, which are treated separately and fused to the final interpolated view. In contrast to previous layered approaches, our algorithm uses two boundary layers and one reliable layer, performs image-based 3D warping only and was generically implemented, i.e. doesn’t necessarily rely on 3D graphics support. Furthermore, different hole-filling and filtering methods are added to provide high-quality intermediate views. As a result, high quality intermediate views for an existing 9-view auto-stereoscopic display as well as other stereo- and multiscopic displays are presented, which prove the suitability of our approach for advanced 3DV systems.

K. Müller, A. Smolic, K. Dix, P. Merkle, P. Kauff, and T. Wiegand,”Reliability-based Generation and View Synthesis in Layered Depth Video“, Proc. MMSP 2008, IEEE International Workshop on Multimedia Signal Processing, Cairns, Australia, October 2008.

Abstract: In this paper, a system for video rendering on multiscopic 3D displays is considered where the data is represented as layered depth video (LDV). This representation consists of one full or central video with associated per-pixel depth and additional residual layers. Thus, only one full view with additional residual data needs to be transmitted. The LDV data is used at the receiver to generate all intermediate views for the display. The paper presents the LDV layer extraction as well as the view synthesis, using a scene reliability-driven approach. Here, unreliable image regions are detected and in contrast to previous approaches the residual data is enlarged to reduce artifacts in unreliable areas during rendering. To provide maximum data coverage, the residual data remains at its original positions and will not be projected towards the central view. The view synthesis process also uses this reliability analysis to provide higher quality intermediate views than previous approaches. As a final result, high quality intermediate views for an existing 9-view auto-stereoscopic display are presented, which prove the suitability of the LDV approach for advanced 3D video (3DV) systems.

A. Smolic, K. Müller, K. Dix, P. Merkle, P. Kauff, and T. Wiegand, ” Intermediate View Interpolation Based on Multiview Video Plus Depth for Advanced 3D Video Systems “, Proc. ICIP 2008, IEEE International Conference on Image Processing, San Diego, CA, USA, October 2008.

Abstract: A system for video on multiscopic 3D displays is considered where the data representation consists of multiview video plus scene depth. At most, 3 multiview video signals are being transmitted and used together with the depth data to generate intermediate views at the receiver. The paper presents an approach to such an intermediate view interpolation that separates unreliable image regions along depth discontinuities from reliable image regions. These image regions are processed with different algorithms and previous layered approaches, two boundary layers and one reliable layer is used. Moreover, the presented technique does not rely on 3D graphics support but uses image-based 3D warping instead. For enhanced quality intermediate view generation, hole-filling and filtering methods are described. As a result, high quality intermediate views for an existing 9-view auto-stereoscopic display are presented, which prove the suitability of the approach for advanced 3D video (3DV) systems.

This document reports first study results for 3D video solutions. The different video formats that were defined in D5.1 are used for this, including three depth-based formats, namely single- and multiview video plus depth and layered depth video, and conventional stereo video. Study results and solutions presented in this deliverable address analysis and synthesis as well as coding algorithms for 3D video. Regarding conventional stereo video, coding approaches are analyzed. For video plus depth results on stereo video rendering for mobile phone video applications are evaluated. For MVD the report presents an extensive study on depth coding and appropriate evaluation methods as well as high quality rendering algorithms. Furthermore extraction and rendering algorithms for layered depth video are investigated. Finally, conclusions and future prospects of study results for 3D video solutions are presented.

Download Document

This deliverable first introduces different 3D video application scenarios on mobile phones, as for instance real-time 3D video communication or 3D video playback. Besides general low complexity requirements, which always apply for mobile phone video applications, the different 3D video application scenarios impose different requirements, which are presented. Then, different 3D video formats are introduced along with related algorithms, which are available and under development. The different 3D video formats have different advantages and drawbacks, which are analysed. An initial assessment of the 3D video formats versus applications scenarios is given, analysing which format might be suitable for which application scenario. Finally, a broad work plan for WP5 is presented.

Download Document