CAMED: Complexity Adaptive Motion Estimation and Mode Decision in Low Power H.264

Project's Home Page | Current Research Areas > Pervasive Media >

Keywords Summary People Publications Links

Keywords

Low power H.264, Complexity efficient H.264, main profile, MPEG-4 Part 10 AVC

Summary

Most of today's video coding systems encode the video bit streams to achieve the best video quality while satisfying certain bitrate constraints. Nevertheless, many media application devices such as mobile handheld devices are getting smaller and lighter. The computational resources available on the handheld devices become relatively scarce, given the increasing functionalities and complexity of applications running on the devices. Therefore, recently there is growing interest in complexity (power) aware video coding solutions.

Emerging video coding standards like H.264 achieves significant advances in improving video quality, reducing bandwidth, but at the cost of greatly increased computational complexity at both the encoder and the decoder. Playing encoded videos produced by such compression standards requires major computational resources and thus power on various handheld devices that are getting increasingly popular in mobile applications.

Among the components in the decoding system, the interpolation procedure used in the motion compensation component consumes the largest computation (about 50%) due to the use of sub-pixel motion vectors. See the complexity break down below.

Break down of H.264 decoding complexity for a test sequence Foreman Diagram
(Reference: Lappalainen, V.; Hallapuro, A.; Hamalainen, TD, "Complexity of optimized H.26L video decoder implementation", vol. 13, iss. 7, IEEE Trans. Circuits Syst. Video Technol., pp. 717- 725, July 2003. )

One way to reduce this major cost is to change the coding algorithm so that the generated compressed bit streams incur less non-integer motion vectors and thus less interpolation operations. The figure below shows the basic coding architecture of H.264. While reducing the number of subpixel motion vectors, we want to select the best block mode and best motion estimation so that minimal video quality loss is introduced. The highlighted box indicates the location for applying our new algorithm, which can be incorporated into any existing H.264 encoder. Note the decoder part is not changed at all.

Our extensive experiments with different video contents, bit rates, and complexity levels show very promising results in reducing the number of interpolation by up to 60% while keeping the video quality almost intact (quality difference less than 0.2dB). For low bit rate such as 100kbps or 200kbps, the performance is even better (up to 80% MC complexity saving with less than 0.1dB impairment).

The following figure shows an example performance when applying the CAMED to the 'stefan' sequence. We can see that by reducing 60% of the motion compensation complexity, the quality degradation is nearly imperceptible (average 0.197dB in this case).

The following table summarizes the performance of complexity control at 1000kbps. Complexity control error measures the ability in achieving a specific complexity target. It is calculated as the difference between the actual resulting complexity and the target complexity, normalized by the target complexity. Complexity Saving is the percentage of the original computational cost that is removed. Quality Degradation is the quality difference (in PSNR) between the bit stream generated using original H.264 and the one using our complexity control method. These results confirm that large savings of the computational complexity (30% to 60%) can be achieved with small quality degradation (0.3dB). Improvements from different video clips are different depending on the type of the content and the complexity of the signal. The most challenging case is the 'Mobile' sequence, which has a steady camera motion (slowly panning left) and thus the SKIP/DIRECT mode is frequently used. However, even for such a challenging case, our proposed CAMED method can still achieve about 33% complexity saving while keeping the video quality more or less intact.

People

Publications

Y. Wang, S.-F. Chang, Motion Estimation and Mode Decision for Low-Complexity H.264 Decoder. Columbia University ADVENT Technical Report #210-2005-4, 2005.