Summary 

Skims are highly condensed clips (say 10 seconds) of long video sequences. They are very useful for content browsing and on-demand summaries. Conventional methods for skim generation primarily relied on text matching or object detection, but did not consider the production syntax and human perception of audio-visual content.

In this project, we study (1) how content condensation affects perceptual quality (2) how production syntax can be preserved and (3) how audio and visual media should be condensed jointly.

One key contribution of ours is the formulation of a rigorous utility-based framework for optimal skim determination, rather than relying on heuristic rules. Subjective experiments show performance improvement by the new method.

People

Hari Sundaram

Prof. Shih-Fu Chang 

Publication

H. Sundaram, L. Xie, S.-F. Chang, A Utility Framework for the Automatic Generation of Audio-Visual Skims, ACM Multimedia, Juan Les Pins, France, December 2002 .
(Best Student Paper Award)
(PS.GZ/PDF)

H. Sundaram, Segmentation, Structure Detection and Summarization of Multimedia Sequences, Doctoral Dissertation, Graduate School of Arts and Sciences, Columbia University, 2002 (Advisor: Prof. Chang).
(PS.GZ/PDF)

H. Sundaram, S.-F. Chang, Constrained Utility Maximization for generating Visual Skims, iEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'2001) Dec. 2001 Kauai, HI USA.
(PS.GZ/PDF)

H. Sundaram and S.-F Chang, Condensing Computable Scenes using Visual Complexity and Film Syntax Analysis, IEEE Conference on Multimedia and Exhibition, Tokyo, Japan, Aug. 22-25, 2001.
(PS.GZ/PDF)

Demo

Download

For problems or questions regarding this web site contact The Web Master.
Last updated: June 12, 2002.