Lexing Xie, Lyndon Kennedy, Shih-Fu Chang, Ajay Divakaran, Huifang Sun, Ching-Yung Lin. Discovering meaningful multimedia patterns with audio-visual concepts and associated text. In IEEE International Conference on Image Processing (ICIP), Singapore, October 2004.

Download [help]

Download paper: Adobe portable document (pdf)

Copyright notice:This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


The work presents the first effort to automatically annotate the semantic meanings of temporal video patterns obtained through unsupervised discovery processes. This problem is interesting in domains where neither perceptual patterns nor semantic concepts have simple structures. The patterns in video are modeled with hierarchical hidden Markov models (HHMM), with efficient algorithms to learn the parameters, the model complexity, and the relevant features; the meanings are contained in words of the speech transcript of the video. The pattern-word association is obtained via co-occurrence analysis and statistical machine translation models. Promising results are obtained through extensive experiments on 20+ hours of TRECVID news videos: video patterns that associate with distinct topics such as el-nino and politics are identified; the HHMM temporal structure model compares favorably to a non-temporal clustering algorithm.


Lexing Xie
Shih-Fu Chang

BibTex Reference

   Author = {Xie, Lexing and Kennedy, Lyndon and Chang, Shih-Fu and Divakaran, Ajay and Sun, Huifang and Lin, Ching-Yung},
   Title = {Discovering meaningful multimedia patterns with audio-visual concepts and associated text},
   BookTitle = {IEEE International Conference on Image Processing (ICIP)},
   Address = {Singapore},
   Month = {Oct},
   Year = {2004}

EndNote Reference [help]

Get EndNote Reference (.ref)

This document was translated automatically from BibTEX by bib2html (Copyright 2003 © Eric Marchand, INRIA, Vista Project).