Content-based
search and retrieval
Content-based search and retrieval
is a comprehensive topic. Its main focuses include extraction, representation
and matching of spatial-temporal features; event detection and classification
based on low-level features and domain knowledge; segmentation and abstraction
at multiple levels (shot, scene, topic), and relevance feedback for user interaction. The following include a
few recent surveys in this field.
[1]Shih-Fu Chang, Qian Huang , Thomas
Huang, Atul Puri, and Behzad Shahraray, Multimedia Search and Retrieval, Book
chapter, in Advances in Multimedia: Systems, Standards, and Networks, A. Puri
and T. Chen (eds.). New York: Marcel Dekker, 1999.
[2]Smoliar, S.W.; HongJiang Zhang,
Content based video indexing and retrieval, IEEE Multimedia , Volume: 1 Issue:
2 , Summer 1994, Page(s): 62 -72
[3]S.-F. Chang, J. R. Smith, M. Beigi,
and A. B. Benitez, Visual Information Retrieval from Large Distributed On-line
Repositories, Communications of the ACM, Vol. 40, No. 12, pp.63-71, Dec 1997.
Sports video analysis
Sports video has inherent structure
constraint as defined in rules of the game and field production. And we
believe this structure makes it easier to explore the correlation and interaction
of domain constraints, low-level features, and high-level semantics.
Prior works include domain-specific
scene classification[4], classification
of audio track into excited/unexcited commentary[6],
audio event spotting via template matching[5][6],
interactive browsing via object tracking[7], slow-motion
detection by still-frame identification in MPEG stream[9],
and incorporation of field model and object tracking[8].
[4]Yihong Gong; Lim Teck Sin; Chua Hock
Chuan; Hongjiang Zhang; Masao Sakauchi,Automatic parsing of TV soccer
programs, Multimedia Computing and Systems, 1995., Proceedings of the
International Conference on , 1995, Page(s): 167 -174
[5]Yuh-Lin Chang; Wenjun Zeng; Kamel,
I.; Alonso, R., Integrated image and speech analysis for content-based video
indexing, Multimedia Computing and Systems, 1996., Proceedings of the Third
IEEE International Conference on , 1996, Page(s): 306 -313
[6]Yong Rui; Anoop Gupta; Alex Acero;
Automatically Extracting Highlights for TV Baseball Programs, Proc. ACM
Multimedia, Oct. 2000, Los Angeles USA, Pages 105 -115
[7]David Rees, Johnson I Agbinya, Nick
Stone, Fu Chen, CLICK-IT: Interactive Television Highlighter for Sports Action
Replay, Pattern Recognition, 1998. Proceedings. Fourteenth International
Conference on , Volume: 2 , 1998
Page(s): 1484 -1487 vol.2
[8]Sudhir, G.; Lee, J.C.M.; Jain, A.K.,
Automatic classification of tennis video for high-level content-based
retrieval, Content-Based Access of Image and Video Database, 1998.
Proceedings., 1998 IEEE International Workshop on , 1998, Page(s): 81 -90
[9]Kobla, V.; DeMenthon, D.; Doermann,
D., Detection of slow-motion replay sequences for identifying sports videos,
Multimedia Signal Processing, 1999 IEEE 3rd Workshop on , 1999, Page(s): 135
-140
[10]
Zhong,
D; Chang, S.-F., Structure Analysis of Sports Video Using Domain Models, IEEE
ICME 2001, Aug. 2001, Tokyo, Japan
[11]
V. Tovinkere , R. J. Qian, “Detecting Semantic Events in Soccer
Games: Towards A Complete Solution”, Proc. ICME 2001, Tokyo, Japan, Aug
22-25, 2001
[12]
D. Yow, B.L.Yeo, M. Yeung, and G. Liu, "Analysis and Presentation
of Soccer Highlights from Digital Video" Proc. ACCV, 1995, Singapore,
Dec. 5-8, 1995
Probabilistic content analysis
Probabilistic reasoning enable inference
based on computable audio-visual features and domain-specific knowledges.
Related works include using Beyesian network to identify multimedia objects
and infer multimedia concepts[11, 12],
and exploiting dynamic programming techniques or hidden Markov model to
distinguish different types of TV program[13].
[13]
L.
R. Rabiner, “A tutorial on hidden Markov models and selected applications in
speech recognition”, Proceedings of the IEEE, v 77 no 2 , P 257 –286, Feb.
1989
[14]
Seungyup
Paek and Shih-Fu Chang, The Case for Image Classification Systems Based on
Probabilistic Reasoning, IEEE International Conference on Multimedia and Expo.
July 30 - August 2, 2000. New York City, NY, USA.
[15]
Naphade,
M.R.; Huang, T.S., A probabilistic framework for semantic video indexing,
filtering, and retrieval, Multimedia, IEEE Transactions on , Volume: 3 Issue:
1 , March 2001, Page(s): 141 -151
[16]
Jincheng
Huang; Zhu Liu; Yao Wang, Joint video scene segmentation and classification
based on hidden Markov model, Multimedia and Expo, 2000. ICME 2000. 2000 IEEE
International Conference on , Volume: 3 , 2000, Page(s): 1551 -1554 vol.3
Front End
Effective and accurate segmentation
and feature extraction is crucial to the performance of the whole content
analysis system. Temporal segmentation output usually include shots, scenes,
or other meaningful units. Video data can be further processed to segment
video into regions or objects from which useful spatio-temporal features
(such as trajectory, motion) can be extracted.
[17]
Di
Zhong; Shih-Fu Chang, An integrated approach for content-based video object
segmentation and retrieval, Circuits and Systems for Video Technology, IEEE
Transactions on , Volume: 9 Issue: 8 , Dec. 1999, Page(s): 1259 -1268
[18]
J.
Meng and S.-F. Chang, Tools for Compressed-Domain Video Indexing and Editing,
Proceedings, IS&T/SPIE Symposium on Electronic Imaging: Science and
Technology (EI'96) -Storage & Retrieval for Image and Video Databases IV,
Vol. 2670, San Jose, CA, February 1996.
[19]
Yap-Peng
Tan; Saur, D.D.; Kulkami, S.R.; Ramadge, P.J., Rapid estimation of camera
motion from compressed video with application to video annotation, Circuits
and Systems for Video Technology, IEEE Transactions on , Volume: 10 Issue: 1 ,
Feb. 2000, Page(s): 133 -146
[20]
Di
Zhong; Shih-Fu Chang, AMOS: an active system for MPEG-4 video object
segmentation, Image Processing, 1998. ICIP 98. Proceedings. 1998 International
Conference on , Volume: 2 , 1998, Page(s): 647 -651 vol.2
[21]
J.S.
Boreczky, and L.A. Rowe, Comparison of Video Shot Boundary Detection
Techniques, Storage and Retrieval for Image and Video Databases IV, Proc. of
IS&T/SPIE 1996 Int'l Symp. on Elec. Imaging: Science and Technology, San
Jose, CA, February 1996
[22]
D.
Zhong and S.-F. Chang, Video Object Model and Segmentation for Content-Based
Video Indexing, IEEE International Symposium on Circuits and Systems
(ISCAS'97), Hong Kong, June 1997, Special Session on Networked Multimedia
Technology and Application
Project: Sports Video Analysis