IMKA - Multimedia Knowledge Extraction & Applications for Annotated Content | ||
|
||
|
SummaryIMKA stands for Intelligent Multimedia Knowledge Application. The project aims at representing and discovering knowledge from multimedia content (such as image collection with annotations) to enhance the classification, navigation and retrieval of multimedia.This project has reviewed research on the understanding and construction of intelligent entities and systems, respectively, in the fields of psychology, artificial intelligence, semiotics and computer vision. Based on these findings, we have defined the novel MediaNet framework that uses multimedia to represent semantic and perceptual knowledge about the world. In addition, we have proposed automatic techniques for discovering perceptual and semantic knowledge from annotated image collection by integrating both the processing of images and text. Perceptual knowledge is constructed by clustering the images based on both visual and text feature descriptors, and by discovering statistical and similarity relationships between the clusters. Semantic knowledge is constructed by disambiguating the senses of words in annotations using WordNet and the images clusters, and by finding semantic relations between the detected senses in WordNet. We have also developed novel automatic techniques for interrelating, summarizing and evaluating the discovered multimedia knowledge. Interdependence among discovered concepts are used to construct Bayesian networks for probabilistic inferencing; knowledge is summarized by grouping similar concepts together; and the conciseness, completeness and consistency of knowledge is evaluated using notions from graph and information theory. The automatically discovered knowledge also provides a systematic and efficient way in browsing and navigating through the concepts and concepts present in the image collection. PeopleAna B. Benitez: Prof. Shih-Fu ChangIn collaboration with Dr. Alex Loui of Eastman Kodak Contact: Ana B. Benitez, Prof. Shih-Fu Chang PublicationsA. B. Benitez and S.-F. Chang, Image Classification Using Multimedia Knowledge Networks, Proceeding of the International Conference on Image Processing (ICIP-2003), Barcelona, Spain, Sep 14-17, 2003.(Paper: PDF) A. B. Benitez and S.-F. Chang, Organization and Browsing of Annotated Images Using Multiresolution Knowledge Networks, Columbia University ADVENT Technical Report #007, 2002. A.
B. Benitez and S.-F. Chang, Multimedia
Knowledge Integration, Summarization and Evaluation, Proceedings of
the 2002 International Workshop on Multimedia Data Mining in conjuction
with the International Conference on Knowledge Discovery & Data Mining
(MDM/KDD-2002), Edmonton, Alberta, Canada, July 23-26, 2002; also Columbia
University ADVENT Technical Report #003, 2002.
A.
B. Benitez and S.-F. Chang, Semantic
Knowledge Construction from Annotated Image Collections, Proceedings
of the 2002 International Conference On Multimedia & Expo (ICME-2002),
Lausanne, Switzerland, Aug 26-29, 2002; also Columbia University ADVENT
Technical Report #002, 2002.
A.
B. Benitez and S.-F. Chang, Perceptual
Knowledge Construction from Annotated Image Collections, Proceedings
of the 2002 International Conference On Multimedia & Expo (ICME-2002),
Lausanne, Switzerland, Aug 26-29, 2002; also Columbia University ADVENT
Technical Report #001, 2002.
A. B. Benitez,
S.-F. Chang, and J. R. Smith, IMKA:
A Multimedia Organization System Combining Perceptual and Semantic Knowledge,
Proceeding of the 2001 ACM International Conference on Multimedia (ACM
MM-2001), Canada, Ottawa, Sep 30-Oct 5, 2001.
A.
B. Benitez and J. R. Smith, New
Frontiers for Intelligent Content-Based Retrieval, Proceedings of the
2001 SPIE Conference on Storage and Retrieval for Media Databases (IS&T/SPIE-2001),
Vol. 4315, San Jose, CA, Jan 24-26, 2001.
A.
B. Benitez, J. R. Smith, and S.-F. Chang, MediaNet:
A Multimedia Information Network for Knowledge Representation, Proceedings
of the 2000 SPIE Conference on Internet Multimedia Management Systems (IS&T/SPIE-2000),
Vol. 4210, Boston, MA, Nov 6-8, 2000.
For problems
or questions regarding this web site contact The
Web Master.
|