Multimedia Knowledge Integration, Summarization and Evaluation

Ana B. Benitez and Shih-Fu Chang


This paper presents new methods for automatically integrating, summarizing and evaluating multimedia knowledge. These are essential for multimedia applications to efficiently and coherently deal with multimedia knowledge at different abstraction levels such as perceptual and semantic knowledge (e.g., image clusters and word senses, respectively). The proposed methods include automatic techniques (1) for interrelating the concepts in the multimedia knowledge using probabilistic Bayesian learning, (2) for reducing the size of multimedia knowledge by clustering the concepts and collapsing the relationships among the clusters, and (3) for evaluating the quality of multimedia knowledge using notions from information and graph theory. Experiments show the potential of knowledge integration techniques for improving the knowledge quality, the importance of good concept distance measures for clustering and summarizing knowledge, and the usefulness of automatic measures for comparing the effects of different processing techniques on multimedia knowledge.

Keywords: Multimedia knowledge, knowledge integration, knowledge summarization, knowledge evaluation, concept distance, concept clustering, Bayesian networks