Hongzhi Li, Joseph G. Ellis, Heng Ji, Shih-Fu Chang. Event Specific Multimodal Pattern Mining for Knowledge Base Construction. In Proceedings of the 24th ACM international conference on Multimedia, Amsterdam, The Netherlands, 2016.
Download paper:
Adobe portable document (pdf) 
Copyright notice:This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
Knowledge bases, which consist of a collection of entities, attributes, and the relations between them are widely used and important for many information retrieval tasks. Knowledge base schemas are often constructed manually using experts with specific domain knowledge for the field of interest. Once the knowledge base is generated then many tasks such as automatic content extraction and knowledge base population can be performed, which have so far been robustly studied by the Natural Language Processing community. However, the current approaches ignore visual information that could be used to build or populate these structured ontologies. Preliminary work on visual knowledge base construction only explores limited basic objects and scene relations. In this paper, we propose a novel multimodal pattern mining approach towards constructing a high-level "event" schema semi-automatically, which has the capability to extend text only methods for schema construction. We utilize a large unconstrained corpus of weakly-supervised image-caption pairs related to high-level events such as "attack" and "demonstration" to both discover visual aspects of an event, and name these visual components automatically. We compare our method with several state-of-the-art visual pattern mining approaches and demonstrate that our proposed method can achieve dramatic improvements in terms of the number of concepts discovered (33% gain), semantic consistence of visual patterns (52% gain), and correctness of pattern naming (150% gain)
Hongzhi Li 
Joseph Ellis
Shih-Fu Chang
@InProceedings{li2016event,
Author = {Li, Hongzhi and Ellis, Joseph G. and Ji, Heng and Chang, Shih-Fu},
Title = {Event Specific Multimodal Pattern Mining for Knowledge Base Construction},
BookTitle = {Proceedings of the 24th ACM international conference on Multimedia},
Address = {Amsterdam, The Netherlands},
Year = {2016}
}
Get EndNote Reference (.ref)
For problems or questions regarding this web site contact The
Web Master.
This document was translated automatically from BibTEX by bib2html (Copyright 2003 © Eric Marchand, INRIA, Vista Project).