Nikolaos Pappas, Miriam Redi, Mercan Topkara, Hongyi Liu, Brendan Jou, Tao Chen, Shih-Fu Chang. Multilingual visual sentiment concept clustering and analysis. International Journal of Multimedia Information Retrieval, 6(1):51-70, 2017.
Download paper:
Adobe portable document (pdf) 
Copyright notice:This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
Visual content is a rich medium that can be used to communicate not only facts and events, but also emotions and opinions. In some cases, visual content may carry a universal affective bias (e.g., natural disasters or beautiful scenes). Often however, to achieve a parity in the affections a visual media invokes in its recipient compared to the one an author intended requires a deep understanding and even sharing of cultural backgrounds. In this study, we propose a computational framework for the clustering and analysis of multilingual visual affective concepts used in different languages which enable us to pinpoint alignable differences (via similar concepts) and nonalignable differences (via unique concepts) across cultures. To do so, we crowdsource sentiment labels for the MVSO dataset, which contains 16 K multilingual visual sentiment concepts and 7.3M images tagged with these concepts. We then represent these concepts in a distribution-based word vector space via (1) pivotal translation or (2) cross-lingual semantic alignment. We then evaluate these representations on three tasks: affective concept retrieval, concept clustering, and sentiment prediction - all across languages. The proposed clustering framework enables the analysis of the large multilingual dataset both quantitatively and qualitatively. We also show a novel use case consisting of a facial image data subset and explore cultural insights about visual sentiment concepts in such portrait-focused images
@article{pappas2017multilingual,
Author = {Pappas, Nikolaos and Redi, Miriam and Topkara, Mercan and Liu, Hongyi and Jou, Brendan and Chen, Tao and Chang, Shih-Fu},
Title = {Multilingual visual sentiment concept clustering and analysis},
Journal = {International Journal of Multimedia Information Retrieval},
Volume = {6},
Number = {1},
Pages = {51--70},
Publisher = {Springer},
Year = {2017}
}
Get EndNote Reference (.ref)
For problems or questions regarding this web site contact The
Web Master.
This document was translated automatically from BibTEX by bib2html (Copyright 2003 © Eric Marchand, INRIA, Vista Project).
