Publications

For the most up-to-date listing of publications, see Brendan's Google Scholar page.

Disclaimer: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

All occurrences of * denote equal contribution.

Conferences

Deep Cross Residual Learning for Multitask Visual Recognition
Brendan Jou and Shih-Fu Chang.
ACM Int'l Conference on Multimedia (MM), Amsterdam The Netherlands, 2016. [PDF][Poster][Model]

Multilingual Visual Sentiment Concept Matching
Nikolaos Pappas, Miriam Redi, Mercan Topkara, Brendan Jou, Hongyi Liu, Tao Chen and Shih-Fu Chang.
ACM Int'l Conference on Multimedia Retrieval (ICMR), New York NY, 2016. [PDF]
Best Multimodal Paper Award

SentiCart: Cartography and Geo-contextualization for Multilingual Visual Sentiment
Brendan Jou*, Margaret Yuying Qian* and Shih-Fu Chang.
ACM Int'l Conference on Multimedia Retrieval (ICMR), New York NY, 2016. [PDF][Poster][Demo]
Best Demo Award

Complura: Exploring and Leveraging a Large-scale Multilingual Visual Sentiment Ontology
Hongyi Liu*, Brendan Jou*, Tao Chen*, Mercan Topkara*, Nikolaos Pappas*, Miriam Redi* and Shih-Fu Chang.
ACM Int'l Conference on Multimedia Retrieval (ICMR), New York NY, 2016. [PDF][Poster][Demo]

Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology
Brendan Jou*, Tao Chen*, Nikolaos Pappas*, Miriam Redi*, Mercan Topkara* and Shih-Fu Chang.
ACM Int'l Conference on Multimedia (MM), Brisbane Australia, 2015. [PDF][Data]

Why We Watch the News: A Dataset for Exploring Sentiment in Broadcast Video News
Joseph G. Ellis, Brendan Jou and Shih-Fu Chang.
ACM Int'l Conference on Multimodal Interaction (ICMI), Istanbul Turkey, 2014. [PDF][Data]

Predicting Viewer Perceived Emotions in Animated GIFs
Brendan Jou, Subhabrata Bhattacharya and Shih-Fu Chang.
ACM Int'l Conference on Multimedia (MM), Orlando FL, 2014. [PDF]
Grand Challenge 3rd Place Award

Structured Exploration of Who, What, When, and Where in Heterogeneous Multimedia News Sources
Brendan Jou*, Hongzhi Li*, Joseph G. Ellis*, Daniel Morozoff-Abegauz and Shih-Fu Chang.
ACM Int'l Conference on Multimedia (MM), Barcelona Spain, 2013. [PDF]
Grand Challenge 1st Place Award

News Rover: Exploring Topical Structures and Serendipity in Heterogeneous Multimedia News
Hongzhi Li*, Brendan Jou*, Joseph G. Ellis*, Dan Morozoff* and Shih-Fu Chang.
ACM Int'l Conference on Multimedia (MM), Barcelona Spain, 2013. [PDF][Poster][Demo]

Robust Object Co-Detection
Xin Guo, Dong Liu, Brendan Jou, Mojun Zhu, Anni Cai and Shih-Fu Chang.
IEEE Int'l Conference on Computer Vision and Pattern Recognition (CVPR), Portland OR, 2013. [PDF][Poster]

Fusion of Region-based Representations for Gender Identification
Si Ying Diana Hu, Brendan Jou, Aaron Jaech and Marios Savvides.
IEEE Int'l Joint Conference on Biometrics (IJCB), Washington DC, 2011. [PDF][Poster]

An Exploration of Gender Identification Using Only the Periocular Region
Jameson Merkow, Brendan Jou and Marios Savvides.
IEEE Int'l Conference on Biometrics: Theory, Applications and Systems (BTAS), Washington DC, 2010. [PDF][Poster]

Journals

A Survey of Multimodal Sentiment Analysis
Mohammad Soleymani, David Garcia, Brendan Jou, Björn Schuller, Shih-Fu Chang and Maja Pantic.
Journal of Image and Vision Computing (IMAVIS), Elsevier, 2017. [PDF]

Multicultural Visual Concept Retrieval and Clustering
Nikolaos Pappas, Miriam Redi, Mercan Topkara, Hongyi Liu, Brendan Jou, Tao Chen and Shih-Fu Chang.
International Journal of Multimedia Information Retrieval (IJMIR), Springer, 2017. [DOI]

From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction
Víctor Campos, Brendan Jou and Xavier Giró-i-Nieto.
Journal of Image and Vision Computing (IMAVIS), Elsevier, 2017. [DOI][arXiv][Code]

Workshops, Technical Reports & Showcases

Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
Víctor Campos, Brendan Jou, Xavier Giró-i-Nieto, Jordi Torres and Shih-Fu Chang.
arXiv preprint arXiv:1708.06834, 2017. [arXiv][Project][Code]

More Cat than Cute? Interpretable Prediction of Adjective-Noun Pairs
Delia Fernandez, Alejandro Woodward, Victor Campos, Xavier Giro-i-Nieto, Brendan Jou and Shih-Fu Chang.
ACM MM Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes (MUSA),
Mountain View, CA, 2017. [arXiv][Project][Code]

Is a "happy dog" image more "happy" than "dog"? Analyzing Adjective and Noun Visual Contributions for Adjective-Noun Pair Prediction
Dèlia Fernández-Canellas, Víctor Campos, Xavier Giró-i-Nieto, Brendan Jou and Shih-Fu Chang.
Annual Workshop for Women in Machine Learning (WiML), Barcelona Spain, 2016. [Poster]

Tamp: A Library for Compact Deep Neural Networks with Structured Matrices
Bingchen Gong, Brendan Jou, Felix Yu and Shih-Fu Chang.
ACM Int'l Conference on Multimedia (MM), Amsterdam The Netherlands, 2016. [PDF][Code]

Going Deeper for Multilingual Visual Sentiment Detection
Brendan Jou and Shih-Fu Chang.
arXiv preprint arXiv:1604.01335, 2016. [arXiv][Data]

Diving Deep into Sentiment: Understanding Fine-tuned CNNs for Visual Sentiment Prediction
Víctor Campos, Amaia Salvador-Aguilera, Xavier Giró-i-Nieto and Brendan Jou.
ACM MM Workshop on Affect and Sentiment in Multimedia (ASM), Brisbane Australia, 2015. [PDF][Project][Code]

News Rover
Joseph G. Ellis*, Brendan Jou*, Hongzhi Li*, Daniel Morozoff-Abegauz* and Shih-Fu Chang.
NYC Media Lab Annual Summit, New York NY, 2014.
2nd Place Demo Award

Structured Exploration of Who, What, When, and Where in Heterogeneous Multimedia News Sources
Joseph G. Ellis, Brendan Jou, Hongzhi Li, Daniel Morozoff-Abegauz and Shih-Fu Chang.
ACM KDD Workshop on Data Science for News Publishing (NewsKDD), New York NY, 2014.

News Rover: Exploring Topical Structures and Serendipity in Heterogeneous Multimedia News
Brendan Jou, Hongzhi Li, Joseph G. Ellis, Daniel Morozoff and Shih-Fu Chang.
Greater New York Multimedia & Vision (GNYMV) Workshop, New York NY, 2013. [Poster]
Best Demo Award

Soft Biometric Identification
Si Ying Diana Hu, Brendan Jou, Jameson Merkow, Tian Seng David Leong and Marios Savvides.
Annual Workshop for Women in Machine Learning (WiML), Vancouver BC, 2010.

Microprocessor-based Control Systems
Brendan Jou.
Growth Opportunities for Leadership Development (GOLD) Program
Consolidated Edison Company of New York, New York NY, 2009.

Automated Music Genre Classification
Brendan Jou, Zian Chen and Peter Zhao.
New York University (NYU) Polytechnic School of Engineering, Brooklyn NY, 2008.