The analysis of emotion, affect and sentiment from visual content has become an
exciting area in the multimedia community allowing to build new applications
for brand monitoring, advertising, and opinion mining.
There exists no corpora for sentiment analysis on visual content, and therefore
limits the progress in this critical area.
To stimulate innovative research on this challenging issue, we constructed a new
benchmark and database (you can browse the database at VSO Browsing Interface
This database contains a Visual Sentiment Ontology (VSO) consisting of
3244 adjective noun pairs (ANP), SentiBank a set of 1200 trained
visual concept detectors providing a mid-level representation of
sentiment, associated training images acquired from Flickr, and a
benchmark containing 603 photo tweets covering a diverse set of 21
This website provides the above mentioned material for download and is structured as the following:
Benchmark and Dataset Citation
Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel and Shih-Fu Chang.
"Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs,"
ACM Multimedia Conference, Barcelona, Oct 2013.
Thanks for checking out VSO, you may also be interested in our new improved and multilingual version MVSO
For each ANP a concept detector has been trained
using 80% of its Flickr sample images. The
remaining 20% have been used for detector testing.
Results can be seen in the Figures on the right and the bottom.
1553 detectors were evaluated and finally 1200 detectors
(all have non-zero AP@20 and most have F-score > 0.6) were retained in the current
release of SentiBank.
Concept detector training was performed
according the following setup:
The following visual features have been used:
RGB Color Histogram (3x256 dim.)
GIST descriptor  (512 dim.)
Local Binary Pattern  (52 dim.)
Bag-of-Words (1,000 dictionary
2-layer spatial pyramid, max pooling)
Attributes  (2,000 dim.)
*The feature extraction code is presented in SentiBank v1.02 and higher.
Detectors have been trained using LibSVM :
Linear SVM classifier
Matlab + C hybrid coding is used to implement
SentiBank in our current version. Windows platform
is recommended to run SentiBank. It is possible
(and validated) that our codes can be also run in
Linux platform with Matlab. However, you might
need to set up the -mex configuration to choose
appropriate C compliers.For a user guide on mex
configuration, see here
Files for Download
1. binary code of SentiBank detectors
(latest version: 1.1)
2. source code of SentiBank detectors
(latest version: 1.1)
3. DeepSentiBank detectors
See paper here
4. MVSO detectors (New!)
See paper here
Comparison of ANP detectors using different features. Performance computed by avg. over 5 runs of the reduced testsets (SentiBank v1.02).
AP@20 of ANP detectors using different fusion approaches. Performance computed by avg. over 5 runs of the reduced testsets (SentiBank v1.02).
Comparison of ANP detectors using different features. Performance computed by avg. over 5 runs of the reduced testsets (SentiBank v1.1).
AP@20 of ANP detectors using different fusion approaches. Performance computed by avg. over 5 runs of the reduced testsets (SentiBank v1.1). For Early Fusion, the average AP@20 of all ANPs increased from 47% (v1.02) to 63% (v1.1).
To download the SentiBank code, please fill out the following fields. We will send you
download instructions via email immediately.
People who request and use the dataset
(or code) must agree that 1) the use of the data is restricted to research or education
purpose only, 2) all copyright and license restrictions associated with the dataset/code
will be followed, and 3) the authors of the above paper and their affiliated
organizations make no warranties regarding the database or software, including but not
limited to non-infringement.
SentiBank's detector performance according to the above described implementation.
For more details of our baseline classifier design, please refer to the paper
AP@20 (avg. over 5 runs of the reduced testsets) vs. frequency of 1,553 ANP detectors ranked by detector performance. Note only a subset of ANP names are shown due to space limit.
AP@1000 vs. frequency of top 600 ANP detectors ranked by detector performance. Note only a subset of ANP names are shown due to space limit.
 A. Olivai and A. Torralba Modeling the Shape of the Scene: a Holistic Representation of the Spatial Envelope. IJCV, 2001 - online resource
 Ojala, Timo, Matti Pietikainen, and Topi Maenpaa. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence. 24.7: 971-987, 2002 - online resource
 Felix, X. Yu, Liangliang Cao, Rogerio S. Feris, John R. Smith, and Shih-Fu Chang. Designing Category-level Attributes for Discriminative Visual Recognition. CVPR, 2013. - online resource
 LIBSVM: a Library for Support Vector Machines, CC Chang, CJ Lin - ACM Transactions on Intelligent Systems and Technology (TIST), 2011 - online resource