The analysis of emotion, affect and sentiment from visual content has become an exciting area in the multimedia community allowing to build new applications for brand monitoring, advertising, and opinion mining. There exists no corpora for sentiment analysis on visual content, and therefore limits the progress in this critical area. To stimulate innovative research on this challenging issue, we constructed a new benchmark and database (you can browse the database at VSO Browsing Interface). This database contains a Visual Sentiment Ontology (VSO) consisting of 3244 adjective noun pairs (ANP), SentiBank a set of 1200 trained visual concept detectors providing a mid-level representation of sentiment, associated training images acquired from Flickr, and a benchmark containing 603 photo tweets covering a diverse set of 21 topics.

This website provides the above mentioned material for download and is structured as the following:
Benchmark and Dataset Citation
Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel and Shih-Fu Chang. "Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs," ACM Multimedia Conference, Barcelona, Oct 2013.
download paper    
download material
Update: Thanks for checking out VSO, you may also be interested in our new expanded and multilingual version MVSO

Visual Sentiment Ontology: Ontology and Concepts Visual Sentiment Ontology: Image Dataset SentiBank: Visual Sentiment Concept Classifiers Photo Tweet
Sentiment Benchmark
System Architecture Construction of the VSO is shown in Fig. 1: process is founded on psychological research. During the the first step - the data-driven discovery - for each of the 24 emotions defined in Plutchik's theory [1] we retrieve images and videos from Flickr and YouTube respectively to extract concurrent tags. The set of all adjectives and all nouns is now used to form adjective noun combinations or Adj. Noun Pairs (ANP) such as "beautiful flowers" or "sad eyes".

VSO Statistics
Analysed Images & Videos: ~316,000
non-empty ANP candidates: ~47,000
VSO ANPs: 3,244
ANPs included in SentiBank: 1,200
most frequent positive adjectives:
- beautiful, amazing, cute
most frequent negative adjectives:
- sad, angry, dark
most frequent nouns:
- face, eyes, sky

VSO Browsing Interface
The entire VSO can be browsed at the VSO
Browsing Interface
. At this interface you can
browse the mapping of emotions to ANP (Fig.2)
and see each ANP and its sample images.

Files for Download
1. VSO: full list of ANP names (3244)
2. VSO (SentiBank): list of ANPs included in
   SentiBank detectors (1200)

   (latest version: 1.1)

Fig.1 Overview of the proposed framework including the construction of the Visual Sentiment Ontology (VSO) and SentiBank.

Fig.2 Plutchik's Wheel of Emotions with its 24 emotions organized by 8 basic emotions placed in a circle with each having 3 valences.

By downloading the VSO dataset, you agree that 1) the use of the data is restricted to research or education purpose only, 2) all copyright and license restrictions associated with the dataset/code will be followed, and 3) the authors of the above paper and their affiliated organizations make no warranties regarding the database or software, including but not limited to non-infringement.


[1] Robert Plutchik. Emotion: A Psychoevolutionary Synthesis. Harper & Row, Publishers, 1980.