Bottom up approach to Complex Event Recognition

Recognition of complex events in unconstrained Internet videos is a challenging research problem. In this symposium proposal, we present a systematic decomposition of complex events into hierarchical components and make an in-depth analysis of how existing research are being used to cater to various levels of this hierarchy. We also identify three key stages where we make novel contributions which are necessary to not only improve the overall recognition performance, but also develop richer understanding of these events. At the lowest level, our contributions include (a) compact covariance descriptors of appearance and motion features used in sparse coding framework to recognize realistic actions and gestures, and (b) a Lie-algebra based representation of dominant camera motion present in video shots which can be used as a complementary feature for video analysis. In the next level, we propose an (c) efficient maximum likelihood estimate based representation from low-level features computed from videos which demonstrates state of the art performance in large scale visual concept detection, and finally, we propose to (d) model temporal interactions between concepts detected in video shots through two new discriminative feature spaces derived from Linear dynamical systems which eventually boosts event recognition performance. In all cases, we conduct thorough experiments to demonstrate promising performance gains over some of the prominent approaches.

This paper is a concise version of the dissertation and was selected for oral presentation at the ACM MM 2013 Doctoral Symposium in Barcelona, Spain [3/13]. The entire dissertation is also available here. A video of the dissertation defense is attached for the interested:

[1] Subhabrata Bhattacharya, "Recognition of Complex Events in Open-Source Web-Scale Videos: A Bottom up approach", In Proc. of ACM International Conference on Multimedia (MM), Barcelona, ES, pp. 1035-1038, 2013.

[2] Subhabrata Bhattacharya, "Recognition of Complex Events in Open-source Web-scale Videos: Features, Intermediate Representations and Their Temporal Interactions", PhD thesis, University of Central Florida, Orlando, FL, USA, 2013.

[3] Yu-Gang Jiang, Subhabrata Bhattacharya, Shih-Fu Chang, Mubarak Shah, "High-Level Event Recognition in Unconstrained Videos", In International Journal of Multimedia Information Retrieval (IJMIR), vol. 2, no. 2, pp. 2:73-101, 2013.

This webpage template is inspired from Imran Saleemi, a friend from UCF. The old one still exists.