|
|
Current Projects
Semantic Concept Detection with Cross Concept Learning
Introduction
Semantic concepts usually do not occur in isolation, and the contextual relationships provide
important information for automatic concept detection in images/videos. Unlike traditional
approaches that independently build binary classifiers to detect individual concepts, we
develop algorithms that consider inter-conceptual relationships.
Context-based Concept Fusion
|
We propose a new context-based concept fusion (CBCF) method for semantic concept detection. Our work
includes two folds. (1) We model the inter-conceptual relationships by a Conditional Random Field (CRF) that
improves detection results from independent detectors by taking into account the inter-correlation among
concepts. CRF directly models the posterior probability of concept labels and is more accurate for the
discriminative concept detection than previous statistical inferencing techniques. The Boosted CRF framework
is incorporated to further enhance performance by combining the power of boosting with CRF. (2) We develop
an effective criterion to predict which concepts may benefit from CBCF. As reported in previous works,
CBCF has inconsistent performance gain on different concepts. With accurate prediction, computational
and data resources can be allocated to enhance concepts that are promising to gain performance.
|
|
Active Context-based Concept Fusion
|
We propose a new framework, called active context-based concept fusion, for effectively improving the
Accuracy of semantic concept detection in images and videos. Our approach solicits user annotations
for a small number of concepts, which are used to refine the detection of the rest of concepts. In
contrast with conventional methods, our approach is active, by using information theoretic criteria
to automatically determine the optimal concepts for user annotation. In addition, we have developed
an effective method to predict concepts that may benefit from context-based fusion.
|
|
Multi-class Concept Detection by Kernel Sharing
|
We proposed a new framework for multi-class concept detection based on kernel sharing and joint learning.
By sharing good kernels among concepts, accuracy of individual weak detectors can be greatly improved;
by joint learning of common detectors among classes, the required kernels and the computational
complexity for detecting each individual concept can be reduced. We demonstrated our approach by
developing an extended JointBoost framework, which was used to choose the optimal kernel and subset
of sharing classes in an iterative boosting process. In addition, we constructed multi-resolution
visual vocabularies by hierarchical clustering and computed kernels based on spatial matching.
Extensive analysis of the results also revealed interesting and important underlying relations
among concepts.
|
|
Publications
- Wei Jiang, Shih-Fu Chang, Alexander C. Loui, "Kernel sharing with joint boosting for multi-class
concept detection", CVPR Workshop, Minneapolis, Minnesota, 2007.
PDF
- Wei Jiang, Shih-Fu Chang, Alexander C. Loui, "Context-based concept fusion with boosted conditional
random fields", ICASSP, (1):949-952, 2007.
PDF
- Wei Jiang, Shih-Fu Chang, Alexander C. Loui, "Active context-based concept fusion with partial user labels",
ICIP, pp.2917-2920, 2006.
PDF
Patent
- Shih-Fu Chang, Wei Jiang, Alexander C. Loui, Active Context-based Concept Fusion, US Patent
Pending, 20070271256, Filed Dec. 2006.
|
