Robust Visual Domain Adaptation with Low-Rank Reconstruction


I-Hong Jhuo, Dong Liu, D.T. Lee and Shih-Fu Chang

National Taiwan University

Columbia University


Visual domain adaptation addresses the problem of adapting the sample distribution of the source domain to the target domain, where the recognition task is intended but the data distributions are different. In this paper, we present a low-rank reconstruction method to reduce the domain distribution disparity. Specifically, we transform the visual samples in the source domain into an intermediate representation such that each transformed source sample can be linearly reconstructed by the samples of the target domain. Unlike the existing work, our method captures the intrinsic relatedness of the source samples during the adaptation process while uncovering the noises and outliers in the source domain that cannot be adapted, making it more robust than previous methods. We formulate our problem as a constrained nuclear norm and l2,1 norm minimization objective and then adopt the Augmented Lagrange Multiplier (ALM) method for the optimization. Extensive experiments on various visual adaptation tasks show that the proposed method consistently and significantly beats the state-of-the-art domain adaptation methods.


l   To effectively adapt the sample distribution of the source domain to match that of the target domain.

l   The common issues with prior works are:

n   Dealing with source samples separately: The adapted distributions may be inconsistent among different source domains.

n   Outliers problem: Not every source sample is useful. There are possibly outliers. Blindly translating outlier samples will hurt performance.


l  Visual Domain Adaptation with Low Rank Reconstruction

Ø  We transform the visual samples in the source domain into an intermediate representation.

Ø  Each transformed source sample can be linearly reconstructed by the samples of the target domain.

Ø  We enforce the reconstruction coefficient matrix corresponding to all source samples to be low rank while removing the noisy features that cannot be adapted through group sparsity. 

l  The problem is solved with a constrained nuclear norm and l2,1 norm optimization by Augmented Lagrange Multiplier (ALM) algorithm.



Fig. Illustration of our proposed method. Each source domain Si contains two classes of samples (marked as purple ellipses and blue triangles) as well as some noisy samples (marked as black diamonds). The samples in the target domain are marked with letter `t'. Our method transforms each source domain Si into an intermediate representation WiSi such that each transformed sample can be linearly reconstructed by the target samples. Within each source domain Si, we enforce the reconstruction of source samples to be related to each other under a low-rank structure while allowing the existence of a sparse set of noisy samples. Furthermore, by enforcing different source domains W1S1,...,WMSM to be jointly low rank, we form a compact source sample set whose distribution is close to the target domain. The whole procedure is unsupervised without utilizing any label information.

Problem Definition

l  Our goal is to find a transformation matrix  to transform the source domain S into an intermediate representation matrix, given a set of n samples   in a source domain, and  in the target domain, where d is the dimension of the feature vector.

l  The relation between samples in source and target domains is the following: , where is the reconstruction coefficient matrix.


Ø   Single Source Domain Adaptation

Adapt one source domain to the target domain.

Capturing intrinsic structure information and handle the outliers/ noises in the source domain.


           , where denotes the nuclear norm of a matrix and encourages the error columns of E to be zero.



Ø   Multiple Source Domain Adaptation

The multi-task low rank domain adaptation can be formulated as:               



²   are two tradeoff parameters.

²  Q is a matrix formed by and  represents the ith transformed source domain.

Experiments and Results

We evaluate the effectiveness of our proposed method, referred to as Robust Domain Adaptation with Low-rank Reconstruction (RDALR), on various challenging visual domain adaptation tasks including three-domain object, Caltech-256, and TRECVID. In each task, the performance of the following domain adaptation methods will be compared.

(1) Naive Combination (NC). We directly augment the target domain with samples from the source domain without any transformation.

(2) Adaptive SVM (A-SVM). In this method, a SVM classifier is first trained in the source domain, and then adjusted to fit the training samples in the target domain.

(3) Noisy Domain Adaptive Reconstruction (NDAR). In this case, we do not consider to remove the noise and outlier information in the source domain.

(4) Our proposed RDALR method.

        Three-domain object




Fig. Performance of different methods on Caltech 256 dataset, where the per category number of training images in the target domain varies from 5 to 50. The per category number of images from the Bing source domain is fixed at 10.




Fig. Per-event performance comparison on the TRECVID MED 2011 dataset




I-Hong Jhuo, Dong Liu, D.T. Lee, Shih-Fu Chang. Robust Visual Domain Adaptation with Low-Rank Reconstruction. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2012. [pdf]