Physics-based Photograph and Computer Graphics Classification
We address the issue of classifying photographic images (PIM) and photorealistic computer graphics (PRCG) images. The motivation is that PRCG can be used as image forgery. The problem is approached by analyzing the physical differences in the image formation process for the photographic images and PRCG. The differences are captured by a set of geometric features (from differential geometry, fractal geometry and local patches) in the linear Gaussian scale-space.
Publications [bibtex]
"Physics-Motivated Features for Distinguishing Photographic Images and Computer Graphics" Tian-Tsong Ng, Shih-Fu Chang, Jessie Hsu, Lexing Xie, Mao-Pei Tsui ACM Multimedia November 2005. [Abstract][pdf][slides] "Classifying Photographic and Photorealistic Computer Graphic Images using Natural Image Statistics" Tian-Tsong Ng, Shih-Fu Chang ADVENT Technical Report #220-2006-6 Columbia University October 2004 [Abstract][pdf]
Dataset [website]
"Columbia Photographic Images and Photorealistic Computer Graphics Dataset" Tian-Tsong Ng, Shih-Fu Chang, Jessie Hsu, Martin Pepeljugoski ADVENT Technical Report #203-2004-3 Columbia University February 2005 [Abstract][pdf]
Online Demo [website]
"An Online System for Classifying Computer Graphics Images from Natural Photographs" Tian-Tsong Ng, Shih-Fu Chang SPIE Electronic Imaging January 2006 [Abstract][pdf][slides]
Technical Illustration
Difference between CG and Photo
We identify three types of differences between CG and Photo by studying the respective image gerenetive process. They are the differences in the image acquisition device, object geometry model and surface property model.
Image Gradient and Camera Response Function
A typical camera response function has the property of compressing signal at the high irradiance region and expanding signal at the low irradiance region. This property can be captured by image gradient.
Principle Curvatures for Capturing local Image Sharp Structures
The rough geometry model can introduce sharp sturctures in CG. These rough structure can be captured by the principle curvatures obtained from the second fundamental form.
Mean Curvature Vectors for Capturing the Correlations between Color Channels
The RGB color in CG are often rendered independently due to the color independent assumption in the surface property model. Mean curvature vector of a 3D (RGB) sub-manifold in 5D Euclidean space offers a way to capture the correlation between the RGB colors.
Columbia Photographic Images and Photorealistic Computer Graphics Dataset
The figure on the left shows example images from Columbia Photographic Images and Photorealistic Computer Graphics Dataset with four sebsets with 800 images each.
Classification Performance
The physics-based feature outperforms the wavelet feature and the cartoon feature in terms of the classification accuracy. The figure on the left shows the ROC curve of the classification.