Further Details on Passive-blind Image Splicing Detection

Blind Image / Video Forensic

Project's Home Page | Current Research Areas > Authentication / Watermarking >

Main Content People Publications Downloads Links Summary Page

Further Details on Passive-blind Image Splicing Detection

Introduction

This page will provide you with the further details on the Passive-blind Image Splicing Detection research, following the brief introductory outline. With a great depth, this page will explain the differences between the active-blind approach and the passive-blind approach that our project is on. Following that, we will provide the rationale of us focusing on the specific problem of image splicing detection, instead of the various other forms of photomontaging techniques. Then, we will give a systematic definition of image authencity with respect to the passive-blind approach of image forgery detection. By the given definition, we could consider this work as image content authentication by means of the natural imaging quality of authentic images.

Before we go into the details of the higher-order statistics (HOS) technique that we adopted, we provide a short description of bicoherence (BIC) and the prior work that used BIC for passive-blind human speech signal splicing detection. Then, we show how how it would be difficult if the same technique were to be applied to images, a different kind of signal. This forms the basis for my work on proposing methods to improve the effectiveness of the BIC technique on image splicing detection.

For the details of the BIC technique, we described the basic features that are used by the prior work. Then, we describe the two general methods that we propose for making the BIC technique more effective in detecting image splicing. Finally, we provide experiment results that show the improved separability between the authentic and spliced image blocks due to the proposed improvement.

Active-Blind versus Passive-Blind Content Authentication Approach

Examples of active-blind content authentication approaches are:

Fragile/Semi Fragile Digital Watermarking: Inserting digital watermark at the source side and verifying the watermark integrity at the detection side.
Authentication Signature: Extracting image features for generating authentication signature at the source side and verifying the image integrity by signature comparison at the receiver side.

The disadvantages are:

Need a fully-secure trustworthy camera infrastructure
Need a common algorithm for the source and the detection side.
Watermark degrades image quality

Passive-blind content authentication approach can do without any prior information (e.g. digital watermark or authentication signature) and verify whether an image is authentic or fake. The advantage is that there is no need for watermark embedding or signature generation at the source side.

The Reasons for Focusing on Spliced Images

Image splicing is a basic and essential operation for all photomontages and photomontaging is one of the main techniques for creating fake images with new semantics.
Image splicing artifact is the direct indication of photomontaging, while post-processing artifacts may not be.

However, a more comprehensive solution for photomontage detection would include detection of post-processing operations and computer graphics techniques for detecting scene internal inconsistencies.

Our Approach through The Quality of Authentic Images

Authentic images have two hallmark qualities:

Natural-imaging Quality: Entailed by natural imaging process with real imaging devices, e.g., camera imposes effects of optical low-pass, sensor noise, lens distortion, etc, on an authentic image.
Natural-scene Quality: Entailed by physical light transport in real-world scene with real-world objects, resulting in
real-looking texture, right shadow, right perspective and shading, etc.

For example, computer graphics and photomontages are not complete with the both qualities.

In this work, we authenticate the Natural-imaging Quality through the detection of image splicing artifact using Higher-order Statistics (HOS) technique, known as Bicoherence (BIC).

Bicoherence

Bispectrum is a third-order moment spectra of a signal, say x(t). Let, X(w) be the Fourier transform of x(t), bispectrum is defined as below:

Whereas bicoherence is the normalized bispectrum, where normalization is by the upper bound the cauchy-schwartz inequality. The mathematical form for bicoherence is given by:

In general, bicoherence is estimated by the segment-averaging of the quadratic periodograms, as shown below:

The properties of bicoherence are:

Bicoherence is zero for a gaussian process.
The magnitude of bicoherence at a bifrequency (w1,w2) is a good estimator of QPC (see definition in Prior Work section) at that bifrequency, under the assumption of phase randomization (each segment used for the computation of a quadratic periodogram can be considered as a random phase realization)
The phase of bicoherence at a bifrequency (w1,w2) would be zero, when there is complete QPC at that bifrequency.

Prior Work

[Farid99] proposed detecting human speech splicing using bicoherence magnitude and phase features, based on the assumption that human speech signal is originally low in quadratic phase coupling (QPC), and considered splicing as a non-linear operation which comprises a linear-quadratic operation that could induce QPC. A linear-quadratic of a signal of two harmonics is illustrated below:

QPC is a phenomena where three harmonics have the following frequency and phase respectively. It can be seen that the relationship of the frequency and phase are the same.

[Krieger97] shows that straight image edges can induce significant energy in 2D bispectrum b(fx1,fy1;fx2,fy2) at the aligned region as shown below:

The significant level of bispectrum energy may imply the originally significant level of QPC in natural images, under phase randomization assumption. Most importantly, the level of bispectrum energy in natural images is image feature dependent. Therefore, the increase in the value of bicoherence features induced by image splicing may be overwhelmed by the variance of the original bicoherence features.

Plain Bicoherence Features

Similar to the prior work of Farid, the plain bicoherence features that we extracted from an image block are:

The flow chart for the extraction of these two features is as below:

The distribution of the plain bicoherence features is as below. We can see that the distribution for magnitude and phase are for the authentic and the spliced image blocks are greatly overlapped, although they have different mean and tail characteristics. The poor separability motivated us to propose two general methods for improving the effectiveness of the bicoherence features

Two General Methods for Improving the Effectiveness of Bicoherence Features

Method 1

We consider other image characteristics than the image splicing artifact for which bicoherence features are also sensitive to. We empirically observe that the level of image texturedness, hence the different interface type of an image block, has an effect on the bicoherence features and we model the texturedness characteristic using Canny edge pixel percentage. The scatter plots for the bicoherence magnitude feature is shown below and those for the bicoherence phase feature has the similar characteristics.

Method 2

We propsed to detect image splicing with respect to the estimated authentic counterpart (i.e., a visually similar image to the spliced one except for the fact that it is actually captured by a natural image acquisition process, see figure below). The reason is that the effect of image splicing is supposed to be more conspicuous if we can have the authentic counterpart as a reference.

To estimate authentic counterpart from an image, we perform a functional texture decomposition on the image and use the structure component as the approximated authentic counterpart of the image. Mathematically, functional texture decomposition using total variation framework [VeseOsher02] is formulated as below:

An example of the decomposition is shown below:

With the authentic counterpart, we detect the absence/presence of the splicing artifact in the original image with respect to the authentic counterpart, in terms of the bicoherence features. The difference between the bicoherence features of the original image blocks and that of the authentic ones is called prediction residual. When there is a presence of splicing artifact in the original image, the prediction residual is expected to be larger than the case when the splicing artifact is absent for an authentic image. The flowchart for the computation of the prediction residual features is shown below:

Results

The plain bicoherence magnitude and phase features achieve only 62% of classification accuracy between the authentic and the spliced image blocks, using Support Vector Machine (SVM). As a reference, random guessing without prior information about the data set, the classification accuracy is 50%.

The features derived from the two proposed methods improve the classification accuracy from 62% to 72% (See figure below). The derived features are the prediction residual for the bicoherence magnitude and phase features and the edge percentage feature. Please refer to the ISCAS 2004 paper, for more details.

Conclusion

In a nutshell, the plain/baseline bicoherence features do not perform well for image splicing detection and the proposal of incorporating image characteristics and the splicing-invariant (with respect to bicoherence) component has resulted in an improvement in the classification accuracy from the 62% obtained by using only the plain bicoherence features to 72% obtained by incorporating the three new features (i.e., the prediction residual for the plain magnitude and phase features, and the edge percentage feature).

Future Work

There is still extensive room for improvement based on this work. Possible directions could be to explore cross-block fusion and incorporate image structure in fusion.

The approach adopted in this report can be considered as a signal processing approach. We can combine the signal processing approach with the computer graphics/computer vision approach to automatically or semi-automatically examine the scene-level internal inconsistencies within an image. Lastly, we can explore beyond bicoherence for other discriminative features for image splicing (in particular) or photomontage (in general) detection.

Reference

[Farid99] H. Farid, "Detecting Digital Forgeries Using Bispectral Analysis," MIT AI Memo AIM-1657, MIT, 1999.

[Krieger97] G. Krieger, C. Zetzsche, and E. Barth, "Higher-order statistics of natural images and their exploitation by operators selective to intrinsic dimensionality," IEEE Signal Processing Workshop on Higher-Order Statistics, Banff, Canada, July 21-23, 1997.

[VeseOsher02] L. A. Vese and S. J. Osher, "Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing," UCLA C.A.M. Report 02-19, May 2002.

People

Publications

Conferences

Tian-Tsong Ng, Shih-Fu Chang, Qibin Sun. Blind Detection of Photomontage Using Higher Order Statistics. In IEEE International Symposium on Circuits and Systems (ISCAS), Vancouver, Canada, May 2004.
Tian-Tsong Ng, Shih-Fu Chang. A Model for Image Splicing. In IEEE International Conference on Image Processing (ICIP), Singapore, October 2004.

Presentation Slides

Presented in IEEE International Symposium on Circuits and Systems (ISCAS), Vancouver, Canada, May 2004

Technical Reports

Tian-Tsong Ng, Shih-Fu Chang. Blind Detection of Digital Photomontage using Higher Order Statistics. ADVENT Technical Report #201-2004-1 Columbia University, June 2004.
Tian-Tsong Ng, Shih-Fu Chang. A Data Set of Authentic and Spliced Image Blocks. ADVENT Technical Report #203-2004-3 Columbia University, June 2004.

Data set Downloads

Columbia Image Splicing Detection Evaluation Dataset