I.    Introduction

The issues of interest are: image quality after watermark embedding, robustness of authentication bits to JPEG compression, authentication sensitivity to malicious manipulation such as crop-and-replace, as well as widely-used but not directly oriented image processing methods such as low pass and median filtering, noise, brightness and contrast change, etc.

This test will also help to further improve SARI system or develop extended authentication schemes for multimedia.

II.    Improvement from SARI 1.0 to 1.1

2.    Solved the non-convergence problem under border conditions

3.    Better system stability.

back to the top

III.    Image Quality Test

    - Keep record of the image PSNR after watermarking embedding
    - System parameter QR denotes the embedding strength related to maximum JPEG  tolerate bound

Subjective  test:

    - Keep record of the maximum acceptable embed strength according to the judgments of the viewers
    - Background of the image viewers and the monitors used are listed below:

Viewer No.1    image-processing             Trinitron 17'                Viewer No.3    non-image-processing           Trinitron 17'
Viewer No.2    image-processing             Sony Laptop                Viewer No.4    image-processing                  Trinitron 17'

Figure 1.    Samples of the test images
Table 1.    Quality Test Statistics

Figure 2.       PSNR for different image types 
(average value of the two images in this type)

Discussion on Image Quality

2.    The embedding capacity of a natural image is generally larger than that of a synthetic image. This is because the former has more textural areas, thus the slight modification caused by authentication bits is less visible. The image quality of human, nature, and still object is generally better than that of synthetic and document image, and both the objective and subjective tests agree at this point.

3.    The quality judgments vary among different viewers. This is because users pay attention to different features of an image and their tolerance bounds can be quite different.  Moreover, different types of monitors have different display effects, e.g. the images that appear not acceptable on a Dell PC look just fine on a Sun Workstation.
        In order to better suit the need of prospective user, extensive test is suggested among a specific user group before an general quality bound is decided.

IV.     Performance Test

Experiment Condition 

2.    JPEG
       Minimum PhotoShop JPEG quality factor (1~10) the watermark can survive under maximum embed strength
       and QR=4 (authentication only), respectively

3.    Manipulation
       Whether the authenticator is sensitive to 1-pixle change or mass crop-&-replacement.
       Case under QR=4 (maximum robustness)

4.    Brightness, Contrast and Gaussian Noise
       Adjustments to selected area, test both BMP and JPEG format
       Case under QR=4 (maximum robustness)

4.1    JPEG Compression and Crop-replace

Notations :   Y--- Authenticator alarm at the exact location   N---- Authenticator no alarm 
* Tested under better visual quality (QR=2)
* Size after watermark embedding (maybe slightly cropped to integer times of 16 or 8 during embedding process)
* Test under better visual quality

Table 2.    Performance under JPEG Compression and Crop-Replace 

Notes

Figure 4.    Detection and Recovery of Crop-and-Replace 
upper left: original; upper right: manipulated; 
lower left: authentication output; lower right: recovery output

 4.2    Image Operations

 Note: the image operations are not directly addressed in this authentication scheme, and these tests are carried on for reference purpose.

Notations :
Y-- Authenticator alarm at the exact location   Y* -- Authenticator alarm but might not at the exact location N---- Authenticator no alarm  
* Tested under max embed depth, i.e. QR=2
* Size after watermark embedding (maybe slightly cropped to integer times of 16 or 8 during embedding process)

Table 3.    Performance under Image Operations

Notes

2.    Small scale tests have also been done on skew, geometry transformation, etc.
       And the result shows authenticator will recognize these changes and issue global alarm

3.    The Recovery Issue:
        - Recovery can be regarded a bonus to the large embedding capacity, and the recovered part is a scaled down version with a quality similar to JPEG generic quality factor 25
        - Recovery bits may be destroyed when the image is modified at several different places

*    The designer's comment (C.-Y. Lin): There might be no good trade-offs in setting a threshold to distinguish these operations from malicious operations. The difficulty is that, for instance, to survive these operations in a 512x512 image, the probability of false alarm (Pfa) in each coefficient should be smaller than 1/12288. This is not likely to happen in the presence of quantization, because even a small Gaussian noise added in the coefficients near the thresholding boundary may introduce large distortion after quantization. Some mathematical analysis can be found in http://www.ctr.columbia.edu/sari/performchart.html reference papers [1] and [3]

back to the top

All test images and results are available upon request to Lexing Xie <xlx@ctr.columbia.edu