We evaluated the retrieval effectiveness of the color indexing approaches on a test set of 3100 color images. The test set includes color images from a variety of subjects such as cities, nature, animals and transportation. For each query the retrieval effectiveness using recall and precision was recorded. The overall query process is illustrated in Figure 10. At each round in the query process the user can (1) initiate a new color set query using the interface tools or (2) use one of the retrieved images as a seed image. After L rounds the user zeros in on the desired image.
Figure 10: Query paths for image queries in VisualSEEk, 
. Starting from color set query 
constructed using interface tools, round 0 yields 
images. After viewing the matches the user makes decision to either (1) formulate a new color set query 
or (2) use a returned image for a give me more color query to find similar images. The process is repeated until desired image is found at round L-1.
The goal of this query is to retrieve all images that contain a significant purple region somewhere in the image. Prior to the trials each of the 3100 images was inspected and assigned a relevance to the purple region query. A three-value relevance was used. The query results are shown in Figure 11(a). The color set indexing method out-performed all others in both retrieval effectiveness and query response time. Furthermore, the color set method was able to retrieve all 97 purple region images with an overall precision of 0.625. For the color histogram queries a seed image was picked from the collection by the user that was most ``typical'' of the desired images. The color set query was formulated in VisualSEEk by simply drawing one purple region and by selecting best match in color and none for match in space.
The goal of this query is to retrieve all images of sunsets which come in various flavors. A sunset image was defined to contain a glowing horizon with the sun still partially visible. Prior to the trials each of the 3100 images was inspected and assigned a relevance. The query results are shown in Figure 11(b). The results are interesting. Since the sunset images in the archive do not have a typical spatial layout of color regions, the color set retrieval precision was initially low, also returning many images of yellow and orange flowers. When a ``typical'' sunset image was used to seed the color histogram queries the retrieval effectiveness was significantly better. However, if the user desires that all sunset images are to be returned with the highest possible precision then the color set method is far superior. In order to return all 83 sunsets using color histograms nearly all (3100) images must be returned. However, using color sets all sunset images are found with a precision of 0.2 (415 returned images).
Figure 11: Retrieval effectiveness (a) Query 1 - Purple regions, the symbol 'o' marks the recall and precision values at point for 
and (b) Query 2 - Sunsets.
The goal of these two queries is to find particular images: (1) image with a red sports car and (2) image with a British double decker bus (see Figure 7). The user was shown the color images and asked to use the system to compose rounds of either (1) color set or (2) give me more color histogram queries to find them. The user was not given an appropriate initial seed image and had to pick from 21 randomly supplied images. The same set was supplied for all queries. In all query rounds the return number was used. At each round l, 
out of images are new matches not returned in rounds 
. The trials are summarized below. The color set method does very well in these types of queries. The color histogram retrievals are largely dependent on the seed images. It takes more rounds to find the desired images using only give me more queries. The best overall performance combines both type of queries: local color set and global histogram. In this case the color set query provides the initial seed images to be used for subsequent histogram queries.
| histogram | histogram | color set + | ||||||
| euclidean | intersection | color set | histogram | |||||
| image | bus | car | bus | car | bus | car | bus | car |
| rounds = L | 8 | 5 | 9 | 5 | 1 | 3 | 1 | 2 |
unique returns  | 101 | 72 | 115 | 75 | 21 | 47 | 21 | 35 |