We presented a system for automated extraction of local color regions which provides for efficient indexing and effective color image retrieval. The system uses the back-projection of binary color sets to identify salient color regions in images. Prior to queries a table of meta-data is created by the systematic iteration over color sets to extract localized color regions. The table is indexed directly by color set value which enables very fast indexing of the image collection. The color set approach also allows queries to include multiple regions and their spatial layout. We also described the implementation of the color set techniques in the VisualSEEk content-based image/video retrieval system. Some sample queries and initial evaluation results based on recall and precision were also presented.