EdSearch: Teaching Visual Arts with Content-Based Image Search Tools

Project's Home Page | Current Research Areas > Multimedia Indexing and Content Management >



In this project, we collaborated with colleagues in Teachers College to develop an intuitive real-time image search system that supports non-textual query cues, such as rough line sketchs and compositions of color regions.

On the technical side, we are interested in new algorithms for extracting invariant edge features and arbitrarily-shaped region features, and search methods for retrieving images that satisfy the visual attributes in user's mental pictures. On the pedagogical side, we are interested in developing new curricula that allow young users (e.g., K-12 students) to easily access art images with intuitive expression in terms of basic visual composition, without formal knowledge of art history.

A sample sketch query and search results are shown below.

The search component supports queries in the form of rough line-sketches that outline the basic form and composition of an image. This permits the user to quickly and easily transform a mental picture of an image into a query. Characterizing archived images with signatures that represents their strongest edges, and comparing these to a dynamically generated signature of the user's sketch achieves this search. The edge signature is generated over multiple scales to account for the variation in detail of the user's line-sketch. Edge-coherence, a measure of the perceptual strength of an edge, based on its continuity, is used in generating the signature. Our search technique is fast enough to allow the system to respond with results at the completion of each stroke of a query sketch. The algorithm is robust and invariant to scale, rotation and translation. We also show how this edge-based search can be merged seamlessly with the more traditional color-region and shape-based searches to produce a search interface that is expressive, intuitive, simple, and works in the absence of sample images.


  Raj Kumar and Prof. Shih-Fu Chang


Offline demo available.


Rajendran Kumar, S.-F. Chang, Image Retrieval with Sketches and Compositions, IEEE International Conference on Multimedia and Expo (ICME), New York, July 2000.



For problems or questions regarding this web site contact The Web Master.
Last updated: June 12, 2002.