Summary
In this project, we collaborated with colleagues in Teachers College to develop an intuitive real-time image search system that supports non-textual query cues, such as rough line sketchs and compositions of color regions.
On the technical side, we are interested in new algorithms for extracting invariant edge features and arbitrarily-shaped region features, and search methods for retrieving images that satisfy the visual attributes in user's mental pictures. On the pedagogical side, we are interested in developing new curricula that allow young users (e.g., K-12 students) to easily access art images with intuitive expression in terms of basic visual composition, without formal knowledge of art history.
A sample sketch query and search results are shown below.
The search component supports queries in the form of rough line-sketches that outline the basic form and composition of an image. This permits the user to quickly and easily transform a mental picture of an image into a query. Characterizing archived images with signatures that represents their strongest edges, and comparing these to a dynamically generated signature of the user's sketch achieves this search. The edge signature is generated over multiple scales to account for the variation in detail of the user's line-sketch. Edge-coherence, a measure of the perceptual strength of an edge, based on its continuity, is used in generating the signature. Our search technique is fast enough to allow the system to respond with results at the completion of each stroke of a query sketch. The algorithm is robust and invariant to scale, rotation and translation. We also show how this edge-based search can be merged seamlessly with the more traditional color-region and shape-based searches to produce a search interface that is expressive, intuitive, simple, and works in the absence of sample images.
People
Raj
Kumar and Prof. Shih-Fu
Chang
Demo
Offline demo available.
Publication
Rajendran Kumar, S.-F. Chang,
Image
Retrieval with Sketches and Compositions, IEEE International Conference
on Multimedia and Expo (ICME), New York, July 2000.
(PS.GZ/PDF)
Download
For problems
or questions regarding this web site contact The
Web Master.
Last updated:
June 12, 2002.