Visually Searching the Web for Content

John R. Smith and Shih-Fu Chang


We describe a system prototype for searching for images and videos on the World Wide Web. New visual information in the form of images, graphics, animations, and videos is being published on the Web at an incredible rate. However, cataloging all this information exceeds the capabilities of current text-based Web search engines. In this article we describe a complete system by which visual information on the Web is collected by automated agents and is catalogued and indexed for fast search and retrieval. We provide an initial evaluation based upon the cataloging of over one half million images and videos from the Web.

Keywords: content-based query, image and video storage and retrieval, image/video subject cataloging, search engines, World-Wide Web