Two metrics for retrieval effectiveness are recall and precision. Recall signifies the proportion of relevant images in the database that are retrieved in response to a query. Precision is the proportion of the retrieved images that are relevant to the query [Jon81]. More precisely, let A be the set of relevant items, let B be the set of retrieved items and a, b, c and d are given in Figure 9 then recall and precision are defined as the following conditional probabilities:
As indicated by the formulas the measures of recall and precision require that the relevance to the query of each item in the database is established ahead of time. For the experiments this was done for each query by subjectively assigning one of three values to each image in the database: relevant = 1, partially or marginally relevant = 0.5 and not relevant = 0. The relevance value corresponds to the probability that the item is relevant to the query.
Figure 9: Retrieval effectiveness: a+b+c+d = full database, a+c = relevant items and a+b = retrieved items.