Read

Relevance Feedback (Rocchio)

A user's first query is often an imperfect description of what they actually want — a couple of keywords that only partially capture their intent. Relevance feedback improves the query using explicit signal from the user: after seeing the initial results, the user marks some as relevant and some as not, and the system uses that feedback to reformulate a better query automatically. The Rocchio algorithm (Rocchio, 1971) is the classic vector-space formulation of this idea, and it plugs directly into the TF-IDF cosine engine you've already built.

The vector space recap

Every document and every query is represented as a TF-IDF weighted vector over the term vocabulary, then L2-normalized so that document length doesn't bias the comparison. Similarity between a query and a document is their cosine similarity — the dot product of the two normalized vectors, which for these already-unit-length vectors reduces to a plain dot product.

The Rocchio update

Rocchio treats the query itself as just another vector that can be nudged around in that same term space. Given:

• q — the original query vector
• D_relevant — the set of document vectors marked relevant
• D_nonrelevant — the set of document vectors marked non-relevant

the reformulated query is:

q_new = alpha * q + beta * mean(D_relevant) - gamma * mean(D_nonrelevant)

Each term's contribution to q_new shifts based on how it appears in the feedback documents: terms common in the relevant set get pulled toward the query (their weight increases), and terms common in the non-relevant set get pushed away (their weight decreases). alpha, beta, gamma control how much weight to give the original query versus the relevant-pull versus the non-relevant-push — this exercise uses the commonly-cited values alpha=1.0, beta=0.75, gamma=0.15, which trust the relevant documents considerably more than the non-relevant ones (a document being "not relevant" is weaker evidence than a document being "relevant," since there are many more ways to be irrelevant than relevant).

Why negative components get clamped to zero

After subtracting the non-relevant contribution, some term weights can go negative. A negative weight in the query vector would imply "documents without this term are more relevant" — but cosine similarity as normally computed doesn't handle that meaningfully (a document with weight 0 for that term would score higher in that dimension than one with a small positive weight, which isn't a coherent notion of relevance). The standard practice — and what real information-retrieval systems do — is to clamp negative weights to zero, effectively saying "this term is no longer part of the query at all" rather than trying to model negative relevance.

Building the query vector the same way as a document vector

For this to work cleanly, the initial query needs to be represented in the exact same TF-IDF space as documents — same tf * idf weighting, same normalization — since Rocchio is literally averaging query and document vectors together. If they lived in different scales, the blend in q_new would be dominated by whichever vector happened to have larger magnitude, not by the alpha/beta/gamma weights you intended.

What changes after feedback

Marking doc 2 relevant and doc 1 non-relevant, for example, pulls terms that co-occur with the query in doc 2 (like rabbit, fast, runs if those are doc 2's other terms) into the reformulated query, and suppresses terms that were specific to doc 1 but not shared with doc 2. The net effect: documents sharing that expanded vocabulary — even ones that scored low or zero on the original two-word query — can rise sharply in the re-ranked results. This is exactly how "more like this" / "not helpful, hide these" feedback loops work in production search systems.

Edge cases

• No relevant docs marked: the beta term contributes nothing; the query only shrinks away from non-relevant terms.
• No non-relevant docs marked: symmetric — the query only grows toward relevant terms.
• A term appearing in feedback docs but not the original query gets added to the reformulated query outright (its alpha * q.get(t, 0) term is just 0).