Turbo Rerank

How it works

The original experiment asked Jev two things per search. First, one score question per candidate passage — how well does this passage answer the query? — on a four-level scale: irrelevant, tangential, partially answers, directly answers. Second, one noul question over the whole candidate set: does at least one of these passages directly answer the query? All 50 candidates went out in a single request.

Here the 598-passage Northwind corpus and the BM25 index are built in your browser, so retrieval costs about 0.05 ms and the left column fills as you type, before any request leaves the page. Pressing Rerank sends the candidates to /yes-no in one call of up to 32 texts — 32 candidates is one call, 50 is two — with the passages as texts and one statement carrying your question:

{
  "texts": [ "<passage 1>", "<passage 2>", … up to 32 ],
  "statement": "This passage answers the question: <your question>",
  "when_true":  "The passage states the specific information the question asks for.",
  "when_false": "The passage is about something else, or only shares words with the question."
}

Candidates are sorted by the returned probability, with the BM25 position as the tiebreak. The four relevance levels of the original become bands over that one probability: answers it above 0.7, partly answers above 0.25, weak signal above 0.02, off topic below. The boundaries come from the benchmark: correct top hits land at 0.09, 0.14, 0.15, 0.30, 0.49 and then 0.75 upwards, so a low probability means low confidence rather than a different subject. The second Jev question needs no extra request: the answer probability of the whole set is the highest probability among the candidates, and the “no good answer in the corpus” banner appears when that stays under 0.02 — four times below the 0.09 floor measured for a correct top hit, and ten times above the 0.002 a question the corpus really cannot answer returns.

Why /yes-no and not /rate. The four levels are ordinal, so /rate is the natural route, and the brief maps them that way. It does not work here: /rate and /classify judge the text they are given, and a query pasted above the passage is read as part of the passage — on the 40-query benchmark every candidate came back at the top level and reranking made the order worse (top-1 50% → 50%, MRR 0.59 → 0.45). Putting the question inside the statement instead is the one shape that carries it, which is what /yes-no allows. Measured on the same 40 questions at 50 candidates: top-1 50% → 90%, top-5 65% → 90%, MRR 0.59 → 0.91, about 500 ms per call.

The Benchmark tab runs those 40 hand-labelled questions through the same path, one at a time. The run is bounded by the list — 40 questions, 74 calls, about a minute — with a 90-second backstop and a Stop button. Nothing fires until you press a button, and every call shares the page-wide queue of 2 per second.

Northwind is fictional and every passage is synthetic. Numbers on this page are measured live, not recorded.