Twenty papers, one rate limit

A summary of twenty papers takes a minute. Defending it takes a week. The reviewer always asks the same three things: where did this claim come from, who disagrees, and how strong is the evidence? Nodus is built to answer those three questions, which it calls lineage, disagreement and quality, for every claim in its report.

To do that it can't just summarise abstracts. It has to actually read each paper: fetch it, split it into sections, pull out the individual claims, embed them, and only then compare claims across papers.

The first pipeline did exactly that, one paper at a time. It was correct, and slow by construction: every paper waited for the one before it, and almost all of the waiting was network time. Fetching a PDF, calling the model and calling the embedding API were each fast to ask for and slow to answer.

asyncio.gather over twenty papers worked for about one run.

Two different limits, two different mechanisms. This distinction is the heart of the fix.

• Capped: an asyncio.Semaphore limits how many papers are in flight (default 10). That protects memory, connections and the database.
• Paced: a separate limiter controls how often a call may start. A cap alone doesn't enforce a rate: four calls that each take two seconds is still 120 requests a minute. Chat and embeddings each get their own pacer, because Google meters them separately.
• Refusals are respected: a 429 is retried after the delay the API itself attaches, plus jitter, so twenty refused calls don't all come back at the same instant.
• Semantic Scholar is throttled in-process to one call every 1.1 seconds. If relevance search is refused, retrieval falls back to bulk search and remembers that choice for the rest of the process.

The cheapest call is the one you don't make.

• Papers are global, not per-query. They're de-duplicated on their Semantic Scholar ID, and normalisation and extraction are reused across questions. A paper read once is never read again.
• Embeddings are cached per claim, keyed by model, so switching provider throws away old vectors rather than comparing across incompatible spaces.
• The extractor gets the results and conclusions, not the whole paper. Study design, population and sample size arrive as a compact context block.
• A paper with no abstract, TLDR or full text is skipped, not sent to the model.

A missing PDF shouldn't sink a run. Nodus walks down a short ladder, and says honestly where it stopped.

A failure only costs one paper. A dead PDF, an unparseable paper or one failed cluster analysis degrades that unit only. The run finishes, and the report states what it couldn't read.

Five minutes is long to stare at a spinner. So progress streams over a WebSocket: each paper as it is read, each cluster as it forms. You watch the evidence arrive, and the wait becomes part of the product.

With the reading solved, the hard problems became judgement.

• Clustering without knowing how many clusters. Greedy leader clustering with running centroids. The similarity threshold has to be tuned per embedding model.
• The chat is grounded or it refuses. It answers from the report only, and says covered: false when the answer isn't there.
• Human edits win. A cluster you edit is pinned, and re-running the analysis never overwrites it.