The Most-Cited Authority in Our Database Was Not a Judgment

Every legal research tool sorts by citation count somewhere. It is the obvious signal — a judgment that a thousand later benches have relied on is probably more load-bearing than one nobody has mentioned since, and unlike relevance scoring it needs no cleverness to compute. Count the inbound references, sort descending, ship.

We did that, and then we looked at what came back. Sorted by citation count, the top of our record of Indian authority reads like this:

Authority	Times cited
The Code of Civil Procedure, 1908	497,050
Article 21 of the Constitution	136,760
The Income Tax Act, 1961	110,052
Gian Singh v. State of Punjab (2012)	53,834
Dataram Singh v. State of Uttar Pradesh (2018)	35,378

The first real judgment is fourth, and it trails a procedural code by roughly ten to one. Nothing there is a counting error. The CPC genuinely is referred to half a million times, because almost every civil order in India mentions it. It is simply not an authority on a point in the way a judgment is, and a list that ranks them together is comparing two things that do not share a unit.

How a statute ends up in a judgment table

We built the record by ingesting a large body of Indian legal material. Judgments came with citations, parties, dates and a court. Alongside them came material that arrives through the same pipes and looks structurally identical — statutes, Law Commission reports, transcripts of Lok Sabha debates. They have titles, dates, citing documents. They sit in rows.

So they went into the same table, and for a long time nothing in the system said they were different. When we finally counted, 159 rows in a table called judgments were not judgments: 126 statutes, 21 Law Commission and committee reports, and 12 parliamentary debates.

Eighty-nine of them had been given a ratio decidendi.

That is the part that stops you. A ratio is the binding principle a court laid down when deciding a case. A statute does not have one. A Law Commission report does not have one. A debate in the Lok Sabha very much does not have one. Somewhere in the pipeline, a process that extracts the ratio from a judgment had been handed a statute, and had done what such processes do when they are given the wrong input — it produced a fluent, well-formed, entirely invented answer, and wrote it into the field.

Nobody lied. Nothing was maliciously constructed. A generic step ran on material it was never designed for, and the output looked exactly like the output it produces on real judgments, which is precisely why it survived so long.

What we did

We added a field. Every row in that table now declares what kind of document it is — judgment, statute, report or debate — and the classification is populated on every row, with none left null. The counts today are 68,698 judgments, 126 statutes, 21 reports and 12 debates.

Then two rules followed from it, and the rules are the actual fix:

Case-law retrieval filters on doc_kind = 'judgment'. When Miss Lucy goes looking for the authority that settles your point, the Code of Civil Procedure is not a candidate, because it is not an answer to that question.

The fabricated ratios are gone. Every non-judgment row now carries an empty ratio field rather than a plausible sentence. We did not try to write a better one. There was nothing to write.

Why "most cited" is a worse signal than it looks

Even inside the judgments, the citation count is not the ranking most people assume it is.

A count is a measure of how often a document is referred to, and documents get referred to for many reasons that have nothing to do with being right. Procedural provisions are cited because every order recites them. Bail matters cite Arnesh Kumar and Dataram Singh by the thousand because they set the standard directions that get reproduced. A judgment that settled a narrow point conclusively, so conclusively that nobody has needed to litigate it since, may be cited a handful of times in forty years and still be the answer to your question.

So the count tells you something real — this authority is in wide circulation — and it does not tell you the thing you asked, which is what governs my facts. Used as a tiebreak between two judgments that both actually apply, it is useful. Used as the ranking, it will hand you the most quoted document in the general area and let you assume it is the most relevant one.

The general shape of the problem

The reason this is worth a blog post rather than a line in a changelog is that the failure is not really about statutes in a judgment table. It is about what happens when a step in a pipeline is confidently applied to material it was not built for.

The extraction step was working correctly. The ingestion was working correctly. The citation counts were arithmetically right. Every individual component did its job, and the composition of them produced 89 sentences of invented law and a ranking that put a procedural code above every judgment in India.

That is the characteristic failure mode of building anything on top of a language model, and it does not announce itself. There is no error, no exception, no empty field. There is a well-formed answer where there should have been nothing at all, and the only way you find it is to go and count.

We now go and count, on a schedule, and we write down what the counts were. It is unglamorous and it is most of the job.

---

Counts run against Miss Lucy's own record on 12 August 2026 and move as the collection grows. Citation counts are inbound references within the ingested corpus and are not comparable across document types — which is, more or less, the point of the post.