Parsing error for citations with defendant 'Thompson'

[ERosendo]

In issue #3924, we identified a bug in Eyecite's parsing method when the defendant's last name is 'Thompson'.

For example, for the citation 'Shapiro v. Thompson, 394 U. S. 618':

• Expected output: volume: 394, reporter: 'U.S.', page: '618'
• Actual output: volume: None, reporter: 'Thompson', page: '394'

Other examples of inputs that are incorrectly parsed are: Adams v. Thompson, 560 F. Supp. 894 and Mozena v. Thompson, 44 A.2d 276.

I've been using the first example to debug this issue, and noticed that Eyecite identifies two tokens within the input string: "Thompson's Unreported Cases (TN)" and "United States Supreme Court Reports.". The problem arises because these tokens overlap (both include "394") and Eyecite's tokenize method prioritizes the rightmost token when encountering overlaps, leading to this results.

[mlissner]

Any idea how easy this is to solve so that it identifies each?

[mlissner]

Per discussion today, seems to be happening when citations appear to overlap. The simple solution here is to find both citations that overlap and then filter out the one that's incomplete.

[quevon24]

The problem here is that we have Thompson as a nominative reporter (optional volume), that's why it is detected as a citation.

We could detect the overlap and see which one is incomplete but according to the reporters-db Thompson regex, Thompson , 394 is a valid citation, we get two valid citations: Thompson , 394 and 394 U. S. 618

This can be replicated with other nominative reporters like:

• Foo v. Cooke (Tennessee Reports, Cooke)
• Foo v. Holmes (Holmes Circuit Court Reports (US))
• Foo v. McCahon (Kansas Reports, McCahon)
• Foo v. Olcott (Olcott's Admiralty Reports)
• Foo v. Taney (Taney's United States (US) Circuit Court Reports)

[mlissner]

Seems like some post processing could detect and remedy this. The Thompson one may be technically complete, but it's the same data. So maybe: If there's overlap and the data overlaps too, then....

[flooie]

@mlissner I like your solution -

if there is an overlapping span - we can check if one is a complete citation and if one is nominative and choose the complete citation.