[Linxiushen]

Summary

EnglishNumberNormalizer.preprocess could silently delete the words "and a half" from a sentence, and failed to normalize the phrase when it followed a digit rather than a spelled-out number. This PR fixes both in the same loop and adds four regression assertions that fail on main.

input	before	after
2 and a half million	2 and a half 1000000	2500000
48 and a half dollars	48 and a half dollars	$48.5
and a half hours later	hours later	and a half hours later
one and a half and a half	1.5	1.5 and a half

Root cause

preprocess rewrites "<number> and a half" as "<number> point five" by splitting the text on \band\s+a\s+half\b and re-joining the pieces. Two defects in that loop:

1. "and a half" could be dropped from the text. The if len(segment.strip()) == 0: continue guard skipped an empty segment and, with it, the "and a half" marker that belonged after it. So a sentence-initial or repeated occurrence lost words outright:

• "and a half hours later" -> "hours later"
• "and a half of the people" -> "of the people"
• "one and a half and a half" -> "1.5"

Because this normalizer is applied to both reference and hypothesis when computing WER, silently dropping words can mask or manufacture errors.

2. Only spelled-out numbers were recognised. The check was last_word in self.decimals or last_word in self.multipliers, and self.decimals holds only word forms ({*self.ones, *self.tens, *self.zeros}). A digit before the phrase fell through to the else branch, producing output that is neither the original text nor a normalized number:

• "2 and a half million" -> "2 and a half 1000000"
• "48 and a half dollars" -> "48 and a half dollars" (expected "$48.5")

Digit input is squarely in scope for this class: process_words already tests next_is_numeric = re.match(r"^\d+(\.\d+)?$", next), and the existing tests cover std("b747"), std("10 th"), std("200 thousand") and std("minus 500"). The spelled-out form was already asserted to work (std("three and a half million") == "3500000"), so the digit form was an inconsistency rather than an unsupported input.

Fix

Restructured the loop so the segment is always emitted and only the marker is decided:

• Append every segment, including empty ones, and break after the last one. Nothing can be dropped any more. The extra whitespace this can introduce is harmless — __call__ already does " ".join(... s.split()).
• Derive last_word from segment.split() with an explicit empty-list fallback. This replaces the strip() guard the old code relied on, and also removes a latent IndexError: the old segment.rsplit(maxsplit=2)[-1] would have raised on a whitespace-only segment.
• Add re.fullmatch(r"\d+", last_word) to the "is this a number?" test, using the same \d convention as process_words' existing next_is_numeric check. Decimals are deliberately excluded, so "1.5 and a half" is left untouched rather than becoming "1.5 point five".

Confined to one method; no public behaviour outside the "and a half" path changes.

Testing

Four assertions added to test_number_normalizer, which is parametrized over both EnglishNumberNormalizer and EnglishTextNormalizer. I verified each one fails on main for both normalizers and passes with the fix — by checking out origin/main in a throwaway git worktree, copying in the patched test file, and running it with PYTHONPATH pinned to that worktree (confirming the worktree's english.py was the module actually imported).

• pytest tests/test_normalizer.py — 4 passed; together with tests/test_tokenizer.py — 8 passed.
• black --check, isort --check-only and flake8 (the repo's pre-commit args) are all clean on the two changed files.
• To bound the blast radius I diffed old against new over generated inputs: 3695 inputs containing no "and a half" produced zero differences. Of 1600 inputs containing the phrase, 529 changed, all in the two intended categories (restoring deleted text, or normalizing a digit form); none became malformed.
• Not run: test_transcribe.py / test_audio.py / test_timing.py, which need model downloads and torch inference and exercise nothing in this change.

One note for reviewers

"0 and a half" yields ".5" whereas "zero and a half" yields "0.5". That asymmetry already exists in the digit path on main ("0 point five" -> ".5"); this change routes the digit form into it rather than introducing it. Happy to normalize that separately if you'd like it addressed here.

🤖 Generated with Claude Code