Batch number field lookups in Dirichlet character search columns (LMFDB#6008)

[roed-math]

The kernel field and value field columns added in LMFDB#6004 issued two database queries per row
each (a nf_fields lookup by coefficients plus a nf_fields_extra lookup), roughly 200
round trips per 50-row results page. This adds a postprocessing step to the search that
computes all defining polynomials up front and finds the matching number field labels in a
single nf_fields query; the column display functions use the precomputed labels (falling
back to the old per-row behavior if the data is absent). Profiling shows the pari
computations (galoissubcyclo/polredabs) take ~9ms per page, so the database round trips were
the entire cost, answering the question raised in the issue. Rendered search-results tables
are byte-identical before and after on sample searches, downloads are unchanged, and page
render time against a remote database drops from 14-30s to under 1s (16-65x). The batch
query uses $or of equalities rather than $in because psycodict omits the ::numeric[]
typecast on the array-$in path.
Addresses LMFDB#6008.

🤖 Generated with Claude Code

[roed-math]

Follow-up commit dabca39 addresses the review of 27cedf0: the batch was incomplete, so a cold results page still did database work proportional to the number of distinct kernel fields on it.

The gap

character_postprocess kept only the label of each matched number field, so formatfield built a WebNumberField holding nothing but that label, and field_pretty reloads a field whenever the pretty name needs more than the label (degree 3 wants disc_abs/disc_sign/coeffs; generic degree 4 and degree 2^k want subfields/subfield_mults). On /Character/Dirichlet/?modulus=13&order=4&search_type=List&showcol=first, whose orbit 13.d has kernel field 4.0.2197.1, that was 5 record-level lookups for the one displayed cell:

nf_fields.lookup('4.0.2197.1')             # field_pretty reloading the quartic
nf_fields.lucky({'label': '4.0.2197.1'})   #   (the query inside that lookup)
nf_fields_extra.lookup('4.0.2197.1')
nf_fields.lucky({'coeffs': [-3, -1, 1]})   # case 5b, resolving the quadratic subfield
nf_fields_extra.lookup('2.2.13.1')

Changes

1. character_postprocess keeps the whole nf_fields record rather than just the label (db.nf_fields.search(nf_query) with no projection, so still one query) and attaches it as the virtual column kernel_field_data, using {"label": "N/A"} when the field is not in the database, which formatfield already reads as missing. display_kernel_field hands that record straight to formatfield. No nf_fields_extra batch is needed: every column field_pretty consults lives in nf_fields.
2. Case 5b of field_pretty (imprimitive quartics) resolved the unique quadratic subfield by looking it up by coefficients. It now reads the squarefree radicand off the stored subfield polynomial the way cases 5a and 7 already do, so no lucky/lookup is left in the render path. The nested helper _quad_label_to_D becomes unused and is removed.
3. Tests: test_field_columns gains the unambiguous label=4.0.2197.1 assertion (that field is not the value field of any character, unlike 4.0.125.1), plus a missing-kernel-field case (modulus=5002&order=12) and a missing-value-field case (order=47); new test_field_columns_no_lookups clears the field_pretty cache, patches nf_fields.lookup, nf_fields.lucky and nf_fields_extra.lookup to raise, and renders the kernel cell; new test_field_columns_download pins the 13.d download row.

Verification

• Reproducer: 5 record lookups down to 0, same knowl \(\Q(\sqrt{-26 -6 \sqrt{13}})\). A 50-row page of order ≤ 12 characters: 0 lookups.

• test_field_columns_no_lookups, rewritten against the old kernel_label API, fails on 27cedf0 with the patched-lookup assertion, so it is testing something real.

• Rendered <table class="ntdata"> fragments are byte-identical before and after on 9 searches (the default page with and without showcol=first, then modulus=13&order=4, order=1-12, modulus=1-200&order=4, modulus=1-500&order=8, modulus=1-500&order=3, order=13-100, is_primitive=yes&modulus=1-1000) and on 4 downloads (text and sage, with and without showcol=first).

• Case 5b parity: 3758 quartics with a unique quadratic subfield (the 1450 such fields among the 4000 smallest-discriminant quartics, plus 2308 more sampled from disc_abs > 1e12, > 1e16 and > 1e20). Old and new radicands agree in every case, the single subfield is always quadratic, and none was missing from nf_fields.

• Value fields were already clean: for all 43 orders that can occur (euler_phi(order2) <= 23) the 28 distinct value fields are all in nf_fields and none needs a record lookup to display, since field_pretty catches them in its cycloinfo branch.

• Cold render times against devmirror, min of 3, clearing the field_pretty and nf_display_knowl caches before each run:

  search (&showcol=first)	before	after
  modulus=1-500&order=4	11.18s	0.92s
  modulus=1-500&order=8	7.91s	0.91s
  modulus=1-500&order=3	4.53s	0.87s
  modulus=13&order=4	1.45s	0.86s
  default page	1.29s	0.45s

• pyflakes, pylint (with CI's flags) and ruff check --preview --select=E722 clean on the three changed files.

• sage -python -m pytest lmfdb/number_fields/test_numberfield.py -q: 34 passed (16m33s). sage -python -m pytest lmfdb/characters/test_characters.py -k field_columns -q: 3 passed (40s). The full characters module is still running locally: devmirror is slow right now and test_nchars (a full-table sum('degree'), unrelated to this change) has been blocked for over an hour. Leaving that to CI, which covers both modules.

One behavior note

Case 5b no longer requires the quadratic subfield to be in nf_fields. Where the old code fell back to the bare label because that subfield was missing, the new code prints the pretty name; the radicand is computed from the subfield polynomial, so it is correct either way. Sampling found no such quartic, and the change can only add pretty names, never remove them.

[roed314]

GPT signed off.

[roed-math]

Superseded by LMFDB#7175, opened upstream from this same branch. Closing here; review continues upstream.