An update purge still deletes live children behind a redirect, and after two failed runs

[xroche]

An update purge still deleted live files in two shapes #1390 left open. A hub that is a 301 has its cache entry stored headers-only, and the read invalidates it for naming no local file, so nothing said the hub had carried its target and everything below it: one interrupted transfer took the whole subtree. A hub that fails on two consecutive runs has no cache entry at all the second time, because a failed fetch writes none, so a child that lost its other parent in that window went with it. Neither is a regression; a build of master from before #1390 loses the same files.

The guard now reads the redirect off the Location the entry recorded, which outlives that invalidation, and when no entry is left it falls back to the copy #746 kept on disk and the name it was saved under. Both new arms sit below the check that the entry answered a status at all, because Location: is parsed and cached whatever the status: read above that check, a 200 blob carrying a stray Location suppresses the whole run's purge. Requiring the kept copy to be present is what stops a brand-new dead .html link from holding the purge forever, the trap #1390 named when it rejected typing a link by its savename.

The issue proposed keying on old.lst's (from URL) column instead. There is no such column: filenote() writes [savename] and nothing else, and (from URL) belongs to hts-cache/new.txt, a debug log. It could not carry either case anyway, since a redirect's child records the grandparent as its referer, and a run that failed never fetched the children at all.

The trade worth your decision. A hold covers the whole run rather than the failed page's subtree, because the subtree is exactly what the run does not know, and opt->links_unqueued is set once with nothing that releases it. Until now that hold could not outlive one run: the second failure wiped the cache entry the guard keyed on, so a page that stayed unreachable started purging again on the next update. With the fallback surviving the missing entry, one previously mirrored .html link that is permanently unreachable holds --purge-old off for the life of the mirror, which then keeps growing with files the site really did drop. The engine says so once per run in hts-log.txt, without naming the page. I took that over the alternative because the files a purge would take in this state are still reachable from the local copy the engine deliberately kept, and a mirror whose pages point at deleted files fails its reader worse than one carrying stale files it reported. Bounding it is implementable: old.lst, new.lst and the kept copy all survive the failure that loses the cache entry, so a per-link counter has somewhere to live. What the bound should be is a policy call about how long a mirror protects a subtree behind an unreachable parent, so the hold is unbounded here and the choice is yours.

Test 354 covers the redirect hub, the double failure, a blob that must hold nothing back, a 200 carrying a Location, and a hub saved under a name its URL does not give. Eight mutants of the guard ran against 348 and 354; seven red the intended assertion, always-on and always-off reding both files. The eighth, dropping the HTTP_IS_REDIRECT arm on its own, stays green: every redirect entry reachable through the engine is invalidated on read and caught by the Location arm below it, and the arm stays as the honest predicate for a 3xx that did name a local file. 348 gains the markers its survival checks never grepped, and premises for the two shapes that had none, since dropping -m,4000 used to leave the size-cap shape green and the answered-500 shape asserted nothing about the cache recovery that happens ahead of the give-up arm.

Two findings from this work are filed rather than fixed. #1415: an empty Content-Type reads as the no-declared-type sentinel and therefore as hypertext, which falsifies the negative control #1395 named and is why 354 controls with an explicit application/octet-stream blob. #1421: a page whose local name comes from its cached type loses that name after a failed fetch, so #746's keep misses and this PR's fallback inherits the miss; 354 now carries a non-.html hub through the first failure, so the suite can see the shape.

Closes #1395