feat(consolidate): delayed retry of transient crawl failures + reconcile permanent ones
Apple's CDN returns sporadic 5xx/429 under load; those pages resolve moments later, but a fresh-corpus build (APPLE_DOCS_HOME is a temp dir on CI) would otherwise ship them missing. Add a bounded, DELAYED retry sweep that re-fetches only transient failures (HTTP 5xx/408/429 + transport errors) after a backoff — a clean crawl finds none and pays no delay. Also reconcile permanent failures that aren't actually missing, so they stop being reported as gaps: - cross-adapter false positives — e.g. swift-compiler / swift-package-manager are served by the swift-docc adapter (docs.swift.org), not apple-docc; the bare developer.apple.com reference 404s but the content is already in the corpus under that adapter's keys. - JSON:API `-data.dictionary` relationship nodes (App Store Connect / Enterprise Program REST schema artifacts) — structural, never standalone pages; their content lives in the parent resource. Verified against the live web that every remaining "Not found" (a dictionary property leaf, a dangling Apple article link) is genuinely non-recoverable.
G
Gigi committed
3bac4c53e6678656fee37ec7fe3ed6fc9b85ece5
Parent: df2a47d