SIGN IN SIGN UP

feat(consolidate): delayed retry of transient crawl failures + reconcile permanent ones

Apple's CDN returns sporadic 5xx/429 under load; those pages resolve moments
later, but a fresh-corpus build (APPLE_DOCS_HOME is a temp dir on CI) would
otherwise ship them missing. Add a bounded, DELAYED retry sweep that re-fetches
only transient failures (HTTP 5xx/408/429 + transport errors) after a backoff —
a clean crawl finds none and pays no delay.

Also reconcile permanent failures that aren't actually missing, so they stop
being reported as gaps:
- cross-adapter false positives — e.g. swift-compiler / swift-package-manager
  are served by the swift-docc adapter (docs.swift.org), not apple-docc; the
  bare developer.apple.com reference 404s but the content is already in the
  corpus under that adapter's keys.
- JSON:API `-data.dictionary` relationship nodes (App Store Connect / Enterprise
  Program REST schema artifacts) — structural, never standalone pages; their
  content lives in the parent resource.

Verified against the live web that every remaining "Not found" (a dictionary
property leaf, a dangling Apple article link) is genuinely non-recoverable.
G
Gigi committed
3bac4c53e6678656fee37ec7fe3ed6fc9b85ece5
Parent: df2a47d