PHAROS DATA ARK · 2026-10-29-01024f5b59
Source availability checks
Use Check now beside Last checked in the landing page's Content sources heading. Anyone who can read the dashboard can request this non-destructive check. It needs the Ark's upstream connection to reach publishers; it does not use the reader device's internet connection and never downloads or replaces library content.
The check runs in the API's background thread, with at most four concurrent publisher probes, a ten-second timeout per HTTP attempt, and one active sweep at a time. It uses HEAD and, for a publisher refusing HEAD with 405/501, a Range GET whose body is not read. A publisher may ignore that Range header; the client still closes the response without consuming its body. A sweep is limited to 2,000 configured assets and a one-minute interval between starts. There is no new periodic timer. Repeated clicks during a running sweep show that sweep rather than starting another.
The button shows progress while the page stays usable. On completion, the timestamp and per-file online source labels update in place. The timestamp retains its regular font, device timezone, and relative age. A failure preserves the previous complete cache and its timestamp. Network failures and blocked requests yield unknown observations, not proof that files are missing. 404/410 means unavailable at the checked URL; other HTTP errors are unknown. Successful source checks do not verify file identity, edition, or checksum, and are distinct from content updates and browser-capture comparisons.
Observations are stored atomically in /opt/dataark/.cache/manifest-upstream.json. The shared lock is .cache/source-check.lock. Both the web control and python3 /opt/dataark/tools/check_upstream.py use it. The page reads the latest observations on load, so a CLI check also appears after refreshing the browser. Regenerating the dashboard refreshes the static fallback too. Observations are runtime cache, not authoritative content configuration. Source URLs remain in dataark.yaml and the portable content catalogues.
API: GET /api/sources/check reads progress/results; POST with an empty JSON object starts a check. No arbitrary URLs, commands, or settings are accepted. Cross-origin browser requests are rejected. The worker is part of dataark.service; restarting that service interrupts an active sweep without replacing the old complete cache. Refresh and start again after a restart. No caretaker password, root helper, or extra package is needed.
Source: app/services/source_check.py, app/routers/source_check.py, app/services/upstream.py, app/static/assets/js/source-check.js, and the heading renderer in app/services/manifest.py. Tests cover job locking, cooldown, failed-check preservation, URL cache validity, bounded concurrency, progress, request restrictions, and manifest rendering. These sources and this guide are included in the recovery source kit and GitHub repository.
On narrow displays the source heading keeps the title and Check now button together on the first row. The smaller timestamp and status move below as a group when there is not enough room; they remain alongside the title on wider displays.
Scheduled release-notice checks
The separate weekly reference-check worker can use an asset’s optional source_check_url instead of its download URL. This is useful for a pinned software archive whose new releases are announced in a small publisher manifest, such as the Ark bootstrap kit. The normal download URL and expected SHA-256 still identify the actual package; the notice has its own change baseline. A change is reported for review and never installs a replacement. This scheduled check is distinct from the heading’s immediate reachability sweep described above.