← Pharos Data Ark

PHAROS DATA ARK · 2026-10-29-01024f5b59

Handbook contents

Content configuration, shipping and restoration

Restoration update: Optional content now requires a capacity-checked preset and explicit queue approval. The legacy content_profile.py restore --apply path is disabled. Fresh bootstrap does not start optional downloads or automatic download timers. See Restore presets and queue. Spare-media acceptance remains pending.

Two copies with different jobs

The authoritative public content configuration lives on the Pi's system drive under /opt/dataark/config/content/. The content SSD carries a readable mirror at /mnt/storage/DATA-ARK-CONFIG/. The repository provides the off-appliance copy on GitHub. Browse the media-drive mirror.

This separates desired content from the medium holding it. Replacing or emptying the SSD does not erase the system-drive selection. Moving the old SSD to another computer carries its source descriptions with it. Runtime download progress, partial files and update locks are not the authoritative selection.

The shell welcome remains on the system drive. Run ark-help library for the everyday starting points.

What is recorded

Generated paper packs are regenerated by their source program. Offline browser tools and their vendored dependencies live in the source repository; they are published by web deployment. They are not redownloaded as giant library payloads.

The intentionally removed Wikipedia-without-images archive is kept as enabled: false. Collection recipes restore their catalogues as groups: to omit an individual form/reference permanently, edit its source catalogue as well as reviewing the profile. This exporter is an explicit snapshot of managed content, not a background watcher of library-manager actions.

An older, unregistered wikipedia_en_medicine_maxi_2026-01.zim was discovered. It is reported as an unmanaged archive, not silently selected. Preserve it separately or add an explicit source recipe if it should be retained. Private bugout backups/drops, system-volume metadata and temporary download caches are outside this public recipe.

Capture and mirror choices

After intentional library changes, capture the current installed selections and review the Git diff:


cd /opt/dataark
sudo /usr/bin/python3 tools/content_profile.py export
git diff -- config/content deploy/reference

Export writes the system-drive profile and its media-drive mirror. It refuses an unmounted or empty content drive; an empty inventory must not overwrite the saved desired library. To edit desired choices without rescanning current files, edit config/content/content.yaml and the referenced catalogues, then copy them to the SSD using:


sudo /usr/bin/python3 tools/content_profile.py mirror

Commit the reviewed public configuration files to the source repository. Neither export nor mirror pushes Git automatically. Refresh the snapshot after adding or removing content through older management interfaces; those interfaces do not yet maintain this profile automatically.

Preview and rebuild

On the configured Ark, open /recovery.html from Manage Library. Review None, Minimal, Recommended and Full with their capacity calculations. No downloads start from that page. Unknown sizes and insufficient space disable preset approval. Disabled selections stay disabled; selected archives still being installed remain part of recovery intent.


cd /opt/dataark
venv/bin/python tools/recovery_queue.py plan --preset minimal
# Review the plan and copy its plan_id before approving:
sudo venv/bin/python tools/recovery_queue.py create --preset minimal --accept-plan PLAN_ID
sudo systemctl start dataark-recovery.service

The queue persists package state and attempts, prioritizes small independent packages, verifies pinned files before reuse, preserves originals on failure and exposes completed direct files immediately. Archive completion refreshes Kiwix and the landing page. Failures block dependent packages while independent work continues. Collection installers retain their own detailed source reports, so successful recipe completion does not imply every publisher file was acquired.


venv/bin/python tools/recovery_queue.py status
sudo venv/bin/python tools/recovery_queue.py pause
sudo venv/bin/python tools/recovery_queue.py cancel
# Resume unfinished packages explicitly:
sudo systemctl start dataark-recovery.service

Cancellation does not delete completed or partial content. Rebooted/interrupted work requires explicit resume. A changed profile or installer source requires a new reviewed plan. The service has no timer or automatic restart. No format or erase operation exists. The legacy content_profile.py restore --apply entry point now refuses to run; its plan --only command remains a diagnostic inventory preview, not an approved recovery queue.

See Restore presets and queue for staging, locks, capacity assumptions, restart behavior and the remaining spare-media acceptance matrix.

Ship a small configuration bundle


/usr/bin/python3 tools/content_profile.py bundle --output /tmp/dataark-content-config.tar.gz

The archive contains only the allowlisted public YAML/JSON profile, source snapshots and referenced content configuration files. It excludes library payloads, passwords, network credentials, private backup directories and unrelated observed machine-configuration files. Keep it with a copy of the matching source repository. Extract the bundle into that repository to put config/content and deploy/reference back in place, preview the plan, then explicitly restore.

This is content restoration for a prepared Ark, not yet a tested bare-Pi-image bootstrap. A replacement SSD must be formatted and mounted by the existing appliance setup; this command does not choose, format or mount a disk. Rebuild-on-boot is a possible separate policy, not enabled by this change. If added, it should read the system-drive desired profile, wait for the actual content mount and internet, retain selections, and expose progress/failures rather than blocking the local library indefinitely.

What configuration alone cannot guarantee

Pinned URLs can disappear. FCC complete files are rolling snapshots; without the preserved source ZIP a later restore imports a later callbook. Some publisher documents change at stable URLs. Rebuilt HTML includes new retrieval metadata and is not byte-identical to the original capture. Exact map extraction may depend on the recorded source and extractor behavior. Missing copyrighted code books and publisher-blocked documents remain gaps, not magically restored content.

For guaranteed recovery without publishers or internet, retain an independent content mirror (including source ZIPs and licensing notices), plus a protected appliance-configuration backup. The readable content profile describes what to restore; it is not a substitute for those bytes or for a hardware/OS recovery image.

Verification

The complete profile was exported and dry-run planned without deleting content. Tests cover traversal and credential-bearing URL rejection, disabled selections, dependency ordering, unknown selections, checksum failure preserving an existing file, successful staged replacement and the bundle allowlist. A small real publisher file is restored into an isolated test directory as an integration check. A full empty-SSD rebuild of the hundreds-of-gigabytes collection has not been performed; that remains an acceptance test before promising unattended disaster recovery.