- database.rs: add has_active_capture_jobs(), list_orphaned_blob_rows(),
all_referenced_file_relpaths(), delete_orphaned_blob_rows()
- routes.rs: GET/DELETE /api/archives/:id/blob-cleanup (ROLE_ADMIN)
- GET returns {orphaned_blob_rows, deletable_files, total_bytes}
- DELETE has two active-capture guards (before and after disk walk)
to prevent deleting files mid-capture; walks raw/ and raw_tweets/
- Referenced set = entry_artifacts.relpath ∪ live blobs' raw_relpath,
so a file is never deleted if any artifact still points at its path
- api.js: scanOrphanBlobs(), deleteOrphanBlobs()
- App.jsx: pass archiveId to SettingsView; add 'storage' to SETTINGS_TABS
so /settings/storage survives refresh/back navigation
- SettingsView.jsx: new Storage tab (admin-only) with idle→scanning→
scanned→deleting→done/error state machine; shows file/record counts
and human-readable byte sizes before a btn-danger confirm
Tests (14 new, 248 total passing):
- database.rs: has_active_capture_jobs for pending/running/completed,
list_orphaned_blob_rows, all_referenced_file_relpaths edge cases,
delete_orphaned_blob_rows preserves referenced rows
- routes.rs: auth (401), active-capture 409 on GET and DELETE,
end-to-end delete preserving referenced file and removing orphan
blob file + extra disk-only file
- ytdlp::download() accepts quality: Option<&str>; quality_format()
maps best/1080p/720p/480p/360p to yt-dlp -f format strings
- perform_capture() threads quality through to the downloader
- CaptureBody gains optional quality field; capture_handler validates
it against the allowlist (400 on unknown values) before spawning
- CLI passes None (preserves existing best-quality behaviour)
- Frontend: isVideoSource() mirrors determine_source() exactly —
shows quality picker only for yt-dlp-backed sources, excludes
playlist/channel shorthands and tweet/thread paths
- submitCapture(archiveId, locator, quality) sends quality in POST body
- CSS: .capture-quality styles the inline select to fit the capture row
- Tests: quality_format unit tests in ytdlp.rs; two new route tests
(valid quality accepted, invalid quality rejected with 400)
- Docs: video quality section added under Supported Platforms
Add --window-size=1920,1080 to the Chromium flags passed via --browser-args.
This makes the existing --remove-unused-styles=false and
--remove-alternative-medias=false effective for real @media rules
and responsive styles (headless default is small).
Also document in ARCHIVR_CHROME_ARGS that users can override by
supplying their own --window-size in the env var.
CaptureDialog:
- Replace single textarea with multi-row inputs; + button adds rows
- Submit fires all pending rows in parallel, dialog stays open/usable
- Polling intervals live on a persistent ref (not cleared on close) so
toasts fire even after the dialog is dismissed
- archiveId stored per item at submit time; page-refresh reconnect uses
it.archiveId instead of the possibly-null prop
- Completed rows flash green then self-remove; failed rows show inline
error + retry button
- Cancel becomes Close while jobs are in flight
ToastStack (new component):
- Fixed bottom-right overlay with spring-in animation
- Error toast: truncated locator, View error / Hide toggle expanding
full error_text in a monospace pre block
- Auto-dismisses after 7 s; timer pauses while detail is expanded
RunsView:
- Failed rows are clickable and expand a full-width detail row showing
error_summary in a scrollable monospace block
capture.rs (archivr-core):
- Staging dir is now "{millis}-{uuid}" — parallel captures in the same
millisecond can no longer collide on temp paths
- create_archive_run moved before URL Content-Type probe so every
attempt appears in /runs regardless of outcome
- Probe failures now call create_archive_run_item with source_metadata
fallback then fail_run, recording error_text on the item and
error_summary on the run with correct failed_count
styles.css:
- Capture dialog: header row, multi-row layout, status dots, spinner,
add-row dashed button, per-row error text
- Toast stack: fixed overlay, error card with coloured left border,
monospace detail expansion
- Run error rows: clickable hover tint, expand hint chevron, detail pre
8082895 removed the ytdlp_metadata_json fetch, local_filename_title
derivation, and entry_title computation, replacing the title arg with
a None stub. Restores all three blocks so YouTube, Instagram, Reddit,
TikTok, Facebook, Snapchat, X, and local file entries receive proper
titles again.
- Save and restore dialog open/closed state in sessionStorage (App.jsx)
- Persist form data: locator, error, busy, jobStatus, jobUid (CaptureDialog.jsx)
- Auto-resume polling if capture job was in progress before page refresh
- Only clear form on fresh user click, not when restoring from refresh
- Clean up sessionStorage when capture completes successfully
Fixes: Capture pop-up disappears on page refresh with unsaved data
Store how many bytes of each entry's artifacts are already on disk from
an earlier entry (content-addressed blob deduplication means shared
blobs are only stored once).
Design
------
- Add `cached_bytes INTEGER NOT NULL DEFAULT 0` to `archived_entries`
- Precompute at capture time via `database::refresh_entry_cached_bytes`
called after all artifacts are saved for every capture path
(web page, generic URL, tweet, yt-dlp/local)
- One-time migration in `initialize_schema`: detects missing column via
PRAGMA table_info, ALTERs the table, then back-fills all existing rows
with the correlated subquery
- `database::cascade_cached_bytes_after_delete` ready for when entry
deletion is implemented; designed to run asynchronously after the
delete is acknowledged to the user
- `cached_bytes` included in `EntrySummary` and all four SELECT paths
(list_root_entries, search_entries, list_entries_for_collection,
entries_for_tag) via the shared ENTRY_SELECT_COLS constant
Frontend
--------
- `EntryRow` shows a `% cached` sub-line under the size when non-zero,
with a tooltip showing the raw cached byte count
- No separate API endpoint or extra fetch — value rides in the existing
entries list response at zero extra query cost per read
* chore: add Dockerfile, docker-compose, and Docker docs
- Multi-stage Dockerfile: Rust builder stage + debian:bookworm-slim runtime
with Chromium, Node/single-file-cli, Python venv (yt-dlp + twitter-api-client)
- docker-compose.yml: wires ARCHIVR_BIND, config volume, and persistent data volume
- docker/config.example.toml: annotated TOML template for Docker deployments
- docs/README.md: add Hosting with Docker section; add ARCHIVR_BIND and
ARCHIVR_STATIC_DIR to the Environment Variables reference
* fix: address code review issues with Docker setup
- .gitignore: whitelist Dockerfile, docker-compose.yml, docker/ so they
are actually tracked (the * catch-all was silently dropping them)
- Dockerfile: build and ship the archivr CLI alongside archivr-server so
users can run `archivr init` inside the container on first setup
- docker/config.example.toml: fix archive_path to point at the .archivr
subdirectory that archivr init creates (not the parent directory), which
is what read_archive_paths expects
- docs/README.md: replace the bare mkdir quickstart step with
`archivr init`, explain why mkdir is insufficient; add a callout that
auth_db_path must be set explicitly to a writable path when the config
mount is read-only
* fix: address second round of Docker review issues
Chromium sandbox (P2):
- singlefile.rs: add ARCHIVR_CHROME_ARGS env var (space-separated flags
appended to Chromium's --browser-args JSON array); Dockerfile sets it
to --no-sandbox because Chromium refuses to start as root without it
Store-path outside volume (P1):
- README: pass explicit absolute store-path as the second positional arg
to `archivr init` so the blob store lands on /data instead of the
container layer (CLI default is ./.archivr/store, resolved from cwd,
which is / with no WORKDIR set)
ENTRYPOINT vs CMD (P2):
- Dockerfile: switch from ENTRYPOINT to CMD so `docker compose run
archivr archivr init …` overrides the full command instead of being
appended to the server invocation
ffmpeg missing (P2):
- Dockerfile: add ffmpeg to the apt-get install block (required by
yt-dlp --merge-output-format mp4 for bestvideo+bestaudio streams)
Node version (P2):
- Dockerfile: replace Debian bookworm's nodejs (18.x) with Node 20 via
the NodeSource setup script (single-file-cli declares engines.node >=20)
Build context secrets (P2):
- Add .dockerignore excluding config/ and docker/ from the build context
so runtime secrets (e.g. twitter-cookies.txt) are never sent to the builder
- Whitelist .dockerignore in .gitignore
docs:
- README: document ARCHIVR_CHROME_ARGS in the Environment Variables section
* fix: third round of Docker review issues
Rust toolchain (P1):
- Dockerfile: bump builder from rust:1.87 to rust:1.88; time@0.3.51,
time-core@0.1.9, and time-macros@0.2.30 (present in Cargo.lock) all
require MSRV 1.88, so the real cargo build --release step was failing
single-file-cli wait mode (P2):
- singlefile.rs: replace --browser-wait-until=networkidle2 with
networkAlmostIdle; the single-file-cli option only accepts
InteractiveTime/networkIdle/networkAlmostIdle/load/domContentLoaded
(verified in options.js); networkidle2 is a Puppeteer concept that the
CLI does not recognise, causing silent fallback to the earliest state
and incomplete captures. networkAlmostIdle is the closest equivalent
(<=2 open connections, matching Puppeteer's networkidle2 semantics)
Build context size (P3):
- .dockerignore: add target/, frontend/node_modules/, frontend/dist/;
these can reach 1.4G+ after a local dev build and are never read by
the Dockerfile, so sending them to the builder wastes time and memory
NixOS module (4 fixes):
- Remove RestrictNamespaces=true: Chromium (launched by single-file for
web captures) needs Linux user namespaces; blocking them broke captures
- Escape TOML string values: labels/paths with quotes or backslashes
would produce invalid archivr-server.toml and prevent startup
- Bracket IPv6 listen addresses: ::1:8080 is rejected by Rust's
SocketAddr parser; [::1]:8080 is the correct form (RFC 2732)
- Add optional storePath per archive: covers archives initialised with
a custom store path outside the default sibling store/ directory
Rust (1 fix):
- extract_client_ip loopback branch: take last XFF value not first.
When a proxy appends to X-Forwarded-For an attacker controls the
first entry; the last entry is always set by the trusted local proxy
Frontend (1 fix):
- Gate fetchArchives on authState === 'authenticated'. The empty-dep
useEffect fired on mount (before login), hit 401, and never retried
after login because authState was not in the dependency array
Fix 1 — NixOS: store/ dir missing from ReadWritePaths
archive_path points to .archivr/; captures write artifacts to the
sibling store/ directory. Whitelisting only .archivr/ caused POST
/captures to fail under ProtectSystem=strict.
Fix: use builtins.dirOf a.path to whitelist the parent, which
contains both .archivr/ and store/.
Fix 2 — Rust: X-Forwarded-For was unconditionally trusted
An attacker could send a different IP on every login attempt,
bypassing LOGIN_MAX_ATTEMPTS entirely.
Fix: use ConnectInfo<SocketAddr> (via into_make_service_with_connect_info)
as the primary rate-limit key; XFF is trusted only when the TCP peer
is loopback (i.e. a local reverse proxy). Tests without a real socket
fall back to XFF unchanged.
Port configuration:
NixOS module: split 'bind' string into separate 'listenAddress'
(default 127.0.0.1) and 'port' (default 8080) options. openFirewall
now uses cfg.port directly instead of parsing it from the bind string.
README NixOS example updated accordingly.
Both handlers accepted AuthUser but served guests.
auth.require_auth()? added as first line closes the gap.
Existing unauthenticated tests updated with session cookies.
5 attempts / 15 min per IP. Excess attempts get 429 + Retry-After.
Client IP read from X-Forwarded-For with loopback fallback.
In-memory; prune inline on each check.
Refactored app() into app_with_state() + thin wrapper.
Uses parking_lot::Mutex (no unwrap, no poisoning).
Injects X-Content-Type-Options, X-Frame-Options, Referrer-Policy,
Content-Security-Policy, and Permissions-Policy on every response.
Registered as outermost layer so it covers setup_guard 503s too.
No new dependencies.
- LoginPage/SetupPage: centered card layout with display font, styled fields and submit button
- Topbar: fix duplicate Settings nav item; add styled user-menu with username + logout button
- RunsView: format ISO timestamps to readable dates, add colored status badges (completed/failed/running)
- AdminView: view-tabs system, styled admin-table, admin-input, status-badge (active/disabled), btn-primary
- SettingsView: replace all inline styles with form-section/form-field/field-input/btn-primary/btn-danger
- CollectionsView: restructure create form with proper field labels and btn-primary
- TagsView: add Tags section heading with separator
- ContextRail: show visibility as human-readable label not raw number; fix assign-error class
- styles.css: add auth-loading, view-tabs, form utilities, btn variants, status badges,
run-status pills, token-banner/row, checkbox-row, coll-create-form, tag-tree-header CSS
