- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
--load-extension); single-file connects via --browser-server instead of
launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
(pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
archivr and archivr-server wrappers
Env vars:
ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)
Adds per-instance cookie rules (admin-only) that are injected into
every network touchpoint during capture.
Storage:
- New cookie_rules table in the auth DB (idempotent migration)
- Rules have pattern_kind (global/wildcard/regex), optional url_pattern,
and cookies_json (validated as string-only JSON object)
Matching (resolve_cookies_for_url):
- Global rules always apply
- Wildcard: * and ? with full metacharacter escaping; matched against
hostname via reqwest::Url when pattern has no ://, full URL otherwise
- Regex: matched against the full URL
- Later rules in ordinal order override earlier ones per cookie name
All six network touchpoints receive resolved cookies:
- http::probe_url_kind and http::download: Cookie request header
- singlefile::save: Netscape cookie file -> --browser-cookies-file
- ytdlp::fetch_metadata and ytdlp::download: Netscape cookie file -> --cookies
- tweets::archive: semicolon credentials file -> --credentials-file
(only when both ct0 and auth_token are present; otherwise falls back
to ARCHIVR_TWITTER_CREDENTIALS_FILE)
Security:
- Cookie files written 0o600 (owner read/write only)
- Exact parsed hostname used as cookie domain (no PSL stripping)
- Files deleted unconditionally before any error propagates,
including spawn failures (hold-result-then-delete pattern)
- No cookie values in process args (no --add-header exposure)
API: GET/POST /api/admin/cookie-rules, PATCH/DELETE /api/admin/cookie-rules/:uid
Frontend: Cookies tab in Settings (admin only) with rule list,
inline edit, pattern-type selector, client-side JSON validation
CLI: CaptureConfig::default() - no behaviour change
254 tests passing (4 new cookie-rule handler tests)
* feat: add YouTube Music and Spotify source detection
- Add Source variants: YouTubeMusicTrack, YouTubeMusicPlaylist,
SpotifyTrack, SpotifyAlbum, SpotifyPlaylist
- ytm:ID shorthand → music.youtube.com/watch?v=ID (audio-only, forced
in core regardless of caller quality hint)
- ytm:playlist/ID and music.youtube.com/playlist URLs detected but
fail with 'not yet implemented' via fail_run
- Spotify URLs/shorthands detected and fail fast with clear DRM error
via fail_run (after run item created, so status is visible in /runs)
- source_metadata: youtube_music/music/audio and spotify/music/audio
(entity_kind='music' for UI pill, representation_kind='audio' stored)
- locator_to_ytdlp_url includes YouTubeMusicTrack for probe endpoint
- generate_entry_title: 'Title — Artist' for YTM tracks
- Frontend: isVideoSource handles ytm: and music.youtube.com/watch;
Spotify returns false (no probe, clear server error on submit)
- Placeholder updated to include ytm:ID
- SOURCE_ICONS: youtube_music (red disc) and spotify (green waves)
- 14 new tests covering all new sources (163 total, all pass)
* fix: prevent yt-dlp playlist expansion and stalled run recovery
- Add --no-playlist to ytdlp::download and fetch_metadata: URLs with a
list= parameter (e.g. music.youtube.com/watch?v=ID&list=RDAMVM…) no
longer cause yt-dlp to expand the full playlist and hang; both the
metadata probe and the download are now single-item only
- Fix fail_stalled_capture_jobs to also recover archive_runs and
archive_run_items: capture_jobs.run_uid is NULL at crash time so a
join is unreliable; instead fail all archive_runs/items still
in_progress directly, then recount failed_count via subquery.
Startup recovery now makes the Runs UI reflect the correct failed
state after a hard shutdown
- Expand fail_stalled_jobs_on_restart test to assert archive_run and
archive_run_item rows are also marked failed, not just capture_jobs
* fix: use play triangle for youtube_music icon
- ytdlp::download() accepts quality: Option<&str>; quality_format()
maps best/1080p/720p/480p/360p to yt-dlp -f format strings
- perform_capture() threads quality through to the downloader
- CaptureBody gains optional quality field; capture_handler validates
it against the allowlist (400 on unknown values) before spawning
- CLI passes None (preserves existing best-quality behaviour)
- Frontend: isVideoSource() mirrors determine_source() exactly —
shows quality picker only for yt-dlp-backed sources, excludes
playlist/channel shorthands and tweet/thread paths
- submitCapture(archiveId, locator, quality) sends quality in POST body
- CSS: .capture-quality styles the inline select to fit the capture row
- Tests: quality_format unit tests in ytdlp.rs; two new route tests
(valid quality accepted, invalid quality rejected with 400)
- Docs: video quality section added under Supported Platforms
Add --window-size=1920,1080 to the Chromium flags passed via --browser-args.
This makes the existing --remove-unused-styles=false and
--remove-alternative-medias=false effective for real @media rules
and responsive styles (headless default is small).
Also document in ARCHIVR_CHROME_ARGS that users can override by
supplying their own --window-size in the env var.
* chore: add Dockerfile, docker-compose, and Docker docs
- Multi-stage Dockerfile: Rust builder stage + debian:bookworm-slim runtime
with Chromium, Node/single-file-cli, Python venv (yt-dlp + twitter-api-client)
- docker-compose.yml: wires ARCHIVR_BIND, config volume, and persistent data volume
- docker/config.example.toml: annotated TOML template for Docker deployments
- docs/README.md: add Hosting with Docker section; add ARCHIVR_BIND and
ARCHIVR_STATIC_DIR to the Environment Variables reference
* fix: address code review issues with Docker setup
- .gitignore: whitelist Dockerfile, docker-compose.yml, docker/ so they
are actually tracked (the * catch-all was silently dropping them)
- Dockerfile: build and ship the archivr CLI alongside archivr-server so
users can run `archivr init` inside the container on first setup
- docker/config.example.toml: fix archive_path to point at the .archivr
subdirectory that archivr init creates (not the parent directory), which
is what read_archive_paths expects
- docs/README.md: replace the bare mkdir quickstart step with
`archivr init`, explain why mkdir is insufficient; add a callout that
auth_db_path must be set explicitly to a writable path when the config
mount is read-only
* fix: address second round of Docker review issues
Chromium sandbox (P2):
- singlefile.rs: add ARCHIVR_CHROME_ARGS env var (space-separated flags
appended to Chromium's --browser-args JSON array); Dockerfile sets it
to --no-sandbox because Chromium refuses to start as root without it
Store-path outside volume (P1):
- README: pass explicit absolute store-path as the second positional arg
to `archivr init` so the blob store lands on /data instead of the
container layer (CLI default is ./.archivr/store, resolved from cwd,
which is / with no WORKDIR set)
ENTRYPOINT vs CMD (P2):
- Dockerfile: switch from ENTRYPOINT to CMD so `docker compose run
archivr archivr init …` overrides the full command instead of being
appended to the server invocation
ffmpeg missing (P2):
- Dockerfile: add ffmpeg to the apt-get install block (required by
yt-dlp --merge-output-format mp4 for bestvideo+bestaudio streams)
Node version (P2):
- Dockerfile: replace Debian bookworm's nodejs (18.x) with Node 20 via
the NodeSource setup script (single-file-cli declares engines.node >=20)
Build context secrets (P2):
- Add .dockerignore excluding config/ and docker/ from the build context
so runtime secrets (e.g. twitter-cookies.txt) are never sent to the builder
- Whitelist .dockerignore in .gitignore
docs:
- README: document ARCHIVR_CHROME_ARGS in the Environment Variables section
* fix: third round of Docker review issues
Rust toolchain (P1):
- Dockerfile: bump builder from rust:1.87 to rust:1.88; time@0.3.51,
time-core@0.1.9, and time-macros@0.2.30 (present in Cargo.lock) all
require MSRV 1.88, so the real cargo build --release step was failing
single-file-cli wait mode (P2):
- singlefile.rs: replace --browser-wait-until=networkidle2 with
networkAlmostIdle; the single-file-cli option only accepts
InteractiveTime/networkIdle/networkAlmostIdle/load/domContentLoaded
(verified in options.js); networkidle2 is a Puppeteer concept that the
CLI does not recognise, causing silent fallback to the earliest state
and incomplete captures. networkAlmostIdle is the closest equivalent
(<=2 open connections, matching Puppeteer's networkidle2 semantics)
Build context size (P3):
- .dockerignore: add target/, frontend/node_modules/, frontend/dist/;
these can reach 1.4G+ after a local dev build and are never read by
the Dockerfile, so sending them to the builder wastes time and memory
- fix(hash): hash raw bytes instead of lossy UTF-8; add hash_bytes
- feat(database): add get_blob_by_sha256 lookup
- feat(font_extractor): extract embedded font data-URIs from archived HTML
- --user-agent: realistic Chrome UA so servers don't block headless string
- --browser-args=[--disable-web-security, --user-data-dir]: lets single-file
inline fonts from any cross-origin CDN (e.g. fonts.gstatic.com) regardless
of ACAO headers; user-data-dir required for --disable-web-security to take
effect in newer Chromium builds (otherwise silently ignored)
Font fidelity:
- --browser-wait-delay=2000: Cloudflare Fonts injects @font-face CSS after
HTML parse; the font hook needs extra time to see it after networkidle2
- --remove-unused-fonts=false: preserve @font-face rules even when fonts
haven't rendered yet (font-display:swap, off-screen text)
- --remove-alternative-fonts=false: preserve unicode-range subsets instead
of stripping them as 'alternatives'
ES module viewer error fix:
- Write a user script (sf-strip-scripts.js) that listens for
single-file-on-before-capture-start and removes all <script> elements
(except application/ld+json) from the live DOM before serialization.
Scripts still execute during capture for CSS fidelity; none end up in
the saved file, so no data:-URL base ES module resolution errors.
Defaults that were destroying CSS fidelity:
- --remove-unused-styles=true: strips CSS nesting rules (site uses & selector)
and any rule targeting JS-applied classes
- --remove-alternative-medias=true: deletes @media blocks that don't match
the capture viewport, breaking responsive layout
- --block-scripts=true: prevents JS from applying classes before CSS snapshot
All three now set to false.
http::download now returns (hash, extension, Option<title_hint>).
Title is derived from Content-Disposition filename header (RFC 5987
filename*= preferred over plain filename=), falling back to the last
path segment of the final URL after redirects (percent-decoded).
capture.rs Source::Url arm passes the title through to record_media_entry
instead of None, so entries like 'Facharbeit.pdf' or 'facharbeit' appear
in the Title column instead of the raw entry_uid.
Adds percent_decode(), title_from_content_disposition(), title_from_url()
helpers with 10 new unit tests (122 total, all green).
- Add PlatformMetadata struct and generate_entry_title() covering all 13
Source variants (YouTubeVideo, YouTubePlaylist, YouTubeChannel, X, Tweet,
TweetThread, Instagram, Facebook, TikTok, Reddit, Snapchat, Local, Other)
- Add downloader/metadata.rs: extract_from_ytdlp_json() parses yt-dlp
--dump-json output into PlatformMetadata; extracts Reddit subreddit
from webpage_url via regex
- Add ytdlp::fetch_metadata(): separate --dump-json invocation that does
not interfere with the actual download call
- Extend record_media_entry() with title: Option<String> param; wire
yt-dlp metadata fetch + local filename extraction in perform_capture
- Restructure record_tweet_entry() to read tweet JSON before entry
creation; extract title via tweet_metadata_from_json()
- 16 title-generation unit tests + 6 metadata extraction unit tests
