Adds an opt-in 'Reader mode' advanced option to the capture dialog.
When enabled, Readability.js is injected as a browser script during
SingleFile capture; it fires on single-file-on-before-capture-start,
replaces the page body with the distilled article content, injects a
clean typographic stylesheet, and adds a header with title/byline/site.
Falls back silently if Readability fails (e.g. non-article pages).
- vendor/readability/Readability.js Apache 2.0, Mozilla, v0.6.0
- singlefile.rs: embed READABILITY_JS + READER_MODE_WRAPPER_JS via
include_str!; write both to temp dir when reader_mode is true;
base_single_file_cmd now accepts &[&Path] for multiple --browser-script
- capture.rs: CaptureConfig.reader_mode: bool
- routes.rs: CaptureBody.reader_mode: Option<bool> (defaults false)
- api.js: submitCapture passes reader_mode in payload
- CaptureDialog.jsx: Reader mode toggle in Advanced options (off by default)
- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
--load-extension); single-file connects via --browser-server instead of
launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
(pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
archivr and archivr-server wrappers
Env vars:
ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)
Adds per-instance cookie rules (admin-only) that are injected into
every network touchpoint during capture.
Storage:
- New cookie_rules table in the auth DB (idempotent migration)
- Rules have pattern_kind (global/wildcard/regex), optional url_pattern,
and cookies_json (validated as string-only JSON object)
Matching (resolve_cookies_for_url):
- Global rules always apply
- Wildcard: * and ? with full metacharacter escaping; matched against
hostname via reqwest::Url when pattern has no ://, full URL otherwise
- Regex: matched against the full URL
- Later rules in ordinal order override earlier ones per cookie name
All six network touchpoints receive resolved cookies:
- http::probe_url_kind and http::download: Cookie request header
- singlefile::save: Netscape cookie file -> --browser-cookies-file
- ytdlp::fetch_metadata and ytdlp::download: Netscape cookie file -> --cookies
- tweets::archive: semicolon credentials file -> --credentials-file
(only when both ct0 and auth_token are present; otherwise falls back
to ARCHIVR_TWITTER_CREDENTIALS_FILE)
Security:
- Cookie files written 0o600 (owner read/write only)
- Exact parsed hostname used as cookie domain (no PSL stripping)
- Files deleted unconditionally before any error propagates,
including spawn failures (hold-result-then-delete pattern)
- No cookie values in process args (no --add-header exposure)
API: GET/POST /api/admin/cookie-rules, PATCH/DELETE /api/admin/cookie-rules/:uid
Frontend: Cookies tab in Settings (admin only) with rule list,
inline edit, pattern-type selector, client-side JSON validation
CLI: CaptureConfig::default() - no behaviour change
254 tests passing (4 new cookie-rule handler tests)
* feat: add YouTube Music and Spotify source detection
- Add Source variants: YouTubeMusicTrack, YouTubeMusicPlaylist,
SpotifyTrack, SpotifyAlbum, SpotifyPlaylist
- ytm:ID shorthand → music.youtube.com/watch?v=ID (audio-only, forced
in core regardless of caller quality hint)
- ytm:playlist/ID and music.youtube.com/playlist URLs detected but
fail with 'not yet implemented' via fail_run
- Spotify URLs/shorthands detected and fail fast with clear DRM error
via fail_run (after run item created, so status is visible in /runs)
- source_metadata: youtube_music/music/audio and spotify/music/audio
(entity_kind='music' for UI pill, representation_kind='audio' stored)
- locator_to_ytdlp_url includes YouTubeMusicTrack for probe endpoint
- generate_entry_title: 'Title — Artist' for YTM tracks
- Frontend: isVideoSource handles ytm: and music.youtube.com/watch;
Spotify returns false (no probe, clear server error on submit)
- Placeholder updated to include ytm:ID
- SOURCE_ICONS: youtube_music (red disc) and spotify (green waves)
- 14 new tests covering all new sources (163 total, all pass)
* fix: prevent yt-dlp playlist expansion and stalled run recovery
- Add --no-playlist to ytdlp::download and fetch_metadata: URLs with a
list= parameter (e.g. music.youtube.com/watch?v=ID&list=RDAMVM…) no
longer cause yt-dlp to expand the full playlist and hang; both the
metadata probe and the download are now single-item only
- Fix fail_stalled_capture_jobs to also recover archive_runs and
archive_run_items: capture_jobs.run_uid is NULL at crash time so a
join is unreliable; instead fail all archive_runs/items still
in_progress directly, then recount failed_count via subquery.
Startup recovery now makes the Runs UI reflect the correct failed
state after a hard shutdown
- Expand fail_stalled_jobs_on_restart test to assert archive_run and
archive_run_item rows are also marked failed, not just capture_jobs
* fix: use play triangle for youtube_music icon
- ytdlp::download() accepts quality: Option<&str>; quality_format()
maps best/1080p/720p/480p/360p to yt-dlp -f format strings
- perform_capture() threads quality through to the downloader
- CaptureBody gains optional quality field; capture_handler validates
it against the allowlist (400 on unknown values) before spawning
- CLI passes None (preserves existing best-quality behaviour)
- Frontend: isVideoSource() mirrors determine_source() exactly —
shows quality picker only for yt-dlp-backed sources, excludes
playlist/channel shorthands and tweet/thread paths
- submitCapture(archiveId, locator, quality) sends quality in POST body
- CSS: .capture-quality styles the inline select to fit the capture row
- Tests: quality_format unit tests in ytdlp.rs; two new route tests
(valid quality accepted, invalid quality rejected with 400)
- Docs: video quality section added under Supported Platforms
CaptureDialog:
- Replace single textarea with multi-row inputs; + button adds rows
- Submit fires all pending rows in parallel, dialog stays open/usable
- Polling intervals live on a persistent ref (not cleared on close) so
toasts fire even after the dialog is dismissed
- archiveId stored per item at submit time; page-refresh reconnect uses
it.archiveId instead of the possibly-null prop
- Completed rows flash green then self-remove; failed rows show inline
error + retry button
- Cancel becomes Close while jobs are in flight
ToastStack (new component):
- Fixed bottom-right overlay with spring-in animation
- Error toast: truncated locator, View error / Hide toggle expanding
full error_text in a monospace pre block
- Auto-dismisses after 7 s; timer pauses while detail is expanded
RunsView:
- Failed rows are clickable and expand a full-width detail row showing
error_summary in a scrollable monospace block
capture.rs (archivr-core):
- Staging dir is now "{millis}-{uuid}" — parallel captures in the same
millisecond can no longer collide on temp paths
- create_archive_run moved before URL Content-Type probe so every
attempt appears in /runs regardless of outcome
- Probe failures now call create_archive_run_item with source_metadata
fallback then fail_run, recording error_text on the item and
error_summary on the run with correct failed_count
styles.css:
- Capture dialog: header row, multi-row layout, status dots, spinner,
add-row dashed button, per-row error text
- Toast stack: fixed overlay, error card with coloured left border,
monospace detail expansion
- Run error rows: clickable hover tint, expand hint chevron, detail pre
8082895 removed the ytdlp_metadata_json fetch, local_filename_title
derivation, and entry_title computation, replacing the title arg with
a None stub. Restores all three blocks so YouTube, Instagram, Reddit,
TikTok, Facebook, Snapchat, X, and local file entries receive proper
titles again.
Store how many bytes of each entry's artifacts are already on disk from
an earlier entry (content-addressed blob deduplication means shared
blobs are only stored once).
Design
------
- Add `cached_bytes INTEGER NOT NULL DEFAULT 0` to `archived_entries`
- Precompute at capture time via `database::refresh_entry_cached_bytes`
called after all artifacts are saved for every capture path
(web page, generic URL, tweet, yt-dlp/local)
- One-time migration in `initialize_schema`: detects missing column via
PRAGMA table_info, ALTERs the table, then back-fills all existing rows
with the correlated subquery
- `database::cascade_cached_bytes_after_delete` ready for when entry
deletion is implemented; designed to run asynchronously after the
delete is acknowledged to the user
- `cached_bytes` included in `EntrySummary` and all four SELECT paths
(list_root_entries, search_entries, list_entries_for_collection,
entries_for_tag) via the shared ENTRY_SELECT_COLS constant
Frontend
--------
- `EntryRow` shows a `% cached` sub-line under the size when non-zero,
with a tooltip showing the raw cached byte count
- No separate API endpoint or extra fetch — value rides in the existing
entries list response at zero extra query cost per read
- capture.rs: add archive_id: Option<&str> to perform_capture; when Some,
call font_extractor::extract_and_rewrite before hashing HTML, register
each font as a deduplicated blob + 'font' artifact
- main.rs: pass None as archive_id (CLI keeps fonts embedded)
- routes.rs: add GET /api/archives/:id/blobs/:sha256 (serve_blob handler),
pass Some(&archive_id) to perform_capture in capture_handler,
add ApiError::internal constructor
http::download now returns (hash, extension, Option<title_hint>).
Title is derived from Content-Disposition filename header (RFC 5987
filename*= preferred over plain filename=), falling back to the last
path segment of the final URL after redirects (percent-decoded).
capture.rs Source::Url arm passes the title through to record_media_entry
instead of None, so entries like 'Facharbeit.pdf' or 'facharbeit' appear
in the Title column instead of the raw entry_uid.
Adds percent_decode(), title_from_content_disposition(), title_from_url()
helpers with 10 new unit tests (122 total, all green).
- perform_capture: fetch yt-dlp metadata (separate --dump-json call)
before download; derive local title from file:// path filename
- Compute entry_title before record_media_entry call; pass it through
instead of the temporary None placeholder
- record_tweet_entry: read tweet JSON before entry creation so title
can be set on NewEntry; add tweet_metadata_from_json helper
- Add PlatformMetadata struct and generate_entry_title() covering all 13
Source variants (YouTubeVideo, YouTubePlaylist, YouTubeChannel, X, Tweet,
TweetThread, Instagram, Facebook, TikTok, Reddit, Snapchat, Local, Other)
- Add downloader/metadata.rs: extract_from_ytdlp_json() parses yt-dlp
--dump-json output into PlatformMetadata; extracts Reddit subreddit
from webpage_url via regex
- Add ytdlp::fetch_metadata(): separate --dump-json invocation that does
not interfere with the actual download call
- Extend record_media_entry() with title: Option<String> param; wire
yt-dlp metadata fetch + local filename extraction in perform_capture
- Restructure record_tweet_entry() to read tweet JSON before entry
creation; extract title via tweet_metadata_from_json()
- 16 title-generation unit tests + 6 metadata extraction unit tests
expand_shorthand_to_url handled instagram:/facebook:/tiktok: etc. but
not yt:/youtube:. yt-dlp doesn't know the yt: scheme, so shorthands
like yt:video/ID failed with 'Unsupported url scheme'.
Add YouTube cases for video/, short/, shorts/, playlist/, channel/,
c/, user/, and @handle. Full https:// URLs pass through unchanged.
Add tests pinning all new expansions.
- Add archivr-core/src/capture.rs: Source enum, all capture helpers,
fail_run (replaces process::exit fail_archive_and_exit), and
perform_capture() as the public entry point
- Slim archivr-cli/src/main.rs to 103 lines: thin adapter over
archivr_core::capture::perform_capture
- Move all source-classification and tweet-entry tests from CLI to
archivr-core/src/capture.rs (they test core logic, belong in core)
- Add POST /api/archives/:archive_id/captures route in archivr-server:
validates non-empty locator (400), resolves archive (404), delegates
to perform_capture, returns {run_uid, status}
- Add capture dialog to browser UI: dialog markup in index.html,
showModal/submit/cancel/loading/error wiring in app.js, dialog
styles in styles.css
- 77 tests pass, clean build (0 warnings)
