* http: realistic UA; fall back to WebPage on probe failure
Replace bare 'archivr/0.1' user agent with a full Chrome 131 UA string
(archivr/0.1 token retained at the end) in both probe_url_kind and download.
More importantly, stop hard-failing when probe_url_kind returns an error.
Sites behind Cloudflare's managed JS challenge (e.g. Medium) return 403
before any header tuning can help — a plain HTTP client cannot pass the
challenge. Instead, log a warning and fall back to Source::WebPage so that
the SingleFile/Chromium path gets a chance; a real browser can solve the
challenge transparently.
* capture: close dialog on Archive; rich per-URL and batch toast notifications
UX changes:
- Pressing Archive closes the capture dialog immediately; jobs continue
polling in the background (component stays mounted).
- Probe failures (e.g. Cloudflare JS challenge, HTTP 403) now fall back to
Source::WebPage so SingleFile/Chromium can attempt the capture instead of
hard-failing at the probe stage.
Toast notifications:
- Single URL: green 'Archived' on success, amber 'Archived with warnings'
(with expandable detail) when uBlock/cookie-ext was skipped, red 'Capture
failed' with expandable error text on failure. Per-item warning/error toasts
include the locator so the user knows which URL was affected.
- Multi-URL batch: per-item failure and warning toasts still fire with
locators; per-item success toasts are suppressed. Once all jobs settle a
single summary toast fires: 'N archived', 'N archived (M with warnings)',
'N archived (M with warnings), F failed', or 'N failed'. The summary Detail
section lists the exact URLs that failed or warned, so the user retains that
information after per-item toasts auto-dismiss.
- Batch summary color: green = all clean; amber = any warnings or failures
present; red = all failed.
- handleIgnoreUblock now only removes per-item warning toasts (those with a
locator) and persists the ignore flag; batch summary warnings (locator=null)
are not swept.
ToastStack improvements:
- All three branches (success/warning/error) support a toast.headline field
so batch summaries can set their own copy.
- Warning branch: Details button conditional on toast.text; Ignore button
conditional on toast.locator (uBlock-specific, not shown on batch summaries).
- Locator display uses hostname/…/last-segment for URLs (preserves domain for
context, tail for identity) and tail-truncation for shorthands; full locator
in title attribute for hover.
- Icon colors: green for success (✓), amber for warning (⚠), red for error (✕).
* feat: uBlock Origin Lite integration for ad-blocking during WebPage captures
- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
--load-extension); single-file connects via --browser-server instead of
launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
(pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
archivr and archivr-server wrappers
Env vars:
ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)
* feat: Extensions settings tab + capture dialog redesign with Advanced options
Settings/Extensions tab (admin-only):
- New 'Extensions' tab between Cookies and Storage
- ExtensionsTab component: shows uBlock Origin Lite card with pill toggle
- Reads ublock_enabled from instance settings; patch via existing PATCH endpoint
- Shows ublock_ext_available status from server (whether ARCHIVR_UBLOCK_EXT is set)
Instance settings:
- Add ublock_enabled BOOLEAN (default true) to instance_settings auth DB table
- Idempotent ALTER TABLE migration in initialize_auth_schema()
- get/update_instance_settings include ublock_enabled
- GET /api/admin/instance-settings now also returns ublock_ext_available (computed
from ARCHIVR_UBLOCK_EXT env var at request time)
- PATCH /api/admin/instance-settings accepts ublock_enabled
Per-capture override:
- CaptureBody gains ublock_enabled: Option<bool>
- CaptureConfig gains ublock_enabled: Option<bool>
- singlefile::save() gains ublock_enabled_override: Option<bool> param
- Capture handler resolves: body override > global instance setting > env var
- submitCapture(aid, loc, qual, extensions) in api.js passes ublock_enabled
Capture dialog redesign:
- Archive button: full-width, 13px padding, min-width 220px, primary CTA
- Cancel: full-width but text-style, below Archive
- ‹Advanced options› chevron toggle (rotates on open)
- Expanded panel shows uBlock toggle for this capture session
- Loads global ublock_enabled default from instance settings on mount
Styles:
- .ext-toggle pill switch (44×24 and 36×20 small variant)
- .ext-card for Settings Extensions tab
- .capture-advanced + .capture-advanced-panel + .capture-chevron
- .capture-ext-row / .capture-ext-label / .capture-ext-name / .capture-ext-desc
- .form-hint utility class
* fix: remove ublock_enabled from INSERT OR IGNORE in DDL batch
The INSERT ran before the ALTER TABLE migration added the column,
causing 'table instance_settings has no column named ublock_enabled'
on existing databases. The INSERT OR IGNORE for the default row only
needs the original columns; the migration's DEFAULT 1 handles the
new column for existing and new rows alike.
* feat: Reader mode via Mozilla Readability.js
Adds an opt-in 'Reader mode' advanced option to the capture dialog.
When enabled, Readability.js is injected as a browser script during
SingleFile capture; it fires on single-file-on-before-capture-start,
replaces the page body with the distilled article content, injects a
clean typographic stylesheet, and adds a header with title/byline/site.
Falls back silently if Readability fails (e.g. non-article pages).
- vendor/readability/Readability.js Apache 2.0, Mozilla, v0.6.0
- singlefile.rs: embed READABILITY_JS + READER_MODE_WRAPPER_JS via
include_str!; write both to temp dir when reader_mode is true;
base_single_file_cmd now accepts &[&Path] for multiple --browser-script
- capture.rs: CaptureConfig.reader_mode: bool
- routes.rs: CaptureBody.reader_mode: Option<bool> (defaults false)
- api.js: submitCapture passes reader_mode in payload
- CaptureDialog.jsx: Reader mode toggle in Advanced options (off by default)
* fix: diagnose single-file no-output-file error + prevent stdout dumping
- Add --dump-content=false to every single-file invocation to prevent
the Docker-detection heuristic from routing HTML to stdout instead of
the output file (the heuristic can trigger in some macOS environments)
- Improve the no-output-file error message to include: temp dir contents,
stderr, and first 200 chars of stdout — this gives enough context to
diagnose any remaining cause without re-running
* fix: switch uBlock loading from --browser-server to --browser-args
The --browser-server (CDP) path caused 'Unexpected server response: 404'
on macOS Chrome because simple-cdp's WebSocket upgrade to the debugger
endpoint failed after Chrome started — likely a version-specific CDP
endpoint shape mismatch.
New approach: single-file always manages Chrome. When ARCHIVR_UBLOCK_EXT
is set, --headless=new, --load-extension, and --disable-extensions-except
are injected via --browser-args. single-file's browser.js prefix-strips
its own conflicting flags before appending ours, so --headless=new
overrides the default --headless (enabling extension support in headless).
Removes allocate_free_port, wait_for_chrome_ready, run_single_file_with_server
(all dead code now). Docblock updated to reflect actual behaviour and notes
the --single-process caveat: uBOL's declarativeNetRequest static rulesets
are expected to work (network-stack level, not service-worker), but this
has not been mechanically verified under --single-process.
Smoke tested on macOS (this machine): capture with --load-extension + all
three browser-scripts (strip, Readability, reader-mode wrapper) produces
output file correctly. Ad-blocking verification deferred to manual test
with a tracker-heavy URL.
* fix: use correct single-file hook event (single-file-on-before-capture-request)
Prior scripts listened on 'single-file-on-before-capture-start' which
does not exist in single-file-core 1.1.49. The real hook is:
single-file-on-before-capture-request (dispatched by initUserScriptHandler
after receiving single-file-user-script-init; userScriptEnabled defaults
to true in args.js so it always fires when --browser-script is passed)
Changes:
- strip-scripts: -start -> -request (no preventDefault needed; synchronous)
- READER_MODE_SCRIPT: -start -> -request; add 'installed' meta marker at
script-evaluation time so artifact inspection can distinguish 'script
not injected' / 'hook never fired' / 'Readability parse failed'
* fix: correct singlefile.rs docstring (scripts.js concatenates, not isolates)
* fix: dispatch single-file-user-script-init so request hook fires
single-file's initUserScriptHandler (in single-file-bootstrap.js) listens
for 'single-file-user-script-init' and only then installs
_singleFile_waitForUserScript. Without that dispatch our scripts'
'single-file-on-before-capture-request' listeners were never reached,
so neither strip-scripts nor reader-mode Readability applied.
Dispatch the init event at the top of strip-scripts (always present) and
redundantly in READER_MODE_SCRIPT. Verified end-to-end: artifact for
run_b3181d6d276e4e56a1a6c356ef9bbe8f has
meta content="applied", max-width:680px CSS, 0 script tags.
* feat: cookie consent extension support (ARCHIVR_COOKIE_EXT)
Mirrors the uBlock Origin Lite integration exactly:
Backend:
- singlefile.rs: resolve_cookie_ext_config() reads ARCHIVR_COOKIE_CONSENT
(default true) + ARCHIVR_COOKIE_EXT path; extension paths comma-joined
into --load-extension / --disable-extensions-except so uBlock and cookie
ext can coexist; SaveResult.cookie_ext_skipped tracks miss
- database.rs: cookie_ext_enabled column on instance_settings (DEFAULT 1);
idempotent ALTER TABLE migration; get/update wired through
- capture.rs: CaptureConfig.cookie_ext_enabled: Option<bool>; threaded to
singlefile::save(); cookie_ext_skipped surfaced in CaptureResult
- routes.rs: CaptureBody + UpdateInstanceSettingsBody get cookie_ext_enabled;
capture handler resolves effective value (body overrides global); notes_json
only includes skipped fields that are true; GET instance-settings includes
cookie_ext_available from env path check
Frontend:
- api.js: submitCapture forwards cookie_ext_enabled
- SettingsView.jsx: 'I Still Don't Care About Cookies' card in Extensions
tab; always-active toggle (user can disable even when ext not installed);
amber 'Not configured' hint + ARCHIVR_COOKIE_EXT guidance when unavailable
- CaptureDialog.jsx: 'Block cookie banners' toggle in Advanced options;
always shown with amber hint when ext not configured; defaults from
global setting
Operator setup: download + unzip the extension from GitHub releases, set
ARCHIVR_COOKIE_EXT=/path/to/unpacked/ext. No Node daemon needed.
* fix: surface cookie_ext_skipped warning toast in CaptureDialog
* feat: package istilldontcareaboutcookies in flake, wire ARCHIVR_COOKIE_EXT
Add isdcac derivation mirroring ublockLite:
- Fetches ISDCAC-chrome-source.zip v1.1.9 from GitHub releases
- Validates manifest.json at extension root before install (guard against
nested-folder zip regressions in future releases)
- Sets ARCHIVR_COOKIE_EXT in both archivr and archivr_server wrappers
Verified: nix build .#archivr-server and .#archivr both succeed;
wrapper scripts export correct store paths; manifest.json present at root.
* fix: gate consent-overlay cleanup on cookie_ext; reset overflow; narrow selectors
- Strip overflow:hidden from body/html only when cookie_ext is active for
the capture — prevents mutating legitimate pages when the feature is off
- Remove .fc-dialog (Google Funding Choices), .qc-cmp2-*, .sp-message-container,
#sp-cc, #usercentrics-root as fallback for CMPs the extension misses
- Removed overbroad [class^="uc-"] and [id^="usercentrics"] selectors
that could match real page content
* fix: remove ad placeholders when uBlock active; kept height causes blank gap
uBlock Origin Lite blocks ad network requests but first-party placeholder
elements (ins.adsbygoogle, #aswift_* iframe hosts) retain their computed
height (e.g. 280px for a top banner), leaving a large blank space at the
top of captured pages.
Gate cleanup on ublock_ext.is_some(): remove ins.adsbygoogle, aswift_*
iframes, and google_ads_* iframes before SingleFile serialises. Also
collapse the parent container if it becomes empty after removal.
* fix: walk up to .top-ad/.google-auto-placed ancestor before removing ad slot
Removing only the inner ins.adsbygoogle left the outer .container.top-ad
wrapper (with pb-4 padding) in the layout, preserving the blank gap.
Now walk up via closest() to the nearest ad-slot container class before
removal so the whole slot including padding collapses.
* feat: add YouTube Music and Spotify source detection
- Add Source variants: YouTubeMusicTrack, YouTubeMusicPlaylist,
SpotifyTrack, SpotifyAlbum, SpotifyPlaylist
- ytm:ID shorthand → music.youtube.com/watch?v=ID (audio-only, forced
in core regardless of caller quality hint)
- ytm:playlist/ID and music.youtube.com/playlist URLs detected but
fail with 'not yet implemented' via fail_run
- Spotify URLs/shorthands detected and fail fast with clear DRM error
via fail_run (after run item created, so status is visible in /runs)
- source_metadata: youtube_music/music/audio and spotify/music/audio
(entity_kind='music' for UI pill, representation_kind='audio' stored)
- locator_to_ytdlp_url includes YouTubeMusicTrack for probe endpoint
- generate_entry_title: 'Title — Artist' for YTM tracks
- Frontend: isVideoSource handles ytm: and music.youtube.com/watch;
Spotify returns false (no probe, clear server error on submit)
- Placeholder updated to include ytm:ID
- SOURCE_ICONS: youtube_music (red disc) and spotify (green waves)
- 14 new tests covering all new sources (163 total, all pass)
* fix: prevent yt-dlp playlist expansion and stalled run recovery
- Add --no-playlist to ytdlp::download and fetch_metadata: URLs with a
list= parameter (e.g. music.youtube.com/watch?v=ID&list=RDAMVM…) no
longer cause yt-dlp to expand the full playlist and hang; both the
metadata probe and the download are now single-item only
- Fix fail_stalled_capture_jobs to also recover archive_runs and
archive_run_items: capture_jobs.run_uid is NULL at crash time so a
join is unreliable; instead fail all archive_runs/items still
in_progress directly, then recount failed_count via subquery.
Startup recovery now makes the Runs UI reflect the correct failed
state after a hard shutdown
- Expand fail_stalled_jobs_on_restart test to assert archive_run and
archive_run_item rows are also marked failed, not just capture_jobs
* fix: use play triangle for youtube_music icon
- ytdlp::download() accepts quality: Option<&str>; quality_format()
maps best/1080p/720p/480p/360p to yt-dlp -f format strings
- perform_capture() threads quality through to the downloader
- CaptureBody gains optional quality field; capture_handler validates
it against the allowlist (400 on unknown values) before spawning
- CLI passes None (preserves existing best-quality behaviour)
- Frontend: isVideoSource() mirrors determine_source() exactly —
shows quality picker only for yt-dlp-backed sources, excludes
playlist/channel shorthands and tweet/thread paths
- submitCapture(archiveId, locator, quality) sends quality in POST body
- CSS: .capture-quality styles the inline select to fit the capture row
- Tests: quality_format unit tests in ytdlp.rs; two new route tests
(valid quality accepted, invalid quality rejected with 400)
- Docs: video quality section added under Supported Platforms
CaptureDialog:
- Replace single textarea with multi-row inputs; + button adds rows
- Submit fires all pending rows in parallel, dialog stays open/usable
- Polling intervals live on a persistent ref (not cleared on close) so
toasts fire even after the dialog is dismissed
- archiveId stored per item at submit time; page-refresh reconnect uses
it.archiveId instead of the possibly-null prop
- Completed rows flash green then self-remove; failed rows show inline
error + retry button
- Cancel becomes Close while jobs are in flight
ToastStack (new component):
- Fixed bottom-right overlay with spring-in animation
- Error toast: truncated locator, View error / Hide toggle expanding
full error_text in a monospace pre block
- Auto-dismisses after 7 s; timer pauses while detail is expanded
RunsView:
- Failed rows are clickable and expand a full-width detail row showing
error_summary in a scrollable monospace block
capture.rs (archivr-core):
- Staging dir is now "{millis}-{uuid}" — parallel captures in the same
millisecond can no longer collide on temp paths
- create_archive_run moved before URL Content-Type probe so every
attempt appears in /runs regardless of outcome
- Probe failures now call create_archive_run_item with source_metadata
fallback then fail_run, recording error_text on the item and
error_summary on the run with correct failed_count
styles.css:
- Capture dialog: header row, multi-row layout, status dots, spinner,
add-row dashed button, per-row error text
- Toast stack: fixed overlay, error card with coloured left border,
monospace detail expansion
- Run error rows: clickable hover tint, expand hint chevron, detail pre
- Save and restore dialog open/closed state in sessionStorage (App.jsx)
- Persist form data: locator, error, busy, jobStatus, jobUid (CaptureDialog.jsx)
- Auto-resume polling if capture job was in progress before page refresh
- Only clear form on fresh user click, not when restoring from refresh
- Clean up sessionStorage when capture completes successfully
Fixes: Capture pop-up disappears on page refresh with unsaved data
