mirror of
https://github.com/thegeneralist01/archivr
synced 2026-07-22 03:05:32 +02:00
feat: uBlock Origin Lite + cookie consent extension + reader mode + ad placeholder cleanup (#21)
* feat: uBlock Origin Lite integration for ad-blocking during WebPage captures
- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
--load-extension); single-file connects via --browser-server instead of
launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
(pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
archivr and archivr-server wrappers
Env vars:
ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)
* feat: Extensions settings tab + capture dialog redesign with Advanced options
Settings/Extensions tab (admin-only):
- New 'Extensions' tab between Cookies and Storage
- ExtensionsTab component: shows uBlock Origin Lite card with pill toggle
- Reads ublock_enabled from instance settings; patch via existing PATCH endpoint
- Shows ublock_ext_available status from server (whether ARCHIVR_UBLOCK_EXT is set)
Instance settings:
- Add ublock_enabled BOOLEAN (default true) to instance_settings auth DB table
- Idempotent ALTER TABLE migration in initialize_auth_schema()
- get/update_instance_settings include ublock_enabled
- GET /api/admin/instance-settings now also returns ublock_ext_available (computed
from ARCHIVR_UBLOCK_EXT env var at request time)
- PATCH /api/admin/instance-settings accepts ublock_enabled
Per-capture override:
- CaptureBody gains ublock_enabled: Option<bool>
- CaptureConfig gains ublock_enabled: Option<bool>
- singlefile::save() gains ublock_enabled_override: Option<bool> param
- Capture handler resolves: body override > global instance setting > env var
- submitCapture(aid, loc, qual, extensions) in api.js passes ublock_enabled
Capture dialog redesign:
- Archive button: full-width, 13px padding, min-width 220px, primary CTA
- Cancel: full-width but text-style, below Archive
- ‹Advanced options› chevron toggle (rotates on open)
- Expanded panel shows uBlock toggle for this capture session
- Loads global ublock_enabled default from instance settings on mount
Styles:
- .ext-toggle pill switch (44×24 and 36×20 small variant)
- .ext-card for Settings Extensions tab
- .capture-advanced + .capture-advanced-panel + .capture-chevron
- .capture-ext-row / .capture-ext-label / .capture-ext-name / .capture-ext-desc
- .form-hint utility class
* fix: remove ublock_enabled from INSERT OR IGNORE in DDL batch
The INSERT ran before the ALTER TABLE migration added the column,
causing 'table instance_settings has no column named ublock_enabled'
on existing databases. The INSERT OR IGNORE for the default row only
needs the original columns; the migration's DEFAULT 1 handles the
new column for existing and new rows alike.
* feat: Reader mode via Mozilla Readability.js
Adds an opt-in 'Reader mode' advanced option to the capture dialog.
When enabled, Readability.js is injected as a browser script during
SingleFile capture; it fires on single-file-on-before-capture-start,
replaces the page body with the distilled article content, injects a
clean typographic stylesheet, and adds a header with title/byline/site.
Falls back silently if Readability fails (e.g. non-article pages).
- vendor/readability/Readability.js Apache 2.0, Mozilla, v0.6.0
- singlefile.rs: embed READABILITY_JS + READER_MODE_WRAPPER_JS via
include_str!; write both to temp dir when reader_mode is true;
base_single_file_cmd now accepts &[&Path] for multiple --browser-script
- capture.rs: CaptureConfig.reader_mode: bool
- routes.rs: CaptureBody.reader_mode: Option<bool> (defaults false)
- api.js: submitCapture passes reader_mode in payload
- CaptureDialog.jsx: Reader mode toggle in Advanced options (off by default)
* fix: diagnose single-file no-output-file error + prevent stdout dumping
- Add --dump-content=false to every single-file invocation to prevent
the Docker-detection heuristic from routing HTML to stdout instead of
the output file (the heuristic can trigger in some macOS environments)
- Improve the no-output-file error message to include: temp dir contents,
stderr, and first 200 chars of stdout — this gives enough context to
diagnose any remaining cause without re-running
* fix: switch uBlock loading from --browser-server to --browser-args
The --browser-server (CDP) path caused 'Unexpected server response: 404'
on macOS Chrome because simple-cdp's WebSocket upgrade to the debugger
endpoint failed after Chrome started — likely a version-specific CDP
endpoint shape mismatch.
New approach: single-file always manages Chrome. When ARCHIVR_UBLOCK_EXT
is set, --headless=new, --load-extension, and --disable-extensions-except
are injected via --browser-args. single-file's browser.js prefix-strips
its own conflicting flags before appending ours, so --headless=new
overrides the default --headless (enabling extension support in headless).
Removes allocate_free_port, wait_for_chrome_ready, run_single_file_with_server
(all dead code now). Docblock updated to reflect actual behaviour and notes
the --single-process caveat: uBOL's declarativeNetRequest static rulesets
are expected to work (network-stack level, not service-worker), but this
has not been mechanically verified under --single-process.
Smoke tested on macOS (this machine): capture with --load-extension + all
three browser-scripts (strip, Readability, reader-mode wrapper) produces
output file correctly. Ad-blocking verification deferred to manual test
with a tracker-heavy URL.
* fix: use correct single-file hook event (single-file-on-before-capture-request)
Prior scripts listened on 'single-file-on-before-capture-start' which
does not exist in single-file-core 1.1.49. The real hook is:
single-file-on-before-capture-request (dispatched by initUserScriptHandler
after receiving single-file-user-script-init; userScriptEnabled defaults
to true in args.js so it always fires when --browser-script is passed)
Changes:
- strip-scripts: -start -> -request (no preventDefault needed; synchronous)
- READER_MODE_SCRIPT: -start -> -request; add 'installed' meta marker at
script-evaluation time so artifact inspection can distinguish 'script
not injected' / 'hook never fired' / 'Readability parse failed'
* fix: correct singlefile.rs docstring (scripts.js concatenates, not isolates)
* fix: dispatch single-file-user-script-init so request hook fires
single-file's initUserScriptHandler (in single-file-bootstrap.js) listens
for 'single-file-user-script-init' and only then installs
_singleFile_waitForUserScript. Without that dispatch our scripts'
'single-file-on-before-capture-request' listeners were never reached,
so neither strip-scripts nor reader-mode Readability applied.
Dispatch the init event at the top of strip-scripts (always present) and
redundantly in READER_MODE_SCRIPT. Verified end-to-end: artifact for
run_b3181d6d276e4e56a1a6c356ef9bbe8f has
meta content="applied", max-width:680px CSS, 0 script tags.
* feat: cookie consent extension support (ARCHIVR_COOKIE_EXT)
Mirrors the uBlock Origin Lite integration exactly:
Backend:
- singlefile.rs: resolve_cookie_ext_config() reads ARCHIVR_COOKIE_CONSENT
(default true) + ARCHIVR_COOKIE_EXT path; extension paths comma-joined
into --load-extension / --disable-extensions-except so uBlock and cookie
ext can coexist; SaveResult.cookie_ext_skipped tracks miss
- database.rs: cookie_ext_enabled column on instance_settings (DEFAULT 1);
idempotent ALTER TABLE migration; get/update wired through
- capture.rs: CaptureConfig.cookie_ext_enabled: Option<bool>; threaded to
singlefile::save(); cookie_ext_skipped surfaced in CaptureResult
- routes.rs: CaptureBody + UpdateInstanceSettingsBody get cookie_ext_enabled;
capture handler resolves effective value (body overrides global); notes_json
only includes skipped fields that are true; GET instance-settings includes
cookie_ext_available from env path check
Frontend:
- api.js: submitCapture forwards cookie_ext_enabled
- SettingsView.jsx: 'I Still Don't Care About Cookies' card in Extensions
tab; always-active toggle (user can disable even when ext not installed);
amber 'Not configured' hint + ARCHIVR_COOKIE_EXT guidance when unavailable
- CaptureDialog.jsx: 'Block cookie banners' toggle in Advanced options;
always shown with amber hint when ext not configured; defaults from
global setting
Operator setup: download + unzip the extension from GitHub releases, set
ARCHIVR_COOKIE_EXT=/path/to/unpacked/ext. No Node daemon needed.
* fix: surface cookie_ext_skipped warning toast in CaptureDialog
* feat: package istilldontcareaboutcookies in flake, wire ARCHIVR_COOKIE_EXT
Add isdcac derivation mirroring ublockLite:
- Fetches ISDCAC-chrome-source.zip v1.1.9 from GitHub releases
- Validates manifest.json at extension root before install (guard against
nested-folder zip regressions in future releases)
- Sets ARCHIVR_COOKIE_EXT in both archivr and archivr_server wrappers
Verified: nix build .#archivr-server and .#archivr both succeed;
wrapper scripts export correct store paths; manifest.json present at root.
* fix: gate consent-overlay cleanup on cookie_ext; reset overflow; narrow selectors
- Strip overflow:hidden from body/html only when cookie_ext is active for
the capture — prevents mutating legitimate pages when the feature is off
- Remove .fc-dialog (Google Funding Choices), .qc-cmp2-*, .sp-message-container,
#sp-cc, #usercentrics-root as fallback for CMPs the extension misses
- Removed overbroad [class^="uc-"] and [id^="usercentrics"] selectors
that could match real page content
* fix: remove ad placeholders when uBlock active; kept height causes blank gap
uBlock Origin Lite blocks ad network requests but first-party placeholder
elements (ins.adsbygoogle, #aswift_* iframe hosts) retain their computed
height (e.g. 280px for a top banner), leaving a large blank space at the
top of captured pages.
Gate cleanup on ublock_ext.is_some(): remove ins.adsbygoogle, aswift_*
iframes, and google_ads_* iframes before SingleFile serialises. Also
collapse the parent container if it becomes empty after removal.
* fix: walk up to .top-ad/.google-auto-placed ancestor before removing ad slot
Removing only the inner ins.adsbygoogle left the outer .container.top-ad
wrapper (with pb-4 padding) in the layout, preserving the blank gap.
Now walk up via closest() to the nearest ad-slot container class before
removal so the whole slot including padding collapses.
This commit is contained in:
parent
dae61e585d
commit
2e8820a0da
18 changed files with 4003 additions and 266 deletions
|
|
@ -608,30 +608,86 @@ select {
|
|||
}
|
||||
.capture-actions {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
justify-content: flex-end;
|
||||
margin-top: 16px;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
margin-top: 18px;
|
||||
}
|
||||
.capture-cancel {
|
||||
border: 1px solid var(--line);
|
||||
background: none;
|
||||
color: var(--ink);
|
||||
padding: 8px 18px;
|
||||
border-radius: var(--r);
|
||||
cursor: pointer;
|
||||
}
|
||||
.capture-cancel:hover { background: var(--paper-2); }
|
||||
.capture-submit {
|
||||
border: 0;
|
||||
background: var(--ink);
|
||||
color: var(--paper);
|
||||
padding: 8px 20px;
|
||||
border-radius: var(--r);
|
||||
padding: 13px 20px;
|
||||
border-radius: var(--r2);
|
||||
cursor: pointer;
|
||||
font-weight: 600;
|
||||
font-size: 15px;
|
||||
width: 100%;
|
||||
min-width: 220px;
|
||||
letter-spacing: 0.01em;
|
||||
}
|
||||
.capture-submit:hover { opacity: 0.85; }
|
||||
.capture-submit:disabled { opacity: 0.45; cursor: default; }
|
||||
.capture-cancel {
|
||||
border: 0;
|
||||
background: none;
|
||||
color: var(--muted);
|
||||
padding: 7px 18px;
|
||||
border-radius: var(--r);
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
text-align: center;
|
||||
width: 100%;
|
||||
}
|
||||
.capture-cancel:hover { color: var(--ink); background: var(--paper-2); }
|
||||
|
||||
/* ── Capture advanced options ────────────────────────────────────────────── */
|
||||
.capture-advanced { margin-top: 12px; }
|
||||
.capture-advanced-toggle {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
background: none;
|
||||
border: 0;
|
||||
cursor: pointer;
|
||||
color: var(--muted);
|
||||
font-size: 12.5px;
|
||||
padding: 4px 0;
|
||||
border-radius: var(--r);
|
||||
}
|
||||
.capture-advanced-toggle:hover { color: var(--ink); }
|
||||
.capture-chevron {
|
||||
width: 14px; height: 14px;
|
||||
flex-shrink: 0;
|
||||
transition: transform 0.18s ease;
|
||||
}
|
||||
.capture-chevron--open { transform: rotate(180deg); }
|
||||
.capture-advanced-panel {
|
||||
margin-top: 8px;
|
||||
padding: 10px 12px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--r2);
|
||||
background: var(--paper-2);
|
||||
}
|
||||
.capture-ext-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
cursor: default;
|
||||
}
|
||||
.capture-ext-label { flex: 1; min-width: 0; }
|
||||
.capture-ext-name {
|
||||
display: block;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
color: var(--ink);
|
||||
}
|
||||
.capture-ext-desc {
|
||||
display: block;
|
||||
font-size: 11.5px;
|
||||
color: var(--muted);
|
||||
margin-top: 1px;
|
||||
}
|
||||
|
||||
/* ── Capture dialog: header + multi-row ─────────────────────────────────── */
|
||||
.capture-dialog-header {
|
||||
|
|
@ -812,6 +868,7 @@ select {
|
|||
to { opacity: 1; transform: translateY(0) scale(1); }
|
||||
}
|
||||
.toast--error { border-left: 3px solid var(--accent); }
|
||||
.toast--warning { border-left: 3px solid #e8a000; }
|
||||
|
||||
.toast-top {
|
||||
display: flex;
|
||||
|
|
@ -892,6 +949,84 @@ select {
|
|||
max-height: 200px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.toast-warning-detail {
|
||||
margin: 0;
|
||||
padding: 10px 14px 12px;
|
||||
font-size: 12px;
|
||||
line-height: 1.55;
|
||||
color: var(--muted);
|
||||
background: var(--paper-2);
|
||||
border-top: 1px solid var(--line);
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
.toast-ignore-btn { color: var(--muted); }
|
||||
|
||||
/* ── Extension toggle (pill switch) ─────────────────────────────────────── */
|
||||
.ext-toggle {
|
||||
flex-shrink: 0;
|
||||
position: relative;
|
||||
width: 44px;
|
||||
height: 24px;
|
||||
border-radius: 12px;
|
||||
background: var(--line);
|
||||
border: 0;
|
||||
cursor: pointer;
|
||||
transition: background 0.18s ease;
|
||||
padding: 0;
|
||||
}
|
||||
.ext-toggle--sm { width: 36px; height: 20px; border-radius: 10px; }
|
||||
.ext-toggle--on { background: var(--ink); }
|
||||
.ext-toggle:disabled { opacity: 0.4; cursor: default; }
|
||||
.ext-toggle-knob {
|
||||
position: absolute;
|
||||
top: 3px; left: 3px;
|
||||
width: 18px; height: 18px;
|
||||
border-radius: 50%;
|
||||
background: var(--paper);
|
||||
transition: transform 0.18s ease;
|
||||
display: block;
|
||||
box-shadow: 0 1px 3px rgba(0,0,0,.2);
|
||||
}
|
||||
.ext-toggle--sm .ext-toggle-knob { width: 14px; height: 14px; top: 3px; left: 3px; }
|
||||
.ext-toggle--on .ext-toggle-knob { transform: translateX(20px); }
|
||||
.ext-toggle--sm.ext-toggle--on .ext-toggle-knob { transform: translateX(16px); }
|
||||
|
||||
/* ── Extension card (Settings / Extensions tab) ──────────────────────────── */
|
||||
.ext-card {
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--r2);
|
||||
padding: 16px 18px;
|
||||
background: var(--paper);
|
||||
}
|
||||
.ext-card-header {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
justify-content: space-between;
|
||||
gap: 16px;
|
||||
}
|
||||
.ext-card-info { flex: 1; min-width: 0; }
|
||||
.ext-card-name {
|
||||
display: block;
|
||||
font-weight: 600;
|
||||
font-size: 14px;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.ext-card-desc {
|
||||
display: block;
|
||||
font-size: 13px;
|
||||
color: var(--muted);
|
||||
line-height: 1.5;
|
||||
}
|
||||
.ext-card-hint {
|
||||
display: block;
|
||||
font-size: 12px;
|
||||
color: #e8a000;
|
||||
margin-top: 6px;
|
||||
}
|
||||
.ext-card-hint code { font-size: 11.5px; }
|
||||
|
||||
.form-hint { font-size: 13px; color: var(--muted); line-height: 1.55; margin: 0; }
|
||||
|
||||
/* ── Utility ─────────────────────────────────────────────────────────────── */
|
||||
.muted { color: var(--muted); }
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue