1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-22 03:05:32 +02:00

feat: uBlock Origin Lite + cookie consent extension + reader mode + ad placeholder cleanup (#21)

* feat: uBlock Origin Lite integration for ad-blocking during WebPage captures

- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
  archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
  --load-extension); single-file connects via --browser-server instead of
  launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
  when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
  ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
  and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
  (pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
  archivr and archivr-server wrappers

Env vars:
  ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
  ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)

* feat: Extensions settings tab + capture dialog redesign with Advanced options

Settings/Extensions tab (admin-only):
- New 'Extensions' tab between Cookies and Storage
- ExtensionsTab component: shows uBlock Origin Lite card with pill toggle
- Reads ublock_enabled from instance settings; patch via existing PATCH endpoint
- Shows ublock_ext_available status from server (whether ARCHIVR_UBLOCK_EXT is set)

Instance settings:
- Add ublock_enabled BOOLEAN (default true) to instance_settings auth DB table
- Idempotent ALTER TABLE migration in initialize_auth_schema()
- get/update_instance_settings include ublock_enabled
- GET /api/admin/instance-settings now also returns ublock_ext_available (computed
  from ARCHIVR_UBLOCK_EXT env var at request time)
- PATCH /api/admin/instance-settings accepts ublock_enabled

Per-capture override:
- CaptureBody gains ublock_enabled: Option<bool>
- CaptureConfig gains ublock_enabled: Option<bool>
- singlefile::save() gains ublock_enabled_override: Option<bool> param
- Capture handler resolves: body override > global instance setting > env var
- submitCapture(aid, loc, qual, extensions) in api.js passes ublock_enabled

Capture dialog redesign:
- Archive button: full-width, 13px padding, min-width 220px, primary CTA
- Cancel: full-width but text-style, below Archive
- ‹Advanced options› chevron toggle (rotates on open)
- Expanded panel shows uBlock toggle for this capture session
- Loads global ublock_enabled default from instance settings on mount

Styles:
- .ext-toggle pill switch (44×24 and 36×20 small variant)
- .ext-card for Settings Extensions tab
- .capture-advanced + .capture-advanced-panel + .capture-chevron
- .capture-ext-row / .capture-ext-label / .capture-ext-name / .capture-ext-desc
- .form-hint utility class

* fix: remove ublock_enabled from INSERT OR IGNORE in DDL batch

The INSERT ran before the ALTER TABLE migration added the column,
causing 'table instance_settings has no column named ublock_enabled'
on existing databases. The INSERT OR IGNORE for the default row only
needs the original columns; the migration's DEFAULT 1 handles the
new column for existing and new rows alike.

* feat: Reader mode via Mozilla Readability.js

Adds an opt-in 'Reader mode' advanced option to the capture dialog.
When enabled, Readability.js is injected as a browser script during
SingleFile capture; it fires on single-file-on-before-capture-start,
replaces the page body with the distilled article content, injects a
clean typographic stylesheet, and adds a header with title/byline/site.
Falls back silently if Readability fails (e.g. non-article pages).

- vendor/readability/Readability.js  Apache 2.0, Mozilla, v0.6.0
- singlefile.rs: embed READABILITY_JS + READER_MODE_WRAPPER_JS via
  include_str!; write both to temp dir when reader_mode is true;
  base_single_file_cmd now accepts &[&Path] for multiple --browser-script
- capture.rs: CaptureConfig.reader_mode: bool
- routes.rs: CaptureBody.reader_mode: Option<bool> (defaults false)
- api.js: submitCapture passes reader_mode in payload
- CaptureDialog.jsx: Reader mode toggle in Advanced options (off by default)

* fix: diagnose single-file no-output-file error + prevent stdout dumping

- Add --dump-content=false to every single-file invocation to prevent
  the Docker-detection heuristic from routing HTML to stdout instead of
  the output file (the heuristic can trigger in some macOS environments)
- Improve the no-output-file error message to include: temp dir contents,
  stderr, and first 200 chars of stdout — this gives enough context to
  diagnose any remaining cause without re-running

* fix: switch uBlock loading from --browser-server to --browser-args

The --browser-server (CDP) path caused 'Unexpected server response: 404'
on macOS Chrome because simple-cdp's WebSocket upgrade to the debugger
endpoint failed after Chrome started — likely a version-specific CDP
endpoint shape mismatch.

New approach: single-file always manages Chrome. When ARCHIVR_UBLOCK_EXT
is set, --headless=new, --load-extension, and --disable-extensions-except
are injected via --browser-args. single-file's browser.js prefix-strips
its own conflicting flags before appending ours, so --headless=new
overrides the default --headless (enabling extension support in headless).

Removes allocate_free_port, wait_for_chrome_ready, run_single_file_with_server
(all dead code now). Docblock updated to reflect actual behaviour and notes
the --single-process caveat: uBOL's declarativeNetRequest static rulesets
are expected to work (network-stack level, not service-worker), but this
has not been mechanically verified under --single-process.

Smoke tested on macOS (this machine): capture with --load-extension + all
three browser-scripts (strip, Readability, reader-mode wrapper) produces
output file correctly. Ad-blocking verification deferred to manual test
with a tracker-heavy URL.

* fix: use correct single-file hook event (single-file-on-before-capture-request)

Prior scripts listened on 'single-file-on-before-capture-start' which
does not exist in single-file-core 1.1.49.  The real hook is:

  single-file-on-before-capture-request  (dispatched by initUserScriptHandler
  after receiving single-file-user-script-init; userScriptEnabled defaults
  to true in args.js so it always fires when --browser-script is passed)

Changes:
- strip-scripts: -start -> -request (no preventDefault needed; synchronous)
- READER_MODE_SCRIPT: -start -> -request; add 'installed' meta marker at
  script-evaluation time so artifact inspection can distinguish 'script
  not injected' / 'hook never fired' / 'Readability parse failed'

* fix: correct singlefile.rs docstring (scripts.js concatenates, not isolates)

* fix: dispatch single-file-user-script-init so request hook fires

single-file's initUserScriptHandler (in single-file-bootstrap.js) listens
for 'single-file-user-script-init' and only then installs
_singleFile_waitForUserScript.  Without that dispatch our scripts'
'single-file-on-before-capture-request' listeners were never reached,
so neither strip-scripts nor reader-mode Readability applied.

Dispatch the init event at the top of strip-scripts (always present) and
redundantly in READER_MODE_SCRIPT.  Verified end-to-end: artifact for
run_b3181d6d276e4e56a1a6c356ef9bbe8f has
  meta content="applied", max-width:680px CSS, 0 script tags.

* feat: cookie consent extension support (ARCHIVR_COOKIE_EXT)

Mirrors the uBlock Origin Lite integration exactly:

Backend:
- singlefile.rs: resolve_cookie_ext_config() reads ARCHIVR_COOKIE_CONSENT
  (default true) + ARCHIVR_COOKIE_EXT path; extension paths comma-joined
  into --load-extension / --disable-extensions-except so uBlock and cookie
  ext can coexist; SaveResult.cookie_ext_skipped tracks miss
- database.rs: cookie_ext_enabled column on instance_settings (DEFAULT 1);
  idempotent ALTER TABLE migration; get/update wired through
- capture.rs: CaptureConfig.cookie_ext_enabled: Option<bool>; threaded to
  singlefile::save(); cookie_ext_skipped surfaced in CaptureResult
- routes.rs: CaptureBody + UpdateInstanceSettingsBody get cookie_ext_enabled;
  capture handler resolves effective value (body overrides global); notes_json
  only includes skipped fields that are true; GET instance-settings includes
  cookie_ext_available from env path check

Frontend:
- api.js: submitCapture forwards cookie_ext_enabled
- SettingsView.jsx: 'I Still Don't Care About Cookies' card in Extensions
  tab; always-active toggle (user can disable even when ext not installed);
  amber 'Not configured' hint + ARCHIVR_COOKIE_EXT guidance when unavailable
- CaptureDialog.jsx: 'Block cookie banners' toggle in Advanced options;
  always shown with amber hint when ext not configured; defaults from
  global setting

Operator setup: download + unzip the extension from GitHub releases, set
ARCHIVR_COOKIE_EXT=/path/to/unpacked/ext. No Node daemon needed.

* fix: surface cookie_ext_skipped warning toast in CaptureDialog

* feat: package istilldontcareaboutcookies in flake, wire ARCHIVR_COOKIE_EXT

Add isdcac derivation mirroring ublockLite:
- Fetches ISDCAC-chrome-source.zip v1.1.9 from GitHub releases
- Validates manifest.json at extension root before install (guard against
  nested-folder zip regressions in future releases)
- Sets ARCHIVR_COOKIE_EXT in both archivr and archivr_server wrappers

Verified: nix build .#archivr-server and .#archivr both succeed;
wrapper scripts export correct store paths; manifest.json present at root.

* fix: gate consent-overlay cleanup on cookie_ext; reset overflow; narrow selectors

- Strip overflow:hidden from body/html only when cookie_ext is active for
  the capture — prevents mutating legitimate pages when the feature is off
- Remove .fc-dialog (Google Funding Choices), .qc-cmp2-*, .sp-message-container,
  #sp-cc, #usercentrics-root as fallback for CMPs the extension misses
- Removed overbroad [class^="uc-"] and [id^="usercentrics"] selectors
  that could match real page content

* fix: remove ad placeholders when uBlock active; kept height causes blank gap

uBlock Origin Lite blocks ad network requests but first-party placeholder
elements (ins.adsbygoogle, #aswift_* iframe hosts) retain their computed
height (e.g. 280px for a top banner), leaving a large blank space at the
top of captured pages.

Gate cleanup on ublock_ext.is_some(): remove ins.adsbygoogle, aswift_*
iframes, and google_ads_* iframes before SingleFile serialises. Also
collapse the parent container if it becomes empty after removal.

* fix: walk up to .top-ad/.google-auto-placed ancestor before removing ad slot

Removing only the inner ins.adsbygoogle left the outer .container.top-ad
wrapper (with pb-4 padding) in the layout, preserving the blank gap.
Now walk up via closest() to the nearest ad-slot container class before
removal so the whole slot including padding collapses.
This commit is contained in:
TheGeneralist 2026-07-08 23:26:48 +02:00 committed by GitHub
parent dae61e585d
commit 2e8820a0da
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 4003 additions and 266 deletions

View file

@ -608,30 +608,86 @@ select {
}
.capture-actions {
display: flex;
gap: 10px;
justify-content: flex-end;
margin-top: 16px;
flex-direction: column;
gap: 8px;
margin-top: 18px;
}
.capture-cancel {
border: 1px solid var(--line);
background: none;
color: var(--ink);
padding: 8px 18px;
border-radius: var(--r);
cursor: pointer;
}
.capture-cancel:hover { background: var(--paper-2); }
.capture-submit {
border: 0;
background: var(--ink);
color: var(--paper);
padding: 8px 20px;
border-radius: var(--r);
padding: 13px 20px;
border-radius: var(--r2);
cursor: pointer;
font-weight: 600;
font-size: 15px;
width: 100%;
min-width: 220px;
letter-spacing: 0.01em;
}
.capture-submit:hover { opacity: 0.85; }
.capture-submit:disabled { opacity: 0.45; cursor: default; }
.capture-cancel {
border: 0;
background: none;
color: var(--muted);
padding: 7px 18px;
border-radius: var(--r);
cursor: pointer;
font-size: 13px;
text-align: center;
width: 100%;
}
.capture-cancel:hover { color: var(--ink); background: var(--paper-2); }
/* ── Capture advanced options ────────────────────────────────────────────── */
.capture-advanced { margin-top: 12px; }
.capture-advanced-toggle {
display: flex;
align-items: center;
gap: 5px;
background: none;
border: 0;
cursor: pointer;
color: var(--muted);
font-size: 12.5px;
padding: 4px 0;
border-radius: var(--r);
}
.capture-advanced-toggle:hover { color: var(--ink); }
.capture-chevron {
width: 14px; height: 14px;
flex-shrink: 0;
transition: transform 0.18s ease;
}
.capture-chevron--open { transform: rotate(180deg); }
.capture-advanced-panel {
margin-top: 8px;
padding: 10px 12px;
border: 1px solid var(--line);
border-radius: var(--r2);
background: var(--paper-2);
}
.capture-ext-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
cursor: default;
}
.capture-ext-label { flex: 1; min-width: 0; }
.capture-ext-name {
display: block;
font-size: 13px;
font-weight: 600;
color: var(--ink);
}
.capture-ext-desc {
display: block;
font-size: 11.5px;
color: var(--muted);
margin-top: 1px;
}
/* ── Capture dialog: header + multi-row ─────────────────────────────────── */
.capture-dialog-header {
@ -812,6 +868,7 @@ select {
to { opacity: 1; transform: translateY(0) scale(1); }
}
.toast--error { border-left: 3px solid var(--accent); }
.toast--warning { border-left: 3px solid #e8a000; }
.toast-top {
display: flex;
@ -892,6 +949,84 @@ select {
max-height: 200px;
overflow-y: auto;
}
.toast-warning-detail {
margin: 0;
padding: 10px 14px 12px;
font-size: 12px;
line-height: 1.55;
color: var(--muted);
background: var(--paper-2);
border-top: 1px solid var(--line);
white-space: pre-wrap;
word-break: break-word;
}
.toast-ignore-btn { color: var(--muted); }
/* ── Extension toggle (pill switch) ─────────────────────────────────────── */
.ext-toggle {
flex-shrink: 0;
position: relative;
width: 44px;
height: 24px;
border-radius: 12px;
background: var(--line);
border: 0;
cursor: pointer;
transition: background 0.18s ease;
padding: 0;
}
.ext-toggle--sm { width: 36px; height: 20px; border-radius: 10px; }
.ext-toggle--on { background: var(--ink); }
.ext-toggle:disabled { opacity: 0.4; cursor: default; }
.ext-toggle-knob {
position: absolute;
top: 3px; left: 3px;
width: 18px; height: 18px;
border-radius: 50%;
background: var(--paper);
transition: transform 0.18s ease;
display: block;
box-shadow: 0 1px 3px rgba(0,0,0,.2);
}
.ext-toggle--sm .ext-toggle-knob { width: 14px; height: 14px; top: 3px; left: 3px; }
.ext-toggle--on .ext-toggle-knob { transform: translateX(20px); }
.ext-toggle--sm.ext-toggle--on .ext-toggle-knob { transform: translateX(16px); }
/* ── Extension card (Settings / Extensions tab) ──────────────────────────── */
.ext-card {
border: 1px solid var(--line);
border-radius: var(--r2);
padding: 16px 18px;
background: var(--paper);
}
.ext-card-header {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 16px;
}
.ext-card-info { flex: 1; min-width: 0; }
.ext-card-name {
display: block;
font-weight: 600;
font-size: 14px;
margin-bottom: 4px;
}
.ext-card-desc {
display: block;
font-size: 13px;
color: var(--muted);
line-height: 1.5;
}
.ext-card-hint {
display: block;
font-size: 12px;
color: #e8a000;
margin-top: 6px;
}
.ext-card-hint code { font-size: 11.5px; }
.form-hint { font-size: 13px; color: var(--muted); line-height: 1.55; margin: 0; }
/* ── Utility ─────────────────────────────────────────────────────────────── */
.muted { color: var(--muted); }