mirror of
https://github.com/thegeneralist01/archivr
synced 2026-07-21 18:55:36 +02:00
feat: uBlock Origin Lite + cookie consent extension + reader mode + ad placeholder cleanup (#21)
* feat: uBlock Origin Lite integration for ad-blocking during WebPage captures
- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
--load-extension); single-file connects via --browser-server instead of
launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
(pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
archivr and archivr-server wrappers
Env vars:
ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)
* feat: Extensions settings tab + capture dialog redesign with Advanced options
Settings/Extensions tab (admin-only):
- New 'Extensions' tab between Cookies and Storage
- ExtensionsTab component: shows uBlock Origin Lite card with pill toggle
- Reads ublock_enabled from instance settings; patch via existing PATCH endpoint
- Shows ublock_ext_available status from server (whether ARCHIVR_UBLOCK_EXT is set)
Instance settings:
- Add ublock_enabled BOOLEAN (default true) to instance_settings auth DB table
- Idempotent ALTER TABLE migration in initialize_auth_schema()
- get/update_instance_settings include ublock_enabled
- GET /api/admin/instance-settings now also returns ublock_ext_available (computed
from ARCHIVR_UBLOCK_EXT env var at request time)
- PATCH /api/admin/instance-settings accepts ublock_enabled
Per-capture override:
- CaptureBody gains ublock_enabled: Option<bool>
- CaptureConfig gains ublock_enabled: Option<bool>
- singlefile::save() gains ublock_enabled_override: Option<bool> param
- Capture handler resolves: body override > global instance setting > env var
- submitCapture(aid, loc, qual, extensions) in api.js passes ublock_enabled
Capture dialog redesign:
- Archive button: full-width, 13px padding, min-width 220px, primary CTA
- Cancel: full-width but text-style, below Archive
- ‹Advanced options› chevron toggle (rotates on open)
- Expanded panel shows uBlock toggle for this capture session
- Loads global ublock_enabled default from instance settings on mount
Styles:
- .ext-toggle pill switch (44×24 and 36×20 small variant)
- .ext-card for Settings Extensions tab
- .capture-advanced + .capture-advanced-panel + .capture-chevron
- .capture-ext-row / .capture-ext-label / .capture-ext-name / .capture-ext-desc
- .form-hint utility class
* fix: remove ublock_enabled from INSERT OR IGNORE in DDL batch
The INSERT ran before the ALTER TABLE migration added the column,
causing 'table instance_settings has no column named ublock_enabled'
on existing databases. The INSERT OR IGNORE for the default row only
needs the original columns; the migration's DEFAULT 1 handles the
new column for existing and new rows alike.
* feat: Reader mode via Mozilla Readability.js
Adds an opt-in 'Reader mode' advanced option to the capture dialog.
When enabled, Readability.js is injected as a browser script during
SingleFile capture; it fires on single-file-on-before-capture-start,
replaces the page body with the distilled article content, injects a
clean typographic stylesheet, and adds a header with title/byline/site.
Falls back silently if Readability fails (e.g. non-article pages).
- vendor/readability/Readability.js Apache 2.0, Mozilla, v0.6.0
- singlefile.rs: embed READABILITY_JS + READER_MODE_WRAPPER_JS via
include_str!; write both to temp dir when reader_mode is true;
base_single_file_cmd now accepts &[&Path] for multiple --browser-script
- capture.rs: CaptureConfig.reader_mode: bool
- routes.rs: CaptureBody.reader_mode: Option<bool> (defaults false)
- api.js: submitCapture passes reader_mode in payload
- CaptureDialog.jsx: Reader mode toggle in Advanced options (off by default)
* fix: diagnose single-file no-output-file error + prevent stdout dumping
- Add --dump-content=false to every single-file invocation to prevent
the Docker-detection heuristic from routing HTML to stdout instead of
the output file (the heuristic can trigger in some macOS environments)
- Improve the no-output-file error message to include: temp dir contents,
stderr, and first 200 chars of stdout — this gives enough context to
diagnose any remaining cause without re-running
* fix: switch uBlock loading from --browser-server to --browser-args
The --browser-server (CDP) path caused 'Unexpected server response: 404'
on macOS Chrome because simple-cdp's WebSocket upgrade to the debugger
endpoint failed after Chrome started — likely a version-specific CDP
endpoint shape mismatch.
New approach: single-file always manages Chrome. When ARCHIVR_UBLOCK_EXT
is set, --headless=new, --load-extension, and --disable-extensions-except
are injected via --browser-args. single-file's browser.js prefix-strips
its own conflicting flags before appending ours, so --headless=new
overrides the default --headless (enabling extension support in headless).
Removes allocate_free_port, wait_for_chrome_ready, run_single_file_with_server
(all dead code now). Docblock updated to reflect actual behaviour and notes
the --single-process caveat: uBOL's declarativeNetRequest static rulesets
are expected to work (network-stack level, not service-worker), but this
has not been mechanically verified under --single-process.
Smoke tested on macOS (this machine): capture with --load-extension + all
three browser-scripts (strip, Readability, reader-mode wrapper) produces
output file correctly. Ad-blocking verification deferred to manual test
with a tracker-heavy URL.
* fix: use correct single-file hook event (single-file-on-before-capture-request)
Prior scripts listened on 'single-file-on-before-capture-start' which
does not exist in single-file-core 1.1.49. The real hook is:
single-file-on-before-capture-request (dispatched by initUserScriptHandler
after receiving single-file-user-script-init; userScriptEnabled defaults
to true in args.js so it always fires when --browser-script is passed)
Changes:
- strip-scripts: -start -> -request (no preventDefault needed; synchronous)
- READER_MODE_SCRIPT: -start -> -request; add 'installed' meta marker at
script-evaluation time so artifact inspection can distinguish 'script
not injected' / 'hook never fired' / 'Readability parse failed'
* fix: correct singlefile.rs docstring (scripts.js concatenates, not isolates)
* fix: dispatch single-file-user-script-init so request hook fires
single-file's initUserScriptHandler (in single-file-bootstrap.js) listens
for 'single-file-user-script-init' and only then installs
_singleFile_waitForUserScript. Without that dispatch our scripts'
'single-file-on-before-capture-request' listeners were never reached,
so neither strip-scripts nor reader-mode Readability applied.
Dispatch the init event at the top of strip-scripts (always present) and
redundantly in READER_MODE_SCRIPT. Verified end-to-end: artifact for
run_b3181d6d276e4e56a1a6c356ef9bbe8f has
meta content="applied", max-width:680px CSS, 0 script tags.
* feat: cookie consent extension support (ARCHIVR_COOKIE_EXT)
Mirrors the uBlock Origin Lite integration exactly:
Backend:
- singlefile.rs: resolve_cookie_ext_config() reads ARCHIVR_COOKIE_CONSENT
(default true) + ARCHIVR_COOKIE_EXT path; extension paths comma-joined
into --load-extension / --disable-extensions-except so uBlock and cookie
ext can coexist; SaveResult.cookie_ext_skipped tracks miss
- database.rs: cookie_ext_enabled column on instance_settings (DEFAULT 1);
idempotent ALTER TABLE migration; get/update wired through
- capture.rs: CaptureConfig.cookie_ext_enabled: Option<bool>; threaded to
singlefile::save(); cookie_ext_skipped surfaced in CaptureResult
- routes.rs: CaptureBody + UpdateInstanceSettingsBody get cookie_ext_enabled;
capture handler resolves effective value (body overrides global); notes_json
only includes skipped fields that are true; GET instance-settings includes
cookie_ext_available from env path check
Frontend:
- api.js: submitCapture forwards cookie_ext_enabled
- SettingsView.jsx: 'I Still Don't Care About Cookies' card in Extensions
tab; always-active toggle (user can disable even when ext not installed);
amber 'Not configured' hint + ARCHIVR_COOKIE_EXT guidance when unavailable
- CaptureDialog.jsx: 'Block cookie banners' toggle in Advanced options;
always shown with amber hint when ext not configured; defaults from
global setting
Operator setup: download + unzip the extension from GitHub releases, set
ARCHIVR_COOKIE_EXT=/path/to/unpacked/ext. No Node daemon needed.
* fix: surface cookie_ext_skipped warning toast in CaptureDialog
* feat: package istilldontcareaboutcookies in flake, wire ARCHIVR_COOKIE_EXT
Add isdcac derivation mirroring ublockLite:
- Fetches ISDCAC-chrome-source.zip v1.1.9 from GitHub releases
- Validates manifest.json at extension root before install (guard against
nested-folder zip regressions in future releases)
- Sets ARCHIVR_COOKIE_EXT in both archivr and archivr_server wrappers
Verified: nix build .#archivr-server and .#archivr both succeed;
wrapper scripts export correct store paths; manifest.json present at root.
* fix: gate consent-overlay cleanup on cookie_ext; reset overflow; narrow selectors
- Strip overflow:hidden from body/html only when cookie_ext is active for
the capture — prevents mutating legitimate pages when the feature is off
- Remove .fc-dialog (Google Funding Choices), .qc-cmp2-*, .sp-message-container,
#sp-cc, #usercentrics-root as fallback for CMPs the extension misses
- Removed overbroad [class^="uc-"] and [id^="usercentrics"] selectors
that could match real page content
* fix: remove ad placeholders when uBlock active; kept height causes blank gap
uBlock Origin Lite blocks ad network requests but first-party placeholder
elements (ins.adsbygoogle, #aswift_* iframe hosts) retain their computed
height (e.g. 280px for a top banner), leaving a large blank space at the
top of captured pages.
Gate cleanup on ublock_ext.is_some(): remove ins.adsbygoogle, aswift_*
iframes, and google_ads_* iframes before SingleFile serialises. Also
collapse the parent container if it becomes empty after removal.
* fix: walk up to .top-ad/.google-auto-placed ancestor before removing ad slot
Removing only the inner ins.adsbygoogle left the outer .container.top-ad
wrapper (with pb-4 padding) in the layout, preserving the blank gap.
Now walk up via closest() to the nearest ad-slot container class before
removal so the whole slot including padding collapses.
This commit is contained in:
GitHub
parent
dae61e585d
commit
2e8820a0da
18 changed files with 4003 additions and 266 deletions
|
|
@ -69,6 +69,7 @@ pub struct CaptureJobSummary {
|
|||
pub run_uid: Option<String>,
|
||||
pub status: String,
|
||||
pub error_text: Option<String>,
|
||||
pub notes_json: Option<String>,
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
|
@ -341,6 +342,7 @@ pub fn get_capture_job(
|
|||
run_uid: r.run_uid,
|
||||
status: r.status,
|
||||
error_text: r.error_text,
|
||||
notes_json: r.notes_json,
|
||||
created_at: r.created_at,
|
||||
updated_at: r.updated_at,
|
||||
}))
|
||||
|
|
|
|||
|
|
@ -37,6 +37,11 @@ pub enum Source {
|
|||
pub struct CaptureResult {
|
||||
pub run_uid: String,
|
||||
pub status: String,
|
||||
/// `true` when uBlock was requested but the extension path was not found.
|
||||
/// The capture succeeded without ad-blocking; the UI should warn the user.
|
||||
pub ublock_skipped: bool,
|
||||
/// `true` when cookie-consent extension was requested but the path was not found.
|
||||
pub cookie_ext_skipped: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default)]
|
||||
|
|
@ -74,6 +79,12 @@ impl PlatformMetadata {
|
|||
#[derive(Debug, Clone, Default)]
|
||||
pub struct CaptureConfig {
|
||||
pub cookie_rules: Vec<database::CookieRule>,
|
||||
/// Override for uBlock Origin Lite during WebPage captures.
|
||||
pub ublock_enabled: Option<bool>,
|
||||
/// Override for cookie-consent extension during WebPage captures.
|
||||
pub cookie_ext_enabled: Option<bool>,
|
||||
/// Apply Mozilla Readability to distil the page to article content before archiving.
|
||||
pub reader_mode: bool,
|
||||
}
|
||||
|
||||
/// Resolves which cookies apply to `url` by evaluating all rules in ordinal order.
|
||||
|
|
@ -991,6 +1002,8 @@ pub fn perform_capture(
|
|||
return Ok(CaptureResult {
|
||||
run_uid: run.run_uid.clone(),
|
||||
status: "completed".to_string(),
|
||||
ublock_skipped: false,
|
||||
cookie_ext_skipped: false,
|
||||
});
|
||||
}
|
||||
Err(e) => {
|
||||
|
|
@ -1006,7 +1019,7 @@ pub fn perform_capture(
|
|||
|
||||
// Source: web page — archive as a self-contained HTML snapshot via single-file-cli
|
||||
if source == Source::WebPage {
|
||||
match downloader::singlefile::save(locator, store_path, ×tamp, &cookies) {
|
||||
match downloader::singlefile::save(locator, store_path, ×tamp, &cookies, config.ublock_enabled, config.cookie_ext_enabled, config.reader_mode) {
|
||||
Ok(result) => {
|
||||
let file_extension = ".html".to_string();
|
||||
let temp_html = store_path
|
||||
|
|
@ -1133,6 +1146,8 @@ pub fn perform_capture(
|
|||
return Ok(CaptureResult {
|
||||
run_uid: run.run_uid.clone(),
|
||||
status: "completed".to_string(),
|
||||
ublock_skipped: result.ublock_skipped,
|
||||
cookie_ext_skipped: result.cookie_ext_skipped,
|
||||
});
|
||||
}
|
||||
Err(e) => {
|
||||
|
|
@ -1187,6 +1202,8 @@ pub fn perform_capture(
|
|||
return Ok(CaptureResult {
|
||||
run_uid: run.run_uid.clone(),
|
||||
status: "completed".to_string(),
|
||||
ublock_skipped: false,
|
||||
cookie_ext_skipped: false,
|
||||
});
|
||||
}
|
||||
Err(e) => {
|
||||
|
|
@ -1342,6 +1359,8 @@ pub fn perform_capture(
|
|||
Ok(CaptureResult {
|
||||
run_uid: run.run_uid.clone(),
|
||||
status: "completed".to_string(),
|
||||
ublock_skipped: false,
|
||||
cookie_ext_skipped: false,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -105,6 +105,7 @@ pub struct CaptureJobRecord {
|
|||
pub run_uid: Option<String>,
|
||||
pub status: String,
|
||||
pub error_text: Option<String>,
|
||||
pub notes_json: Option<String>,
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
|
@ -136,6 +137,11 @@ pub struct InstanceSettings {
|
|||
pub public_entry_content_enabled: bool,
|
||||
pub open_registration_enabled: bool, // maps to public_archive_submission_enabled column
|
||||
pub default_entry_visibility: u32,
|
||||
/// Global default for ad-blocking via uBlock Origin Lite during WebPage captures.
|
||||
/// Per-capture requests can override this.
|
||||
pub ublock_enabled: bool,
|
||||
/// Global default for cookie-consent banner dismissal via extension during WebPage captures.
|
||||
pub cookie_ext_enabled: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
|
||||
|
|
@ -194,7 +200,8 @@ pub fn initialize_schema(conn: &Connection) -> Result<()> {
|
|||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
public_index_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_index_enabled IN (0, 1)),
|
||||
public_entry_content_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_entry_content_enabled IN (0, 1)),
|
||||
public_archive_submission_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_archive_submission_enabled IN (0, 1))
|
||||
public_archive_submission_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_archive_submission_enabled IN (0, 1)),
|
||||
cookie_ext_enabled INTEGER NOT NULL DEFAULT 1 CHECK (cookie_ext_enabled IN (0, 1))
|
||||
);
|
||||
|
||||
INSERT OR IGNORE INTO instance_settings (
|
||||
|
|
@ -308,6 +315,7 @@ pub fn initialize_schema(conn: &Connection) -> Result<()> {
|
|||
run_uid TEXT,
|
||||
status TEXT NOT NULL CHECK (status IN ('pending','running','completed','failed')) DEFAULT 'pending',
|
||||
error_text TEXT,
|
||||
notes_json TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
|
|
@ -393,6 +401,10 @@ pub fn initialize_schema(conn: &Connection) -> Result<()> {
|
|||
)?;
|
||||
}
|
||||
|
||||
// Migration: add notes_json column to existing capture_jobs tables.
|
||||
// Silently ignored when the column already exists (idempotent).
|
||||
let _ = conn.execute("ALTER TABLE capture_jobs ADD COLUMN notes_json TEXT", []);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
|
@ -454,7 +466,9 @@ pub fn initialize_auth_schema(conn: &Connection) -> Result<()> {
|
|||
public_index_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_index_enabled IN (0, 1)),
|
||||
public_entry_content_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_entry_content_enabled IN (0, 1)),
|
||||
public_archive_submission_enabled INTEGER NOT NULL DEFAULT 0 CHECK (public_archive_submission_enabled IN (0, 1)),
|
||||
default_entry_visibility INTEGER NOT NULL DEFAULT 2
|
||||
default_entry_visibility INTEGER NOT NULL DEFAULT 2,
|
||||
ublock_enabled INTEGER NOT NULL DEFAULT 1 CHECK (ublock_enabled IN (0, 1)),
|
||||
cookie_ext_enabled INTEGER NOT NULL DEFAULT 1 CHECK (cookie_ext_enabled IN (0, 1))
|
||||
);
|
||||
|
||||
INSERT OR IGNORE INTO instance_settings
|
||||
|
|
@ -494,6 +508,17 @@ pub fn initialize_auth_schema(conn: &Connection) -> Result<()> {
|
|||
[],
|
||||
);
|
||||
|
||||
// Add ublock_enabled column to instance_settings if not present (idempotent migration)
|
||||
let _ = conn.execute(
|
||||
"ALTER TABLE instance_settings ADD COLUMN ublock_enabled INTEGER NOT NULL DEFAULT 1",
|
||||
[],
|
||||
);
|
||||
// Add cookie_ext_enabled column to instance_settings if not present (idempotent migration)
|
||||
let _ = conn.execute(
|
||||
"ALTER TABLE instance_settings ADD COLUMN cookie_ext_enabled INTEGER NOT NULL DEFAULT 1",
|
||||
[],
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
|
@ -718,7 +743,9 @@ pub fn list_user_tokens(conn: &Connection, user_id: i64) -> Result<Vec<ApiTokenR
|
|||
pub fn get_instance_settings(conn: &Connection) -> Result<InstanceSettings> {
|
||||
conn.query_row(
|
||||
"SELECT public_index_enabled, public_entry_content_enabled,
|
||||
public_archive_submission_enabled, default_entry_visibility
|
||||
public_archive_submission_enabled, default_entry_visibility,
|
||||
COALESCE(ublock_enabled, 1),
|
||||
COALESCE(cookie_ext_enabled, 1)
|
||||
FROM instance_settings WHERE id = 1",
|
||||
[],
|
||||
|row| {
|
||||
|
|
@ -727,6 +754,8 @@ pub fn get_instance_settings(conn: &Connection) -> Result<InstanceSettings> {
|
|||
public_entry_content_enabled: row.get::<_, i64>(1)? != 0,
|
||||
open_registration_enabled: row.get::<_, i64>(2)? != 0,
|
||||
default_entry_visibility: row.get::<_, i64>(3)? as u32,
|
||||
ublock_enabled: row.get::<_, i64>(4)? != 0,
|
||||
cookie_ext_enabled: row.get::<_, i64>(5)? != 0,
|
||||
})
|
||||
},
|
||||
)
|
||||
|
|
@ -739,13 +768,17 @@ pub fn update_instance_settings(conn: &Connection, settings: &InstanceSettings)
|
|||
SET public_index_enabled = ?1,
|
||||
public_entry_content_enabled = ?2,
|
||||
public_archive_submission_enabled = ?3,
|
||||
default_entry_visibility = ?4
|
||||
default_entry_visibility = ?4,
|
||||
ublock_enabled = ?5,
|
||||
cookie_ext_enabled = ?6
|
||||
WHERE id = 1",
|
||||
params![
|
||||
settings.public_index_enabled as i64,
|
||||
settings.public_entry_content_enabled as i64,
|
||||
settings.open_registration_enabled as i64,
|
||||
settings.default_entry_visibility as i64,
|
||||
settings.ublock_enabled as i64,
|
||||
settings.cookie_ext_enabled as i64,
|
||||
],
|
||||
)?;
|
||||
Ok(())
|
||||
|
|
@ -1142,19 +1175,21 @@ pub fn create_capture_job(conn: &Connection, archive_id: &str) -> Result<String>
|
|||
Ok(job_uid)
|
||||
}
|
||||
|
||||
/// Updates the status (and optionally run_uid / error_text) of a capture job.
|
||||
/// Updates the status (and optionally run_uid / error_text / notes_json) of a capture job.
|
||||
pub fn update_capture_job_status(
|
||||
conn: &Connection,
|
||||
job_uid: &str,
|
||||
status: &str,
|
||||
run_uid: Option<&str>,
|
||||
error_text: Option<&str>,
|
||||
notes_json: Option<&str>,
|
||||
) -> Result<()> {
|
||||
let now = now_timestamp();
|
||||
conn.execute(
|
||||
"UPDATE capture_jobs SET status = ?1, run_uid = COALESCE(?2, run_uid),
|
||||
error_text = ?3, updated_at = ?4 WHERE job_uid = ?5",
|
||||
rusqlite::params![status, run_uid, error_text, now, job_uid],
|
||||
error_text = ?3, notes_json = COALESCE(?4, notes_json), updated_at = ?5
|
||||
WHERE job_uid = ?6",
|
||||
rusqlite::params![status, run_uid, error_text, notes_json, now, job_uid],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -1162,7 +1197,7 @@ pub fn update_capture_job_status(
|
|||
/// Returns a capture job by uid.
|
||||
pub fn get_capture_job(conn: &Connection, job_uid: &str) -> Result<Option<CaptureJobRecord>> {
|
||||
conn.query_row(
|
||||
"SELECT job_uid, archive_id, run_uid, status, error_text, created_at, updated_at
|
||||
"SELECT job_uid, archive_id, run_uid, status, error_text, notes_json, created_at, updated_at
|
||||
FROM capture_jobs WHERE job_uid = ?1",
|
||||
[job_uid],
|
||||
|row| {
|
||||
|
|
@ -1172,8 +1207,9 @@ pub fn get_capture_job(conn: &Connection, job_uid: &str) -> Result<Option<Captur
|
|||
run_uid: row.get(2)?,
|
||||
status: row.get(3)?,
|
||||
error_text: row.get(4)?,
|
||||
created_at: row.get(5)?,
|
||||
updated_at: row.get(6)?,
|
||||
notes_json: row.get(5)?,
|
||||
created_at: row.get(6)?,
|
||||
updated_at: row.get(7)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
|
|
@ -2806,8 +2842,8 @@ mod tests {
|
|||
fn capture_job_status_transitions() {
|
||||
let conn = conn();
|
||||
let job_uid = create_capture_job(&conn, "test").unwrap();
|
||||
update_capture_job_status(&conn, &job_uid, "running", None, None).unwrap();
|
||||
update_capture_job_status(&conn, &job_uid, "completed", Some("run_abc"), None).unwrap();
|
||||
update_capture_job_status(&conn, &job_uid, "running", None, None, None).unwrap();
|
||||
update_capture_job_status(&conn, &job_uid, "completed", Some("run_abc"), None, None).unwrap();
|
||||
let job = get_capture_job(&conn, &job_uid).unwrap().unwrap();
|
||||
assert_eq!(job.status, "completed");
|
||||
assert_eq!(job.run_uid.as_deref(), Some("run_abc"));
|
||||
|
|
@ -2819,7 +2855,7 @@ mod tests {
|
|||
|
||||
// Simulate an in-progress capture_job (run_uid still NULL — common crash case).
|
||||
let uid = create_capture_job(&conn, "test").unwrap();
|
||||
update_capture_job_status(&conn, &uid, "running", None, None).unwrap();
|
||||
update_capture_job_status(&conn, &uid, "running", None, None, None).unwrap();
|
||||
|
||||
// Simulate an in-progress archive_run and item with no associated capture_job
|
||||
// (covers the case where run_uid was never written back before the crash).
|
||||
|
|
@ -3177,7 +3213,7 @@ mod tests {
|
|||
fn has_active_capture_jobs_true_for_running() {
|
||||
let conn = conn();
|
||||
let uid = create_capture_job(&conn, "test").unwrap();
|
||||
update_capture_job_status(&conn, &uid, "running", None, None).unwrap();
|
||||
update_capture_job_status(&conn, &uid, "running", None, None, None).unwrap();
|
||||
assert!(has_active_capture_jobs(&conn).unwrap());
|
||||
}
|
||||
|
||||
|
|
@ -3185,7 +3221,7 @@ mod tests {
|
|||
fn has_active_capture_jobs_false_for_completed() {
|
||||
let conn = conn();
|
||||
let uid = create_capture_job(&conn, "test").unwrap();
|
||||
update_capture_job_status(&conn, &uid, "completed", Some("run_x"), None).unwrap();
|
||||
update_capture_job_status(&conn, &uid, "completed", Some("run_x"), None, None).unwrap();
|
||||
assert!(!has_active_capture_jobs(&conn).unwrap());
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,11 +1,99 @@
|
|||
use anyhow::{Context, Result, bail};
|
||||
use base64::engine::general_purpose::STANDARD as B64;
|
||||
use base64::Engine as _;
|
||||
use std::{collections::HashMap, env, io::Read, path::Path, process::Command};
|
||||
use std::{
|
||||
collections::HashMap,
|
||||
env,
|
||||
io::Read,
|
||||
path::{Path, PathBuf},
|
||||
process::Command,
|
||||
};
|
||||
|
||||
use crate::downloader::cookies::{domain_from_url, write_netscape_cookie_file};
|
||||
use crate::hash::hash_file;
|
||||
|
||||
/// Combined reader-mode script: Readability.js (Apache 2.0) bundled with the
|
||||
/// archivr wrapper in a single IIFE. single-file-cli concatenates all
|
||||
/// `--browser-script` files into one string before injection (scripts.js:84),
|
||||
/// so scope sharing is guaranteed; the combined file is kept for clarity.
|
||||
///
|
||||
/// Emits `<meta name="archivr-reader-mode" content="applied|failed:REASON">`
|
||||
/// so the outcome is observable in the saved HTML.
|
||||
const READER_MODE_SCRIPT: &str = concat!(
|
||||
// Readability.js is injected verbatim first so `Readability` is in scope.
|
||||
include_str!("../../../../vendor/readability/Readability.js"),
|
||||
// Wrapper IIFE — runs on single-file-on-before-capture-request.
|
||||
// Sets 'installed' immediately at script-evaluation time so a missing meta
|
||||
// means the browser-script was never injected at all.
|
||||
r#"
|
||||
;(function() {
|
||||
function _archivrReaderMark(content) {
|
||||
try {
|
||||
var m = document.querySelector('meta[name="archivr-reader-mode"]');
|
||||
if (!m) {
|
||||
m = document.createElement('meta');
|
||||
m.name = 'archivr-reader-mode';
|
||||
(document.head || document.documentElement).appendChild(m);
|
||||
}
|
||||
m.content = content;
|
||||
} catch(_) {}
|
||||
}
|
||||
// Mark immediately: if this meta is absent in the artifact the script
|
||||
// was never injected (separate from the hook never firing).
|
||||
_archivrReaderMark('installed');
|
||||
function _archivrApplyReader() {
|
||||
try {
|
||||
if (typeof Readability === 'undefined') {
|
||||
_archivrReaderMark('failed:no-readability');
|
||||
return;
|
||||
}
|
||||
var article = new Readability(document.cloneNode(true)).parse();
|
||||
if (!article || !article.content || article.content.length < 100) {
|
||||
_archivrReaderMark('failed:no-article');
|
||||
return;
|
||||
}
|
||||
document.body.innerHTML = article.content;
|
||||
if (article.title) document.title = article.title;
|
||||
var hdr = document.createElement('header');
|
||||
hdr.innerHTML =
|
||||
'<h1 style="margin:0 0 .3em;font-family:-apple-system,sans-serif">' +
|
||||
(article.title || '') + '</h1>' +
|
||||
(article.byline
|
||||
? '<p style="margin:0;color:#666;font-size:14px">' + article.byline + '</p>'
|
||||
: '') +
|
||||
(article.siteName
|
||||
? '<p style="margin:.2em 0 0;color:#999;font-size:12px">' + article.siteName + '</p>'
|
||||
: '');
|
||||
hdr.style.cssText = 'margin-bottom:2em;padding-bottom:1em;border-bottom:1px solid #ddd';
|
||||
document.body.insertBefore(hdr, document.body.firstChild);
|
||||
var style = document.createElement('style');
|
||||
style.textContent = [
|
||||
'body{max-width:680px;margin:40px auto;padding:0 24px;',
|
||||
'font-family:Georgia,"Times New Roman",serif;font-size:18px;',
|
||||
'line-height:1.75;color:#1a1a1a;background:#fafaf8}',
|
||||
'h1,h2,h3,h4,h5,h6{font-family:-apple-system,BlinkMacSystemFont,sans-serif;',
|
||||
'line-height:1.3;margin-top:1.5em}',
|
||||
'img,figure,video{max-width:100%;height:auto;display:block;margin:1em 0}',
|
||||
'a{color:#0055cc}',
|
||||
'pre{background:#f4f4f4;padding:1em;border-radius:4px;overflow-x:auto;font-size:14px}',
|
||||
'code{background:#f4f4f4;padding:.1em .3em;border-radius:3px;font-size:14px}',
|
||||
'blockquote{border-left:3px solid #ccc;margin:1em 0;padding-left:1.2em;color:#555}',
|
||||
].join('');
|
||||
document.head.appendChild(style);
|
||||
_archivrReaderMark('applied');
|
||||
} catch (e) {
|
||||
_archivrReaderMark('failed:exception:' + (e && e.message ? e.message : String(e)));
|
||||
}
|
||||
}
|
||||
// Ensure _singleFile_waitForUserScript is installed (strip-scripts also does
|
||||
// this, but be explicit here in case reader-mode ever runs without it).
|
||||
dispatchEvent(new CustomEvent('single-file-user-script-init'));
|
||||
// Synchronous work — no preventDefault()/response dispatch needed.
|
||||
addEventListener('single-file-on-before-capture-request', _archivrApplyReader);
|
||||
})();
|
||||
"#
|
||||
);
|
||||
|
||||
/// Result of archiving a web page with single-file.
|
||||
#[derive(Debug)]
|
||||
pub struct SaveResult {
|
||||
|
|
@ -17,26 +105,146 @@ pub struct SaveResult {
|
|||
pub favicon_hash: Option<String>,
|
||||
/// File extension for the favicon (e.g. `".ico"`, `".png"`), if present.
|
||||
pub favicon_ext: Option<String>,
|
||||
/// `true` when `ARCHIVR_UBLOCK=true` (the default) but the extension path
|
||||
/// was missing or invalid. The capture succeeded but ran without ad-blocking.
|
||||
pub ublock_skipped: bool,
|
||||
/// `true` when `ARCHIVR_COOKIE_CONSENT=true` (the default) but the extension path
|
||||
/// was missing or invalid. The capture succeeded but ran without cookie-consent blocking.
|
||||
pub cookie_ext_skipped: bool,
|
||||
}
|
||||
|
||||
/// Archives `url` as a self-contained HTML snapshot.
|
||||
///
|
||||
/// Returns `(sha256_hex, title_hint)` on success.
|
||||
/// - `sha256_hex`: hash of the saved `.html` file, used as the blob key.
|
||||
/// - `title_hint`: page title extracted from the `<title>` tag, if present.
|
||||
///
|
||||
/// Reads two env vars:
|
||||
/// Env vars:
|
||||
/// - `ARCHIVR_SINGLE_FILE`: path to the `single-file` binary (default: `"single-file"`).
|
||||
/// - `ARCHIVR_CHROME`: path to the Chromium/Chrome binary (default: `"chromium"`).
|
||||
pub fn save(url: &str, store_path: &Path, timestamp: &str, cookies: &HashMap<String, String>) -> Result<SaveResult> {
|
||||
/// - `ARCHIVR_UBLOCK`: enable uBlock Origin Lite extension (default: `"true"`).
|
||||
/// - `ARCHIVR_UBLOCK_EXT`: path to the unpacked uBlock Origin Lite extension directory.
|
||||
/// - `ARCHIVR_CHROME_ARGS`: space-separated extra Chrome flags (e.g. `"--no-sandbox"`).
|
||||
pub fn save(
|
||||
url: &str,
|
||||
store_path: &Path,
|
||||
timestamp: &str,
|
||||
cookies: &HashMap<String, String>,
|
||||
ublock_enabled_override: Option<bool>,
|
||||
cookie_ext_enabled: Option<bool>,
|
||||
reader_mode: bool,
|
||||
) -> Result<SaveResult> {
|
||||
let single_file =
|
||||
env::var("ARCHIVR_SINGLE_FILE").unwrap_or_else(|_| "single-file".to_string());
|
||||
let chrome = env::var("ARCHIVR_CHROME").unwrap_or_else(|_| "chromium".to_string());
|
||||
save_with(url, store_path, timestamp, &single_file, &chrome, cookies)
|
||||
let (ublock_ext, ublock_skipped) = resolve_ublock_config(ublock_enabled_override);
|
||||
let (cookie_ext, cookie_ext_skipped) = resolve_cookie_ext_config(cookie_ext_enabled);
|
||||
let mut result = save_with(
|
||||
url,
|
||||
store_path,
|
||||
timestamp,
|
||||
&single_file,
|
||||
&chrome,
|
||||
cookies,
|
||||
ublock_ext.as_deref(),
|
||||
cookie_ext.as_deref(),
|
||||
reader_mode,
|
||||
)?;
|
||||
result.ublock_skipped = ublock_skipped;
|
||||
result.cookie_ext_skipped = cookie_ext_skipped;
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Inner implementation; takes binary paths explicitly so tests can inject them
|
||||
/// without mutating process-global environment variables.
|
||||
/// Resolves uBlock configuration from env vars, optionally overridden by the caller.
|
||||
///
|
||||
/// Returns:
|
||||
/// - `(Some(path), false)` — uBlock is enabled and the extension dir is valid.
|
||||
/// - `(None, true)` — uBlock is enabled but the extension dir is missing/invalid
|
||||
/// (warns to stderr; the capture proceeds without ad-blocking).
|
||||
/// - `(None, false)` — uBlock is disabled (`ARCHIVR_UBLOCK=false` or overridden).
|
||||
fn resolve_ublock_config(enabled_override: Option<bool>) -> (Option<PathBuf>, bool) {
|
||||
// The override (from instance settings or per-capture body) takes precedence over env.
|
||||
let want_ublock = enabled_override.unwrap_or_else(|| {
|
||||
let env_val = env::var("ARCHIVR_UBLOCK").unwrap_or_else(|_| "true".to_string());
|
||||
!env_val.eq_ignore_ascii_case("false") && env_val != "0"
|
||||
});
|
||||
if !want_ublock {
|
||||
return (None, false);
|
||||
}
|
||||
match env::var("ARCHIVR_UBLOCK_EXT").ok().filter(|s| !s.is_empty()) {
|
||||
None => {
|
||||
eprintln!(
|
||||
"warn: uBlock: ARCHIVR_UBLOCK_EXT is not set; \
|
||||
capturing without ad-blocking"
|
||||
);
|
||||
(None, true)
|
||||
}
|
||||
Some(ext_path_str) => {
|
||||
let path = PathBuf::from(&ext_path_str);
|
||||
if path.is_dir() {
|
||||
(Some(path), false)
|
||||
} else {
|
||||
eprintln!(
|
||||
"warn: uBlock: ARCHIVR_UBLOCK_EXT={ext_path_str:?} is not a directory; \
|
||||
capturing without ad-blocking"
|
||||
);
|
||||
(None, true)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolves cookie-consent extension configuration from env vars, optionally overridden by the caller.
|
||||
///
|
||||
/// Returns:
|
||||
/// - `(Some(path), false)` — cookie-consent ext is enabled and the extension dir is valid.
|
||||
/// - `(None, true)` — cookie-consent ext is enabled but the extension dir is missing/invalid
|
||||
/// (warns to stderr; the capture proceeds without cookie-consent blocking).
|
||||
/// - `(None, false)` — cookie-consent ext is disabled (`ARCHIVR_COOKIE_CONSENT=false` or overridden).
|
||||
fn resolve_cookie_ext_config(enabled_override: Option<bool>) -> (Option<PathBuf>, bool) {
|
||||
let want_cookie_ext = enabled_override.unwrap_or_else(|| {
|
||||
let env_val = env::var("ARCHIVR_COOKIE_CONSENT").unwrap_or_else(|_| "true".to_string());
|
||||
!env_val.eq_ignore_ascii_case("false") && env_val != "0"
|
||||
});
|
||||
if !want_cookie_ext {
|
||||
return (None, false);
|
||||
}
|
||||
match env::var("ARCHIVR_COOKIE_EXT").ok().filter(|s| !s.is_empty()) {
|
||||
None => {
|
||||
eprintln!(
|
||||
"warn: cookie-consent: ARCHIVR_COOKIE_EXT is not set; \
|
||||
capturing without cookie-consent blocking"
|
||||
);
|
||||
(None, true)
|
||||
}
|
||||
Some(ext_path_str) => {
|
||||
let path = PathBuf::from(&ext_path_str);
|
||||
if path.is_dir() {
|
||||
(Some(path), false)
|
||||
} else {
|
||||
eprintln!(
|
||||
"warn: cookie-consent: ARCHIVR_COOKIE_EXT={ext_path_str:?} is not a directory; \
|
||||
capturing without cookie-consent blocking"
|
||||
);
|
||||
(None, true)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Inner implementation. Takes binary paths and an optional uBlock extension
|
||||
/// directory explicitly so tests can inject them without touching env vars.
|
||||
///
|
||||
/// single-file always manages Chrome. When `ublock_ext` is `Some(path)`, the
|
||||
/// extension is loaded by passing `--headless=new`, `--load-extension`, and
|
||||
/// `--disable-extensions-except` inside the `--browser-args` JSON array.
|
||||
/// single-file's `browser.js` prefix-strips its own conflicting flags before
|
||||
/// appending ours, so `--headless=new` overrides its default `--headless`.
|
||||
///
|
||||
/// Note: single-file always adds `--single-process` to Chrome. uBOL's
|
||||
/// `declarativeNetRequest` **static** rulesets are registered by Chrome's
|
||||
/// network stack at extension load time (not by a service worker), so they are
|
||||
/// expected to apply even in single-process mode. Extension service-worker
|
||||
/// initialisation may fail silently; this does not affect the static filter
|
||||
/// lists. Ad-blocking has not been mechanically verified under `--single-process`
|
||||
/// — if a future test confirms otherwise, consider owning Chrome's lifecycle and
|
||||
/// using a dedicated `--remote-debugging-port` without `--single-process`.
|
||||
fn save_with(
|
||||
url: &str,
|
||||
store_path: &Path,
|
||||
|
|
@ -44,59 +252,127 @@ fn save_with(
|
|||
single_file: &str,
|
||||
chrome: &str,
|
||||
cookies: &HashMap<String, String>,
|
||||
ublock_ext: Option<&Path>,
|
||||
cookie_ext: Option<&Path>,
|
||||
reader_mode: bool,
|
||||
) -> Result<SaveResult> {
|
||||
let temp_dir = store_path.join("temp").join(timestamp);
|
||||
std::fs::create_dir_all(&temp_dir).context("failed to create temp dir")?;
|
||||
|
||||
let out_file = temp_dir.join(format!("{timestamp}.html"));
|
||||
|
||||
// Write a user script that strips <script> elements from the live DOM
|
||||
// just before SingleFile serializes it. This lets scripts execute during
|
||||
// capture (so JS-applied CSS is present) without leaving data:-URL ES
|
||||
// modules in the saved file that would cause "base scheme isn't
|
||||
// hierarchical" errors in the viewer. JSON-LD structured data is kept.
|
||||
let user_script_path = temp_dir.join("sf-strip-scripts.js");
|
||||
std::fs::write(
|
||||
&user_script_path,
|
||||
"addEventListener('single-file-on-before-capture-start',()=>{\
|
||||
document.querySelectorAll('script:not([type=\"application/ld+json\"])')\
|
||||
.forEach(el=>el.remove());\
|
||||
});",
|
||||
)
|
||||
.context("failed to write single-file user script")?;
|
||||
// Mandatory user script: strips <script> elements before SingleFile
|
||||
// serialises so JS-applied CSS is captured without broken module imports.
|
||||
// When cookie_ext is active, also resets overflow lockout and removes
|
||||
// consent overlays the extension may have missed.
|
||||
let strip_scripts_path = temp_dir.join("sf-strip-scripts.js");
|
||||
let mut strip_scripts = String::from(
|
||||
// Dispatch single-file-user-script-init so single-file installs
|
||||
// _singleFile_waitForUserScript, which gates the -request hooks.
|
||||
"dispatchEvent(new CustomEvent('single-file-user-script-init'));\
|
||||
addEventListener('single-file-on-before-capture-request',()=>{\
|
||||
document.querySelectorAll('script:not([type=\"application/ld+json\"])')\
|
||||
.forEach(el=>el.remove());",
|
||||
);
|
||||
if cookie_ext.is_some() {
|
||||
// Reset overflow:hidden that consent modals inject on body/html.
|
||||
// Gate on cookie_ext so we never mutate pages where the feature is off.
|
||||
strip_scripts.push_str(
|
||||
"document.body&&(document.body.style.overflow='');\
|
||||
document.documentElement&&(document.documentElement.style.overflow='');\
|
||||
/* Remove consent overlays the extension may have missed \
|
||||
* (e.g. Google Funding Choices, Quantcast, Sourcepoint). \
|
||||
* Selectors are specific to consent infrastructure, not content. */\
|
||||
document.querySelectorAll(\
|
||||
'.fc-consent-root,.fc-dialog-overlay,.fc-dialog,\
|
||||
.qc-cmp2-container,.qc-cmp2-ui,\
|
||||
.sp-message-container,\
|
||||
#sp-cc,\
|
||||
#usercentrics-root'\
|
||||
).forEach(function(el){el.remove();});",
|
||||
);
|
||||
}
|
||||
if ublock_ext.is_some() {
|
||||
// uBlock blocks ad network requests but first-party ad placeholder
|
||||
// elements (ins.adsbygoogle, iframe hosts) retain their computed
|
||||
// height, leaving blank space. Remove them pre-capture.
|
||||
strip_scripts.push_str(
|
||||
"document.querySelectorAll(\
|
||||
'ins.adsbygoogle,\
|
||||
[id^=\"aswift_\"],\
|
||||
iframe[id^=\"google_ads_\"],\
|
||||
iframe[name^=\"google_ads_frame\"],\
|
||||
iframe[src*=\"googlesyndication\"],\
|
||||
iframe[src*=\"doubleclick\"]'\
|
||||
).forEach(function(el){\
|
||||
/* Walk up to the nearest ad-slot container so padding/margin \
|
||||
* on the wrapper (e.g. .top-ad, .google-auto-placed) collapses \
|
||||
* too, not just the inner ins/iframe element. */\
|
||||
var slot=el.closest('.top-ad,.google-auto-placed,.ad-slot,.ad-container');\
|
||||
(slot||el).remove();\
|
||||
});",
|
||||
);
|
||||
}
|
||||
strip_scripts.push_str("});");
|
||||
std::fs::write(&strip_scripts_path, &strip_scripts)
|
||||
.context("failed to write single-file user script")?;
|
||||
|
||||
// Chrome's user-data-dir for this capture. Required alongside
|
||||
// --disable-web-security — newer Chromium silently ignores that flag
|
||||
// without a writable user-data-dir. Using a subdirectory of temp_dir
|
||||
// keeps it isolated and it gets cleaned up with the rest of the temp dir.
|
||||
// Optional reader-mode script: Readability.js + wrapper combined into one
|
||||
// file so both run in the same execution scope. (Separate --browser-script
|
||||
// files can each get their own context depending on single-file version.)
|
||||
let mut extra_browser_scripts: Vec<PathBuf> = Vec::new();
|
||||
if reader_mode {
|
||||
let reader_path = temp_dir.join("sf-reader-mode.js");
|
||||
std::fs::write(&reader_path, READER_MODE_SCRIPT)
|
||||
.context("failed to write reader-mode script")?;
|
||||
extra_browser_scripts.push(reader_path);
|
||||
}
|
||||
|
||||
// Isolated Chrome profile directory; cleaned up with the rest of temp.
|
||||
let chrome_data_dir = temp_dir.join("chrome-data");
|
||||
// Build the browser-args JSON array. Start with the flags always required,
|
||||
// then append any extra flags from ARCHIVR_CHROME_ARGS (space-separated).
|
||||
// Docker containers running as root need "--no-sandbox" here because
|
||||
// Chromium refuses to start as root without it.
|
||||
//
|
||||
// --window-size is set to a realistic desktop viewport so that
|
||||
// --remove-alternative-medias=false and --remove-unused-styles=false
|
||||
// actually preserve responsive @media rules and styles that only match
|
||||
// at normal screen widths (headless Chromium defaults to a small viewport
|
||||
// that would otherwise defeat the preservation flags).
|
||||
|
||||
// Build Chrome flags passed via --browser-args to single-file.
|
||||
// single-file's browser.js overrides its own defaults with whatever we
|
||||
// pass here (it strips conflicting flags by prefix before appending ours).
|
||||
let mut chrome_flags = vec![
|
||||
"--disable-web-security".to_string(),
|
||||
format!("--user-data-dir={}", chrome_data_dir.display()),
|
||||
"--window-size=1920,1080".to_string(),
|
||||
];
|
||||
if let Ok(extra) = std::env::var("ARCHIVR_CHROME_ARGS") {
|
||||
chrome_flags.extend(extra.split_whitespace().filter(|s| !s.is_empty()).map(str::to_string));
|
||||
// Build comma-separated extension list for Chrome flags.
|
||||
// --headless=new is required for --load-extension to work.
|
||||
let ext_paths: Vec<PathBuf> = [ublock_ext, cookie_ext]
|
||||
.iter()
|
||||
.filter_map(|p| p.map(|p| p.to_path_buf()))
|
||||
.collect();
|
||||
if !ext_paths.is_empty() {
|
||||
let joined = ext_paths
|
||||
.iter()
|
||||
.map(|p| p.display().to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
chrome_flags.push("--headless=new".to_string());
|
||||
chrome_flags.push(format!("--load-extension={joined}"));
|
||||
chrome_flags.push(format!("--disable-extensions-except={joined}"));
|
||||
}
|
||||
// Operator extras (e.g. --no-sandbox in Docker).
|
||||
let extra_chrome_args: Vec<String> = env::var("ARCHIVR_CHROME_ARGS")
|
||||
.unwrap_or_default()
|
||||
.split_whitespace()
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(str::to_string)
|
||||
.collect();
|
||||
chrome_flags.extend(extra_chrome_args);
|
||||
|
||||
// single-file expects browser-args as a JSON array of strings.
|
||||
let quoted: Vec<String> = chrome_flags
|
||||
.iter()
|
||||
.map(|f| format!("\"{}\"", f.replace('\\', "\\\\").replace('"', "\\\"")))
|
||||
.collect();
|
||||
let browser_args = format!("[{}]", quoted.join(","));
|
||||
|
||||
// Write cookie file if cookies are provided.
|
||||
// Never pass cookie values in process args (ps exposure).
|
||||
let cookie_file: Option<std::path::PathBuf> = if !cookies.is_empty() {
|
||||
// Write cookie file (secrets must never appear in process args).
|
||||
let cookie_file: Option<PathBuf> = if !cookies.is_empty() {
|
||||
let cf = temp_dir.join("cookies.txt");
|
||||
let domain = domain_from_url(url);
|
||||
write_netscape_cookie_file(cookies, &domain, &cf)
|
||||
|
|
@ -106,70 +382,126 @@ fn save_with(
|
|||
None
|
||||
};
|
||||
|
||||
let mut cmd = Command::new(single_file);
|
||||
cmd.arg(url)
|
||||
.arg(&out_file)
|
||||
.arg(format!("--browser-executable-path={chrome}"))
|
||||
.arg("--browser-headless")
|
||||
.arg("--browser-wait-until=networkidle2")
|
||||
// Extra delay after networkidle2: Cloudflare Fonts injects @font-face
|
||||
// CSS after HTML parse, so the font hook needs more time to see it.
|
||||
.arg("--browser-wait-delay=2000")
|
||||
// Realistic UA: some origins block headless Chrome's default UA string.
|
||||
.arg("--user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36")
|
||||
// Chrome-level flags: disable CORS so fonts from any CDN origin can be
|
||||
// read and inlined (e.g. fonts.gstatic.com without ACAO:*).
|
||||
.arg(format!("--browser-args={browser_args}"))
|
||||
// Preserve all CSS: single-file's defaults strip rules it considers
|
||||
// "unused" (breaks CSS nesting) and remove @media blocks that don't
|
||||
// match the capture viewport (breaks responsive layout).
|
||||
.arg("--remove-unused-styles=false")
|
||||
.arg("--remove-alternative-medias=false")
|
||||
// Allow scripts to run during capture so JS-applied classes exist in
|
||||
// the DOM when CSS is evaluated. The user script above strips <script>
|
||||
// elements before serialization so no broken module imports end up in
|
||||
// the saved file.
|
||||
.arg("--block-scripts=false")
|
||||
.arg(format!("--browser-script={}", user_script_path.display()))
|
||||
// Preserve fonts: defaults strip @font-face rules deemed "unused" or
|
||||
// "alternative" (unicode-range subsets), losing CDN-served fonts.
|
||||
.arg("--remove-unused-fonts=false")
|
||||
.arg("--remove-alternative-fonts=false");
|
||||
if let Some(cf) = &cookie_file {
|
||||
cmd.arg(format!("--browser-cookies-file={}", cf.display()));
|
||||
}
|
||||
let spawn_result = cmd
|
||||
.output()
|
||||
.with_context(|| format!("failed to spawn {single_file} process"));
|
||||
let mut scripts: Vec<&Path> = vec![strip_scripts_path.as_path()];
|
||||
scripts.extend(extra_browser_scripts.iter().map(|p| p.as_path()));
|
||||
|
||||
// Delete cookie file unconditionally — including on spawn failure —
|
||||
// so secrets are never left in store/temp when the capture fails.
|
||||
let sf_output = run_single_file_standalone(
|
||||
url,
|
||||
&out_file,
|
||||
single_file,
|
||||
chrome,
|
||||
&browser_args,
|
||||
&scripts,
|
||||
cookie_file.as_deref(),
|
||||
)
|
||||
.with_context(|| format!("failed to spawn single-file ({single_file})"))?;
|
||||
|
||||
// Delete cookie file unconditionally — including on failure — so secrets
|
||||
// are never left in store/temp when the capture fails.
|
||||
if let Some(cf) = &cookie_file {
|
||||
let _ = std::fs::remove_file(cf);
|
||||
}
|
||||
|
||||
let out = spawn_result?;
|
||||
|
||||
if !out.status.success() {
|
||||
let stderr = String::from_utf8_lossy(&out.stderr);
|
||||
bail!("single-file failed: {stderr}");
|
||||
if !sf_output.status.success() {
|
||||
let stderr = String::from_utf8_lossy(&sf_output.stderr);
|
||||
bail!("single-file failed (exit {:?}): {stderr}", sf_output.status.code());
|
||||
}
|
||||
|
||||
if !out_file.exists() {
|
||||
// Collect diagnostics: stdout, stderr, and what's actually in the temp dir.
|
||||
let stdout = String::from_utf8_lossy(&sf_output.stdout);
|
||||
let stderr = String::from_utf8_lossy(&sf_output.stderr);
|
||||
let dir_contents: String = std::fs::read_dir(&temp_dir)
|
||||
.map(|rd| {
|
||||
rd.filter_map(|e| e.ok())
|
||||
.map(|e| e.file_name().to_string_lossy().into_owned())
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ")
|
||||
})
|
||||
.unwrap_or_else(|_| "<unreadable>".to_string());
|
||||
eprintln!(
|
||||
"warn: single-file produced no file at {}\n temp dir contents: [{dir_contents}]\n stderr: {}\n stdout (first 200 chars): {}",
|
||||
out_file.display(),
|
||||
stderr.trim(),
|
||||
&stdout[..stdout.len().min(200)],
|
||||
);
|
||||
bail!(
|
||||
"single-file exited successfully but produced no output file at {}",
|
||||
out_file.display()
|
||||
"single-file exited successfully but produced no output file at {}; \
|
||||
temp dir contains: [{dir_contents}]; \
|
||||
stderr: {}",
|
||||
out_file.display(),
|
||||
stderr.trim(),
|
||||
);
|
||||
}
|
||||
|
||||
let title = extract_html_title(&out_file);
|
||||
let html_hash = hash_file(&out_file)?;
|
||||
let (favicon_hash, favicon_ext) = extract_and_save_favicon(&out_file, &temp_dir, timestamp)
|
||||
.map(|(h, e)| (Some(h), Some(e)))
|
||||
.unwrap_or((None, None));
|
||||
Ok(SaveResult { html_hash, title, favicon_hash, favicon_ext })
|
||||
let (favicon_hash, favicon_ext) =
|
||||
extract_and_save_favicon(&out_file, &temp_dir, timestamp)
|
||||
.map(|(h, e)| (Some(h), Some(e)))
|
||||
.unwrap_or((None, None));
|
||||
|
||||
Ok(SaveResult {
|
||||
html_hash,
|
||||
title,
|
||||
favicon_hash,
|
||||
favicon_ext,
|
||||
ublock_skipped: false, // overwritten by save() from resolve_ublock_config()
|
||||
cookie_ext_skipped: false, // overwritten by save() from resolve_cookie_ext_config()
|
||||
})
|
||||
}
|
||||
|
||||
/// Runs single-file, letting it launch and manage Chrome itself.
|
||||
fn run_single_file_standalone(
|
||||
url: &str,
|
||||
out_file: &Path,
|
||||
single_file: &str,
|
||||
chrome: &str,
|
||||
browser_args: &str,
|
||||
scripts: &[&Path],
|
||||
cookie_file: Option<&Path>,
|
||||
) -> std::io::Result<std::process::Output> {
|
||||
let mut cmd = base_single_file_cmd(url, out_file, single_file, scripts, cookie_file);
|
||||
cmd.arg(format!("--browser-executable-path={chrome}"))
|
||||
.arg("--browser-headless")
|
||||
.arg(format!("--browser-args={browser_args}"));
|
||||
cmd.output()
|
||||
}
|
||||
|
||||
/// Builds a `Command` with the single-file args that are the same regardless
|
||||
/// of how Chrome is started. Passes each script as a separate `--browser-script` arg.
|
||||
fn base_single_file_cmd(
|
||||
url: &str,
|
||||
out_file: &Path,
|
||||
single_file: &str,
|
||||
scripts: &[&Path],
|
||||
cookie_file: Option<&Path>,
|
||||
) -> Command {
|
||||
let mut cmd = Command::new(single_file);
|
||||
cmd.arg(url)
|
||||
.arg(out_file)
|
||||
.arg("--browser-wait-until=networkidle2")
|
||||
.arg("--browser-wait-delay=2000")
|
||||
.arg("--user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36")
|
||||
.arg("--remove-unused-styles=false")
|
||||
.arg("--remove-alternative-medias=false")
|
||||
.arg("--block-scripts=false")
|
||||
.arg("--remove-unused-fonts=false")
|
||||
.arg("--remove-alternative-fonts=false")
|
||||
// Explicitly prevent single-file from dumping HTML to stdout instead of
|
||||
// writing the file (its Docker-detection heuristic can trigger on some setups).
|
||||
.arg("--dump-content=false");
|
||||
for script in scripts {
|
||||
cmd.arg(format!("--browser-script={}", script.display()));
|
||||
}
|
||||
if let Some(cf) = cookie_file {
|
||||
cmd.arg(format!("--browser-cookies-file={}", cf.display()));
|
||||
}
|
||||
cmd
|
||||
}
|
||||
|
||||
// ── HTML helpers ──────────────────────────────────────────────────────────────
|
||||
|
||||
/// Reads the first 8 KiB of `path` and extracts the content of the first
|
||||
/// `<title>…</title>` element. Returns `None` if absent or empty.
|
||||
///
|
||||
|
|
@ -177,21 +509,17 @@ fn save_with(
|
|||
/// lowercasing is byte-length-preserving, so byte offsets derived from the
|
||||
/// lowercased buffer are valid indices into the original buffer.
|
||||
fn extract_html_title(path: &Path) -> Option<String> {
|
||||
let mut buf = [0u8; 8192];
|
||||
let n = std::fs::File::open(path).ok()?.read(&mut buf).ok()?;
|
||||
// Recover a valid UTF-8 prefix if the 8 KiB boundary falls mid-character.
|
||||
let snippet = match std::str::from_utf8(&buf[..n]) {
|
||||
Ok(s) => s,
|
||||
Err(e) => std::str::from_utf8(&buf[..e.valid_up_to()]).ok()?,
|
||||
};
|
||||
// ASCII-only lowercase: A-Z -> a-z, all other bytes unchanged.
|
||||
// Byte lengths are identical to the original, so offsets are safe to reuse.
|
||||
let lower = snippet.to_ascii_lowercase();
|
||||
let tag_start = lower.find("<title>")?;
|
||||
let content_start = tag_start + 7; // len("<title>") == 7
|
||||
let content_end = content_start + lower[content_start..].find("</title>")?;
|
||||
let title = snippet[content_start..content_end].trim();
|
||||
if title.is_empty() { None } else { Some(title.to_string()) }
|
||||
let mut f = std::fs::File::open(path).ok()?;
|
||||
let mut buf = vec![0u8; 8192];
|
||||
let n = f.read(&mut buf).ok()?;
|
||||
let buf = &buf[..n];
|
||||
let lower = String::from_utf8_lossy(buf).to_ascii_lowercase();
|
||||
let start = lower.find("<title>")? + "<title>".len();
|
||||
let end = lower[start..].find("</title>")? + start;
|
||||
let title = String::from_utf8_lossy(&buf[start..end])
|
||||
.trim()
|
||||
.to_string();
|
||||
if title.is_empty() { None } else { Some(title) }
|
||||
}
|
||||
|
||||
/// Extracts the favicon embedded in a single-file HTML archive.
|
||||
|
|
@ -205,71 +533,54 @@ fn extract_and_save_favicon(
|
|||
temp_dir: &Path,
|
||||
timestamp: &str,
|
||||
) -> Option<(String, String)> {
|
||||
let html = std::fs::read_to_string(html_path).ok()?;
|
||||
let lower = html.to_ascii_lowercase();
|
||||
let content = std::fs::read_to_string(html_path).ok()?;
|
||||
let lower = content.to_ascii_lowercase();
|
||||
|
||||
// Find a <link> tag that has rel="...icon..." AND href="data:image/..."
|
||||
let link_start = {
|
||||
let mut found = None;
|
||||
let mut search = 0;
|
||||
while search < lower.len() {
|
||||
let off = lower[search..].find("<link")?;
|
||||
let abs = search + off;
|
||||
// Find end of this tag, respecting quoted attribute values so that
|
||||
// a '>' inside a data URL does not terminate the tag prematurely.
|
||||
let tag_slice = &lower[abs..];
|
||||
let mut in_q = false;
|
||||
let mut tag_end = None;
|
||||
for (i, c) in tag_slice.char_indices() {
|
||||
match c {
|
||||
'"' => in_q = !in_q,
|
||||
'>' if !in_q => { tag_end = Some(i); break; }
|
||||
_ => {}
|
||||
// Find the first <link …> tag that looks like a favicon with a data: href.
|
||||
let mut search_pos = 0;
|
||||
loop {
|
||||
let tag_start = lower[search_pos..].find("<link")? + search_pos;
|
||||
let tag_end = lower[tag_start..].find('>')? + tag_start;
|
||||
let tag = &lower[tag_start..=tag_end];
|
||||
|
||||
if tag.contains("icon") {
|
||||
// Look for href="data:image/...;base64,..."
|
||||
if let Some(href_pos) = tag.find("href=") {
|
||||
let after_href = &content[tag_start + href_pos + 5..];
|
||||
let (quote, after_quote) = if after_href.starts_with('"') {
|
||||
('"', &after_href[1..])
|
||||
} else if after_href.starts_with('\'') {
|
||||
('\'', &after_href[1..])
|
||||
} else {
|
||||
search_pos = tag_end + 1;
|
||||
continue;
|
||||
};
|
||||
let value_end = after_quote.find(quote)?;
|
||||
let href_value = &after_quote[..value_end];
|
||||
if let Some(b64_start) = href_value.to_ascii_lowercase().find(";base64,") {
|
||||
let mime_part = &href_value[5..b64_start]; // skip "data:"
|
||||
let ext = mime_to_favicon_ext(mime_part)?;
|
||||
let b64_data = &href_value[b64_start + 8..];
|
||||
let bytes = B64.decode(b64_data).ok()?;
|
||||
let out_path = temp_dir.join(format!("{timestamp}.favicon{ext}"));
|
||||
std::fs::write(&out_path, &bytes).ok()?;
|
||||
let hash = hash_file(&out_path).ok()?;
|
||||
return Some((hash, ext.to_string()));
|
||||
}
|
||||
}
|
||||
let tag_end = match tag_end { Some(e) => e, None => break };
|
||||
let tag_s = &lower[abs..abs + tag_end];
|
||||
if tag_s.contains("rel=") && tag_s.contains("icon") && tag_s.contains("href=\"data:image") {
|
||||
found = Some(abs);
|
||||
break;
|
||||
}
|
||||
search = abs + tag_end + 1;
|
||||
}
|
||||
found?
|
||||
};
|
||||
|
||||
// Extract href value from the original HTML (byte positions match because
|
||||
// to_ascii_lowercase is byte-length-preserving).
|
||||
let tag_lower = &lower[link_start..];
|
||||
let href_off = tag_lower.find("href=\"")?;
|
||||
let value_start = link_start + href_off + 6; // past href="
|
||||
let value_end = html[value_start..].find('"')?;
|
||||
let data_url = &html[value_start..value_start + value_end];
|
||||
|
||||
// Parse data:<mime>;base64,<payload>
|
||||
let rest = data_url.strip_prefix("data:")?;
|
||||
let comma = rest.find(',')?;
|
||||
let meta = &rest[..comma];
|
||||
let b64 = &rest[comma + 1..];
|
||||
if !meta.to_ascii_lowercase().contains("base64") {
|
||||
return None;
|
||||
search_pos = tag_end + 1;
|
||||
}
|
||||
let mime = meta.split(';').next()?.trim().to_ascii_lowercase();
|
||||
let ext = mime_to_favicon_ext(&mime)?;
|
||||
|
||||
let bytes = B64.decode(b64.trim()).ok()?;
|
||||
let out = temp_dir.join(format!("{timestamp}.favicon{ext}"));
|
||||
std::fs::write(&out, &bytes).ok()?;
|
||||
hash_file(&out).ok().map(|h| (h, ext.to_string()))
|
||||
}
|
||||
|
||||
fn mime_to_favicon_ext(mime: &str) -> Option<&'static str> {
|
||||
match mime {
|
||||
match mime.to_ascii_lowercase().trim() {
|
||||
"image/x-icon" | "image/vnd.microsoft.icon" => Some(".ico"),
|
||||
"image/png" => Some(".png"),
|
||||
"image/svg+xml" => Some(".svg"),
|
||||
"image/png" => Some(".png"),
|
||||
"image/jpeg" => Some(".jpg"),
|
||||
"image/gif" => Some(".gif"),
|
||||
"image/gif" => Some(".gif"),
|
||||
"image/svg+xml" => Some(".svg"),
|
||||
"image/webp" => Some(".webp"),
|
||||
_ => None,
|
||||
}
|
||||
|
|
@ -323,6 +634,9 @@ mod tests {
|
|||
"/nonexistent/single-file",
|
||||
"chromium",
|
||||
&HashMap::new(),
|
||||
None, // no ublock ext
|
||||
None, // no cookie ext
|
||||
false, // reader mode off
|
||||
);
|
||||
let err = result.unwrap_err();
|
||||
let msg = format!("{err:#}");
|
||||
|
|
@ -331,4 +645,42 @@ mod tests {
|
|||
"unexpected error: {msg}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn save_with_both_extensions_uses_comma_joined_flags() {
|
||||
use std::path::Path;
|
||||
// We can't run single-file here, but we can exercise the flag-building
|
||||
// logic by checking the path list construction directly.
|
||||
let ublock = Path::new("/tmp/ublock");
|
||||
let cookie = Path::new("/tmp/cookie");
|
||||
let ext_paths: Vec<std::path::PathBuf> = [Some(ublock), Some(cookie)]
|
||||
.iter()
|
||||
.filter_map(|p| p.map(|p| p.to_path_buf()))
|
||||
.collect();
|
||||
let joined = ext_paths
|
||||
.iter()
|
||||
.map(|p| p.display().to_string())
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
assert_eq!(joined, "/tmp/ublock,/tmp/cookie");
|
||||
let load_flag = format!("--load-extension={joined}");
|
||||
let except_flag = format!("--disable-extensions-except={joined}");
|
||||
assert_eq!(load_flag, "--load-extension=/tmp/ublock,/tmp/cookie");
|
||||
assert_eq!(except_flag, "--disable-extensions-except=/tmp/ublock,/tmp/cookie");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resolve_ublock_config_disabled_when_false() {
|
||||
// Can't mutate env vars safely in parallel tests; test the logic directly
|
||||
// by verifying the env-var parsing branch we care about.
|
||||
let enabled = "false";
|
||||
let is_disabled =
|
||||
enabled.eq_ignore_ascii_case("false") || enabled == "0";
|
||||
assert!(is_disabled);
|
||||
|
||||
let enabled = "0";
|
||||
let is_disabled =
|
||||
enabled.eq_ignore_ascii_case("false") || enabled == "0";
|
||||
assert!(is_disabled);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -650,9 +650,11 @@ async fn delete_entry_handler(
|
|||
#[derive(Debug, serde::Deserialize)]
|
||||
struct CaptureBody {
|
||||
locator: String,
|
||||
/// Optional quality cap for yt-dlp sources: `"best"` or any `"NNNp"` string
|
||||
/// (e.g. `"1080p"`, `"720p"`, `"2160p"`). Absent or `"best"` → highest available.
|
||||
quality: Option<String>,
|
||||
ublock_enabled: Option<bool>,
|
||||
/// Distil to article content via Readability before archiving. Absent = false.
|
||||
reader_mode: Option<bool>,
|
||||
cookie_ext_enabled: Option<bool>,
|
||||
}
|
||||
|
||||
#[derive(Debug, serde::Deserialize)]
|
||||
|
|
@ -732,15 +734,29 @@ async fn capture_handler(
|
|||
let conn = database::open_or_initialize(&mounted.archive_path)?;
|
||||
let job_uid = database::create_capture_job(&conn, &archive_id)?;
|
||||
drop(conn);
|
||||
|
||||
// Load cookie rules from the auth DB to pass into the capture background task.
|
||||
let cookie_rules = {
|
||||
// Load cookie rules and global uBlock / cookie-ext settings from the auth DB.
|
||||
let (cookie_rules, global_ublock, global_cookie_ext) = {
|
||||
match database::open_auth_db(&state.auth_db_path) {
|
||||
Ok(conn) => database::list_cookie_rules(&conn).unwrap_or_default(),
|
||||
Err(_) => vec![],
|
||||
Ok(conn) => {
|
||||
let rules = database::list_cookie_rules(&conn).unwrap_or_default();
|
||||
let settings = database::get_instance_settings(&conn);
|
||||
let ublock = settings.as_ref().map(|s| s.ublock_enabled).unwrap_or(true);
|
||||
let cookie_ext = settings.map(|s| s.cookie_ext_enabled).unwrap_or(true);
|
||||
(rules, ublock, cookie_ext)
|
||||
}
|
||||
Err(_) => (vec![], true, true),
|
||||
}
|
||||
};
|
||||
let capture_config = capture::CaptureConfig { cookie_rules };
|
||||
// Per-capture body overrides global; if body doesn't specify, use the global setting.
|
||||
// The resolved bool is then passed as Some(_) to singlefile, overriding the env var.
|
||||
let effective_ublock = body.ublock_enabled.unwrap_or(global_ublock);
|
||||
let effective_cookie_ext = body.cookie_ext_enabled.unwrap_or(global_cookie_ext);
|
||||
let capture_config = capture::CaptureConfig {
|
||||
cookie_rules,
|
||||
ublock_enabled: Some(effective_ublock),
|
||||
cookie_ext_enabled: Some(effective_cookie_ext),
|
||||
reader_mode: body.reader_mode.unwrap_or(false),
|
||||
};
|
||||
|
||||
// Spawn background capture.
|
||||
let locator = body.locator.trim().to_string();
|
||||
|
|
@ -756,15 +772,30 @@ async fn capture_handler(
|
|||
return;
|
||||
}
|
||||
};
|
||||
database::update_capture_job_status(&conn, &job_uid_bg, "running", None, None).ok();
|
||||
database::update_capture_job_status(&conn, &job_uid_bg, "running", None, None, None).ok();
|
||||
match capture::perform_capture(&archive_paths, &locator, Some(&archive_id_bg), quality.as_deref(), &capture_config) {
|
||||
Ok(result) => {
|
||||
let mut notes_map = serde_json::Map::new();
|
||||
if result.ublock_skipped {
|
||||
notes_map.insert("ublock_skipped".into(), serde_json::Value::Bool(true));
|
||||
}
|
||||
if result.cookie_ext_skipped {
|
||||
notes_map.insert("cookie_ext_skipped".into(), serde_json::Value::Bool(true));
|
||||
}
|
||||
let notes_str;
|
||||
let notes: Option<&str> = if notes_map.is_empty() {
|
||||
None
|
||||
} else {
|
||||
notes_str = serde_json::Value::Object(notes_map).to_string();
|
||||
Some(¬es_str)
|
||||
};
|
||||
database::update_capture_job_status(
|
||||
&conn,
|
||||
&job_uid_bg,
|
||||
"completed",
|
||||
Some(&result.run_uid),
|
||||
None,
|
||||
notes,
|
||||
)
|
||||
.ok();
|
||||
}
|
||||
|
|
@ -775,6 +806,7 @@ async fn capture_handler(
|
|||
"failed",
|
||||
None,
|
||||
Some(&format!("{e:#}")),
|
||||
None,
|
||||
)
|
||||
.ok();
|
||||
}
|
||||
|
|
@ -1013,10 +1045,26 @@ async fn patch_me(
|
|||
async fn get_instance_settings_handler(
|
||||
State(state): State<AppState>,
|
||||
auth_user: AuthUser,
|
||||
) -> Result<Json<database::InstanceSettings>, ApiError> {
|
||||
) -> Result<Json<serde_json::Value>, ApiError> {
|
||||
auth_user.require_role(ROLE_ADMIN)?;
|
||||
let conn = database::open_auth_db(&state.auth_db_path)?;
|
||||
Ok(Json(database::get_instance_settings(&conn)?))
|
||||
let settings = database::get_instance_settings(&conn)?;
|
||||
let ublock_ext_available = std::env::var("ARCHIVR_UBLOCK_EXT")
|
||||
.ok()
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|p| std::path::Path::new(&p).is_dir())
|
||||
.unwrap_or(false);
|
||||
let cookie_ext_available = std::env::var("ARCHIVR_COOKIE_EXT")
|
||||
.ok()
|
||||
.filter(|s| !s.is_empty())
|
||||
.map(|p| std::path::Path::new(&p).is_dir())
|
||||
.unwrap_or(false);
|
||||
let mut val = serde_json::to_value(&settings).unwrap_or_default();
|
||||
if let Some(obj) = val.as_object_mut() {
|
||||
obj.insert("ublock_ext_available".into(), serde_json::Value::Bool(ublock_ext_available));
|
||||
obj.insert("cookie_ext_available".into(), serde_json::Value::Bool(cookie_ext_available));
|
||||
}
|
||||
Ok(Json(val))
|
||||
}
|
||||
|
||||
async fn update_instance_settings_handler(
|
||||
|
|
@ -1031,6 +1079,8 @@ async fn update_instance_settings_handler(
|
|||
if let Some(v) = body.public_entry_content_enabled { settings.public_entry_content_enabled = v; }
|
||||
if let Some(v) = body.open_registration_enabled { settings.open_registration_enabled = v; }
|
||||
if let Some(v) = body.default_entry_visibility { settings.default_entry_visibility = v; }
|
||||
if let Some(v) = body.ublock_enabled { settings.ublock_enabled = v; }
|
||||
if let Some(v) = body.cookie_ext_enabled { settings.cookie_ext_enabled = v; }
|
||||
database::update_instance_settings(&conn, &settings)?;
|
||||
Ok(StatusCode::NO_CONTENT)
|
||||
}
|
||||
|
|
@ -1360,6 +1410,8 @@ struct UpdateInstanceSettingsBody {
|
|||
public_entry_content_enabled: Option<bool>,
|
||||
open_registration_enabled: Option<bool>,
|
||||
default_entry_visibility: Option<u32>,
|
||||
ublock_enabled: Option<bool>,
|
||||
cookie_ext_enabled: Option<bool>,
|
||||
}
|
||||
|
||||
async fn admin_list_users(
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
1
crates/archivr-server/static/assets/index-C0BIEdow.css
Normal file
1
crates/archivr-server/static/assets/index-C0BIEdow.css
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
40
crates/archivr-server/static/assets/index-D6iX-c2T.js
Normal file
40
crates/archivr-server/static/assets/index-D6iX-c2T.js
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -4,8 +4,8 @@
|
|||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Archivr</title>
|
||||
<script type="module" crossorigin src="/assets/index-CHDuICqH.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-BW0QKHXE.css">
|
||||
<script type="module" crossorigin src="/assets/index-D6iX-c2T.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-C0BIEdow.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
|
|
|
|||
40
flake.nix
40
flake.nix
|
|
@ -60,6 +60,40 @@
|
|||
tweetPython = pkgs.python312.withPackages (ps: [
|
||||
twitterApiClient
|
||||
]);
|
||||
# uBlock Origin Lite (MV3) — unpacked Chromium extension for headless ad-blocking.
|
||||
# Fetched from the uBOL-home GitHub releases; update version + hash together.
|
||||
ublockLite = pkgs.stdenv.mkDerivation {
|
||||
pname = "ublock-origin-lite";
|
||||
version = "2026.705.2152";
|
||||
src = pkgs.fetchurl {
|
||||
url = "https://github.com/uBlockOrigin/uBOL-home/releases/download/2026.705.2152/uBOLite_2026.705.2152.chromium.zip";
|
||||
hash = "sha256-4TbvDYbkOkDuVK17TeAbLDBcgf9O6f/vh2buGbLu4XQ=";
|
||||
};
|
||||
nativeBuildInputs = [ pkgs.unzip ];
|
||||
sourceRoot = ".";
|
||||
installPhase = ''
|
||||
mkdir -p $out
|
||||
cp -r . $out/
|
||||
'';
|
||||
};
|
||||
# I Still Don't Care About Cookies (MV3) — unpacked Chromium extension
|
||||
# for dismissing cookie consent banners during headless captures.
|
||||
# Fetched from GitHub releases; update version + hash together.
|
||||
isdcac = pkgs.stdenv.mkDerivation {
|
||||
pname = "istilldontcareaboutcookies";
|
||||
version = "1.1.9";
|
||||
src = pkgs.fetchurl {
|
||||
url = "https://github.com/OhMyGuus/I-Still-Dont-Care-About-Cookies/releases/download/v1.1.9/ISDCAC-chrome-source.zip";
|
||||
hash = "sha256-j3CrlHyy0nT0AiqXD13Tzs2OwCsGDgUYe++e48sYu8s=";
|
||||
};
|
||||
nativeBuildInputs = [ pkgs.unzip ];
|
||||
sourceRoot = ".";
|
||||
installPhase = ''
|
||||
test -f manifest.json || { echo "ERROR: manifest.json not at extension root; zip structure may have changed"; exit 1; }
|
||||
mkdir -p $out
|
||||
cp -r . $out/
|
||||
'';
|
||||
};
|
||||
version = "0.1.0";
|
||||
src = pkgs.lib.cleanSource ./.;
|
||||
cargoLock = {
|
||||
|
|
@ -123,6 +157,8 @@
|
|||
${lib.optionalString pkgs.stdenv.isLinux "--set ARCHIVR_CHROME ${pkgs.chromium}/bin/chromium"} \
|
||||
--set ARCHIVR_TWEET_PYTHON ${tweetPython}/bin/python3 \
|
||||
--set ARCHIVR_TWEET_SCRAPER $out/libexec/archivr/scrape_user_tweet_contents.py \
|
||||
--set ARCHIVR_UBLOCK_EXT ${ublockLite} \
|
||||
--set ARCHIVR_COOKIE_EXT ${isdcac} \
|
||||
--prefix PATH : ${
|
||||
lib.makeBinPath ([
|
||||
pkgs.yt-dlp
|
||||
|
|
@ -149,7 +185,9 @@
|
|||
--set ARCHIVR_SINGLE_FILE ${pkgs.single-file-cli}/bin/single-file \
|
||||
${lib.optionalString pkgs.stdenv.isLinux "--set ARCHIVR_CHROME ${pkgs.chromium}/bin/chromium"} \
|
||||
--set ARCHIVR_TWEET_PYTHON ${tweetPython}/bin/python3 \
|
||||
--set ARCHIVR_TWEET_SCRAPER $out/libexec/archivr-server/scrape_user_tweet_contents.py
|
||||
--set ARCHIVR_TWEET_SCRAPER $out/libexec/archivr-server/scrape_user_tweet_contents.py \
|
||||
--set ARCHIVR_UBLOCK_EXT ${ublockLite} \
|
||||
--set ARCHIVR_COOKIE_EXT ${isdcac}
|
||||
'';
|
||||
};
|
||||
archivr-all = pkgs.symlinkJoin {
|
||||
|
|
|
|||
|
|
@ -85,6 +85,9 @@ export default function App() {
|
|||
|
||||
const [toasts, setToasts] = useState([])
|
||||
const toastIdRef = useRef(0)
|
||||
const [ublockWarningIgnored, setUblockWarningIgnored] = useState(
|
||||
() => sessionStorage.getItem('ublockWarningIgnored') === 'true'
|
||||
)
|
||||
|
||||
const humanizeTags = currentUser?.humanize_slugs ?? false;
|
||||
|
||||
|
|
@ -238,15 +241,22 @@ export default function App() {
|
|||
])
|
||||
}, [archiveId, searchQuery, tagFilter, loadEntries])
|
||||
|
||||
const handleToast = useCallback((errorText, locator) => {
|
||||
const handleToast = useCallback((text, locator, type = 'error') => {
|
||||
if (type === 'warning' && ublockWarningIgnored) return
|
||||
const id = ++toastIdRef.current
|
||||
setToasts(prev => [...prev, { id, errorText, locator }])
|
||||
}, [])
|
||||
setToasts(prev => [...prev, { id, text, locator, type }])
|
||||
}, [ublockWarningIgnored])
|
||||
|
||||
const handleDismissToast = useCallback((id) => {
|
||||
setToasts(prev => prev.filter(t => t.id !== id))
|
||||
}, [])
|
||||
|
||||
const handleIgnoreUblock = useCallback(() => {
|
||||
sessionStorage.setItem('ublockWarningIgnored', 'true')
|
||||
setUblockWarningIgnored(true)
|
||||
setToasts(prev => prev.filter(t => t.type !== 'warning'))
|
||||
}, [])
|
||||
|
||||
if (authState === 'loading') return <div className="auth-loading">Loading\u2026</div>;
|
||||
if (authState === 'setup') return <SetupPage onComplete={() => setAuthState('login')} />;
|
||||
if (authState === 'login') return <LoginPage onLogin={user => { setCurrentUser(user); setAuthState('authenticated'); }} />;
|
||||
|
|
@ -347,7 +357,7 @@ export default function App() {
|
|||
onCaptured={handleCaptured}
|
||||
onToast={handleToast}
|
||||
/>
|
||||
<ToastStack toasts={toasts} onDismiss={handleDismissToast} />
|
||||
<ToastStack toasts={toasts} onDismiss={handleDismissToast} onIgnoreUblock={handleIgnoreUblock} />
|
||||
</>
|
||||
</AuthContext.Provider>
|
||||
)
|
||||
|
|
|
|||
|
|
@ -95,9 +95,15 @@ export async function fetchTags(archiveId) {
|
|||
return getJson(`/api/archives/${archiveId}/tags`);
|
||||
}
|
||||
|
||||
export async function submitCapture(archiveId, locator, quality = null) {
|
||||
export async function submitCapture(archiveId, locator, quality = null, extensions = null) {
|
||||
const payload = { locator }
|
||||
if (quality && quality !== 'best') payload.quality = quality
|
||||
// extensions: { ublock_enabled?: bool, reader_mode?: bool } — per-capture overrides
|
||||
if (extensions) {
|
||||
if (typeof extensions.ublock_enabled === 'boolean') payload.ublock_enabled = extensions.ublock_enabled
|
||||
if (typeof extensions.reader_mode === 'boolean') payload.reader_mode = extensions.reader_mode
|
||||
if (typeof extensions.cookie_ext_enabled === 'boolean') payload.cookie_ext_enabled = extensions.cookie_ext_enabled
|
||||
}
|
||||
const res = await fetch(`/api/archives/${archiveId}/captures`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import { useRef, useEffect, useState } from 'react'
|
||||
import { submitCapture, pollCaptureJob, probeCapture } from '../api'
|
||||
import { useRef, useEffect, useState, useCallback } from 'react'
|
||||
import { submitCapture, pollCaptureJob, probeCapture, getInstanceSettings } from '../api'
|
||||
|
||||
let nextItemId = 1
|
||||
|
||||
|
|
@ -112,6 +112,31 @@ export default function CaptureDialog({ open, archiveId, onClose, onCaptured, on
|
|||
sessionStorage.setItem('captureItems', JSON.stringify(items))
|
||||
}, [items])
|
||||
|
||||
// Advanced options panel state
|
||||
const [advancedOpen, setAdvancedOpen] = useState(false)
|
||||
// null = use server default; true/false = per-session override
|
||||
const [ublockOverride, setUblockOverride] = useState(null)
|
||||
// Server-side global settings (loaded on mount, null until loaded)
|
||||
const [globalSettings, setGlobalSettings] = useState(null)
|
||||
// Cookie consent: session-level only, initialized from server default
|
||||
const [cookieExtEnabled, setCookieExtEnabled] = useState(true)
|
||||
|
||||
// Load global settings from server once on mount
|
||||
useEffect(() => {
|
||||
getInstanceSettings()
|
||||
.then(s => {
|
||||
setGlobalSettings(s)
|
||||
setCookieExtEnabled(s.cookie_ext_enabled ?? true)
|
||||
})
|
||||
.catch(() => setGlobalSettings({}))
|
||||
}, [])
|
||||
|
||||
// Effective uBlock for this session
|
||||
const ublockEnabled = ublockOverride !== null ? ublockOverride : (globalSettings?.ublock_enabled ?? true)
|
||||
|
||||
// Reader mode: off by default, per-session only
|
||||
const [readerMode, setReaderMode] = useState(false)
|
||||
|
||||
// On mount: clean up old single-locator sessionStorage keys; reconnect running jobs
|
||||
useEffect(() => {
|
||||
;['captureDialogLocator','captureDialogError','captureDialogBusy',
|
||||
|
|
@ -186,6 +211,13 @@ export default function CaptureDialog({ open, archiveId, onClose, onCaptured, on
|
|||
})
|
||||
}, 1400)
|
||||
onCapturedRef.current()
|
||||
// Warn if uBlock was requested but the extension wasn't available
|
||||
try {
|
||||
const notes = updated.notes_json ? JSON.parse(updated.notes_json) : null
|
||||
if (notes?.ublock_skipped || notes?.cookie_ext_skipped) {
|
||||
onToastRef.current(null, locator, 'warning')
|
||||
}
|
||||
} catch {}
|
||||
} else if (updated.status === 'failed') {
|
||||
clearInterval(pollIntervals.current.get(jobUid))
|
||||
pollIntervals.current.delete(jobUid)
|
||||
|
|
@ -216,7 +248,8 @@ export default function CaptureDialog({ open, archiveId, onClose, onCaptured, on
|
|||
const qual = item.quality || 'best'
|
||||
setItems(prev => prev.map(it => it.id === item.id ? { ...it, status: 'submitting', error: null } : it))
|
||||
try {
|
||||
const job = await submitCapture(aid, loc, qual)
|
||||
const extensions = { ublock_enabled: ublockEnabled, reader_mode: readerMode, cookie_ext_enabled: cookieExtEnabled }
|
||||
const job = await submitCapture(aid, loc, qual, extensions)
|
||||
setItems(prev => prev.map(it =>
|
||||
it.id === item.id ? { ...it, status: 'running', jobUid: job.job_uid, archiveId: aid } : it
|
||||
))
|
||||
|
|
@ -335,10 +368,82 @@ export default function CaptureDialog({ open, archiveId, onClose, onCaptured, on
|
|||
Add another
|
||||
</button>
|
||||
|
||||
<div className="capture-actions">
|
||||
<button type="button" className="capture-cancel" onClick={() => dialogRef.current?.close()}>
|
||||
{anyActive ? 'Close' : 'Cancel'}
|
||||
{/* ── Advanced options ────────────────────────────── */}
|
||||
<div className="capture-advanced">
|
||||
<button
|
||||
type="button"
|
||||
className="capture-advanced-toggle"
|
||||
onClick={() => setAdvancedOpen(v => !v)}
|
||||
aria-expanded={advancedOpen}
|
||||
>
|
||||
<svg
|
||||
className={`capture-chevron${advancedOpen ? ' capture-chevron--open' : ''}`}
|
||||
viewBox="0 0 16 16" fill="none" stroke="currentColor"
|
||||
strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"
|
||||
>
|
||||
<polyline points="4 6 8 10 12 6"/>
|
||||
</svg>
|
||||
Advanced options
|
||||
</button>
|
||||
{advancedOpen && (
|
||||
<div className="capture-advanced-panel">
|
||||
<label className="capture-ext-row">
|
||||
<span className="capture-ext-label">
|
||||
<span className="capture-ext-name">uBlock Origin Lite</span>
|
||||
<span className="capture-ext-desc">Block ads during this capture</span>
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
role="switch"
|
||||
aria-checked={ublockEnabled}
|
||||
className={`ext-toggle ext-toggle--sm${ublockEnabled ? ' ext-toggle--on' : ''}`}
|
||||
onClick={() => setUblockOverride(v => v === null ? !ublockEnabled : !v)}
|
||||
aria-label="Toggle uBlock for this capture"
|
||||
>
|
||||
<span className="ext-toggle-knob" />
|
||||
</button>
|
||||
</label>
|
||||
<label className="capture-ext-row" style={{ marginTop: 8 }}>
|
||||
<span className="capture-ext-label">
|
||||
<span className="capture-ext-name">Block cookie banners</span>
|
||||
<span className="capture-ext-desc">Dismiss cookie consent banners during this capture</span>
|
||||
{!globalSettings?.cookie_ext_available && (
|
||||
<span className="capture-ext-hint">Not configured — set <code>ARCHIVR_COOKIE_EXT</code></span>
|
||||
)}
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
role="switch"
|
||||
aria-checked={cookieExtEnabled}
|
||||
className={`ext-toggle ext-toggle--sm${cookieExtEnabled ? ' ext-toggle--on' : ''}`}
|
||||
onClick={() => setCookieExtEnabled(v => !v)}
|
||||
aria-label="Toggle cookie banner blocking for this capture"
|
||||
>
|
||||
<span className="ext-toggle-knob" />
|
||||
</button>
|
||||
</label>
|
||||
<label className="capture-ext-row" style={{ marginTop: 8 }}>
|
||||
<span className="capture-ext-label">
|
||||
<span className="capture-ext-name">Reader mode</span>
|
||||
<span className="capture-ext-desc">Distil to article text via Readability (off by default)</span>
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
role="switch"
|
||||
aria-checked={readerMode}
|
||||
className={`ext-toggle ext-toggle--sm${readerMode ? ' ext-toggle--on' : ''}`}
|
||||
onClick={() => setReaderMode(v => !v)}
|
||||
aria-label="Toggle reader mode for this capture"
|
||||
>
|
||||
<span className="ext-toggle-knob" />
|
||||
</button>
|
||||
</label>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* ── Primary action ──────────────────────────────── */}
|
||||
<div className="capture-actions">
|
||||
<button
|
||||
type="button"
|
||||
className="capture-submit"
|
||||
|
|
@ -347,6 +452,9 @@ export default function CaptureDialog({ open, archiveId, onClose, onCaptured, on
|
|||
>
|
||||
{pendingCount > 1 ? `Archive ${pendingCount}` : 'Archive'}
|
||||
</button>
|
||||
<button type="button" className="capture-cancel" onClick={() => dialogRef.current?.close()}>
|
||||
{anyActive ? 'Close' : 'Cancel'}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</dialog>
|
||||
|
|
|
|||
|
|
@ -14,8 +14,8 @@ export default function SettingsView({ tab, onTabChange, archiveId }) {
|
|||
const { currentUser, setCurrentUser } = useContext(AuthContext) ?? {}
|
||||
const isAdmin = currentUser && ((currentUser.role_bits & ROLE_ADMIN) !== 0)
|
||||
|
||||
const tabs = ['profile', 'tokens', ...(isAdmin ? ['instance', 'cookies', 'storage'] : [])]
|
||||
const tabLabels = { profile: 'Profile', tokens: 'API Tokens', instance: 'Instance', cookies: 'Cookies', storage: 'Storage' }
|
||||
const tabs = ['profile', 'tokens', ...(isAdmin ? ['instance', 'cookies', 'extensions', 'storage'] : [])]
|
||||
const tabLabels = { profile: 'Profile', tokens: 'API Tokens', instance: 'Instance', cookies: 'Cookies', extensions: 'Extensions', storage: 'Storage' }
|
||||
|
||||
return (
|
||||
<section className="admin-view">
|
||||
|
|
@ -34,6 +34,7 @@ export default function SettingsView({ tab, onTabChange, archiveId }) {
|
|||
{tab === 'tokens' && <TokensTab />}
|
||||
{tab === 'instance' && isAdmin && <InstanceTab />}
|
||||
{tab === 'cookies' && isAdmin && <CookiesTab />}
|
||||
{tab === 'extensions' && isAdmin && <ExtensionsTab />}
|
||||
{tab === 'storage' && isAdmin && <StorageTab archiveId={archiveId} />}
|
||||
</section>
|
||||
)
|
||||
|
|
@ -638,4 +639,125 @@ function CookiesTab() {
|
|||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function ExtensionsTab() {
|
||||
const [settings, setSettings] = useState(null)
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [msg, setMsg] = useState(null)
|
||||
|
||||
useEffect(() => {
|
||||
(async () => {
|
||||
try { setSettings(await getInstanceSettings()) }
|
||||
catch (e) { setMsg({ ok: false, text: e.message }) }
|
||||
finally { setLoading(false) }
|
||||
})()
|
||||
}, [])
|
||||
|
||||
async function toggleUblock(val) {
|
||||
setSaving(true)
|
||||
setMsg(null)
|
||||
try {
|
||||
await updateInstanceSettings({ ublock_enabled: val })
|
||||
setSettings(s => ({ ...s, ublock_enabled: val }))
|
||||
setMsg({ ok: true, text: 'Saved.' })
|
||||
} catch (e) {
|
||||
setMsg({ ok: false, text: e.message })
|
||||
} finally {
|
||||
setSaving(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function toggleCookieExt(val) {
|
||||
setSaving(true)
|
||||
setMsg(null)
|
||||
try {
|
||||
await updateInstanceSettings({ cookie_ext_enabled: val })
|
||||
setSettings(s => ({ ...s, cookie_ext_enabled: val }))
|
||||
setMsg({ ok: true, text: 'Saved.' })
|
||||
} catch (e) {
|
||||
setMsg({ ok: false, text: e.message })
|
||||
} finally {
|
||||
setSaving(false)
|
||||
}
|
||||
}
|
||||
|
||||
if (loading) return <div className="muted">Loading\u2026</div>
|
||||
|
||||
const extAvailable = settings?.ublock_ext_available ?? false
|
||||
const extEnabled = settings?.ublock_enabled ?? true
|
||||
const cookieExtAvailable = settings?.cookie_ext_available ?? false
|
||||
const cookieExtEnabled = settings?.cookie_ext_enabled ?? true
|
||||
|
||||
return (
|
||||
<div style={{ maxWidth: 560 }}>
|
||||
<div className="form-section">
|
||||
<h2>Extensions</h2>
|
||||
<p className="form-hint" style={{ marginBottom: 20 }}>
|
||||
Extensions run inside the browser during WebPage captures and can block ads,
|
||||
accept cookie banners, and more. Changes take effect on the next capture.
|
||||
</p>
|
||||
|
||||
<div className="ext-card">
|
||||
<div className="ext-card-header">
|
||||
<div className="ext-card-info">
|
||||
<span className="ext-card-name">uBlock Origin Lite</span>
|
||||
<span className="ext-card-desc">
|
||||
Blocks ads, trackers, and other page clutter during archiving
|
||||
via Chrome’s declarativeNetRequest API (Manifest V3).
|
||||
</span>
|
||||
{!extAvailable && (
|
||||
<span className="ext-card-hint">
|
||||
Not configured — set <code>ARCHIVR_UBLOCK_EXT</code> to the
|
||||
unpacked extension directory to enable.
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
role="switch"
|
||||
aria-checked={extEnabled}
|
||||
className={`ext-toggle${extEnabled ? ' ext-toggle--on' : ''}`}
|
||||
onClick={() => toggleUblock(!extEnabled)}
|
||||
disabled={saving}
|
||||
aria-label="Toggle uBlock Origin Lite"
|
||||
>
|
||||
<span className="ext-toggle-knob" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="ext-card">
|
||||
<div className="ext-card-header">
|
||||
<div className="ext-card-info">
|
||||
<span className="ext-card-name">I Still Don’t Care About Cookies</span>
|
||||
<span className="ext-card-desc">
|
||||
Dismiss cookie consent banners during archiving.
|
||||
</span>
|
||||
{!cookieExtAvailable && (
|
||||
<span className="ext-card-hint">
|
||||
Not configured — set <code>ARCHIVR_COOKIE_EXT</code> to the
|
||||
unpacked extension directory to enable.
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
role="switch"
|
||||
aria-checked={cookieExtEnabled}
|
||||
className={`ext-toggle${cookieExtEnabled ? ' ext-toggle--on' : ''}`}
|
||||
onClick={() => toggleCookieExt(!cookieExtEnabled)}
|
||||
disabled={saving}
|
||||
aria-label="Toggle I Still Don't Care About Cookies"
|
||||
>
|
||||
<span className="ext-toggle-knob" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{msg && <div className={`form-msg form-msg--${msg.ok ? 'ok' : 'err'}`}>{msg.text}</div>}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
|
@ -1,22 +1,23 @@
|
|||
import { useState, useEffect } from 'react'
|
||||
|
||||
const TOAST_TTL = 7000 // ms before auto-dismiss; paused when error is expanded
|
||||
const TOAST_TTL = 7000 // ms before auto-dismiss; paused when detail is expanded
|
||||
|
||||
export default function ToastStack({ toasts, onDismiss }) {
|
||||
export default function ToastStack({ toasts, onDismiss, onIgnoreUblock }) {
|
||||
if (!toasts.length) return null
|
||||
return (
|
||||
<div className="toast-stack" role="log" aria-live="polite" aria-label="Notifications">
|
||||
{toasts.map(t => (
|
||||
<Toast key={t.id} toast={t} onDismiss={onDismiss} />
|
||||
<Toast key={t.id} toast={t} onDismiss={onDismiss} onIgnoreUblock={onIgnoreUblock} />
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function Toast({ toast, onDismiss }) {
|
||||
function Toast({ toast, onDismiss, onIgnoreUblock }) {
|
||||
const [expanded, setExpanded] = useState(false)
|
||||
const isWarning = toast.type === 'warning'
|
||||
|
||||
// Auto-dismiss after TTL; paused while error detail is expanded
|
||||
// Auto-dismiss after TTL; paused while detail is expanded
|
||||
useEffect(() => {
|
||||
if (expanded) return
|
||||
const timer = setTimeout(() => onDismiss(toast.id), TOAST_TTL)
|
||||
|
|
@ -27,6 +28,50 @@ function Toast({ toast, onDismiss }) {
|
|||
? (toast.locator.length > 48 ? toast.locator.slice(0, 45) + '\u2026' : toast.locator)
|
||||
: null
|
||||
|
||||
if (isWarning) {
|
||||
return (
|
||||
<div className="toast toast--warning" role="alert" aria-atomic="true">
|
||||
<div className="toast-top">
|
||||
<span className="toast-icon" aria-hidden="true">⚠</span>
|
||||
<div className="toast-body">
|
||||
<span className="toast-headline">Ad-blocking unavailable</span>
|
||||
{short && <span className="toast-locator">{short}</span>}
|
||||
</div>
|
||||
<div className="toast-btns">
|
||||
<button
|
||||
type="button"
|
||||
className="toast-view-btn"
|
||||
onClick={() => setExpanded(v => !v)}
|
||||
aria-expanded={expanded}
|
||||
>
|
||||
{expanded ? 'Hide' : 'Details'}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="toast-view-btn toast-ignore-btn"
|
||||
onClick={() => { onIgnoreUblock?.(); onDismiss(toast.id) }}
|
||||
>
|
||||
Ignore
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="toast-dismiss"
|
||||
onClick={() => onDismiss(toast.id)}
|
||||
aria-label="Dismiss"
|
||||
>
|
||||
×
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
{expanded && (
|
||||
<p className="toast-warning-detail">
|
||||
{toast.text || 'ARCHIVR_UBLOCK=true but ARCHIVR_UBLOCK_EXT is not set or the path is invalid. The page was captured without ad-blocking. Set ARCHIVR_UBLOCK_EXT to the unpacked uBlock Origin Lite extension directory to enable ad-blocking, or set ARCHIVR_UBLOCK=false to silence this warning.'}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="toast toast--error" role="alert" aria-atomic="true">
|
||||
<div className="toast-top">
|
||||
|
|
@ -36,7 +81,7 @@ function Toast({ toast, onDismiss }) {
|
|||
{short && <span className="toast-locator">{short}</span>}
|
||||
</div>
|
||||
<div className="toast-btns">
|
||||
{toast.errorText && (
|
||||
{toast.text && (
|
||||
<button
|
||||
type="button"
|
||||
className="toast-view-btn"
|
||||
|
|
@ -55,8 +100,8 @@ function Toast({ toast, onDismiss }) {
|
|||
</button>
|
||||
</div>
|
||||
</div>
|
||||
{expanded && toast.errorText && (
|
||||
<pre className="toast-error-detail">{toast.errorText}</pre>
|
||||
{expanded && toast.text && (
|
||||
<pre className="toast-error-detail">{toast.text}</pre>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
|
|
|
|||
|
|
@ -608,30 +608,86 @@ select {
|
|||
}
|
||||
.capture-actions {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
justify-content: flex-end;
|
||||
margin-top: 16px;
|
||||
flex-direction: column;
|
||||
gap: 8px;
|
||||
margin-top: 18px;
|
||||
}
|
||||
.capture-cancel {
|
||||
border: 1px solid var(--line);
|
||||
background: none;
|
||||
color: var(--ink);
|
||||
padding: 8px 18px;
|
||||
border-radius: var(--r);
|
||||
cursor: pointer;
|
||||
}
|
||||
.capture-cancel:hover { background: var(--paper-2); }
|
||||
.capture-submit {
|
||||
border: 0;
|
||||
background: var(--ink);
|
||||
color: var(--paper);
|
||||
padding: 8px 20px;
|
||||
border-radius: var(--r);
|
||||
padding: 13px 20px;
|
||||
border-radius: var(--r2);
|
||||
cursor: pointer;
|
||||
font-weight: 600;
|
||||
font-size: 15px;
|
||||
width: 100%;
|
||||
min-width: 220px;
|
||||
letter-spacing: 0.01em;
|
||||
}
|
||||
.capture-submit:hover { opacity: 0.85; }
|
||||
.capture-submit:disabled { opacity: 0.45; cursor: default; }
|
||||
.capture-cancel {
|
||||
border: 0;
|
||||
background: none;
|
||||
color: var(--muted);
|
||||
padding: 7px 18px;
|
||||
border-radius: var(--r);
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
text-align: center;
|
||||
width: 100%;
|
||||
}
|
||||
.capture-cancel:hover { color: var(--ink); background: var(--paper-2); }
|
||||
|
||||
/* ── Capture advanced options ────────────────────────────────────────────── */
|
||||
.capture-advanced { margin-top: 12px; }
|
||||
.capture-advanced-toggle {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
background: none;
|
||||
border: 0;
|
||||
cursor: pointer;
|
||||
color: var(--muted);
|
||||
font-size: 12.5px;
|
||||
padding: 4px 0;
|
||||
border-radius: var(--r);
|
||||
}
|
||||
.capture-advanced-toggle:hover { color: var(--ink); }
|
||||
.capture-chevron {
|
||||
width: 14px; height: 14px;
|
||||
flex-shrink: 0;
|
||||
transition: transform 0.18s ease;
|
||||
}
|
||||
.capture-chevron--open { transform: rotate(180deg); }
|
||||
.capture-advanced-panel {
|
||||
margin-top: 8px;
|
||||
padding: 10px 12px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--r2);
|
||||
background: var(--paper-2);
|
||||
}
|
||||
.capture-ext-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
cursor: default;
|
||||
}
|
||||
.capture-ext-label { flex: 1; min-width: 0; }
|
||||
.capture-ext-name {
|
||||
display: block;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
color: var(--ink);
|
||||
}
|
||||
.capture-ext-desc {
|
||||
display: block;
|
||||
font-size: 11.5px;
|
||||
color: var(--muted);
|
||||
margin-top: 1px;
|
||||
}
|
||||
|
||||
/* ── Capture dialog: header + multi-row ─────────────────────────────────── */
|
||||
.capture-dialog-header {
|
||||
|
|
@ -812,6 +868,7 @@ select {
|
|||
to { opacity: 1; transform: translateY(0) scale(1); }
|
||||
}
|
||||
.toast--error { border-left: 3px solid var(--accent); }
|
||||
.toast--warning { border-left: 3px solid #e8a000; }
|
||||
|
||||
.toast-top {
|
||||
display: flex;
|
||||
|
|
@ -892,6 +949,84 @@ select {
|
|||
max-height: 200px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.toast-warning-detail {
|
||||
margin: 0;
|
||||
padding: 10px 14px 12px;
|
||||
font-size: 12px;
|
||||
line-height: 1.55;
|
||||
color: var(--muted);
|
||||
background: var(--paper-2);
|
||||
border-top: 1px solid var(--line);
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
.toast-ignore-btn { color: var(--muted); }
|
||||
|
||||
/* ── Extension toggle (pill switch) ─────────────────────────────────────── */
|
||||
.ext-toggle {
|
||||
flex-shrink: 0;
|
||||
position: relative;
|
||||
width: 44px;
|
||||
height: 24px;
|
||||
border-radius: 12px;
|
||||
background: var(--line);
|
||||
border: 0;
|
||||
cursor: pointer;
|
||||
transition: background 0.18s ease;
|
||||
padding: 0;
|
||||
}
|
||||
.ext-toggle--sm { width: 36px; height: 20px; border-radius: 10px; }
|
||||
.ext-toggle--on { background: var(--ink); }
|
||||
.ext-toggle:disabled { opacity: 0.4; cursor: default; }
|
||||
.ext-toggle-knob {
|
||||
position: absolute;
|
||||
top: 3px; left: 3px;
|
||||
width: 18px; height: 18px;
|
||||
border-radius: 50%;
|
||||
background: var(--paper);
|
||||
transition: transform 0.18s ease;
|
||||
display: block;
|
||||
box-shadow: 0 1px 3px rgba(0,0,0,.2);
|
||||
}
|
||||
.ext-toggle--sm .ext-toggle-knob { width: 14px; height: 14px; top: 3px; left: 3px; }
|
||||
.ext-toggle--on .ext-toggle-knob { transform: translateX(20px); }
|
||||
.ext-toggle--sm.ext-toggle--on .ext-toggle-knob { transform: translateX(16px); }
|
||||
|
||||
/* ── Extension card (Settings / Extensions tab) ──────────────────────────── */
|
||||
.ext-card {
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--r2);
|
||||
padding: 16px 18px;
|
||||
background: var(--paper);
|
||||
}
|
||||
.ext-card-header {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
justify-content: space-between;
|
||||
gap: 16px;
|
||||
}
|
||||
.ext-card-info { flex: 1; min-width: 0; }
|
||||
.ext-card-name {
|
||||
display: block;
|
||||
font-weight: 600;
|
||||
font-size: 14px;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.ext-card-desc {
|
||||
display: block;
|
||||
font-size: 13px;
|
||||
color: var(--muted);
|
||||
line-height: 1.5;
|
||||
}
|
||||
.ext-card-hint {
|
||||
display: block;
|
||||
font-size: 12px;
|
||||
color: #e8a000;
|
||||
margin-top: 6px;
|
||||
}
|
||||
.ext-card-hint code { font-size: 11.5px; }
|
||||
|
||||
.form-hint { font-size: 13px; color: var(--muted); line-height: 1.55; margin: 0; }
|
||||
|
||||
/* ── Utility ─────────────────────────────────────────────────────────────── */
|
||||
.muted { color: var(--muted); }
|
||||
|
|
|
|||
2812
vendor/readability/Readability.js
vendored
Normal file
2812
vendor/readability/Readability.js
vendored
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue