1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-22 03:05:32 +02:00

feat: add user-configurable cookie rules (#20)

Adds per-instance cookie rules (admin-only) that are injected into
every network touchpoint during capture.

Storage:
- New cookie_rules table in the auth DB (idempotent migration)
- Rules have pattern_kind (global/wildcard/regex), optional url_pattern,
  and cookies_json (validated as string-only JSON object)

Matching (resolve_cookies_for_url):
- Global rules always apply
- Wildcard: * and ? with full metacharacter escaping; matched against
  hostname via reqwest::Url when pattern has no ://, full URL otherwise
- Regex: matched against the full URL
- Later rules in ordinal order override earlier ones per cookie name

All six network touchpoints receive resolved cookies:
- http::probe_url_kind and http::download: Cookie request header
- singlefile::save: Netscape cookie file -> --browser-cookies-file
- ytdlp::fetch_metadata and ytdlp::download: Netscape cookie file -> --cookies
- tweets::archive: semicolon credentials file -> --credentials-file
  (only when both ct0 and auth_token are present; otherwise falls back
  to ARCHIVR_TWITTER_CREDENTIALS_FILE)

Security:
- Cookie files written 0o600 (owner read/write only)
- Exact parsed hostname used as cookie domain (no PSL stripping)
- Files deleted unconditionally before any error propagates,
  including spawn failures (hold-result-then-delete pattern)
- No cookie values in process args (no --add-header exposure)

API: GET/POST /api/admin/cookie-rules, PATCH/DELETE /api/admin/cookie-rules/:uid
Frontend: Cookies tab in Settings (admin only) with rule list,
  inline edit, pattern-type selector, client-side JSON validation
CLI: CaptureConfig::default() - no behaviour change

254 tests passing (4 new cookie-rule handler tests)
This commit is contained in:
TheGeneralist 2026-07-06 19:01:34 +02:00 committed by GitHub
parent 21b11c211f
commit dae61e585d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 1013 additions and 103 deletions

View file

@ -365,6 +365,49 @@ export async function deleteOrphanBlobs(archiveId) {
return res.json()
}
// ── Cookie rules ──────────────────────────────────────────────────────────────
export async function listCookieRules() {
return getJson('/api/admin/cookie-rules')
}
export async function createCookieRule(urlPattern, patternKind, cookiesJson) {
const res = await fetch('/api/admin/cookie-rules', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
url_pattern: urlPattern || null,
pattern_kind: patternKind,
cookies_json: cookiesJson,
}),
})
if (!res.ok) {
const body = await res.json().catch(() => ({}))
throw new Error(body.message || `HTTP ${res.status}`)
}
return res.json()
}
export async function updateCookieRule(ruleUid, patch) {
const res = await fetch(`/api/admin/cookie-rules/${ruleUid}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(patch),
})
if (!res.ok) {
const body = await res.json().catch(() => ({}))
throw new Error(body.message || `HTTP ${res.status}`)
}
}
export async function deleteCookieRule(ruleUid) {
const res = await fetch(`/api/admin/cookie-rules/${ruleUid}`, { method: 'DELETE' })
if (!res.ok) {
const body = await res.json().catch(() => ({}))
throw new Error(body.message || `HTTP ${res.status}`)
}
}
// ── 401 interceptor ───────────────────────────────────────────────────────────
const _origFetch = window.fetch;
window.fetch = async (...args) => {