Server (routes.rs):
POST /api/util/resolve-tco — unauthenticated, accepts JSON array of
https://t.co/<alphanumeric> URLs only (strict regex validation, no SSRF
via input), capped at 50 per batch, 3 s timeout, redirect(Policy::none)
so the server only ever touches t.co itself. HEAD first, GET fallback if
HEAD returns no Location. Location sanitized to http/https only —
javascript:/data:/etc. fall back to the original t.co.
api.js:
resolveTcoUrls(urls) — project-convention wrapper for the new endpoint.
Returns {} on failure (callers degrade gracefully to bare t.co links).
TweetPreview.jsx:
After tweet data loads, per-tweet range-based coverage detection:
builds covered [start,end) from existing entity fromIndex/toIndex or
indices fallback, then finds regex matches whose span is NOT covered.
Resolves unique uncovered t.co URLs via resolveTcoUrls(), synthesises
one entity per occurrence (with exact fromIndex/toIndex so normalizeUrlAnn
gets correct bounds even for duplicate t.co URLs in the same tweet).
Augments entities.urls before setTweets() so all rendering paths
see expanded URLs.
Adds per-instance cookie rules (admin-only) that are injected into
every network touchpoint during capture.
Storage:
- New cookie_rules table in the auth DB (idempotent migration)
- Rules have pattern_kind (global/wildcard/regex), optional url_pattern,
and cookies_json (validated as string-only JSON object)
Matching (resolve_cookies_for_url):
- Global rules always apply
- Wildcard: * and ? with full metacharacter escaping; matched against
hostname via reqwest::Url when pattern has no ://, full URL otherwise
- Regex: matched against the full URL
- Later rules in ordinal order override earlier ones per cookie name
All six network touchpoints receive resolved cookies:
- http::probe_url_kind and http::download: Cookie request header
- singlefile::save: Netscape cookie file -> --browser-cookies-file
- ytdlp::fetch_metadata and ytdlp::download: Netscape cookie file -> --cookies
- tweets::archive: semicolon credentials file -> --credentials-file
(only when both ct0 and auth_token are present; otherwise falls back
to ARCHIVR_TWITTER_CREDENTIALS_FILE)
Security:
- Cookie files written 0o600 (owner read/write only)
- Exact parsed hostname used as cookie domain (no PSL stripping)
- Files deleted unconditionally before any error propagates,
including spawn failures (hold-result-then-delete pattern)
- No cookie values in process args (no --add-header exposure)
API: GET/POST /api/admin/cookie-rules, PATCH/DELETE /api/admin/cookie-rules/:uid
Frontend: Cookies tab in Settings (admin only) with rule list,
inline edit, pattern-type selector, client-side JSON validation
CLI: CaptureConfig::default() - no behaviour change
254 tests passing (4 new cookie-rule handler tests)
5 attempts / 15 min per IP. Excess attempts get 429 + Retry-After.
Client IP read from X-Forwarded-For with loopback fallback.
In-memory; prune inline on each check.
Refactored app() into app_with_state() + thin wrapper.
Uses parking_lot::Mutex (no unwrap, no poisoning).
- 5 tag routes (list tree, create, assign, remove, search with tag=)
- Tags nav, tag tree view, tag filter badge
- Entry tag pills with remove, assign-tag form in context rail
