feat(core): add more Freedium domains + restrict Freedium routing to supported domains (#36)

Browse source

Previously, any WebPage capture with via_freedium=true was routed
through the Freedium mirror regardless of the URL's host. This caused
non-paywall sites like borretti.me to be fetched through Freedium,
which is wrong — Freedium only knows how to handle a specific set of
publications.

Add is_freedium_supported_url() with a static allowlist of the 7 hosts
Freedium explicitly supports per its homepage announcement:

  Medium, NYT, WaPo, Bloomberg, Reuters, Economist, Financial Times

The gate matches on the bare domain or any subdomain (e.g.
towardsdatascience.medium.com). Unparseable URLs fall through to a
direct fetch. The existing freedium-mirror.cfd re-wrap guard is kept
as a belt-and-suspenders check after the new allowlist test.

Fixes: https://borretti.me/article/notes-on-managing-adhd archived via
Freedium despite not being a Medium article.

...

This commit is contained in:

TheGeneralist 2026-07-29 17:40:07 +02:00 • committed by GitHub

parent 6a0f59ea94

commit 95cd46978d

No known key found for this signature in database

GPG key ID: B5690EEEBB952194

1 changed files with 500 additions and 176 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

676

crates/archivr-core/src/capture.rs

View file

File diff suppressed because it is too large Load diff