1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-21 18:55:36 +02:00
Commit graph

9 commits

Author SHA1 Message Date
71e176cbd1
security: fix two Codex review findings + add port option
Fix 1 — NixOS: store/ dir missing from ReadWritePaths
  archive_path points to .archivr/; captures write artifacts to the
  sibling store/ directory. Whitelisting only .archivr/ caused POST
  /captures to fail under ProtectSystem=strict.
  Fix: use builtins.dirOf a.path to whitelist the parent, which
  contains both .archivr/ and store/.

Fix 2 — Rust: X-Forwarded-For was unconditionally trusted
  An attacker could send a different IP on every login attempt,
  bypassing LOGIN_MAX_ATTEMPTS entirely.
  Fix: use ConnectInfo<SocketAddr> (via into_make_service_with_connect_info)
  as the primary rate-limit key; XFF is trusted only when the TCP peer
  is loopback (i.e. a local reverse proxy). Tests without a real socket
  fall back to XFF unchanged.

Port configuration:
  NixOS module: split 'bind' string into separate 'listenAddress'
  (default 127.0.0.1) and 'port' (default 8080) options. openFirewall
  now uses cfg.port directly instead of parsing it from the bind string.
  README NixOS example updated accordingly.
2026-06-29 20:32:08 +02:00
f9de282b0e
feat(nix): add NixOS module for archivr-server
modules/nixos/archivr-server.nix — services.archivr-server NixOS module:
- enable/bind/archives/user/group/openFirewall options
- generates archivr-server.toml from options via pkgs.writeText
- pins auth DB to /var/lib/archivr-server/ (StateDirectory)
- dedicated archivr system user + group
- hardened systemd unit: ProtectSystem=strict, NoNewPrivileges,
  PrivateTmp, RestrictNamespaces, etc.; archive paths whitelisted
- openFirewall parses the port from the bind string automatically

flake.nix:
- add self to outputs args
- add aarch64-linux to systems (Raspberry Pi / ARM servers)
- expose nixosModules.archivr-server + nixosModules.default

.gitignore: whitelist modules/ directory

docs/README.md: add 'Hosting on NixOS' section with full example
2026-06-29 16:51:20 +02:00
e70c6b37cd
docs: document single-file env vars, check off web page archiving milestone 2026-06-24 18:39:12 +02:00
03abfb4d18
feat(core): generic HTTP/S file URL capture (Track 1)
- Add crates/archivr-core/src/downloader/http.rs
  - download(url, store_path, timestamp) -> Result<(hash, extension)>
  - Rejects text/html responses with a clear error
  - Derives extension from URL path or Content-Type header
  - Follows redirects (capped at 10), user-agent archivr/0.1
  - 10 unit tests for extension/content-type helpers
- Add Source::Url variant to capture::Source enum
- determine_source: unmatched http/https URLs route to Source::Url
- source_metadata: Source::Url => ("web", "file", "file")
- generate_entry_title: Source::Url arm -> "Downloaded File" fallback
- perform_capture: Source::Url arm calls http::download, uses existing
  temp -> hash_exists -> move_temp_to_raw -> record_media_entry pipeline
- Update test expectations: 3 plain https:// cases now expect Source::Url
- Add reqwest 0.12 (blocking) to workspace and archivr-core deps
- Mark URLs milestone done in docs/README.md
- Update NEXT.md Track 1 status
2026-06-24 14:37:08 +02:00
2d7a4f1766
feat(server): configurable bind address with loopback default and non-loopback warning
- Add optional `bind` field to ServerRegistry (TOML + ARCHIVR_BIND env var)
- Default bind address remains 127.0.0.1:8080; non-loopback prints a warning
- Add route security classification comment block (READ/ADMIN/WRITE/STATIC)
- Add Security and Deployment section to docs/README.md
- Replace vague auth note in ARCHIVR-MENTAL-MODEL.md with concrete model description
- Add three registry tests covering bind field round-trip and defaults
2026-06-23 17:24:24 +02:00
44c17eee3d
docs: document runtime entrypoints 2026-06-21 19:40:56 +02:00
5552591f4f
feat: add X Article archiving (#6)
* Add X Article archiving
* All TOML formats changed to JSON
2026-04-04 19:14:06 +02:00
6bee5adfdd
feat: add Twitter tweet/thread archiving and platform shorthand support (#5)
* Add Twitter tweet and thread archiving support

* Fix tweet scraper path resolution and error reporting

* Flatten tweet archives and rearchive tweet assets

* refactor: simplify archive source parsing

* Refactor tweet archive source handling

* Clean up some clanker-written code

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Rename resolve_from_cwd to absolutize_path

Update call sites and tests to use the new API. Adjust tweet scraper
path/credentials handling and make small tweaks to local path hashing
and
raw store helpers.

Signed-off-by: TheGeneralist
<180094941+thegeneralist01@users.noreply.github.com>
Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Add docs for supported platforms, shorthands, and env vars

* Minor clean up

* Implement social shorthand URL expansion and tweet alias parsing

* Extract store and Twitter helpers into shared modules

---------

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>
2026-04-03 15:34:26 +02:00
2d59ab0af5
feat: add archiving of platform media files (#1)
* chore: specify non-ignored `.md` files

* refactor: rename youtube downloader to ytdlp

More generic name since yt-dlp supports many sites beyond YouTube.

* feat: add local file downloader

Supports file:// URLs for archiving local files.

* deps: add regex crate for URL pattern matching

* feat: expand source detection with granular YouTube types

- Split Source::YouTube into YouTubeVideo, YouTubePlaylist, YouTubeChannel
- Add Source::X for Twitter/X posts
- Add Source::Local for file:// URLs
- Add regex-based URL pattern matching for YouTube URLs
- Add shorthand schemes (yt:video/ID, youtube:playlist/ID, etc.)
- Add comprehensive tests for all URL patterns

* docs: update README milestones

Mark YouTube videos, Twitter videos, and local files as done.

* chore: update flake.lock

* feat: add shorthand schemes for X/Twitter media

* chore: move docs into docs dir

* Remove temp file using timestamp path

Delete the temp entry at store_path/temp/<timestamp> in both
the hash-exists and success paths. Stop constructing the full filename
with extension and remove the early process::exit to de-duplicate
cleanup.

* Add Nix caches and default flake package

* Add social platform source detection and update milestones

* Tighten social URL matching to avoid false positives

* Mark media archiving milestone complete
2026-03-31 12:39:35 +02:00
Renamed from README.md (Browse further)