1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-22 03:05:32 +02:00
Commit graph

13 commits

Author SHA1 Message Date
2f7724f928
feat: package istilldontcareaboutcookies in flake, wire ARCHIVR_COOKIE_EXT
Add isdcac derivation mirroring ublockLite:
- Fetches ISDCAC-chrome-source.zip v1.1.9 from GitHub releases
- Validates manifest.json at extension root before install (guard against
  nested-folder zip regressions in future releases)
- Sets ARCHIVR_COOKIE_EXT in both archivr and archivr_server wrappers

Verified: nix build .#archivr-server and .#archivr both succeed;
wrapper scripts export correct store paths; manifest.json present at root.
2026-07-08 17:03:03 +02:00
8a90259a76
feat: uBlock Origin Lite integration for ad-blocking during WebPage captures
- singlefile.rs: when ARCHIVR_UBLOCK=true and ARCHIVR_UBLOCK_EXT is set,
  archivr owns Chrome's lifecycle (--headless=new, --remote-debugging-port,
  --load-extension); single-file connects via --browser-server instead of
  launching its own Chrome. Falls back to old behaviour with ublock_skipped=true
  when the ext path is missing or invalid.
- capture.rs: thread ublock_skipped through CaptureResult
- database.rs: add notes_json TEXT column to capture_jobs (DDL + idempotent
  ALTER TABLE migration); update_capture_job_status gains notes_json param
- archive.rs: expose notes_json in CaptureJobSummary
- routes.rs: store {"ublock_skipped":true} in notes_json on completed captures
- ToastStack.jsx: warning toast variant (toast--warning) with Details expander
  and Ignore button
- CaptureDialog.jsx: fire warning toast when poll result has ublock_skipped
- App.jsx: sessionStorage-backed Ignore suppression for ublock warnings
- styles.css: .toast--warning (amber left border) + .toast-warning-detail
- flake.nix: ublockLite derivation fetches uBOLite_2026.705.2152.chromium.zip
  (pinned SHA256) from uBlockOrigin/uBOL-home; sets ARCHIVR_UBLOCK_EXT in both
  archivr and archivr-server wrappers

Env vars:
  ARCHIVR_UBLOCK=true (default) — enable uBlock during WebPage captures
  ARCHIVR_UBLOCK_EXT — path to unpacked uBOL extension dir (set by Nix)
2026-07-07 17:18:55 +02:00
f9de282b0e
feat(nix): add NixOS module for archivr-server
modules/nixos/archivr-server.nix — services.archivr-server NixOS module:
- enable/bind/archives/user/group/openFirewall options
- generates archivr-server.toml from options via pkgs.writeText
- pins auth DB to /var/lib/archivr-server/ (StateDirectory)
- dedicated archivr system user + group
- hardened systemd unit: ProtectSystem=strict, NoNewPrivileges,
  PrivateTmp, RestrictNamespaces, etc.; archive paths whitelisted
- openFirewall parses the port from the bind string automatically

flake.nix:
- add self to outputs args
- add aarch64-linux to systems (Raspberry Pi / ARM servers)
- expose nixosModules.archivr-server + nixosModules.default

.gitignore: whitelist modules/ directory

docs/README.md: add 'Hosting on NixOS' section with full example
2026-06-29 16:51:20 +02:00
46ad816c4c
fix(flake): add openssl to buildInputs for Rust packages (#10)
Garnix CI was failing because openssl-sys couldn't find the OpenSSL
library. Added pkgs.openssl to buildInputs for both archivr_cli_unwrapped
and archivr_server_unwrapped to provide the necessary system dependency
and allow pkg-config to locate openssl.pc.
2026-06-27 11:54:31 +02:00
ad1c4f8ee5
fix(nix): guard pkgs.chromium behind isLinux for aarch64-darwin 2026-06-24 18:46:11 +02:00
c9ce0ee00b
feat(nix): add single-file-cli and chromium to both wrappers 2026-06-24 18:39:05 +02:00
b3f1932b62
chore: update gitignore and flake.nix 2026-06-23 20:33:41 +02:00
e3face0a1b
fix: bundle tweet scraper in archivr-server Nix wrapper
archivr_server now includes the Python scraper and sets
ARCHIVR_TWEET_PYTHON and ARCHIVR_TWEET_SCRAPER, matching what the
archivr CLI wrapper already does. Without this, browser capture of
tweets failed with 'No such file or directory' because the server
resolved the scraper relative to cwd.
2026-06-22 22:37:58 +02:00
b56c969624
feat: add db and multi-archive web UI foundation (#8)
* Add SQLite metadata database support

* Implement archive metadata database

* chore: let's guess cargoHash because there's something wrong with nixpkgs!

* Gate test-only database helpers behind cfg(test)

* Fix archive database row identity

* Use serde for archive metadata JSON

* Finalize archive runs at command level

* Handle archive command errors without panics

* Cover tweet entry metadata recording

* Document static regex invariants

* docs: add web UI design spec

* docs: add web UI implementation plan

* chore: move cli into workspace crate

* chore: track workspace crates directory

* refactor: extract archive core crate

* refactor: add core archive opening APIs

* refactor: rename taxonomy model to tags

* feat: add archive query APIs

* feat: add web server registry

* feat: expose archive server APIs

* feat: add archive table web UI

* fix: complete web UI smoke path

* docs: add architecture mental model

* docs: remove private superpowers plans

* nix: split cli and server packages

* chore: remove PLAN.md
2026-06-14 00:27:16 +02:00
5552591f4f
feat: add X Article archiving (#6)
* Add X Article archiving
* All TOML formats changed to JSON
2026-04-04 19:14:06 +02:00
6bee5adfdd
feat: add Twitter tweet/thread archiving and platform shorthand support (#5)
* Add Twitter tweet and thread archiving support

* Fix tweet scraper path resolution and error reporting

* Flatten tweet archives and rearchive tweet assets

* refactor: simplify archive source parsing

* Refactor tweet archive source handling

* Clean up some clanker-written code

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Rename resolve_from_cwd to absolutize_path

Update call sites and tests to use the new API. Adjust tweet scraper
path/credentials handling and make small tweaks to local path hashing
and
raw store helpers.

Signed-off-by: TheGeneralist
<180094941+thegeneralist01@users.noreply.github.com>
Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Add docs for supported platforms, shorthands, and env vars

* Minor clean up

* Implement social shorthand URL expansion and tweet alias parsing

* Extract store and Twitter helpers into shared modules

---------

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>
2026-04-03 15:34:26 +02:00
2d59ab0af5
feat: add archiving of platform media files (#1)
* chore: specify non-ignored `.md` files

* refactor: rename youtube downloader to ytdlp

More generic name since yt-dlp supports many sites beyond YouTube.

* feat: add local file downloader

Supports file:// URLs for archiving local files.

* deps: add regex crate for URL pattern matching

* feat: expand source detection with granular YouTube types

- Split Source::YouTube into YouTubeVideo, YouTubePlaylist, YouTubeChannel
- Add Source::X for Twitter/X posts
- Add Source::Local for file:// URLs
- Add regex-based URL pattern matching for YouTube URLs
- Add shorthand schemes (yt:video/ID, youtube:playlist/ID, etc.)
- Add comprehensive tests for all URL patterns

* docs: update README milestones

Mark YouTube videos, Twitter videos, and local files as done.

* chore: update flake.lock

* feat: add shorthand schemes for X/Twitter media

* chore: move docs into docs dir

* Remove temp file using timestamp path

Delete the temp entry at store_path/temp/<timestamp> in both
the hash-exists and success paths. Stop constructing the full filename
with extension and remove the early process::exit to de-duplicate
cleanup.

* Add Nix caches and default flake package

* Add social platform source detection and update milestones

* Tighten social URL matching to avoid false positives

* Mark media archiving milestone complete
2026-03-31 12:39:35 +02:00
01cc7826bf
feat: finish YouTube downloading; primitive "raw" archiving 2025-10-15 00:29:08 +02:00