1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-21 18:55:36 +02:00
Commit graph

63 commits

Author SHA1 Message Date
fe9ff2dafe
feat(capture): add Source::WebPage, route HTML URLs to singlefile downloader 2026-06-24 18:39:02 +02:00
fd06632073
feat(singlefile): add single-file-cli downloader module 2026-06-24 18:38:57 +02:00
b7ab12898e
feat(http): add UrlKind enum and probe_url_kind HEAD probe 2026-06-24 18:38:52 +02:00
143f71bc17
feat(core): auto-title for HTTP/S URL downloads
http::download now returns (hash, extension, Option<title_hint>).
Title is derived from Content-Disposition filename header (RFC 5987
filename*= preferred over plain filename=), falling back to the last
path segment of the final URL after redirects (percent-decoded).

capture.rs Source::Url arm passes the title through to record_media_entry
instead of None, so entries like 'Facharbeit.pdf' or 'facharbeit' appear
in the Title column instead of the raw entry_uid.

Adds percent_decode(), title_from_content_disposition(), title_from_url()
helpers with 10 new unit tests (122 total, all green).
2026-06-24 14:56:48 +02:00
03abfb4d18
feat(core): generic HTTP/S file URL capture (Track 1)
- Add crates/archivr-core/src/downloader/http.rs
  - download(url, store_path, timestamp) -> Result<(hash, extension)>
  - Rejects text/html responses with a clear error
  - Derives extension from URL path or Content-Type header
  - Follows redirects (capped at 10), user-agent archivr/0.1
  - 10 unit tests for extension/content-type helpers
- Add Source::Url variant to capture::Source enum
- determine_source: unmatched http/https URLs route to Source::Url
- source_metadata: Source::Url => ("web", "file", "file")
- generate_entry_title: Source::Url arm -> "Downloaded File" fallback
- perform_capture: Source::Url arm calls http::download, uses existing
  temp -> hash_exists -> move_temp_to_raw -> record_media_entry pipeline
- Update test expectations: 3 plain https:// cases now expect Source::Url
- Add reqwest 0.12 (blocking) to workspace and archivr-core deps
- Mark URLs milestone done in docs/README.md
- Update NEXT.md Track 1 status
2026-06-24 14:37:08 +02:00
80828951e2
feat: wire metadata titles through perform_capture and record_tweet_entry
- perform_capture: fetch yt-dlp metadata (separate --dump-json call)
  before download; derive local title from file:// path filename
- Compute entry_title before record_media_entry call; pass it through
  instead of the temporary None placeholder
- record_tweet_entry: read tweet JSON before entry creation so title
  can be set on NewEntry; add tweet_metadata_from_json helper
2026-06-23 22:12:34 +02:00
3e289383a9
feat: platform-aware entry titling for all 13 source types
- Add PlatformMetadata struct and generate_entry_title() covering all 13
  Source variants (YouTubeVideo, YouTubePlaylist, YouTubeChannel, X, Tweet,
  TweetThread, Instagram, Facebook, TikTok, Reddit, Snapchat, Local, Other)
- Add downloader/metadata.rs: extract_from_ytdlp_json() parses yt-dlp
  --dump-json output into PlatformMetadata; extracts Reddit subreddit
  from webpage_url via regex
- Add ytdlp::fetch_metadata(): separate --dump-json invocation that does
  not interfere with the actual download call
- Extend record_media_entry() with title: Option<String> param; wire
  yt-dlp metadata fetch + local filename extraction in perform_capture
- Restructure record_tweet_entry() to read tweet JSON before entry
  creation; extract title via tweet_metadata_from_json()
- 16 title-generation unit tests + 6 metadata extraction unit tests
2026-06-23 21:49:34 +02:00
5ea0c396b8
fix: expand yt:/youtube: shorthands to real URLs before yt-dlp
expand_shorthand_to_url handled instagram:/facebook:/tiktok: etc. but
not yt:/youtube:. yt-dlp doesn't know the yt: scheme, so shorthands
like yt:video/ID failed with 'Unsupported url scheme'.

Add YouTube cases for video/, short/, shorts/, playlist/, channel/,
c/, user/, and @handle. Full https:// URLs pass through unchanged.

Add tests pinning all new expansions.
2026-06-22 22:42:41 +02:00
8d0352c2c4
feat: browser capture button (Track 4)
- Add archivr-core/src/capture.rs: Source enum, all capture helpers,
  fail_run (replaces process::exit fail_archive_and_exit), and
  perform_capture() as the public entry point
- Slim archivr-cli/src/main.rs to 103 lines: thin adapter over
  archivr_core::capture::perform_capture
- Move all source-classification and tweet-entry tests from CLI to
  archivr-core/src/capture.rs (they test core logic, belong in core)
- Add POST /api/archives/:archive_id/captures route in archivr-server:
  validates non-empty locator (400), resolves archive (404), delegates
  to perform_capture, returns {run_uid, status}
- Add capture dialog to browser UI: dialog markup in index.html,
  showModal/submit/cancel/loading/error wiring in app.js, dialog
  styles in styles.css
- 77 tests pass, clean build (0 warnings)
2026-06-22 22:18:44 +02:00
db953de67a
feat(core): hierarchical tag API
- Tag/TagNode types, create_tag, list_tag_tree, assign_entry_tag,
  remove_entry_tag, entries_for_tag
- Tag-filtered search returns child entries (no root restriction)
- parent_entry_uid added to EntrySummary
- 8 new tests
2026-06-22 14:48:46 +02:00
83f909585f
feat: server-side search with structured prefix query language 2026-06-22 14:48:46 +02:00
dcf9e127bb
feat: entry detail view and artifact file serving 2026-06-22 14:48:46 +02:00
b56c969624
feat: add db and multi-archive web UI foundation (#8)
* Add SQLite metadata database support

* Implement archive metadata database

* chore: let's guess cargoHash because there's something wrong with nixpkgs!

* Gate test-only database helpers behind cfg(test)

* Fix archive database row identity

* Use serde for archive metadata JSON

* Finalize archive runs at command level

* Handle archive command errors without panics

* Cover tweet entry metadata recording

* Document static regex invariants

* docs: add web UI design spec

* docs: add web UI implementation plan

* chore: move cli into workspace crate

* chore: track workspace crates directory

* refactor: extract archive core crate

* refactor: add core archive opening APIs

* refactor: rename taxonomy model to tags

* feat: add archive query APIs

* feat: add web server registry

* feat: expose archive server APIs

* feat: add archive table web UI

* fix: complete web UI smoke path

* docs: add architecture mental model

* docs: remove private superpowers plans

* nix: split cli and server packages

* chore: remove PLAN.md
2026-06-14 00:27:16 +02:00