1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-22 03:05:32 +02:00
Commit graph

10 commits

Author SHA1 Message Date
5fba6987a3
fix(capture): remove debug println! from hash_exists 2026-06-25 16:19:33 +02:00
351e74df09
feat(fonts): wire font extraction into capture pipeline + blob serving route
- capture.rs: add archive_id: Option<&str> to perform_capture; when Some,
  call font_extractor::extract_and_rewrite before hashing HTML, register
  each font as a deduplicated blob + 'font' artifact
- main.rs: pass None as archive_id (CLI keeps fonts embedded)
- routes.rs: add GET /api/archives/:id/blobs/:sha256 (serve_blob handler),
  pass Some(&archive_id) to perform_capture in capture_handler,
  add ApiError::internal constructor
2026-06-25 16:07:50 +02:00
653efa9705
feat(capture): store favicon artifact for WebPage entries 2026-06-24 19:16:21 +02:00
fe9ff2dafe
feat(capture): add Source::WebPage, route HTML URLs to singlefile downloader 2026-06-24 18:39:02 +02:00
143f71bc17
feat(core): auto-title for HTTP/S URL downloads
http::download now returns (hash, extension, Option<title_hint>).
Title is derived from Content-Disposition filename header (RFC 5987
filename*= preferred over plain filename=), falling back to the last
path segment of the final URL after redirects (percent-decoded).

capture.rs Source::Url arm passes the title through to record_media_entry
instead of None, so entries like 'Facharbeit.pdf' or 'facharbeit' appear
in the Title column instead of the raw entry_uid.

Adds percent_decode(), title_from_content_disposition(), title_from_url()
helpers with 10 new unit tests (122 total, all green).
2026-06-24 14:56:48 +02:00
03abfb4d18
feat(core): generic HTTP/S file URL capture (Track 1)
- Add crates/archivr-core/src/downloader/http.rs
  - download(url, store_path, timestamp) -> Result<(hash, extension)>
  - Rejects text/html responses with a clear error
  - Derives extension from URL path or Content-Type header
  - Follows redirects (capped at 10), user-agent archivr/0.1
  - 10 unit tests for extension/content-type helpers
- Add Source::Url variant to capture::Source enum
- determine_source: unmatched http/https URLs route to Source::Url
- source_metadata: Source::Url => ("web", "file", "file")
- generate_entry_title: Source::Url arm -> "Downloaded File" fallback
- perform_capture: Source::Url arm calls http::download, uses existing
  temp -> hash_exists -> move_temp_to_raw -> record_media_entry pipeline
- Update test expectations: 3 plain https:// cases now expect Source::Url
- Add reqwest 0.12 (blocking) to workspace and archivr-core deps
- Mark URLs milestone done in docs/README.md
- Update NEXT.md Track 1 status
2026-06-24 14:37:08 +02:00
80828951e2
feat: wire metadata titles through perform_capture and record_tweet_entry
- perform_capture: fetch yt-dlp metadata (separate --dump-json call)
  before download; derive local title from file:// path filename
- Compute entry_title before record_media_entry call; pass it through
  instead of the temporary None placeholder
- record_tweet_entry: read tweet JSON before entry creation so title
  can be set on NewEntry; add tweet_metadata_from_json helper
2026-06-23 22:12:34 +02:00
3e289383a9
feat: platform-aware entry titling for all 13 source types
- Add PlatformMetadata struct and generate_entry_title() covering all 13
  Source variants (YouTubeVideo, YouTubePlaylist, YouTubeChannel, X, Tweet,
  TweetThread, Instagram, Facebook, TikTok, Reddit, Snapchat, Local, Other)
- Add downloader/metadata.rs: extract_from_ytdlp_json() parses yt-dlp
  --dump-json output into PlatformMetadata; extracts Reddit subreddit
  from webpage_url via regex
- Add ytdlp::fetch_metadata(): separate --dump-json invocation that does
  not interfere with the actual download call
- Extend record_media_entry() with title: Option<String> param; wire
  yt-dlp metadata fetch + local filename extraction in perform_capture
- Restructure record_tweet_entry() to read tweet JSON before entry
  creation; extract title via tweet_metadata_from_json()
- 16 title-generation unit tests + 6 metadata extraction unit tests
2026-06-23 21:49:34 +02:00
5ea0c396b8
fix: expand yt:/youtube: shorthands to real URLs before yt-dlp
expand_shorthand_to_url handled instagram:/facebook:/tiktok: etc. but
not yt:/youtube:. yt-dlp doesn't know the yt: scheme, so shorthands
like yt:video/ID failed with 'Unsupported url scheme'.

Add YouTube cases for video/, short/, shorts/, playlist/, channel/,
c/, user/, and @handle. Full https:// URLs pass through unchanged.

Add tests pinning all new expansions.
2026-06-22 22:42:41 +02:00
8d0352c2c4
feat: browser capture button (Track 4)
- Add archivr-core/src/capture.rs: Source enum, all capture helpers,
  fail_run (replaces process::exit fail_archive_and_exit), and
  perform_capture() as the public entry point
- Slim archivr-cli/src/main.rs to 103 lines: thin adapter over
  archivr_core::capture::perform_capture
- Move all source-classification and tweet-entry tests from CLI to
  archivr-core/src/capture.rs (they test core logic, belong in core)
- Add POST /api/archives/:archive_id/captures route in archivr-server:
  validates non-empty locator (400), resolves archive (404), delegates
  to perform_capture, returns {run_uid, status}
- Add capture dialog to browser UI: dialog markup in index.html,
  showModal/submit/cancel/loading/error wiring in app.js, dialog
  styles in styles.css
- 77 tests pass, clean build (0 warnings)
2026-06-22 22:18:44 +02:00