1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-21 18:55:36 +02:00
Commit graph

2 commits

Author SHA1 Message Date
21b11c211f
feat: YouTube Music audio capture (ytm: shorthand, Spotify detection, stalled job recovery) (#19)
* feat: add YouTube Music and Spotify source detection

- Add Source variants: YouTubeMusicTrack, YouTubeMusicPlaylist,
  SpotifyTrack, SpotifyAlbum, SpotifyPlaylist
- ytm:ID shorthand → music.youtube.com/watch?v=ID (audio-only, forced
  in core regardless of caller quality hint)
- ytm:playlist/ID and music.youtube.com/playlist URLs detected but
  fail with 'not yet implemented' via fail_run
- Spotify URLs/shorthands detected and fail fast with clear DRM error
  via fail_run (after run item created, so status is visible in /runs)
- source_metadata: youtube_music/music/audio and spotify/music/audio
  (entity_kind='music' for UI pill, representation_kind='audio' stored)
- locator_to_ytdlp_url includes YouTubeMusicTrack for probe endpoint
- generate_entry_title: 'Title — Artist' for YTM tracks
- Frontend: isVideoSource handles ytm: and music.youtube.com/watch;
  Spotify returns false (no probe, clear server error on submit)
- Placeholder updated to include ytm:ID
- SOURCE_ICONS: youtube_music (red disc) and spotify (green waves)
- 14 new tests covering all new sources (163 total, all pass)

* fix: prevent yt-dlp playlist expansion and stalled run recovery

- Add --no-playlist to ytdlp::download and fetch_metadata: URLs with a
  list= parameter (e.g. music.youtube.com/watch?v=ID&list=RDAMVM…) no
  longer cause yt-dlp to expand the full playlist and hang; both the
  metadata probe and the download are now single-item only

- Fix fail_stalled_capture_jobs to also recover archive_runs and
  archive_run_items: capture_jobs.run_uid is NULL at crash time so a
  join is unreliable; instead fail all archive_runs/items still
  in_progress directly, then recount failed_count via subquery.
  Startup recovery now makes the Runs UI reflect the correct failed
  state after a hard shutdown

- Expand fail_stalled_jobs_on_restart test to assert archive_run and
  archive_run_item rows are also marked failed, not just capture_jobs

* fix: use play triangle for youtube_music icon
2026-07-06 15:34:14 +02:00
339076e6a2
feat: add orphan blob cleanup to Settings > Storage tab (#18)
- database.rs: add has_active_capture_jobs(), list_orphaned_blob_rows(),
  all_referenced_file_relpaths(), delete_orphaned_blob_rows()
- routes.rs: GET/DELETE /api/archives/:id/blob-cleanup (ROLE_ADMIN)
  - GET returns {orphaned_blob_rows, deletable_files, total_bytes}
  - DELETE has two active-capture guards (before and after disk walk)
    to prevent deleting files mid-capture; walks raw/ and raw_tweets/
  - Referenced set = entry_artifacts.relpath ∪ live blobs' raw_relpath,
    so a file is never deleted if any artifact still points at its path
- api.js: scanOrphanBlobs(), deleteOrphanBlobs()
- App.jsx: pass archiveId to SettingsView; add 'storage' to SETTINGS_TABS
  so /settings/storage survives refresh/back navigation
- SettingsView.jsx: new Storage tab (admin-only) with idle→scanning→
  scanned→deleting→done/error state machine; shows file/record counts
  and human-readable byte sizes before a btn-danger confirm

Tests (14 new, 248 total passing):
- database.rs: has_active_capture_jobs for pending/running/completed,
  list_orphaned_blob_rows, all_referenced_file_relpaths edge cases,
  delete_orphaned_blob_rows preserves referenced rows
- routes.rs: auth (401), active-capture 409 on GET and DELETE,
  end-to-end delete preserving referenced file and removing orphan
  blob file + extra disk-only file
2026-07-06 14:01:38 +02:00