1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-22 03:05:32 +02:00
Commit graph

6 commits

Author SHA1 Message Date
03abfb4d18
feat(core): generic HTTP/S file URL capture (Track 1)
- Add crates/archivr-core/src/downloader/http.rs
  - download(url, store_path, timestamp) -> Result<(hash, extension)>
  - Rejects text/html responses with a clear error
  - Derives extension from URL path or Content-Type header
  - Follows redirects (capped at 10), user-agent archivr/0.1
  - 10 unit tests for extension/content-type helpers
- Add Source::Url variant to capture::Source enum
- determine_source: unmatched http/https URLs route to Source::Url
- source_metadata: Source::Url => ("web", "file", "file")
- generate_entry_title: Source::Url arm -> "Downloaded File" fallback
- perform_capture: Source::Url arm calls http::download, uses existing
  temp -> hash_exists -> move_temp_to_raw -> record_media_entry pipeline
- Update test expectations: 3 plain https:// cases now expect Source::Url
- Add reqwest 0.12 (blocking) to workspace and archivr-core deps
- Mark URLs milestone done in docs/README.md
- Update NEXT.md Track 1 status
2026-06-24 14:37:08 +02:00
2d7a4f1766
feat(server): configurable bind address with loopback default and non-loopback warning
- Add optional `bind` field to ServerRegistry (TOML + ARCHIVR_BIND env var)
- Default bind address remains 127.0.0.1:8080; non-loopback prints a warning
- Add route security classification comment block (READ/ADMIN/WRITE/STATIC)
- Add Security and Deployment section to docs/README.md
- Replace vague auth note in ARCHIVR-MENTAL-MODEL.md with concrete model description
- Add three registry tests covering bind field round-trip and defaults
2026-06-23 17:24:24 +02:00
44c17eee3d
docs: document runtime entrypoints 2026-06-21 19:40:56 +02:00
5552591f4f
feat: add X Article archiving (#6)
* Add X Article archiving
* All TOML formats changed to JSON
2026-04-04 19:14:06 +02:00
6bee5adfdd
feat: add Twitter tweet/thread archiving and platform shorthand support (#5)
* Add Twitter tweet and thread archiving support

* Fix tweet scraper path resolution and error reporting

* Flatten tweet archives and rearchive tweet assets

* refactor: simplify archive source parsing

* Refactor tweet archive source handling

* Clean up some clanker-written code

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Rename resolve_from_cwd to absolutize_path

Update call sites and tests to use the new API. Adjust tweet scraper
path/credentials handling and make small tweaks to local path hashing
and
raw store helpers.

Signed-off-by: TheGeneralist
<180094941+thegeneralist01@users.noreply.github.com>
Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Add docs for supported platforms, shorthands, and env vars

* Minor clean up

* Implement social shorthand URL expansion and tweet alias parsing

* Extract store and Twitter helpers into shared modules

---------

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>
2026-04-03 15:34:26 +02:00
2d59ab0af5
feat: add archiving of platform media files (#1)
* chore: specify non-ignored `.md` files

* refactor: rename youtube downloader to ytdlp

More generic name since yt-dlp supports many sites beyond YouTube.

* feat: add local file downloader

Supports file:// URLs for archiving local files.

* deps: add regex crate for URL pattern matching

* feat: expand source detection with granular YouTube types

- Split Source::YouTube into YouTubeVideo, YouTubePlaylist, YouTubeChannel
- Add Source::X for Twitter/X posts
- Add Source::Local for file:// URLs
- Add regex-based URL pattern matching for YouTube URLs
- Add shorthand schemes (yt:video/ID, youtube:playlist/ID, etc.)
- Add comprehensive tests for all URL patterns

* docs: update README milestones

Mark YouTube videos, Twitter videos, and local files as done.

* chore: update flake.lock

* feat: add shorthand schemes for X/Twitter media

* chore: move docs into docs dir

* Remove temp file using timestamp path

Delete the temp entry at store_path/temp/<timestamp> in both
the hash-exists and success paths. Stop constructing the full filename
with extension and remove the early process::exit to de-duplicate
cleanup.

* Add Nix caches and default flake package

* Add social platform source detection and update milestones

* Tighten social URL matching to avoid false positives

* Mark media archiving milestone complete
2026-03-31 12:39:35 +02:00
Renamed from README.md (Browse further)