thegeneralist01/archivr
1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-05-30 08:36:47 +02:00
Code Issues Activity
14 commits 2 branches 0 tags 481 KiB
Commit graph

14 commits

This branch
This branch
All branches
Author SHA1 Message Date
TheGeneralist
a79e7d7dba
Fix archive database row identity 2026-05-29 16:40:23 +02:00
TheGeneralist
b712bdd5db
Gate test-only database helpers behind cfg(test) 2026-05-29 16:30:29 +02:00
TheGeneralist
5fae7c71af
chore: let's guess cargoHash because there's something wrong with nixpkgs! 2026-05-29 15:22:18 +02:00
TheGeneralist
13e1ed586e
Merge branch 'codex/implement-database-design' into codex/plan-database-design 2026-05-04 20:27:57 +02:00
TheGeneralist
1389847b94
Implement archive metadata database 2026-05-04 20:27:54 +02:00
TheGeneralist
f38621231f
Add SQLite metadata database support 2026-04-30 21:55:53 +04:00
TheGeneralist
cc380ec5ba
fix: extract full tweet text from note_tweet field when available 2026-04-06 11:05:32 +02:00
TheGeneralist
5552591f4f
feat: add X Article archiving (#6) 
* Add X Article archiving
* All TOML formats changed to JSON
 2026-04-04 19:14:06 +02:00
TheGeneralist
51e986b6b2
chore: add isolate_cookies script 
Add `isolate_cookies` script for creating Twitter credentials file
 2026-04-03 15:42:20 +02:00
TheGeneralist
6bee5adfdd
feat: add Twitter tweet/thread archiving and platform shorthand support (#5) 
* Add Twitter tweet and thread archiving support

* Fix tweet scraper path resolution and error reporting

* Flatten tweet archives and rearchive tweet assets

* refactor: simplify archive source parsing

* Refactor tweet archive source handling

* Clean up some clanker-written code

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Rename resolve_from_cwd to absolutize_path

Update call sites and tests to use the new API. Adjust tweet scraper
path/credentials handling and make small tweaks to local path hashing
and
raw store helpers.

Signed-off-by: TheGeneralist
<180094941+thegeneralist01@users.noreply.github.com>
Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>

* Add docs for supported platforms, shorthands, and env vars

* Minor clean up

* Implement social shorthand URL expansion and tweet alias parsing

* Extract store and Twitter helpers into shared modules

---------

Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>
 2026-04-03 15:34:26 +02:00
TheGeneralist
9441a9d9fb
fix: use remove_dir_all for removal of temp directories 
Signed-off-by: TheGeneralist <180094941+thegeneralist01@users.noreply.github.com>
 2026-03-31 12:49:44 +02:00
TheGeneralist
2d59ab0af5
feat: add archiving of platform media files (#1) 
* chore: specify non-ignored `.md` files

* refactor: rename youtube downloader to ytdlp

More generic name since yt-dlp supports many sites beyond YouTube.

* feat: add local file downloader

Supports file:// URLs for archiving local files.

* deps: add regex crate for URL pattern matching

* feat: expand source detection with granular YouTube types

- Split Source::YouTube into YouTubeVideo, YouTubePlaylist, YouTubeChannel
- Add Source::X for Twitter/X posts
- Add Source::Local for file:// URLs
- Add regex-based URL pattern matching for YouTube URLs
- Add shorthand schemes (yt:video/ID, youtube:playlist/ID, etc.)
- Add comprehensive tests for all URL patterns

* docs: update README milestones

Mark YouTube videos, Twitter videos, and local files as done.

* chore: update flake.lock

* feat: add shorthand schemes for X/Twitter media

* chore: move docs into docs dir

* Remove temp file using timestamp path

Delete the temp entry at store_path/temp/<timestamp> in both
the hash-exists and success paths. Stop constructing the full filename
with extension and remove the early process::exit to de-duplicate
cleanup.

* Add Nix caches and default flake package

* Add social platform source detection and update milestones

* Tighten social URL matching to avoid false positives

* Mark media archiving milestone complete
 2026-03-31 12:39:35 +02:00
TheGeneralist
553cca99ca
docs: add README and LICENSE 2025-10-15 00:43:54 +02:00
TheGeneralist
01cc7826bf
feat: finish YouTube downloading; primitive "raw" archiving 2025-10-15 00:29:08 +02:00