1
Fork 0
mirror of https://github.com/thegeneralist01/archivr synced 2026-07-21 18:55:36 +02:00

feat(tweets): add re-archive button and fix thread-tweet orphan cleanup (#23)

Orphan cleanup bug: archiving x🧵A downloaded D/C/B/A JSONs and
media, but only registered artifacts for A. D/C/B files had no
entry_artifacts rows and were deleted as orphans.

Fix (staged scraper output, precise touched set):
- tweets::archive() stages all scraper output in temp/{ts}/tweet_stage/,
  validates, then renames JSONs to raw_tweets/. Return type changed from
  Result<bool> to Result<Vec<String>> (store-relative relpaths of every
  produced tweet JSON, i.e. the exact touched set).
- tweets::rearchive() (new): same staged approach but always runs the
  scraper. On scraper failure (tweet deleted/private), errors before
  touching raw_tweets/ so existing data is preserved.
- register_tweet_artifacts() (new private helper in capture.rs): registers
  every JSON in the touched set as a raw_tweet_json artifact, parses each
  for media blobs, registers those too. JSON read failure is a hard error
  with context, not a silent skip.
- record_tweet_entry() now accepts tweet_json_relpaths: &[String] and
  delegates artifact registration to register_tweet_artifacts().
- perform_capture() passes the returned vec from tweets::archive().

Re-archive feature:
- capture::perform_rearchive(): looks up entry by uid, validates
  tweet/tweet_thread, runs tweets::rearchive(), atomically swaps
  entry_artifacts in a DB transaction. archived_at, title, tags,
  collections are untouched.
- database: add get_entry_for_rearchive() and delete_entry_artifacts().
- POST /api/archives/:id/entries/:uid/rearchive: requires ROLE_USER,
  creates capture job, returns 202 + job_uid, runs perform_rearchive in
  spawn_blocking.
- Frontend: re-archive button in ContextRail for tweet/tweet_thread
  entries; polls job at 500ms; refreshes entry detail on success; shows
  error text on failure. Poll interval cleared before early-return on
  entry deselect to prevent stale updates.
This commit is contained in:
TheGeneralist 2026-07-11 16:14:58 +02:00 committed by GitHub
parent 03390362c5
commit 2779afee2d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 528 additions and 174 deletions

View file

@ -2391,6 +2391,43 @@ fn humanize_slug(slug: &str) -> String {
.join(" ")
}
/// A minimal view of an archived entry needed for re-archiving.
pub struct EntryForRearchive {
pub id: i64,
pub entity_kind: String,
pub source_metadata_json: String,
}
/// Looks up an entry by its public UID for re-archive purposes.
pub fn get_entry_for_rearchive(
conn: &Connection,
entry_uid: &str,
) -> Result<Option<EntryForRearchive>> {
conn.query_row(
"SELECT id, entity_kind, source_metadata_json \
FROM archived_entries WHERE entry_uid = ?1",
[entry_uid],
|row| {
Ok(EntryForRearchive {
id: row.get(0)?,
entity_kind: row.get(1)?,
source_metadata_json: row.get(2)?,
})
},
)
.optional()
.map_err(anyhow::Error::from)
}
/// Deletes all `entry_artifacts` rows for the given entry.
/// Call within a transaction for atomicity with re-insertion.
pub fn delete_entry_artifacts(conn: &Connection, entry_id: i64) -> Result<usize> {
Ok(conn.execute(
"DELETE FROM entry_artifacts WHERE entry_id = ?1",
[entry_id],
)?)
}
#[cfg(test)]
mod tests {
use super::*;