mirror of
https://github.com/thegeneralist01/archivr
synced 2026-07-21 18:55:36 +02:00
feat(tweets): add re-archive button and fix thread-tweet orphan cleanup (#23)
Orphan cleanup bug: archiving x🧵A downloaded D/C/B/A JSONs and
media, but only registered artifacts for A. D/C/B files had no
entry_artifacts rows and were deleted as orphans.
Fix (staged scraper output, precise touched set):
- tweets::archive() stages all scraper output in temp/{ts}/tweet_stage/,
validates, then renames JSONs to raw_tweets/. Return type changed from
Result<bool> to Result<Vec<String>> (store-relative relpaths of every
produced tweet JSON, i.e. the exact touched set).
- tweets::rearchive() (new): same staged approach but always runs the
scraper. On scraper failure (tweet deleted/private), errors before
touching raw_tweets/ so existing data is preserved.
- register_tweet_artifacts() (new private helper in capture.rs): registers
every JSON in the touched set as a raw_tweet_json artifact, parses each
for media blobs, registers those too. JSON read failure is a hard error
with context, not a silent skip.
- record_tweet_entry() now accepts tweet_json_relpaths: &[String] and
delegates artifact registration to register_tweet_artifacts().
- perform_capture() passes the returned vec from tweets::archive().
Re-archive feature:
- capture::perform_rearchive(): looks up entry by uid, validates
tweet/tweet_thread, runs tweets::rearchive(), atomically swaps
entry_artifacts in a DB transaction. archived_at, title, tags,
collections are untouched.
- database: add get_entry_for_rearchive() and delete_entry_artifacts().
- POST /api/archives/:id/entries/:uid/rearchive: requires ROLE_USER,
creates capture job, returns 202 + job_uid, runs perform_rearchive in
spawn_blocking.
- Frontend: re-archive button in ContextRail for tweet/tweet_thread
entries; polls job at 500ms; refreshes entry detail on success; shows
error text on failure. Poll interval cleared before early-return on
entry deselect to prevent stale updates.
This commit is contained in:
parent
03390362c5
commit
2779afee2d
12 changed files with 528 additions and 174 deletions
|
|
@ -2391,6 +2391,43 @@ fn humanize_slug(slug: &str) -> String {
|
|||
.join(" ")
|
||||
}
|
||||
|
||||
/// A minimal view of an archived entry needed for re-archiving.
|
||||
pub struct EntryForRearchive {
|
||||
pub id: i64,
|
||||
pub entity_kind: String,
|
||||
pub source_metadata_json: String,
|
||||
}
|
||||
|
||||
/// Looks up an entry by its public UID for re-archive purposes.
|
||||
pub fn get_entry_for_rearchive(
|
||||
conn: &Connection,
|
||||
entry_uid: &str,
|
||||
) -> Result<Option<EntryForRearchive>> {
|
||||
conn.query_row(
|
||||
"SELECT id, entity_kind, source_metadata_json \
|
||||
FROM archived_entries WHERE entry_uid = ?1",
|
||||
[entry_uid],
|
||||
|row| {
|
||||
Ok(EntryForRearchive {
|
||||
id: row.get(0)?,
|
||||
entity_kind: row.get(1)?,
|
||||
source_metadata_json: row.get(2)?,
|
||||
})
|
||||
},
|
||||
)
|
||||
.optional()
|
||||
.map_err(anyhow::Error::from)
|
||||
}
|
||||
|
||||
/// Deletes all `entry_artifacts` rows for the given entry.
|
||||
/// Call within a transaction for atomicity with re-insertion.
|
||||
pub fn delete_entry_artifacts(conn: &Connection, entry_id: i64) -> Result<usize> {
|
||||
Ok(conn.execute(
|
||||
"DELETE FROM entry_artifacts WHERE entry_id = ?1",
|
||||
[entry_id],
|
||||
)?)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue