mirror of
https://github.com/thegeneralist01/archivr
synced 2026-07-21 18:55:36 +02:00
feat: entry previews (#24)
* feat: entry previews (video, tweet, article, iframe, image, audio bar)
- Add PreviewPanel dispatch hub routing by entity_kind + artifact extension
- VideoPreview: HTML5 <video> for YouTube/Instagram/TikTok/Reddit/X posts
- TweetPreview: tweet card, thread, and X article renderer (ported from x-article-renderer)
- IframePreview: sandboxed iframe for SingleFile web pages and PDFs
- ImagePreview: image viewer with click-to-open-fullsize
- AudioBar: persistent fixed-bottom player (Spotify-style) that survives entry navigation
- Lift entryDetail to App.jsx, shared between PreviewPanel and ContextRail
- 3-column layout (workspace | 300px preview | 340px rail) when preview active
- Stale-guard fixes: seq incremented before early returns in all async effects
- handleRearchive: capture startSeq/entryUid at call time, guard every async resume
- TweetPreview: reset loading/error/tweets before early-return branches
* feat: entry previews — tweet/thread/article/video/audio/image/iframe/pdf
- PreviewModal: modal overlay with new-tab link (↗) and keyboard close
- PreviewPanel: routes by entity_kind + primary_media extension to the
correct viewer (tweet/video/audio/pdf/html/image/fallback)
- TweetPreview: full X-style tweet, thread, and article renderer with
local artifact map for archived media (CDN fallback)
- AudioBar: persistent fixed bottom player, triggered via ContextRail
Play button; body.has-audio-bar pads content above it
- VideoPreview, IframePreview, ImagePreview: inline viewers
- PreviewPage: standalone /preview/:archiveId/:entryUid route
- ContextRail: Play/Preview buttons; isAudio/isPreviewable detection
- App.jsx: preview modal state, currentAudio state, preview route guard,
has-audio-bar body class effect
- routes.rs: CSP updated (media-src self blob https; frame-ancestors self;
Google Fonts + external images/scripts whitelisted)
- styles.css: preview modal, tweet-wrap scroll (min-height:0), audio bar
body padding, newtab button, preview panel flex layout
* style: tighten article/tweet preview spacing
- aMeta padding: 14px → 10px
- article title marginBottom: 10px → 8px
- aAuthorRow marginBottom: 10px → 8px
- bH1 top margin: 20px → 16px
- bH2 top margin: 18px → 14px
- bHr margin: 20px → 14px
- .preview-tweet-wrap padding: 20px → 12px (already committed)
More content visible above the fold in both modal and standalone views.
* feat: tweet/article preview quality pass
HTML entities: decode > < & etc. on sliced segments only
(entity offsets index the stored string; decoding before slicing shifts them)
Image lightbox: click any tweet/thread/article image to open full-screen
viewer; cmd+click follows <a> to open in new tab; arrow-key + ‹ › nav;
Escape closes; 1/N counter; ↗ open-in-new-tab link
Multi-image grid: 2 photos → side-by-side (180px rows); 3 → left spans
both rows; 4 → 2×2 (140px rows); single image unchanged
Empty media grid ghost: build photos/videoItems arrays first, render
.mediaGrid div only when at least one item resolved (advisory: never gate
on raw media.length when map items can all return null)
QT indicator: ↻ QT badge on tweet.is_quote_status === true entries
Modal shrinks for short content: height: 88vh → max-height: 88vh;
.preview-modal-body gets max-height: calc(88vh - 52px) so long threads
still scroll (advisory: don't rely on flex:1 once parent has no fixed height)
Video scrollbar leak: .preview-modal-body overflow: auto → hidden; each
child (tweet-wrap, video-wrap, iframe) manages its own scroll surface
Styled thin scrollbar on .preview-tweet-wrap (matches workspace rail)
ArticleRenderer: cover image and body images are lightbox-clickable;
opts thread through renderBlocksJSX → renderBlockJSX → renderAtomicJSX
* fix: move artifact fetch to api.js; stop Escape propagation from lightbox
- Export fetchEntryArtifacts(archiveId, entryUid, indices) from api.js
using Promise.all + getJson (follows project convention: all /api calls
go through api.js, never inline fetch in components)
- TweetPreview: import fetchEntryArtifacts, replace inline Promise.all
- MediaLightbox keydown handler: stopPropagation + preventDefault for
Escape/ArrowLeft/ArrowRight so the parent PreviewModal window listener
does not also fire and close the modal behind the lightbox
* fix: iframe/page preview height chain and toolbar UX
Problem: changing .preview-modal from height to max-height broke iframe
previews - <iframe style='flex:1'> needs a concrete ancestor height, which
max-height alone doesn't supply when content is shorter than the cap.
Fix - CSS:
.preview-modal--full { height: 88vh } applied to non-tweet modals
.preview-modal--full .preview-modal-body { max-height: none }
.preview-iframe-toolbar span: remove text-transform/letter-spacing
(was uppercasing the URL/title in shouty caps)
Fix - PreviewModal: className adds --full when entity_kind is not
tweet/tweet_thread; tweet previews keep shrink-to-fit behavior.
Fix - PreviewPanel: pass title + original_url from summary to IframePreview
for both HTML and PDF; wrappers use flex:1/minHeight:0 not height:100%.
Fix - IframePreview:
- Accept title + originalUrl props; show originalUrl in toolbar (falls
back to artifact src only when original_url absent); show title above
URL when available
- flex:1 + minHeight:0 instead of height:100% on the wrap div
- Single unified layout for page + pdf (both just show the iframe)
* feat: expand t.co links; linkify bare URLs in tweet and article text
Frontend:
- resolveEntityBounds: try multiple candidate strings in order (u.url
first, since that's the t.co short URL that appears in full_text)
- normalizeUrlAnn: multi-candidate search; href = expanded > url,
display = display_url > expanded > url
- linkifyText(): regex linkifier for entity-less bare URLs; trims
trailing punctuation [.,;:!?)] before linking; used in both
renderTweetTextJSX and renderInlineJSX including their early-return
paths (anns.length === 0) that previously bypassed linkification
- renderInlineJSX: fix mention href mention.name → screen_name;
replace t.co segment text with url.display when entity covers it
Scraper (vendor/twitter/scrape_user_tweet_contents.py):
- extract_tweet_data: when note_tweet text is used, pull urls/mentions/
hashtags/symbols from note_result.entity_set (correct indices for the
note text); keep media from legacy.entities (no note media downloads)
* feat: server-side t.co resolver + frontend augmentation
Server (routes.rs):
POST /api/util/resolve-tco — unauthenticated, accepts JSON array of
https://t.co/<alphanumeric> URLs only (strict regex validation, no SSRF
via input), capped at 50 per batch, 3 s timeout, redirect(Policy::none)
so the server only ever touches t.co itself. HEAD first, GET fallback if
HEAD returns no Location. Location sanitized to http/https only —
javascript:/data:/etc. fall back to the original t.co.
api.js:
resolveTcoUrls(urls) — project-convention wrapper for the new endpoint.
Returns {} on failure (callers degrade gracefully to bare t.co links).
TweetPreview.jsx:
After tweet data loads, per-tweet range-based coverage detection:
builds covered [start,end) from existing entity fromIndex/toIndex or
indices fallback, then finds regex matches whose span is NOT covered.
Resolves unique uncovered t.co URLs via resolveTcoUrls(), synthesises
one entity per occurrence (with exact fromIndex/toIndex so normalizeUrlAnn
gets correct bounds even for duplicate t.co URLs in the same tweet).
Augments entities.urls before setTweets() so all rendering paths
see expanded URLs.
* feat: suppress rendered media attachment URLs from tweet text
renderTweetTextJSX now accepts skipSpans=[] as third param.
Skip-span boundaries are added to the pts split set so a trailing
media t.co inside a plain segment still gets isolated and suppressed—
not re-linked by linkifyText. Early return only when both anns and
skipSpans are empty.
TweetCard computes mediaSkipSpans after building photos/videoItems:
for each rawMedia item whose src resolved (photo src match; any video),
resolveEntityBounds(m, ft, m.url) gives the precise [s,e] span using
indices/fromIndex first, indexOf fallback—then the span is passed to
renderTweetTextJSX so the t.co attachment URL is silently dropped.
This commit is contained in:
GitHub
parent
2779afee2d
commit
d610d37793
21 changed files with 2792 additions and 92 deletions
|
|
@ -21,6 +21,7 @@ serde_json.workspace = true
|
|||
rusqlite.workspace = true
|
||||
parking_lot.workspace = true
|
||||
regex.workspace = true
|
||||
reqwest.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile.workspace = true
|
||||
|
|
|
|||
|
|
@ -91,36 +91,61 @@ async fn setup_guard(
|
|||
/// Tower middleware: injects HTTP security response headers on every response.
|
||||
/// HSTS is intentionally omitted — that belongs at the reverse-proxy layer.
|
||||
async fn security_headers(req: Request, next: Next) -> Response {
|
||||
// Capture path before consuming req for next.run()
|
||||
let is_artifact = req.uri().path().contains("/artifacts/");
|
||||
let mut response = next.run(req).await;
|
||||
let headers = response.headers_mut();
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("x-content-type-options"),
|
||||
axum::http::HeaderValue::from_static("nosniff"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("x-frame-options"),
|
||||
axum::http::HeaderValue::from_static("DENY"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("referrer-policy"),
|
||||
axum::http::HeaderValue::from_static("strict-origin-when-cross-origin"),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("content-security-policy"),
|
||||
axum::http::HeaderValue::from_static(
|
||||
"default-src 'self'; \
|
||||
script-src 'self'; \
|
||||
style-src 'self' 'unsafe-inline'; \
|
||||
img-src 'self' data: blob:; \
|
||||
font-src 'self'; \
|
||||
connect-src 'self'; \
|
||||
frame-ancestors 'none'",
|
||||
),
|
||||
);
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("permissions-policy"),
|
||||
axum::http::HeaderValue::from_static("camera=(), microphone=(), geolocation=()"),
|
||||
axum::http::HeaderValue::from_static("camera=(), microphone=(), geolocation=(), autoplay=()"),
|
||||
);
|
||||
if is_artifact {
|
||||
// Artifact responses are iframed by the preview modal (sandboxed, no allow-scripts).
|
||||
// When opened directly in a new tab scripts must still be blocked so archived
|
||||
// pages cannot make same-origin API calls with the user's session.
|
||||
// Only styles, images, fonts and media need to be relaxed for rendering.
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("content-security-policy"),
|
||||
axum::http::HeaderValue::from_static(
|
||||
"default-src 'none'; \
|
||||
script-src 'none'; \
|
||||
style-src 'self' 'unsafe-inline' https:; \
|
||||
img-src 'self' data: blob: https:; \
|
||||
font-src 'self' https:; \
|
||||
media-src 'self' blob:; \
|
||||
connect-src 'none'; \
|
||||
frame-ancestors 'self'",
|
||||
),
|
||||
);
|
||||
} else {
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("x-frame-options"),
|
||||
axum::http::HeaderValue::from_static("DENY"),
|
||||
);
|
||||
// Main app CSP — allow Google Fonts and external images for tweet previews
|
||||
headers.insert(
|
||||
axum::http::header::HeaderName::from_static("content-security-policy"),
|
||||
axum::http::HeaderValue::from_static(
|
||||
"default-src 'self'; \
|
||||
script-src 'self'; \
|
||||
style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; \
|
||||
img-src 'self' data: blob: https:; \
|
||||
font-src 'self' https://fonts.gstatic.com; \
|
||||
media-src 'self' blob: https:; \
|
||||
connect-src 'self'; \
|
||||
frame-src 'self'; \
|
||||
frame-ancestors 'none'",
|
||||
),
|
||||
);
|
||||
}
|
||||
response
|
||||
}
|
||||
|
||||
|
|
@ -303,6 +328,7 @@ pub fn app_with_state(state: AppState) -> Router {
|
|||
"/api/archives/:archive_id/blob-cleanup",
|
||||
get(blob_cleanup_scan_handler).delete(blob_cleanup_delete_handler),
|
||||
)
|
||||
.route("/api/util/resolve-tco", post(resolve_tco_handler))
|
||||
.nest_service("/assets", ServeDir::new(static_dir.join("assets")))
|
||||
.fallback_service(ServeFile::new(static_dir.join("index.html")))
|
||||
.layer(axum::middleware::from_fn_with_state(state.clone(), setup_guard))
|
||||
|
|
@ -1883,6 +1909,74 @@ async fn delete_collection_handler(
|
|||
}
|
||||
}
|
||||
|
||||
// ── t.co resolver ──────────────────────────────────────────────────────────────
|
||||
// POST /api/util/resolve-tco
|
||||
// Body: JSON array of t.co short URLs (max 50).
|
||||
// Returns: JSON object mapping each input URL to its expanded destination.
|
||||
//
|
||||
// Security:
|
||||
// - Input restricted to https://t.co/<alphanumeric token> only (no SSRF via input).
|
||||
// - redirect(Policy::none()): makes ONE HEAD to t.co, reads Location header, never
|
||||
// fetches the expanded destination (no open-proxy).
|
||||
// - 3 s timeout, 1-hop max.
|
||||
// - No auth required (t.co is public; no data exposed).
|
||||
|
||||
static TCO_RE: std::sync::LazyLock<regex::Regex> = std::sync::LazyLock::new(|| {
|
||||
regex::Regex::new(r"^https://t\.co/[A-Za-z0-9]+$").unwrap()
|
||||
});
|
||||
|
||||
async fn resolve_tco_handler(
|
||||
Json(urls): Json<Vec<String>>,
|
||||
) -> Result<Json<std::collections::HashMap<String, String>>, ApiError> {
|
||||
const MAX_BATCH: usize = 50;
|
||||
let urls: Vec<String> = urls
|
||||
.into_iter()
|
||||
.filter(|u| TCO_RE.is_match(u))
|
||||
.take(MAX_BATCH)
|
||||
.collect();
|
||||
|
||||
let client = reqwest::Client::builder()
|
||||
.redirect(reqwest::redirect::Policy::none())
|
||||
.timeout(Duration::from_secs(3))
|
||||
.build()
|
||||
.map_err(|e| ApiError::internal(&format!("http client: {e}")))?;
|
||||
|
||||
let mut map = std::collections::HashMap::new();
|
||||
let futs: Vec<_> = urls.iter().map(|url| {
|
||||
let client = client.clone();
|
||||
let url = url.clone();
|
||||
tokio::spawn(async move {
|
||||
// Try HEAD first; fall back to GET if HEAD returns no Location.
|
||||
// Neither follows redirects (Policy::none), so the server only
|
||||
// ever connects to t.co itself — never to the destination.
|
||||
// Only accept http/https destinations — never javascript:, data:, etc.
|
||||
let safe_location = |resp: reqwest::Response| {
|
||||
resp.headers()
|
||||
.get(reqwest::header::LOCATION)
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string())
|
||||
.filter(|s| s.starts_with("http://") || s.starts_with("https://"))
|
||||
};
|
||||
let expanded = match client.head(&url).send().await.ok().and_then(safe_location) {
|
||||
Some(loc) => loc,
|
||||
None => match client.get(&url).send().await.ok().and_then(safe_location) {
|
||||
Some(loc) => loc,
|
||||
None => url.clone(),
|
||||
},
|
||||
};
|
||||
(url, expanded)
|
||||
})
|
||||
}).collect();
|
||||
|
||||
for fut in futs {
|
||||
if let Ok((k, v)) = fut.await {
|
||||
map.insert(k, v);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(Json(map))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
47
crates/archivr-server/static/assets/index-Gm5awz10.js
Normal file
47
crates/archivr-server/static/assets/index-Gm5awz10.js
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -4,8 +4,8 @@
|
|||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Archivr</title>
|
||||
<script type="module" crossorigin src="/assets/index-DiHAMIVH.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-DwI288Sc.css">
|
||||
<script type="module" crossorigin src="/assets/index-Gm5awz10.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-C89vW3ol.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue