`update` fetches the latest release tag from the GitHub API (or takes
--version), downloads the cross-platform python zipapp, and installs it
into archivr's state dir. The install is atomic — staged as yt-dlp.new,
chmod +x'd, then renamed over the target — so a concurrently running
capture never sees a half-written binary. A sibling .version file makes a
repeat update a no-op instead of a 3MB re-download.
The download is checked for the python3 shebang before install, which
catches the usual failure mode of getting an HTML error page back. python3
itself is only warned about, not required: the server may run under a nix
wrapper with its own PATH.
`status` prints all three candidates (env / state-dir / PATH fallback) with
their versions and stars whichever the resolver picks, so it is obvious
which yt-dlp a capture will actually use.
reqwest is pulled from the existing workspace dependency; the GitHub JSON is
parsed with serde_json so the "json" feature is not needed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The nix flake wrapper pins a yt-dlp via ARCHIVR_YT_DLP, but yt-dlp rots
fast — extractors break within weeks of a pin. Add a resolver that probes
`--version` on both the pinned binary and a user-installed copy under the
mutable state dir, and runs whichever is newer.
Version strings are YYYY.MM.DD, so plain string ordering is chronological.
Ties resolve toward the state dir: a user who installed it there did so
deliberately. ARCHIVR_YT_DLP_FORCE bypasses the comparison entirely, and
with no candidate at all we fall back to bare `yt-dlp` on PATH — exactly
the previous behaviour.
Resolution is cached in a OnceLock so `--version` costs one subprocess per
process, and all four inline env::var lookups now go through it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Text captures land as `.md` (Markdown) or `.txt` (plain) blobs, but
PreviewPanel only dispatched on video/audio/image/pdf/html extensions,
so opening a text entry hit the 'No preview available' fallback with
the raw artifact path exposed.
- New `TextPreview` component fetches the primary artifact as text,
renders it in a monospace `<pre>` with word-wrap, and shows the
entry title on top and the MIME as a small trailing tag. Handles
loading/error states.
- `PreviewPanel` gains a `TEXT_EXTS` set + a branch that dispatches
to `TextPreview` for `md` / `markdown` / `txt`.
- CSS is padded and centered to ~780px so a text note reads like a
document rather than an edge-to-edge terminal dump.
v1 intentionally does NOT parse Markdown: keeping frontend deps at
react+react-dom only. Bump to a real Markdown renderer if we start
capturing Markdown-authored notes.
Tweet and tweet_thread entries store their payload under artifact_role
`raw_tweet_json`, not `primary_media`. `build_summary_input` filtered
strictly for `primary_media LIMIT 1`, so both cases silently failed
with 'entry X has no primary_media artifact to summarize'.
Threads compound the problem: the tweet scraper writes ONE json file
per status, so even a fixed lookup that took the first row would
summarize only the initial tweet and lose the rest of the conversation.
Fixes:
- New `load_summary_artifacts` helper returns every artifact for a
role in insertion order.
- For entity_kind `tweet` / `tweet_thread`, load all
`raw_tweet_json` artifacts (falling back to `primary_media` for
archives predating that role convention).
- Iterate artifacts, extract text per file with the existing
markdown/html/json branches, then join thread pieces with a
`---` separator so the model sees a real paragraph break between
statuses instead of one flowing document.
Single-tweet entries produce one piece and the separator never
renders. Non-tweet entries behave exactly as before.
The text row shipped with semantic classnames (`capture-text-inputs`,
`capture-text-title`, `capture-text-body`, `capture-text-mime`,
`capture-text-icon`) but no CSS rules. Falling through to the parent
`.capture-row-main` flex-row (`display: flex; align-items: center`)
meant the title, textarea, and mime-select stacked as intrinsic-width
boxes centered on the tall body, producing a layout where the body
floated to the top-right, the title box appeared BELOW it, and the mime
selector rendered as an unstyled OS dropdown.
Fix:
- `.capture-text-row .capture-row-main` uses `align-items: flex-start`
so the leading icon and trailing × pin to the top of the block.
- `.capture-text-inputs` is now a full-width column-flex container with
proper gaps.
- `.capture-text-title` reuses the 44px input height and typography of
`.capture-input`; `.capture-text-body` gets a 140px min-height,
vertical resize, and matching border/focus treatment.
- `.capture-text-mime` is styled as a small chip with a custom caret
so it matches `.capture-quality` and stops looking like a raw
`<select>`. Sits in a right-aligned footer under the body.
- `.capture-text-icon` gets a 44px column so it aligns with the title
input; remove button gets a small top-margin for the same reason.
Rebuilt static bundle bumped as well (`index-BLxoi9rt.css`,
`index-CQcpPA_I.js`).
Two related fixes for the codex_cli summary provider:
1. Executable discovery. `ARCHIVR_CODEX_CLI` was already respected, but
without it the code resolved to bare `codex` and relied on PATH.
The ChatGPT desktop app installs codex at
`/Applications/ChatGPT.app/Contents/Resources/codex` and does not
put it on PATH, so users who only have the desktop app saw
'No such file or directory' with no hint. `resolve_cli` now walks
env override → a small set of well-known absolute paths → HOME
/.local/bin/<bare> → bare fallback. Same treatment applied to
claude_cli for symmetry (/opt/homebrew/bin/claude, /usr/local/bin/
claude, HOME/.local/bin/claude).
2. Clean output. `codex exec -` writes a runtime header ("OpenAI
Codex vX", session id, sandbox, model), the assistant reply, and a
footer ("tokens used", replay of the reply) to stdout. The JSON
extractor took the first '{' from the *user prompt echo* and the
last '}' from the trailing replay, producing invalid text that
fell through to the "raw text under summary" fallback path. Now
uses `--output-last-message <tempfile>` and reads only the final
assistant message. Fallback (positional prompt) uses the same
flag. Tempfile is cleaned up on all paths, incl. spawn failure.
New "Summary" rail section between the URL/Preview controls and .meta-list.
A completed summary renders as bold tl;dr, body paragraph, tag chips, and a
provider · model footer; missing or failed shows Generate; pending/running
shows an inline spinner and polls GET every 1500 ms until terminal.
- api.js: fetchEntrySummary + requestEntrySummary. The POST helper unwraps
ApiError's { "error": ... } body so the missing-env-var message reaches
the user verbatim rather than as a bare status code.
- ContextRail.jsx: state seeds from detail.latest_summary so the section
renders immediately on selection. Polling is anchored on the summary
status rather than started inside the click handler, so a job still
running when the user navigates away and back is picked up again. A
transient poll failure is swallowed — the next tick retries, and a real
failure arrives as status === 'failed'.
- Regenerate passes force:true only when a completed summary is already
shown; otherwise the request can take the server's 200 cache-hit path.
- Provider choice persists in sessionStorage under archivr:summary:provider,
with try/catch around both accessors for private-mode browsers.
- Public sessions never see the selector or the Generate button, and the
section renders at all only when a completed summary made it through the
server's visibility gate.
- styles.css: .rail-summary-* only; spacing and the action button reuse
.rail-section and .rail-rearchive-btn. The spinner honours
prefers-reduced-motion — the text alone conveys the state.
- AGENTS.md: document the summary env vars alongside the existing
external-tool convention.
Smoke-tested end to end against a scratch archive with a seeded markdown
entry: claude_cli produced a real summary (pending → running → completed in
~11s); a local mock server exercised the openai_compatible transport and
confirmed the Bearer header, model, and system/user role split on the wire;
unconfigured providers return 400 naming the exact variable; a video entry
returns 400 "v1 unsupported"; a repeat POST returns 200 from cache without
adding a row.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GET is read-only and gated exactly like entry detail, so a guest can read a
summary only for an entry whose content they could already read. POST
requires ROLE_USER, matching capture / tags / patch / rearchive; no auth
roles change.
Both the provider config and the content extraction resolve on the request
thread, before spawn_blocking. That is what lets a missing env var come back
as a synchronous 400 naming the exact variable, and an unsummarizable
artifact (video, audio) as a 400 saying so, rather than becoming a
background job the caller must poll only to learn about a config typo.
The pending row is claimed before spawning so the 202 can name a summary_uid
the client can poll immediately. summarize_entry owns the
pending → running → completed/failed transitions for that same row — the
cache key is identical, so both upserts resolve to one row — leaving the
handler to catch only the case where it fails before recording anything.
When !force and an identical cache key already completed, the existing row
comes back as a 200 with no new work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per-entry LLM summaries as a regenerable child record, not a column on
archived_entries and not an on-disk artifact: an entry may carry several
summaries (one per provider/model/prompt version), any of which can be
discarded and recomputed. Generation is manual-only — nothing in capture.rs
calls into this module.
- database.rs: entry_summaries table + index, EntrySummaryRecord, and
upsert/update/find/latest helpers mirroring the capture_jobs style.
provider_model is stored as '' rather than NULL because SQLite treats
NULLs as distinct inside a UNIQUE index, which would stop the CLI
providers (no model) from ever deduping on the cache key.
- summarizer.rs: SummaryProvider trait with four implementations —
Anthropic Messages API, OpenAI-compatible chat completions, `claude -p`
and `codex exec -`. Configuration comes from env vars only (never TOML),
matching how yt-dlp / single-file / tweet-scraper are resolved, which
also keeps API keys out of anything the archive persists.
- archive.rs: EntryDetail gains latest_summary, populated by one extra
LIMIT 1 query in get_entry_detail. EntrySummaryView aliases the DB row
rather than duplicating it.
Implementation notes:
- No tokio in core. CLI timeouts are enforced structurally: stdout is
drained on its own thread and handed back over a channel so the calling
thread can recv_timeout and kill an overrunning child; stdin is written
on a third thread so a 48 KB prompt cannot deadlock against a child
waiting for us to read.
- HTML is reduced with regex rather than a parser: html5ever is not in the
tree, and a model tolerates imperfect whitespace. Paired tags are spelled
out per tag because Rust's regex engine has no backreferences by design.
- reqwest is declared with only the `blocking` feature here, so bodies are
serialized via .body(value.to_string()) instead of widening the
workspace dependency for .json().
- input_sha256 holds a SHA3-256 digest via hash::hash_bytes, the tree's one
hashing primitive; the content is truncated to 48 KB *before* hashing so
the cache key describes exactly the bytes the model saw.
Tests: no mockito/wiremock in dev-deps, and adding a mock HTTP server for
one JSON shape is a poor trade, so the two halves that can actually break
are tested directly — request-body builders and response parsers — leaving
only reqwest's own transport uncovered. Plus schema idempotency, cache-key
dedupe, cascade-on-delete, provider_from_env happy/missing-var paths, HTML
and tweet extraction, output normalization, and the CLI runner's stdin
round-trip, timeout kill, and nonzero-exit paths.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Add submitTextCapture API client function with same error handling as submitCapture
- Create makeTextItem() factory for text capture state
- Implement CaptureTextRow component with title, body textarea, and MIME selector
- Add 'Add text' button in capture dialog toolbar
- Update handleArchive to filter and route text submissions
- Modify submitBgJob to detect and submit text items via submitTextCapture
- Skip probe and conflict checks for text items
- Reuse job tracking and batch settlement for text captures
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Add CaptureTextBody struct for title, body, and optional MIME type
- Implement capture_text_handler with validation for empty fields and MIME type
- Route text submissions to perform_text_capture() in background
- Reuse existing capture job tracking and polling infrastructure
- Default MIME type to text/markdown when not specified
- Include route tests covering happy path, validation, auth, and error cases
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Add downloader/text.rs module with save() function that stages and hashes text content
- Support text/markdown and text/plain MIME types with .md and .txt extensions
- Add perform_text_capture() function for capturing user-supplied text
- Validates title (non-empty, max 500 chars) and body (non-empty, max 2 MiB)
- Creates blob records and entries with source_kind='text', entity_kind='document'
- Includes comprehensive unit tests for markdown, plain text, and validation
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
list_entries_for_collection was missing AND e.parent_entry_id IS NULL,
causing child entries (playlist/channel videos) to appear alongside
their parent in the flat archive view. list_root_entries and
search_entries already had this guard; the collection path did not.
Also removes garnix from flake.nix nixConfig.
Previously, any WebPage capture with via_freedium=true was routed
through the Freedium mirror regardless of the URL's host. This caused
non-paywall sites like borretti.me to be fetched through Freedium,
which is wrong — Freedium only knows how to handle a specific set of
publications.
Add is_freedium_supported_url() with a static allowlist of the 7 hosts
Freedium explicitly supports per its homepage announcement:
Medium, NYT, WaPo, Bloomberg, Reuters, Economist, Financial Times
The gate matches on the bare domain or any subdomain (e.g.
towardsdatascience.medium.com). Unparseable URLs fall through to a
direct fetch. The existing freedium-mirror.cfd re-wrap guard is kept
as a belt-and-suspenders check after the new allowlist test.
Fixes: https://borretti.me/article/notes-on-managing-adhd archived via
Freedium despite not being a Medium article.
* Add Archivr brand identity and favicon
- Brand style guide (docs/branding/style-guide.html): logo, clear
space, don'ts, color palette, typography, in-context mockup
- Logo assets (docs/branding/assets/): lockup SVGs for dark/light
backgrounds, standalone mark, favicon.svg, multi-size favicon.ico
- Wire favicon into frontend/index.html (SVG + ICO fallback);
frontend/public/ seeded so Vite copies assets on next build
- Revert display strings archivr back to Archivr in UI
(Topbar, LoginPage, SetupPage, index.html titles, flake.nix)
- Update .gitignore to allow docs/branding/
* Build frontend with favicon links and assets
* Fix static file serving: serve all of static/ with SPA index.html fallback
Previously only /assets/* was served from disk; all other paths
(including /favicon.ico, /favicon.svg) hit the index.html fallback.
Replace nest_service("/assets") + fallback_service(index.html) with
ServeDir on the full static dir and not_found_service(index.html),
so any file that exists in static/ is served directly.
* feat(core): add requires_auth to collections; include name in entry-collection memberships
- Add `requires_auth INTEGER NOT NULL DEFAULT 1` column to the
collections DDL and as an idempotent ALTER TABLE migration in
initialize_schema (archive DB), not initialize_auth_schema.
- CollectionRecord and CollectionSummary gain `requires_auth: bool`.
- create_collection() and update_collection() accept the new field.
- get_entry_collection_memberships() now returns collection name as the
third tuple element; EntryCollectionMembership gains a `name` field
so the sidebar can show human-readable names instead of raw UIDs.
* feat(server): conditional auth for public collections; add requires_auth + original_url to API
- CreateCollectionBody gains requires_auth (default true).
- PatchCollectionBody gains requires_auth: Option<bool>.
- get_collection_handler: load record first, then skip auth.require_auth()
when record.requires_auth == false so public collections are accessible
to unauthenticated callers; caller_bits falls back to ROLE_GUEST (1)
so only visibility_bits=3 entries are returned to guests.
- Collection JSON response includes requires_auth and each entry now
includes original_url for use by the public collection page.
- list_collections_handler keeps require_auth (management UI).
* feat(frontend): public collection page at /c/:archiveId/:collUid
- Detect PUBLIC_COLL_ROUTE at module load time (like PREVIEW_ROUTE) and
return <PublicCollectionPage> before any auth checks so unauthenticated
users can view public collections without hitting the login gate.
- PublicCollectionPage fetches via getCollection() and renders the
server-filtered entry list (no client-side bitmask filtering - the
server already applies caller_bits=GUEST for unauthenticated requests).
Entry titles link to original_url when present; fall back to plain
text when original_url is null.
- api.js createCollection() gains requiresAuth param (default true),
sent as requires_auth in the request body.
- Storybook story covers WithEntries, Empty, and LoadError states.
* feat(frontend): collections view improvements
- addVis in the 'Add entry' form now syncs to the selected collection's
default_visibility_bits via useEffect on collDetail, so the default
matches the collection's configured entry visibility.
- Rename 'Default visibility' label to 'Entries\' default visibility'
in both the detail pane and the create form to distinguish it from
the new collection-level access setting.
- Add 'Require authentication to view' checkbox in the detail pane
backed by a PATCH to requires_auth; reads collDetail?.requires_auth
with fallback to the list-level selected record.
- Create form gains a matching requires_auth checkbox (default: true),
passed as 5th arg to createCollection().
* feat(frontend): context rail collection improvements
- Show collection name (c.name) instead of raw UID in the sidebar
Collections section; names now come from the updated
EntryCollectionMembership API response.
- Fix horizontal overflow on long collection names: coll-name gains
overflow:hidden + text-overflow:ellipsis + white-space:nowrap +
min-width:0; coll-row gets overflow:hidden.
- Single-entry 'Add to collection' UI: dropdown + button inside the
Collections rail section lets users add the current entry to any
non-default collection without multi-selecting. After add, the
membership list refreshes automatically.
- Collections section now shows even when entryCollections is empty,
as long as non-default collections exist (so the add form is
accessible for un-membered entries).
- Bulk 'Add to collection' now uses the target collection's
default_visibility_bits instead of hardcoded 2 (Users only).
- Both bulk and single-entry dropdowns filter out slug='_default_'
to match the backend rejection in add_entry_to_collection_handler.
- Collections list is now fetched on archiveId change (not just on
bulk mode entry) so it is available for single-entry mode too.
* build(frontend): update static assets
* feat(frontend): public collection link UX + app-styled public page
CollectionsView:
- When a collection has requires_auth=false, show a read-only URL input
and Copy button below the auth checkbox so the public link is
immediately discoverable. The input auto-selects on focus so manual
copy always works. Copy button tries navigator.clipboard.writeText
first; falls back to execCommand('copy') for HTTP deployments where
the Clipboard API is unavailable in non-secure contexts.
PublicCollectionPage:
- Rewritten to use the app's CSS classes and variables instead of
bare inline styles, so it visually matches the main archive UI.
Dark topbar (.pub-coll-topbar) with brand + collection name, paper
background body, entry list via .coll-entries-list / .coll-entry-row /
.coll-entry-info / .coll-entry-kind — the same classes used in the
authenticated Collections view.
styles.css:
- .coll-public-link-row / -wrap / -input / .coll-copy-btn for the
new link field in CollectionsView detail pane.
- .pub-coll-* classes for the public page layout and typography.
* feat(core): add get_collection_by_slug; scope search to active collection
- get_collection_by_slug(): new function mirroring get_collection_by_uid
but matching on slug, used to resolve the _default_ collection when no
?collection param is supplied.
- SearchEntriesQuery gains collection_id: Option<i64>. When set, the
search SQL adds an EXISTS subquery that checks collection_entries cef
for both membership (cef.collection_id = ?) and visibility bits in
that specific collection — preventing cross-collection visibility
leaks where an entry is public in one collection but private in the
current one. Without collection_id the original cross-collection
visibility fallback is kept.
* feat(server): collection-scoped entries/search with uniform auth gate
All entry listing and search now route through the active collection:
list_entries (?collection=<uid>|main|<omitted>):
- Resolves the target collection; omitted or 'main' resolves to _default_.
- Checks requires_auth on that collection; gates auth conditionally.
- Returns list_entries_for_collection() — same EntrySummary shape.
search_entries_handler:
- Same collection resolution + conditional auth as list_entries.
- Sets search_query.collection_id so SQL scopes membership + visibility
to the specific collection, not cross-collection fallback.
list_collections_handler:
- Dropped require_auth() — collection summaries (name/slug/uid/
requires_auth/default_visibility_bits) are public metadata needed for
the guest collection-switcher dropdown.
Tests:
- list_collections_requires_auth → list_collections_is_public (200).
- list_entries_requires_auth and search coverage still pass.
* feat(frontend): integrate collection switching into main Archive view
Replaces the standalone /c/:archiveId/:collUid public page with a
unified main-view approach where all collection logic lives at /.
URL param:
- ?collection=<uid> selects a collection; omitted or 'main' = default.
- 'main' is normalized to null in parseLocation() so the dropdown shows
'All entries' and the URL stays clean.
Collection switcher (Topbar):
- Dropdown always visible (guests need it to navigate public collections).
- Non-default collections only (All entries = no param = _default_).
- Guest selecting an auth-required collection calls onSignInClick().
- handleCollectionChange checks both named and _default_ requires_auth
before proceeding, redirecting guests to login if needed.
listCollections fetched for all users (guests too) since the endpoint
is now public; used to populate the switcher without auth.
Public-session mode (authenticated state, no currentUser):
- Auth gate: fetchArchives() + fetchEntries() with collection param;
401 falls through to login, 200 proceeds as guest.
- auth:expired suppressed when !currentUser.
- fetchEntryDetail skipped; ContextRail shows entry summary + sign-in prompt.
- ContextRail selection effect skips tag/collection API calls.
- runs/tags not fetched in guest mode.
- Child row expansion disabled in EntryRow (hasChildren = false).
api.js:
- fetchEntries/searchEntries both thread ?collection=<uid> to server.
Deleted: PublicCollectionPage.jsx, PublicCollectionPage.stories.jsx,
copy-link UI from CollectionsView, pub-coll-*/copy-link CSS.
* build(frontend): update static assets
* feat(core): add is_entry_publicly_accessible; checks entry+parent vs public collections
* feat(server): allow guests to fetch detail/children/artifacts for public entries
* feat(frontend): guest collection dropdown filtering; public entry detail without auth wall
* build(frontend): update static assets
* test(server): public entry detail/artifact/children contract for guests
* feat(server): filter auth-required collections from guest list_collections response
