mirror of
https://github.com/thegeneralist01/archivr
synced 2026-07-21 18:55:36 +02:00
TheGeneralist
2414acf0df
* chore: add Dockerfile, docker-compose, and Docker docs - Multi-stage Dockerfile: Rust builder stage + debian:bookworm-slim runtime with Chromium, Node/single-file-cli, Python venv (yt-dlp + twitter-api-client) - docker-compose.yml: wires ARCHIVR_BIND, config volume, and persistent data volume - docker/config.example.toml: annotated TOML template for Docker deployments - docs/README.md: add Hosting with Docker section; add ARCHIVR_BIND and ARCHIVR_STATIC_DIR to the Environment Variables reference * fix: address code review issues with Docker setup - .gitignore: whitelist Dockerfile, docker-compose.yml, docker/ so they are actually tracked (the * catch-all was silently dropping them) - Dockerfile: build and ship the archivr CLI alongside archivr-server so users can run `archivr init` inside the container on first setup - docker/config.example.toml: fix archive_path to point at the .archivr subdirectory that archivr init creates (not the parent directory), which is what read_archive_paths expects - docs/README.md: replace the bare mkdir quickstart step with `archivr init`, explain why mkdir is insufficient; add a callout that auth_db_path must be set explicitly to a writable path when the config mount is read-only * fix: address second round of Docker review issues Chromium sandbox (P2): - singlefile.rs: add ARCHIVR_CHROME_ARGS env var (space-separated flags appended to Chromium's --browser-args JSON array); Dockerfile sets it to --no-sandbox because Chromium refuses to start as root without it Store-path outside volume (P1): - README: pass explicit absolute store-path as the second positional arg to `archivr init` so the blob store lands on /data instead of the container layer (CLI default is ./.archivr/store, resolved from cwd, which is / with no WORKDIR set) ENTRYPOINT vs CMD (P2): - Dockerfile: switch from ENTRYPOINT to CMD so `docker compose run archivr archivr init …` overrides the full command instead of being appended to the server invocation ffmpeg missing (P2): - Dockerfile: add ffmpeg to the apt-get install block (required by yt-dlp --merge-output-format mp4 for bestvideo+bestaudio streams) Node version (P2): - Dockerfile: replace Debian bookworm's nodejs (18.x) with Node 20 via the NodeSource setup script (single-file-cli declares engines.node >=20) Build context secrets (P2): - Add .dockerignore excluding config/ and docker/ from the build context so runtime secrets (e.g. twitter-cookies.txt) are never sent to the builder - Whitelist .dockerignore in .gitignore docs: - README: document ARCHIVR_CHROME_ARGS in the Environment Variables section * fix: third round of Docker review issues Rust toolchain (P1): - Dockerfile: bump builder from rust:1.87 to rust:1.88; time@0.3.51, time-core@0.1.9, and time-macros@0.2.30 (present in Cargo.lock) all require MSRV 1.88, so the real cargo build --release step was failing single-file-cli wait mode (P2): - singlefile.rs: replace --browser-wait-until=networkidle2 with networkAlmostIdle; the single-file-cli option only accepts InteractiveTime/networkIdle/networkAlmostIdle/load/domContentLoaded (verified in options.js); networkidle2 is a Puppeteer concept that the CLI does not recognise, causing silent fallback to the earliest state and incomplete captures. networkAlmostIdle is the closest equivalent (<=2 open connections, matching Puppeteer's networkidle2 semantics) Build context size (P3): - .dockerignore: add target/, frontend/node_modules/, frontend/dist/; these can reach 1.4G+ after a local dev build and are never read by the Dockerfile, so sending them to the builder wastes time and memory
106 lines
4.4 KiB
Docker
106 lines
4.4 KiB
Docker
# syntax=docker/dockerfile:1
|
||
|
||
###############################################################################
|
||
# Stage 1 – Build the Rust server and CLI binaries
|
||
###############################################################################
|
||
FROM rust:1.88-slim-bookworm AS builder
|
||
|
||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||
pkg-config \
|
||
libssl-dev \
|
||
&& rm -rf /var/lib/apt/lists/*
|
||
|
||
WORKDIR /build
|
||
|
||
# Layer the dependency build separately for better cache reuse.
|
||
# Stub out every crate so Cargo can resolve and compile all dependencies
|
||
# before we copy the real source.
|
||
COPY Cargo.toml Cargo.lock ./
|
||
COPY crates/archivr-core/Cargo.toml crates/archivr-core/Cargo.toml
|
||
COPY crates/archivr-server/Cargo.toml crates/archivr-server/Cargo.toml
|
||
COPY crates/archivr-cli/Cargo.toml crates/archivr-cli/Cargo.toml
|
||
|
||
RUN mkdir -p \
|
||
crates/archivr-core/src \
|
||
crates/archivr-server/src \
|
||
crates/archivr-cli/src \
|
||
&& touch crates/archivr-core/src/lib.rs \
|
||
&& echo 'fn main() {}' > crates/archivr-server/src/main.rs \
|
||
&& echo 'fn main() {}' > crates/archivr-cli/src/main.rs \
|
||
&& cargo build --release -p archivr-server -p archivr-cli || true
|
||
|
||
# Build the real binaries; touch source files to force Cargo to relink.
|
||
COPY crates/ crates/
|
||
RUN touch \
|
||
crates/archivr-core/src/lib.rs \
|
||
crates/archivr-server/src/main.rs \
|
||
crates/archivr-cli/src/main.rs \
|
||
&& cargo build --release -p archivr-server -p archivr-cli
|
||
|
||
###############################################################################
|
||
# Stage 2 – Runtime image
|
||
###############################################################################
|
||
FROM debian:bookworm-slim
|
||
|
||
# Runtime dependencies:
|
||
# chromium used by single-file-cli for full-page archiving
|
||
# nodejs (20+) runtime for single-file-cli (requires Node >=20; Debian
|
||
# bookworm ships 18, so we install from the NodeSource repo)
|
||
# ffmpeg required by yt-dlp to merge separate audio/video streams
|
||
# (e.g. YouTube bestvideo+bestaudio format selection)
|
||
# python3 + pip + venv twitter scraper
|
||
# ca-certificates outbound HTTPS from the server and NodeSource HTTPS
|
||
# libssl3 OpenSSL linked by the Rust binary
|
||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||
curl \
|
||
ca-certificates \
|
||
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
|
||
&& apt-get install -y --no-install-recommends \
|
||
chromium \
|
||
nodejs \
|
||
ffmpeg \
|
||
python3 \
|
||
python3-pip \
|
||
python3-venv \
|
||
libssl3 \
|
||
&& rm -rf /var/lib/apt/lists/*
|
||
|
||
# Install single-file-cli globally so `single-file` is on PATH.
|
||
RUN npm install -g single-file-cli
|
||
|
||
# Install yt-dlp and twitter-api-client into an isolated venv to avoid
|
||
# conflicts with Debian's system Python packages.
|
||
RUN python3 -m venv /opt/archivr-venv \
|
||
&& /opt/archivr-venv/bin/pip install --no-cache-dir \
|
||
yt-dlp \
|
||
twitter-api-client
|
||
|
||
# Server and CLI binaries (CLI is needed to run `archivr init` on first setup)
|
||
COPY --from=builder /build/target/release/archivr-server /usr/local/bin/archivr-server
|
||
COPY --from=builder /build/target/release/archivr /usr/local/bin/archivr
|
||
|
||
# Pre-built frontend assets (already compiled; no Vite build step needed)
|
||
COPY crates/archivr-server/static/ /usr/share/archivr-server/static/
|
||
|
||
# Twitter scraper script
|
||
COPY vendor/twitter/scrape_user_tweet_contents.py \
|
||
/usr/local/lib/archivr/scrape_user_tweet_contents.py
|
||
|
||
# Wire up env vars that the server (and archivr-core) read at runtime.
|
||
# ARCHIVR_BIND and ARCHIVR_TWITTER_CREDENTIALS_FILE are intentionally left
|
||
# unset here — set them in docker-compose.yml or at `docker run` time.
|
||
ENV ARCHIVR_STATIC_DIR=/usr/share/archivr-server/static \
|
||
ARCHIVR_CHROME=/usr/bin/chromium \
|
||
ARCHIVR_SINGLE_FILE=/usr/local/bin/single-file \
|
||
ARCHIVR_TWEET_PYTHON=/opt/archivr-venv/bin/python3 \
|
||
ARCHIVR_TWEET_SCRAPER=/usr/local/lib/archivr/scrape_user_tweet_contents.py \
|
||
ARCHIVR_YT_DLP=/opt/archivr-venv/bin/yt-dlp \
|
||
ARCHIVR_CHROME_ARGS=--no-sandbox
|
||
|
||
EXPOSE 8080
|
||
|
||
# Expects the TOML config at /config/archivr-server.toml (mount a volume).
|
||
# Copy docker/config.example.toml as a starting point.
|
||
# Using CMD (not ENTRYPOINT) so `docker compose run archivr archivr init …`
|
||
# can override the whole command for first-time archive initialisation.
|
||
CMD ["archivr-server", "/config/archivr-server.toml"]
|