CTAO Science Portal: static Astro site with git-based CMS (Sveltia + Gitea)

Portal code only; editorial content lives in the ctao/content repo on the
demo machine's Gitea and is overlaid at build time (see README.md and
deploy/README.md). Live demo: https://astro.isl-dev.grid.cyfronet.pl
This commit is contained in:
2026-09-08 13:49:21 +02:00
parent 845c1ccf0d
commit 2817353c46
50 changed files with 11564 additions and 44 deletions
+8
View File
@@ -0,0 +1,8 @@
# Build image for the CTAO portal demo: pinned Node + git.
# Why: the host has node 24 but NO git (and we have no sudo); a 2-line image
# keeps node pinned and independent of host packages (git comes from the
# alpine repo unpinned — alpine drops old package versions, pinning is moot).
# Built ONCE at install, never pulled again at runtime:
# podman build -t localhost/ctao-portal-build:1 -f Containerfile.build .
FROM docker.io/library/node:24-alpine@sha256:a0b9bf06e4e6193cf7a0f58816cc935ff8c2a908f81e6f1a95432d679c54fbfd
RUN apk add --no-cache git
+127
View File
@@ -0,0 +1,127 @@
# Deploy — CTAO portal demo on strapi-experimental.cyfronet
Everything runs rootless as user `strapi`. Footprint on the machine:
ONE directory (`~/ctao-portal-demo/`) + 4 unit files in `~/.config/`.
Uninstall restores the machine exactly (see bottom). No secrets in any file.
Two Gitea repos by design — `ctao/portal` (code, developed by the team) and
`ctao/content` (Markdown + uploads, committed by the CMS). The build overlays
content onto code; a push to EITHER republishes the site.
```
~/ctao-portal-demo/
├── gitea-data/ # Gitea state (repos, SQLite, accounts) ← the ONLY thing worth backing up
├── gitea-config/ # Gitea app.ini (generated from env)
├── repo/ # clone of the code repo (created by first build)
├── content/ # clone of the content repo (created by first build)
├── releases/ # <code>-<content>/ dirs + `current` symlink (what nginx serves)
├── state/ # last-built release id
├── npm-cache/ # npm cache for the build container (created by build.sh)
├── bin/build.sh # copied from deploy/ (source of truth stays in the code repo)
└── config/nginx.conf
```
Host prerequisites: `podman`, `curl`, `jq` (build.sh checks and says which is
missing). Everything else runs inside containers.
| Port | What | Exposed as |
|---|---|---|
| 3000 | Gitea | https://astro-git.isl-dev.grid.cyfronet.pl (ingress vhost) |
| 8080 | portal (nginx, static) | https://astro.isl-dev.grid.cyfronet.pl (ingress vhost) |
## Install (each step reviewed before running; [W] = writes to the machine)
1. **[W]** `loginctl enable-linger $USER` — without lingering every user unit
dies at logout and nothing starts after a reboot. Verify:
`loginctl show-user $USER -p Linger` → `Linger=yes`.
2. **[W]** `mkdir -p ~/ctao-portal-demo/{gitea-data,gitea-config,releases,state,bin,config} ~/.config/containers/systemd ~/.config/systemd/user`
3. **[W]** Copy files from this dir (scp):
- `ctao-demo-gitea.container`, `ctao-demo-web.container` → `~/.config/containers/systemd/`
- `ctao-portal-build.service`, `ctao-portal-build.timer` → `~/.config/systemd/user/`
- `build.sh` → `~/ctao-portal-demo/bin/` (`chmod +x`)
- `nginx.conf` → `~/ctao-portal-demo/config/`
- `Containerfile.build` → anywhere (needed once, for the next step)
4. **[W]** Pull the images at the digests pinned in the unit files (one-time,
needs internet) and build the build image:
`podman pull docker.io/gitea/gitea@sha256:<digest from ctao-demo-gitea.container>`
`podman pull docker.io/library/nginx@sha256:<digest from ctao-demo-web.container>`
`podman build -t localhost/ctao-portal-build:1 -f Containerfile.build .`
(Upgrading later = pick new digests deliberately, update the pins in the
`.container` files / `Containerfile.build`, re-pull, re-build.)
5. **[W]** `systemctl --user daemon-reload && systemctl --user start ctao-demo-gitea`
6. **[W]** Create the Gitea admin — run interactively in a terminal so the
password never lands in a file or shell history:
`podman exec -it ctao-demo-gitea gitea admin user create --admin --username <you> --email <you@…> --random-password`
7. **[W]** In the Gitea UI: create org `ctao` with repos `portal` and
`content` (both public read). Then, from your workstation, push both over
the vhost with a repo-scoped token:
`git push https://<user>:<token>@astro-git.isl-dev.grid.cyfronet.pl/ctao/portal.git main`
`git push https://<user>:<token>@astro-git.isl-dev.grid.cyfronet.pl/ctao/content.git main`
8. **[W]** `systemctl --user enable --now ctao-portal-build.timer` — first run
clones + `npm ci` + builds (minutes); later runs are seconds. Wait until
`journalctl --user -u ctao-portal-build -n 5` shows `published <sha>`
(starting nginx earlier just serves 404s until the first build lands).
9. **[W]** `systemctl --user start ctao-demo-web`
10. **[R]** Verify: `curl -s -o /dev/null -w '%{http_code}' http://localhost:8080/`
and `journalctl --user -u ctao-portal-build -n 20` (shows measured build times).
Editor accounts need WRITE access to `ctao/content` (Gitea → repo →
Collaborators, or a team) — public read alone lets them sign in but every
save fails.
## CMS sign-in (Sveltia ↔ Gitea OAuth)
- Gitea OAuth2 app (PKCE, `confidential_client=false`, no secret) with
redirect `https://astro.isl-dev.grid.cyfronet.pl/admin/`; its client id is
the `app_id` in `public/admin/config.yml`.
- CORS is pinned in `ctao-demo-gitea.container`: `ALLOW_DOMAIN` takes the
FULL portal origin with scheme (a bare hostname silently disables CORS in
Gitea 1.27; there is no `SCHEME` key) and `HEADERS` must include
`Authorization` or authenticated API calls from the browser fail.
- OAuth requires a secure context: the vhosts must stay HTTPS.
- Changing origins later = edit the quadlet env + the OAuth app's redirect
URI + `config.yml`, then `systemctl --user daemon-reload && systemctl --user restart ctao-demo-gitea`.
## Uninstall (leaves only podman's own storage metadata)
```
systemctl --user disable --now ctao-portal-build.timer
systemctl --user stop ctao-demo-web ctao-demo-gitea
rm ~/.config/containers/systemd/ctao-demo-*.container \
~/.config/systemd/user/ctao-portal-build.{service,timer}
systemctl --user daemon-reload
podman rmi localhost/ctao-portal-build:1 docker.io/gitea/gitea:1.27-rootless \
docker.io/library/nginx:stable-alpine docker.io/library/node:24-alpine
rm -rf ~/ctao-portal-demo
loginctl disable-linger $USER # only if nothing else of yours should survive logout
```
## Notes
- A release that fails to build is not retried until either repo gets a new
commit (`state/last-failed` guards against a 10 s rebuild loop). To force a
retry of the SAME release (e.g. after fixing `bin/build.sh` itself):
`rm ~/ctao-portal-demo/state/last-failed`.
- **Only pushes to the machine's Gitea auto-deploy** (code and content repos
alike). Changes to `deploy/*` need a manual
re-copy: `build.sh` → `bin/`, `nginx.conf` → `config/` +
`systemctl --user restart ctao-demo-web`, unit files →
`~/.config/…` + `systemctl --user daemon-reload` (+ restart). This is
deliberate: the build pipeline must not execute host-side code straight
from the content repo.
- Publish latency = poll (≤10 s) + build (measured: astro build 5–7 s on the
2 vCPU VM, whole build.sh 16–21 s; every build's time lands in the journal).
- Internet needed only for: image pulls (install) and `npm ci` when the
lockfile changes. Routine rebuilds are fully offline.
- Memory caps (`MemoryHigh`) keep us polite next to Outline + Strapi;
the build container is capped at 1 GB via `podman run --memory`. Watch the
first `npm ci` + build in the journal — if it OOMs inside its cgroup
(contained, just retries), raise the cap.
- Build isolation: build containers join the Gitea container's network
namespace (`--network=container:ctao-demo-gitea`) — repo/npm code sees
Gitea on localhost:3000 but cannot reach host loopback services. (Rootless
netavark bridges don't work here: no `ip_tables` kernel module, no sudo.)
- Secrets inventory: Gitea admin password (typed interactively, lives only
in Gitea's DB) and repo-scoped push tokens (managed in Gitea) — that's the
complete list. Build/poll/serve use none.
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env bash
# CTAO portal — poll the code AND content repos, rebuild on any change,
# publish atomically. Triggered every 10 s by ctao-portal-build.timer; systemd
# oneshot semantics guarantee runs never overlap. The 99.9% case is two local
# curls and exit 0. No secrets anywhere: both repos are public-read on the
# local Gitea.
#
# Two repositories by design: code (templates/CSS, developed by the team) and
# content (Markdown + uploads, committed by the CMS). The build overlays
# content onto code, so an editor publishing an article and a developer
# shipping CSS never mix histories — either change republishes the site.
set -euo pipefail
# Defaults match the machine; every var is env-overridable so the whole
# pipeline can be tested locally against a sandbox dir + local Gitea.
BASE="${BASE:-$HOME/ctao-portal-demo}"
GITEA_URL="${GITEA_URL:-http://localhost:3000}" # published by ctao-demo-gitea
CODE_REPO="${CODE_REPO:-ctao/portal}" # owner/repo in Gitea
CONTENT_REPO="${CONTENT_REPO:-ctao/content}"
BRANCH="${BRANCH:-main}"
BUILD_IMAGE="${BUILD_IMAGE:-localhost/ctao-portal-build:1}"
# The build joins the Gitea container's network namespace: localhost inside
# the build = Gitea's loopback (port 3000), host loopback stays unreachable.
# (A netavark bridge would be equivalent, but rootless bridges need the
# ip_tables kernel module, absent on the machine — pasta needs nothing.)
BUILD_NETNS="${BUILD_NETNS:-container:ctao-demo-gitea}"
CODE_URL="${CODE_URL:-http://localhost:3000/$CODE_REPO.git}"
CONTENT_URL="${CONTENT_URL:-http://localhost:3000/$CONTENT_REPO.git}"
KEEP="${KEEP:-3}" # previous releases kept besides `current` (rollback targets)
mkdir -p "$BASE/repo" "$BASE/content" "$BASE/releases" "$BASE/state" "$BASE/npm-cache"
# Host prerequisites (everything else runs inside containers). Fail loud —
# a missing tool is permanent, unlike a Gitea hiccup below.
for tool in curl jq podman; do
command -v "$tool" >/dev/null || { echo "missing host tool: $tool"; exit 1; }
done
# --- 1. Cheap poll: both branch heads via the local Gitea API ---
head_of() {
curl -fsS --max-time 5 "$GITEA_URL/api/v1/repos/$1/branches/$BRANCH" \
| jq -r '.commit.id' || true
}
code_sha=$(head_of "$CODE_REPO")
content_sha=$(head_of "$CONTENT_REPO")
# Gitea down/unreachable is a transient, not a unit failure — exit 0 quietly
# instead of painting the journal red every 10 s. Name the repo: a 404 here
# also means "repo/branch missing or renamed", not just "Gitea down".
[[ "$code_sha" =~ ^[0-9a-f]{40}$ ]] || { echo "poll failed for $CODE_REPO@$BRANCH (gitea down, or repo/branch missing) — skipping"; exit 0; }
[[ "$content_sha" =~ ^[0-9a-f]{40}$ ]] || { echo "poll failed for $CONTENT_REPO@$BRANCH (gitea down, or repo/branch missing) — skipping"; exit 0; }
release="${code_sha:0:12}-${content_sha:0:12}" # code+content pin the release
# Skip only if this pair is both recorded AND still present in releases/
# (a deleted release dir must trigger a rebuild, not an eternal skip).
[[ "$release" == "$(cat "$BASE/state/last-built" 2>/dev/null)" \
&& -d "$BASE/releases/$release" ]] && exit 0
# A release that already failed is not retried until either repo moves —
# otherwise one bad commit (e.g. broken frontmatter) turns into a full
# rebuild every 10 s on a shared VM. The failure is loud once, then quiet.
if [[ "$release" == "$(cat "$BASE/state/last-failed" 2>/dev/null)" ]]; then
echo "skipping $release — build failed before; push a fix to retry"
exit 0
fi
echo "building code=$code_sha content=$content_sha"
t0=$(date +%s)
# --- 2. Build in the ephemeral container (git + pinned node live there).
# SECURITY: the container runs npm lifecycle scripts from the repo, so it is
# confined to Gitea's netns — it reaches Gitea on localhost:3000 and the
# internet (for `npm ci` when the lockfile changed), but NOT the host's
# loopback services. Never use --network=host here.
# node_modules and .deps-hash are untracked, so they survive checkouts.
podman run --rm --network="$BUILD_NETNS" --memory=1g \
-e ASTRO_TELEMETRY_DISABLED=1 \
-e CODE_SHA="$code_sha" -e CODE_URL="$CODE_URL" \
-e CONTENT_SHA="$content_sha" -e CONTENT_URL="$CONTENT_URL" \
-e RELEASE="$release" -e BRANCH="$BRANCH" \
-v "$BASE/repo:/work/repo:z" \
-v "$BASE/content:/work/content:z" \
-v "$BASE/releases:/work/releases:z" \
-v "$BASE/npm-cache:/root/.npm:z" \
-w /work "$BUILD_IMAGE" sh -ec '
git config --global safe.directory "/work/repo"
git config --global --add safe.directory "/work/content"
sync_clone() { # $1 dir $2 url $3 sha
[ -d "$1/.git" ] || git clone --branch "$BRANCH" "$2" "$1"
git -C "$1" remote set-url origin "$2" # self-heal if the URL changes
git -C "$1" fetch --quiet origin "$BRANCH"
# --force: the working copy is disposable; a stray tracked-file edit
# must not wedge every future build.
git -C "$1" checkout --quiet --force "$3"
}
sync_clone repo "$CODE_URL" "$CODE_SHA"
sync_clone content "$CONTENT_URL" "$CONTENT_SHA"
# Overlay content onto code (these paths are gitignored in the code repo).
# mkdir -p: checkout prunes the emptied parent dirs, cp needs them back.
rm -rf repo/src/content/news repo/src/content/pages repo/public/uploads
mkdir -p repo/src/content repo/public
cp -a content/news repo/src/content/news
cp -a content/pages repo/src/content/pages
cp -a content/uploads repo/public/uploads
cd repo
lock=$(sha256sum package-lock.json | cut -d" " -f1)
if [ ! -d node_modules ] || [ "$lock" != "$(cat .deps-hash 2>/dev/null)" ]; then
# --ignore-scripts: (1) removes the install-time postinstall vector from
# npm deps (build-time repo code still runs `npm run build` below — the
# netns confinement is the control for that), (2) avoids the esbuild
# ETXTBSY postinstall race in rootless containers. esbuild ships its
# binary as an optional dep, so nothing here needs lifecycle scripts.
npm ci --ignore-scripts --no-audit --no-fund
echo "$lock" > .deps-hash
fi
npm run build
rm -rf "../releases/$RELEASE"
cp -a dist "../releases/$RELEASE"
' || { echo "$release" > "$BASE/state/last-failed"; echo "BUILD FAILED for $release (see above) — will not retry until a new commit"; exit 1; }
rm -f "$BASE/state/last-failed"
# --- 3. Atomic publish: symlink flip via rename(2) — no half-published moment.
# mv -T is GNU (the target host is Rocky); when testing on macOS put a
# coreutils `mv` (gmv) first in PATH.
rm -f "$BASE/releases/".current.* # stale temps from a crash mid-flip
ln -s "$release" "$BASE/releases/.current.$$"
mv -Tf "$BASE/releases/.current.$$" "$BASE/releases/current"
echo "$release" > "$BASE/state/last-built"
# --- 4. Prune old releases. `current`'s target is excluded explicitly —
# mtime ordering makes it newest today, but nothing should depend on that.
# `|| true`: an empty match must not fail the unit after a successful publish
# (grep exits 1 under pipefail when there is nothing to prune).
cd "$BASE/releases"
cur=$(readlink current || true)
ls -1t | grep -vx current | grep -vx -- "$cur" | tail -n +"$((KEEP + 1))" | while read -r old; do
rm -rf -- "$old"
done || true
echo "published $release in $(( $(date +%s) - t0 ))s"
+62
View File
@@ -0,0 +1,62 @@
# CTAO portal demo — Gitea (CMS backend: content repo, editor accounts, OAuth for Sveltia)
# Quadlet unit. Install: copy to ~/.config/containers/systemd/ on the machine,
# then `systemctl --user daemon-reload` → service name: ctao-demo-gitea.service.
# Rootless image + SQLite by design: one container, no DB server, all state
# lives in ~/ctao-portal-demo/gitea-{data,config} — nothing else on the machine.
[Unit]
Description=CTAO portal demo — Gitea
Wants=network-online.target
After=network-online.target
[Container]
ContainerName=ctao-demo-gitea
# Pinned by digest at install (2026-07-28); tag kept for readability.
Image=docker.io/gitea/gitea:1.27-rootless@sha256:36cce26be71609091e1236d5b5de2c66a81fb8a7d45756a5fd3b7a28c11733b7
# The rootless image runs as uid 1000 inside; keep-id maps it to the host user
# so the bind-mounted dirs stay owned by `strapi` (no chown, no root anywhere).
UserNS=keep-id:uid=1000,gid=1000
# Networking: default rootless pasta (userspace — the machine's kernel lacks
# ip_tables for rootless netavark bridges, and we have no sudo). The build
# container joins THIS container's netns (--network=container:ctao-demo-gitea),
# so it sees Gitea on localhost:3000 while the HOST loopback stays invisible.
Volume=%h/ctao-portal-demo/gitea-data:/var/lib/gitea:Z
Volume=%h/ctao-portal-demo/gitea-config:/etc/gitea:Z
# Bound on all interfaces DELIBERATELY: the ingress that terminates the
# public vhost runs on a separate box and reaches this VM over the network —
# a 127.0.0.1 bind would cut it off.
PublishPort=3000:3000
# Env-driven config — re-applied on every start, no hand-edited app.ini
# (see .skills/gitea/SKILL.md). Secrets: none here; the admin account is
# created interactively after first start (README).
Environment=GITEA__server__HTTP_PORT=3000
# Public URL via the Cyfronet ingress vhost.
Environment=GITEA__server__ROOT_URL=https://astro-git.isl-dev.grid.cyfronet.pl/
Environment=GITEA__server__DISABLE_SSH=true
Environment=GITEA__database__DB_TYPE=sqlite3
Environment=GITEA__security__INSTALL_LOCK=true
Environment=GITEA__service__DISABLE_REGISTRATION=true
Environment=GITEA__mailer__ENABLED=false
# Repos default to public so the build pipeline can clone anonymously —
# content is the public site anyway. The `ctao/portal` repo itself is
# created in the UI (README step 7).
Environment=GITEA__repository__DEFAULT_PRIVATE=public
# Sveltia is served from the portal vhost and calls the Gitea API cross-origin
# — CORS locked to exactly that origin. ALLOW_DOMAIN takes FULL origins with
# scheme (verified in the 1.27 config cheat sheet; a SCHEME key no longer
# exists). Authorization must be listed in HEADERS — the default set
# (Content-Type,User-Agent) would block Sveltia's authenticated API calls.
Environment=GITEA__cors__ENABLED=true
Environment=GITEA__cors__ALLOW_DOMAIN=https://astro.isl-dev.grid.cyfronet.pl
Environment=GITEA__cors__HEADERS=Authorization,Content-Type,User-Agent
Environment=GITEA__cors__METHODS=GET,HEAD,POST,PUT,PATCH,DELETE,OPTIONS
[Service]
Restart=on-failure
# Shared 3.6 GB VM (Outline + Strapi live here too) — keep Gitea on a leash.
MemoryHigh=400M
MemoryMax=600M
[Install]
WantedBy=default.target
+28
View File
@@ -0,0 +1,28 @@
# CTAO portal demo — static web server. Serves ~/ctao-portal-demo/releases/current
# (a symlink the build flips atomically). Read-only mounts: nginx can only read.
# Quadlet unit → service name: ctao-demo-web.service.
[Unit]
Description=CTAO portal demo — static web server (nginx)
Wants=network-online.target
After=network-online.target
[Container]
ContainerName=ctao-demo-web
# Pinned by digest at install (2026-07-28); tag kept for readability.
Image=docker.io/library/nginx:stable-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
# releases/ is shared with the build container (:z shared label);
# the config file is exclusive to nginx (:Z).
Volume=%h/ctao-portal-demo/releases:/srv/releases:ro,z
Volume=%h/ctao-portal-demo/config/nginx.conf:/etc/nginx/conf.d/default.conf:ro,Z
# All-interfaces bind is deliberate — the ingress box reaches us over the
# network (see the matching note in ctao-demo-gitea.container).
PublishPort=8080:80
[Service]
Restart=on-failure
MemoryHigh=64M
MemoryMax=128M
[Install]
WantedBy=default.target
+15
View File
@@ -0,0 +1,15 @@
# CTAO portal demo — rebuild-on-new-commit worker (fired by the .timer).
# Install: copy to ~/.config/systemd/user/ → systemctl --user daemon-reload.
# Type=oneshot + timer means runs can never overlap: the timer will not
# activate a service that is still running.
[Unit]
Description=CTAO portal demo — rebuild if the code or content repo has new commits
[Service]
Type=oneshot
ExecStart=%h/ctao-portal-demo/bin/build.sh
# Polite neighbour on the shared VM:
Nice=10
# First-ever npm ci on 2 vCPU can be slow; normal rebuilds are seconds.
TimeoutStartSec=900
+18
View File
@@ -0,0 +1,18 @@
# CTAO portal demo — poll cadence for the build worker.
# OnUnitInactiveSec: next tick 10 s after the previous run FINISHES (it is
# deactivation-relative; OnUnitActiveSec would be start-relative). Overlap is
# impossible either way — a timer never activates a still-running unit.
# OnActiveSec covers `systemctl --user enable --now` (first tick 5 s later);
# OnBootSec covers reboot. Enable: systemctl --user enable --now ctao-portal-build.timer
[Unit]
Description=CTAO portal demo — poll the code and content repos every 10 s
[Timer]
OnActiveSec=5
OnBootSec=30
OnUnitInactiveSec=10
AccuracySec=1s
[Install]
WantedBy=timers.target
+49
View File
@@ -0,0 +1,49 @@
# CTAO portal demo — static serving. The cache split is what makes the
# 10-second publish loop feel instant: HTML is revalidated on every request
# (a symlink flip shows up on the next refresh), fingerprinted assets are
# cached forever.
server {
listen 80;
server_name _;
root /srv/releases/current;
charset utf-8;
server_tokens off;
error_page 404 /404.html; # Astro emits 404.html at the site root
# Directory redirects (/admin -> /admin/) must stay relative: an absolute
# redirect is built from listen port 80 and loses the real port whenever the
# site is reached through a tunnel or a proxy on a non-default port.
absolute_redirect off;
# Security headers are REPEATED in every location on purpose: nginx
# `add_header` inheritance is all-or-nothing — any add_header in a location
# discards ALL server-level ones, so server-level headers would silently
# vanish. `always` keeps them on error responses (404) too.
# Fingerprinted build assets (/_astro/<name>.<hash>.*) — immutable
location /_astro/ {
add_header Cache-Control "public, max-age=31536000, immutable" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header X-Frame-Options "SAMEORIGIN" always;
}
# Editor-uploaded media (stable paths, may be re-uploaded) — short cache
location /uploads/ {
add_header Cache-Control "public, max-age=3600" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header X-Frame-Options "SAMEORIGIN" always;
}
# Everything else: HTML pages, feeds, /admin (Sveltia is static files too)
location / {
try_files $uri $uri/ =404;
add_header Cache-Control "no-cache" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header X-Frame-Options "SAMEORIGIN" always;
}
gzip on;
gzip_types text/css application/javascript application/json image/svg+xml application/rss+xml text/xml application/xml;
}