Devops Observability Quality · Key takeaways
1 min readKey takeaways
runs the nightly tests against production (keeping test load off prod).
reproducible and every rollback exact; images travel through a private registry over WireGuard. manage.sh deploy <app> drives it; Traefik is the ingress (Let's Encrypt TLS, host routing, middleware).
preventing runaway rebuilds and wasted RAM, and giving local + CI a shared vocabulary.
isolation to dodge rate-limit flakes; it runs on dev PCs, via Tilt, or as an in-cluster Job + nightly CronJob** targeting production.
(metrics) + Loki (logs) + health/canary + Alertmanager email.
health and escalates email-send failures.
spam-safe contact, Lighthouse ≥ 80, axe-core a11y — continuously enforced by the static-smoke cron. Umami is self-hosted + consent-gated, and Web Vitals are reported into analytics per app.
- K3s on Hetzner, one
dloizidesnamespace, with a separate staging cluster that - Digest-pinned deploys (
@sha256:<digest>, not mutable tags) make every deploy - Tilt owns local development with manual-trigger resources for every check —
- GitHub Actions enforces the same gates in CI.
- Playwright E2E is split per product/area, repro-first, and run **in
- Observability = Sentry (errors) + OpenTelemetry (traces) + Prometheus
- Smoke tests prove services are up; the Daily Environment Report summarises
- Public web apps must pass a mandatory baseline — Umami analytics, SEO,