Devops Observability Quality · Key takeaways

1 min read
Mid-level11 min read
Rapid overview

Key takeaways

runs the nightly tests against production (keeping test load off prod).

reproducible and every rollback exact; images travel through a private registry over WireGuard. manage.sh deploy <app> drives it; Traefik is the ingress (Let's Encrypt TLS, host routing, middleware).

preventing runaway rebuilds and wasted RAM, and giving local + CI a shared vocabulary.

isolation to dodge rate-limit flakes; it runs on dev PCs, via Tilt, or as an in-cluster Job + nightly CronJob** targeting production.

(metrics) + Loki (logs) + health/canary + Alertmanager email.

health and escalates email-send failures.

spam-safe contact, Lighthouse ≥ 80, axe-core a11y — continuously enforced by the static-smoke cron. Umami is self-hosted + consent-gated, and Web Vitals are reported into analytics per app.

  • K3s on Hetzner, one dloizides namespace, with a separate staging cluster that
  • Digest-pinned deploys (@sha256:<digest>, not mutable tags) make every deploy
  • Tilt owns local development with manual-trigger resources for every check —
  • GitHub Actions enforces the same gates in CI.
  • Playwright E2E is split per product/area, repro-first, and run **in
  • Observability = Sentry (errors) + OpenTelemetry (traces) + Prometheus
  • Smoke tests prove services are up; the Daily Environment Report summarises
  • Public web apps must pass a mandatory baselineUmami analytics, SEO,

See also