Node Right-Sizing Study Portal

TL;DR

On a small node, memory is almost always the binding constraint — not CPU and not "too many containers." A single-node k3s box (4 vCPU, 7.9 GB RAM, no swap) running ~89 pods was OOM-killing pods even though CPU sat at 26%, because memory was 95% requested with only ~880 MB free and limits overcommitted to 244% on a node with no swap. The biggest lesson is a plot twist: measure before you attribute. The plan assumed the fat .NET services were heavy because of Server GC and that flipping them to Workstation GC would save ~150 MB each — but inspecting the live pods showed the fat services were already on Workstation GC and the lean ones (poueni 85 MB, dynalux 78 MB) were on default Server GC. The gap was real application working set driven mostly by uptime accumulation — an A/B test showed the same service at 94 MB fresh vs 284 MB aged (and kefi-api 419 MB after 16h), while DOTNET_GCConserveMemory=5 made a fresh pod worse (121 MB). Long-running .NET services accumulate caches, large-object-heap growth and broker buffers that the GC holds rather than returning. The genuine levers, in order: lower the container memory limit (the GC heap hard limit is ~75% of it, so a tighter cap forces the GC compact — roll one at a time with a swap cushion), periodic restart (non-durable band-aid), app-level reduction, and scale idle services to zero with KEDA (sidesteps accumulation entirely). Verify every assumption against ground truth, change one service, measure, then roll out — never big-bang on production.