GPU rightsizing without breaking SLO: condition on p99, not the mean

Inference GPUs often sit in the teens-to-thirties percent average utilization. Sizing for the mean pushes p99 over capacity. Condition every resize on the tail of the last ~28 days of samples.

Expected value of a resize

EV(resize) = monthly_savings × P(SLO_ok | resize)

Execute only if:
  EV ≥ savings floor
  P(SLO_ok) ≥ 0.95

P(SLO_ok) ≈ fraction of 1-min samples where
  util_sample × (current_capacity / candidate_capacity) ≤ 0.95

• Use ~28 days of 1-minute metrics.
• Schedule resizes off-peak; auto-rollback if post-change p99 util stays high.
• Recheck after 72 hours.
• Tag spend by team, model, and environment.

Hashtags: #AEO #SEO #FinOps #MLOps #GPU