Compare a canary with the baseline. Each has {"requests": int, "errors": int, "p95_ms": float}.
Rules, checked in order (collect every failed rule):
"insufficient-traffic" if the canary has fewer than min_requests requests — then return
{"decision": "wait", "reasons": [...]} immediately."error-rate" if canary error rate > baseline error rate + max_error_delta (absolute, e.g. 0.01)."latency" if canary p95 > baseline p95 × (1 + max_latency_ratio).Return {"decision": "promote", "reasons": []} if no rule failed, else "rollback" with the reasons.
Treat an error rate with zero requests as 0.
Implement canary_decision. Input: baseline: dict, canary: dict, min_requests: int, max_error_delta: float, max_latency_ratio: float. Returns: dict.
def canary_decision(baseline, canary, min_requests, max_error_delta, max_latency_ratio):
return {"decision": "promote", "reasons": []}
Run executes the public tests. Submit grades against public and hidden tests.
Compare a canary with the baseline. Each has {"requests": int, "errors": int, "p95_ms": float}.
Rules, checked in order (collect every failed rule):
"insufficient-traffic" if the canary has fewer than min_requests requests — then return
{"decision": "wait", "reasons": [...]} immediately."error-rate" if canary error rate > baseline error rate + max_error_delta (absolute, e.g. 0.01)."latency" if canary p95 > baseline p95 × (1 + max_latency_ratio).Return {"decision": "promote", "reasons": []} if no rule failed, else "rollback" with the reasons.
Treat an error rate with zero requests as 0.
Implement canary_decision. Input: baseline: dict, canary: dict, min_requests: int, max_error_delta: float, max_latency_ratio: float. Returns: dict.
def canary_decision(baseline, canary, min_requests, max_error_delta, max_latency_ratio):
return {"decision": "promote", "reasons": []}
Run executes the public tests. Submit grades against public and hidden tests.