Skip to content

Drill: noisy neighbor

Moduleops.09 · drill · ops, docs · Pass 11 · 1 to 2 h
You builddocs/runbooks/NoisyNeighbor.md
Testscourse/tests/ops.09 (named below, with why each exists)
Needsnone
Used byno code call site; this is an independent artifact
MilestoneMS-P11
  • Tie each claim to the exact platform evidence that Drill: noisy neighbor produces.
  • State the owner, success condition, and recovery action before declaring the work complete.
  • Record unresolved assumptions so the next review can test them.
Terminal window
ol start ops.09
ol tests ops.09
ol check ops.09

Shared compute lets one tenant degrade another tenant’s latency. This drill tests detection, attribution, and fair recovery.

Measure per-tenant queueing and resource use. Protect reserved capacity, apply bounded admission, and avoid penalizing unrelated tenants.

Generate a high-rate workload from one tenant while a steady control workload runs. Compare queue depth, TTFT, 429s, and utilization by tenant.

TestKINDChecksWhy it matters downstream
test_artifact_paths_are_declaredartifactConfirms the submitted paths and required evidence exist, so a plausible narrative cannot pass without the reviewable deliverable.
#PitfallSymptomCaught by
1The canary advances before the rollback trigger is recordedThe exercise reports success without proving the old format or route can be restored.test_artifact_paths_are_declared
2The team watches request errors but omits the migration-specific signalA partial rollout can corrupt or strand work while the generic health check stays green.test_artifact_paths_are_declared
3No owner records the decision and verification evidenceThe next on-call cannot tell whether the incident is contained.test_artifact_paths_are_declared
DirectionModuleConnection
Backops.08Read this declared prerequisite before producing the artifact; use it to verify the relevant contract or evidence.
Backload.01Read this declared prerequisite before producing the artifact; use it to verify the relevant contract or evidence.
ResourceWhat to inspect
Incident drill catalogCompare this inject with the adjacent failure drills and use the same containment and verification fields.