Would you expand this migration?
The overall comparison is good. One consumer still gets the wrong answer.
The question
Morrow is changing the storage behind a scheduling limit service. Product teams depend on exact counts in different time windows. The old path still serves users; the new path runs comparison requests. All data are invented.
| Compared requests | Different answers | |
|---|---|---|
| Standard windows | 99000 | 0 |
| Boundary windows | 1000 | 20 |
The table has 100,000 compared requests and 20 mismatches. An overall 0.02% mismatch does not establish that the boundary-window rule is acceptable.
“State what the comparison proves and decide the next scope of live traffic.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: The overall mismatch is 0.02%. Is that enough to expand?
What would you do here? Open the move you’d make.
Expand because the overall mismatch is under one percent
Tempting, but it misses something. No acceptable mismatch threshold or consumer impact was agreed.
Inspect the affected consumer and the boundary cohort
The strong move. The concentrated differences can affect an important contract.
Stop every project because any comparison can differ
Tempting, but it misses something. A difference needs explanation; this packet does not establish that all work must stop.
See how a strong candidate says it
“No decision follows from the average alone. All 20 differences are in the 1,000 boundary-window requests. I would ask which consumer contract is affected and whether any different answer changes a user decision.”
Why it works: The boundary cohort has a two-percent mismatch, even though the overall average looks small.
Step 2: New fact: 12 of the 20 differences would deny valid bookings. The limit team owns that contract. Who must review?
What would you do here? Open the move you’d make.
Ask the storage team to sign off alone
Tempting, but it misses something. Storage-level agreement does not establish the booking-limit contract.
Tell support the new behavior is close enough
Tempting, but it misses something. Twelve compared requests would wrongly deny valid bookings.
Bring the contract owner into the decision and isolate the cohort
The strong move. This keeps the affected users out of the expansion and names the accountable team.
See how a strong candidate says it
“The limit team must review the changed behavior with the storage team. I would keep users on the old path for this cohort, record the 12 false denials and assign the repair and test owner.”
Why it works: The storage team can explain rows, but the consumer team owns what a denial means.
Step 3: New fact: a boundary-rounding repair passes the saved examples. What evidence is still missing?
What would you do here? Open the move you’d make.
Check full-path boundary behavior and the return path
The strong move. These are the remaining named risks, rather than an unrelated broad test sweep.
Expand immediately because the twenty examples pass
Tempting, but it misses something. The examples do not establish representative behavior or recovery.
Require an arbitrary month of comparisons
Tempting, but it misses something. Duration alone does not answer which boundary transitions or contracts were covered.
See how a strong candidate says it
“The saved examples show the known defect is repaired. Before live expansion, compare representative full requests over the boundary transitions and exercise the return to the old path. Agree the stop condition with the limit team.”
Why it works: The known examples do not cover the full request behavior or the rollback operation.
Step 4: New fact: representative boundary checks and return-path rehearsal pass; the limit owner accepts a small live cohort. What now?
What would you do here? Open the move you’d make.
Keep comparing forever without a remaining question
Tempting, but it misses something. The agreed risks for this next step have evidence.
Expand to the accepted cohort and observe the contract
The strong move. This matches the commitment to the evidence and preserves a clear operator.
Delete the old path as soon as the cohort starts
Tempting, but it misses something. A first live cohort does not establish the retirement conditions.
See how a strong candidate says it
“Expand to the agreed cohort with the named operator and stop condition. Compare the affected consumer outcomes, not only storage latency. Retire the old path only after the separate retention and ownership decision.”
Why it works: The evidence now supports a bounded next move. It does not establish final retirement.
The answer, in short
- Decision
- Expand only after the affected contract and return path have evidence.
- Because
- A low aggregate mismatch hid false booking denials in one cohort.
- What would change it
- New false denials, an untested cohort or an unavailable return path stops expansion.
Common traps
- Using an aggregate as a consumer contract
- Letting one team approve another team’s behavior
- Equating a first live cohort with final retirement
Check your answer
- Did I inspect the cohort?
- Did I name the user effect?
- Did the consumer owner accept the next step?
- Did I separate expansion from retirement?
Last reviewed Nov 10, 2026