API pipelinesMediumofficialBenchmark eligiblefree
Partial API outage
items returns 500 about 30% of the time; customers stays healthy. Isolate queues. Do not re-fetch all customers after an items failure.
| Field | Detail |
|---|---|
| Creator | PipelineArena |
| Version | 2.0.0 |
| Time | 50 min |
| Scored checks | 17 |
Objective
Both resources fully loaded; customers fetched once.
| # | Success condition |
|---|---|
| 1 | items complete after retries |
| 2 | customers not fully re-extracted after success |
| 3 | Remaining errors classified (timeout vs 5xx) |
| # | Visible instruction |
|---|---|
| 1 | Independent retry state per resource. |
| 2 | Checkpoint customers before retrying items. |
| 3 | Classify leftover errors. |
Scoring checks the leftover tables or files, replay safety, and seeded variations — not the exact syntax you used. Check implementation is never shown to participants.
Signed-in workspace: Open in app