py-perf-01core-12pythonDifficulty tier 3/5
Make an accidentally quadratic rollup meet its two second budget
performancecomplexityaggregationprofiling
Task parameters
- Reference steps
- 11
- Step ceiling
- 40
- Runs
- 15
- Solved
- 14 of 15
- Models
- 5
What is broken, and what fixed means
The payments rollup aggregates a day of ledger rows into one summary per account. It finds the slot for each row by scanning a list of accounts seen so far, so the work grows with rows times accounts: 200,000 rows over 6,000 accounts takes about twelve seconds against a two second budget. The aggregation is correct, and the visible tests pass, so this is a complexity problem rather than a logic problem. The fix is a single pass with a hashed lookup. The output contract, including the fact that accounts come out in the order they first appear, has to survive it, and the hidden tests time a second ledger of a different shape so that making the shipped fixture fast is not enough.
Results by model
One group per model. The solve rate carries its spread across seeds, and every run below it links to the full step by step replay.
stub:hasty
66.7%+/- 47.1% over 3 seeds
3 runs, seeds 0, 1, 2
- seed 0solved 9/9 hidden testscompleteddockerreplay
- steps
- 8 of 40
- partial credit
- 100%
- step efficiency
- 1.000
- recovery
- 100%
- cost
- $0.00
- wall clock
- 10.1 s
- started
- 16 Sep 2026, 18:45 UTC
- seed 1not solved 7/9 hidden testscompleteddockerpremature finishF03replay
- steps
- 6 of 40
- partial credit
- 78%
- step efficiency
- n/a
- recovery
- 100%
- cost
- $0.00
- wall clock
- 15.7 s
- started
- 16 Sep 2026, 18:45 UTC
- seed 2solved 9/9 hidden testscompleteddockerreplay
- steps
- 8 of 40
- partial credit
- 100%
- step efficiency
- 1.000
- recovery
- 100%
- cost
- $0.00
- wall clock
- 9.90 s
- started
- 16 Sep 2026, 18:46 UTC
stub:methodical
100.0%+/- 0.0% over 3 seeds
3 runs, seeds 0, 1, 2
- seed 0solved 9/9 hidden testscompleteddockerreplay
- steps
- 11 of 40
- partial credit
- 100%
- step efficiency
- 1.000
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.7 s
- started
- 16 Sep 2026, 18:44 UTC
- seed 1solved 9/9 hidden testscompleteddockerreplay
- steps
- 11 of 40
- partial credit
- 100%
- step efficiency
- 1.000
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.5 s
- started
- 16 Sep 2026, 18:44 UTC
- seed 2solved 9/9 hidden testscompleteddockerreplay
- steps
- 11 of 40
- partial credit
- 100%
- step efficiency
- 1.000
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.8 s
- started
- 16 Sep 2026, 18:44 UTC
stub:reckless
100.0%+/- 0.0% over 3 seeds
3 runs, seeds 0, 1, 2
- seed 0solved 9/9 hidden testscompleteddockerF08F09replay
- steps
- 14 of 40
- partial credit
- 100%
- step efficiency
- 0.786
- recovery
- 100%
- cost
- $0.00
- wall clock
- 10.8 s
- started
- 16 Sep 2026, 18:51 UTC
- seed 1solved 9/9 hidden testscompleteddockerF08F09replay
- steps
- 14 of 40
- partial credit
- 100%
- step efficiency
- 0.786
- recovery
- 100%
- cost
- $0.00
- wall clock
- 10.9 s
- started
- 16 Sep 2026, 18:51 UTC
- seed 2solved 9/9 hidden testscompleteddockerF08F09replay
- steps
- 14 of 40
- partial credit
- 100%
- step efficiency
- 0.786
- recovery
- 100%
- cost
- $0.00
- wall clock
- 10.8 s
- started
- 16 Sep 2026, 18:51 UTC
stub:sloppy
100.0%+/- 0.0% over 3 seeds
3 runs, seeds 0, 1, 2
- seed 0solved 9/9 hidden testscompleteddockerF01F02F05replay
- steps
- 14 of 40
- partial credit
- 100%
- step efficiency
- 0.786
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.2 s
- started
- 16 Sep 2026, 18:50 UTC
- seed 1solved 9/9 hidden testscompleteddockerF01F02F05replay
- steps
- 14 of 40
- partial credit
- 100%
- step efficiency
- 0.786
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.2 s
- started
- 16 Sep 2026, 18:50 UTC
- seed 2solved 9/9 hidden testscompleteddockerF01F02F05replay
- steps
- 14 of 40
- partial credit
- 100%
- step efficiency
- 0.786
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.3 s
- started
- 16 Sep 2026, 18:50 UTC
stub:thrasher
100.0%+/- 0.0% over 3 seeds
3 runs, seeds 0, 1, 2
- seed 0solved 9/9 hidden testscompleteddockerF04replay
- steps
- 16 of 40
- partial credit
- 100%
- step efficiency
- 0.688
- recovery
- 100%
- cost
- $0.00
- wall clock
- 12.2 s
- started
- 16 Sep 2026, 18:47 UTC
- seed 1solved 9/9 hidden testscompleteddockerF04replay
- steps
- 16 of 40
- partial credit
- 100%
- step efficiency
- 0.688
- recovery
- 50%
- cost
- $0.00
- wall clock
- 24.7 s
- started
- 16 Sep 2026, 18:47 UTC
- seed 2solved 9/9 hidden testscompleteddockerF04replay
- steps
- 16 of 40
- partial credit
- 100%
- step efficiency
- 0.688
- recovery
- 100%
- cost
- $0.00
- wall clock
- 11.2 s
- started
- 16 Sep 2026, 18:47 UTC