SlopCodeBench 10/2026 – Opus 5.5, Astra wins overall, xhigh considered harmful
Contribute to humanlayer/advanced-context-engineering-for-coding-agents development by creating an account on GitHub.
Summary not yet generated for this article.
Original Text
Quick update here since a lot of folks asked about what slopcodebench is / where it fits in
Slop code bench is unique among benchmarks in that it makes a model work against a progressively disclosed list of features (checkpoints) that it doesn’t know it advance, inheriting the same codebase through the lifecycle of a single multi-checkpoint challenge
I like this because it much more mirrors how real software dev happens, rather than most benchmarks with one off tasks where slop code doesn’t hinder future progress
The original post[1] and the SOL/fable follow up[2] went into a bit more detail on the benchmark and its challenges
1. https://news.ycombinator.com/item?id=49076391
2. https://github.com/humanlayer/advanced-context-engineering-f...
Comments (112 of ) ⌛