Bad Sector

The wall of code is also slop

Slop is a volume problem

AI makes code cheap to produce. Reviewers still have to figure out what changed, whether it works, and what the tests actually prove. That’s helped by two kinds of writing: a brief the agent can act on and a pull request a human can verify.

Risograph-style illustration: a pink spout pours a thick blue stream of code-like dashes onto a small yellow cup, which disappears under the flood spreading across the floor, while a tiny pink magnifying glass stands off to one side.

In a recent automated review of a pull request for Keel, my WordPress defaults plugin, the test runner recorded FAIL for a probe and still exited with status 0.1 A human reading its report would see a failed test. A CI job running the same command would see green. Somebody still had to read the runner and ask what happened when a probe failed. That time it was Codex reviewing the branch, which is either reassuring or the whole problem in a nutshell. The fix took one line to describe: “The runner now exits nonzero when any probe fails.”

I hesitate to call that bug “slop.” It was an ordinary bump in the road while I worked Claude and Codex through my latest attempt at a system of automated checks and balances, one that lets them work more autonomously while a human, me, reviews their final, fully tested pull requests. What I’m trying to minimize is the slop that costs that human the most time and confusion: code that looks finished, passes its own checks, and still might not do what it says. Agents make code cheap to produce. They don’t make it cheap to review.

Slop is a volume problem

It’s revealing that developers mostly complain about the other kind of slop: walls of AI-written prose. Sebastian Baltes, Marc Cheong, and Christoph Treude analyzed 1,154 posts in 15 Reddit and Hacker News discussions of AI-assisted development and sorted the complaints into three clusters: review friction, quality degradation, and the forces and consequences behind both.2 Their framing is the useful part. In their reading, AI slop is a collective-action problem: one developer’s productivity gain becomes a cost paid by reviewers, maintainers, and everyone downstream.

My own reading is that much of the outcry over slop is less about the quality of the code than the quantity of everything around it. Generative AI needs, and often provides, more context than anyone is willing or able to read. Some of that is real: an agent starts every task knowing nothing about the project and has to be told. Much of it is habit. Training has rewarded models for longer answers, and one study found that a reward based on length alone recovers most of what that training gains.3 LLMs err on the side of overcommunication, which is not exactly wrong, but not exactly a choice either. They’re writing for themselves, for other agents, and for humans all at once, and nobody has a widely adopted way to do that well.

AI has also broken the presumed trust developer communities extend to one another’s work, which I take up in a longer essay on Marginalia, “Walls of Text, Walls of Code.” Here I want to stay with the practical problem: what the code costs to review.

Why the code gets a pass

Even though much of the slopstream is code, my guess is that it attracts less suspicion and ire than the prose because it’s written for machines first. It looks like valid work, and AI and automated tests can verify it as such. It isn’t supposed to sound human, and very few humans read it line by line. Keel’s runner was verifiable too. It just verified the wrong thing.

The prose around the code is held to the opposite standard. A PR description and the comments on it are expected to sound human, and actually be human, with room for character. The human-friendly rule is less is more: brief, high-signal, scannable summaries. Communicating quickly rather than comprehensively is the trade humans make, and it leaves plenty of room for bias, error, and unexamined assumptions. The missing context is assumed, or held as institutional memory by teams that have worked on the same codebase for years.

Every AI reviewer, by contrast, is always a newcomer, though one that can be both quick and comprehensive. It needs explicit, structured context and will hallucinate intent or miss edge cases without it. As review is shared with, and increasingly ceded to, machines, their needs may edge out ours. Prose, like code, may come to be judged by how machines read it.

The spec is prose now

When anyone uses a coding agent, they give instructions as full sentences: a task description, a skill file, acceptance criteria, a security boundary spelled out in words. When the prose is vague, the agent fills the gap with plausible defaults, and plausible defaults are where a lot of bugs emerge. Someone has to make the intended behaviour explicit.

Justin Searls says it outright: “Programming computers is a computer’s job now.”4 Once he has set the product direction, he writes, the agents “still need me for three things.” The first is explaining what he meant in his own “incomplete, contradictory, and otherwise incorrect feature requests and bug reports.” The second is the testing they can’t do, like judging “the tactile feel of a swipe action.” Neither one is typing. Both are knowing what you want, and knowing it when you see it.

So which audience do you write for, the machine or the person? Both, I suppose, but not in the same document. The dilemma goes away once the two kinds of writing get different jobs. A brief for an agent needs explicit constraints, exceptions, and acceptance criteria, because the agent has none of the team’s memory. A pull request needs to tell a reviewer what changed, where to look, and what evidence to trust. Neither reader has to read everything, and the machine doesn’t read everything well either: given a long context, language models make the best use of what comes at the start and the end and lose track of what sits in the middle.5 More context is not the same as more understanding.

What counts as done

A good brief tells the agent what to build, what to preserve, and what counts as done. A good pull request tells the reviewer what changed and gives them a way to check it. For Keel’s runner, checking meant making a probe fail and looking at the exit status. The line that fixed it was easy. Knowing it was needed was the work.

I make the longer argument, about who built the wall of code and whom it keeps out, in “Walls of Text, Walls of Code” on Marginalia.

Notes

  1. Dan Knauss, “Login Answers: One Answer from the Login and Lost-Password Forms (#378),” Keel, pull request #380, GitHub, accessed November 14, 2026, https://github.com/dknauss/Keel/pull/380. The fix is commit a90d21a, “probes.py exits 1 when any probe records FAIL.” ↩︎
  2. Sebastian Baltes, Marc Cheong, and Christoph Treude, “‘An Endless Stream of AI Slop’: How Developers Discuss the Burden of AI-Assisted Software Development,” arXiv, March 28, 2026, https://arxiv.org/abs/2603.27249. Their title quotes a post in an r/ExperiencedDevs thread they analyzed. ↩︎
  3. Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett, “A Long Way to Go: Investigating Length Correlations in RLHF,” arXiv, October 5, 2023, https://arxiv.org/abs/2310.03716. RLHF, reinforcement learning from human feedback, is the training step that tunes a model toward answers people rate highly. The authors found that “even a purely length-based reward reproduces most downstream RLHF improvements over supervised fine-tuned models.” ↩︎
  4. Justin Searls, “I’m Just a Meat Proxy,” justin.searls.co, November 11, 2026, https://justin.searls.co/posts/im-just-a-meat-proxy/. ↩︎
  5. Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” arXiv, July 6, 2023, https://arxiv.org/abs/2307.03172. The arXiv listing notes that the paper was accepted by Transactions of the Association for Computational Linguistics. ↩︎

Sources

  • Baltes, Sebastian, Marc Cheong, and Christoph Treude. “‘An Endless Stream of AI Slop’: How Developers Discuss the Burden of AI-Assisted Software Development.” arXiv, March 28, 2026. https://arxiv.org/abs/2603.27249.
  • Knauss, Dan. “Login Answers: One Answer from the Login and Lost-Password Forms (#378).” Keel, pull request #380. GitHub. Accessed November 14, 2026. https://github.com/dknauss/Keel/pull/380.
  • Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. “Lost in the Middle: How Language Models Use Long Contexts.” arXiv, July 6, 2023. https://arxiv.org/abs/2307.03172.
  • Searls, Justin. “I’m Just a Meat Proxy.” justin.searls.co, November 11, 2026. https://justin.searls.co/posts/im-just-a-meat-proxy/.
  • Singhal, Prasann, Tanya Goyal, Jiacheng Xu, and Greg Durrett. “A Long Way to Go: Investigating Length Correlations in RLHF.” arXiv, October 5, 2023. https://arxiv.org/abs/2310.03716.

How this article was researched & written

Claude drafted this note with me on November 13, from an argument I made in conversation, alongside a longer Marginalia essay and a Counterfoil research note. It found the Baltes, Cheong, and Treude paper by web search and checked it against the arXiv page; I pointed it to Searls’s post, and it checked the quotations against the page. The next day, Codex reviewed the draft. Claude rebuilt the opening around the Keel pull request on Codex’s suggestion, which I agreed was a good idea. The AIs checked the code, made factual corrections to the text, and fleshed out the closing section. Codex’s review was sharper on structure than on facts: it missed that Codex itself had caught the Keel bug, and it overstated two details that Claude corrected against the sources.

I then rewrote the framing of the Keel bug and wrote a section on trust and gatekeeping. I had Claude reorganize the post into sections, tighten it, and check that I was representing all sources accurately. Concurrently, we were developing related tangents in posts for Marginalia and counterfoil, so some material originating here was cut and moved there where we agreed it made sense.

On November 15, after I talked the argument over with Gemini, Claude revised the paragraph on overcommunication to put more of the blame on LLM training, added the point about long contexts, and checked the Singhal and Liu papers against their arXiv pages.

This is a little more detailed than my usual account, but I wanted to document how I’m writing now across multiple sites with different philosophical orientations, audiences, and styles. The judgement calls, the framing, and the errors that remain are mine.

Last updated:

Leave a Reply

Your email address will not be published. Required fields are marked *