Vibe coding toward the incident horizon

We are living through a golden age of generative AI: a time when progress is both breathtaking and somehow still unable to reliably open a PDF without hallucinating the author’s middle name. The curve is real, the funding is real, and the demos are so real that they must be watched on a stage with dramatic lighting, because ordinary lighting reveals too much. The modern model does many things remarkably well—summarization, translation, code generation—and then it will confidently assert that 9 is a prime number “depending on your threat model,” which is how you know you’re witnessing history.

A lot of this is because we trained the future on the internet, which is like training a gourmet chef by locking them in a gas station for three years with unlimited energy drinks and a copy of “Culinary Theory (Unofficial Fan Wiki).” The corpus contains Shakespeare, medical textbooks, and the kind of Reddit thread where a man explains that welding is safer without a mask because “my uncle never wore one and he only glows on weekends.” The model digests all of it with equal serenity. It does not learn truth so much as it learns the shape of sentences that arrive after confidence. This is why it can produce a flawless explanation of distributed consensus while simultaneously insisting that TCP stands for “Total Cloud Persistence.” The system is not lying; it is performing a statistically accurate reenactment of a person online.

Naturally, companies respond with marketing claims about “PhD-level reasoning.” And sure: the model can generate a literature review at a speed that implies it is either brilliant or committing a crime. It can draft a grant proposal that contains all required sections, three new sections you didn’t ask for, and a concluding paragraph that reads like it was written by a very polite fog machine. Yet the same system will fail at tasks that toddlers solve using pure spite, like “put the triangle in the triangle hole.” It can explain category theory and then forget what a category is mid-sentence, the way a browser can throw seven errors and keep rendering anyway, because consequences are a legacy feature.

The most reliable promise is also the simplest: AGI is always one year away. Not a year—the year, a mythical constant like π or “next quarter,” eternally approached and never reached. This is convenient for everyone involved. Executives can announce imminent transcendence while remaining safely employed in the pre-transcendence economy. Investors can fund the revolution without the awkward part where the revolution arrives and asks for accountability. And the rest of us can enjoy the comfort of knowing that whatever is happening now is not the real thing; the real thing will arrive next year, fully aligned, thoroughly audited, and carrying a tasteful slide deck.

Meanwhile we are told that non-programmers can build serious applications through vibe coding, which is true in the same way that non-pilots can land a plane if you redefine “land” as “become part of the landscape.” Vibe coding produces software the way a séance produces architectural drawings: everyone is emotionally involved, nobody can reproduce the result, and at the end a table has moved slightly. The app usually works in the demo because the demo is the model’s natural habitat: a carefully curated universe in which users behave correctly and the network never blinks. Then you ship it, and a user does something monstrous like entering an emoji into a phone-number field, and the system responds by inventing a new category of error that HR cannot classify.

From here the conclusion is always delivered with the seriousness of a prophecy: there is no future in programming; programming jobs will disappear. This is plausible if you define “programming” as “typing code” and define “disappear” as “mutate into ten other jobs that still require engineers.” Code will be generated. So will bugs. So will security vulnerabilities that look like they were handcrafted by a bored demon with an excellent CI pipeline. The remaining human work will be debugging, governance, and the ancient art of explaining to stakeholders that “it worked in the demo” is not a compliance framework.

And then come the humanoid robots, inevitably described as replacing all human labor because they have hands, feet, and the haunting, unblinking optimism of a product page. Humanoids are appealing because the world is built for humans: stairs, doorknobs, forklifts, keyboards, the entire category of “things that were designed without asking permission from physics.” But reality is rude. A robot can carry boxes until it meets a wet floor, a narrow hallway, a dog, or a toddler—each of which is a chaos engine disguised as a small problem. The robot will do your job flawlessly right up to the moment it encounters the office kitchen, where it will stare into the sink like it is reading an ancient prophecy, and then it will file a ticket: Cannot proceed: dishes are unstructured data.

So yes: progress is astonishing. Models write code, images, prose, and occasional small pieces of legal doctrine that would get a real lawyer gently escorted out of the building. The failures are also astonishing, because they are not subtle failures; they are cliff-edge failures, the kind that make you wonder if the system is brilliant, broken, or simply experiencing a deep and personal disagreement with the concept of “Monday.” The future will arrive, probably one year from now, and it will be amazing. In the meantime, we will keep shipping systems that can draft a dissertation but cannot reliably tell whether “seven” is greater than “elephant,” and we will call this “reasoning,” because optimism is a powerful runtime.^¶

^¶ With due reverence to James Mickens’s talent for weaponized hyperbole and to ChatGPT’s tireless production of statistically plausible sentences.

Comments   Post Toot! Tweet Share