[{"data":1,"prerenderedAt":104},["ShallowReactive",2],{"blog-post-en-the-trust-problem-coming-for-vibe-coded-software":3},{"id":4,"title":5,"author":6,"body":7,"category":86,"date":87,"description":88,"excerpt":89,"extension":90,"gradient":91,"icon":92,"meta":93,"navigation":94,"path":95,"seo":96,"stem":97,"tags":98,"__hash__":103},"posts_en\u002Fblog\u002Fthe-trust-problem-coming-for-vibe-coded-software.md","The Trust Problem Coming for Vibe-Coded Software","mercedes",{"type":8,"value":9,"toc":79},"minimark",[10,20,23,28,31,34,37,40,44,47,50,57,60,63,67,70,73,76],[11,12,13,14,19],"p",{},"The ",[15,16,18],"a",{"href":17},"\u002Fblog\u002Fwhy-the-200-dollar-seat-exists","previous post"," argued that a small team can now build and operate a product that used to require forty people. That argument is straightforwardly true, and it comes with a consequence that is less comfortable: the same collapse in build cost applies to everyone, including people shipping software they have not read.",[11,21,22],{},"This is not a complaint about AI-assisted development. We use it heavily, on every product we run. It is an observation about what happens when the cost of producing plausible code drops faster than the cost of verifying it — because those two costs did not fall together, and the gap between them is where the problem lives.",[24,25,27],"h2",{"id":26},"the-near-term-failure-modes-are-specific","The near-term failure modes are specific",[11,29,30],{},"\"Vibe-coded software is bad\" is not a useful claim. The useful version names the failure classes, because they are not random — they cluster in predictable places.",[11,32,33],{},"Generated code is reliably competent at the shape of a solution and unreliable at its boundaries. Authorization is the clearest case: a model asked for an endpoint that returns a user's records will produce one that works perfectly when you are that user. Whether it checks that you are that user depends on whether someone thought to ask. The happy path is what gets demonstrated, so the happy path is what gets verified.",[11,35,36],{},"The same pattern shows up in multi-tenancy, where a missing scope filter is invisible until you have a second customer; in error handling, where the difference between a caught exception and a stack trace containing a connection string is one line nobody looked at; and in dependency selection, where a model will confidently reach for a package that solved this problem well in 2021 and has had three advisories since.",[11,38,39],{},"None of these are exotic. They are the ordinary content of a code review. The problem is not that AI produces uniquely dangerous code — it is that it produces a volume of code that makes skipping review feel affordable, and these particular mistakes are the ones that survive a demo.",[24,41,43],{"id":42},"where-this-goes","Where this goes",[11,45,46],{},"Here is the part that is a forecast rather than an observation, and it's worth labeling it as such, because confident prediction is exactly the failure mode this post is about.",[11,48,49],{},"Software has enjoyed a default presumption of competence for a long time. When you enter a card number or upload a document, you are not usually evaluating whether that application's authorization logic was reviewed by anyone. You assume that shipping to production implied someone competent was involved, because for most of the industry's history that assumption was cheap and roughly correct.",[11,51,52,53],{},"That assumption gets more expensive as the cost of shipping falls. I would expect the sequence to be gradual and unglamorous: a rising volume of ordinary breaches in small and mid-sized products, few of them individually newsworthy; a slow accumulation of the sense that a certain category of application is not to be trusted with anything sensitive; and eventually a purchase criterion that barely exists today — some version of ",[54,55,56],"em",{},"who built this, and what did they check?",[11,58,59],{},"The category where this bites first is not consumer apps. It is the software holding things people cannot re-issue. A leaked password is an afternoon of annoyance. A leaked medical history is permanent. We build in healthcare, so this is not abstract for us: the asymmetry between how cheap the software was to produce and how unrecoverable the damage is when it fails is the entire risk, and it is not visible in a product demo.",[11,61,62],{},"I could be wrong about this. Consumers have absorbed a great deal of breach fatigue without changing behavior much, and it is entirely possible that this follows the same path — genuine harm, no behavioral change, regulation eventually arriving late and unevenly. But the direction seems clear even if the timeline isn't, and the cost of being early to caution here is low.",[24,64,66],{"id":65},"what-this-actually-implies-for-building","What this actually implies for building",[11,68,69],{},"The practical conclusion is narrower than \"be careful.\" AI should write most of the code — that part of the argument is settled, and refusing it is just choosing to be slow. What cannot be delegated is the judgment about which parts of the system are expensive to get wrong.",[11,71,72],{},"For us that means a small number of non-negotiables. Anything touching authorization, tenant isolation, or personal data gets read by a human who understands the threat model, not skimmed for style. The test pipeline runs unit, integration, and end-to-end before anything reaches production, and a red pipeline stops the release regardless of how confident the change looks. Dependencies are chosen deliberately and audited, not accepted because they appeared in a suggestion.",[11,74,75],{},"None of that is novel. It is what careful teams did before any of this, and the reason to say it out loud is that the economics now actively discourage it. When generating a feature takes twenty minutes and reviewing it properly takes two hours, the review is what gets cut, and nothing about that decision is visible to the customer until it is very visible indeed.",[11,77,78],{},"The trust asymmetry is what makes this worth taking seriously. Being trusted with someone's data is slow to earn, immediate to lose, and effectively impossible to rebuild. That has not been repriced by anything — which is the one part of the old cost structure worth keeping.",{"title":80,"searchDepth":81,"depth":81,"links":82},"",2,[83,84,85],{"id":26,"depth":81,"text":27},{"id":42,"depth":81,"text":43},{"id":65,"depth":81,"text":66},"engineering","2026-09-14","The previous post argued that a small team can now build and operate a product that used to require forty people. That argument is straightforwardly true, and it comes with a consequence that is less comfortable: the same collapse in build cost applies to everyone, including people shipping software they have not read.",null,"md","linear-gradient(135deg, rgba(79,70,229,0.4), rgba(99,102,241,0.2))","i-heroicons-shield-exclamation",{},true,"\u002Fblog\u002Fthe-trust-problem-coming-for-vibe-coded-software",{"title":5,"description":88},"blog\u002Fthe-trust-problem-coming-for-vibe-coded-software",[99,100,101,102],"Software Quality","Security","AI","Trust","-902fwHEXtmRQWhxlLpCYo5EnVudOfHcM0g6hcK79Hg",1789948885991]