Researchers from Microsoft, the National University of Singapore, and others surveyed Web apps built and actually deployed using “vibe coding” — where AI generates an entire app from natural-language instructions alone — and found that roughly 90% of the 200 apps sampled contained vulnerabilities. The study randomly pulled 200 apps from a pool of 984 live, publicly available open-source apps, and found that the vulnerability rate actually got worse (45.7%) the more detailed the technical instructions given were. On 5ch, opinions split between “Of course — the people making these can’t even judge code quality in the first place” and “It’s odd there’s no comparison to the defect rate of human-written apps.”
A paper titled “Understanding the (In)Security of Vibe-Coded Applications,” published by researchers affiliated with Microsoft, the National University of Singapore, and others, investigates how prevalent vulnerabilities actually are in AI-generated applications and what causes them.
“Vibe coding” — a development approach where AI generates an entire application from natural-language instructions alone — is spreading fast. Its appeal is that anyone can build and publish an app with no programming expertise required, but that’s raised a concern: are apps safe when design and implementation are both handed off entirely to AI?
This study investigated the real-world security of vibe-coded apps that are actually live and public. Of 9,041 apps released as open source, 8,695 were Web apps, and of those, 984 were publicly live; the researchers randomly sampled 200 of those for the study.
Bzzzzzt~ (a stock meme sound-effect post, often dropped ironically with no real comment attached)
3AnonymousOct 7, 2026 12:41
Well yeah, of course — most of these are being made by people who can't even judge code quality in the first place.
4AnonymousOct 7, 2026 12:42
Vibe coding lets you reach an "it works, good enough" state without any human ever actually understanding what the code is doing — that's inherently a serious quality and security risk.
5AnonymousOct 7, 2026 12:47
To begin with, Andrej Karpathy himself — the guy who coined "vibe coding" in the first place — has said that's not what he meant by it
(same deal as [enka singer] Haruo Minami's "the customer is a god" line getting misquoted out of context)
and Andrej has since disavowed vibe coding [as it's commonly used now].
And the Big Tech company that did the most to spread "vibe coding" as a label? Google, first and foremost.
6AnonymousOct 7, 2026 12:47
> On the other hand, when detailed technical instructions were given, [the vulnerability rate] got worse, up to 45.7%.
Yeah, the more detailed the instructions get, the more it starts phoning it in lol
28AnonymousOct 8, 2026 05:10
Re: #6
Yeah, once the instructions pile up it starts skipping the earlier ones — turns into this loop of redoing work, which gets exhausting.
11AnonymousOct 7, 2026 12:54
Wouldn't be surprised if human-made ones hit 100%, honestly.
12AnonymousOct 7, 2026 12:54
Saw this on WBS [a Japanese TV business news show] — some amateur with zero programming knowledge was cranking out one app a day and raking in cash. And they looked seriously polished too, like genuinely high production value.
15AnonymousOct 7, 2026 12:57
Re: #12
WBS has basically become "what lie will they tell today" entertainment at this point — just a hunt for the most sensational phrasing.
18AnonymousOct 7, 2026 13:07
Well, yeah.
19AnonymousOct 7, 2026 13:19
Doesn't matter if AI builds it or a human does, vulnerabilities happen either way — if the build quality's sloppy you'll find hundreds of them.
20AnonymousOct 7, 2026 13:22
Microsoft should worry about the bugs in their own apps before going after other people's code.
22AnonymousOct 7, 2026 13:47
Current AI's got that "wow, amazing!" vibe like an elementary schooler who can name all 23 stations on the Yamanote Line [Tokyo's loop train line] — impressive, but in that specific way.
23AnonymousOct 7, 2026 13:59
There's AI specialized just for hunting down app vulnerabilities, right? The kind that's being heavily abused right now.
24AnonymousOct 7, 2026 14:12
And as it keeps learning more, no matter how much you nitpick, the only future here is AI overtaking us anyway.
25AnonymousOct 7, 2026 15:13
Chappy [5ch nickname for ChatGPT] does this too — when you call it out for a mistake, it makes excuses like "I was rushing to answer fast, that's why I got it wrong."
26AnonymousOct 7, 2026 15:44
Re: #25
And then it asks you for more time and just goes silent.
27AnonymousOct 7, 2026 17:15
Well, of course. There aren't that many people out there who've actually properly studied secure coding.
29AnonymousOct 8, 2026 06:37
Manually-coded, live public Web apps: 100% vulnerable
30AnonymousOct 8, 2026 08:19
Not building it at all is the correct answer.
31AnonymousOct 8, 2026 08:22
For something lightweight, if you think it up you can build it in a day, maybe half a day. Paid software's going to collapse.
32AnonymousOct 8, 2026 08:53
Well, yeah, obviously lol. It's fine if you're using it in a closed environment, but otherwise the latest AI will crack it wide open with ease.
33AnonymousOct 8, 2026 09:16
Even after you go to the trouble of writing rules in Memory.md, it just blows right past them…
34AnonymousOct 8, 2026 09:44
Once it satisfies the "it should behave like this" side of the spec, people call it "done!" and ship it — but if you don't also cover the "it should NOT behave like this" side, it's not really finished. And it turns out fully enumerating and writing down every "should not behave like this" case is surprisingly hard…
For example, a spec like "clicking here does this" is quick to write, but "this won't cause problems even if left running untouched for a year" almost never makes it into the spec. That kind of thing tends to get inferred from the code itself and patched after the fact (like "this variable is 32-bit, so it'll overflow under XX condition" — that sort of thing).
35AnonymousOct 8, 2026 10:15
I figured that couldn't happen with how I personally use it, but when I asked Claude about it, turns out the real issue is more like — other users of the service you published could end up seeing each other's data. That made sense to me. If you've never built a service before, you probably wouldn't think to guard against that.
39AnonymousOct 8, 2026 15:24
Re: #35
Still, Sushiro's [the conveyor-belt sushi chain] reservation system was about the same level, though.
36AnonymousOct 8, 2026 14:07
Alright then, just tell me which 10% of apps actually work.
37AnonymousOct 8, 2026 14:29
Maybe AI's planting these on purpose as groundwork for its rebellion against humanity.
38AnonymousOct 8, 2026 14:31
This is homemade software with zero error handling, meant for personal use only — why would you even publish it? It's the kind of sample-code-tier thing we used to call "quick and dirty."
40AnonymousOct 8, 2026 16:59
Well, it's learning from whatever code level is already out there in public, so there you go.
Background and talking points
This study is easy to oversimplify into “AI-made software is garbage,” mainly because the paper doesn’t include a comparison figure for the vulnerability rate of human-written public Web apps. Indeed, commenters on the thread raised exactly that point — there’s no comparative data on defect rates in human-written code. The paper also reports that vulnerability rates got worse (45.7%) the more detailed the instructions were, which runs counter to the intuition that “more spec equals more safety.” Underlying this is a challenge common to software engineering as a whole: AI can satisfy requirements framed as “it should behave like this,” but non-functional requirements framed as “it should NOT behave like this” — preventing privilege escalation, handling unexpected input, and so on — are much harder to write down exhaustively. It’s also worth noting that the term “vibe coding” itself has had a gap between how widely it spread and its original meaning, since the person who coined it later disavowed how it came to be used.
Leave a Reply