Sean Percival

Essays · October 2026

Everyone Can Build with AI. But Should They?

Brussels
A Bruegel-style oil painting of the Tower of Babel under construction, scaffolding and hundreds of workers swarming its arches, a group of men in Renaissance dress studying plans on a table in the foreground, a harbor of sailing ships below, and the starship Enterprise gliding through the clouds overhead

I have very fond memories of The Next Generation. (And a moment here for Sir Patrick Stewart, who died this week. Picard was the first boss I ever wanted to work for, and I was like fifteen.)

It was not the phasers or the transporters that stayed with me. It was the computer. Someone would walk onto the bridge, say “Computer, run a simulation of the hull under these conditions,” and it would just do it. A full program, analysis, charts, from a sentence. No ticket, no sprint, no one asking which environment.

Now we basically have that. A few lines of instructions, or you just talk to the damn thing, and you can make anything from a simple script to a tool your whole department runs on. It is the most fun I have had with a computer since Myspace let me set a tiled jpeg as the background.

So the question is whether that is a good thing.

Annoyed in Unboxed

I was at the Unboxed Summit recently, an e-commerce event, and one of the keynotes, from Michael Connolly, a VP at Bloom & Wild, had a line that stuck. Just because everyone can build with AI doesn't mean everyone should.

This annoyed me, because it is the opposite of what I have been saying. To my team, to strangers, to anyone who stands still long enough. Need a solution? Build it. Don't wait for a vendor. Don't wait for IT. Just build.

I still believe that. The people who build their own tools are faster, they understand their own problems better, and they send me fewer Teams messages that start with “quick question.” I am not giving that up.

But he is right about the other half, and it took me a week to admit it.

Getting output from AI is easy now. A report, a summary, a slide, a chart. Everyone can do that and most people do. But the gap between getting output and building something, meaning an app, a service, a thing with a URL that other humans touch, is still enormous. I have conversations every week with smart people who do not know what is local, what is on the internet, and what is an artifact. These are three different places. Things can leave one and end up in another, and once they're on the internet they are, by definition, on the internet.

I would not want most of them pushing too many things live. The failure modes are real: security, data privacy, customer data in a Supabase table with the default policies still on. The only reason I feel comfortable shipping is that I have made most of these mistakes already, some of them in production, at least one of them on a Friday.

Why nobody has blown up yet

We are not hearing many vibe-coding horror stories. Not many “intern pushed the customer database to a public link” headlines. I think that is because most people are still on the holodeck, and not by choice. They don't know how to make the leap from the Claude app to the open web. The artifact stays an artifact. The safeties are on because nobody has found the switch.

For right now, that is a feature.

Because we already know what AI is doing to the security of the web. PHP2Shell dropped two weeks ago: two critical holes in the language running 70 percent of websites, found by AI models, sitting there for over a decade, patched in an emergency release while government sites went dark. Multiple frontier models have been gated this year because they are, functionally, a nuke in the cybersecurity world.

So the attackers got the power tools first. The moment the rest of us start pushing to production in volume, the issues go exponential. Not because people are careless. Because the gap between “I made a thing” and “I understand what the thing can do when a stranger touches it” is still wide, and the tools have not closed it yet.

And before any of that, the two questions nobody asks at eleven at night when the thing finally works. Should this exist at all. And if it should, who is going to maintain it. Because it will need maintaining. The API will change, the token will expire, the person who built it will leave, and the tool their whole department runs on will become the tool their whole department is stuck with. I have a folder of these. Some of them still run. I'm not sure which.

There is a lot of talk about AI slop. Some have used the phrase slop grenades, which I liked. Mostly it means AI junk thrown into the mix of work life. Occasionally it means something with a pin pulled out, sitting in your system with full write access.

Safety protocols

So just because everyone can build doesn't mean everyone should. Not yet.

The job for those of us who have been doing this a while is not to gatekeep. It is to teach what it means for something to leave the relative safety of your AI app and go out into the web at large. What a public URL is. What a database policy is. What happens when you give an agent a token with more permissions and agency than you have yourself.

The holodeck had safety protocols on by default. Every episode where something went wrong, someone had turned them off. And “make it so” was always the easy part. Captain Picard also had a Riker standing next to him to say “are you sure” first.