The security holes AI code has by default

Why this keeps happening

A model optimises for code that works when you try it. You try it as yourself, logged in, doing the intended thing. Security is about what happens when someone does the unintended thing, and nobody in the loop tested that.

These four are not exotic. They are in a large share of AI-built apps right now.

1. Secrets in the browser

The fastest way to make an API call work is to put the key where the calling code is. If that code runs in the browser, the key is public. Not hidden. Not obscured. Anyone can open developer tools and read it.

Watch for a key inside a "use client" component, or an environment variable whose name starts with NEXT_PUBLIC_, VITE_, or REACT_APP_. Those prefixes exist precisely to mean “ship this to the browser.” A key with that prefix is published.

The fix is structural: the browser calls your server, your server holds the key and calls the provider. “Move this API call to a server route so the key is never sent to the browser” is a request the model will handle well, once asked.

The other route is a .env file committed to a public repository. Automated scanners find those within minutes of the push, and the first sign is usually the bill. If it happens: rotate the key immediately. Deleting the file does not help, because the value is still in the history.

2. Permission checks that only exist in the UI

Hiding a Delete button from users who should not delete is not a permission check. The endpoint still accepts the request from anyone who sends it, and sending it takes about fifteen seconds with the browser’s own network tools.

So the question for every endpoint is: who is allowed to call this, and which line on the server enforces it?

And there are two separate questions hiding inside that. Are you logged in is authentication. Are you allowed to touch this particular thing is authorization. AI code very often has the first and not the second.

3. The one that is everywhere

// What the model wrote
app.get('/api/invoices/:id', requireLogin, async (req, res) => {
  const invoice = await db.query(
    'SELECT * FROM invoices WHERE id = $1',
    [req.params.id]
  )
  res.json(invoice)
})

This checks that you are logged in. It never checks that the invoice is yours. Change the number in the URL and you are reading another customer’s invoice. Every customer’s, one at a time, with a script.

// What it needed
app.get('/api/invoices/:id', requireLogin, async (req, res) => {
  const invoice = await db.query(
    'SELECT * FROM invoices WHERE id = $1 AND user_id = $2',
    [req.params.id, req.session.userId]   // from the session, never from the request
  )
  if (!invoice) return res.status(404).json({ error: 'Not found' })
  res.json(invoice)
})

One extra condition. Note where the user id comes from: the session on the server. If it arrives in the URL, the body, or a header the browser set, the caller controls it, and a value the caller controls proves nothing.

This is invisible during normal use because you only ever test with your own account and your own data. Make a second account. Log in as user B and try to open user A’s things by editing the URL. Do this before launch, every time.

4. Injection

User input treated as instructions rather than data.

The classic is SQL built by gluing strings together: "SELECT * FROM users WHERE email = '" + email + "'". Someone types a quote mark and the rest of their text becomes part of the query. Modern libraries mostly prevent this, so look specifically for string concatenation or a raw(...) call near a query.

The newer version: your app passes user text to a model that can use tools or read private data. Text in a document or a message can carry instructions, and the model may follow them. If your app has an AI feature that can act (send, delete, fetch) treat every piece of text it reads as written by someone who wants something from you.

The review that catches most of it

Before anyone else uses your app, spend twenty minutes:

1. Open developer tools, Network tab, and use your app. Look at what is sent. Any key visible is public.
2. Search the code for sk-, key, secret, password. Anything in a client file is exposed.
3. Make a second account. Try to reach the first account’s data by changing ids in URLs.
4. Confirm .env is in .gitignore and was never committed.
5. Ask for a specific review, not a general one: “For every API route, tell me which line checks that the requester owns this record. List any route where there is no such line.”

General requests to “check security” produce reassurance. Specific ones produce findings.