blog

The AI pen test: an agent that attacks your own app

ProductNovember 11, 20266 min

A scanner can tell you a header is missing. It cannot tell you that order #418 belongs to someone else and your API hands it over anyway. That gap — between checking configuration and actually trying things — is where real penetration testing lives, and it is usually expensive and occasional. Today we are closing it in Coaade IDE: the AI pen test, an agent that opens your deployed app and goes after it the way a security engineer would.

It tests, it does not just read

The agent runs in a real AWS cloud browser and can also send crafted HTTP requests. So it does both halves of the job: it clicks through your app like a user, and it pokes your API like an attacker. It maps the surface first, writes a plan, then works through it:

Authentication     sign-in, sessions, cookie flags, logout, weak credentials
Access control     IDOR: change an id and see whose data comes back
Injection          XSS, SQL and template markers — benign, detection only
Data exposure      secrets, stack traces, source maps, published files
Transport          https, HSTS, CSP, clickjacking, CORS, cache-control
QA                 broken flows, dead ends, missing feedback, mobile

It signs up its own throwaway accounts with made-up details, then uses them against each other — which is how it catches the access-control bugs that only appear when two users exist. It reads the page after every action instead of assuming a click worked, and it waits for slow screens before judging them.

Every finding comes with proof

A finding you cannot reproduce is noise. Each one the agent reports carries the request or the steps that produced it, what a correct app should have done, what yours did, and a specific fix:

[HIGH] (access-control) Orders are readable across accounts
  GET /api/orders/418 with account A's session returned
  account B's order, including name and address.
  Expected: 403 or 404. Actual: 200 with another user's data.
  Fix: check ownership on the server for every /api/orders/:id
  read, not just in the UI that lists them.

At the end you get a 0–100 security score and a verdict: secure, issues found, or at risk. The score is derived from the findings, not from the model's mood — a critical finding costs far more than a hardening note, and the agent cannot award itself a clean bill of health while a high-severity issue sits in its own report. Fix, redeploy, run it again, and watch the number move.

Aggressive, but fenced in

An AI that attacks web apps is only a good idea if it cannot be turned on someone else. Two limits are enforced in our code, not left to the model's judgement:

  • Your app only. The target is resolved to your project's own coaade.app deploy or a custom domain you have verified. Any other address is refused before a single request goes out, so Coaade cannot be used as an attack platform.
  • Nothing destructive. Only safe HTTP verbs are allowed — GET, HEAD, POST, OPTIONS. Deletes and overwrites are blocked. Injection probes are benign markers that reveal whether input is escaped; there is no DROP TABLE, no flooding, no touching real users' data.

It proves a weakness exists. It does not exploit it. That is the difference between a pen test and an incident.

One tab with your code review

The IDE now has a single Security & Code review tab. One button scores your code — languages, hardcoded secrets, things to improve — and runs the quick security scan of your live site at the same time. Run AI pen test sits next to it for the deep pass, and findings stream in as the agent works. Fix with AI Chat hands the worst of all three to Coaade Code, which can go and fix them.

It works from your own AI tools too. With Coaade MCP connected, ask ChatGPT, Claude or Cursor to “run a pen test on my site and fix what it finds” and the score and findings come back in the conversation.

Availability

The AI pen test is a paid feature, on Pro and above, like App Test and the security scan — it runs a cloud browser and a capable model for several minutes per run, and it uses your Coaade AI allowance. Connecting your AI tools over Coaade MCP stays free on every plan; the testing and security tools are the paid part. Deploy your project, open Security & Code review in the IDE, and set it loose on your own app.

One honest caveat: this finds the issues a thorough tester checks first, and it finds them every time you ship rather than once a year. It is not a substitute for a manual audit or a bug bounty on a high-risk application.