TL;DR

cloudflare/security-audit-skill is the open-sourced prompt pack that seeded Cloudflare’s fleet-wide vulnerability discovery harness. Install it into Claude Code, Codex, or any agent with parallel sub-agents, type security audit this codebase, and it runs a six-phase workflow: reconnaissance, coverage-led hunting by isolated agents, adversarial validation where a different agent tries to disprove every finding, schema-validated findings.json, a second independent verification pass, and a target-neutral report. It is MIT-licensed and hit 17,270 stars by 2026-09-20 after a 211-point Hacker News thread on 2026-09-17.

Key facts:

  • 17,270 stars, 947 forks, 42 open issues as of 2026-09-20; repo created 2026-06-18 alongside Cloudflare’s Build your own vulnerability harness post; last push 2026-09-14
  • Not a scanner — it’s ~300 KB of Markdown prompts (15 files) plus two zero-dependency Node validators (validate-findings.cjs, validate-coverage-ledger.cjs) and a JSON schema
  • Three verdicts only: confirmed (full source trace + observed result), needs_validation (one exact unresolved fact, no severity), rejected (disproved, kept so future runs don’t repeat it)
  • Hard requirement: an OS-enforced sandbox with no network for anything that executes target code; without it, leads stay needs_validation rather than being run
  • Cost is the loudest complaint: 150K+ tokens on a small FastAPI project, one report of 1M tokens with nothing to show, and Cloudflare’s own admission that a single run finds about half of what repeated runs find
  • Independent blind test (issue #20): median 90% precision, zero hits on decoy findings, but 0% on dependency CVEs by design

Install: npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit.

What it actually is

There is a temptation to file this next to Strix or Codex Security as “another AI vulnerability scanner.” It isn’t. There is no binary, no model, no runner. The repo is a single directory, skills/security-audit/, containing:

FileSizeRole
SKILL.md22 KBOperating modes, sandbox rules, write isolation, run profiles, budget gate
RECONNAISSANCE.md16 KBPhase 1 prompts → architecture.md + coverage-ledger.json
HUNTING.md22 KBPhase 2 hunter prompt contract, coverage-critic waves
ATTACK-CLASSES.md16 KBInjection, access control, resource handling, crypto, business logic, feature abuse, chained trust, wildcard, “obvious things”
10 companion files7–12 KB eachMemory safety, AI/LLM, web protocol & auth, client-side, supply chain, cloud, RPC/messaging, resource exhaustion, data isolation, desktop/mobile IPC
VALIDATION-AND-REPORTING.md18 KBPhases 3–6
report-schema.json14 KBThe contract for findings.json
validate-findings.cjs, validate-coverage-ledger.cjs29 KB + 34 KBMechanical checks, with test suites

The design premise, spelled out in Cloudflare’s blog, is that “the real value lives in the prompts themselves.” Cloudflare started with a ~450-line skill, tuned it until it surfaced real bugs in one repo, then lifted each phase into a separate agent backed by SQLite to scan 128 repositories. The prompts in that harness “carry the initial skill’s attacker scenarios, bug classes, and anti-pattern detections nearly unchanged.” What you’re downloading is the seed, cleaned up.

The six phases

The skill is agent-neutral by construction. It talks about a parent (the coordinating agent that owns shared state), a Task tool (whatever your platform calls sub-agent delegation), and two delegated roles: research (read-only source exploration) and general (broad investigation plus bounded local execution). Map those onto Claude Code’s subagents, Codex’s, or whatever you run.

  1. Reconnaissance. Parallel research agents map architecture, trust boundaries, input surfaces, and prior evidence into architecture.md. The parent then builds coverage-ledger.json — a deterministic list of units (subsystem × boundary × attack class) and runs validate-coverage-ledger.cjs on it.
  2. Coverage-led hunting. The parent assigns planned ledger units to isolated general hunters. Each hunter prompt has a fixed nine-part structure: role preamble, architecture.md verbatim, assigned coverage IDs, the selected attack-class blocks copied verbatim (never by name), explicit exclusions, the core hunting method, validation rules, prior-run exclusions, and the structured-result contract. After a wave, coverage critics look for gaps and trigger another wave.
  3. Candidate validation. Every unique candidate goes to a fresh verifier that didn’t hunt it, whose job is to disprove it. It must re-read every cited source location and independently re-run any decisive check.
  4. Structured output. The parent writes findings.json and runs validate-findings.cjs. Malformed or prose-wrapped agent results are discarded, not repaired.
  5. Independent record verification. One more fresh research verifier per final record checks the structured record, not the hunter’s write-up. If a replacement promotes a verdict or materially changes the trace, it goes to yet another verifier.
  6. Reporting. REPORT.md, FINDINGS-DETAIL.md, and NEEDS-VALIDATION.md are derived from the verified records. Prose never changes a verdict.

The hunting method itself is worth quoting because it’s the opposite of checklist review:

WORK FROM A CONCRETE INVARIANT:
1. Name the lower-trust principal and starting capability.
2. Name the accepted value, action, state transition, or resource selector.
3. Locate the control that should reject, bind, isolate, limit, or revoke it.
4. Trace the exact source path after that decision.
5. Stop at the smallest affected dummy record, wrong return value,
   process-integrity effect, or locally observable shared-resource effect.
6. State a source-level change and regression case that enforce the invariant.

And the “Obvious things” class carries the rule that kills most false positives: “A flag is not a finding — trace the impact before reporting.” Missing HttpOnly? Check whether the cookie holds anything sensitive and whether JS reads it by design first.

Setup and first run

# project-level
npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit

# or user-level
npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit --global

Then, inside your agent, any of:

security audit this codebase
find security vulnerabilities in ./src
do a security review, output to ~/audits/my-project

Two things to know before you hit Enter. First, the skill has two modes. A plain security question or a focused review triggers guidance mode — it uses relevant parts of the methodology, launches focused agents if useful, and writes no files. Only an explicit “audit / pen-test / full review / give me a report” request triggers full audit mode, which runs all six phases. If your request is ambiguous, it’s instructed to ask one question first.

Second, full audit mode writes outside your repo by default, to ~/security-audit-skill/<repo-name>/run-<N>, and refuses to write inside the target unless the directory is git-ignored. The output tree looks like:

~/security-audit-skill/my-api/run-1/
├── run-metadata.json        # run_id, repo, source_ref, profile, scope_paths, budget
├── architecture.md
├── coverage-ledger.json
├── findings.json
├── REPORT.md
├── FINDINGS-DETAIL.md
├── NEEDS-VALIDATION.md
└── agents/
    ├── hunter-auth-01/
    │   ├── scratch/         # the only place the agent (or target code) may write
    │   └── artifacts/       # parent-owned; promoted from scratch via allowlist
    └── verifier-auth-01/

The profile matters for cost. quick runs one hunter wave, one critic pass, and merges Phases 3 and 5 into one verifier per candidate. standard is the workflow as written. deep splits ledger units per subsystem and lifecycle mode and runs critic waves until clean. You can also scope a run to named paths, one subsystem, or the diff between two refs — everything else is recorded out_of_scope, never covered. And you can set a budget as a maximum number of agent invocations; the parent has to reserve four recon calls, the critic passes, and at least one verifier before launching anything, or refuse to start.

Practical advice from the HN thread, which matches the skill’s own design: “These work best on a targeted section of the code, like a PR.” Start with quick on a scope, not standard on a monorepo.

What a finding looks like

The schema is the most opinionated part of the repo. A confirmed record requires, among other things: a stable fingerprint, root_cause, intended_behavior, a trace array where every entry is typed entrypoint | propagation | sink with file and line, evidence with file/line, typed conditions, an execution block, remediation, and severity as likelihood × impact.

{
  "verdict": "confirmed",
  "fingerprint": "api/export:idor:tenant_id",
  "title": "Export endpoint accepts caller-supplied tenant_id",
  "root_cause": "handleExport reads tenant_id from the request body instead of the session",
  "intended_behavior": "Exports are scoped to the authenticated tenant",
  "trace": [
    { "kind": "entrypoint", "file": "src/api/export.ts", "line": 41,
      "scope": "handleExport", "description": "tenant_id parsed from JSON body" },
    { "kind": "propagation", "file": "src/api/export.ts", "line": 58,
      "scope": "handleExport", "description": "passed unchanged to buildExportQuery" },
    { "kind": "sink", "file": "src/db/export.ts", "line": 12,
      "scope": "buildExportQuery", "description": "WHERE tenant_id = $1 with attacker value" }
  ],
  "evidence": [
    { "file": "src/api/export.ts", "line": 41, "description": "no comparison against session.tenantId" }
  ],
  "conditions": [
    { "kind": "authentication_level", "description": "any authenticated user of any tenant" }
  ],
  "execution": {
    "attacker_perspective": "Low-privilege user in tenant A",
    "payloads": ["{\"tenant_id\":\"tenant-b\",\"format\":\"csv\"}"],
    "instructions": ["POST /api/export with the payload using a tenant-A session (local fixture)"],
    "observed_result": "CSV containing the dummy tenant-B rows from the fixture"
  },
  "remediation": {
    "strategy": "Derive tenant_id from the session; reject body-supplied values",
    "code_changes": [{ "file_name": "src/api/export.ts", "fixed_code": "const tenantId = session.tenantId;" }]
  },
  "severity": { "likelihood": "high", "impact": "high", "rating": "high" }
}

Note what the schema prevents. A needs_validation record has no severity field at all — you cannot assign “High” to something you couldn’t demonstrate. The execution block must describe an observed result against a local fixture with dummy principals; the universal safety rules forbid probing deployed endpoints, spending paid API quota, or installing dependencies. And the validator enforces the shape mechanically:

node ~/.claude/skills/security-audit/validate-findings.cjs ~/security-audit-skill/my-api/run-1/findings.json

The validators also carry their own paranoia: input capped at 5 MB, nesting depth 64, Windows reserved device names rejected in paths, control and format characters stripped from diagnostics. Someone at Cloudflare has clearly thought about the audit output itself being a prompt-injection vector.

The sandbox requirement, explained

The README’s “Requirements” section stops people cold: an OS-enforced sandbox for target-controlled builds, tests, processes, browsers, emulators, fuzzers, and fixtures. It must disable external networking, use a sanitized allowlisted environment, enforce resource limits, and allow writes only to assigned scratch paths.

The HN thread asked why. The answer is in SKILL.md: source inspection is read-only and always allowed, but the moment a hunter wants to run target code to prove a bug — compile a parser fragment, spin up a fixture server, fuzz a decoder — it may only do so under those four controls. If it can’t, the lead is recorded as needs_validation with the missing sandbox capability as the blocker and a safe validation plan. The skill degrades gracefully rather than executing untrusted code on your workstation.

Cloudflare’s harness built this on unshare; their blog notes the biggest jump in finding quality came from giving hunters a sandbox to crash binaries in, and warns that if you run inside Docker you need seccomp=unconfined and apparmor=unconfined or nested sandboxing silently fails. The skill even specifies the artifact-promotion procedure — eleven steps with O_NOFOLLOW, fstat, link-count checks, and byte limits — so that a hostile build can’t smuggle a symlink into retained output. Issue #31 is about exactly that code hanging on platforms without O_NOFOLLOW/O_NONBLOCK.

What the community is saying

The 2026-09-17 HN thread (211 points, 38 comments) split cleanly.

  • Token cost dominates. “At least 150k on my relatively small FastAPI project, but hit my session limit.” Another: “I threw 1M tokens for nothing in a medium codebase.” A third pointed out that 150K is “where you start hitting the ‘dumb zone’.” Nobody disputed the numbers; the counter-advice was to scope to a PR.
  • Context pollution. Several commenters objected to “very huge markdown files” and “dumping 14 full schemas into the prompt.” That’s partly a misreading — the hunter prompt copies only selected attack-class blocks, and only the confirmed/needs_validation schema branches — but SKILL.md alone is 22 KB and gets loaded on any security-flavored request.
  • Model refusals. A security professional noted that framing a task as “security research” sometimes triggers refusals from top OpenAI and Anthropic models; their workaround is bug-class-specific skills without the security framing, plus a combining skill. Another replied that going through the vendors’ security-researcher validation removes the refusals, though a third said their applications sat unanswered for weeks.
  • Cloudflare listens. A tangential complaint about Cloudflare’s other skills repo polluting context got a reply from Kenton Varda: “I forwarded this to the right person, and it sounds like it’s being worked on.”
  • Alternatives surfaced. Synthesia shared their in-house audit-skill recipe as a cheaper option; another commenter built atgreen/secscan-skill on Visa’s open-sourced Glasswing harness.

On the issue tracker, the most useful thread is #20, a blind seeded-target comparison (n=3 per arm, one target, pre-registered answer key, scorer blind to the arms). The tool “did well on what it’s built for: precision held at a median of 90% across rounds, zero hits on either deliberate look-alike decoy in any round, and every claim carried a source trace.” The gaps: a real disclosed CVE in a pinned dependency went unfound in all three rounds (the sandbox correctly declined to guess at external facts), and locale-specific validator bugs “never became a ledger unit at all” — not deferred, not out of scope, just unmapped. #11 reports Google’s Antigravity killing the parallel sub-agents with Gemini 3.1 Pro. #21 reports the coverage validator accepting nonexistent local-check artifacts.

Honest limitations

  • It is expensive, and one run is half a run. Cloudflare’s own number: a single run finds roughly half of the bugs found across repeated runs, skewed toward the simpler ones. Their conclusion was to build a persistent harness. Yours might be to run quick on every PR and deep quarterly.
  • No dependency-CVE coverage, and the README doesn’t say so. By design the skill won’t guess at advisories it can’t verify from source. Pair it with npm audit, pip-audit, or Trivy.
  • Coverage is only as good as recon. Issue #20’s second gap — defects outside the attack-class taxonomy leaving no trace on the ledger — is the scary one, because the ledger is what tells you what was checked.
  • Platform dependence is real. It needs true parallel sub-agents. Antigravity reportedly kills them; single-agent tools will serialize and blow context. Claude Code and Codex are the tested paths.
  • The sandbox is on you. No sandbox means no dynamic confirmation; you’ll get a NEEDS-VALIDATION.md full of plausible leads and a REPORT.md that’s thinner than you hoped.
  • Refusals happen. The vendor models occasionally balk at “find exploits” framing regardless of how defensive the prompt is.
  • The harness isn’t released. The blog says “hopefully the harness itself will follow shortly.” Three months later, the SQLite orchestration, dedup agents, cross-repo tracer, and the wishlist mechanism remain Cloudflare-internal.

How it compares

Cloudflare security-audit skillStrixCodex SecuritySemgrep / CodeQL
What it isPrompt pack + validatorsRunnable agent w/ browser + proxyOpenAI CLI + serviceStatic analysis engines
Runs onAny agent with sub-agentsIts own harnessCodexCI
ApproachSource-first, invariant tracing, adversarial verifyDynamic pentest against running appScan + validate in sandboxPattern/dataflow rules
Outputfindings.json (schema), 3 reportsFindings + PoCsFindings + patchesSARIF
Executes target codeOnly in OS sandboxYes, against targetYes, sandboxedNo
Cost modelYour tokens, unboundedYour tokensOpenAI creditsFixed
LicenseMITApache-2.0ProprietaryLGPL / MIT

The most telling data point from Cloudflare’s harness: they plumbed Semgrep all the way through and “the Hunters invoked it zero times in a month of runs. They would rather read and run the code.” Static analysis and this skill aren’t substitutes — but the agents don’t reach for SAST when they have source and a sandbox.

FAQ

Does it work with Claude Code, Codex, or Cursor?

It’s written to be agent-neutral and requires only a model with tool use and a parallel sub-agent mechanism. npx skills add handles the per-agent install location. Claude Code and Codex are what the community reports success with; Antigravity is reported broken (issue #11).

How much does a run cost?

There’s no fixed number. HN reports range from 150K tokens (small FastAPI app, standard, hit the session cap) to 1M tokens on a medium codebase. Use quick, scope to a subsystem or diff, and set an explicit invocation budget; the parent will refuse to launch if the budget can’t cover recon + critics + one verifier.

Do I need the sandbox to get anything useful?

No — source-only runs still produce traced findings. But anything that requires executing target code to confirm stays needs_validation, so you’ll get more leads and fewer confirmed records. The sandbox needs no network, an allowlisted environment, resource limits, and scratch-only writes.

Is it a replacement for Semgrep, CodeQL, or a pentest?

No. It doesn’t do dependency-CVE lookups, it’s non-deterministic (run it more than once), and it never touches deployed systems. Think of it as a very thorough code reviewer with a strict evidence bar, not a scanner or a red team.

Can I run it repeatedly on the same repo?

Yes, and you should. Runs are additive: the parent reads prior coverage-ledger.json and findings.json, carries forward confirmed records whose source is unchanged (still re-verified), creates revalidation units for changed source, and never treats a prior quick or scoped run as “the rest is fine.”

Bottom line

Most “AI security” launches are a scanner with a model behind it. This is the reverse: no scanner, just the methodology Cloudflare used to produce 20,799 raw candidates and 7,245 actionable findings across 145 repositories, encoded as prompts with a machine-checked evidence contract. The evidence bar — named principal, traced invariant, observed result, adversarial disproof by a different agent — is the best I’ve seen in an open skill, and the blind test in issue #20 backs it up at 90% precision.

The costs are equally real: it eats tokens, one run is half a run, it can’t see CVEs, and the ledger only covers what recon thought to map. If you run Claude Code or Codex on a codebase that matters, install it, run quick scoped to your auth or export layer, and read NEEDS-VALIDATION.md as carefully as REPORT.md. If you’re expecting a npm audit replacement, this isn’t it.

Repo: github.com/cloudflare/security-audit-skill. Install: npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit.

Sources