Skip to main content

Reducing false positives (Jev)

Every autonomous scanner produces some false positives. Pentest Swarm AI already grades and de-duplicates findings before it writes the report — but you can add one more pass that drops findings a second, independent model is confident are not real. That optional pass is Jev.

Jev is TypeSafe's "System One" model. It is not an LLM — instead of generating text, it returns typed, probabilistic decisions in a single fast, cheap pass. Pentest Swarm AI asks it one yes/no question per finding — "is this a real, exploitable vulnerability?" — and gets back a probability for each. Findings below the threshold are removed.

This is opt-in and off by default

The swarm's own grading is the primary quality control. Jev is a second opinion for teams who want to trim the last of the noise before triage. If you don't turn it on, nothing changes.

Turning it on​

You need a TypeSafe API key — get one at typesafe.ai. Provide it via the TYPESAFE_API_KEY environment variable (preferred) or the --jev-key flag.

In the launcher (pentestswarm run)​

The interactive launcher has a Jev FP filter toggle, off by default. Switch it on with the arrow keys. If TYPESAFE_API_KEY isn't already set, a masked TypeSafe key field appears for you to paste the key into; when the env var is present, the launcher uses it and skips the field.

On the command line​

export TYPESAFE_API_KEY=your-typesafe-key
pentestswarm scan example.com \
  --scope example.com \
  --swarm --follow \
  --jev

Or pass the key inline:

pentestswarm scan example.com --scope example.com --swarm --jev --jev-key your-typesafe-key

How it behaves​

  • Where it runs. As the final step of the report agent — after the swarm's findings are graded and duplicates are collapsed, and just before the report is rendered. It sees the clean, de-duplicated set.

  • What it drops. Each surviving finding is scored 0.0–1.0 for P(real vulnerability). Anything below the threshold (default 0.5) is dropped. You'll see a campaign event like:

    Jev false-positive filter: kept 7, dropped 2 (P(real) < 0.50)

  • It fails OPEN. If the Jev API errors, times out, or is unreachable, the filter is skipped and every finding is kept. A transient hiccup will never silently discard a real bug — you'll see a Jev false-positive filter skipped … — keeping all N findings event instead.

  • Preflight. When --jev is set, the run checks the key up front. A missing key stops the run with a clear message; a bad key fails the Jev health check fast, so you find out immediately rather than at report time.

Cost​

Jev is billed by TypeSafe, separately from your LLM provider. It's a single typed pass over the final finding set (not per-token generation), so it's cheap and doesn't count against your spend cap, which only meters LLM spend.

Why not LiteLLM / an OpenAI-compatible endpoint?

Jev isn't a chat model, so it doesn't fit the OpenAI-compatible provider path the other providers use. Pentest Swarm AI talks to its typed /v1/systemone API directly — no LiteLLM or extra proxy required.

Related: adaptive attack-path scoring (--jev-adaptive)​

Jev's fast typed decisions aren't only useful at the end of a run. With --jev-adaptive, the swarm uses Jev during exploitation as a real-time scoring brain: when the exploit agent has several candidate attack strategies for a finding, it asks Jev to score each strategy against live state (what's been discovered, which steps just worked), then pursues the best-scored path first and grades that chain's pheromone by the score — so the swarm's attention concentrates on what's actually landing.

pentestswarm scan <target> --scope <target> --swarm --jev-adaptive

Off by default; needs a TypeSafe key (same as the filter); fails open to a heuristic ranking if Jev is unavailable, so the swarm never stalls. You can run --jev and --jev-adaptive together.