one flow.
one request.
one verdict.
An intrusion detection system that asks TypeSafe's Jev two typed questions about one network flow and gets a verdict back, with no text to parse.
Put to the test on NSL-KDD, the classic intrusion detection benchmark, against an LLM (GPT-5.6 Luna) and a Random Forest.
-
4.8×
faster per verdict
than GPT-5.6 Luna -
3.8×
cheaper per verdict
than GPT-5.6 Luna -
1.5×
more zero-day attacks
caught than GPT-5.6 Luna -
15×
fewer false alarms
than a Random Forest
input
flow one connection record from NSL-KDD, plus k labeled examples
request: two typed questions to jev
is_attack noul Is the connection an intrusion attempt?
category choice Which category does it belong to?
output
is_attack → 0.82
category → dos, confidence 0.84
verdict → attack p_attack ≥ 0.5, the same cut for every detector
the request
Jev IDS runs on a new kind of AI, the System One Model (SOM). Instead of generating text, a SOM takes a state and standardized questions about it and answers each one with a probability. Jev IDS hands it one monitored network flow and gets back one verdict, attack or normal, with the probability behind it.
-
input
one flow
The monitoring record of a single connection: duration, protocol, service, bytes in and out.
-
jev
one request
State: the instructions, a few labeled examples and the flow. Questions:
is_attackandcategory. -
output
one verdict
attack · p_attack 0.82
dos · confidence 0.84
Everything Jev is told lives in one versioned file,
prompts/nsl-kdd/jev.json. Python adds only the flow and the examples, and the file's sha256
rides with every prediction, so any verdict traces back to the exact
prompt. Examples are labeled by category only; attack names never
reach a model. Both questions are answered in parallel over the same
state, in one round trip.
Two baselines follow the same protocol: an LLM agent with a JSON output schema and a scikit-learn Random Forest. Same frozen split, same examples, same seeds.
evidence
Pilot split of NSL-KDD: 300 flows, 160 of them attacks and 39 of those of a kind absent from KDDTrain+. Three seeds of examples, so 900 predictions per detector and k. Means over the three seeds, from the runs of 2026-09-21.
| detector | k | f1 | precision | recall | novel recall | latency | cost / 1M flows |
|---|---|---|---|---|---|---|---|
| jev (typesafe-ai/jev) | 1 | 0.859 | 0.941 | 0.790 | 0.838 | 504 ms | $74 |
| gpt-5.6 (gpt-5.6-luna) | 1 | 0.776 | 0.914 | 0.675 | 0.547 | 2,410 ms | $283 |
| random forest (100 trees) | 1 | 0.728 | 0.574 | 1.000 | 1.000 | 3 ms | local |
| jev | 2 | 0.839 | 0.929 | 0.765 | 0.761 | 494 ms | $106 |
| gpt-5.6 | 2 | 0.807 | 0.930 | 0.715 | 0.573 | 2,476 ms | $399 |
| random forest | 2 | 0.766 | 0.629 | 0.990 | 0.991 | 2 ms | local |
Jev against the Random Forest at k = 1: of 900 paired verdicts, 439 differ. Jev is right in 338 of them and the forest in 101 (McNemar p ≈ 6 × 10⁻³¹). A forest trained on five rows calls almost everything an attack, which is why its recall is perfect and its precision is not: on the 420 benign flows it raised 362 false alarms, Jev 24.
limits worth knowing
-
Pilot numbers, taken to settle the protocol. Final results will come
from the disjoint
papersplit with k up to 16. NF-UQ-NIDS-v2 has not run yet. - Latency is the wall clock of the successful HTTP call through the Vercel AI Gateway. 1,350 of the 1,800 Jev rows needed a retry, and retries are not counted.
- Cost is tokens times list price, not what was billed: $0.042 per million input tokens for Jev, output free. Jev was free under a promotion until 2026-09-25, and GPT-5.6 ran through the ChatGPT Codex backend at the public API rate.
-
The gateway masks Jev's version as
typesafe-ai/jev; TypeSafe direct reportsjev-1.13.0. The run date is the only pin. -
Jev's
noulanswer carries no confidence; only thechoiceanswer does.
try it
git clone https://github.com/jev-ids/jev-ids.git
cd jev-ids
uv sync
.env: AI_GATEWAY_API_KEY for Jev through the Vercel AI Gateway;
DEEPSEEK_API_KEY and CHATGPT_CLIENT_ID only for the LLM baselines.
Download
NSL-KDD
into data/raw/nsl-kdd/ and prepare it once.
uv run python -m scripts.prepare_nsl_kdd
uv run jev-ids run --dataset data/nsl-kdd/dataset.json --detector jev --split smoke --k 0,1
The smoke split is five flows. The run writes
results/<timestamp>-nsl-kdd-jev-smoke/ with
config.json, one JSON row per flow in
predictions.jsonl and the raw answers in
responses.jsonl.
bring your own flows
Jev IDS is an independent research prototype, not a product. Its numbers come from one pilot on NSL-KDD. It can still sit inside a commercial solution, and this is how.
where jev fits
A commercial IDS already has sensors, a flow exporter and a signature engine feeding a SIEM. Jev IDS replaces none of them. It sits beside the pipeline, off the packet path, and judges one flow record at a time.
-
Second opinion on alerts. Send Jev the flow behind
each alert the signature engine raised.
p_attackranks the queue, and the SOC reads the top first. In the pilot, Jev raised 24 false alarms on 420 benign flows where a Random Forest raised 362. - A net behind the signatures. Signatures miss what they have never seen. Sample the flows the engine passed as clean, or every flow to a critical asset, and let Jev judge them: it caught 84% of attacks of a kind absent from its examples.
-
A category for the playbook. The
choiceanswer names the category with a confidence, so the SIEM routes dos, probe, r2l and u2r, or your own taxonomy, to different runbooks with no parser in between. - Coverage from day one. A new site or tenant has no training set. Jev needs one labeled flow per category, so it covers the segment while a classical model is still collecting data.
Half a second per verdict and a rate-limited gateway make this an
asynchronous side channel, fed from the exporter (NetFlow, IPFIX, Zeek
conn.log) through a queue, never an inline filter.
six steps
-
Get a key. Jev is served through the Vercel AI
Gateway; set
AI_GATEWAY_API_KEYin.env. Pricing and terms are TypeSafe's and Vercel's. -
Describe your flows. Write a dataset card like
data/nsl-kdd/dataset.json: the columns your flow exporter emits, in order, which of them are symbolic, your categories and which one is benign. -
Write the request. Copy
prompts/nsl-kdd/jev.json, replacecolumnsand the category descriptions with yours, and keep the two questions. - Pick examples. One labeled flow per category from your own network is enough to start; k = 1 is what the pilot used.
-
Call the detector.
JevDetector(load_prompt(path)).predict(flow, examples)returnsp_attack,category_predandconfidencefor one flow in about half a second. Routep_attackto your alerting with the cut your alarm budget allows; 0.5 was the benchmark's choice, not a rule. -
Measure before you trust. Run
jev-ids runandjev-ids metricson a labeled split of your own flows. The numbers above are NSL-KDD's, not yours.
Only flow features leave your network, never payloads, but they do leave it: every request goes to the gateway.