Jev vs decider-4b
Open Qwen3.5-4B System One rebuild, 8.4 GB of Apache-2.0 weights.
Jev 1.13.0
TypeSafe · System One model
63.3JevBench rank #4
Highest intelligence in the top ten: 53.1.
decider-4b
Mapika · Open Qwen3.5-4B System One rebuild, 8.4 GB of Apache-2.0 weights
64.1JevBench rank #3
Third of 91 on the composite — ahead of Jev on the score, behind it on the answers.
Key differences
- Where it runs JevHosted API, generally availabledecider-4bSelf-hosted on one GPU, about 8.4 GB in bf16
- Licence JevProprietary, hosteddecider-4bApache-2.0 weights and package
- Speed Jev83.3 · 0.65s median over the networkdecider-4b92.9 · 17ms raw on the benchmark’s GPU
- Evaluation-style cases Jev94.5% correctdecider-4b87.7% correct
- Hardest test cases Jev74.1% correctdecider-4b67.3% correct
- Sealed decisions Jev36.7% correctdecider-4b34.7% correct
- Context Jev64k tokens per requestdecider-4b32k tokens
JevBench v1.4.2.2, measured the same way
One benchmark measured 95 systems under one method and ranked 91, so these figures are comparable in a way vendor-published numbers are not. The composite weighs four axes — intelligence, calibration, speed and cost — and hides where systems actually differ, so the charts below break it apart. Full method at the source ↗
Composite score
The top four finish within 4.1 points of each other, then the board falls away sharply.
Capability, higher is better
Accuracy by how hard the decision is
Easy and standard decisions separate almost nothing. The hard tier is where these systems stop agreeing, and the sealed tier shows how much of that holds on questions nobody could have tuned for. A dash means the benchmark published no combined figure. The composite also weighs a fourth axis — cost — which we do not reproduce; see the source table.
Which decider?
Mapika publishes several decider models and releases come quickly. JevBench has ranked three of them. The ranked one is decider-4b v2: Qwen3.5-4B-Base, 4.2B parameters, about 8.4 GB in bf16, released 24 September — the version kept under the Hugging Face tag v2.
decider-4b v2.1 is the current default, released the same day: v2 plus a further LoRA stage. Its author reports it slightly weaker on JevBench’s public hard items (0.649 vs 0.676) and less well calibrated on hard items, so it is not the version the board measured.
decider-35b-a3b (Qwen3.5-35B-A3B-Base, 34.7B parameters with 3B active, ~65 GB bf16 or 19.6 GB NVFP4) ranks #21 at 41.2, held back by cost. decider-2b (Qwen3.5-2B-Base, 1.9B) ranks #41 at 30.7, held back by calibration. decider-0.8b and decider-2b-vision are not on the board.
Where decider-4b falls short
- The ranked checkpoint is v2 — the newer v2.1 default is weaker on hard items per its own author.
- 32k-token context is half of Jev’s 64k request budget.
- Judge-tier accuracy trails Jev by nearly 7 points: 87.7% vs 94.5%.
When to use which
Choose Jev if
- Evaluation-style prompts are your workload: judge-tier accuracy is 94.5% vs 87.7%.
- Your ambiguous cases matter — hard tier 74.1% vs 67.3%, sealed 36.7% vs 34.7%.
- You want a hosted, versioned endpoint instead of GPU operations.
Choose decider-4b if
- You need Apache-2.0 weights on your own GPU — about 8.4 GB in bf16.
- Raw latency matters: 17ms measured raw on the benchmark GPU vs a networked call.
- You want to fine-tune the decision head on your own labels.
FAQ
Is decider-4b better than Jev?
On the JevBench composite, yes — #3 at 64.1 vs Jev’s #4 at 63.3, mostly through speed and cost. On the accuracy tiers Jev leads everywhere it counts: judge 94.5% vs 87.7%, hard 74.1% vs 67.3%, sealed 36.7% vs 34.7%, and intelligence 53.1 vs 49.4.
Which decider-4b version should I run?
JevBench ranked v2 (the Hugging Face tag v2). The newer v2.1 default adds a LoRA stage its author reports is slightly weaker on hard items and less calibrated — check the repo’s notes before defaulting to it.
Is decider-4b open source?
Yes — Apache-2.0 weights and package from Mapika, a Qwen3.5-4B-Base rebuild of the System One request shape. Independent of TypeSafe.
Can decider-4b replace a Jev API call?
It speaks the same state-plus-questions contract at 32k tokens. For judge-style and ambiguous cases, test your own prompts first — Jev leads those tiers by several points.