Jev vs deck-31B
Frozen Gemma-4-31B-it run in TorchAO FP8, second on Capability.
Jev 1.13.0
TypeSafe · hosted System One model
77.1Capability · unranked reference
Intelligence 63.6 · calibration 90.6.
deck-31B
krishna765 · Frozen Gemma-4-31B-it run in TorchAO FP8, second on Capability
77.6Capability #2
Capability #2 at 77.6. Intelligence 73.0, calibration 82.2.
Key differences
- Where it runs JevHosted APIdeck-31BSelf-hosted frozen Gemma-4-31B-it, TorchAO FP8 dynamic
- Open weights JevNo public Jev weightsdeck-31BBenchmark Heaven marks the evidence as unknown
- Capability (v1.6.1) Jev77.1 · unranked referencedeck-31B77.6 · rank #2
- Intelligence / calibration Jev63.6 / 90.6deck-31B73.0 / 82.2
- Cost and latency Jev$0.032 · 0.2 sdeck-31B$0.048 estimated · 0.4 s
JevBench v1.6.1, same method
Capability is the mean of intelligence and calibration, ranked only inside Benchmark Heaven’s cost and median-latency caps. Jev 1.13.0 stays on the board as the reference row and is not given a rank. A model can score above that reference and still be left off the numbered list when its estimated cost is too high. Full table ↗
Capability axes, higher is better
Cost $0.048 per 1,000 decisions (estimated). Median latency 0.4 s. Jev’s reference row is $0.032 per 1,000 decisions (not published as a list price), median 0.2 s. Both dollar figures are the benchmark’s, except Jev’s, which Benchmark Heaven marks as not a published list price.
What deck-31B is
deck-31B is krishna765’s frozen Gemma-4-31B-it decision system, measured by Benchmark Heaven in TorchAO FP8. It does not fine-tune the base. On the v1.6.1 Capability board it sits at #2 with 77.6, a hair above Jev’s 77.1 reference.
The two scores are not the same shape. deck-31B’s intelligence is 73.0 against Jev’s 63.6. Jev’s calibration is 90.6 against 82.2. If you auto-act on a probability threshold, that calibration gap is the whole decision.
Read this before you switch
- Benchmark Heaven did not record the weights as confirmed open. Read the licence before you ship it.
- Calibration 82.2 is the weak axis. A stated 0.9 is less trustworthy here than on Jev.
- Estimated cost is higher than Jev ($0.048 vs $0.032 per 1,000 decisions) and the median call is slower (0.4 s vs 0.2 s).
When to use which
Choose Jev if
- You threshold on probabilities and need calibration near 90.
- You want a hosted model with nothing to quantise or pin.
- Cost and median latency are part of the choice, not just the right label.
Choose deck-31B if
- Intelligence on this board is the constraint, and 73.0 against 63.6 is the gap you care about.
- You already run Gemma-4-31B and can add a frozen decision readout in FP8.
- You have checked the licence yourself and it fits the deployment.
FAQ
Does deck-31B beat Jev?
On Capability, barely: 77.6 vs 77.1. It is well ahead on intelligence (73.0 vs 63.6) and well behind on calibration (82.2 vs 90.6).
Is deck-31B open source?
Benchmark Heaven lists the open-weights evidence as unknown. Treat it as unconfirmed until you have the author’s licence in hand.
Who made deck-31B?
krishna765. The measured setup is a frozen Gemma-4-31B-it with TorchAO FP8 dynamic quantisation.