Jev vs Plumb-4B
A LoRA on JevK5 v0.2, small enough to self-host.
Jev 1.13.0
TypeSafe · hosted System One model
77.1Capability · unranked reference
Intelligence 63.6 · calibration 90.6.
Plumb-4B
crh225 · A LoRA on JevK5 v0.2, small enough to self-host
65.4Open-weights Capability #12
Capability #12 at 65.4 on the October board. Composite #2 at 65.8 on the September board.
Key differences
- Where it runs JevHosted APIPlumb-4BSelf-hosted GPU, LoRA on JevK5 v0.2
- October Capability Jev77.1 · referencePlumb-4B65.4 · rank #12
- October intelligence / calibration Jev63.6 / 90.6Plumb-4B43.0 / 87.9
- September composite (v1.4.2.2) Jev#4 at 63.3Plumb-4B#2 at 65.8
- September hard / sealed Jev74.1% / 36.7%Plumb-4B77.7% / 38.0%
- Estimated October cost Jev$0.032 per 1,000 decisionsPlumb-4B$0.017 per 1,000 decisions
JevBench v1.6.1, same method
Capability is the mean of intelligence and calibration, ranked only inside Benchmark Heaven’s cost and median-latency caps. Jev 1.13.0 stays on the board as the reference row and is not given a rank. A model can score above that reference and still be left off the numbered list when its estimated cost is too high. Full table ↗
Capability axes, higher is better
Cost $0.017 per 1,000 decisions (estimated). Median latency 0.2 s. Jev’s reference row is $0.032 per 1,000 decisions (not published as a list price), median 0.2 s. Both dollar figures are the benchmark’s, except Jev’s, which Benchmark Heaven marks as not a published list price.
What Plumb-4B is
Plumb-4B is crh225’s LoRA on allebee’s JevK5 v0.2, which itself is an open Qwen3.5-4B decision model. On the September JevBench v1.4.2.2 composite it was #2 at 65.8, ahead of Jev’s #4 at 63.3, and it led Jev on the hard tier (77.7% vs 74.1%) and the sealed set (38.0% vs 36.7%).
The October board changed the headline. JevBench v1.6.1 Capability — the mean of intelligence and calibration, inside a cost and latency cap — puts Plumb at #12 with 65.4. Intelligence on that release is 43.0 against Jev’s 63.6. Calibration is 87.9 against 90.6. Estimated cost is the friendly part: $0.017 per 1,000 decisions against $0.032, at the same 0.2 s median. The September and October figures are different tests; quote the version next to the number.
Read this before you switch
- Benchmark Heaven marks Plumb’s open-weights evidence as unknown on the v1.6.1 alternatives table. Confirm the licence with the author before you ship it.
- A #2 from September is a #12 on the October Capability list. Both can be true because the score changed.
- Intelligence 43.0 on v1.6.1 is the number to read if the new pool is your workload. The September hard-tier win does not transfer automatically.
When to use which
Choose Jev if
- You are buying the October Capability score: 77.1 vs 65.4, and intelligence 63.6 vs 43.0.
- You want the hosted model and a calibration score of 90.6.
- You do not want to operate a LoRA on top of JevK5.
Choose Plumb-4B if
- Your eval still looks like the September hard and sealed tiers, where Plumb led (77.7% and 38.0%).
- Estimated cost around $0.017 per 1,000 decisions is the constraint, and you can host a 4B LoRA.
- You already run JevK5 and want crh225’s extra LoRA on that base.
FAQ
Did Plumb used to rank above Jev?
On JevBench v1.4.2.2’s composite, yes: #2 at 65.8 vs Jev at #4 with 63.3. On v1.6.1 Capability it is #12 at 65.4, and Jev’s reference is 77.1.
Why did the rank change so much?
The headline changed. September’s composite mixed intelligence, calibration, speed, and cost. October’s Capability rank averages intelligence and calibration inside a cost and latency cap, on a larger decision pool. Speed no longer lifts a small model up the list.
What is Plumb built on?
crh225’s LoRA on JevK5 v0.2, which is allebee’s Qwen3.5-4B decision model. See the Jev vs JevK5 page for the base.