Jev vs JevK5: Hosted Decisions or Open-Weight Control?

Jev vs JevK5 starts with a familiar problem: your app needs one clear answer from a messy piece of text. A customer describes a late delivery and a missing refund. Which team should receive the case? A chatbot could write a helpful paragraph, but your software still has to turn that paragraph into a route. Jev and JevK5 take a shorter path. You give them the text, a focused question, and the allowed answers. They return a structured decision.

The names sound related, but JevK5 is not a Jev release. Jev is TypeSafe AI's hosted decision model. JevK5 is an independent, open-weight project built on Qwen3.5-4B. The most useful Jev vs JevK5 question is therefore practical: do you want a managed model with a larger documented input budget, or do you want to run and inspect the weights yourself? This guide explains the trade-off in plain language, then shows how to try a decision on the Jev AI homepage Playground.

What is JevK5?

JevK5 is a model and runtime for typed decisions. Its project repository publishes Apache-2.0 code and model weights. The weights combine Qwen3.5-4B with a trained LoRA that has been merged into the released model. For each question, JevK5 looks at the supplied answer options and returns a probability for each one without writing a free-form reply. The project uses a one-pass readout adapted from SemIf; it does not use Jev's undisclosed architecture or TypeSafe's weights.

The three question types are easy to picture. Choice picks from named answers, such as Billing, Shipping, or Other. Score rates a case on an ordered scale that you define. Noul asks whether a statement is true and returns a yes/no probability. Suppose a customer says, “The parcel was marked delivered to number 17, but I live at number 71.” Choice could identify a delivery problem; Noul could ask whether the address numbers conflict. Neither answer proves what happened to the parcel. It gives your app a small, testable judgment.

JevK5 can serve a TypeSafe-style request shape, which makes side-by-side experiments easier. Matching input fields do not make two models interchangeable. Their training, probability estimates, and limits differ. If your application will act on a high-confidence answer, test that behavior rather than assuming a compatible JSON response means compatible decisions.

Jev vs JevK5: the main differences

These limits come from TypeSafe's Jev model reference, its Choice guide, and the JevK5 model card. The JevK5 repository now describes a multi-pass method for more than 16 choices, but that is different from selecting 16 choices in one pass. More passes mean more work, and accuracy on large label sets should be checked separately. It would be misleading to treat an updated runtime feature as proof that the older benchmark covered every option count.

What do the published Jev vs JevK5 results say?

The clearest public head-to-head comes from JevBench v1.4. It scored Jev 1.13.0 at 63.29 and JevK5 v0.2.0 at 62.04 on a combined measure of decision accuracy, probability quality, speed, and estimated cost. Jev ranked first and JevK5 second in that version. A gap of 1.25 points on a mixed score is interesting, but it is not a promise that Jev will win every task or that JevK5 will be cheaper in your deployment.

The task-level numbers are easier to use. On 220 hard decisions, Jev was correct on 74.1% and JevK5 on 70.0%. On 146 answer-judging cases, they both reached 94.5%. A fresh set of 308 sealed questions gave 36.7% for Jev and 33.1% for JevK5. Those sealed results are low for both models; the benchmark describes the set as unusually difficult. The useful lesson is that performance depends on the type of decision, not just the model's place on a chart.

Speed needs the same care. JevK5's author reports roughly 13 milliseconds for short decisions on an H100 GPU after loading, measured inside the process. JevBench measured Jev through a hosted API, where network time is part of the experience. Comparing those numbers directly would mix two different routes. If a real user waits for the answer, measure from your app to the result, including queueing and any retry. If you self-host JevK5, also measure how it behaves when several requests arrive together; its reference server evaluates questions separately and serializes requests on one GPU.

Neither model comes with a universal confidence threshold. Jev is trained to return calibrated probabilities, while JevK5 fits a temperature to its option scores. JevK5's own model card reports that its standard-tier answers can be underconfident, even though its hard-tier calibration looks better. A probability of 0.8 is useful only if cases your team sees with that score are right about eight times in ten. Check that with labeled examples before using a threshold to approve a refund or close a case.

One disclosure is worth reading with the scores. The JevK5 maintainer's changelog says some MMLU-Pro test items were included in its training data. That issue concerns MMLU-Pro results; the maintainer says JevBench items were not used for training. It is still a reminder to ask where evaluation examples came from. A score is most useful when the model has not seen the answers and the test resembles your real workload.

When would you choose one over the other?

Jev is an easy starting point for text decisions when you want to avoid operating model hardware. It gives you a hosted route, a versioned model name, and a larger published context budget. That can matter for a long support history or a policy question with several relevant paragraphs. Longer input is not automatically better, though. If the decisive sentence is buried under pages of unrelated text, shorten the state or select the relevant passage before testing.

JevK5 is worth a closer look if local control is essential. You can inspect the code and weights, run the model in your own environment, and decide when to upgrade it. The standard BF16 setup needs about 9 GB of GPU memory according to the project. Its newer GGUF builds offer other local options, including CPU and Mac use, but those are different runtime conditions from the H100 speed report. Local use replaces a hosted model bill with hardware, setup, monitoring, and maintenance work.

Consider the shape of your work as well. A three-team ticket router with short English messages fits JevK5's basic one-pass design. A large taxonomy, multilingual inbox, or long policy packet needs a more careful trial. Jev may be the simpler starting point for those cases, but it should still be tested on your language and category list. And if the task is to write a customer reply, use a writing model after the routing step. Neither Jev nor JevK5 is meant to draft a friendly email.

A fair comparison you can run yourself

You do not need a huge benchmark to learn something useful. Save 30 to 50 cases your team already understands. For each one, record the original text, one question, the allowed answers, and the answer you expect. Include easy cases, borderline cases, and cases where Other should win. Keep the exact same question and option wording for both models. When you compare results, note the selected answer, probability, response time, and whether the model gave a valid response.

Look closely at disagreements. Did the input leave out a key fact? Are two options too similar? Did the model make a confident mistake on a rare but important case? A single accuracy percentage hides those details. If wrong routes cost minutes, you may accept more automation. If a mistake changes a payment or blocks an account, you may want a wider review band. A good Jev vs JevK5 comparison reflects the cost of errors in your own workflow.

You can begin that exercise on this website. Go to the Jev AI homepage Playground, choose Your own case, paste one of your text examples, and write a Choice, Score, or Yes / No question. Press Run Jev when the service is available. Try changing one fact in the message while keeping the question fixed. If the answer does not change when it should, that is a useful case to save for further testing. The What Is Jev? guide explains the question types, and the examples page gives fuller walkthroughs.

The homepage currently runs Jev, not JevK5. Use it to learn what a well-defined decision request looks like; compare JevK5 separately if you run the open model. That keeps the comparison honest and gives you a practical starting point today.

Jev vs JevK5 FAQ

Is JevK5 an official Jev model?

No. JevK5 is independent of TypeSafe AI. It accepts a similar request shape, but its weights and implementation are different.

Is JevK5 free to run?

Its code and weights are published under Apache-2.0. Running them still uses your hardware, electricity, and engineering time.

Is JevK5 faster than Jev?

Its published in-process GPU timing is very fast, but it is not directly comparable to a hosted Jev request. Measure both from the same point in your own application.

Can I use JevK5 in the Jev AI Playground?

The current Playground runs Jev decisions. You can prepare and test the same text question there, then evaluate JevK5 in a separate environment.