Type-Safe LLM Calls: Making AI Agents Return Decisions Code Can Trust

Look closely at almost any AI agent in production and you will find that most of its model calls are not asking for writing. They are asking for a decision. Which department should this email go to? Is this extracted date really the date of birth? Is this code-review finding a real bug? Is the user still typing?

The usual way to get those answers is to prompt a general-purpose LLM, ask it to reply in JSON, parse the reply and hope the label it chose is one you recognise. It works in a demo. In production it gives you three problems: the output is text you have to validate, the "confidence" is a number the model made up, and every decision costs a full generation.

Left: a model's free-text reply that code must parse and hope is valid. Right: a typed answer, one valid option, that code can act on directly. — Most agent steps need a decision, not an essay. Ask for the type you will use.

The alternative is to move these decision steps to type-safe LLM calls: model calls that return a typed value instead of free text. This post covers the pattern, the kinds of agent steps it suits, and the engineering rules that make it safe in production.

What a type-safe LLM call is

Instead of asking a model to write an answer, you hand it your application state and a set of typed questions. Each question comes back as a value your code already knows how to handle. TypeSafe's Jev model is one example of this approach. Nearly every decision in an agent fits one of three types:

Choice: pick exactly one option from a list you supply. Route an email, select a template, classify a document type. The answer is always one of your options, never a near-miss spelling.
Yes/no probability: a calibrated probability that a statement is true. "Is this value supported by the source text?" comes back as 0.08 or 0.93, and your code decides what to do at each level.
Score: a graded rating against criteria you define, such as relevance or groundedness on a fixed scale.

Application state feeds three typed questions: a choice from a list, a yes/no probability between 0 and 1, and a score on a fixed scale. — Nearly every agent decision fits one of three types.

Several questions can ride in one request, so a chat turn that needs a route, a rerank of ten knowledge-base articles and three safety checks is still one fast call. Generation is left to a generative model, and only for the steps that genuinely have to write text.

Use case 1: Routing and triage

Routing is the most common decision an agent makes, and the one most often done badly with keyword rules or embedding similarity.

Conversational assistants. One typed request per turn can decide where a message should go (a data question, a how-to, an action, or something that needs clarification) and, in the same call, rank which knowledge sources are worth retrieving. Fewer wrong turns means fewer "could you clarify?" replies.
Inbox and ticket triage. Incoming messages are classified into a fixed taxonomy and given an urgency score, so the queue sorts itself. The taxonomy lives in data, so adding a category is a configuration change, not a prompt rewrite.
Choosing from approved options. When an agent must pick one of a set of pre-approved responses, templates or workflows, a typed choice selects it (or "none"), and a generative model writes only the text that choice needs.
Pre-screening before an expensive step. A cheap yes/no check decides whether an item needs the large model at all. Items that clearly do not skip it; anything uncertain still gets the full treatment.

Use case 2: Verifying what another model extracted

Document AI has a well-known failure: the extractor returns a confident, well-formatted, wrong value. Rather than trusting the extractor's own confidence, a typed check can follow extraction and ask narrow yes/no questions of every field against the source text:

Is this value absent from, or unsupported by, the source?
Was it taken from the wrong label, such as an issue date returned as a date of birth?
Does it violate the expected format: truncated, garbled or carrying extra text?
If the field is empty, does the document in fact contain it?

Fields with no confident red flag are accepted automatically. Any single flag above the threshold sends that field, not the whole document, to a stronger model or a human reviewer. The expensive path is spent only where it is needed.

Extracted fields each pass a check for unsupported values, wrong labels and bad formats; clean fields are accepted automatically and one flagged field goes to review. — Escalate the field, not the document. The expensive path runs only where a check fires.

A related pattern: let code find candidate values (amounts, dates, names, places) and let the typed model only choose among them, or map a document to one category from a list the business maintains. The model never invents a value; it only picks between values that already exist.

Use case 3: Cutting false positives in review agents

Review agents fail in an unglamorous way: they report too much. Once an automated reviewer raises five non-issues, people stop reading it.

A single yes/no question per finding ("is this a real problem, judged against the evidence?") filters the noise before anything is stored, sent or ticketed. Dropped findings are logged, so the threshold can be tuned on real data instead of guessed.

Many review findings enter a gate that asks whether each is a real problem; three are reported and the rest are logged but not sent. — One yes/no per finding keeps a review agent worth reading.

The same gate works for classification at scale: let rules settle the obvious cases, let a typed check settle the clear remainder cheaply, and send only the genuinely ambiguous items to a full model or a person.

Use case 4: Small conversational judgments

Some of the most useful decisions are tiny ones nobody would write a full prompt for.

Is the user still typing? On chat channels people often split one request across several short messages. A yes/no check on each fragment lets an agent wait for the rest instead of replying to half a sentence.
Is this input genuine? Flagging throwaway or junk submissions that a static block list misses, before they enter a sales or support process.
Is this detail really there? When an agent summarises a conversation or document, each captured detail can be checked against the source, so people are not handed facts the summariser filled in on its own.

Use case 5: Evaluating AI answers

Every retrieval-augmented system needs ongoing evaluation, and using a large LLM as the judge is slow, costly and inconsistent from run to run. The standard quality measures (context relevance, groundedness, answer relevance) map naturally to typed scores in one request, which makes evaluation cheaper, repeatable and comparable over time.

Engineering rules that make it safe

The model is only half of it. These rules are what make the pattern safe to put in front of customers:

Fail open, with a known fallback. Every typed call has a defined behaviour on timeout or error: keep the previous route, keep every finding, reply now. An outage degrades the agent; it never breaks it.
Ship in shadow mode first. Give each integration an off / shadow / live switch. In shadow, the typed decision is logged next to the existing one and never acted on, so agreement can be measured on real traffic before switching.
Thresholds belong to code. The model returns a probability. The decision of what 0.7 means for your business is a constant in your code, tuned on your logged data and changeable without touching the model.
Options are data, not prompt. Categories, templates and labels are passed in as lists. The business changes the list; nobody rewrites a prompt.
Ask narrow questions. "Is this field from the wrong label?" beats "Is this extraction correct?". Narrow questions are easier to calibrate and easier to explain to an auditor.
Keep a self-check beside the code. Give each gate a small offline test of its decision logic, so a refactor cannot quietly flip an accept into an escalate.

Where it does not fit

Typed calls are for decisions. When the output is prose (an email reply, a summary, an explanation), use a generative model and constrain it with a schema. The strongest design is usually the pairing: a typed call decides what to do, and a generative model writes only the text that decision requires. That keeps the judgement step small and testable, which is what keeps an agent predictable.

A typed call decides what to do and passes one value to a generative model, which writes only the text that decision needs. — A typed call decides what to do. A generative model writes only the text it needs.

The takeaway

If your agent asks an LLM a question and then parses the answer with string matching, that step is a candidate for a typed call. You get answers that are always valid values, probabilities you can threshold, one request in place of several, and a decision trail an auditor can follow.

The pattern applies wherever an agent has to decide rather than write. If you are building agents that need to make reliable decisions, not just produce fluent text, talk to us.