Technology
Decision API: Moving Beyond “Reply in JSON” Prompts to Typed, Calibrated Answers
Almost every team that has tried to make a language model drive real software hits the same wall. You write a careful prompt, ask politely for JSON, and get back a code-fenced reply with three sentences of commentary wrapped around a malformed object. So you add a parser, then a repair pass, then a retry loop, then a validator, and suddenly a single business question has turned into a small distributed system with its own failure modes. The prompt drifts, the model changes, and the eval suite you built last quarter no longer reflects what production is doing. A decision api is the structural answer to that problem: instead of generating text and hoping it parses, it returns typed, calibrated decisions that code can branch on directly. This guide breaks down what that contract looks like in practice, why calibrated confidence matters more than raw model scores, and how to budget latency and cost when decisions sit on the hot path of an application.
What Separates a Decision API From Prompt-Engineered JSON
The difference is not just formatting — it is the shape of the contract. With a text-generation model, the input is a prompt and the output is a token stream whose structure you infer after the fact. With a decision engine, the input is state and the output is a set of typed answers to questions you defined in code. DecisionNode, for example, exposes a single HTTPS endpoint (POST /v1/decide) that accepts state as text, JSON, or up to sixteen images or video frames, alongside a question set expressed as a schema. It returns every answer in one pass, and each answer is constrained to one of a small number of machine-friendly types: choice (exactly one of your labels), score (a level on an ordered scale), truth (a calibrated probability between 0.00 and 1.00), number (a value on a grid), and point or box (locations within an image).
That constrained output space changes what your integration code has to do. There is no free-form prose to strip, no accidental markdown fence to unwrap, and no valid-but-unexpected key showing up at 2 a.m. If you asked for one label out of five, you get one label out of five. The “System One” model behind the service is designed to produce a decision rather than a narrative, which also means the response is deterministic: the same request produces the same answer. Determinism is quietly one of the most valuable properties here, because it makes decisions replayable, diffable in code review, and cacheable — three things that are effectively impossible when sampling is in play.
It also simplifies the parts of the stack that surround the call. SDKs for TypeScript, Python, Go, Ruby, and plain curl mean the endpoint drops into existing services without a bespoke client, and the same request shape works whether the caller is a backend job, an ops automation script, or an AI agent that needs to decide before it acts. Instead of maintaining prompt templates as a second, informal codebase, teams keep the question definitions in the repository next to the code that consumes them — versioned, reviewed, and testable like anything else. The practical result is that ambiguity moves out of the parser and into the schema, where it can be expressed deliberately: which labels are legal, which scale a score belongs to, and what confidence is required before the answer is trusted.
Calibrated Confidence Turns Ambiguity Into Control Flow
The hardest part of automating a real decision is not producing an answer — it is knowing when the answer is good enough to act on. A model that is confidently wrong is worse than one that escalates, and a system that cannot distinguish the two will eventually fail in a way nobody can trace. This is where calibration earns its place. Every answer from DecisionNode carries a calibrated confidence, meaning the reported probability is intended to correspond to observed accuracy rather than merely reflect internal model excitement. A 0.90 truth value should be right roughly ninety times out of a hundred, which is what makes it safe to attach a business rule to it.
Once confidence is calibrated, threshold logic becomes the backbone of the integration. Above a threshold, code acts instantly and without a human in the loop. Below it, the same request can trigger a re-check with additional state, route to a reviewer queue, or take a deliberately conservative safe path — hold the order, block the promo, ask for another document. Consider a promo-abuse check: a truth probability of 0.97 that an account is abusing a referral code can auto-block, a probability around 0.6 can route to manual review with the evidence attached, and a probability near 0.05 can auto-approve. The same pattern maps cleanly onto ticket routing and triage, moderation queues, order and shipping exceptions, and gating decisions in a product funnel. In each case the model supplies a number and the business supplies the boundary, which keeps policy ownership where it belongs.
This design also makes escalation cheap to reason about. Instead of treating a low-confidence answer as an error requiring retries, you treat it as a legitimate outcome that the system is expected to handle — one branch among several. That distinction matters at scale, because it caps the cost of uncertainty: ambiguous cases go to humans or to a slower path, and clear cases never touch either.
Throughput is the other half of the control-flow story. For loops that exceed roughly ten decisions per second — drones, robots, simulators, machine monitoring on a production line — DecisionNode offers sessions over WebSocket, so continuous decision-making does not pay a connection setup cost on every call. Pipelining decisions this way is what makes per-frame quality control or per-reading anomaly checks financially and technically realistic rather than a batch job that runs overnight.
Latency, Cost, and Where Decisions Belong in Production
Decision endpoints are only useful if they fit inside the latency budget of the thing they are deciding about. DecisionNode ships two models aimed at different points on that curve. DecisionNode Flash runs at roughly 5 ms server-side and about 19 ms end to end, priced at $0.035 per 1M input tokens. DecisionNode MoE runs at roughly 9 ms and $0.042 per 1M input tokens, trading a few milliseconds for additional capability on harder questions. Output tokens are always free, batch discounts are available, and there is a free daily tier, which means a team can prototype against production infrastructure instead of a mock and measure real end-to-end latency before committing to an architecture.
Choosing between the two is usually an empirical question rather than a philosophical one. If the decision is a routing label over well-structured JSON, Flash is generally the right default and the 19 ms end-to-end figure is small enough to sit inline in a request handler. If the question depends on richer visual or textual context — a damaged package photo, a dense support thread — the extra milliseconds of MoE are easy to justify. Mixed strategies work well too: run the fast model first and only re-check low-confidence answers on the stronger one, which keeps the average cost near the Flash price while reserving capacity for the cases that need it.
Input modality is often the deciding factor in where these calls get placed. Because a single request can carry text, JSON, and up to sixteen images or video frames, decisions that used to require a separate vision pipeline can be expressed directly. Boosters extend the base decision types for specific sub-problems: human age estimation, AI-generated image detection, face detection and recognition, OCR, and depth and segmentation. A warehouse monitoring job can ask whether a bay is occupied and where the pallet sits in the frame; a traffic monitoring job can score congestion levels on an ordered scale; a production line can gate units on a quality choice with a confidence threshold. None of these require assembling a chain of models — the state goes in, the typed answers come out.
The same endpoint serves the agentic side of the stack. AI agents can call it to decide before acting, MCP tools expose it to agent runtimes, and webhooks push results into existing automation. Common production patterns include ticket routing and triage, fraud and promo-abuse checks, order and shipping decisions, moderation, scoring, gating, and quality control on production lines. The engineering discipline that makes these work is unglamorous but consistent: define questions in code, version the schema alongside the consuming service, log the request and the returned confidence for every decision, and replay stored requests after any change to confirm answers stay stable. With deterministic outputs and calibrated probabilities, that replay step is a real regression test rather than a best-effort smoke check.
Raised in São Paulo’s graffiti alleys and currently stationed in Tokyo as an indie game translator, Yara writes about street art, bossa nova, anime economics, and zero-waste kitchens. She collects retro consoles and makes a mean feijoada.