Why System One Models Could Change the Economics of Clinical Trial Decisioning

Enterprise AI teams are running into a familiar problem: not every workflow needs the biggest model in the room. In clinical trial operations, many high-value tasks still come down to structured judgment: interpreting a protocol deviation, understanding its risk and determining the right reporting path. These decisions are bounded, but their consequences are not. That is exactly where System One models such as Jev become interesting. Enterprises should stop treating every AI use case as an LLM use case and start asking a more useful question: what kind of decision does this workflow actually require?

System One models are designed for fast, constrained decisioning over predefined choices or scores rather than open-ended natural language generation. Instead of generating long sequences of text, they work within a defined decision space at the logit level. The user specifies the decision type, such as classification, scoring or choice, provides the relevant state, typically as a JSON dataset and receives a deterministic JSON response in a specified format.

Fit the model to the decision pattern

System One models are not competing with LLMs on breadth as they are built for a different kind of work. General-purpose LLMs are powerful when the task calls for synthesis, explanation, broad reasoning or natural language generation. But when the task is bounded, repeatable and already shaped as a set of possible decisions, scores or classifications, a constrained model can be a better fit because it returns structured outputs for the business to act quickly upon.

Routing every problem through a large LLM, however, would mean paying for model power that the workflow does not need while adding latency and complexity to work that demands speed and reliability.

The clinical trial decision problem

Clinical trial operations make this architecture question concrete. Trials generate a steady stream of structured decisions: whether a deviation affects data quality, creates patient safety risk, requires CAPA or needs to be reported. These are high-stakes judgments, but they do not always require open-ended generation. They require consistency, speed and outputs that can be acted on.

Clinical trials are long, expensive and tightly regulated, so the cost of getting these decisions wrong can be significant. A single administrative error, missed visit window or incorrect classification can affect data quality, patient eligibility, regulatory reporting and downstream trial outcomes. CRAs already monitor deviations across sites and protocols, so the operational burden is not just recognizing that something went wrong, but deciding what action is required in the context of the trial.

That became the research question: when a protocol deviation must be classified into known categories, can a constrained decision model provide a better operational fit than a large general-purpose LLM?

Research approach

We created a dataset of trial protocols and protocol deviations and used the models to identify the next best action. We first tested the Llama 3.3 70B parameter model through Persistent’s GenAI Hub. The example below shows the prompt and response for a single test case.

Prompt Sent

[SYSTEM]

You are a clinical research expert. Given a protocol deviation context, return ONLY a valid JSON object with exactly these keys: capa_required (boolean), severity (string: minor|major), regulatory_reportability (string: not_required|IRB_only|sponsor_and_IRB), immediate_risk (boolean). No other text.

[USER]

Deviation context:

Trial: Hemoadsorption for Prevention of Vasodilatory Shock in Cardiac Surgery Patients With Infective Endocarditis (NCT03266302) | Sponsor: Jena University Hospital | Phase: NA | Condition: Infective Endocarditis. Site: Royal Edinburgh Clinical Research Unit (UK, LOW risk profile). Subject SUBJ-0046 at Follow-up Visit 2. Deviation occurred: 2023-11-10, discovered: 2023-11-16 (6 days later). Deviation: Visit Conducted Outside Protocol Window. Severity: MINOR. Protocol requirement: Visit 4 must be conducted within Day 77–91 of study entry. Data collected outside this window may not be usable for primary endpoint analysis. Description: Subject’s Month 3 visit (Visit 4) was scheduled for Day 84 ±7 (window: Day 77–91). Actual visit occurred on Day 95 — 4 days outside the upper boundary. Subject had international travel conflict.

Raw Response

Model: meta-llama/llama-3.3-70b-instruct
Usage: prompt=339 completion=65
Content: ```json
{
 "capa_required": false,
 "severity": "minor",
 "regulatory_reportability": "IRB_only",
 "immediate_risk": false
}
```

The Llama model returned a valid JSON-formatted response, showing that it could follow the requested output structure. However, the response took about 4 seconds, which limits its usefulness for high-volume, low-latency decision workflows.

Next, we ran the same experiment using a Jev model from TypeSafe AI, hosted in a similar environment.

Payload Sent

Model: typesafe/jev-1.13
state:
Clinical Trial: Hemoadsorption for Prevention of Vasodilatory Shock in Cardiac Surgery Patients With Infective Endocarditis
(NCT03266302)
Sponsor: Jena University Hospital | Phase: NA
Condition: Infective Endocarditis
Site: Royal Edinburgh Clinical Research Unit (UK, LOW risk profile)
Investigator: Dr. Helen Bradshaw | CRA: CRA_Amara_Diallo
Subject: SUBJ-0046 | Visit: Follow-up Visit 2
Occurrence date: 2023-11-10 | Discovery date: 2023-11-16 (6 days gap)
Deviation type: Visit Conducted Outside Protocol Window — Severity: MINOR
Protocol section: Section 7.3 – Visit Schedule and Windows
Requirement violated: Visit 4 must be conducted within Day 77–91 of study entry. Data collected outside this window may not be
usable for primary endpoint analysis.
What happened: Subject's Month 3 visit (Visit 4) was scheduled for Day 84 ±7 (window: Day 77–91). Actual visit occurred on Day 95 — 4 days outside the upper boundary. Subject had international travel conflict.
Immediate risk: NO — Primary endpoint labs collected outside window may be excluded from per-protocol analysis.
Root causes: Subject work travel conflict not communicated until 72 hours before scheduled visit; No alternate visit date within window could be arranged; Telemedicine visit not pre-authorized for this visit type
questions:
{
"capa_required": {
"type": "noul",
"instructions": "Is a Corrective and Preventive Action (CAPA) required for this protocol deviation?",
"criteria": {
"true": "A CAPA is needed \u2014 process failure, systemic issue or major deviation requiring formal corrective action",
"false": "No CAPA needed \u2014 minor deviation, no systemic issue, documentation fix is sufficient"
}
},
"immediate_risk": {
"type": "noul",
"instructions": "Does this protocol deviation pose an immediate risk to subject safety?",
"criteria": {
"true": "Subject is currently at risk \u2014 safety action or IP suspension needed now",
"false": "No immediate risk \u2014 subject safety is not in jeopardy at this moment"
}
},

"severity": {
"type": "choice",
"instructions": "What is the severity classification of this protocol deviation?",
"criteria": {
"major": "Significant deviation that may affect subject safety, data integrity or regulatory compliance \u2014 requires immediate sponsor notification",
"minor": "Minor deviation with negligible impact on subject safety or data quality \u2014 document and notify sponsor per standard timeline"
}
},
"regulatory_reportability": {
"type": "choice",
"instructions": "What is the required regulatory reporting level for this deviation?",
"criteria": {
"not_required": "No regulatory reporting required \u2014 internal documentation only",
"IRB_only": "Must be reported to the Institutional Review Board (IRB/Ethics Committee) only",
"sponsor_and_IRB": "Must be reported to both the sponsor and the IRB/Ethics Committee"
}
},
"corrective_action_scope": {
"type": "score",
"instructions": "How broad does the corrective action need to be?",
"criteria": [
"Individual fix \u2014 update one record or retrain one staff member",
"Site-level fix \u2014 revise site SOP or retrain site team",
"Multi-site fix \u2014 sponsor-level process change affecting multiple sites",
"Systemic overhaul \u2014 regulatory authority notification and trial-wide corrective action"
]
}
}

Raw Response

HTTP status: 200
Response body:
{
"model": "typesafe/jev-1.13-20260917",
"answers": {
"capa_required": {
"type": "noul",
"noul": 0.41
},
"immediate_risk": {
"type": "noul",
"noul": 0.08
},
"severity": {
"type": "choice",
"choice": "minor",
"probabilities": {
"minor": 0.99,
"major": 0.01
},
"confidence": 0.99
},
"regulatory_reportability": {
"type": "choice",
"choice": "not_required",
"probabilities": {
"IRB_only": 0.04,
"not_required": 0.62,
"sponsor_and_IRB": 0.34
},
"confidence": 0.43
},
"corrective_action_scope": {
"type": "score",
"score": 0.97,
"legend": {
"0": "Individual fix \u2014 update one record or retrain one staff member",
"1": "Site-level fix \u2014 revise site SOP or retrain site team",
"2": "Multi-site fix \u2014 sponsor-level process change affecting multiple sites",
"3": "Systemic overhaul \u2014 regulatory authority notification and trial-wide corrective action"
},
"probabilities": {
"0": 0.08,
"1": 0.88,
"2": 0.04,
"3": 0
},
"confidence": 0.88
}
},
"usage": {
"input_tokens": 1249,
"output_tokens": 220,
"cost": 5.2458e-05
},
"id": "gen-dec-1790017891-1T1nNHP94qWkwqtSu5Kq",
"provider": "TypeSafe"
}

The Jev model responded much faster and mapped the deviation to specific decision categories with associated scores and confidence values. Although it required more upfront setup to define the payload, the output was faster, more structured and easier to use programmatically. It also surfaced the token profile behind the response, with 1,249 input tokens, 220 output tokens and a cost of 5.2458e-05, making the economics of the decision visible alongside the decision itself.

We then ran several similar tests and compared both quality and latency across the two models.

What the results show

We evaluated how well each model understood the protocol context and classified the required action across CAPA requirement, severity, regulatory reportability and immediate risk. A human CRA rated the deviations to create a golden dataset for comparison. The charts below show that the System One model delivered higher overall accuracy on this structured decision task, reaching 91% composite accuracy compared with 74% for Llama 3.3 70B.

The latency difference was even more significant: mean latency was 3.03 seconds for the Llama model and 0.45 seconds for the Jev model. Taken together, the results suggest that constrained class generation can be more effective than verbose reasoning for repeatable, structured decision workflows because it returns the required answer faster, in a usable format and with clearer visibility into token use and cost.

Evaluation Summary

For pharmaceutical companies and CROs, the takeaway is clear: System One models should be evaluated as specialized decisioning engines for structured clinical workflows, not as replacements for LLMs. Protocol deviation classification is one example. Risk assessment, reportability checks and other repeatable operational decisions may present similar opportunities.

The token impact is central to the business case. In workflows where the answer is a bounded choice, score or classification, forcing every request through a large language model means paying for unnecessary token generation, longer prompts and more latency than the task requires.

Jev’s constrained decisioning approach reduces that burden by limiting the model to the decision space needed for the workflow while still returning structured, machine-readable outputs.

As token consumption becomes a larger part of enterprise AI economics, AI maturity will depend less on choosing the largest available model and more on choosing the architecture that delivers the required answer with the fewest tokens, lowest latency and clearest path to scale.