01 / CHOOSE
Route or classify
Pick from named options, such as billing, technical support or sales.
A practical deep dive · 2026
Decisions
without the
text generator.
Laya turns a piece of text into structured decisions in one model pass: choose a label, score a rubric, or estimate yes/no probability.
Open source · Apache 2.0Python SDK + HTTP APIEnglish + multilingual
Input → typed output
intent → billingurgency → 2 / criticalneeds human → 0.18
NO TOKEN-BY-TOKEN GENERATION01 / 15
01 · Plain English
Laya is a fast decision layer for text.
Instead of asking a large language model to write an answer and then parsing it, you define the decision you need. Laya returns typed outputs and probability distributions.
02 / SCORE
Apply a rubric
Estimate an ordered level, such as urgency from routine to blocking.
03 / YES–NO
Estimate a probability
Score a proposition, such as “does this message explicitly request a refund?”
Why it can help: fixed output shapes, low-latency classification, and a simpler path to batching and human-review rules. It still needs evaluation on your own data.
02 · Benefits & use cases
Put it on the decision path.
Laya is most useful when the next step is a bounded decision, not an open-ended explanation.
SUPPORT OPS
Ticket triage
Label a case, estimate urgency, flag churn signals, then send uncertain cases to a person.
AGENT SYSTEMS
Tool or model routing
Choose a workflow path or whether a request should use a smaller model or a stronger one.
TRUST & SAFETY
Fast screening
Score a prompt or post against a defined policy, with review gates for risky decisions.
Design pattern: let Laya make the cheap, repeatable first decision; reserve generative models and people for cases that need reasoning, explanation or exceptions.
Deep dive · 01
Under the hood: one pass, explicit decisions.
The useful abstraction is a state, a question schema and a checkpoint. The model scores possible outputs directly; it does not compose a free-form response token by token.
03 · Core API
Describe the shape of the answer.
Inputs
- State: text, email, ticket or a structured object.
- Question: what you want to decide.
- Criteria: options or an ordered rubric.
Outputs
- Typed answer: selected label, expected score or probability.
- Distribution: probabilities over candidate answers.
- Metadata: checkpoint, routing reason, elapsed time and usage.
questions = {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {"billing": "invoices, payments, refunds",
"technical": "bugs, outages, errors"}
},
"urgent": {"type": "score", "criteria": ["routine", "soon", "critical"]},
"refund": {"type": "noul", "instructions": "Is a refund requested?"}
}
result = router.predict(state, questions)
04 · Decision types
Three primitives cover many workflow gates.
choice
Choose among named options; return the selected label and a distribution. Use for intent, category or destination.
score
Return an expected level on an ordinal rubric, plus a distribution. Use for urgency, severity or frustration.
noul
Return a probability from 0 to 1 that a proposition is true. Use for a defined yes/no signal such as “contains a refund request”.
Schema-driven option: Laya also accepts JSON Schema or Pydantic models through
decide() when you want one structured object.05 · Model routing
Router picks a checkpoint before inference.
The built-in Router uses script and language heuristics to choose between the English and multilingual checkpoint. You can provide your own language guess or force a model.
INPUT STATEText / JSON
→Language and script detection
ROUTINGCheckpoint choice
→English, multilingual, or explicit override
DECISIONTyped answers
Single request or compatible batch
The repo warns that short Latin-script inputs may not reveal their language. If routing accuracy matters, pass a trusted language-ID result using
lang_guess and test ambiguous cases.06 · What the repo reports
Promising speed. Specialise before trusting.
Repository-reported benchmark results show the multilingual checkpoint outperforming the English model across the sampled non-English tasks. They do not establish performance on your workflow.
17,416questions in the shared benchmark described in the README
32.8 msreported one-question latency for multilingual on one T4 GPU
0.766accuracy reported for a fine-tuned typed-decisions checkpoint; base was 0.362
| Reported measure | English | Multilingual / routed |
|---|---|---|
| MASSIVE, 13 other languages | 0.306 | 0.451 |
| XNLI, 14 other languages | 0.521 | 0.731 |
| Languages above 3× random | 23 / 51 | 45 / 51 |
Read the caveats: base checkpoints are near chance on the repo’s typed-decisions benchmark; shipped confidence is overconfident without temperature fitting; long context and many-option questions need special handling. Treat benchmark values as author-reported, not an independent audit.
07 · Comparison
Same decision pattern. Different trade-offs.
Both target fast, typed decisions inside software and agent workflows. Jev is TypeSafe AI’s hosted model; Laya is an open-weight implementation you can run and specialise yourself.
| Dimension | Laya | Jev · TypeSafe AI |
|---|---|---|
| Access | Open weights / self-hosted; Apache 2.0 repository licence | Hosted API; weights are not offered in the Laya comparison |
| Decision interface | choice, score, noul; Python SDK and Jev-compatible HTTP wire protocol | choice, score, noul via TypeSafe API |
| Adaptation | Can fine-tune a checkpoint on domain decisions | Use the hosted model; no model-weight fine-tuning path described by Laya |
| Where it can lead | High-cardinality choice is a weak spot in the cited comparison: Banking77 0.425 | Banking77 reported at 0.870 (72 vs 77 labels) |
Evidence caveat: Laya’s README says Jev figures are third-party published and were not measured by the Laya team against the same samples/prompts. The reported typed-decisions scores (Jev 0.727, Laya fine-tuned 0.766) are indicative, not a controlled head-to-head. Choose by running both on identical, held-out examples from your workflow.
Laya README: “Laya (with routing) vs Jev” ↗ · Comparison table, protocol compatibility and caveats.
Jev / TypeSafe overview ↗ · TypeSafe describes Jev as a “System One decision model” for fast structured decisions in agent pipelines.
08 · System shape
SDK first; serve it when you need a boundary.
Local Python
Load a checkpoint with Router or laya.load(). Run in-process and batch compatible states to use accelerator capacity efficiently.
Good for: prototypes, offline jobs, an existing Python service.
HTTP / integrations
Optional extras add a FastAPI server, MCP server, LangChain / LangGraph integrations and ONNX support.
Good for: shared internal service, agent tool boundary or deployment independent of the app.
For production, make the deployment choice explicit: preload the checkpoints you serve for predictable latency, size memory for model and tokenizer assets, secure the API boundary, and observe routing and answer quality.
Implementation guide · 02
From install to a measured decision.
The hard part isn’t calling predict(). It’s defining the label space, validating edge cases and setting a safe fallback.
09 · Step by step
A practical first integration.
Install in an isolated environment
Use Python 3.10+ and install laya. First model use downloads weights from Hugging Face.
Define a decision contract
Start with one workflow and a small, mutually exclusive label set. Write examples for ambiguous and negated language.
Run predictions in shadow mode
Keep the existing workflow in charge. Log output, distributions, routing and errors against labelled examples.
Evaluate and calibrate
Measure per-class precision/recall, language and subgroup slices, calibration and abstention. Fit temperatures on held-out data.
Add a human fallback
Choose thresholds from your risk tolerance and validation set. Escalate low-confidence or high-impact cases.
Serve and monitor
Preload chosen checkpoints, batch where useful, add hooks for redaction/audit, then watch drift and error samples.
10 · Working skeleton
Start small, then measure.
# Python 3.10+ · in a virtual environment
# python -m pip install laya
from laya import Router
router = Router() # lazy load; first prediction downloads a checkpoint
state = {"body": "I was billed twice. Please refund the duplicate."}
questions = {
"intent": {
"type": "choice",
"instructions": "What is the main request?",
"criteria": {
"billing": "payment, invoice, refund",
"technical": "bug, outage, error",
"other": "anything else"
}
},
"refund_requested": {
"type": "noul",
"instructions": "Does the customer explicitly request a refund?"
}
}
result = router.predict(state, questions)
print(result["answers"])
print(result["routing"]) # inspect the selected checkpoint
# Next: run a labelled evaluation set; don't set production
# thresholds from a few hand-picked examples.
For long mostly-English documents, the README says to select
model="multilingual" explicitly and set max_len=8192; its reported accuracy varies beyond roughly 4,000 input tokens.11 · Decision guide
Use Laya where the decision is bounded.
Strong fit to investigate
- High-volume classification or triage
- Known categories or an ordered rubric
- Latency or predictable output shape matters
- You can maintain labelled evaluation data
- Human escalation is available for risky cases
Validate carefully
- Negation, subtle language and high-stakes outcomes
- Long inputs and very large option sets
- Less-supported languages or code-switching
- Confidence thresholds from another model or task
- Any claim that “no generation” means “no errors”
Recommendation: run a time-boxed shadow evaluation on one workflow. Compare task quality, latency and serving cost with your current classifier or LLM pipeline before expanding.
12 · Sources
Read the repository at the source.
Facts, benchmark values, examples and implementation notes in this deck are based on the project’s current README and linked documentation.
Laya repository README ↗
Installation, quickstart, checkpoint overview, benchmarks, limits, deployment and fine-tuning. README phrase: “typed questions … over any state … in a single forward pass.”
Installation, quickstart, checkpoint overview, benchmarks, limits, deployment and fine-tuning. README phrase: “typed questions … over any state … in a single forward pass.”
Project documentation ↗
API reference and guides for hooks, schema-driven decisions, Docker and LangChain / LangGraph.
API reference and guides for hooks, schema-driven decisions, Docker and LangChain / LangGraph.
Laya checkpoints on Hugging Face ↗
Model artifacts referenced by the repository. Verify the exact checkpoint and version you evaluate.
Model artifacts referenced by the repository. Verify the exact checkpoint and version you evaluate.
Apache 2.0 licence ↗
Check upstream licence and model-card terms before adopting in a product.
Check upstream licence and model-card terms before adopting in a product.
Prepared 27 September 2026 · Author-reported metrics are labelled as such. Recheck upstream docs before implementation.