TL;DR

Laya (NandhaKishorM/laya) is an open-weight, Apache-2.0 “System 1” decision model: you hand it a state (an email, a ticket, a JSON blob, an agent observation) plus typed questions, and it returns typed answers — a choice among your labels, an ordinal score, or a yes/no probability (noul) — in a single non-autoregressive forward pass. No text generation, nothing to parse, no JSON to repair. It is the open-source answer to TypeSafe’s Jev, and it has become the fastest-growing AI repo of September 2026: created September 18, it hit 23,805 stars, 2,045 forks and 158 open issues by September 25, with v0.3.20 shipped on September 24. Key facts:

  • Three checkpoints + a router: laya (ModernBERT-large, 421M, English), laya-multilingual (mmBERT-base, 322M, 100+ languages, up to 8,192 tokens), and laya-typed-decisions (fine-tuned for four agent workflows). Router() picks per request from script/language detection in under 0.5 ms.
  • Speed: 32.8–39.5 ms per question on a Tesla T4, 7.2 ms/question batched — 6–7× faster than third-party Jev measurements (236–276 ms p50).
  • Jev-compatible wire protocol: laya-serve exposes POST /v1/systemone; existing Jev clients only need a new baseUrl.
  • Integrations: LangGraph conditional edges, an MCP stdio server, ONNX Runtime, a TileLang GPU fast path, Docker, and a Nix flake.
  • Reality check: the base checkpoints are near chance on the typed-decisions benchmark zero-shot (0.36 vs 0.318 random); the 0.766 headline comes from fine-tuning. Options share a small token budget (Banking77: 0.425 vs Jev’s 0.870). Calibration is over-confident as shipped. The maintainers say so, in a section titled “Honest limits.”

If you run bounded decisions in an agent loop — which model to route to, does this alert need a human, is this prompt an injection — Laya turns a 1–3 second LLM call into a 30 ms local classifier. Just plan to fine-tune it.

Quick Reference

Repositorygithub.com/NandhaKishorM/laya
Weightshuggingface.co/convaiinnovations/laya (all three checkpoints; only the requested one downloads)
Docsnandhakishorm.github.io/laya
AuthorNandakishor M, Convai Innovations
LicenseApache-2.0 (code and weights)
LanguagePython 3.10+ (PyTorch 2.14, transformers 5.x)
Stars23,805 (+2,045 forks) as of September 25, 2026; created September 18
Latest releasev0.3.20 (September 24, 2026), PyPI laya
Installpython -m pip install laya
Question typeschoice, score, noul (P(true))
DemoHugging Face Space

What It Is

A “System 1” decision model is a classifier with a schema. Instead of prompting a chat model to “reply with JSON containing department, urgency and churn_risk” and parsing whatever comes back, you declare the questions and their allowed answers up front. The model reads the state and every option in one bidirectional encoder pass, a small decision head scores a marker token before each option, and a softmax turns those scores into a probability distribution per question. Nothing is generated token by token, so latency is a fixed function of input length and there is no output to hallucinate.

TypeSafe coined the “System One” framing and shipped it as the hosted Jev API in early September 2026 at $0.042 per million input tokens. Laya is the same interface with open weights. Its author, Nandakishor M, had published a non-autoregressive decision model and paper a year earlier; when Jev went viral he posted “I literally built the Jev architecture one year back” to r/LocalLLaMA, trained a larger general-purpose model on a single RTX 6000 Pro (96 GB), and released it as Laya. The Hacker News launch thread reached 1,352 points and 316 comments.

Architecturally: a ModernBERT-large (or mmBERT-base for multilingual) encoder with a from-scratch Transformer head that scores [MASK] option markers. Training is what the author calls RLCD — reinforcement learning against strictly proper scoring rules (log + spherical, plus ranked probability score for ordinal questions). Proper scoring rules are maximised only by reporting honest probabilities, which is the theoretical basis for the “calibrated confidence” claim; the practical basis is temperature fitting on your own data.

Three things landed at once. Jev made typed decisions a category — Browser Use’s Jev Ultrafast completed a Google Flights search in 7 seconds by letting a decision model pick instead of an LLM write. Jev is a waitlisted, closed, hosted API. And many engineers had the same reaction as HN user Oras, who tested Jev for classification: “as someone who trained NLP models prior to LLMs, it’s just BERT with more data … a wake up call for the tech community to go back to basics for most tasks instead of relying solely on generic LLMs.”

Laya is that wake-up call in pip install form. Within a week the ecosystem included laya.cpp (an optimised CPU port), laymbda (AWS Lambda with SnapStart), laya-jolt (a Jev-API-compatible server), VisionLaya (a SmolVLM-256M variant that takes images), an oh-my-pi judge plugin, and a Tetris-playing demo. That is the long tail a closed API cannot grow.

Getting Started

python3 -m venv .venv
.venv/bin/python -m pip install laya
.venv/bin/python -I -c "import laya; print(laya.__version__)"

The Router is the recommended entry point. It downloads a checkpoint on first use (or all three with preload=True):

from laya import Router

router = Router()

state = "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
questions = {
    "department": {"type": "choice", "instructions": "Which department should handle this?",
                   "criteria": {"billing": "invoices, payments, refunds",
                                "technical": "bugs, outages, system errors",
                                "other": "everything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?",
                "criteria": ["not urgent", "soon", "blocking"]},
    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
}

result = router.predict(state, questions)
print(result["answers"]["department"]["choice"])   # billing
print(result["answers"]["churn_risk"]["noul"])     # e.g. 0.892 — P(yes)
print(result["routing"]["model"])                  # english

Send Hindi or Spanish and the same call routes to laya-multilingual automatically; the response carries a routing.reason such as "non-Latin script (devanagari, 100% of letters); the English checkpoint cannot read it". There is also a CLI (laya "My payment failed twice" --preset triage) and a FastAPI request-builder UI in examples/server.py.

If you prefer schemas, decide() takes a JSON schema or a pydantic model and returns plain typed values:

schema = {"type": "object", "properties": {
    "department": {"type": "string", "enum": ["billing", "support", "sales"]},
    "urgency": {"type": "integer", "minimum": 0, "maximum": 2},
    "needs_human": {"type": "boolean"}}}

agent = laya.load("convaiinnovations/laya")
agent.decide("I was charged twice, refund me.", schema=schema)
# {"department": "billing", "urgency": 2, "needs_human": True}

Key Features

Batching and the GPU fast path

predict_batch(states, questions, batch_size=64) packs a backlog of tickets against the same questions into shared forward passes; Router.predict_batch() groups requests by checkpoint and schema first so mixed-language workloads don’t churn models. pip install laya[fast] adds fused TileLang kernels with bf16-resident weights and each (batch, length) bucket captured as a CUDA graph, removing ~200 Python kernel launches per call; parity against fp32 is documented at max |Δp| ≤ 0.05.

Self-hosting with a Jev-compatible server

pip install "laya[serve]"
LAYA_DEVICE=cuda LAYA_PRELOAD=1 laya-serve   # 0.0.0.0:8000, all three checkpoints resident

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"body": "billed twice, refund please or we cancel"},
  "questions": {"dept": {"type": "choice", "instructions": "which team?",
                "criteria": {"billing": "refunds", "tech": "bugs"}}}}'

LAYA_API_KEY enables bearer auth, LAYA_THREADS caps CPU threads, LAYA_MODELS selects what to preload; the Nix flake ships a hardened systemd unit. Three things differ from Jev when you port a client: options share a head_max_len token budget rather than a 255-option cap, every score level needs a description, and confidence is 1 − normalised entropy rather than Jev’s formula — thresholds don’t transfer.

LangGraph, MCP and hooks

from laya.integrations.langchain import LayaRouter, LayaGuardrail

router = LayaRouter(criteria={"billing": "invoices, charges", "tech": "bugs, outages"},
                    confidence_threshold=0.80, fallback="human_agent")
workflow.add_conditional_edges("triage", router)

guard = LayaGuardrail(action="raise")   # LayaGuardrailError on jailbreak/injection

pip install laya[mcp] and laya-mcp-server expose laya_predict, laya_route, laya_preset and laya_status as MCP tools over stdio for OpenClaw, Claude Desktop or Cursor. Built-in presets cover model routing, prompt guardrails, moderation and ticket triage; prediction hooks (on_predict_start, on_predict_end, on_route, on_error) add PII redaction, caching or audit logging without forking.

Benchmarks: Laya vs Jev

All Laya numbers are measured by the maintainers on a T4 with a fixed seed; the Jev column is third-party published figures, since the author has no TypeSafe API access. Treat the comparison as indicative, not a controlled head-to-head.

Jev 1.13.0 (published)Laya (routed)
typed-decisions, 2,000 decisions0.7270.766 (fine-tuned checkpoint)
AG News, 4 labels0.9100.950
DAIR Emotion, 6 labels0.4800.595
Banking77 (72 vs 77 labels)0.8700.425
ECE after temperature fitting (lower is better)0.2460.081
p50 latency, 1 question236–276 ms32.8 ms
Context~32K (per HN commenters)512 / 1,024 (multilingual up to 8,192)
Weights / costclosed API, $0.042 per 1M input tokensApache-2.0, $0 self-hosted

The multilingual story is the strongest part. Across 51 languages of MASSIVE intent (20 options), the English checkpoint is usable (>3× random) in only 23 languages and scores 0.000 on Khmer at 95.2% confidence — confidently wrong, which is why routing happens before the forward pass. laya-multilingual clears 45 of 51 languages and scores 0.731 on non-English XNLI vs 0.521 for the English checkpoint. One detail from the DAIR Emotion run: Jev assigned zero probability to the true label on 16% of examples, a hard failure for anything branching on confidence.

Honest Limitations

The README’s “Honest limits” section is unusually candid, and independent testing bears it out:

  • Zero-shot is weak on real agent decisions. On the typed-decisions benchmark (invoice processing, security incidents, customer service, agent-trace observability) the base checkpoints score 0.362 and 0.352 — below the 0.461 majority class. The 0.766 number comes from laya-typed-decisions, fine-tuned on that benchmark’s own training split. The maintainers’ framing: “Laya is a fast base to specialise, not a zero-shot decision engine.”
  • Context is short. 512 tokens on the English checkpoint, 1,024 default on the others. HN commenter cube2222 noted Jev is “seemingly 32k” and was surprised the gap wasn’t surfaced; v0.3.20 partly answers with max_len=8192 on the multilingual model, reliable to roughly 4,000 tokens and “more variable beyond.”
  • Many options degrade fast. Options share a 192–256-token head budget, so 77 Banking77 labels get 3–4 tokens each. Workarounds: raise head_max_len, split into coarse/fine questions, or predict_shortlist with an embedding to keep the top-k labels (one user reported 54.3% → 60.8%).
  • Label sensitivity. noul can follow its false:/true: labels instead of the state on the English checkpoint (#156); negated cancellation requests still selected cancel_account in 4 of 4 cases (#377); laya-multilingual has a position bias on score questions (#131); action.act_probability “carries no usable signal yet” (#185).
  • Calibration needs fitting. Both checkpoints are over-confident as shipped; laya-multilingual ships with no fitted temperatures. Refitting per (type, option count) moves mean ECE from 0.466 → 0.081.
  • Common sense is not in scope. HN user edot fed the demo “a 6-sided die rolled a 3” and asked “Is the number odd?” — Laya answered 9% with 91% confidence. A bounded classifier is not a reasoning model.

Community Reactions

The HN thread split between engineers glad to see classical ML back in fashion and sceptics of the launch post’s tone. kilroy123: “the cheapest LLM request is no request at all.” Rui Carmo confirmed ~400M parameters runs on CPU, “not instantly though.” kamranjon pointed out that GLiNER2 (Fastino) has an older claim to the underlying trick than either Jev or Laya. Several found the author’s “a frontier lab called it a breakthrough” framing bitter; dcow’s take: “both Jev and Laya are based on the research of countless prior papers” — the difference is TypeSafe built a product. wren6991: “this VC-backed firm could have been a single arXiv preprint.”

The most useful field report is from As The Geek Learns, which swapped a deterministic model-routing rule in a coding-agent gateway on an M3 Ultra Mac Studio for laya-typed-decisions. On a frozen 40-decision held-out replay, acceptable routing decisions went from 33/40 (82.5%) to 37/40 (92.5%), and Laya never selected an ineligible model. The caveats matter as much: p95 routing latency was 143 ms on a quiet host but 467 ms on a busy one — past their 300 ms decision timeout, which fired for real — and a recovery-decision profile failed acceptance and stayed disabled. Their rule: “Laya only ranks survivors. Permission never comes from the model.”

Who Should Use This (and Who Shouldn’t)

Use Laya if you have a bounded decision that fires often inside an agent loop or pipeline — routing, triage, guardrails, moderation, “does this need a human” — and you can collect a few thousand labelled decisions to fine-tune on. The Kaggle notebook runs the whole RLCD loop on free 2×T4s in 4–5 hours over ~30k questions. The worked browser-agent example is the proof: a Laya head specialised for Jev Ultrafast went from 10% to 66% element top-1 among ~45 candidates, and real-task success from 0% to 62% at 17–23 ms per step, on one 16 GB GPU with no paid API. It also fits when data residency matters: tickets and alerts never leave your box, and there is no waitlist.

Stay with Jev (or an LLM) if you need 50+ options in one question without tuning, long inputs, soft-distribution fidelity (Jev’s 0.580 vs Laya’s 0.471), or you don’t want to own inference in production — as HN user zurfer put it, “managing GPUs in production is a non trivial problem.” And if the decision needs reasoning across context, that is a System 2 problem; a bigger model is the right tool.

Comparison with Alternatives

LayaTypeSafe JevZero-shot NLI (DeBERTa)LLM + JSON schema
Interfacetyped choice/score/noulsamelabel-per-hypothesisfree-form JSON
WeightsApache-2.0closedopenvaries
Latency (1 question)~33 ms GPU, 190–460 ms CPU236–276 ms p50 hostedone pass per label500 ms–3 s
Multilingual100+ (mmBERT)unpublishedmodel-dependentyes
Options per questionbudget-limited (~20 default)up to 255linear costunbounded
Fine-tuningopen notebook, RLCDnoyesprompt only

FAQ

Is Laya a drop-in replacement for Jev? Mostly. laya-serve implements Jev’s POST /v1/systemone wire format with schema-identical answers and usage blocks, so clients like hs-jev only change baseUrl. Expect differences in option budgets, mandatory score-level descriptions, and the confidence formula — refit any thresholds.

Does Laya run on CPU? Yes. Preloaded, a single question takes roughly 193–464 ms on CPU versus 33 ms on a T4; laya.cpp, ONNX Runtime (laya[onnx]) and Apple MPS are all supported paths. Lazy loading costs a 7–10 s rebuild per checkpoint switch, so preload on servers.

How accurate is it out of the box? Strong on benchmarks in the training mix (AG News 0.947, BoolQ 0.830, English XNLI 0.860), weak on ordinal scoring (SST-5 0.372), and near chance on agent-workflow decisions until fine-tuned.

What languages does it support? laya-multilingual is usable in 45 of the 51 languages benchmarked, including Devanagari, Arabic and CJK scripts. The router chooses it automatically; set Router(default="multilingual") if most traffic is short non-English text.

Can I use it from Claude Code, Cursor or OpenClaw? Yes — pip install laya[mcp] and register laya-mcp-server as a stdio MCP server; the tools return JSON with answers, probabilities, routing metadata and latency_ms. Weights and code are Apache-2.0, so the only cost is hardware.

Verdict

Laya is the most complete open implementation of the typed-decision idea available today, with a maturity that belies a one-week-old repo: routing, batching, CUDA-graph kernels, a Jev-compatible server, MCP, LangGraph, Nix, and a benchmark report that lists its own failures by issue number. The headline numbers deserve scepticism — Jev figures are second-hand, and the 0.766 win comes from a checkpoint fine-tuned on the benchmark’s own split — but the honest version is still compelling: a 421M-parameter model you own, answering in 33 ms, that beats a hosted API once specialised on your data. Fine-tune it, fit the temperatures, gate on answer_confidence, and keep permission in deterministic code.

Sources