Just Smarter Eval

Benchmark, govern, and ship LLMs — on EU infrastructure.

Product datasheet
v2026-04-24

The evaluation and governance layer for regulated AI teams. Compare 37+ frontier LLMs on your own data, produce compliance-grade audit trails, and run the entire workflow inside the EU. Purpose-built for AI platform leads, governance teams, and CISOs who can't deploy LLMs on vibes.

What it does

Benchmark

Upload a dataset. Fan out across every model in the Brain Orchestra catalog (30+ today, growing as BO adds them). Get a scorecard — latency, token count, credits, quality — sortable by any column. Export to PDF / CSV / JSON.

Judge

LLM-as-judge scoring with configurable rubric. Claude Opus is the default judge; swap for domain-specific grounding. Per-row quality score per model.

Govern

Every eval run produces an admissible audit artifact: dataset hash, model versions, hosting tier, per-row I/O, timestamps, EU-region attestation. Covers AI Act Art. 15 accuracy-logging requirements.

Ship

Save suites. Detect prompt regressions before deploy. Compare model upgrades objectively. Wire into CI/CD via REST API. Prompt changes caught in dev, not by customers.

Model coverage — 37+ models across 7 providers

Provider	Models	Hosting tier eligibility
Mistral (France, EU-owned)	Small · Medium · Large · Codestral · Devstral · Pixtral (Large + 12B) · Magistral (Small + Medium)	EU StrictEU Cloud
Anthropic Claude	Haiku · Sonnet · Opus (latest)	EU CloudUnrestricted
OpenAI	GPT-4o · GPT-4o mini · GPT-4.1 (full + mini + nano) · GPT-5 (full + mini + nano) · GPT-o3 · GPT-o4 mini	Unrestricted
Google	Gemini Flash · Gemini Pro	Unrestricted
Amazon (Bedrock Stockholm + Frankfurt)	Nova Lite Sweden · Devstral Sweden	EU CloudUnrestricted
Moonshot (PRC, opt-in)	Kimi K2.6	Unrestricted

Live catalog at justsmarter.ai/eval — auto-syncs from Brain Orchestra every 48h, with the full per-change history at /changelog.

Eval kits — shipping at launch

Model Selection for RAG — upload reference docs + questions; get a Pareto curve of cost vs faithfulness across the full model catalog.
EU Legal Reasoning — 40 pre-built prompts on GDPR, AI Act, DORA, DPIA drafting; scored in EN / SV / DE / FR / IT / ES.
Hallucination Detection via Council — uses Just Smarter's multi-model Council + live web fact-check as ground truth; scores single-model runs against it. Unique to this platform.

Pricing at a glance

Just Smarter Eval is a free surface on top of Brain Orchestra. Connect a Brain Orchestra API key and your own Brain Orchestra account is billed only for the inference your evals run — there is no separate Just Smarter subscription, no seats, and no per-eval fee.

Tier	Price	Includes
Just Smarter Eval	Free	Every model in the catalog · batch eval runner · drift dashboard · CSV/JSON/Markdown exports · all hosting tiers (incl. EU Strict) · requires a connected Brain Orchestra key to run
Brain Orchestra	Pay-per-use	The inference layer Just Smarter runs on · you pay Brain Orchestra only for the tokens you use · EU residency + compliance tiers · one key powers Just Smarter and your own apps
Enterprise	On request	Dedicated EU environment · SAML / SCIM · counter-signed DPA + SCCs · custom eval kits · dedicated support · SLA

Why enterprises pick Just Smarter Eval over US-hosted alternatives

EU-native by design — Xalerate AB, Sweden. No US parent, no CLOUD Act reach. EU Strict tier keeps data with EU-owned vendors only.
Multi-model bake-off as the primary workflow, not a buried playground feature. Competitors treat it as a side panel.
Council / Superpower synthesis — novel ground-truth primitive using ensemble + web-grounded fact-check. No competitor offers this.
Compliance artifacts per eval run — AI Act Art. 15, GDPR Art. 28 DPA, SCCs, sub-processor transparency. See justsmarter.ai/trust.
UI-first — non-engineer AI platform leads can run evals themselves. No Python, no API-only workflow.

Try it now: The runner is live at justsmarter.ai/eval — paste prompts, pick models, run. Or book a 30-minute demo — you leave with a sample eval report on your own data and a clear pilot proposal. Email support@justsmarter.ai.