A unit-test runner for AI output.
“AI成果物のUnit Test”
Define acceptance criteria as deterministic rules — contains, regex, JSON checks. Submit an AI output and get pass/fail plus a score back. Drops straight into CI/CD. It never calls an AI provider itself.
Screens
What it does
- 01A deterministic rule engine
- contains, not_contains, regex, min/max_length, json_valid, json_key_exists — six types, zero ambiguity.
- 02Organised as Suites and Cases
- Group related test cases into a Suite; each case can carry multiple AND-combined rules.
- 03Per-rule breakdown
- See exactly which rule failed a case, so debugging never means guessing.
- 04Run history and score trend
- Every run is kept, so you can track how the score moved after each change.
- 05A CLI for CI/CD
- The bundled ovcli posts JSON from stdin to the API and exits 1 on failure.
- 06Works with any AI system
- It grades whatever output you hand it, from any source — it never calls a provider.
How it works
Create a suite and cases
Create a suite, then add cases carrying acceptance-criteria rules.
Submit an AI output
Paste it manually, or POST it from CI via ovcli. Every rule gets checked.
Review the result
Pass/fail, score, and a breakdown of any failed rule. History tracks the trend.
Good to know
- Deterministic verdicts — the same output and rules always produce the same result.
- An unknown case_id fails immediately — a typo is never silently skipped.
- It never calls an AI provider — no API keys or provider integration to manage.