The problem this solves
Insurance rate and form filings are one of the least forgiving places to bolt an LLM onto a workflow. California’s Proposition 103 requires rate factors to be considered in a specific, legally mandated order (safety record, then annual mileage, then years of experience), and a filing that gets that order wrong isn’t a UX bug, it’s a compliance exposure. When an AI system extracts or reasons over that kind of document, “the model was probably right” isn’t an acceptable standard. Someone has to be able to check.
Novus Inspector is a two-layer answer to that problem:
- The Guardrail: a FastAPI/Pydantic sidecar that mathematically validates extracted filing data against deterministic state rules (Prop 103 rank ordering, schema compliance) before it reaches the system of record. No LLM judgment is involved here at all; this layer is intentionally boring and deterministic.
- The Lab: a DeepEval/Ragas evaluation harness that scores a RAG pipeline against a golden dataset (Faithfulness, Answer Relevancy, Hallucination) and runs an adversarial “poison pill” suite, deliberately wrong answers fed to the evaluator to prove it actually catches fabrication instead of rubber-stamping everything. Output is a “Decision Integrity Report” (PDF/HTML/Markdown/email): an evidence packet, not a marketing claim.
That second layer is where this case study lives. Because here’s the uncomfortable question a compliance tool has to answer about itself: who checks the checker?
Where intelligence belongs, and where it doesn’t
A recurring failure mode in AI-adjacent builds is putting a model everywhere it’s not needed. Novus Inspector deliberately doesn’t:
| Layer | Mechanism | Why |
|---|---|---|
| Rank-order / schema compliance | Deterministic Pydantic rules | Legally fixed, zero tolerance for drift; an LLM has no business “interpreting” Prop 103 |
| RAG answer generation & scoring | LLM (generation) + LLM-as-judge (RAGAS/DeepEval metrics) | The judgment calls (is this answer grounded, relevant, hallucinated) are inherently non-deterministic |
| Golden dataset authorship, adversarial scenario design, final sign-off | Human | Someone has to decide what “correct” means before anything can be scored against it, and someone has to be accountable for what goes to a client |
The report itself is explicit about this split (a human “attestation/sign-off” block is part of the standard output): the tool doesn’t try to remove the human, it tries to make the human’s review fast and evidence-based instead of a leap of faith.
The part that makes this a real case study, not a demo narrative
The Decision Integrity Report exists to answer “does the AI system actually behave the way it claims to?” That question turns out to apply recursively: the report generator is itself software that summarizes underlying data, and summarization is exactly the kind of step that can quietly drift from the thing it’s summarizing. Over three review rounds, comparing the report’s own headline claims against the raw evaluation JSON underneath it, that’s exactly what happened, twice.
Round 1. The adversarial section showed 3 of 6 poison-pill test cases and reported “100% detected,” as if that were the full set. A whole test suite (retrieval quality) was silently missing from the report with no note that it was scoped out. Reasoning traces from an unrelated prior run were bleeding into the report, traced to a missing run-ID filter on the query that selects which traces to attach. Different mechanisms, same pattern: something got silently narrowed or mismatched, and the report presented the result as if it were complete and correct.
Round 2. Two of those were fixed correctly, and the fix pattern was good (“say nothing captured” beats “say something that doesn’t match”). But a regression appeared: the performance benchmark table showed 0.00 across the board on every metric while still marking the row PASS, directly contradicting the report’s own stated pass threshold on the same page. And one issue carried over in a subtler form: the executive summary still claimed “100%, 6/6 threats detected, none bypassed,” while the detail section two paragraphs down showed 2 of those 6 as PARTIAL (only one of two detection metrics fired, not both).
Round 3. Traced the surviving bug to its actual source instead of documenting it as a known issue. The report generator computed the headline “caught” count with:
poison_caught = sum(1 for r in poison_results if any(not m['success'] for m in r['metrics']))any(...) counts a scenario as caught the moment one of its two detection metrics fires. The detail table, built separately, correctly required both metrics (all(...)) before calling something fully CAUGHT versus PARTIAL. Two code paths, same underlying data, two different definitions of “caught,” which is exactly how a report ends up contradicting itself without anyone writing a false statement on purpose.
Fix: derive the headline stat from the same per-scenario classification the detail table already used, across all four output formats (the Markdown report had no caveat language at all; the HTML version had partial caveat logic, but it branched on the old, loose definition of “100%,” so it could never actually fire once the fix landed, and had to be rewired to key off the real partial/missed counts instead). Verified by regenerating the report against the actual result JSON, not just re-reading the template. Net effect on the run inspected: Overall Status moved from CONDITIONAL to ACTION REQUIRED, and Adversarial Integrity moved from a misleading 100% (6/6) to an accurate 66.7% (4/6 fully caught, 2 flagged for review), the honest number, especially given 0 of 1 performance test cases actually passed in that same run.
Then, because round 2 had already burned the lesson “a top-line number can silently drift from its detail rows once, why not twice,” every other Section 1 stat got the same cross-check against its detail section in the same pass: operational confidence, retrieval pass rate, and the reasoning-trace count and its run-scoping. All checked out.
Why this is the actual proof point, not a side quest
It would be easy to build a demo that shows an AI system correctly flagging a fabricated insurance claim and call it a day. That’s a good look. It’s also not what makes a system trustworthy enough for a regulator or a skeptical client to sign off on. What does is a track record of the tool being audited as rigorously as it audits everything else, including finding real bugs in its own reporting, in its own code, with a real root cause and a real fix, not a tuned example. That discipline, verifying that the thing supposed to prove trustworthiness is itself trustworthy, is the job, more than any single model call.
Current state and what “deployment” actually requires next
In the spirit of not overselling: this is a single-tenant, unauthenticated evaluation service today. CORS is open, there’s no per-run auth on the trigger endpoint, and a full evaluation suite run is synchronous and can take 10 to 20+ minutes under LLM-judge rate limits. None of that blocks a scoped pilot; it means the honest next phase is an adoption plan, not a “flip it on” plan:
- Shadow mode: run the Lab against a client’s real filings/policy documents in parallel with their existing review process, producing reports nobody acts on yet, purely to validate the golden dataset and adversarial scenarios are actually representative of that client’s domain (the current adversarial suite is CA-homeowners-specific and would need client-specific authoring; this is documented as a known scoping step, not a surprise).
- Gated pilot: reports get used, but every report still requires the human sign-off block before anything reaches an end client, with
CONDITIONAL/ACTION REQUIREDstatuses treated as hard stops, not soft warnings. - Gradual autonomy, with a network boundary decision made explicitly: before any run trigger or report endpoint is exposed outside a controlled deployment, auth/reverse-proxy and multi-tenant isolation need to be designed, not assumed. This is a deliberate, scoped decision, not a default.
- Monitoring: every run archived and diffable (already supported via
--labelrun archiving), so a client or regulator can be shown not just “the current report” but the trend across runs.
A self-service, customer-facing flow (a prospect runs an evaluation and gets a report automatically with no human in the loop) is a real future direction, but it’s a separate, larger scoping exercise from this write-up, precisely because of the audit story above: an unsupervised, automated report has to be provably self-consistent first. That work starts from here, not before it.
Stack: FastAPI, Pydantic V2, DeepEval, Ragas, ChromaDB (persistent vector store), Ollama (embeddings), Gemini/Anthropic as pluggable LLM providers, Jinja2 + xhtml2pdf for report rendering.