kovacEvaluations
Measure answer quality before you change anything
Test questions written from your own documents, a gate before go-live, and an AI judge on live traffic.
Golden sets from your own documents
kovac writes test questions and ground-truth answers straight from a collection's own chunks.
A quality gate before go-live
A new agent or retrieval change runs against the golden set, and a regression blocks it from shipping.
An AI judge on live traffic
Sampled real conversations are scored for groundedness and relevance by a judge model you configure.
Full traces
Every run is traced end to end, with the candidates retrieved before and after reranking kept as separate records, so you can see exactly what the agent saw.