How to evaluate AI agents before launch

Write the intended outcome, allowed tools, forbidden actions, evidence source, maximum cost and latency, approval points, and a stop condition before choosing a metric. A fluent answer can still accompany a wrong tool call or partial write.

Build cases from normal tasks, difficult edge cases, past failures, denied actions, stale or hostile content, tool timeouts, empty retrieval, and recovery. Keep every case tied to an exact product revision and expected behavior.

Use deterministic checks where the rule is exact and calibrated model-based evaluators where quality needs judgment. Human review remains necessary for ambiguous criteria, high-impact changes, and the launch decision.

A few successful chats are examples, not a dataset. Average scores can hide one irreversible failure, so high-impact actions need exact denial and approval cases.

An LLM judge is not ground truth. Calibrate it against human labels, keep deterministic checks for exact rules, record failures as failures, and never turn model or billing errors into passes.

Evaluation is not security testing, production monitoring, or customer validation. Keep those evidence lanes separate even when they share cases, traces, or release thresholds.

Prostir has one implemented Evaluations module shared by Agent, Skill, and Company. It stores typed cases, evaluator profiles, bounded runs, results, failure signals, and improvement proposals under the exact product owner.

Evidence enters only after explicit selection and consent. Deterministic checks can run without a judge; judge-backed checks use Microsoft.Extensions.AI.Evaluation and require explicit model-credit authorization and a positive budget.

A run never mutates or publishes a product. A cited proposal stays pending until a person accepts it, and application rechecks the exact current draft revision before changing only that draft.