Deterministic Verification vs AI-Based Evaluation

A deterministic verifier uses explicit comparisons and canonical transformations. If measured latency is 642 milliseconds and the maximum is 900, the rule outcome is reproducible. The engine version, threshold and observation can be recorded in a compact receipt that another implementation checks without consulting the original service.

This model is well suited to bounded facts: numeric thresholds, timestamps, identifiers, signatures, hashes and state transitions. It is easier to test with known-answer cases and easier to explain during a dispute because the decision path is visible.

A model can assess natural-language quality, classify an answer or compare an output with a rubric that is difficult to encode as simple arithmetic. That flexibility is valuable, but the result may depend on model version, prompt, sampling settings and context. Even when temperature is reduced, the evaluation still represents a model-mediated judgement rather than a universal fact.

Model evaluations therefore need their own provenance. Preserve the model identifier, evaluation prompt, rubric, input commitment and raw response where permitted. A later reviewer must be able to distinguish the model's assessment from independently observed service facts.

A practical hybrid pipeline can ask a model to extract candidate facts or score a qualitative property, then apply deterministic validation around the result. Schema checks can reject missing fields. Confidence or disagreement thresholds can route uncertain cases to human review. Cryptographic commitments can preserve the exact prompt and output used.

The receipt should label each field by source and method. A latency value from a monitor, a classification from a model and a final threshold comparison are different kinds of evidence. Combining them is legitimate when the distinction remains visible.

Use deterministic checks when the agreement can be expressed as objective conditions. Introduce model judgement only for genuinely semantic or ambiguous questions, and design an escalation path for low-confidence cases. This reduces cost, variability and hidden policy changes.

PactVerity's operating Proof Engine deliberately focuses on deterministic conditions and caller-supplied evidence. It does not use a language model to decide whether an SLA passed. That boundary supports repeatability while leaving source collection and qualitative evaluation to separate components.