Interpretive variability assessment qualifies how and why generated interpretations differ across controlled contexts.
The framework exists to prevent over-reading isolated tests. It separates normal wording variation from variation that affects meaning, source authority, category, proof, or recommendation.
Minimum observation record
Each observation must preserve:
- target entity, source, claim, product, or category;
- canonical source or baseline, when available;
- query and language;
- system or surface observed;
- web-search or retrieval status, when visible;
- approximate context such as region, session mode, or device class, when relevant;
- timestamp;
- cited or activated sources, when visible;
- raw output;
- qualification decision and reason.
Assessment axes
IVA-1: retrieval variation
Cited or activated sources differ across contexts.
IVA-2: selection variation
Comparable sources exist, but different sources dominate the answer.
IVA-3: representation variation
The role, category, limits, proof, or differentiators of the target change.
IVA-4: recommendation variation
The target moves between absent, mentioned, recommended, first recommended, or displaced by a competitor.
IVA-5: context variation
Region, language, session state, tool mode, or time window changes the answer substance.
IVA-6: delivery-layer variation or fixation
The observed answer may be fresh, cached, routed, approved, refined, or assembled from an older retrieval state. Identical outputs can therefore indicate either stability or delivery-layer fixation.
Qualification states
- Stable: wording differs, meaning holds.
- Fragile: minor displacement appears but canon remains recoverable.
- Unstable: category, proof, or recommendation changes materially.
- Drifted: output contradicts canon or baseline.
- Inconclusive: evidence is insufficient or context is uncontrolled.
Evidence rule
A single output cannot establish stability. A repeated pattern can support a stability, fragility, or drift finding only if the test conditions and raw outputs remain reconstructible.
Relationship to interpretive observability
Interpretive variability assessment is a component of interpretive observability. Observability provides the traces, metrics, and evidence needed to decide whether variability is noise, an early warning, or a governed drift condition.
Native vs delivered object
A valid assessment should state whether the observed object is a native model interrogation or an application-delivered answer. If the path is unknown, claims must be downgraded to delivered-surface observation.