The operational question
In this illustrative research scenario, a evaluation lead uses AI to compare behavior under a defined test. The workflow draws on scenario envelopes and recorded model responses. Its central risk is an aggregate score concealing a mission-critical error class. The design question is how to publish an evaluation finding for the authorized review audience while preserving the conditions covered by the evaluation. This is a proposed evaluation scenario, not a report of an Archetypal customer deployment or a demonstrated operational outcome.
Preserve source provenance
An answer becomes reviewable when its relationship to the underlying evidence remains visible. Keep the source identifier, relevant time, permitted use, and transformation history attached to the information that matters for the decision. Distinguish a direct observation from a report about an observation, and distinguish both from the model’s own interpretation. These distinctions should survive summarization. Do not ask a citation to establish more than its source can support. A source may confirm that a report was written without confirming that the reported event occurred. A reference may be current while the underlying observation is old. Reviewers need enough context to identify those differences.
Put the control in the workflow
Place this review immediately before the team can publish an evaluation finding. The evaluation lead should see the proposed result beside the relevant parts of scenario envelopes and recorded model responses. Identify which statement is supported by a source, which is an interpretation, and which remains unresolved. Carry the conditions covered by the evaluation into the decision record rather than relying on a reviewer to remember it from another screen. If the evidence does not establish the condition required for release, route the case to its owner with a concrete question. The interface should make the missing fact discoverable and the next action clear.
A test that can change the design
A source is replaced with an older but similarly worded record. The expected result is a visible freshness difference and a review of whether the evidence still applies. Run the case using a fixed version of the scenario and the policy under review. Ask an independent reviewer to identify the decisive fact before seeing the system’s disposition. Compare that interpretation with the result. Where they disagree, preserve both explanations and inspect whether the difference comes from the rule, the available evidence, or the interface. For frontier-model evaluation, include test version, model configuration, and labeled failure evidence in the review packet. Repeat the test after a correction and retain the original failure as part of the evidence.
Evidence to retain
The minimum useful record connects the purpose of the task, test version, model configuration, and labeled failure evidence, the applicable policy version, and the final disposition. Add the identity or role of the responsible reviewer, the conditions attached to approval, and the unresolved questions. If the team proceeds, distinguish the approval from an observed completion. If it stops, explain what evidence or authorization would allow another review. Keep source permissions attached to the record when it moves to the authorized review audience. Do not assume that permission to read the initial source includes permission to reproduce it in every downstream system.
What a result would establish
A successful run would show that this configuration recognizes the tested boundary for frontier-model evaluation and gives the evaluation lead an interpretable next step. It would not establish complete coverage of other audiences, source conditions, applications, or mission environments. Report the scope with the finding. Review any decision to publish an evaluation finding under changed conditions as a new applicability question. The strongest next experiment is usually the smallest change that could make the current conclusion false.
Review before wider use
Ask the workflow owner whether the proposed control is understandable at the point of use. Ask the policy owner whether it preserves the source requirement. Ask the evaluator whether the test can distinguish a real improvement from a change in presentation. Finally, ask the deployment owner what happens when scenario envelopes and recorded model responses are unavailable or the integration no longer observes the required event. Agreement among these roles should be documented as a set of decisions and remaining conditions, not compressed into an unsupported statement that the system is universally ready.
Purpose in this scenario
State the immediate task and the downstream use separately. The same output may be acceptable for an internal draft but unsuitable for a consequential decision or a broader audience. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Authority in this scenario
Identify the role empowered to make the decision. Record the source of that authority and any conditions attached to a delegation. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Audience in this scenario
Name the intended receiving group. Reassess the decision if the result is forwarded, summarized for another group, or included in a new workflow. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Freshness in this scenario
Record when a source was observed and when its current applicability was checked. A recently generated summary does not make old evidence current. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Completeness in this scenario
Identify the information required to make the decision and the information actually present. The absence of a required fact should remain visible. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Conflict in this scenario
Preserve contradictory sources and competing rule interpretations long enough for an authorized reviewer to resolve them. Do not make disagreement disappear through summarization. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Identity in this scenario
Use the identity information required for attribution and authority checks. Keep uncertain or incomplete identity evidence distinct from a verified assignment. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Permissions in this scenario
Connect each meaningful action to its permitted resources, purpose, duration, and owner. Review the difference between technical capability and delegated authority. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Exceptions in this scenario
Give an exception a reason, owner, scope, and expiration. Explain whether it applies to this case alone or creates a candidate for a wider policy change. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Disposition in this scenario
Define what each outcome does in the workflow. Allow, block, controlled approval, escalation, and indeterminate should lead to observable and different next steps. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Explanation in this scenario
Include the facts that caused the decision and the conditions that could change it. A restatement of the rule is not enough to explain its application. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Review in this scenario
Show who inspected the case, what evidence they had, and whether they had the authority required for the disposition. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Retention in this scenario
Keep the information necessary for the authorized governance purpose. Document whether the record stores source content, identifiers, or a reference to a controlled source. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Correction in this scenario
Define how a mistaken source, label, or attribution can be corrected without erasing the history needed to understand the original decision. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Recovery in this scenario
Assign responsibility for restoring a known configuration when a change produces an unacceptable result. Record the criteria for resuming ordinary operation. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Outcome in this scenario
Distinguish intended action, attempted action, and observed completion. A generated recommendation is not evidence that the real-world result occurred. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Transfer in this scenario
Review what crosses an organizational or system boundary. Preserve source restrictions and avoid copying more information than the receiving decision requires. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Expiration in this scenario
Attach an appropriate review trigger to permissions, policies, and precedents. Time alone is one trigger; a change in circumstances can be another. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Coverage in this scenario
Record which events and applications were tested. An inventory of connected systems does not establish complete visibility into every relevant workflow. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Dissent in this scenario
Keep material disagreement available to later reviewers. A final disposition should not obscure a plausible contrary interpretation or the evidence supporting it. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Configuration in this scenario
Connect the result to the actual model, application, policy, and environment used. Names without versions may be insufficient for comparison. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Reproduction in this scenario
Keep enough of the scenario and procedure for another reviewer to understand how the result was produced and where exact replay is not possible. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Impact in this scenario
Describe the consequence of a wrong result in the terms of this workflow. This helps determine which failures deserve separate reporting and review. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Ownership in this scenario
Make recurring follow-up work someone’s responsibility. An identified gap without an owner and a next step can remain unresolved indefinitely. In frontier-model evaluation, the evaluation lead should apply this check to the proposed decision to publish an evaluation finding. Use test version, model configuration, and labeled failure evidence to make the review concrete. Explain how the result changes if the condition is absent, disputed, or no longer current. Record the expected disposition before running the scenario so that the evaluator cannot quietly redefine success after seeing the output. The receiving audience is the authorized review audience; preserve the limitations they need to interpret the result.
Related control: define the mission boundary
The first governance decision is where the system’s authority begins and ends. Write the intended task as a bounded activity with an identifiable owner. Specify the information the workflow may use, the audience it serves, and the decisions it may support. Then identify the actions that remain outside that permission. A description of what a model can do is not an authorization to do it. A good boundary makes change visible. If the audience, information class, tool access, or intended use changes, the workflow should be able to recognize that it is operating under a new set of conditions. The owner can then decide whether an existing approval still applies or whether another review is required. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: translate policy into a decision
A useful rule preserves the meaning of its authority while making the next action clear. Identify the source authority, the condition that activates the rule, the facts needed to evaluate that condition, and the permitted dispositions. Separate requirements from guidance and ordinary cases from exceptions. Keep ambiguous interpretations available for review rather than hiding them inside an implementation. A rule should describe more than a prohibited phrase. The same words can have different meanings in different contexts, and the same prohibited act can be described without the expected words. Scenario design should examine both directions: harmless requests that resemble a violation and violations expressed indirectly. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: design a usable human review
Oversight works only when a person has the information, authority, and time to exercise judgment. Present the proposed action, the facts that could change the decision, the applicable rule, and the unresolved uncertainty together. Make the available dispositions understandable. A reviewer should be able to accept with conditions, reject, request clarification, or route the case to a more appropriate authority. Avoid turning review into a ritual of confirmation. If the interface makes acceptance easier than understanding, a human approval can become a weak signal. Study the effort needed to find contrary evidence, the clarity of exception conditions, and whether the reviewer can identify who remains responsible after approval. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: keep permissions bounded
A delegated goal needs an equally explicit account of what the system is allowed to do. Define permissions at the level of meaningful actions and resources. Connect them to a task owner, a purpose, an operating period, and conditions for revocation. Granting access to a tool is different from authorizing every action that tool can perform. Record both the technical access and the decision authority. Review permission changes as part of the workflow. New resources, a broader audience, or a different execution environment can change the risk of an otherwise familiar task. A permission that was appropriate for a test may not be appropriate for an operational record or an external communication. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: make uncertainty actionable
Uncertainty is useful when it changes the next step rather than merely qualifying the prose. Identify what is unknown, why it matters, and what evidence could resolve it. Separate uncertainty in the source, uncertainty in interpretation, and uncertainty about policy applicability. Each may require a different response. An indeterminate outcome should have a defined owner and a path to clarification. Do not compress every kind of uncertainty into one confidence number. A model can sound confident while missing an essential source or misreading an exception. Describe the specific gap in terms the reviewer can act on, and preserve it if the result is summarized or transferred to another system. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: test the difficult boundaries
The most useful evaluation cases are the ones that distinguish a working rule from an attractive demonstration. Start with a permitted baseline, a clearly prohibited case, an ambiguous case, and a legitimate exception. Change one material factor at a time before testing combinations. Preserve the scenario, policy, model, configuration, response, and review label so that another person can reconstruct the result. Report failure classes separately. A missed restriction, an unnecessary block, an unsupported explanation, and an unusable escalation path affect the mission in different ways. Aggregate performance may help compare configurations, but it should not erase the particular boundary a deployment depends on. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: build a useful decision record
An audit trail earns its value by explaining a consequential decision after the original context has changed. Record the request’s governed purpose, the decisive facts, the applicable rule version, the disposition, and the responsible reviewer or system. Include conditions, unresolved questions, and the observed outcome when available. Keep the difference between a proposed action and a completed action explicit. Retain enough information to reconstruct the decision without treating unlimited capture as the default. Consider who may inspect the record, which source material can be linked rather than copied, and how the record should respond when a source is corrected or a policy is retired. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: control change after deployment
A system that passed an evaluation can leave its tested operating envelope without changing its product name. Track the versions and assumptions that matter for behavior: model, policy, application integration, source schema, permission configuration, and operating environment. Define which changes require a targeted regression test and which require a new deployment decision. Give recovery a named owner. Treat rollout as an evidence-generating activity. Start with a scope that makes failures observable, retain the prior configuration needed for recovery, and compare the intended behavior with the results seen in the workflow. A quiet system is not necessarily a correctly governed system. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Related control: learn from tested outcomes
A durable lesson carries the conditions that made it true, not just the conclusion that was convenient to remember. Connect a resolved case to its source rule, decisive facts, reviewer, dissent, and observed outcome. When retrieving it for another scenario, compare the conditions explicitly. Preserve reasons not to apply the precedent. A similar phrase is weak evidence that the same decision should follow. Use outcomes to decide what to test next. A recurring exception may reveal a policy ambiguity; a repeated false block may expose a poor scenario boundary; a successful intervention may depend on a reviewer who had information absent from the formal record. Each finding calls for a different improvement. Applied to frontier-model evaluation, this means examining the conditions covered by the evaluation before the team can publish an evaluation finding. The evaluation lead should decide whether the related control changes the evidence needed for the current decision. Keep that judgment connected to test version, model configuration, and labeled failure evidence and describe any condition that requires another owner’s review.
Exercise 1: purpose
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with purpose. State the immediate task and the downstream use separately. The same output may be acceptable for an internal draft but unsuitable for a consequential decision or a broader audience. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 2: authority
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with authority. Identify the role empowered to make the decision. Record the source of that authority and any conditions attached to a delegation. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 3: audience
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with audience. Name the intended receiving group. Reassess the decision if the result is forwarded, summarized for another group, or included in a new workflow. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 4: freshness
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with freshness. Record when a source was observed and when its current applicability was checked. A recently generated summary does not make old evidence current. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 5: completeness
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with completeness. Identify the information required to make the decision and the information actually present. The absence of a required fact should remain visible. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 6: conflict
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with conflict. Preserve contradictory sources and competing rule interpretations long enough for an authorized reviewer to resolve them. Do not make disagreement disappear through summarization. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 7: identity
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with identity. Use the identity information required for attribution and authority checks. Keep uncertain or incomplete identity evidence distinct from a verified assignment. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 8: permissions
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with permissions. Connect each meaningful action to its permitted resources, purpose, duration, and owner. Review the difference between technical capability and delegated authority. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 9: exceptions
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with exceptions. Give an exception a reason, owner, scope, and expiration. Explain whether it applies to this case alone or creates a candidate for a wider policy change. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 10: disposition
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with disposition. Define what each outcome does in the workflow. Allow, block, controlled approval, escalation, and indeterminate should lead to observable and different next steps. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 11: explanation
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with explanation. Include the facts that caused the decision and the conditions that could change it. A restatement of the rule is not enough to explain its application. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 12: review
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with review. Show who inspected the case, what evidence they had, and whether they had the authority required for the disposition. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 13: retention
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with retention. Keep the information necessary for the authorized governance purpose. Document whether the record stores source content, identifiers, or a reference to a controlled source. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 14: correction
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with correction. Define how a mistaken source, label, or attribution can be corrected without erasing the history needed to understand the original decision. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 15: recovery
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with recovery. Assign responsibility for restoring a known configuration when a change produces an unacceptable result. Record the criteria for resuming ordinary operation. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 16: outcome
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with outcome. Distinguish intended action, attempted action, and observed completion. A generated recommendation is not evidence that the real-world result occurred. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 17: transfer
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with transfer. Review what crosses an organizational or system boundary. Preserve source restrictions and avoid copying more information than the receiving decision requires. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 18: expiration
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with expiration. Attach an appropriate review trigger to permissions, policies, and precedents. Time alone is one trigger; a change in circumstances can be another. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 19: coverage
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with coverage. Record which events and applications were tested. An inventory of connected systems does not establish complete visibility into every relevant workflow. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 20: dissent
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with dissent. Keep material disagreement available to later reviewers. A final disposition should not obscure a plausible contrary interpretation or the evidence supporting it. Ask whether the evaluation lead can identify the change from the evidence packet and whether the system’s disposition responds to it. Record the rationale for any difference. If both cases produce the same result, determine whether the unchanged response is appropriate or whether the control failed to recognize a material fact. Preserve the conditions covered by the evaluation throughout the comparison. Do not introduce real restricted information merely to make the example realistic; an authorized synthetic case can express the same decision boundary.
Exercise 21: configuration
Create two versions of the frontier-model evaluation scenario. Keep the task, audience, and expected action constant, then change only the condition associated with configuration.
Authority Purpose.