<aside> 🎯 Refine ensures the solution keeps working as conditions change. This is not a one-time phase; it is ongoing.
</aside>
Objective: Respond to AI failures without making them worse. Detect, contain, investigate, fix, recover, learn.
When to use: When the AI system produces incorrect, harmful, or unexpected outputs at scale. Not for isolated edge cases handled by normal support.
Inputs: Incident detection (alert, user report, or monitoring finding), system access for investigation, rollback procedure documentation.
Position: Sentinel owns incident response. Smith executes containment and fixes. Architect determines root cause.
How to fill out: One row per incident. Document severity, what happened, root cause, resolution, and lessons learned. Update status as you move through phases: Investigating, Contained, Resolved, Post-mortem complete.
<aside> ⚠️ Containment often means increasing human oversight (moving from Tier 1 back to Tier 2), not shutting down. Full shutdown creates its own problems. Fix the root cause, not just the symptom.
</aside>
Objective: Define how much human oversight each AI decision type requires. Autonomy is earned through evidence, not granted by assumption.
When to use: When deploying AI that makes decisions with varying risk levels. Initialize during Solve; maintain throughout Refine.
Inputs: Complete list of AI decision types, historical accuracy data per type (if available), risk assessment framework, stakeholder input on risk tolerance.
Position: Sentinel owns tier assignments and reviews. Scout validates risk assessments.
How to fill out: One row per AI decision type. Assess risk level, assign initial tier (most start at Tier 3: Full Review), set accuracy thresholds for graduation, and document graduation criteria. Track days at current tier and next review date.