Stop Trying to Make AI Safe
Engineer for failure before it reaches the outcome
A guardrail is another component that can fail. What contains the error when it does?
Model failure should be expected. System failure should not.
The conversation
A model can pass an evaluation and still make a dangerous decision in production. Better models, grounding, and evaluations matter. The architecture must also answer what happens when those measures miss an error.
This talk places responsibility for reliability in the system surrounding the model. Matt Vegas introduces the Layered Safety and Accuracy System (LSAS), his specification for safety orchestration. Its five stages cover intent and risk classification, grounding, validation and filtering, escalation, and continuous learning and policy change. Each layer needs a defined job, a failure signal, and an owner who can intervene.
A hypothetical agent handling a financial adjustment provides the walkthrough. The model recommends an incorrect amount. The architecture must validate the proposed action against authoritative records, limit what can execute, preserve the evidence, and provide a path to review or recovery. Adding another model to approve the first is insufficient if both rely on the same faulty evidence.
Layers also create costs: latency, false alarms, operational burden, and new failure paths. Human review can become a queue or a rubber stamp. The engineering task is to choose controls according to the consequence of failure, test them independently, and decide when automation should stop.
The goal is practical containment and visible accountability. No model, guardrail, or framework removes every risk.
Model failure should be expected. System failure should not.
What your audience takes away
- Trace a model error to the action or outcome it could affect before choosing a control.
- Give each LSAS layer a specific responsibility, observable failure signal, and accountable owner.
- Validate consequential actions against authoritative evidence and independently enforced constraints.
- Test shared failure paths, escalation capacity, and recovery instead of counting guardrails.
- Choose when to abstain or stop automation, and make remaining risk explicit.
The frameworks behind the talk
LSAS — Layered Safety and Accuracy System
Matt Vegas' five-stage specification for AI safety orchestration: intent and risk classification; grounding; validation and filtering; escalation; continuous learning and policy change. Controls require implementation and verification in the system that uses them.