As LLM agents increasingly execute high-impact actions through tool usage, the necessity for robust runtime safety monitoring has become critical. Researchers have introduced NEXUS (Neural EXecution Utility and Safety), a structured-plan safety monitor designed to apply a formal intervention policy during agent execution.
A Four-Tiered Intervention Strategy
Unlike traditional binary safety filters, NEXUS utilizes a graded escalation policy to select among four specific actions:
- Allow: Permits the agent to proceed with the action.
- Block: Halts the execution due to safety violations.
- Request Confirmation: Interrupts the process to seek human approval.
- Request Revision: Prompts the agent to reformulate its plan.
Technical Framework and Benchmarking
NEXUS integrates deterministic safety rules with argument-level inspection and a calibrated logistic-regression risk score. This hybrid approach allows for nuanced decision-making. On a 128-instance synthetic benchmark, NEXUS achieved an F1 score of 0.949, significantly outperforming rule-only systems by 27.3 percentage points in intervention selection accuracy.
Efficiency remains a core feature of the system. With a median latency of 0.205 ms, NEXUS introduces less than 0.1% overhead to typical agent processing loops. The researchers have publicly released the code, benchmarks, and the calibrated risk scorer to support further development in the field of AI safety.
