NeurIPS 2026: A Ten-Dimension Intervention Gives Agents a Safety Shield with over 97% Accuracy

After the AI agent began to take over the terminal, security issues also entered the execution field. When the intelligent body is able to read files, write scripts, run commands, and continuously adjust the next step according to system feedback, an erroneous decision can affect the actual file and system status.

How to make it possible for an intelligent organism to identify risks that may occur in the course of completing a task in a timely manner is an important issue in research into the safety of intelligent organisms. Long-range tasks involving multiple rounds of execution in particular test protection mechanisms.

A directory inspection, a script creation, a configuration modification, all require a combination of task objectives, risk assessment of previous operations and recent environmental feedback. Initial command review for single-cycle execution is difficult to cover the entire execution process of long-range tasks.

A team of researchers from institutions such as the University of Georgia, the University of Rogers, and the University of Southern California have proposed Agent MechSuits to try to put on a set of "security avionics" that can detect internal states for command line intelligences: read the security signals inside the model before each step of action is generated, detect risk, and follow-up behavior through 10 hidden characteristic dimensional interventions in a single layer. The main model weight has been frozen in the process, and has been received by NeurIPS 2026.[3]

Address of the paper:https://arxiv.org/abs/2606.22673

The main page of the project is as follows:

Agent MechSuits is providing security protection to multiple command line smart phones.CLIInternal state of the Agent, implementation of sparse interventions within a single model layer. Current steps of three test modelsThe risksThe average accuracy of detection is 97.32%.

The CLI smart body interacts with the computer via the command line. It needs to incorporate the results of previous actions, the system returns, and the original tasks into the context, and then decide what to execute next. This means that the security state will change with the execution history.

For example, an intelligent organism may first examine a directory, then create a script, and then modify the configuration. The previous operation may appear to be just preparatory work, while the real dangerous behavior appears in the subsequent steps.

External safety pads can check behavior and provide protection for the execution process according to the rules. Agent MechSuits further introduces concealed features within the model.Whether these internal states that are updated as tasks are executed will help the system identify the currentThe risksIt's time to change the next move.

Figure 1 of the paper depicts a scenario of a multi-stage risk demonstration. In the face of destructive requests, some system checks performed in the early stages of a smartphone do not appear to be dangerous, but follow-up may generate operations that lead to data loss.

Figure 1 demonstrates the risk of multiple rounds of CLI and the current steps of security intervention.

The paths that are not included in the chart continue to lead to harmful execution. After joining Agent MechSuits, the system detects the risk at the current steps and publishes the signal-level intervention, while the model explains the risk and refuses to provide the relevant code. This demonstration scenario explains the need for continued inspection and timely intervention.

Agent MechSuits consists of two stages of interconnectedness: detecting the current execution status and intervening in high-risk status.

Read security signals with a linear probe

At each interaction step, the system executes a complete historical input model, extracts the hidden state of the specified middle layer's last input token, and hands it to a lightweight linear probe. This probe can be understood as a simple classifier used to determine whether the current internal state is associated with malicious execution.

The research team trains a probe for each target model and replicates the different execution steps of the model. The state of the probe is constantly changing, so that it can monitor the progress of the task for risk detection.

Demonstrated intervention before action is generated

Based on the weight of the well-trained probe, the research team pre-selected the maximum 10 hidden dimensions for the category to distinguish the weight and constructed the intervention vector.

Once the risk is detected, the system applies the vector to each input token position in the same layer. After adjustment, the characteristic continues to participate in subsequent calculations, guiding the model to generate safer actions or reject harmful requests. Here, the "10D" refers to the range of intervention when running.

Adjusting the intensity of intervention according to current steps

The same intensity of intervention may have different effects in different tasks. For this reason, the research team introduced an external language model controller to evaluate the safety and practicality of candidate outputs under different intensities, selecting the results appropriate to the current steps. The intervention was triggered only after the probe identified the risk.

Figure 2 Risk detection and adaptation of Agent MechSuits interventions.

In order to study the relationship between internal states and execution risks, data is required to be able to track the entire interaction process. The research team built Mechanistic AgentSafety, or the MAS benchmark.

The MAS contains 194 missions, including 98 malicious scenarios and 96 normal scenarios. The study uses LLaMA-3.1-8B, Qwen-2.5-7B, and Gemma-2-9B to build white box CLI smartphones that allow them to perform tasks in a replicable Docker environment, recording actions, environmental feedback, and corresponding internal states.

Data labeling is refined into interactive steps, focusing on whether the current action directly implements or triggers dangerous behavior. The research team also constructed normal examples of surface structures similar to hazardous commands, reducing the likelihood that detectors rely on the appearance of commands to make judgments.

Training sets and test sets are divided into 6 to 4 tasks, with different steps of the same task not going in both directions at the same time.

This design allows the study to analyze two issues simultaneously:The risksWhere the execution chain is located and what states within the model can support timely intervention.

In the current step-by-step testing of MAS, AgentMechSuits achieved a higher accuracy with F1 scores on three tested models.

Table 1 Results of the current risk detection step.

The accuracy of the three models was an average of 97.32%. Agent MechSuits achieved the highest current step detection accuracy in all three models compared to the external language model arbitrators and security guards embedded in the paper. The five random seed tests in the appendix also maintained similar detection performance.

Layer-by-layer analysis further found that safety-related signals were already more apparent in the early to middle layers of the model. Simple linear probes were able to extract effective information from the hidden states of these layers, providing an interface to be studied further for operational monitoring.

Figure 3 Linear probe detection accuracy for different model layers. The horizontal axis is layer numbering and the longitudinal axis is testing accuracy.

The three curves in the diagram correspond respectively to the three tested models. The detection performance of the different layers is not exactly the same, and some subsequent layers will drop significantly. The results support the reading of safety signals in the more appropriate intermediate layer and further testing whether the layer can be used for behavioural intervention.

After identifying the risk, to what extent can internal interventions change the behavior of an intelligent organism? The research team selected the first harmful step in a malicious testing task, comparing the effects of different defense methods on the results generated. In the paper's risk mitigation assessment, Agent MechSuits lowered the attack success rate indicator ASR in all three models.

The success rate of attacks in risk mitigation assessments is given in Table 2. The lower the numerical value, the better.

Among these, Qwen-2.5-7B's ASR decreased from 91.30% to 4.35%, down 86.95 percentage points. The three models using the ASR after Agent MechSuits were also below the baseline of RepE expression engineering and Self Reminder self-reminder in the paper.

The results of this group used the quality of the paper's language model arbitrators, and the output of rejection or risk awareness can only be judged as safe. The key steps chosen for understanding these figures are to consider both the specific objects of assessment and the way in which they are judged.

The study also examined the impact of the entire defense process on normal tasks. On the normal tasks that were successfully completed by the original models, after enabling detection and intervention, the task success retention rates for LLaMA, Qwen and Gemma were 95.71%, 91.30% and 93.18%, respectively, and the three-model equality average was 93.40%.

At the same time, interventions are still costly. The average incidence of generated crashes, such as repeated output or sudden interruption, has risen from 2.24% in the original model to 18.54% in the evaluation of malicious steps. How to maintain stability and availability of output while reducing risk remains an important direction for further improvement.

In order to further test whether the direction of the selected expression affects actual behaviour, the research team started from a model that had explicitly rejected the harmful request and observed changes after reverse imposition of the expression intervention.

Figure 4 reverses the example of a model-rejecting behavior with a reverse representation of the model-changing intervention.

Related experiments covered 16 selective rejection states in LLaMA and 27 selective rejection states in Gemma. The observed behavioural changes provide additional evidence for the causal effects of the chosen direction of expression.

The team further tested a question: had the model already contained a warning signal in the first step of the dangerous move?

The study repeated the current steps to test the probe for the internal state prior to the harmful action. In 13 samples of LLaMA, the probe identified 12 samples. In 31 samples of Qwen, the probe identified 30 samples, with a corresponding recall rate of 96.77%.

This discovery provides preliminary evidence for anticipating the risk of execution. The specific sources of harmful signals will be explored in subsequent work.

Agent MechSuits demonstrates the dual value of internal representation in intelligent security: it can provide risk detection signals and can be an intervention point for adjustment behavior. For systems that require continuous execution of tasks, this adds a control interface that can be updated with context.

The current approach relies on access to the internal state of the white box model. Risk-triggered intervention intensity searches require the invention of an external language model, with an average of about eight rounds of candidate generation and evaluation per intervention in the paper, which still entails additional costs.

The paper's suggestion injected pressure test found that the current probe was not yet able to stabilize the identification of malicious targets entering from environmental feedback. After external triggering intervention, the same signaling direction could still change behavior, indicating that risk detection and behavior control need to be tested and improved separately.

In Figure 6, the user requests to find a file containing "123" in the specified directory and perform the operation specified by the file.

Figure 6 Environmental suggestions to inject an external touch into the scene to produce a signal to intervene. The model identifies hazards and ends execution.

Before the intervention, the model is prepared to comply with the hazardous operations in the file. After an external triggering intervention, the model clearly indicates that the operation will destroy the system file and subsequently stop the execution and end the task.

Current probes are unable to reliably automatically identify such injections, so this example shows that in the same set of externally triggered intervention experiments, Qwen's ASR decreased from 86.7% to 6.7% and the failure rate remained at 0%.

These numbers responded to the suggestion of injecting pressure tests, evaluating conditions that were different from the direct malicious mission experiments of the previous MAS. The direction of more applied prospects is to allow internal expression monitoring to work in sync with sandbox, authorization control, command inspection and environmental content monitoring.

How to continuously perceive the risks in the execution process and intervene appropriately in key steps will become an important issue for building a reliable intelligent body as the intelligent body takes on an increasingly long chain of tasks.

The first author, Weidi Luo, is now a PhD student in Computer Science at the University of Georgia in the United States, where her research focuses on AI security and AI security. As the first author (with co-first author), she has published numerous papers at international academic conferences such as ACL, NAACL, EMNLP, COLM, ICLR, NeurIPS. The University of Georgia's Intelligent Security team is led by Professor Liu, whose main research areas include big model and AI security, as well as applications in science, medicine, and education.