Authenticated RCE through an AI agent's execute tool
During a real penetration test, any signed-in member could get a product's built-in AI assistant to run shell commands on the server — not through an exploit chain, but by asking. The model's refusal to run 'sensitive' commands looked like a control. It wasn't.
About these case studies. Every finding in this series is a real vulnerability CyberStrike discovered in a live penetration test. Customer names, hostnames, and any detail that could identify the target have been removed under the confidentiality agreement that governs the engagement. The technique, the impact, and the lesson are exactly as we found them — only the target is deliberately unidentifiable.
During a recent engagement against a multi-tenant AI product, we found that any signed-in member could get the product’s built-in assistant to run shell commands on the server. Not through a memory-corruption chain or a clever bypass — by asking the model to do it.
Severity: HIGH · CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H · CWE-78 (OS command injection) · vector: request body.
What the agent exposed
The product’s chat feature runs on the CopilotKit agent runtime (1.68.1). Behind it, the default agent was wired with an execute tool — a general-purpose server-side shell — and offered to the LLM with no allow-list of permitted commands. The design trusted the model to decide what was safe to run.
Any authenticated tenant member can POST an agent/run request to the chat endpoint that instructs the model to call execute. The model complies with commands it doesn’t read as dangerous, runs them in its backend sandbox, and returns the real output straight back into the conversation.
What we confirmed
An agent/run request telling the model to run id; hostname; pwd; uname -a came back with genuine shell output: a low-privilege service account on an Amazon Linux 2023 (aarch64) host. A second command, echo $((123456+654321)), returned 777777 with exit code 0. The shell was real, and it belonged to us for the length of a chat message.
The refusal that looks like a control
The model did decline some things. Ask it to read /etc/passwd, dump environment variables, or reach the cloud metadata endpoint, and it refuses — it recognizes those as sensitive.
That refusal is the trap. It feels like a security boundary, so it gets mistaken for one. But it is model policy, not authorization: nothing in the system stops the tool from running those commands. The only thing standing in the way is a model that currently chooses not to — and a model’s choices are steerable. During testing, indirect prompt injection through a document uploaded or pasted into the chat already drove read_file and execute tool calls on its own. “It won’t do that” is one well-crafted instruction away from “it will.”
Why it matters
This is code execution in a backend tool sandbox reachable by any authenticated member of any tenant. Where it goes from there depends entirely on how well that sandbox is contained — its network egress, secrets on disk or in the environment, and the instance’s IAM role and metadata service. A low-privilege uid is not the end of the story; it’s the opening move. The distance between “annoying” and “breach” is sandbox hygiene the application can’t see from inside the chat window.
The fix
The generic advice — “validate input, follow secure coding practices” — misses what actually went wrong here. The specific fixes:
- Treat an agent’s tools as attack surface, not convenience. Scope
executeto an explicit allow-list of commands and arguments, or remove it. A general-purpose shell behind an LLM is remote code execution by design, no matter how well-behaved the model is today. - Put the authorization boundary in the system, not the model. The sandbox itself should enforce what the tool can touch — no network egress by default, no secrets in the process environment, no route to the metadata endpoint — so a jailbroken or injected model hits a wall the code owns rather than a policy the model picked.
- Treat every input to the agent as hostile. Uploaded and inline documents are code paths to tool calls. Indirect prompt injection is the delivery mechanism, and any text the model reads can rewrite what the model does.
- Assume the tool will be abused. The service account, its IAM role, and its network reach should all be scoped as if an attacker already controls the shell — because, through the model, any member can.
The pattern
This is the shape of a whole class of AI-product findings: a capable model, a powerful tool, and a safety story that lives in the model’s judgment instead of in the system around it. The model’s reluctance is real, but it is not a control you can count on, because the one thing an attacker gets to influence is the model’s reasoning. We keep finding the authorization boundary drawn in the wrong place — inside the thing the attacker is allowed to talk to.
Testing for it means thinking like the attacker who has a conversation, not just the one who has a packet. That is what CyberStrike’s AI red teaming is built to do.
Found by CyberStrike — AI-powered security testing. Target and identifying details withheld under NDA.
