LLM proxy, MCP gateway, AI gateway: why the three converge, and what the enterprise should demand of the one it keeps

A tool that changed its story

In April 2025, a security research team published a short notice about tools that lie. An MCP server, the kind of plug-in that lets an AI agent read files or send messages, describes each of its tools in a few sentences. The model reads those sentences. The person who installed the server usually never does. The researchers showed that a description could carry instructions of its own, and that a server could change a tool’s description after a user had approved it. They called the second move a rug pull.[1]

Think about what a conventional LLM gateway would have recorded while that happened. Requests to an approved model. Token counts well inside the team’s budget. Prompts with no card numbers or Social Security numbers in them. Every check the gateway knows how to run would have come back clean, because the problem was never in the request. It was in what the model had been told to do next, and in the tool that did it.

That is the gap this piece is about. Managing requests is not managing actions, and an enterprise that governs only the first is governing the part of its AI estate that is easiest to see.

Requests and actions

LLM proxies grew up answering questions about requests. Which model may this application call, through which provider, with which key, at what rate? What happens when the provider is down? How many tokens did the claims team use last month, and what did they cost? Those are good questions, and the products that answer them — routing, failover, key management, token accounting, some inspection of what leaves in a prompt — became ordinary infrastructure within a couple of years.

An agent puts different demands on the infrastructure around it. It does not just call a model. It calls a model, reads the reply, picks a tool, passes it arguments, reads what comes back and calls the model again, sometimes dozens of times for one task, sometimes handing part of the work to a sub-agent it started itself. The model call is the least consequential step in that loop. The tool call is where a record gets updated, a message gets sent or a file leaves the building.

The industry’s own security lists draw the same line. OWASP’s Top 10 for LLM applications treats excessive agency, an agent given more functionality, permission or autonomy than its task needs, as a category separate from prompt injection.[2] The OWASP list for agentic applications, published last December, puts tool misuse and identity and privilege abuse among its first three entries.[3]

A widely read essay from the summer of 2025 reduced the danger to a combination it called the lethal trifecta. An agent with access to private data, exposure to untrusted content and a way to communicate externally can be talked into sending the data out by whoever controls the content.[4] None of those is a property of a model request. All of them are properties of the tools an agent holds and the order in which it uses them.

So a control point that sees only requests watches the steps where the least can go wrong. Governing an agent means seeing its actions, enforcing permissions on each one, and holding the thread of a multi-step interaction long enough to know that the tool call in step seven is acting on something read in step three.

Why the three are converging

The market has reached the same conclusion from several directions at once. Kong added MCP governance to its AI gateway in October 2025, with an OAuth 2.1 plugin and observability for tool use.[5] Microsoft describes the AI gateway in Azure API Management as one set of capabilities spanning model APIs, MCP servers and agent-to-agent APIs, built into the API gateway rather than sold beside it.[6] LiteLLM’s documentation describes a single control plane for models, MCP servers and agents.[7] Cloudflare put MCP servers behind portals with per-user access policies in August 2025, and AWS describes its AgentCore Gateway as one entry point for agent traffic to tools, other agents and models.[8][9] The open-source project once called Envoy AI Gateway, now Agent Router, added MCP routing, tool filtering and OAuth authorization in a release last November, and Apigee began aggregating MCP tool lists across approved API products in September.[10][11][12]

These are capable engineering teams arriving at the same place independently. The LLM proxy, the MCP gateway and the AI gateway are becoming one category, because the thing being governed is one interaction.

The protocol has moved the same way. For servers that require authorization over HTTP, the current MCP specification, revised in July, treats the server as an OAuth resource server.[13] The server has to publish where a client can obtain a token for it, the client has to name the server it wants the token for, and the server has to refuse tokens issued for anyone else.[14][15] It may not pass a token it received through to the services behind it.[13][16] Each of those rules is about authority over actions. None of them can be enforced by a component that sees only the model call.

Government is asking similar questions. In January, NIST’s Center for AI Standards and Innovation asked industry how to secure AI agent systems, naming least privilege and zero trust among the practices it wanted views on, along with limits on tools and approvals for consequential actions.[17]

The case for keeping them apart

There is a respectable argument for keeping these separate, and platform teams make it often. The LLM proxy belongs to the platform team, which cares about cost, latency and uptime. Tool governance belongs to security, which cares about permissions and data. Merge them, the argument goes, and you get one product that is mediocre at both jobs, two teams’ concerns in one change queue, and a single component whose failure stops every agent in the company.

The last point deserves a straight answer, because it is right about the risk. A converged gateway concentrates authority, and a control point that fails closed across the whole estate is an outage waiting for a bad deploy. No enterprise should accept one that cannot say what it does when its control plane is unreachable, how long it keeps enforcing the last policy it verified, and how a person overrides it in an emergency without erasing the record.

But separation fails at the seam where the risk now lives. The dangerous sequence — untrusted content read in one step, a sensitive tool called in another — crosses both products. If the proxy sees the read and the tool gateway sees the call, neither can stop the call because of the read. Correlating the two afterwards in a log pipeline produces an excellent incident report and no prevention. Two policies written by two teams in two languages will also disagree somewhere, and an agent will find the disagreement before the auditor does.

The worry about two teams is real, and it is organizational. It argues for one policy with clear ownership of each control. It does not argue for two enforcement points that cannot see each other.

What to demand of the one you keep

If the three are converging, the useful question is not which product to buy but what the one you keep must be able to prove. Six demands follow from the argument so far.

Receipts, not logs. Every decision, on a model call or a tool call, should produce a signed record of who asked, under which policy version, what was decided and why, chained so that a missing record shows. A log is what the vendor chose to write down. A receipt can be checked.

One policy, compiled everywhere. A rule should be written once and compiled into every enforcement point the company runs, including gateways from other vendors, and an enforcement point that cannot express a rule should refuse it rather than approximate it. One policy is also how two teams can share a control point without sharing a queue.

A job for every agent. An agent should run under a defined job, with exactly the models, tools, data classes and budget the job needs, rather than under a developer’s token or a shared service account. The gateway should be able to show what was granted beside what was used. We make that case at length in a companion piece.

Inspection in both directions. Checking what leaves in a prompt is necessary and not sufficient, because tool results and model responses carry injected instructions and sensitive data back in. Ask which direction a vendor inspects, and ask for its miss rates on a published corpus. A detector without a published miss rate is a promise.

Verification without the vendor. An auditor should be able to verify the record offline, with a public key obtained independently and no access to the vendor’s systems. If verification needs the vendor’s console, the evidence is the vendor’s opinion.

An appliance you can inspect. The data plane should run in the enterprise’s own environment from images pinned by digest, with signed configuration, egress limited to named destinations and a software bill of materials, so the platform team can check what it installs before it installs it.

One more request sits under the other six. Ask the vendor to say, for each of them, what runs in production, what runs in a lab and what is planned. A converging market produces a great many roadmaps described in the present tense.

Where ARC1 stands

We hold ourselves to the same list. ARC1’s signed governance receipts, two-person approval, policy compilation and offline verification are in our released product and run on our hosted deployment with synthetic data; no customer deployment is recorded yet. Our LLM proxy and inline MCP proxy are built and run in our lab. The MCP proxy refuses a call when a pinned tool’s definition has changed since approval, and on loopback in our lab it added 8.2 milliseconds at the 95th percentile.

Some of the six we do not yet meet in production. Inspection of responses and tool results is designed and not built, OAuth 2.1 for our MCP gateway is next, and agent personas and the appliance are in our lab. Our documentation marks every part LIVE, LAB or NEXT, which is the least a buyer should ask of anyone selling the one control point they keep.