6. API Management & AI Gateway

AI Gateway Concepts

The AI Gateway sits between clients and AI models/agents, providing rate limiting, content safety, load balancing, and observability — all through Azure API Management.

MCP Deep Dive

Model Context Protocol (MCP) standardizes how agents discover and call tools. Foundry's AI Gateway natively supports MCP, allowing agents to use any MCP-compatible tool server.

# MCP Tool Definition (JSON)
{
  "name": "get_weather",
  "description": "Get current weather for a location",
  "inputSchema": {
    "type": "object",
    "properties": {
      "location": { "type": "string" }
    },
    "required": ["location"]
  }
}

MCP Server Architecture

An MCP server exposes tools via a standardized endpoint. The AI Gateway translates between agent tool calls and MCP server requests.

Custom MCP Servers

# Python MCP Server Example
from mcp.server import Server, NotificationOptions
from mcp.server.models import InitializationOptions

server = Server("weather-server")

@server.list_tools()
async def list_tools():
    return [
        Tool(name="get_weather", description="Get weather",
             inputSchema={"type": "object", "properties": {"location": {"type": "string"}}})
    ]

@server.call_tool()
async def call_tool(name, arguments):
    if name == "get_weather":
        return {"temperature": 22, "conditions": "sunny"}
    raise ValueError(f"Unknown tool: {name}")

Rate Limiting

Configure rate limiting per user, per API key, or per model deployment to prevent abuse and manage costs.