Run MCP servers in Kubernetes

Prerequisites

A Kubernetes cluster (v1.19+)
Permissions to create resources in the cluster
kubectl configured to communicate with your cluster
The ToolHive operator installed in your cluster (see Deploy the operator using Helm)

Overview

The ToolHive operator deploys MCP servers in Kubernetes by creating proxy pods that manage the actual MCP server containers. Here's how the architecture works:

High-level architecture

This diagram shows the basic relationship between components. The ToolHive operator watches for MCPServer resources and automatically creates the necessary infrastructure to run your MCP servers securely within the cluster.

STDIO transport flow

For MCP servers using STDIO transport, the proxy directly attaches to the MCP server pod's standard input/output streams.

SSE transport flow

For MCP servers using Server-Sent Events (SSE) transport, the proxy creates both a pod and a headless service. This allows direct HTTP/SSE communication between the proxy and MCP server while maintaining network isolation and service discovery.

Create an MCP server

You can create MCPServer resources in namespaces based on how the operator was deployed.

Cluster mode (default): Create MCPServer resources in any namespace
Namespace mode: Create MCPServer resources only in allowed namespaces

See Deploy the operator to learn about the different deployment modes.

To create an MCP server, define an MCPServer resource and apply it to your cluster. This minimal example creates the osv MCP server which queries the Open Source Vulnerability (OSV) database for vulnerability information.

my-mcpserver.yaml
apiVersion: toolhive.stacklok.dev/v1alpha1
kind: MCPServer
metadata:
  name: osv
  namespace: my-namespace # Update with your namespace
spec:
  image: ghcr.io/stackloklabs/osv-mcp/server
  transport: sse
  port: 8080
  permissionProfile:
    type: builtin
    name: network
  resources:
    limits:
      cpu: '100m'
      memory: '128Mi'
    requests:
      cpu: '50m'
      memory: '64Mi'

Apply the resource:

kubectl apply -f my-mcpserver.yaml

What's happening?

When you apply an MCPServer resource, here's what happens:

The ToolHive operator detects the new resource (if it's in an allowed namespace)
The operator automatically creates the necessary RBAC resources in the target namespace:
- A ServiceAccount with the same name as the MCPServer
- A Role with minimal permissions for StatefulSets, Services, Pods, and Pod logs/attach operations
- A RoleBinding that connects the ServiceAccount to the Role
The operator creates a new Deployment containing a ToolHive proxy pod and service to handle client connections
The proxy creates the actual MCPServer pod containing your specified container image
For STDIO transport, the proxy attaches directly to the pod; for SSE transport, a headless service is created for direct pod communication
Clients can now connect through the service → proxy → MCP server chain to use the tools and resources (note: external clients will need an ingress controller or similar mechanism to access the service from outside the cluster)

Automatic RBAC management

The ToolHive operator automatically handles RBAC (Role-Based Access Control) for each MCPServer instance, providing better security isolation and multi-tenant support. Here's what the operator creates automatically:

ServiceAccount: A dedicated ServiceAccount with the same name as your MCPServer
Role: A namespace-scoped Role with minimal permissions for:
- StatefulSets (create, get, list, watch, update, patch, delete)
- Services (create, get, list, watch, update, patch, delete)
- Pods (get, list, watch)
- Pod logs and attach operations (get, list)
RoleBinding: Connects the ServiceAccount to the Role

This approach provides:

Each MCPServer operates with its own minimal set of permissions
No manual RBAC setup required
Better security isolation between different MCPServer instances
Support for multi-tenant deployments across different namespaces

For more examples of MCPServer resources, see the example MCP server manifests in the ToolHive repo.

Customize server settings

You can customize the MCP server by adding additional fields to the MCPServer resource. Here are some common configurations.

Customize the MCP server pod

You can customize the MCP server pod that gets created by the proxy using the podTemplateSpec field. This gives you full control over the pod specification, letting you set security contexts, resource limits, node selectors, and other pod-level configurations.

The podTemplateSpec field follows the standard Kubernetes PodTemplateSpec format, so you can use any valid pod specification options.

This example sets security contexts and resource limits. It lets the MCP container to run as root, an unfortunate requirement for the Fetch MCP server image, while still applying some security restrictions.

my-mcpserver-custom-pod.yaml
apiVersion: toolhive.stacklok.dev/v1alpha1
kind: MCPServer
metadata:
  name: fetch
  namespace: development # Can be any namespace
spec:
  image: docker.io/mcp/fetch
  transport: stdio
  port: 8080
  permissionProfile:
    type: builtin
    name: network
  podTemplateSpec:
    spec:
      containers:
        - name: mcp # This name must be "mcp"
          securityContext:
            allowPrivilegeEscalation: false
            runAsNonRoot: false # Allows the MCP container to run as root
            runAsUser: 0
            capabilities:
              drop:
                - ALL
          resources: # These resources apply to the MCP container
            limits:
              cpu: '500m'
              memory: '512Mi'
            requests:
              cpu: '100m'
              memory: '128Mi'
      securityContext:
        runAsNonRoot: true # The pod itself can run as a non-root user
        seccompProfile:
          type: RuntimeDefault
  resources: # These resources apply to the proxy container
    limits:
      cpu: '100m'
      memory: '128Mi'
    requests:
      cpu: '50m'
      memory: '64Mi'

Container name requirement

When customizing containers in podTemplateSpec, you must use name: mcp for the main container. This ensures the proxy can properly manage the MCP server process.

Run a server with secrets

For MCP servers that require authentication tokens or other secrets, add the secrets field to the MCPServer resource. This example shows how to use a Kubernetes secret to pass a GitHub personal access token to the github MCP server.

my-mcpserver-with-secrets.yaml
apiVersion: toolhive.stacklok.dev/v1alpha1
kind: MCPServer
metadata:
  name: github
  namespace: production # Can be any namespace
spec:
  image: ghcr.io/github/github-mcp-server
  transport: stdio
  port: 8080
  permissionProfile:
    type: builtin
    name: network
  secrets:
    - name: github-token
      key: token
      targetEnvName: GITHUB_PERSONAL_ACCESS_TOKEN

First, create the secret. Note that the secret must be created in the same namespace as the MCP server and the key must match the one specified in the MCPServer resource.

kubectl -n production create secret generic github-token --from-literal=token=<YOUR_TOKEN>

Apply the MCPServer resource:

kubectl apply -f my-mcpserver-with-secrets.yaml

Mount a volume

You can mount volumes into the MCP server pod to provide persistent storage or access to data. This is useful for MCP servers that need to read/write files or access large datasets.

To do this, add a standard volumes field to the podTemplateSpec in the MCPServer resource and a volumeMounts section in the container specification. Here's an example that mounts a persistent volume claim (PVC) to the /projects path in the Filesystem MCP server. The PVC must already exist in the same namespace as the MCPServer.

my-mcpserver-with-volume.yaml
apiVersion: toolhive.stacklok.dev/v1alpha1
kind: MCPServer
metadata:
  name: filesystem
  namespace: data-processing # Can be any namespace
spec:
  image: docker.io/mcp/filesystem
  transport: stdio
  port: 8080
  permissionProfile:
    type: builtin
    name: none
  podTemplateSpec:
    spec:
      volumes:
        - name: my-mcp-data
          persistentVolumeClaim:
            claimName: my-mcp-data-claim
      containers:
        - name: mcp
          # ... other container settings ...
          volumeMounts:
            - mountPath: /projects/my-mcp-data
              name: my-mcp-data
              readOnly: true

Check MCP server status

To check the status of your MCP servers in a specific namespace:

kubectl -n <NAMESPACE> get mcpservers

To check MCP servers across all namespaces:

kubectl get mcpservers --all-namespaces

The status, URL, and age of each MCP server is displayed.

For more details about a specific MCP server:

kubectl -n <NAMESPACE> describe mcpserver <NAME>

Configuration reference

MCPServer spec

Field	Description	Required	Default
`image`	Container image for the MCP server	Yes	-
`transport`	Transport method (stdio or sse)	No	stdio
`port`	Port to expose the MCP server on	No	8080
`targetPort`	Port to use for the MCP server (for SSE)	No
`args`	Additional arguments to pass to the MCP server	No	-
`env`	Environment variables to set in the container	No	-
`resources`	Resource requirements for the container	No	-
`secrets`	References to secrets to mount in the container	No	-
`permissionProfile`	Permission profile configuration	No	-
`podTemplateSpec`	Custom pod specification for the MCP server	No	-

Secrets

The secrets field has the following parameters:

name: The name of the Kubernetes secret (required)
key: The key in the secret (required)
targetEnvName: The environment variable to be used when setting up the secret in the MCP server (optional). If left unspecified, it defaults to the key.

Permission Profiles

Permission profiles can be configured in two ways:

Using a built-in profile:

permissionProfile:
  type: builtin
  name: network # or "none"

Using a ConfigMap:

permissionProfile:
  type: configmap
  name: my-permission-profile
  key: profile.json

The ConfigMap should contain a JSON permissions profile.

Next steps

See the Client compatibility reference to learn how to connect to MCP servers using different clients.

Deploy the operator using Helm - Install the ToolHive operator
Custom permissions - Configure permission profiles

Troubleshooting

MCPServer resource not creating pods

If your MCPServer resource is created but no pods appear, first ensure you created the MCPServer resource in an allowed namespace. If the operator runs in namespace mode and you didn't include the namespace in the allowedNamespaces list, the operator ignores the resource. Check the operator's configuration:

helm get values toolhive-operator -n toolhive-system

Check the operator.rbac.scope and operator.rbac.allowedNamespaces properties. If the operator runs in namespace mode, add the namespace where you created the MCPServer to the allowedNamespaces list. See Operator deployment modes.

If the operator runs in cluster mode (default) or the MCPServer is in an allowed namespace, check the operator logs and resource status:

# Check MCPServer status
kubectl -n <NAMESPACE> describe mcpserver <NAME>

# Check operator logs
kubectl -n toolhive-system logs -l app.kubernetes.io/name=toolhive-operator

# Verify the operator is running
kubectl -n toolhive-system get pods -l app.kubernetes.io/name=toolhive-operator

Other common causes include:

Operator not running: Ensure the ToolHive operator is deployed and running
Invalid image reference: Verify the container image exists and is accessible
RBAC issues: The operator automatically creates RBAC resources, but check for cluster-level permission issues
Resource quotas: Check if namespace resource quotas prevent pod creation

MCP server pod fails to start

If the MCP server pod is created but fails to start or is in CrashLoopBackOff:

# Check pod status
kubectl -n <NAMESPACE> get pods

# Describe the failing pod
kubectl -n <NAMESPACE> describe pod <POD_NAME>

# Check pod logs
kubectl -n <NAMESPACE> logs <POD_NAME> -c mcp

Common causes include:

Image pull errors: Verify the container image is accessible and the image name is correct
Missing secrets: Ensure required secrets exist and are properly referenced
Resource constraints: Check if the pod has sufficient CPU and memory resources
Permission issues: Verify the security context and permission profile are correctly configured
Invalid arguments: Check if the args field contains valid arguments for the MCP server

Proxy pod connection issues

If the proxy pod is running but clients cannot connect:

# Check proxy pod status
kubectl -n <NAMESPACE> get pods -l app.kubernetes.io/instance=<MCPSERVER_NAME>

# Check proxy logs
kubectl -n <NAMESPACE> logs -l app.kubernetes.io/instance=<MCPSERVER_NAME>

# Verify service is created
kubectl -n <NAMESPACE> get services

Common causes include:

Service not created: Ensure the proxy service exists and has the correct selectors
Port configuration: Verify the port field matches the MCP server's listening port
Transport mismatch: Ensure the transport field (stdio/sse) matches the MCP server's capabilities
Network policies: Check if network policies are blocking communication

Secret mounting issues

If secrets are not being properly mounted or environment variables are missing:

# Check if secret exists
kubectl -n <NAMESPACE> get secret <SECRET_NAME>

# Verify secret content
kubectl -n <NAMESPACE> describe secret <SECRET_NAME>

# Check environment variables in the pod
kubectl -n <NAMESPACE> exec <POD_NAME> -c mcp -- env | grep <ENV_VAR_NAME>

Common causes include:

Secret doesn't exist: Create the secret in the correct namespace
Wrong key name: Ensure the key field matches the actual key in the secret
Namespace mismatch: Secrets must be in the same namespace as the MCPServer
Permission issues: The operator automatically creates the necessary RBAC resources, but verify the ServiceAccount has access to read secrets

Volume mounting problems

If persistent volumes or other volumes are not mounting correctly:

# Check PVC status
kubectl -n <NAMESPACE> get pvc

# Describe the PVC
kubectl -n <NAMESPACE> describe pvc <PVC_NAME>

# Check volume mounts in the pod
kubectl -n <NAMESPACE> describe pod <POD_NAME>

Common causes include:

PVC not bound: Ensure the PersistentVolumeClaim is bound to a PersistentVolume
Namespace mismatch: The PVC must be in the same namespace as the MCPServer
Storage class issues: Verify the storage class exists and is available
Access mode conflicts: Check that the access mode is compatible with your setup
Mount path conflicts: Ensure mount paths don't conflict with existing directories

Permission profile errors

If the MCP server fails due to permission profile issues:

# Check if ConfigMap exists (for custom profiles)
kubectl -n <NAMESPACE> get configmap <CONFIGMAP_NAME>

# Verify ConfigMap content
kubectl -n <NAMESPACE> describe configmap <CONFIGMAP_NAME>

# Check operator logs for permission errors
kubectl -n toolhive-system logs -l app.kubernetes.io/name=toolhive-operator | grep -i permission

Common causes include:

Invalid profile name: Ensure built-in profile names are correct (none, network)
ConfigMap not found: Create the ConfigMap with the custom permission profile
Invalid JSON: Verify the permission profile JSON is valid
Missing key: Ensure the specified key exists in the ConfigMap

Resource limit issues

If pods are being killed due to resource constraints:

# Check resource usage
kubectl -n <NAMESPACE> top pods

# Check for resource limit events
kubectl -n <NAMESPACE> get events --sort-by='.lastTimestamp'

# Describe the pod for resource information
kubectl -n <NAMESPACE> describe pod <POD_NAME>

Solutions:

Increase resource limits: Adjust resources.limits in the MCPServer spec
Optimize resource requests: Set appropriate resources.requests values
Check node capacity: Ensure cluster nodes have sufficient resources
Review resource quotas: Check namespace resource quotas and limits

Debugging connectivity

To test connectivity between components:

# Port-forward to test direct access to the proxy
kubectl -n <NAMESPACE> port-forward service/<MCPSERVER_NAME> 8080:8080

# Test the connection locally
curl http://localhost:8080/health

# Check service endpoints
kubectl -n <NAMESPACE> get endpoints

Getting more information

For additional debugging information:

# Get all resources related to your MCP server
kubectl -n <NAMESPACE> get all -l app.kubernetes.io/instance=<MCPSERVER_NAME>

# Check operator events
kubectl -n <NAMESPACE> get events --field-selector involvedObject.kind=MCPServer

# Export MCPServer resource for inspection
kubectl -n <NAMESPACE> get mcpserver <NAME> -o yaml

Prerequisites​

Overview​

High-level architecture​

STDIO transport flow​

SSE transport flow​

Create an MCP server​

Automatic RBAC management​

Customize server settings​

Customize the MCP server pod​

Run a server with secrets​

Mount a volume​

Check MCP server status​

Configuration reference​

MCPServer spec​

Secrets​

Permission Profiles​

Next steps​

Related information​

Troubleshooting​

MCPServer resource not creating pods​

MCP server pod fails to start​

Proxy pod connection issues​

Secret mounting issues​

Volume mounting problems​

Permission profile errors​

Resource limit issues​

Debugging connectivity​

Getting more information​

Prerequisites

Overview

High-level architecture

STDIO transport flow

SSE transport flow

Create an MCP server

Automatic RBAC management

Customize server settings

Customize the MCP server pod

Run a server with secrets

Mount a volume

Check MCP server status

Configuration reference

MCPServer spec

Secrets

Permission Profiles

Next steps

Related information

Troubleshooting

MCPServer resource not creating pods

MCP server pod fails to start

Proxy pod connection issues

Secret mounting issues

Volume mounting problems

Permission profile errors

Resource limit issues

Debugging connectivity

Getting more information