Use AI Guard outside your cluster

Views:

Expose AI Guard and LiteLLM to clients running outside your Kubernetes cluster by deploying an ingress controller and creating ingress resources for each service.

By default, you can only reach AI Guard inside the cluster. To call AI Guard from an external application, like a LiteLLM proxy running outside the cluster, you must expose it through an ingress controller. The steps below use the NGINX Ingress Controller as an example, but any ingress controller that can proxy TCP traffic to port 8080 works.

After completing the below procedure, use the resulting external hostname as the endpoint URL in your integration. For LiteLLM configuration, see LiteLLM integration.

Prerequisites:

AI Guard deployed in the trend-ai-security namespace

See Deploy on a Kubernetes cluster.
Helm 3.0+ installed
kubectl configured with access to the target cluster

Procedure

Install the NGINX Ingress Controller.

helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace

Retrieve the external hostname assigned to the ingress controller.
```
kubectl get svc ingress-nginx-controller -n ingress-nginx \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
```
Wait until the command returns a hostname before continuing. The value becomes the base URL for all Ingress resources you create in the following steps.

Create and apply an Ingress resource for AI Guard.

Save the following as ai-guard-ingress.yaml then apply it.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ai-guard-ingress
  namespace: trend-ai-security
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
    - http:
        paths:
          - path: /applyGuardrails
            pathType: Prefix
            backend:
              service:
                name: ai-guard
                port:
                  number: 8080

kubectl apply -f ai-guard-ingress.yaml

If you deploy LiteLLM in the same cluster and it requires external access, create and apply an ingress resource for LiteLLM.

Save the following as litellm-ingress.yaml then apply it. Replace <litellm-namespace> with the namespace where LiteLLM is deployed.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: litellm-ingress
  namespace: <litellm-namespace>
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: litellm
                port:
                  number: 4000

kubectl apply -f litellm-ingress.yaml

Verify that AI Guard is reachable through the ingress.
Replace <external-hostname> with the hostname retrieved above.
```
curl http://<external-hostname>/applyGuardrails
```
A response from AI Guard confirms that the ingress is routing traffic correctly. Use http://<external-hostname> as the endpoint URL in any integration that requires an external ingress URL.