Hands-On Lab: Zero-Trust AKS Workload Identity with Key Vault CSI Driver

Lab Overview & Production Architecture

In modern microservices architectures hosting LLMs and agentic workflows, inference containers need secure access to OpenAI endpoints, database connection strings, and encryption keys. Mounting static JSON keys into configmaps creates severe audit failures.

This lab demonstrates the zero-trust workflow:

1. An AKS cluster with --enable-oidc-issuer and --enable-workload-identity.
2. A user-assigned managed identity with Key Vault Secrets User permissions on an Azure Key Vault.
3. An Entra ID Federated Identity Credential that binds a specific Kubernetes ServiceAccount to the managed identity.
4. A SecretProviderClass resource mounting secrets into pod volumes without storing secrets in Git.

Step 1: Provision Cluster with OIDC Issuer & Workload Identity

Workload identity requires the OIDC issuer URL from your AKS cluster:

# Create resource group and Key Vault
az group create --name rg-ai-security --location eastus
az keyvault create --name kv-ai-infer-99 --resource-group rg-ai-security --enable-rbac-authorization true

# Store sample OpenAI API key secret
az keyvault secret set --vault-name kv-ai-infer-99 --name "OpenAIKey" --value "sk-mock-live-enterprise-token"

# Provision AKS cluster with OIDC & Workload Identity enabled
az aks create \
  --resource-group rg-ai-security \
  --name aks-ai-zero-trust \
  --node-count 2 \
  --enable-oidc-issuer \
  --enable-workload-identity \
  --enable-addons azure-keyvault-secrets-provider \
  --generate-ssh-keys

Step 2: Create Managed Identity & Federated Credential

Next, establish trust between the Kubernetes service account and Azure Active Directory (Entra ID):

# Retrieve cluster OIDC Issuer URL
OIDC_URL=$(az aks show --resource-group rg-ai-security --name aks-ai-zero-trust --query "oidcIssuerProfile.issuerUrl" -o tsv)

# Create User-Assigned Managed Identity
az identity create --name id-ai-inference --resource-group rg-ai-security
UAMI_CLIENT_ID=$(az identity show --name id-ai-inference --resource-group rg-ai-security --query "clientId" -o tsv)

# Assign Key Vault Secrets User Role
KV_ID=$(az keyvault show --name kv-ai-infer-99 --resource-group rg-ai-security --query "id" -o tsv)
az role assignment create \
  --role "Key Vault Secrets User" \
  --assignee $UAMI_CLIENT_ID \
  --scope $KV_ID

# Create Federated Identity Credential
az identity federated-credential create \
  --name fed-cred-inference \
  --identity-name id-ai-inference \
  --resource-group rg-ai-security \
  --issuer "$OIDC_URL" \
  --subject "system:serviceaccount:ai-system:sa-ai-inference" \
  --audience "api://AzureADTokenExchange"

Step 3: Define Kubernetes ServiceAccount & SecretProviderClass

Configure the SecretProviderClass custom resource pointing to your Azure Key Vault:

apiVersion: v1
kind: Namespace
metadata:
  name: ai-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-ai-inference
  namespace: ai-system
  annotations:
    azure.workload.identity/client-id: "<UAMI_CLIENT_ID>"
---
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kv-provider
  namespace: ai-system
  spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    clientID: "<UAMI_CLIENT_ID>"
    keyvaultName: "kv-ai-infer-99"
    cloudName: ""
    objects: |
      array:
        - |
          objectName: OpenAIKey
          objectType: secret
          objectVersion: ""
    tenantId: "<AZURE_TENANT_ID>"

Step 4: Deploy Inference Pod & Verify Injection

When the pod mounts the CSI volume, the Workload Identity webhook injects the short-lived JWT token, authorizing the CSI driver to fetch the secret directly into RAM:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-inference-app
  namespace: ai-system
spec:
  replicas: 2
  selector:
    matchLabels:
      app: inference
  template:
    metadata:
      labels:
        app: inference
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: sa-ai-inference
      containers:
        - name: inference-worker
          image: mcr.microsoft.com/azure-cli:latest
          command: ["/bin/sh", "-c", "cat /mnt/secrets-store/OpenAIKey && sleep 3600"]
          volumeMounts:
            - name: secrets-store-inline
              mountPath: "/mnt/secrets-store"
              readOnly: true
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: "azure-kv-provider"

Lab Observations & Performance Verification

During the verification run, we logged several critical metrics:

Review Checklist for AI-200 Exam Day

• Cluster must have both --enable-oidc-issuer and --enable-workload-identity.
• ServiceAccount requires annotation azure.workload.identity/client-id: <id>.
• Pod template spec requires label azure.workload.identity/use: "true".
• Managed Identity requires Key Vault Secrets User role assignment, NOT Contributor.
• The federated credential subject format must follow: system:serviceaccount:<namespace>:<serviceaccount-name>.