Deploy Storage to MicroK8s

Goal: Deploy the containerized local filesystem storage service to MicroK8s using Kubernetes YAML (namespace, PVC, Deployment, Service) and validate it with port-forwarding.

With the service containerized and tested in Docker, deploy it to MicroK8s. You will set up MicroK8s, push the image, then create Kubernetes resources and deploy.

Important

Before proceeding, stop the Docker container from the previous phase so it does not hold ports 8011 or 50051 when MicroK8s port-forwarding starts.

docker stop localfilesystem-test

Service Ports

Port	Protocol	Description
8011	HTTP/REST	REST API endpoint
50051	gRPC	gRPC API endpoint

MicroK8s setup

Install and configure MicroK8s for the examples.

# install Micro K8s
sudo snap install microk8s --classic

# Make sure to add the user to the microk8s group (this avoids having to run as root)
sudo usermod -a -G microk8s $USER
# create the ~/.kube directory if it doesn't exist
mkdir -p ~/.kube
chmod 0700 ~/.kube
# re-endter the session to apply the changes
su - $USER

You can also install from the MicroK8s website.

Enable the required add-ons:

# make sure microk8s is running and the registry is enabled
microk8s.start
# enable the registry for local docker images
microk8s enable registry
# enable DNS for service discovery
microk8s enable dns
# enable ingress controller for external access
microk8s enable ingress
# enable storage for persistent volumes on the localhost filesystem
microk8s enable hostpath-storage

microk8s status --wait-ready

Validate the status of Micro K8s and check the nodes and pods.

# check the status of Micro K8s
microk8s kubectl get nodes
microk8s kubectl get pods --all-namespaces

Push the container to MicroK8s

Make the Docker image you built available to MicroK8s so it can be used by the deployment. Choose one of the two options below.

Option 1: Use the MicroK8s built-in registry (recommended)

MicroK8s includes a built-in container registry at localhost:32000 (enabled earlier with microk8s enable registry). Tag and push your image:

docker tag storageapi_localfilesystem_service:local localhost:32000/storageapi_localfilesystem_service:local
docker push localhost:32000/storageapi_localfilesystem_service:local

# validate the image was pushed
curl -s http://localhost:32000/v2/_catalog

Note

If you use Option 1, update the Deployment YAML image reference to localhost:32000/storageapi_localfilesystem_service:local and set imagePullPolicy: Always. See the Deployment YAML section below for both variants.

Option 2: Direct import (requires sudo)

Export the Docker image as a tarball and import it directly into MicroK8s’s containerd runtime. This method requires sudo because the containerd socket has root-only permissions.

docker save storageapi_localfilesystem_service:local -o storageapi_localfilesystem_service.tar
sudo microk8s ctr image import ./storageapi_localfilesystem_service.tar

# validate the container was imported
sudo microk8s ctr image ls | grep storageapi_localfilesystem_service

The container is now available to MicroK8s and can be used in the deployment.

Deploying

You will create a single YAML file and add one Kubernetes resource at a time. This is not a Kubernetes primer; see the Kubernetes documentation for deeper concepts.

Namespace

Use the storage-apis-dev namespace for this deployment. You can choose your own namespace name; just substitute it consistently in all commands throughout this guide.

# create the namespace
microk8s kubectl create namespace storage-apis-dev

# validate the namespace was created
microk8s kubectl get namespaces | grep storage-apis-dev

Kubernetes Deployment

Create a single YAML file and add resources one at a time so you can see how each part fits together.

Create the deployment file

# create the file
touch storage-service.yaml

First, create a new file called storage-service.yaml in your working directory. We’ll add resources to this file step by step.

# open the file in your favorite text editor, we use vs code
code storage-service.yaml

PersistentVolumeClaim

The filesystem service uses local storage, so create a PersistentVolumeClaim (PVC) to request storage from the cluster. The PVC is like requesting a “disk” for your application; the Deployment will use it for persistent storage.

Add the following to your storage-service.yaml file:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: storage-pvc  # Name of this PVC - referenced by the Deployment below
  namespace: storage-apis-dev  # Must match the namespace above
spec:
  accessModes:
    - ReadWriteOnce  # Single node access mode (suitable for MicroK8s hostpath)
                    # This means only one pod can mount this volume at a time
  resources:
    requests:
      storage: 10Gi  # Request 10GB of storage space
  storageClassName: microk8s-hostpath  # Use MicroK8s hostpath storage class

storage-service.yaml

Apply the file to create the PVC.

microk8s kubectl apply -f storage-service.yaml

Verify the PVC was created.

microk8s kubectl get pvc -n storage-apis-dev

Define the Deployment

A Deployment manages pods (containers) running your application. It ensures the specified number of replicas (pods) are running and handles updates. This Deployment creates a pod running the storage service container using the Docker image you built earlier.

Add the following to your storage-service.yaml file (after the PVC, separated by ---). The image and imagePullPolicy values depend on how you pushed the container in the previous step.

Set args: ["filesystem"] so the backend is explicit in container startup. This ensures backend-specific options (for example FILESERVICE_STATIC_DIR and FILESERVICE_SERVER_BASE_URI) are applied consistently, including with older images.

If you used Option 1 (MicroK8s registry):

# Make sure to add this after the PVC, separated by ---
---

# Define the Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: storage-service  # Name of this deployment
  namespace: storage-apis-dev  # Must match the namespace above
  labels:
    app: storage-service  # Label for selection and organization
spec:
  replicas: 1  # Run 1 pod (single replica since we're using persistent storage)
  selector:
    matchLabels:
      app: storage-service
  template:
    metadata:
      labels:
        app: storage-service
    spec:
      containers:
      - name: storage  # Container name within the pod
        image: localhost:32000/storageapi_localfilesystem_service:local
        imagePullPolicy: Always  # Pull from the MicroK8s registry
        args: ["filesystem"]  # Explicit backend selection for consistent config parsing
        ports:
        - containerPort: 8011  # HTTP/REST API port (matches Docker setup)
          name: http
        - containerPort: 50051  # gRPC API port (matches Docker setup)
          name: grpc
        env:
        # Environment variables configure the storage service
        # These match the -e flags used in the Docker containerize step
        - name: FILESERVICE_STATIC_DIR
          value: "/data/storage"  # This matches the mountPath below where the PVC is mounted
        - name: FILESERVICE_SERVER_BASE_URI
          value: "file-storage://fileservice"
        - name: FILESERVICE_TEST_FOLDER_MODE
          value: "native"
        - name: GRPC_SERVER_PORT
          value: "50051"
        - name: HTTP_SERVER_PORT
          value: "8011"
        - name: REDIRECT_HOST
          value: "http://localhost"
        - name: REDIRECT_PORT
          value: "8011"
        volumeMounts:
        - name: storage-data
          mountPath: /data/storage  # This matches FILESERVICE_STATIC_DIR above
      volumes:
      - name: storage-data
        persistentVolumeClaim:
          claimName: storage-pvc  # Reference to the PVC created in Step 1

If you used Option 2 (direct import):

# Make sure to add this after the PVC, separated by ---
---

# Define the Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: storage-service  # Name of this deployment
  namespace: storage-apis-dev  # Must match the namespace above
  labels:
    app: storage-service  # Label for selection and organization
spec:
  replicas: 1  # Run 1 pod (single replica since we're using persistent storage)
  selector:
    matchLabels:
      app: storage-service
  template:
    metadata:
      labels:
        app: storage-service
    spec:
      containers:
      - name: storage  # Container name within the pod
        image: storageapi_localfilesystem_service:local
        imagePullPolicy: Never  # Use the locally imported image, don't try to pull from a registry
        args: ["filesystem"]  # Explicit backend selection for consistent config parsing
        ports:
        - containerPort: 8011  # HTTP/REST API port (matches Docker setup)
          name: http
        - containerPort: 50051  # gRPC API port (matches Docker setup)
          name: grpc
        env:
        # Environment variables configure the storage service
        # These match the -e flags used in the Docker containerize step
        - name: FILESERVICE_STATIC_DIR
          value: "/data/storage"  # This matches the mountPath below where the PVC is mounted
        - name: FILESERVICE_SERVER_BASE_URI
          value: "file-storage://fileservice"
        - name: FILESERVICE_TEST_FOLDER_MODE
          value: "native"
        - name: GRPC_SERVER_PORT
          value: "50051"
        - name: HTTP_SERVER_PORT
          value: "8011"
        - name: REDIRECT_HOST
          value: "http://localhost"
        - name: REDIRECT_PORT
          value: "8011"
        volumeMounts:
        - name: storage-data
          mountPath: /data/storage  # This matches FILESERVICE_STATIC_DIR above
      volumes:
      - name: storage-data
        persistentVolumeClaim:
          claimName: storage-pvc  # Reference to the PVC created in Step 1

storage-service.yaml

Apply the updated file:

microk8s kubectl apply -f storage-service.yaml

Wait for the pod to start running. This may take 30-60 seconds:

microk8s kubectl get pods -n storage-apis-dev -w

Press Ctrl+C once the pod shows status Running and 1/1 ready.

Verify the pod is running:

microk8s kubectl get pods -n storage-apis-dev | grep storage-service

Define the Service

At this point, we have a pod running the storage service container with persistent storage. However, we need a way to access the service consistently from other pods in the cluster or via port-forwarding from your host machine.

A Service provides a stable network endpoint (DNS name and IP) for accessing pods. Even if pods restart and get new IPs, the Service provides a consistent way to connect to your application. This Service exposes the storage service pods so they can be accessed from other pods in the cluster or via port-forwarding from your host machine.

Add the following to your storage-service.yaml file (after the Deployment, separated by ---):

# Make sure to add this after the Deployment, separated by ---
---

# Define the Service
apiVersion: v1
kind: Service
metadata:
  name: storage-service  # Name of this service
  namespace: storage-apis-dev  # Must match the namespace above
  labels:
    app: storage-service  # Label for organization and selection
spec:
  selector:
    app: storage-service  # Routes traffic to pods with this label
                        # This must match the pod labels in the Deployment above
  ports:
  - name: http  # HTTP/REST API port name
    port: 8011  # Port that other services/clients will connect to
    targetPort: 8011  # Port on the pods that traffic will be forwarded to
    protocol: TCP
  - name: grpc
    port: 50051  # Port that other services/clients will connect to
    targetPort: 50051  # Port on the pods that traffic will be forwarded to
    protocol: TCP
  type: ClusterIP  # Only accessible within the cluster (internal)

Apply the updated file.

microk8s kubectl apply -f storage-service.yaml

Verify the service endpoints.

microk8s kubectl get endpoints -n storage-apis-dev storage-service

You should see the pod IP and ports listed. This confirms the Service can find and route traffic to your pods.

Set up port forwarding

The Service is ClusterIP (internal only). Use port-forwarding from your host to test; it forwards the service’s port 8011 to localhost:8011 so you can use the same curl commands as in the Docker step.

microk8s kubectl port-forward -n storage-apis-dev service/storage-service 8011:8011

Keep this running while you test the service. The port-forward will forward traffic from localhost:8011 to the service inside the cluster.

Note

Troubleshooting port-forward issues

• TLS certificate mismatch (API server): If the error references x509 and the API server, regenerate the API server certificate and restart MicroK8s:

  sudo microk8s refresh-certs --cert server.crt
  sudo microk8s stop && sudo microk8s start
  

• TLS certificate mismatch (kubelet): If the error mentions port 10250 or lists IPs that don’t match your current network interfaces, the kubelet certificate needs regeneration. This commonly happens when a VPN interface (e.g., GlobalProtect) is added after MicroK8s was installed. kubectl port-forward, kubectl logs, and kubectl exec all route through the kubelet and will all fail.

  Check the kubelet certificate’s SANs:

  openssl s_client -connect 127.0.0.1:10250 2>/dev/null | openssl x509 -noout -text | grep -A2 "Subject Alternative"
  

  If your current IP is missing, regenerate the kubelet certificate:

  sudo microk8s refresh-certs --cert kubelet.crt
  sudo microk8s stop && sudo microk8s start
  

• Port already in use: Find and stop the process using the port, or forward to a different local port:

  ss -tlnp | grep {port}
  kill -9 {pid}
  

Validate the storage service is working

# test writing to the storage service
export PAYLOAD="Hello from Storage API!"
export PAYLOAD_SIZE=$(printf "%s" "${PAYLOAD}" | wc -c)
# the non-percent encoded URL is: file-storage://fileservice/hello.txt
curl -X PUT "http://localhost:8011/v1beta/fileobject/by-address/file-storage%3A%2F%2Ffileservice%2Fhello.txt?data_object_size=${PAYLOAD_SIZE}" -H "Content-Type: application/octet-stream" --data "${PAYLOAD}"

# test reading from the storage service
curl http://localhost:8011/v1beta/fileobject/by-address/file-storage%3A%2F%2Ffileservice%2Fhello.txt

Note

If you changed FILESERVICE_SERVER_BASE_URI in the Deployment, use the percent-encoded version of that URI in the curl file-address path above instead of file-storage://fileservice.

Output should be similar to this:

>> "Hello from Storage API!"

You can inspect the storage mount path; it matches the Docker testing layout. Replace {pvc-uuid} with the PVC UUID from your deployment (from the previous step).

# this is the default mount path for microk8s hostpath storage
cd /var/snap/microk8s/common/default-storage/

# storage path (replace {pvc-uuid} with your PVC UUID)
cd storage-apis-dev/storage-pvc-{pvc-uuid}/

# you should see the file you wrote in the Docker testing step
ls -lR

Cleanup

To clean up, delete the namespace.

microk8s kubectl delete namespace storage-apis-dev

Next Steps

Next: Add a Discovery service so you can extend the deployment with additional services.