Accessing faster builds with /dev/fuse

You can configure your pods with the /dev/fuse device to enable faster and more efficient container image builds, particularly for unprivileged users. This device allows unprivileged pods to mount overlay filesystems, which can be leveraged by tools like Podman.

Configuring /dev/fuse for unprivileged builds in pods

By exposing the /dev/fuse device to an unprivileged pod, you grant it the capability to perform Filesystem in Userspace (FUSE) mounts. This is achieved by adding the io.kubernetes.cri-o.Devices: "/dev/fuse" annotation to your pod definition. This setup allows an unprivileged user within the pod to use tools like podman with storage drivers such as fuse-overlayfs by mimicking privileged build capabilities in a secure and efficient manner without granting full privileged access to the pod.

Procedure

1. Define the pod with /dev/fuse access:

   1. Create a YAML file named fuse-builder-pod.yaml with the following content:

      apiVersion: v1
      kind: Pod
      metadata:
        name: fuse-builder-pod
        annotations:
          io.kubernetes.cri-o.Devices: "/dev/fuse"
      spec:
        containers:
        - name: build-container
          image: quay.io/podman/stable
          command: ["/bin/sh", "-c"]
          args: ["echo 'Container is running. Use oc exec to get a shell.'; sleep infinity"]
          securityContext:
            runAsUser: 1000

      where:

      io.kubernetes.cri-o.Devices

      The io.kubernetes.cri-o.Devices: "/dev/fuse" annotation makes the FUSE device available.

      image

      This annotation specifies a container that uses an image that includes podman (for example, quay.io/podman/stable).

      args

      This command keeps the container running so you can exec into it.

      securityContext

      This annotation specifies a securityContext that runs the container as an unprivileged user (for example, runAsUser: 1000).

      Note

      Depending on your cluster’s Security Context Constraints (SCCs) or other policies, you might need to further adjust the securityContext specification, for example, by allowing specific capabilities if /dev/fuse alone is not sufficient for fuse-overlayfs to operate.

   2. Create the pod by running the following command:

      $ oc apply -f fuse-builder-pod.yaml

2. Verify that the pod is running:

   $ oc get pods fuse-builder-pod

3. Access the pod and prepare the build environment:

   After the fuse-builder-pod pod is in the Running state, open a shell session into the build-container environment:

   $ oc exec -ti fuse-builder-pod -- /bin/bash

   You are now inside the container. Because the default working directory might not be writable by the unprivileged user, change to a writable directory like /tmp:

   $ cd /tmp

   $ pwd

   /tmp

4. Create a dockerfile and build an image using Podman:

   Inside the pod’s shell and within the /tmp directory, you can now create a Dockerfile and use podman to build a container image. If fuse-overlayfs is the default or configured storage driver, Podman is able to leverage fuse-overlayfs because of the available /dev/fuse device.

   1. Create a sample Dockerfile:

      $ cat > Dockerfile <<EOF
      FROM registry.access.redhat.com/ubi9/ubi-minimal
      RUN microdnf install -y findutils && microdnf clean all
      RUN echo "This image was built inside a pod with /dev/fuse by user $(id -u)" > /app/build_info.txt
      COPY Dockerfile /app/Dockerfile_copied
      WORKDIR /app
      CMD ["sh", "-c", "cat /app/build_info.txt && echo '--- Copied Dockerfile ---' && cat /app/Dockerfile_copied"]
      EOF

   2. Build the image using podman. The -t flag tags the image:

      $ podman build -t my-fuse-built-image:latest .

      You should see Podman executing the build steps.

5. Optional: Test the built image:

   Still inside the fuse-builder-pod, you can run a container from the image you just built to test it:

   $ podman run --rm my-fuse-built-image:latest

   This should output the content of the /app/build_info.txt file and the copied Dockerfile.

6. Exit the pod and clean up:

   1. After you are done, exit the shell session in the pod:

      $ exit

   2. Delete the pod if it’s no longer needed:

      $ oc delete pod fuse-builder-pod

   3. Remove the local YAML file:

      $ rm fuse-builder-pod.yaml