This document will detail the procedure to deploy a private docker registry as a pod on Kubernetes. Having your own private repository for docker images gives absolute control over storage options, tightening access control and implementing authentication practices, all customized per your needs.
A Kubernetes docker registry is a docker registry running as a Kubernetes pod. Like any storage service, there will be a volume attached to the pod that stores your private docker images, and you can set up access controls for the registry via the access controls of the Kubernetes pod.
Good examples of a docker registry are the public Dockerhub, there is also Amazon Elastic Container Registry, Azure Container Registry, and Artifact Registry from Google. There are also tools like Sonatype’s Nexus, JFrog’s Artifactory, and VMware Harbor that are popularly used tools for systems that prefer staying cloud agnostic.
Namespace
We will deploy everything in a namespace to make management easier and cleaner.
kubectl create namespace docker-registrySecurity
We will start with authentication to have basic security.
htpasswd
We will generate the user / password for authentication. For this, we will use htpasswd to generate a file that will be used to access the registry. For this demonstration, we will use myuser as the username and mypasswd as the password.
$ mkdir auth
$ docker run --rm --entrypoint htpasswd registry:2.7.0 -Bbn myuser mypasswd > auth/htpasswd
Unable to find image 'registry:2.7.0' locally
2.7.0: Pulling from library/registry
cd784148e348: Pull complete
0ecb9b11388e: Pull complete
918b3ddb9613: Pull complete
5aa847785533: Pull complete
adee6f546269: Pull complete
Digest: sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57
Status: Downloaded newer image for registry:2.7.0Generic secret
Here is the generic secret created in the cluster
$ kubectl create secret generic auth-secret --from-file=auth/htpasswd --namespace docker-registry
secret/auth-secret createdPersistent Storage
Now that we have basic security done, we need to create storage for our registry. For this, we will use Persistent Volume and Claim.
At Leaseweb, we will use the default storage class cloudstack-custom. We will not need to specify it during deployment, as this is the default and only storage class on the new cluster. This might differ if you are using S3 or another storage class and would need to be specified.
A Persistent Volume (PV) is a piece of volume and a cluster resource that you provision just like the nodes of the cluster.
A Persistent Volume Claim (PVC) is a request for storage by a user in the cluster. Just like a Pod consumes node resources, PVCs request and consume PV resources.
– From Kubernetes documentation
Click here to read more about PV and PVC.
For this, we will need to first create a manifest and then deploy it on the cluster.
Manifest
In a file called “registry-pvc.yml”, we will define a PersistentVolumeClaim. You can change the storage size to your needs. We do not need to specify the StorageClassName as the CloudStack CSI is defined as the default on all deployed clusters at Leaseweb.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: docker-registry-pvc
namespace: docker-registry
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 10GiDeployment
We then apply this to our cluster
$ kubectl apply -f registry-pvc.yml --namespace docker-registry
persistentvolumeclaim/docker-registry-pvc createdNow, we have a PVC to use the volume resources from. Let us start using this volume.
Back to the top ↑Registry
We will use Helm to deploy the registry. Helm is the package manager for Kubernetes, focused on automating the installation of all kinds of Kubernetes applications. We will add the repo, then create a values.yml to configure it and then deploy the registry.
Adding
First, let’s add the repo twuni/docker-registry, which is the successor of the original stable docker registry helm chart, and then update the local cache to make sure we have the latest charts.
pod.yml
$ helm repo add twuni https://twuni.github.io/docker-registry.helm
"twuni" has been added to your repositories
$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "twuni" chart repository
Update Complete. ⎈Happy Helming!⎈We can take a look and see the version we will install.
$ helm search repo docker-registry
NAME CHART VERSION APP VERSION DESCRIPTION
twuni/docker-registry 2.2.2 2.8.1 A Helm chart for Docker RegistryConfiguring
We will now create the values.yml that will be used to deploy the registry. Again, here we do not need to specify the storageClass as it is defaulted to the cloudstack csi
values.yml
---
replicaCount: 1
persistence:
enabled: true
size: 10Gi # <- adapt this value with the size you want to use
deleteEnabled: true
existingClaim: docker-registry-pvc # <- here we specify the pvc we created earlier
secrets:
htpasswd: myuser:$2y$05$mDso3c3.tzcPUAE5NQsqmuYX0JoP8ihttoNBa59VaFPn8Dz6RQbkS # <- This is the username / password we created in the security section. You can replace this with your own username and password
ingress:
enabled: true
className: traefik
path: /
hosts:
- registry.example.org
tls:
- secretName: docker-registry-tls
hosts:
- registry.example.orgDeployment
We then deploy to our cluster:
$ helm install -f values.yml docker-registry --namespace docker-registry twuni/docker-registry
NAME: docker-registry
LAST DEPLOYED: Fri Nov 10 14:09:53 2023
NAMESPACE: docker-registry
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
https://registry.example.org/Usage
Now that everything has been deployed, we will go into how to use this.
For this, we will use a simple nginx image to pull and push to our new private Docker registry.
Login
We will log in to our new registry:
$ docker login https://registry.example.org/
Username: myuser
Password:
Login SucceededWe pull an image, for example, nginx:latest
$ docker pull nginx
Using default tag: latest
latest: Pulling from library/nginx
578acb154839: Pull complete
e398db710407: Pull complete
85c41ebe6d66: Pull complete
7170a263b582: Pull complete
8f28d06e2e2e: Pull complete
6f837de2f887: Pull complete
c1dfc7e1671e: Pull complete
Digest: sha256:86e53c4c16a6a276b204b0fd3a8143d86547c967dc8258b3d47c3a21bb68d3c6
Status: Downloaded newer image for nginx:latest
docker.io/library/nginx:latest
What's Next?
View a summary of image vulnerabilities and recommendations → docker scout quickview nginxWe will now tag this with the private docker registry domain name and a custom tag
$ docker tag nginx registry.example.org/my-nginxAnd finally, we push on the private docker registry our newly tag image.
$ docker push registry.example.org/my-nginx
Using default tag: latest
The push refers to repository [registry.example.org/my-nginx]
505f49f13fbe: Pushed
9920f1ebf52b: Pushed
768e28a222fd: Pushed
715b32fa0f12: Pushed
e503754c9a26: Pushed
609f2a18d224: Pushed
ec983b166360: Pushed
latest: digest: sha256:d2e65182b5fd330470eca9b8e23e8a1a0d87cc9b820eb1fb3f034bf8248d37ee size: 1778Now we have an image in our private docker registry hosted on our Kubernetes cluster. Let’s use this in a deployment.
Back to the top ↑Using Docker Registry To Pull Images In Your Kubernetes Cluster
As you have a Docker registry deployed in your Kubernetes cluster, you can start using it by pulling previously pushed images for your Kubernetes Pods.
To learn how a private Docker registry can be used for pulling images, you will create a simple Kubernetes pod in a newly created test namespace. This Kubernetes Pod will use the previously pushed image registry.example.org/my-nginx.
Namespace
First, you have to create a test Kubernetes namespace
$ kubectl create namespace test
namespace/test createdSecrets
Kubernetes Secret
A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image. Using a Secret means that you don’t need to include confidential data in your application code. Because Secrets can be created independently of the Pods that use them, there is less risk of the Secret (and its data) being exposed during the workflow of creating, viewing, and editing Pods. Kubernetes, and applications that run in your cluster, can also take additional precautions with Secrets, such as avoiding writing sensitive data to nonvolatile storage. So we will now create a TLS type of secret and a Generic type of secret to mount our certificate and password, respectively.
– Excerpt from the official Kubernetes documentation
We will reuse the username and password that were created earlier in this demonstration to create a Kubernetes secret that will be used by the deployment to access the registry.
Warning
Be mindful of the namespace where you deploy the secret, as this might break your deployment.
We create the secret in the test namespace that we created a minute ago.
$ kubectl create secret docker-registry regcred --docker-server=registry.example.org --docker-username=myuser --docker-password=mypasswd -n test
secret/regcred createdDeployment
We will now deploy a pod to test all of this.
Manifest
We will create a pod manifest name test-nginx.yml
test-nginx.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: test
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: registry.example.org/my-nginx
ports:
- containerPort: 80
imagePullSecrets:
- name: regcred # <- This point to the secret we created earlier so we can authenticate against the registry.
Deploy
We then apply our manifest
$ kubectl apply -f test-nginx.yml
deployment.apps/nginx-deployment createdWe look at the deployment
$ kubectl get pods -n test
NAME READY STATUS RESTARTS AGE
nginx-deployment-5fc878d7b5-gjvsh 1/1 Running 0 2m8s
$ kubectl describe pods nginx-deployment-5fc878d7b5-gjvsh -n test
Name: nginx-deployment-5fc878d7b5-gjvsh
Namespace: test
Priority: 0
Service Account: default
[...]
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 75s default-scheduler Successfully assigned test/nginx-deployment-5fc878d7b5-gjvsh to k8s-0000001-worker-0-2z6dj
Normal Pulling 75s kubelet Pulling image "registry.example.org/my-nginx"
Normal Pulled 71s kubelet Successfully pulled image "registry.example.org/my-nginx" in 3.885317258s (3.885349506s including waiting)
Normal Created 71s kubelet Created container nginx
Normal Started 71s kubelet Started container nginxHopefully, this article gave you a good overview of how to set up a private Docker registry in your Kubernetes cluster. Keep in mind that having a Private Docker registry is essential if deploying Docker services that are not Open Source or need more security.
Back to the top ↑
