Provide users access to Kubernetes cluster
Kubernetes allows external credentials for authentication, and these are:
- Static Token Files.
- Certificates.
- 3rd Party Identity Services.
All these are managed externally to the cluster by us because Kubernetes does not provide by itself any support for users. In this case we will configure the access to the Kubernetes cluster for an individual user using the certificates method.
The process is simple,
First, the cluster administrators or the user generate the client key for the user (user.key). This key will be used to authenticate to the cluster therefore in order to be recognized as legit by the cluster, this key has to be signed by the cluster certificate authority.
Second, it is necessary to generate, from the key, a certificate signing request (user.csr) which is a file that contain the key.
Third, The key is signed by the cluster certification authority.
Fourth, the signed key (user.crt) handed out to the user.
The following step is get the certificated signed which produce a .crt file (user.crt) and finally the signed corticate has to be provided to the user so she can install on her local machine and start communicating with the cluster as shown above.
First, we verify if the cluster is configured to accept client certificates, so let’s run the following commands.
Run a command to find the pod name that runs the cluster api and check it.
root@km1:~# k get pods -n kube-system | grep kube-apiserver
kube-apiserver-km1 1/1 Running 32 (18m ago) 91d
now that we have the pod name we can verify if it accepts client certificates:
root@km1:~# k describe pod kube-apiserver-km1 -n kube-system
Name: kube-apiserver-km1
Namespace: kube-system
. . . <edited>
Command:
kube-apiserver
--advertise-address=192.168.100.250
--allow-privileged=true
--authorization-mode=Node,RBAC
--client-ca-file=/etc/kubernetes/pki/ca.crt
--enable-admission-plugins=NodeRestriction
--enable-bootstrap-token-auth=true
. . .
We see in the highlighted line that we can use client certificate.
We have to generate the user certificate and sign it with the K8s cluster CA certificate. Notice that we are now work from the cluster master node virtual machine so to making this process easier to follow.
Since we are in the cluster master node, we have access to the cluster certificates that we need to use for signing the user private certificate.
root@km1:~# ll /etc/kubernetes/pki
total 68
. . . (edited)
-rw-r--r-- 1 root root 1164 Jan 4 17:30 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 J an 4 17:30 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1099 Jan 4 17:30 ca.crt
-rw------- 1 root root 1675 Jan 4 17:30 ca.key
drwxr-xr-x 2 root root 4096 Jan 4 17:30 etcd/
-rw-r--r-- 1 root root 1115 Jan 4 17:30 front-proxy-ca.crt
. . .
The ca.crt and ca.key are the cluster certificate we have to use.
Create .key file
So now we can start creating the .key file in the user local computer:tmp-ssl ✌️ $ openssl genrsa -out willy.key 2048 tmp-ssl ✌️ $ openssl genrsa -out willy.key 2048
Generating RSA private key, 2048 bit long modulus
.................+++
...........................................+++
e is 65537 (0x10001)
tmp-ssl ✌️ $ ls -l
total 8
-rw-r--r-- 1 willy staff 1.6K Apr 6 09:58 willy.key
tmp-ssl ✌️ $
Create .csr file
For creating the .csr file we need to keep in mind that if we want to create this certificate for a particular user we will need a user name (in certs lingo it is called “common name”) and also is convenient have a user group (in this context certs lingo “organization”), so we will use “willy” for the user name and “developers” for the group name.
We run the command as follow:
tmp-ssl ✌️ $ openssl req -new -key willy.key -out willy.csr -subj "/CN=willy/O=developers"
tmp-ssl ✌️ $ ls -l
total 16
-rw-r--r-- 1 willy staff 915 Apr 6 10:14 willy.csr
-rw-r--r-- 1 willy staff 1679 Apr 6 09:58 willy.key
tmp-ssl ✌️ $
So we have the .csr file, and now we have two options for getting the user cert signed:
1. Create the user certificate manually by signing our .csr with ca.crt and cs.key from the cluster using openssl command.
2. Use the K8s to do it for us by creating a K8s csr resource with the .csr file and after that we can approve it with a kubectl command.
Generate .crt certificate manually
We copy our .csr file to the master node in the cluster server that has the name “km1”:
tmp-ssl ✌️ $ scp ./willy.csr root@km1:~/tmp
willy.csr 100% 915
495.9KB/s 00:00
mp-ssl ✌️ $
So in the cluster master node we have:
root@km1:~/tmp# ls -l
total 4
-rw-r--r-- 1 root root 915 Apr 6 16:57 willy.csr
root@km1:~/tmp#
now we can run the command to sign the cert:
root@km1:~/tmp# openssl x509 -req -in willy.csr \
> -CA /etc/kubernetes/pki/ca.crt \
> -CAkey /etc/kubernetes/pki/ca.key \
> -CAcreateserial \
> -out willy.crt \
> -days 365
Signature ok
subject=CN = willy, O = developers
Getting CA Private Key
root@km1:~/tmp# ls -l
total 8
-rw-r--r-- 1 root root 1 017 Apr 7 00:51 willy.crt
-rw-r--r-- 1 root root 915 Apr 6 16:57 willy.csr
root@km1:~/tmp#
Now we have the certificate we need for the user to access the cluster. But we have this file in the master node of the cluster (km1) so we could have access to the cluster “ca.” files. We need to copy the file willy.crt into the user local machine where the user will use it.
For simplicity I just created a new empty local file named willy.crt and pasted in it the content of the file in the cluster master node. Once done we have in the local server all what we need:
tmp-ssl ✌️ $ ls -l
total 24
-rw-r--r-- 1 willy staff 1017 Apr 6 21:03 willy.crt
-rw-r--r-- 1 willy staff 915 Apr 6 10:14 willy.csr
-rw-r--r-- 1 willy staff 1679 Apr 6 09:58 willy.key
tmp-ssl ✌️ $
Now we create the kubeconfig file for the user in his local machine:
tmp-ssl ✌️ $ k config set-credentials willy \
> --client-certificate willy.crt \
> --client-key willy.key
User "willy" set.
tmp-ssl ✌️ $
if we check the kubeconfig file we will find that the user willy was injected there:
. . .
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJT
server: https://192.168.100.250:6443
name: kc-austin
- cluster:
. . .
- name: kubernetes-admin2
user:
client-certificate-data: LS0tLS1CRU
client-key-data: LS0tLS1CR
- name: willy
user:
client-certificate: /Users/willy/tmp-ssl/willy.crt
client-key: /Users/willy/tmp-ssl/willy.key
we also need to add a context for this user in the kubeconfig file and for that we need to have the cluster name which in this case is “kc-austin”.
tmp-ssl ✌️ $ k config set-context willy-kc-austin --user willy --cluster kc-austin
Context "willy-kc-austin" created.
tmp-ssl ✌️ $
if we check the kubeconfig file we see:
. . . <edited>
contexts:
- context:
cluster: kc-austin
user: kc-austin-admin
name: kc-austin-admin@kc-austin
- context:
cluster: kubernetes2
user: kubernetes-admin2
name: kubernetes-admin2@kubernetes2
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context:
cluster: kc-austin
user: willy
name: willy@kc-austin
current-context: kc-austin-admin@kc-austin
. . .
so now we can see the contexts we have:
tmp-ssl ✌️ $ k config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kc-austin-admin@kc-austin kc-austin kc-austin-admin
kubernetes-admin2@kubernetes2 kubernetes2 kubernetes-admin2
kubernetes-admin@kubernetes kubernetes kubernetes-admin
willy@kc-austin ck-austin willy
tmp-ssl ✌️ $
the list shows several clusters that are available in the network, but we care just the one highlighted, so we now change the context to test:
tmp-ssl ✌️ $ k config use-context willy@kc-austin
Switched to context "willy@kc-austin".
tmp-ssl ✌️ $ k config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
kc-austin-admin@kc-austin kc-austin kc-austin-admin
kubernetes-admin2@kubernetes2 kubernetes2 kubernetes-admin2
kubernetes-admin@kubernetes kubernetes kubernetes-admin
* willy@kc-austin ck-austin willy
but if we try to use this context we will get rejected because even when we configure a authentication for the user willy, we didn’t define any authorization for him.
For solve this we need to create a role and a role binding for the user but for that we have to switch context back to the administrator as the administrator has authority to do so.
tmp-ssl ✌️ $ k config use-context kc-austin-admin@kc-austin
Switched to context "kc-austin-admin@kc-austin".
So now we can create a role for listing pods for example and its corresponding role binding to the user willy:
tmp-ssl ✌️ $ k create role allow_pod_list --resource pods --verb list
role.rbac.authorization.k8s.io/allow_pod_list created
tmp-ssl ✌️ $ k create rolebinding allow_pod_list --role allow_pod_list --user willy
rolebinding.rbac.authorization.k8s.io/allow_pod_list created
so now we switch context back to willy and test:
tmp-ssl ✌️ $ k config use-context willy@kc-austin
Switched to context "willy@kc-austin".
tmp-ssl ✌️ $ k config current-context
willy@kc-austin
tmp-ssl ✌️ $ k get pods
NAME READY STATUS RESTARTS AGE
coffee-7c86d7d67c-429hb 1/1 Running 19 (8m38s ago) 69d
coffee-7c86d7d67c-s9v5r 1/1 Running 19 (8m38s ago) 69d
tmp-ssl ✌️ $ k get nodes
Error from server (Forbidden): nodes is forbidden: User "willy" cannot list resource "nodes" in API group "" at the cluster scope
as we see the user can list pods but cannot list nodes which is the expected behavior.
Create user .crt using Kubernetes Api
Let’s generate a new .key for a new user named "eddy"
tmp-ssl ✌️ $ openssl genrsa -out eddy.key 2048
Generating RSA private key, 2048 bit long modulus
...................+++
.....................+++
e is 65537 (0x10001)
tmp-ssl ✌️ $ ls -l
total 32
-rw-r--r-- 1 willy staff 1. 675 Apr 8 09:19 eddy.key
We create a .csr from the .key already created (answers highlighted):
tmp-ssl ✌️ $ openssl req -new -key eddy.key -out eddy.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:US
State or Province Name (full name) []:FL
Locality Name (eg, city) []:O
Organization Name (eg, company) []:NC
Organizational Unit Name (eg, section) []:CS
Common Name (eg, fully qualified host name) []:eddy
Email Address []:eddy@gmail.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:tmp-ssl ✌️ $ ls -l e*
-rw-r--r-- 1 willy staff 1013 Apr 8 09:37 eddy.csr
-rw-r--r-- 1 willy staff 1675 Apr 8 09:19 eddy.key
Now we create a .yaml manifest so we can ask the cluster to create the .crt for eddy.
tmp-ssl ✌️ $ cat eddy.csr
-----BEGIN CERTIFICATE REQUEST-----
MIICszCCAZsCAQAwbjELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkZMMQowCAYDVQQH
Gekyx96SnB6LOX33EoCTm5iagr2gNQk=
-----END CERTIFICATE REQUEST-----
We have to change this cert format to base64 in order to include it in our yaml manifest and for that we run the following command:
tmp-ssl ✌️ $ cat eddy.csr | base64 | tr -d "\n"
this command grab the certificate, encode it in base64 and strip out the new lines and returns us the result that we copy and paste in the .yaml file.
Our eddy-csr.yaml files looks like this:
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: eddy
spec:
groups:
- developers
request: LS0tLS1CRUdJTiBDRVJUSUZJ. . . <edited>. . .
signerName: kubernetes.io/kube-apiserver-client
expirationSeconds: 86400 # one day
usages:
- client auth
We now create the .csr request in the cluster as follow:
tmp-ssl ✌️ $ k create -f eddy-csr.yaml
certificatesigningrequest.certificates.k8s.io/eddy created
tmp-ssl ✌️ $
and we check if it was processed:
tmp-ssl ✌️ $ k get csr
NAME AGE SIGNERNAME REQUESTOR REQDUR. CONDITION
eddy 38s kubernetes.io/kube-. . .kubernetes-admin 24h Pending
tmp-ssl ✌️ $
the certificate is shown “Pending” which means is pending for approval by the cluster administrator, se we go ahead and approve it:
tmp-ssl ✌️ $ k certificate approve eddy
certificatesigningrequest.certificates.k8s.io/eddy approved
tmp-ssl ✌️ $ k get csr
NAME AGE SIGNERNAME REQUESTOR REQDUR. CONDITION
eddy 79m kubernetes.io/kube- kubernetes-admin 24h Approved,Issued
tmp-ssl ✌️ $
as we see the certificate was approved and issued. Now we need to retrieve it so we can send it to the user
tmp-ssl ✌️ $ k get csr eddy -o yaml
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
creationTimestamp: "2022-04-08T14:00:57Z"
name: eddy
resourceVersion: "2124723"
uid: 79c926d6-28c4-43c9-b0ea-68bde8808af2
spec:
expirationSeconds: 86400
groups:
- system:masters
- system:authenticated
request: LS0tLS1CRU . . .
signerName: kubernetes.io/kube-apiserver-client
usages:
- client auth
username: kubernetes-admin
status:
certificate: LS0tLS1CRUdJTi . . .
conditions:
- lastTransitionTime: "2022-04-08T15:19:33Z"
lastUpdateTime: "2022-04-08T15:19:33Z"
message: This CSR was approved by kubectl certificate approve.
reason: KubectlApprove
status: "True"
type: Approved
tmp-ssl ✌️ $
now we extract the cert that is in the status field and translate from encoded base64
tmp-ssl ✌️ echo ‘LS0tLS1CRUdJTi . .
tmp-ssl ✌️ $ ls -l e*
-rw-r--r-- 1 willy staff 1599 Apr 8 09:56 eddy-csr.yaml
-rw-r--r-- 1 willy staff 1172 Apr 8 11:47 eddy.crt
-rw-r--r-- 1 willy staff 1013 Apr 8 09:37 eddy.csr
-rw-r--r-- 1 willy staff 1675 Apr 8 09:19 eddy.key
tmp-ssl ✌️ $
We pass the certificates to user eddy.
Here we can create a kubeconfig file for user eddy, let’s say we called eddy.config using as a base the file we have in ~/.kube/config and changing the certificate files as shown below.
. . .
- name: willy
user:
client-certificate: /Users/willy/tmp-ssl/eddy.crt
client-key: /Users/willy/tmp-ssl/eddy.key
note that alternatively we could insert the base64 encoded version instead of providing the path to the files. If we do that, we need to change the name of the parameters as shown below:
. . .
client-certificate-data: LS0tLS1CRUdJ
client-key-data: LS0tLS1CR
we concluded our task.