K8s in Action reading notes——[7] ConfigMaps and Secrets: configuring applications
7.1 Configuring containerized applications
Before we discuss how to pass configuration data to applications running in Kubernetes, let's look at how containerized applications are typically configured.
If you ignore the fact that you can embed configuration information into the application itself, when developing a new application it is common to configure the application via command line parameters. Then, as the list of configuration options grows, you can move the configuration into configuration files.
Another widely popular way to pass configuration options in containerized applications is through environment variables . Instead of reading a configuration file or command line parameters, the application looks for the value of an environment variable. For example, the official MySQL container image uses MYSQL_ROOT_PASSWORDan environment variable named to set the password for the root superuser account.
But why are environment variables so popular in containers? Using configuration files inside a Docker container is a bit tricky because you have to embed the configuration file into the container image itself or mount the volume containing the files into the container. It's obvious that embedding files into an image is similar to hardcoding configuration into your application's source code, in that it requires you to rebuild the image every time you want to change the configuration. Additionally, anyone with access to the static can see the configuration, including any information that should be kept private, such as credentials or encryption keys. Using a volume is better, but you still need to make sure the files are written to the volume before starting the container.
If you read the previous chapters, you might consider using a gitRepo volume as a configuration source. This is not a bad idea as it allows you to keep versions of the configuration and enables you to easily roll back configuration changes if needed. However, there is an easier way to put the configuration data into a top-level Kubernetes resource and store it and all other resource definitions in the same Git repository or any other file-based storage. The Kubernetes resource used to store configuration data is called a ConfigMap . We'll learn how to use it in this chapter.
Regardless of whether you use a ConfigMap to store configuration data, you can configure your application in the following ways:
- Pass command line parameter settings to the container.
- Custom environment variables for each container.
- Configuration files are mounted into the container via a special type of volume.
We'll cover all of these ways in the next few sections, but before we get started, let's look at configuration options from a security perspective. Although most configuration options do not contain any sensitive information, several do, such as credentials, private encryption keys, and similar data that need to be kept secret. This type of information needs to be handled with special care, which is why Kubernetes provides another object type called Secret. We'll learn more about it in the last part of this chapter.
7.2 Passing command-line arguments to containers
In all the examples so far, the containers you have created have run the default commands defined in the container image. Kubernetes allows overriding commands in a Pod's container definition when you want to run a different executable, or want to run it with different command line arguments.
7.2.1 Defining the command and arguments in Docker
The command executed in the container consists of two parts: command and parameters.
Understand ENTRYPOINT and CMD
CMD
Similar to the RUN instruction, it is used to run programs, but they run at different points in time:
- CMD is run when docker run.
- RUN is in docker build.
Note : If there are multiple CMD instructions in the Dockerfile, only the last one will take effect.
Format:
CMD <shell 命令>
CMD ["<可执行文件或命令>","<param1>","<param2>",...]
CMD ["<param1>","<param2>",...] # 该写法是为 ENTRYPOINT 指令指定的程序提供默认参数
It is recommended to use the second format, as the execution process is clearer. The first format will actually be automatically converted into the second format during operation, and the default executable file is sh.
ENTRYPOINT
Similar to the CMD command, but it will not be overridden by commands specified by the command line parameters of docker run, and these command line parameters will be used as parameters to the program specified by the ENTRYPOINT command.
However, if you use the --entrypoint option when running docker run, the program specified by the ENTRYPOINT directive will be overwritten.
Advantages : When executing docker run, you can specify the parameters required for ENTRYPOINT operation.
Note : If there are multiple ENTRYPOINT instructions in the Dockerfile, only the last one will take effect .
Format:
ENTRYPOINT ["<executeable>","<param1>","<param2>",...]
It can be used with the CMD command: CMD is generally used to change parameters. CMD here is equivalent to passing parameters to ENTRYPOINT, as will be mentioned in the following examples.
Example:
Assume that the nginx:test image has been built through the Dockerfile:
FROM nginx
ENTRYPOINT ["nginx", "-c"] # 定参
CMD ["/etc/nginx/nginx.conf"] # 变参
1. Run without passing parameters
$ docker run nginx:test
The following command will be run by default in the container to start the main process.
nginx -c /etc/nginx/nginx.conf
2. Parameter transfer operation
$ docker run nginx:test -c /etc/nginx/new.conf
The following command will be run by default in the container to start the main process (/etc/nginx/new.conf: assuming this file already exists in the container)
nginx -c /etc/nginx/new.conf
Understand shell format and exec format
A command has two formats
shellFormat:ENTRYPOINT node app.js.execFormat:ENTRYPOINT ["node", "app.js"]
The difference is whether the specified command is called in the shell.
In the kubia image we created earlier, we used a format execthat would run the node process directly (rather than inside a shell).
If you used the shell form ( ENTRYPOINT node app.js), then these will be the container's processes.
Make the interval of fortune mirroring configurable through parameters
Let's modify the fortune script and image to make the delay interval in the loop configurable. You will add an INTERVAL variable and initialize it to the value of the first command line argument, as shown in the following listing.
#!/bin/bash
trap "exit" SIGINT
INTERVAL=$1
echo Configured to generate new fortune every $INTERVAL seconds
mkdir -p /var/htdocs
while :
do
echo $(date) Writing fortune to /var/htdocs/index.html
/usr/games/fortune > /var/htdocs/index.html
sleep $INTERVAL
done
The Dockerfile will be modified so that it uses the exec version of the ENTRYPOINT instruction and sets the default interval to 10 seconds using the CMD instruction as shown below:
FROM ubuntu:latest
RUN apt-get update ; apt-get -y install fortune
ADD fortuneloop.sh /bin/fortuneloop.sh
ENTRYPOINT ["/bin/fortuneloop.sh"]
CMD ["10"]
Then push it to Dockerhub, here I use the author's image:docker.io/luksa/fortune:args
$ docker pull docker.io/luksa/fortune:args
$ docker run -it docker.io/luksa/fortune:args
Configured to generate new fortune every 10 seconds
Mon May 29 11:53:31 UTC 2023 Writing fortune to /var/htdocs/index.html
The default sleep interval can also be overridden by passing parameters:
$ docker run -it docker.io/luksa/fortune:args 15
Configured to generate new fortune every 15 seconds
Mon May 29 11:54:58 UTC 2023 Writing fortune to /var/htdocs/index.html
7.2.2 Overriding the command and arguments in Kubernetes
In Kubernetes, you can optionally override ENTRYPOINT and CMD when specifying a container. To do this, set the command and args properties in the container specification as shown:
kind: Pod
sepc:
containers:
- image: some/image
command: ["/bin/command"]
args: ["arg1", "arg2"]
In most cases, just set custom parameters and rarely override commands.
The corresponding relationship between commands and parameters in Docker and Kubernetes is as follows:
The Pod configuration file is as follows:
# fortune-pod-args.yaml
apiVersion: v1
kind: Pod
metadata:
name: fortune2s
spec:
volumes:
- name: html
emptyDir: {
}
containers:
# 镜像替换
- image: luksa/fortune:args
# 传入参数，睡眠间隔修改为2秒
args: ["2"]
name: html-generator
volumeMounts:
- name: html
mountPath: /var/htdocs
- image: nginx:alpine
name: web-server
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
readOnly: true
ports:
- containerPort: 80
protocol: TCP
If you need to pass in multiple parameters, you can pass them like this:
args:
- foo
- bar
- "15"
The numbers must be enclosed in quotation marks.
7.3 Setting environment variables for a container
Containerized applications often use environment variables as configuration options. Kubernetes allows you to specify a custom list of environment variables for each Pod's container , as shown in Figure 7.1. Although it would be useful to define environment variables at the Pod level and let the container inherit them, there is currently no such option.
Make the interval of fortune mirror configurable through environment variables
Modify fortuneloop.sh to make the sleep time configurable through environment variables:
#!/bin/bash
trap "exit" SIGINT
echo Configured to generate new fortune every $INTERVAL seconds
mkdir -p /var/htdocs
while :
do
echo $(date) Writing fortune to /var/htdocs/index.html
/usr/games/fortune > /var/htdocs/index.html
sleep $INTERVAL
done
Just delete the one in the previous script INTERVAL=$1.
7.3.1 Specifying environment variables in a container definition
After you build the new image (which the author tagged luksa/fortune:env), and push it to Docker Hub, you can run it by creating a new Pod, where you pass the environment variables to the script by including them in the container definition, as follows Shown:
apiVersion: v1
kind: Pod
metadata:
name: fortune2s
spec:
volumes:
- name: html
emptyDir: {
}
containers:
# 镜像替换
- image: luksa/fortune:env
# 设置环境变量，睡眠间隔修改为30秒
env:
- name: INTERVAL
value: "30"
name: html-generator
volumeMounts:
- name: html
mountPath: /var/htdocs
- image: nginx:alpine
name: web-server
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
readOnly: true
ports:
- containerPort: 80
protocol: TCP
As mentioned before, set environment variables in the container definition, not at the Pod level.
Don’t forget that within each container, Kubernetes also automatically exposes environment variables for each service in the same namespace. These environment variables are basically injected automatically.
7.3.2 Referring to other environment variables in a variable’s value
In the previous example, you set a fixed value for the environment variable, but you can also use the $(VAR)syntax to reference a previously defined environment variable or any other existing variable. If you define two environment variables, the second may contain the value of the first, like this:
env:
- name: FIRST_VAR
value: "foo"
- name: SECOND_VAR
value: "$(FIRST_VAR)bar"
7.3.3 Understanding the drawback of hardcoding environment variables
Adding hardcoded values in the Pod definition means you need to write different Pod definitions for production Pods and development Pods. To reuse the same Pod definition in multiple environments, it is best to decouple configuration from Pod descriptors . You can do this using a ConfigMap resource and use valueFromit (instead of the value field) as the source of the environment variable value.
7.4 Decoupling configuration with a ConfigMap
7.4.1 Introducing ConfigMaps
Kubernetes allows configuration options to be separated into a separate object called a ConfigMap, which is a Map containing key/value pairs, with values ranging from simple text to full configuration files.
The application does not need to read directly or even know that the ConfigMap exists. Instead, key/value pairs are passed to the container as environment variables or as files (see Figure 7.2). Because environment variables can be referenced in command line arguments using the $(ENV_VAR) syntax, you can also pass ConfigMap entries as command line arguments to the process.
Of course, if desired, the application can also read the contents of the ConfigMap directly through the Kubernetes REST API endpoint, but unless you really need to do this, you should make your application as Kubernetes agnostic as possible.
Regardless of how your application uses ConfigMaps, placing configuration information in separate objects allows you to maintain multiple ConfigMaps with the same name for different environments (development, test, QA, production, etc.). Because Pods reference ConfigMap by name, it is possible to use the same Pod specification in all environments while using different configuration information in each environment (see Figure 7.3).
7.4.2 Creating a ConfigMap
A special kubectl create configmapcommand will be used to create the ConfigMap, rather than a generic kubectl create -fcommand to publish YAML.
Entries in the Map can be defined by passing the command directly to the kubectl command, or a ConfigMap can be created from a file stored on disk. Start by typing a simple command:
$ kubectl create configmap fortune-config --from-literal=sleep-interval=25
configmap/fortune-config created
This creates a ConfigMap named fortune-config with a single entry of sleep-interval = 25 (see Figure 7.4).
ConfigMap usually contains multiple entries. If you want to create a ConfigMap with multiple literal entries, you need to add multiple --from-literal parameters.
$ kubectl create configmap myconfigmap --from-literal=foo=bar --from-literal=bar=baz --from-literal=one=two
Let us use the kubectl get command to view the YAML descriptor of the created ConfigMap as shown below:
$ kubectl get configmap fortune-config -o yaml
apiVersion: v1
data:
sleep-interval: "25"
kind: ConfigMap
metadata:
creationTimestamp: "2023-05-29T13:56:08Z"
name: fortune-config
namespace: default
resourceVersion: "2881457"
uid: 090a67b4-776b-4295-ad59-37d2f5206c8c
This is no different, you can $ kubectl create -f fortune-config.yamleasily configure it via YAML files.
Create ConfigMap from file
ConfigMap can also store coarse-grained configuration data, such as complete configuration files . To enable this, the kubectl create configmap command also supports reading files from disk and storing them as individual entries in the ConfigMap.
$ kubectl create configmap my-config --from-file=config-file.conf
When you execute the previous command, kubectl will look for the config-file.conf file in the directory where you run kubectl. The contents of the file are then stored under a key in the ConfigMap config-file.conf(the filename is used as the map key), but you can also specify a key manually, for example:
$ kubectl create configmap my-config --from-file=customkey=config-file.conf
This stores the contents of the file customkeyunder the key, and can also be used to --from-filecreate multiple entries.
Create a ConfigMap from files in a directory
It's even possible to import all files from a directory of files without importing each file individually:
$ kubectl create configmap my-config --from-file=/path/to/dir
In this case, kubectl will create a separate map entry for each file in the specified directory, but only for files whose filenames are valid ConfigMap keys.
Combine different options
When creating a ConfigMap, you can combine all the options mentioned in this chapter
$ kubectl create configmap my-config --from-file=foo.json --from-file=bar=foobar.conf --from-file=config-opts/ --from-literal=some=thing
7.4.3 Passing a ConfigMap entry to a container as an environment variable
So, how to pass the value in the Map to the Pod's container? There are three options. Let's start with the simplest - setting environment variables. The Pod descriptor should look like this:
# fortune-pod-env-configmap.yaml
apiVersion: v1
kind: Pod
metadata:
name: fortune-env-from-configmap
spec:
volumes:
- name: html
emptyDir: {
}
containers:
# 镜像替换
- image: luksa/fortune:env
env:
- name: INTERVAL
valueFrom:
configMapKeyRef:
key: sleep-interval
name: fortune-config
name: html-generator
volumeMounts:
- name: html
mountPath: /var/htdocs
- image: nginx:alpine
name: web-server
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
readOnly: true
ports:
- containerPort: 80
protocol: TCP
You define an environment variable called INTERVAL and set its value to whatever value is stored under the sleep-interval key in the fortune-config ConfigMap. When the process running in the html-generator container reads the INTERVAL environment variable, it will see the value 25 (as shown in Figure 7.6).
Reference to a ConfigMap that does not exist in the Pod
What happens if the ConfigMap referenced when creating the Pod does not exist. Kubernetes will schedule the Pod normally and try to run its container. Containers that reference a non-existent ConfigMap will fail to start, but other containers will start normally . If you subsequently create the missing ConfigMap, you can start the failed container without recreating the Pod.
References to ConfigMap can also be marked as optional (via Settings Configuration
configMapKeyRef.optional: true). In this case, the container will start even if the ConfigMap does not exist.
This example shows you how to separate configuration from Pod specifications. This allows you to keep all configuration options tightly coupled (even for multiple Pods) rather than having them scattered across the Pod definition (or duplicated in multiple Pod configuration files).
7.4.4 Passing all entries of a ConfigMap as environment variables at once
When your ConfigMap contains more than a few entries, creating environment variables from each entry individually becomes tedious and error-prone. Fortunately, Kubernetes provides a way to expose all entries of a ConfigMap as environment variables.
There is a ConfigMap named FOO, BAR and FOO-BAR. Instead of using env like the previous example, you can expose them all as environment variables using the envFrom attribute. As follows:
spec:
containers:
- image: some-image
envFrom: # 用envFrom代替env
- prefix: CONFIG_ # 所有环境变量都将以CONFIG_作为前缀。
configMapRef:
name: my-config-map # 引用名为my-config-map的ConfigMap
As you can see, you can also specify a prefix for environment variables (CONFIG_ in this case). This will result in the following two environment variables within the container: CONFIG_FOOand CONFIG_BAR.
The prefix is optional, so if you omit it, the environment variable has the same name as the key.
Did you notice that I said there are two variables, but earlier I said the ConfigMap has three entries (FOO, BAR and FOO-BAR)? Why don't FOO-BARConfigMap entries have environment variables?
The reason is that CONFIG_FOO-BAR is not a valid environment variable name because it contains hyphens. Kubernetes does not convert keys in any way (for example, it does not convert dashes to underscores). If a ConfigMap key is malformed, the entry will be skipped (but it will log an event to let you know it was skipped).
7.4.5 Passing a ConfigMap entry as a command-line argument
Now, let us see how to pass the values from the ConfigMap as parameters to the main process running in the container. You cannot pod.spec.containers.argsreference a ConfigMap entry directly in a field, but you can first initialize an environment variable from the ConfigMap entry and then reference the variable in a parameter as shown in Figure 7.7.
# fortune-pod-args-configmap.yaml
apiVersion: v1
kind: Pod
metadata:
name: fortune-env-from-configmap
spec:
volumes:
- name: html
emptyDir: {
}
containers:
# 镜像替换
- image: luksa/fortune:env
env:
- name: INTERVAL
valueFrom:
configMapKeyRef:
key: sleep-interval
name: fortune-config
args: ["$(INTERVAL)"]
name: html-generator
volumeMounts:
- name: html
mountPath: /var/htdocs
- image: nginx:alpine
name: web-server
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
readOnly: true
ports:
- containerPort: 80
protocol: TCP
Environment variables are defined as before, but then $(ENV_VARIABLE_NAME)syntax is used to let Kubernetes inject the value of the variable into the parameter.
7.4.6 Using a configMap volume to expose ConfigMap entries as files
Passing configuration options as environment variables or command line arguments is typically used for short variable values. As you can see, a ConfigMap can also contain entire configuration files. When you want to expose them to containers, you can use a special volume type mentioned in the previous chapter, the ConfigMap volume.
A ConfigMap volume exposes each ConfigMap entry as a file. A process running in the container can obtain the value of the entry by reading the file contents.
Although this method is mainly used for passing large configuration files into the container, there is nothing preventing short single values from being passed in this way.
# 启用了gzip压缩的Nginx配置：my-nginx-config.conf
server{
listen 80;
server_name www.kubia-example.com;
# 为纯文本和XML文件启用了gzip压缩。
gzip on;
gzip_types text/plain application/xml;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
}
Now use the kubectl delete configmap fortune-config command to delete the existing fortune-config ConfigMap so that it can be replaced with a new ConfigMap that will include the Nginx configuration files. A ConfigMap will be created from a file stored on local disk.
Create a new directory called configmap-files and store the Nginx configuration files from the previous article into the configmap-files/my-nginx-config.conf file. In order for the ConfigMap to also contain a sleep-interval entry, add a plain text file called sleep-interval in the same directory and store the number 25 in it (see Figure 7.8).
Now create a ConfigMap from all the files in the directory as follows:
$ kubectl create configmap fortune-config --from-file=configmap-files/
configmap/fortune-config created
View its YAML content:
$ kubectl get configmap fortune-config -o yaml
apiVersion: v1
data:
my-nginx-config.conf: |-
server{
listen 80;
server_name www.kubia-example.com;
# 为纯文本和XML文件启用了gzip压缩。
gzip on;
gzip_types text/plain application/xml;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
}
sleep-interval: "25"
kind: ConfigMap
metadata:
creationTimestamp: "2023-05-30T02:25:57Z"
name: fortune-config
namespace: default
resourceVersion: "2973738"
uid: 4411ddfa-7491-4d4b-a1a4-fb9900faba3b
In the first line of two entries, the pipe character after the colon indicates that a literal multiline value is to follow.
This ConfigMap contains two entries, with keys corresponding to the names of the actual files they create. This ConfigMap will now be used in both pod containers.
Using ConfigMap entries in volumes
Creating a volume containing the contents of a ConfigMap is as simple as creating a reference to the ConfigMap by name and mounting the volume in the container. You've learned how to create a volume and mount it, so all you need to learn is how to create a file using ConfigMap entries to initialize the volume.
Nginx /etc/nginx/nginx.confreads its configuration files from files. The Nginx image already contains the default configuration options for this file, and you don't want to overwrite this file, so you don't want to replace it. Fortunately, the default configuration file automatically /etc/nginx/conf.d/includes all .conf files in the subdirectory, so you should add the configuration file in that directory. Figure 7.9 shows what you want to achieve.
The YAML file of Pod is as follows:
# fortune-pod-configmapvolume.yaml
apiVersion: v1
kind: Pod
metadata:
name: fortune-configmap-volume
spec:
containers:
- image: luksa/fortune:env
env:
- name: INTERVAL
valueFrom:
configMapKeyRef:
name: fortune-config
key: sleep-interval
name: html-generator
volumeMounts:
- name: html
mountPath: /var/htdocs
- image: nginx:alpine
name: web-server
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
readOnly: true
- name: config
# 将在此位置挂载configMap卷
mountPath: /etc/nginx/conf.d
readOnly: true
- name: config
mountPath: /tmp/whole-fortune-config-volume
readOnly: true
ports:
- containerPort: 80
name: http
protocol: TCP
volumes:
- name: html
emptyDir: {
}
- name: config
# 引用了ConfigMap
configMap:
name: fortune-config
verify
The web server should now be configured to compress the responses it sends. This can be verified by enabling port forwarding from localhost:8080 to the pod's port 80 and checking the server's response using curl
$ kubectl port-forward fortune-configmap-volume 8080:80 &
# [1] 9788
# Forwarding from 127.0.0.1:8080 -> 80
$ curl -H "Accept-Encoding: gzip" -I localhost:8080
# HTTP/1.1 200 OK
# Server: nginx/1.21.5
# Date: Tue, 30 May 2023 02:41:54 GMT
# Content-Type: text/html
# Last-Modified: Tue, 30 May 2023 02:41:44 GMT
# Connection: keep-alive
# ETag: W/"64756268-49"
# Content-Encoding: gzip
Let's take a look at /etc/nginx/conf.dthe contents of the directory
$ kubectl exec fortune-configmap-volume -c web-server ls /etc/nginx/conf.d
# my-nginx-config.conf
# sleep-interval
Both entries for the ConfigMap have been added as files to the directory. Although the sleep-interval entry is included, it does not belong in this directory because it only applies to fortuneloop containers. You can create two different ConfigMaps and use one to configure the fortuneloop container and the other to configure the web-server container. However, somehow it feels wrong to use multiple ConfigMaps to configure containers in the same pod. After all, putting containers in the same pod means that the containers are closely related and should probably be configured as a unit.
Show some ConfigMap entries in the volume
It is possible to populate a configMap volume using only a portion of the ConfigMap entries, such as only the my-nginx-config.conf entry. This does not affect the fortuneloop container because you pass the sleep-interval entry to it through an environment variable, not through the volume.
To define which entries should be exposed as files in the configMap volume, use the items property of the volume as shown
volumes:
- name: html
emptyDir: {
}
- name: config
# 引用了ConfigMap
configMap:
name: fortune-config
items:
- key: my-nginx-config.conf
path: gzip.conf
When specifying individual entries, you need to set the file name and entry key for each entry. If you run the Pod from the previous list, /etc/nginx/conf.d only the gzip.conf file and no other files are included.
Mounting a directory hides existing files in that directory
The volume is mounted as a directory, which means any files stored in the /etc/nginx/conf.d directory in the container image are hidden.
This is usually what happens in Linux when a file system is mounted to a non-empty directory. The directory then contains only files from the mounted file system, and the original files in that directory are inaccessible during the file system mount.
Imagine mounting a volume into the /etc directory, which typically contains many important files. This will most likely break the entire container, as all the original files that were supposed to be in the /etc directory will no longer exist. If you need to add files to a directory like /etc, you cannot use this method at all .
Mount a single ConfigMap entry as a file without hiding other files in the directory
volumeMountAnother property on subPathallows mounting individual files or individual directories from the volume instead of mounting the entire volume. As shown below:
Suppose you have a myconfig.confconfigMap volume containing files and want to add it to /etcthe directory as someconfig.conf. It can be mounted into this directory using subPaththe attribute without affecting any other files in the directory. The following Pod definition shows the relevant parts:
spec:
containers:
- image: some/image
volumeMounts:
- name: myVolume
mountPath: /etc/someconfig.conf
# 只需挂载myconfig.conf条目，而不是挂载整个卷
subPath: myconfig.conf
By default, permissions for all files in a ConfigMap volume are set to 644（-rw-r--r--）. You can change this property by setting the defaultMode property in the volume specification,
volumes:
- name: config
configMap:
name: fortune-config
defaultMode: "6600" # 权限设置为-rw-rw------
7.4.7 Updating an app’s config without having to restart the app
When you update a ConfigMap, the files in all volumes that reference it are updated. It is then up to the process to detect that they have been changed and reload them.
EditConfigMap
Now let's see how to change the ConfigMap and have processes in the Pod reload the files exposed in the configMap volume. You will modify the Nginx configuration file from the previous example so that it uses the new configuration without restarting the Pod. Try turning off gzip compression by kubectl editediting the ConfigMap using:fortune-config
$ kubectl edit configmap fortune-config
configmap/fortune-config edited
Changegzip on to gzip off, save the file, and then close the editor. The ConfigMap is then updated, and shortly after, the actual files in the volume are updated as well. This can be confirmed by printing the contents of the file using kubectl exec:
$ kubectl exec fortune-configmap-volume -c web-server cat /etc/nginx/conf.d/my-nginx-config.conf
server{
listen 80;
server_name www.kubia-example.com;
# 为纯文本和XML文件启用了gzip压缩。
gzip off;
gzip_types text/plain application/xml;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
}r
You will see the changes in the configuration files, but you will find that this has no impact on Nginx as it does not automatically watch the files and reload them
Send signal to Nginx to reload configuration
Nginx will continue to compress its responses until it is told to reload its configuration files, which can be done using the following command:
kubectl exec fortune-configmap-volume -c web-server -- nginx -s reload
$ curl -H "Accept-Encoding: gzip" -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Tue, 30 May 2023 03:12:37 GMT
Content-Type: text/html
Content-Length: 51
Last-Modified: Tue, 30 May 2023 03:12:35 GMT
Connection: keep-alive
ETag: "647569a3-33"
Accept-Ranges: bytes
It can be found that the response is no longer compressed
Because files in a ConfigMap volume are not updated synchronously across all running instances, files in a single Pod may go out of sync for up to a full minute .
7.5 Using Secrets to pass sensitive data to containers
7.5.1 Introducing Secrets
To store and distribute this information, Kubernetes provides an independent object called a Secret. Secrets are very similar to ConfigMap - they are also maps that hold key-value pairs. They are used in the same way as ConfigMap, you can:
- Pass the Secret to the container as an environment variable.
- Map the Secret as a file in Volume.
Kubernetes keeps your Secrets secure by ensuring that each Secret is only distributed to nodes running pods that need access to the Secret. Additionally, on a node, Secrets are always stored in memory and are never written to physical storage .
On the control node (more specifically in etcd), starting with Kubernetes version 1.7, etcd stores Secrets in encrypted form, making the system more secure. Therefore, you must choose to use Secret or ConfigMap in the right scenario. The difference between them is simple:
- Use ConfigMap to store non-sensitive general configuration data.
- Use Secrets to store data that is of a sensitive nature and needs to be encrypted. If the configuration file includes sensitive and non-sensitive data, the file should be stored in a Secret.
7.5.2 Introducing the default token Secret
You can start learning about secrets by examining the secrets mounted in each container. You may have noticed this when viewing pods using the kubectl describe command. The output of the command always contains the following:
Volumes:
default-token-cfee9:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-cfee9
Each pod will automatically have a secret volume attached . The volume in the above kubectl describeoutput refers to a Secret named default-token-cfee9. Since Secrets are resources, you can kubectl get secrets list them using , and find the default-token Secret in the list. Let's take a look:
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-lqrbh kubernetes.io/service-account-token 3 16d
You can also use the kubectl describe command to learn more, as shown below.
$ kubectl describe secrets
Name: default-token-lqrbh
Namespace: default
Labels: <none>
Annotations: kubernetes.io/service-account.name: default
kubernetes.io/service-account.uid: 43b73c21-1084-4115-b8e5-0b51580bb18d
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImhwNkxjWFIzSC1KSHNNZ0VaN09YWHdGY1ItYlh4YXNvR3Z2c1NfRmd3encifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZmF1bHQtdG9rZW4tbHFyYmgiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGVmYXVsdCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjQzYjczYzIxLTEwODQtNDExNS1iOGU1LTBiNTE1ODBiYjE4ZCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OmRlZmF1bHQifQ.P3zG24SkN2FPmTIR_a0ZYVl4rH-CF1NTNsjw3VkOjU2J4or5bwUVTs8d1zBBl6YmQEc16v06cgpijXzRJJ6bX99dFfg4eT8Jw1O9_QKhpQp3Wv97LdXgoIRd0PVsMNVfh4IsD0yQwv4hoZsC8thG1muN7sYPSPWH0WFV42EiQ_VZKP7gv-oB3eZxWpEbdP747FvBOR4py-NcCaI6XZBT2xKJWxASguIbv_Pt7Jf-Qt2EUZkSfW1ACbkxBbV3lcqm-b_Kk2XngN5hfcmX0ukyAI2MqgftzGxYCmQzexDKqI-T3wTi_y_7B1RK6vgzWkYr5E1-vDJyHKL4kJkdp0yqrg
ca.crt: 1066 bytes
namespace: 7 bytes
You can see that the Secret contains three entries: ca.crt, namespace, and token, which represent everything you need to securely communicate with the Kubernetes API server from within the pod when needed. The following figure shows the mounting location of the secret volume:
7.5.3 Creating a Secret
Now, you will create your own small Secret. You'll improve your fortune-serving Nginx container by configuring it to also serve HTTPS traffic. To do this, a certificate and private key need to be created. The private key needs to be kept secret, so you will put it and the certificate into Secret.
First, generate the certificate and private key files on your local machine:
$ openssl genrsa -out https.key 2048
$ openssl req -new -x509 -key https.key -out https.cert -days 3650 -subj /CN=www.kubia-example.com
To better illustrate something about Secrets, now create a dummy file named foo that contains the string bar.
Use kubectl create secretthe command to create a Secret from these three files:
$ kubectl create secret generic fortune-https --from-file=https.key --from-file=https.cert --from-file=foo
secret/fortune-https created
This is not much different from creating ConfigMaps. In this case, you are creating a generic Secret called fortune-https, which includes two entries (https.key contains the contents of the https.key file, and https.cert does the same). As you learned earlier, you can also use --from-file=fortune-httpsto include an entire directory instead of specifying each file individually.
7.5.4 Comparing ConfigMaps and Secrets
View the YAML description of the created Secret:
$ kubectl get secret fortune-https -o yaml
apiVersion: v1
data:
foo: YmFyCg==
https.cert: LS0tLS1CRUdJTiBDRV......
https.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBL......
kind: Secret
metadata:
creationTimestamp: "2023-05-30T07:00:51Z"
name: fortune-https
namespace: default
resourceVersion: "3007580"
uid: 2154f404-8cd3-4cec-af76-f37ab51a2e36
type: Opaque
Compare this to the YAML of the ConfigMap created earlier:
$ kubectl get configmap fortune-config -o yaml
apiVersion: v1
data:
my-nginx-config.conf: |-
server{
listen 80;
server_name www.kubia-example.com;
# 为纯文本和XML文件启用了gzip压缩。
gzip off;
gzip_types text/plain application/xml;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
}
sleep-interval: "25"
kind: ConfigMap
metadata:
creationTimestamp: "2023-05-30T02:25:57Z"
name: fortune-config
namespace: default
resourceVersion: "2978884"
uid: 4411ddfa-7491-4d4b-a1a4-fb9900faba3b
The contents of the Secret entry are displayed as a Base64-encoded string, while the contents of the ConfigMap are displayed in clear text .
The reason for using Base64 encoding is simple. Entries for Secrets can contain binary values, not just plain text . Base64 encoding allows you to include binary data in YAML or JSON, which are both plain text formats.
You can even use Secrets for non-sensitive binary data, but be aware that the maximum size of Secrets is limited to 1MB .
Since not all sensitive data exists in binary form, Kubernetes also allows setting the value of Secret through the stringData field:
kind: Secret
apiVersion: v1
stringData: # 可以用于非二进制的Secret数据。
foo: plain text
data:
https.cert: LS0tLS1CRUdJTiBDRV......
https.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBL......
The stringData field is write-only(Please note: write only, not read only). It can only be used to set values. When you use kubectl get -o yaml to retrieve the Secret's YAML, stringData is not displayed. Instead, all entries you specify in the stringData field are displayed under data and Base64 encoded like other entries.
When you expose a Secret to a container via a secret volume, the value of the Secret entry is decoded and written to the file in its actual form (whether it is plain text or binary). The same is true when you expose a Secret entry through an environment variable. In both cases, the application does not need to decode, but can read the file contents or find the environment variable value and use it directly.
7.5.5 Using the Secret in a pod
Modify fortune-config to enable HTTPS
$ kubectl edit configmap fortune-config
# 修改后的conf
my-nginx-config.conf: |-
server{
listen 80;
listen 443 ssl;
server_name www.kubia-example.com;
ssl_certificate certs/https.cert;
ssl_certificate_key certs/https.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
}
This will configure the server to read certificate and key files from /etc/nginx/certs, so you need to mount the secret volume there.
Next, you will create a new fortune-https pod and mount the secret volume to the correct location in the web-server container, as shown in the following listing.
apiVersion: v1
kind: Pod
metadata:
name: fortune-https
spec:
containers:
- image: luksa/fortune:env
name: html-generator
env:
- name: INTERVAL
valueFrom:
configMapKeyRef:
name: fortune-config
key: sleep-interval
volumeMounts:
- name: html
mountPath: /var/htdocs
- image: nginx:alpine
name: web-server
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
readOnly: true
- name: config
mountPath: /etc/nginx/conf.d
readOnly: true
- name: certs # 已将Nginx配置为从/etc/nginx/certs读取证书和密钥文件，因此需要在那里挂载Secret卷
mountPath: /etc/nginx/certs/
readOnly: true
ports:
- containerPort: 80
- containerPort: 443
volumes:
- name: html
emptyDir: {
}
- name: config
configMap:
name: fortune-config
items:
- key: my-nginx-config.conf
path: https.conf
- name: certs # 在此处定义Secret卷
secret:
secretName: fortune-https
Once the pod is running you can see if it is serving HTTPS traffic by opening a port forward tunnel to the pod port 443 and using curl to send a request to the server
$ kubectl port-forward fortune-https 8443:443 &
[1] 32574
$ curl https://localhost:8443 -k
Write yourself a threatening letter and pen a defiant reply.
If you configured the server correctly, you should get a response. You can check the server's certificate to see if it matches the certificate you generated earlier. This can also be done by turning on verbose logging with curl (using the -v option) as follows:
$ curl https://localhost:8443 -k -v
* Rebuilt URL to: https://localhost:8443/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate: # 证书与你创建并存储在Secret中的证书匹配。
* subject: CN=www.kubia-example.com
* start date: May 30 06:57:31 2023 GMT
* expire date: May 27 06:57:31 2033 GMT
* issuer: CN=www.kubia-example.com
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost:8443
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.21.5
< Date: Tue, 30 May 2023 07:40:27 GMT
< Content-Type: text/html
< Content-Length: 79
< Last-Modified: Tue, 30 May 2023 07:40:18 GMT
< Connection: keep-alive
< ETag: "6475a862-4f"
< Accept-Ranges: bytes
<
Q: How many Martians does it take to screw in a light bulb?
A: One and a half.
* Connection #0 to host localhost left intact
By mounting the Secret volume in /etc/nginx/certs in the directory tree, you have successfully passed the certificate and private key to the container. Secret volumes use the memory file system (tmpfs) to store Secret files. If you list the container's mount points, you'll see:
$ kubectl exec fortune-https -c web-server -- mount | grep certs
tmpfs on /etc/nginx/certs type tmpfs (ro,relatime)
Because tmpfs is used, sensitive data stored in Secret will never be written to disk and will not be leaked.
In addition to using volumes, you can expose individual entries for Secrets as environment variables just like you did for the sleep-interval entry in ConfigMap. For example, if you want to expose the foo key in Secret as the environment variable FOO_SECRET, you would add the following code snippet to the container definition:
env:
- name: FOO_SECRET
valueFrom:
secretKeyRef:
name: fortune-https
key: foo
This is pretty much the same way as setting the INTERVAL environment variable, except this time you use secretKeyRefinstead of configMapKeyRefreferencing the Secret.
Think twice before using environment variables to pass your confidential data to the container, as they may be inadvertently exposed . For security reasons, always use a Secret volume to expose a Secret.
7.5.6 Understanding image pull Secrets
You've learned how to pass a Secret to an application and use its data. But sometimes Kubernetes itself requires you to provide it with credentials, such as when you want to use an image from a private container image registry . This is also done via Secret.
Until now, all of your container images have been stored in public image registries, which do not require any special credentials to pull images from. But most organizations don't want their images to be available to everyone, so use private image registries. When deploying a Pod whose container image is in a private registry, Kubernetes needs to know the credentials required to pull the image. Let's see how to do this.
In addition to public image repositories, Docker Hub also allows you to create private repositories. You can mark a repository as private by logging in at http://hub.docker.com, finding the repository and checking a checkbox.
To run a Pod that uses an image from a private repository, you need to do two things:
- Create a Secret to save the credentials for the Docker registry.
imagePullSecretsReference the Secret in the field of the Pod manifest .
Creating a Secret that holds the Docker registry authentication credentials is not much different than creating the generic Secret created in Section 7.5.3. You use the same kubectl create secret command, but with different types and options:
$ kubectl create secret docker-registry mydockerhubsecret \ --docker-username=myusername --docker-password=mypassword \ --docker-email=[email protected]
What you created is a docker-registry Secret named mydockerhubsecret, which specifies your Docker Hub username, password, and email. If you use kubectl describe to view the contents of the newly created Secret, you will see that it contains an entry called dockercfg. This is equivalent to the .dockercfg file in your home directory, which Docker creates when you run the docker login command.
In order for Kubernetes to use this secret when pulling images from your private Docker Hub repository, you need to specify the name of the secret in the Pod specification, as shown in the following code:
apiVersion: v1
kind: Pod
metadata:
name: private-pod
spec:
imagePullSecrets:
- name: mydockerhubsecret
containers:
- image: username/private:tag
name: main