Continuous integration deployment-k8s-advanced scheduling-CronJob: application of scheduled tasks

Advanced Scheduling-CronJob: Application of scheduled tasks

1 Introduction
2. CronJob scheduled tasks on Linux
3. Create K8s CronJob
4. Delete k8s CronJob

1 Introduction

In Kubernetes, CronJob is a resource object used to create scheduled tasks. It allows us to schedule and run regular jobs in the cluster, similar to cron tasks in Linux systems.

CronJob resources make it easy to create scheduled tasks on Kubernetes without relying on external tools or services. Here are some of the key features and applications of CronJob:

定时调度: CronJob allows you to define task scheduling rules based on Cron expressions, such as execution every morning, execution every Friday, etc.
作业管理: The task created by CronJob will generate the corresponding Job resource and execute the task at the specified time. If the task completes successfully, the Job is marked as completed; if the task fails, the Job retains the failed status and is retried according to the configured retry policy.
并行处理: CronJob allows you to configure concurrency, i.e. the number of tasks executed simultaneously. This can help you control resource usage and avoid running out of resources due to too many tasks running simultaneously.
灵活的配置: CronJob supports many configuration options, such as defining task container images, environment variables, command parameters, etc., making the execution of scheduled tasks more flexible.

2. CronJob scheduled tasks on Linux

On Linux,CronJob is a cron tool that allows you to run a command or script at a specified interval.

Cron expression:

# ┌───────────── 分钟 (0 - 59)
# │ ┌───────────── 小时 (0 - 23)
# │ │ ┌───────────── 月的某天 (1 - 31)
# │ │ │ ┌───────────── 月份 (1 - 12)
# │ │ │ │ ┌───────────── 周的某天 (0 - 6)（周日到周一；在某些系统上，7 也是星期日）
# │ │ │ │ │                          或者是 sun，mon，tue，web，thu，fri，sat
# │ │ │ │ │
# │ │ │ │ │
# * * * * *

Here are the general steps for using CronJob on Linux:

编辑 Cron 表: Open a terminal and use the following command to edit the cron table.

crontab -e

If this is your first time editing a Cron table, you will be prompted to choose a default editor.

添加定时任务: In the Cron table, each row represents a scheduled task. Each line contains six fields, representing the minute, hour, date, month, day of the week, and the command to be executed. You can adjust these parameters as needed.
For example, to run the /path/to/command command every day at 2 AM, add the following to the Cron table:

0 2 * * * /path/to/command

This means executing the /path/to/command command at minute 0 and hour 2 (2 a.m.) every day.

保存 Cron 表: When you have finished editing, save and close the Cron table. The editor will automatically write the new cron table to the system.
验证 Cron 任务: You can use the following command to view the contents of the current user's Cron table.

crontab -l

This will display all scheduled tasks for the current user.

管理 Cron 任务: By editing the Cron table, you can add, modify or delete scheduled tasks. When you need to make changes to the scheduled task, use the crontab -e command to edit the Cron table again.

In addition, there are some other tips for using CronJob:

You can use the * wildcard to represent any value, for example * * * * * means run every minute.
You can specify multiple values using comma-separated values, for example 0,15,30,45 * * * * to run at minutes 0, 15, 30, and 45 of every hour.
You can use hyphens to specify a range, for example 0 9-17 * * 1-5 to run every hour on the hour between 9am and 5pm, Monday through Friday.

Please note that CronJob accuracy is minute-level and will be affected by system time. Therefore, ensure that the system time is accurate.

Take a look at my scheduled tasks here:

[root@docker-54 config]# crontab -l
0-59/10 * * * * /usr/sbin/ntpdate us.pool.ntp.org | logger -t NTP
[root@docker-54 config]#

The meaning of this scheduled task is to execute the /usr/sbin/ntpdate us.pool.ntp.org command every 10 minutes, and pass the output of the command to logger -t NTP through a pipe.

Let me explain what each part means:

0-59/10: This represents the minute field, indicating that the task will be executed every 10 minutes. The range is from 0 to 59, and the number after the slash indicates the step size, which is the interval. So, 0-59/10 means that the task is executed every 10 minutes.
*: This is a wildcard character that means "every".
*: This is a wildcard character, indicating no restriction.
*: This is a wildcard character, indicating no restriction.
/usr/sbin/ntpdate us.pool.ntp.org: This is the command to execute, which will get the time from us.pool.ntp.org and synchronize it using NTP protocol.
|: This is the pipe symbol used to pass the output of a command to the next command.
logger -t NTP: This is to record the output of the command to the system log through the logger tool and mark it as NTP.

Therefore, the meaning of this scheduled task is to use the NTP protocol to obtain the time from us.pool.ntp.org every 10 minutes and synchronize it, and at the same time record the output of the command to the system log for later viewing. Or track time synchronization.

3. Create K8s CronJob

Create a new configuration file:cron-job-pd.yaml

apiVersion: batch/v1
kind: CronJob # 定时任务
metadata:
  name: cron-job-test # 定时任务名字
spec:
  concurrencyPolicy: Allow # 并发调度策略：Allow 允许并发调度，Forbid：不允许并发执行，Replace：如果之前的任务还没执行完，就直接执行新的，放弃上一个任务
  failedJobsHistoryLimit: 1 # 保留多少个失败的任务
  successfulJobsHistoryLimit: 3 # 保留多少个成功的任务
  suspend: false # 是否挂起任务，若为 true 则该任务不会执行
#  startingDeadlineSeconds: 30 # 间隔多长时间检测失败的任务并重新执行，时间不能小于 10
  schedule: "* * * * *" # 调度策略
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: busybox
            image: busybox:1.28
            imagePullPolicy: IfNotPresent
            command:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure

Here, use the busybox image to create a container and execute the date; echo Hello from the Kubernetes cluster command once every minute.

[root@docker-54 jobs]# kubectl apply -f cron-job-pd.yaml 
cronjob.batch/cron-job-test created
[root@docker-54 jobs]# 
[root@docker-54 jobs]# kubectl get cronjob
NAME            SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cron-job-test   * * * * *   False     0        <none>          5s
[root@docker-54 jobs]#
[root@docker-54 jobs]# kubectl get cj
NAME            SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cron-job-test   * * * * *   False     0        <none>          36s
[root@docker-54 jobs]#

Displayed here LAST SCHEDULE is none, which means that the last scheduling record does not exist, proving that it has not been executed yet.

[root@docker-54 jobs]# kubectl describe cj cron-job-test
Name:                          cron-job-test
Namespace:                     default
Labels:                        <none>
Annotations:                   <none>
Schedule:                      * * * * *
Concurrency Policy:            Allow
Suspend:                       False
Successful Job History Limit:  3
Failed Job History Limit:      1
Starting Deadline Seconds:     <unset>
Selector:                      <unset>
Parallelism:                   <unset>
Completions:                   <unset>
Pod Template:
  Labels:  <none>
  Containers:
   busybox:
    Image:      busybox:1.28
    Port:       <none>
    Host Port:  <none>
    Command:
      /bin/sh
      -c
      date; echo Hello from the Kubernetes cluster
    Environment:     <none>
    Mounts:          <none>
  Volumes:           <none>
Last Schedule Time:  Sun, 26 Nov 2023 22:35:00 +0800
Active Jobs:         cron-job-test-28350155
Events:
  Type    Reason            Age   From                Message
  ----    ------            ----  ----                -------
  Normal  SuccessfulCreate  57s   cronjob-controller  Created job cron-job-test-28350155
[root@docker-54 jobs]#

You can download detailed information about this task.

Then look at the status:

[root@docker-54 jobs]# kubectl get cj
NAME            SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cron-job-test   * * * * *   False     0        32s             3m30s
[root@docker-54 jobs]#

You can see that 32s has passed since the last scheduling.

Also take a look at the execution record:

[root@docker-54 jobs]# kubectl get po | grep cron
cron-job-test-28350156--1-xn5xf   0/1     Completed   0               2m46s
cron-job-test-28350157--1-9mfwx   0/1     Completed   0               106s
cron-job-test-28350158--1-75v5k   0/1     Completed   0               46s
[root@docker-54 jobs]# 
[root@docker-54 jobs]# kubectl logs -f cron-job-test-28350158--1-75v5k
Sun Nov 26 14:38:01 UTC 2023
Hello from the Kubernetes cluster
[root@docker-54 jobs]#

You can see that these execution tasks are in a successful execution status.

This is also the case in our actual use. We just need to change our script to a custom implementation.

4. Delete k8s CronJob

Next let's delete this CronJob:

[root@docker-54 jobs]# kubectl get cj
NAME            SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cron-job-test   * * * * *   False     0        11s             7m9s
[root@docker-54 jobs]# 
[root@docker-54 jobs]# kubectl delete cj cron-job-test
cronjob.batch "cron-job-test" deleted
[root@docker-54 jobs]# 
[root@docker-54 jobs]# kubectl get cj
No resources found in default namespace.
[root@docker-54 jobs]# 
[root@docker-54 jobs]# kubectl get po | grep cron
[root@docker-54 jobs]#

As you can see, after I deleted the CronJob, the corresponding Pod record was also deleted.