Article directory
Job
Job is used to batch short one-time tasks and guarantee a specified number of Pods to end successfully.
K8S supports the following methods:
- Non-parallel Job:
- Usually only one Pod runs, and the Pod exits when the Job ends successfully.
- Parallel Jobs with a fixed number of completions:
- Run the specified number of Pods concurrently until the specified number of Pods succeed and the Job ends.
- Parallel Job with Work Queue:
- Users can specify the number of pods in parallel, when any pod ends successfully, no new pods will be created
- Once a Pod ends successfully, and all Pods end, the Job ends successfully.
- Once a Pod ends successfully, the other Pods are ready to exit.
Job Spec
The complete Job field can refer to [Job]. Job has several main parameters that are used to specify the number of completions, run concurrently, retry on errors, etc.:
- .spec.completions: Specifies the number of times the job needs to successfully run Pods. Default: 1
- .spec.parallelism: Specifies the number of Pods the job should run concurrently at any one time. Default: 1
- .spec.activeDeadlineSeconds: Specifies the time period for which the job can be run. The system will try to terminate it before the time expires.
- .spec.backoffLimit: Specifies the number of times to retry the job after it fails. The default is 6 times. There will be a delay time for retry after each failure. This time increases exponentially, and the maximum time is 6min.
Known Issues
Issue #54870
, When .spec.template.spec.restartPolicy is set to "Onfailure", it will conflict with .spec.backoffLimit. You can temporarily set restartPolicy to "Never" to avoid it.
Note 1: .spec.activeDeadlineSeconds has a higher priority than .spec.backoffLimit. If the time is up, but the backoffLimit has not yet arrived, the Job will also be forced to stop.
Job mode
There are several typical modes of Job applied to different business scenarios:
- Extend based on Job template:
- You need to write a general job template first, and generate multiple Job json/yml files for job creation according to different parameters. You can use the same tags for job management.
- Queue by each work item:
- Users are required to prepare a message queue service in advance, such as rabbitMQ, which is a public component, and each work item can insert task messages into it.
- Users can create parallel jobs that need to be able to apply to the message queue, and then consume tasks from the message queue and process them until the message is processed.
- In this mode, the user needs to fill in according to the number of items
spec.completions, and the parallel quantity.spec.parallelismcan be filled in according to the actual situation. In this mode, the job will end successfully when all tasks are successfully completed.
- Queue with variable number of tasks:
- Users are required to prepare a storage service in advance to save the work queue, such as Redis. Each item can populate the storage service with messages.
- Users can start multiple parallel jobs applicable to the work queue for message processing. The difference from the previous Rabbit message queue is that each Job task can know that the work queue is empty and can exit successfully at this time.
- In this mode, it
spec.completionsneeds to be set to 1, and the parallel number.spec.parallelismcan be filled in according to the actual situation. As long as one of the tasks completes successfully, the job will end successfully.
- Ordinary static task
CronJob
cronJob is the timing management of tasks based on time:
- Run a task at a specific point in time
- Repeatedly run tasks at specified time points: such as scheduled database backups, scheduled emails, etc.
CronJob Spec
For the complete spec field, you can refer to [CronJob], which introduces several main fields:
- .spec.schedule: Specify the task running period, the specific format reference [Cron - Wikipedia]
- .spec.startingDeadlineSeconds: Specifies the deadline for the task to run
- .spec.concurrencyPolicy: Specifies the concurrency policy of the task. The parameters support Allow, Forbid and Replace.
- .spec.jobTemplate: Specify the task to be run, the format is the same as [Job]. So in fact, cronJob is implemented based on Job.