1 Introduction

1.1 paascloud-master

github open source project - paascloud-master: https://github.com/paascloud/paascloud-master
paascloud-masterReliable sources Design Document: https://github.com/paascloud/paascloud-master/wiki/ reliable sources

Document Backup Address: https://blog.csdn.net/liu_zhaoming/article/details/79603036

1.2 Elastic-Job

githubOpen Source Address: https://github.com/elasticjob/elastic-job-lite
Official Documents
Official Development Guide
Use github project
Reference blog: Elastic-timed task Job-- distributed framework .
Reference blog: Application of Distributed regular tasks Elastic-Job framework SpringBoot project (a)

2 How to use

2.1 project requirements

The following items in 5种case of using a timed task Elastic-Job .

Reference: https://github.com/paascloud/paascloud-master/wiki/ reliable sources

Regular cleaning of all subscribers to 7 days prior to 消费成功the message data [0:00] of the day;
Regular cleaning of all producers 7 days prior to 发送成功the message data [1:00] of the day;
Regularly clean up invalid OSS files (pictures) [0:00] every day;
Processing 发送中the message data to perform a] [30 seconds;
Processing 待确认the message data to perform a [10 minutes];
Updating offline token timeout, when the token valid duration of 2 hours per day [0:00];

Here we have the first case analysis of how to use Elastic-Jobthe.

2.2 Timing scavenging messages

Task: message data regularly clean up subscribers consumption of success .

DeleteRpcConsumerMessageJob.java

Here SimpleJobyou can refer to the documentation: http://elasticjob.io/docs/elastic-job-lite/01-start/dev-guide/

/**
 * 定时清理所有订阅者消费成功的消息数据.
 *
 * @author paascloud.net @gmail.com
 */
@Slf4j
@ElasticJobConfig(cron = "0 0 0 1/1 * ?")	// 每天00:00:00
public class DeleteRpcConsumerMessageJob implements SimpleJob {
	@Resource
	private PaascloudProperties paascloudProperties;
	@Resource
	private TpcMqMessageService tpcMqMessageService;

	/**
	 * Execute.
	 *
	 * @param shardingContext the sharding context
	 */
	@Override
	public void execute(final ShardingContext shardingContext) {
		ShardingContextDto shardingContextDto = new ShardingContextDto(shardingContext.getShardingTotalCount(), shardingContext.getShardingItem());
		final TpcMqMessageDto message = new TpcMqMessageDto();
		// 将 Elastic-Job 该定时任务分片上下文设置到消息体中
		message.setMessageBody(JSON.toJSONString(shardingContextDto));
		// 设置清理任务的消息标签：删除消费者历史消息
		message.setMessageTag(AliyunMqTopicConstants.MqTagEnum.DELETE_CONSUMER_MESSAGE.getTag());
		// 设置清理任务的消息主题
		message.setMessageTopic(AliyunMqTopicConstants.MqTopicEnum.TPC_TOPIC.getTopic());
		// 设置该服务的生产组
		message.setProducerGroup(paascloudProperties.getAliyun().getRocketMq().getProducerGroup());
		String refNo = Long.toString(UniqueIdGenerator.generateId());
		message.setRefNo(refNo);
		message.setMessageKey(refNo);
		// 发送清理所有订阅者消费成功的消息数据
		tpcMqMessageService.saveAndSendMessage(message);
	}
}

Annotation @ElasticJobConfig(cron = "0 0 0 1/1 * ?")set the timer task-related information.

Custom annotation reference blog: https://www.cnblogs.com/liaojie970/p/7879917.html

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Component
public @interface ElasticJobConfig {

	/**
	 * cron表达式，用于控制作业触发时间
	 *
	 * @return String string
	 */
	String cron();

	/**
	 * 作业分片总
	 *
	 * @return int int
	 */
	int shardingTotalCount() default 1;

	/**
	 * 分片序列号和个性化参数对照表.
	 * 分片序列号和参数用等号分隔, 多个键值对用逗号分隔. 类似map.
	 * 分片序列号从0开始, 不可大于或等于作业分片总数.
	 * 如:
	 * 0=a,1=b,2=c
	 *
	 * @return String string
	 */
	String shardingItemParameters() default "";

	/**
	 * 作业自定义参数.
	 * 作业自定义参数，可通过传递该参数为作业调度的业务方法传参，用于实现带参数的作业
	 * 例：每次获取的数据量、作业实例从数据库读取的主键等
	 *
	 * @return String string
	 */
	String jobParameter() default "";

	/**
	 * 是否开启任务执行失效转移，开启表示如果作业在一次任务执行中途宕机，
	 * 允许将该次未完成的任务在另一作业节点上补偿执行
	 *
	 * @return boolean boolean
	 */
	boolean failover() default false;

	/**
	 * 是否开启错过任务重新执行
	 *
	 * @return boolean boolean
	 */
	boolean misfire() default true;

	/** ... **/
}

Then the first step, the listener listens topicmessage to perform delete operations:MqMessageServiceImpl.java

@Override
	@Transactional(rollbackFor = Exception.class)
	public void deleteMqMessage(final int shardingTotalCount, final int shardingItem, final String tags) {
		// 分页参数每页5000条
		int pageSize = 1000;
		int messageType;
		// 1. 删除所有生产者发送成功的消息数据
		if (AliyunMqTopicConstants.MqTagEnum.DELETE_PRODUCER_MESSAGE.getTag().equals(tags)) {
			messageType = MqMessageTypeEnum.PRODUCER_MESSAGE.messageType();
		// 2. 删除所有订阅者消费成功的消息数据
		} else {
			messageType = MqMessageTypeEnum.CONSUMER_MESSAGE.messageType();
		}

		int totalCount = mqMessageDataMapper.getBefore7DayTotalCount(shardingTotalCount, shardingItem, messageType);
		if (totalCount == 0) {
			return;
		}
		// 分页参数, 总页数
		int pageNum = (totalCount - 1) / pageSize + 1;

		for (int currentPage = 1; currentPage < pageNum; currentPage++) {
			List<Long> idList = mqMessageDataMapper.getIdListBefore7Day(shardingTotalCount, shardingItem, messageType, currentPage, pageSize);
			mqMessageDataMapper.batchDeleteByIdList(idList);
		}
	}

2.3 zk registry data structure

After the job starts, zookeeperthe registry creates a job-related data node:

Registration center in the namespace definition, create job name node com.paascloud.elastic.demo.SimpleJobDemo , used to distinguish between different jobs, so the job can not be modified once created job name, if you modify the name as a new job .
Job name and the node comprises 5个data sub-node, respectively config, instances, sharding, serversand leader.

Reference specific meaning: http://elasticjob.io/docs/elastic-job-lite/03-design/lite-design/

2.4 elastic-job-console operation and maintenance platform

Abstract: Elastic-Job After the deployment of clusters in a production environment, how to monitor the timing of the operation and maintenance tasks online ran it?

If deployed on a large scale server clusters production of integrated Elastic-Jobbusiness project, but there is no corresponding operation and maintenance monitoring tools to monitor the status of scheduled tasks execution and dynamically modify the scheduled task execution time, modify the configuration database or have to manually update configuration file, then it will give the operation and maintenance and R & D engineers add a lot of trouble. Used Quartzcluster program students should have had the same feelings, modify the scheduled task execution time configuration and monitoring tasks of the state are too much trouble, they want a fully functional operation and maintenance monitoring platform have their own specially to develop. Fortunately, Elastic-Jobthe open source community very early consideration to the issue, released early in the project that is providing a function relatively complete Elastic-Joboperation and maintenance monitoring consoleplatform.