Each article has its own part of the story, not the story of the article is not the soul of the article. And I am the soul ferryman.
Moumou hero, here to disclose the names, Zhang called it. Zhang took office in a small team of Internet entrepreneurs.
Jobs is back-end Java development, so the overall business code and deal with inevitable.
Before there is a search-related needs, and the number of relatively large amount can be considered, on the use of ElasticSearch to do the search. Because of the time to catch the first edition of comparison, do rough. More to the back to find the code more difficult to write anymore, mainly in updating the index data did not handle the scene, only today's story.
Basics
Spring Event
Spring event is the observer design pattern, after the end of a task requires notification task to the next step, you can use the event to drive.
To use the event mechanism in Spring is as follows:
- Custom event objects, inheritance ApplicationEvent
- Custom event listeners to realize ApplicationListener or comment on methods to achieve by listening @EventListener
- Custom publisher, () event by publishing applicationContext.publishEvent
Spring Event in many open source frameworks in both use cases, such as Spring Cloud in Eureka which will have to use
event package
Custom Event
Published Event
Guava EventBus
EventBus event handling mechanism of Guava, almost at the level of use and Spring Event. Do not do too much to explain here, mainly about Spring Event today.
Business Background
All data will have a task to synchronize the timing of data to ElasticSearch in business directly back ElasticSearch query data from the caller.
The reason why all data stored ElasticSearch is to facilitate, if only stores the search field, then search out the need to go after other information assembled a database query.
Since all data is stored in ElasticSearch, so when the data changed, we need to refresh the data ElasticSearch in, this is the core of our background article today.
We assume that the data is ElasticSearch article information, that is, we often see technical articles, this article is stored in the traffic, thumbs amount, the amount of comments and other information.
When these actions occur, we need to update the data ElasticSearch job, our default action is to update data in the database, ElasticSearch is a task to synchronize the timing, synchronization will cycle, do not update milliseconds.
Implementation - stubborn bronze
Bronze is stubborn in each involve local data changes, the calling code to manual data refresh operation, the drawbacks that have to go to every place calls, this is the simplest scenario, there are complex business scenario, a business operation possible refresh will involve a lot of data, which is the need to call many times, simulation code is as follows:
// 浏览
public void visit() {
articleIndexService.reIndex(articleId);
XXXIndexService.reIndex(articleId);
........
}
// 评论
public void comment() {
articleIndexService.reIndex(articleId);
}
复制代码Implementation - Silver Order
Malpractice stubborn bronze that is not decoupled, and is a synchronous call, time will be longer if the transaction in the transaction. So we need a program to perform asynchronous logic rebuild the index.
After all the heated discussion, and the project is also based on Spring Boot-based, so I chose Spring Event as asynchronous scheme.
The definition of a rebuilding article index Event, code is as follows:
public class ArticleReIndexEvent extends ApplicationEvent {
private String id;
public ArticleReIndexEvent(Object source, String id) {
super(source);
this.id = id;
}
public String getId() {
return id;
}
}
复制代码Then write a EventListener to monitor events, business logic processing code is as follows:
@Component
public class MyEventListener {
@EventListener
public void onEvent(ArticleReIndexEvent event) {
System.out.println(event.getId());
}
}
复制代码Local use only need to publish an Event can be, this action is synchronous by default, if we want this operation will not clog, become asynchronous only need to add a note in @EventListener @Async above.
// 浏览
public void visit() {
applicationContext.publishEvent(new ArticleReIndexEvent(this, articleId));
applicationContext.publishEvent(new XXXReIndexEvent(this, articleId));
}
// 评论
public void comment() {
applicationContext.publishEvent(new ArticleReIndexEvent(this, articleId));
}
复制代码Implementation - Gold Glory
Silver program order in the code level is indeed decoupled, but the person who posted the event of interest has too many points, that is, I changed the data in a table, I have to know what the index will use this data table I shall have to send out the relevant events, such data will be asynchronous refresh.
When the business complex or have new colleagues, not so understanding of the business, are they not know that I changed this data impact on those other indexes, so this program still has room for optimization.
Glory Gold program is all the events are united in one, then in Riga event attribute to distinguish where the modified data is yes. Each data needs to be synchronized to change the index has its own listeners, listening to this unified event, so I just need to send an event for publishers who tell you, I am here to change the data, you do not want to consume, to Do not update the index I do not care.
Define a data table modified occurrence of an event, as follows:
public class TableUpdateEvent extends ApplicationEvent {
private String table;
private String id;
public TableUpdateEvent(Object source, String id, String table) {
super(source);
this.id = id;
this.table = table;
}
public String getId() {
return id;
}
public String getTable() {
return table;
}
}
复制代码Then each index needs to consume this event, only need to focus on the source of the data in the table this index has not changed, if there are changes to refresh the index.
For example, the index data are article A table over, so long as the article data in the table has been changed at the index have to do A corresponding process, so the index A listener need only concern table has no modification to article .
@Component
public class MyEventListener {
private List<String> consumerTables = Arrays.asList("article");
@Async
@EventListener
public void onEvent(TableUpdateEvent event) {
System.out.println(event.getId() + "\t" + event.getTable());
if (consumerTables.contains(event.getTable())) {
System.out.println("消费自己关注的表数据变动，然后处理。。。");
}
}
}
复制代码Such as data indexing B is coming from comment and comment_reply two tables, so long as the comment and the two tables of data comment_reply change occurs, need to do B index corresponding process, so the index B listeners only need to focus comment and comment_reply two tables have no modifications.
@Component
public class MyEventListener2 {
private List<String> consumerTables = Arrays.asList("comment", "comment_replay");
@Async
@EventListener
public void onEvent(TableUpdateEvent event) {
System.out.println(event.getId() + "\t" + event.getTable());
if (consumerTables.contains(event.getTable())) {
System.out.println("消费自己关注的表数据变动，然后处理。。。");
}
}
}
复制代码Implementation - Premier Platinum
Glory Gold program was perfect, not to decouple the code that the user focus less, and less error prone.
But there is a fatal problem is that all relates to a method of modifying business, have to manually send out an event code is decoupled from the scene, it still remained a little flaw, so at least there is still a line of code to send events.
Premier Platinum program will be fully decoupled, time does not need to manually write the code to send events. We will send events by subscribing to unify MySql's binlog.
binlog is a binary log of the MySQL database, SQL statements used to record the user information database operations, MySQL master-slave synchronization is also based binlog to achieve, for us this scene heterogeneous data appropriate.
There are many ways binlog subscription, open source framework are generally used to achieve the canal.
canal：github.com/alibaba/can…
If you buy cloud database, as there dts ALI cloud data subscription services like canal.
After embodiment of FIG follows:
Implementation - Diamond Eternity
Nothing is eternal scheme and architecture, along with the change and the evolution of the business, in line with the current needs of the business is crucial. The more you consider something back, after all, is the last to upgrade to the most powerful king, ha ha.
Interested can follow my micro letter public number ape world , the first time to read more technical articles. I also have some GitHub open source code github.com/yinjihuan