Message queue processing scheme

Basic introduction:
agent: load the life cycle management of the container, and the health check of the container, notify the master when the container process is down and start
master: the message received from the agent is placed in a queue, and a thread loops from the queue Get the appid, and send the latest application instance list to the router (because many business scenarios will trigger the router update, and the order must be guaranteed, so a queue is used, and the put and the take are separated)
router:router receives The message from the master updates the instance list of the application to ensure that requests are forwarded to available instances.

Here comes the problem:
some application processing times out, resulting in frequent failures and successes of health checks. Every change will notify the master, and the master will notify the router. Too frequent state changes will cause the router to fail to process.

Solution 1:
The appid and addtime are already placed in the queue, and a map is added, key=appid, value=the time of the last time put into the queue.
After each time the appid is taken out from the queue, the addtime is compared with the time of the appid in the map, < Do not process, >= notify the router, so as to reduce the frequency of notifications and reduce the pressure on the router. Problem: If the state of the container changes frequently, the time taken from the queue is always less than the time in the map, and it will never be notified.

Solution 2:
Change the queue to the priority queue PriorityQueue, so that all the appid messages are together, the next appid taken from the queue is different from the previous one, notify the rouer, and pass the same. Disadvantage: Not guaranteed to be notified first of changes.


Solution three:
Using queue and set, each time you put a queue, first determine whether the set has an appid, a pass, and no set and queue are added at the same time. This can filter out duplicates when they are placed. In order to ensure that the processing is too fast and the effect of filtering repetition is not good, each time an element in the queue is processed, sleep for 1 second. This sleep1s is acceptable for business. This solution should be no problem for the current concurrency. Be sure to pay attention to locking when placing sets and queues, and deleting sets and queues to prevent high concurrency problems

. Considering this, we have adopted the third solution.