1. serious talk about: service hang.
It was from the storage mechanism ActiveMQ. Under normal circumstances, the non-persistent messages are in memory, persistent message is stored in a file, which limits the maximum storage configuration <systemUsage> node configuration file. However, non-persistent messages accumulate to a certain extent, the memory ran out of time, ActiveMQ will be in memory of the non-persistent messages are written to a temporary file to free up memory. Although are saved to the file, but the difference between it and the message is persistent, persistent message after the restart to recover from the file, the non-persistent temporary files will be deleted directly.
So what if the file has reached the maximum limit increases configuration time will occur? I did the following experiment:
Set around 2G persistent file limit, the mass production of persistent messages until the file reaches the maximum limit, this time blocking producers, but consumers may be connected properly and consume messages, and so consumed part of the message, delete the file and then make room, producers but also continue to send the message, the service automatically return to normal.
Set around a temporary 2G file limit, the mass production of non-persistent messages and write temporary files when it reaches the maximum limit, blocking producers, consumers may be connected properly but can not consume messages, or otherwise slow consumer spending, consumer suddenly stopped. The entire system can be connected, but can not provide the service, it is so hung up.
The specific reasons unknown, Solution: Try not to use non-persistent messages, have to use it, will be limited as much as possible to transfer large temporary files.
2. lost messages.
This unusually java from the start of java.net.SocketException. Simply put, it is when the sender sends a bunch of network data, and then call Close to close the connection. These data are sent in the cache of the recipient, the recipient to read the data if you call the method still read from the cache, even though the other side has closed the connection. But when the receiver attempts to send data at this time since the connection has been closed, so the exception will occur, this is well understood. But note that, when the SocketException occurs, the data in the buffer zone had also set aside, at this time again the recipient calls the read method to read data in the cache, it will report Software caused connection abort: recv failed error.
Capture by that, every 10 seconds the ActiveMQ will send a heartbeat packet, the server is to send the heartbeat packet to the client, the client is determined to death did not die. If you read the first article above, you know that the accumulation of non-persistent messages to a certain extent be written to a file, the writing process will block all actions and will last 20-30 seconds, and with the increased memory increases. When the client finished his message calling connection.close (), the server will look for answers to close the connection, if more than 15 seconds did not answer directly call close socket layer close tcp connected. At this time the message sent by the client is actually still waiting to be processed in the server's cache, but since the server heartbeat packets, leading to an abnormal java.net.SocketException, the data cache set aside, the message did not deal with the lost.
Solution: use persistent messages, or non-persistent messages do not accumulate in time, or start a transaction, start a transaction, commit () method will be responsible to wait for the server to return, it will not close the connection that caused the message is lost.
3. Persistent messages very slowly.
By default, non-persistent messages are sent asynchronously, persistent messages are sent synchronously, encountered little slower hard drive, fast to send messages is intolerable. But in the case of open transactions, messages are sent asynchronously, efficiency will be two orders of magnitude improvement. So when sending persistent messages, be sure to activate transaction mode. In fact, when the transaction is sent also recommended to open a non-persistent messages, because there will not affect performance.
4. non-uniform consumption of the message.
Sometimes after sending some messages, open two consumers to process the message. You will find a consumer handles all the messages and one that consumers do not receive the message. The reason is that ActiveMQ the prefetch mechanism. When consumers go to get the message, not one by one to get, but a number of one-time acquisition, the default is 1000. These pre-acquisition news, not confirmed prior to consumption, the management console can still see these messages, but not reassigned to other consumers, this time the status of these messages should be considered "non-consumption has been assigned" If the message is finally consumption, it will be deleted on the server side, if consumers collapse, these messages will be reassigned to new customers. But if the consumer does not consume confirmation, not collapse, and that these messages will never lie consumer buffer zone can not handle. More often the case, the consumption of these messages very time consuming, you opened 10 consumer to deal with, and found that only one machine Hangchihangchi process, in addition to nine losers quit.
The dead letter queue.
If you want to post a message in the process fails, the server is not deleted, can also be treated by other consumers or retry can be closed AUTO_ACKNOWLEDGE, the ack referred to the program with their own. If you use that AUTO_ACKNOWLEDGE, when the message is to be confirmed, there is a method to confirm the news did not stop? Have!
Consumer news there are two ways, one is calling consumer.receive () method, which will block until and returns a message. In this case, after the message is returned to the caller is automatically method was confirmed. Another method is to use a callback function listener, when the message arrives, the onMessage method will be called listener interface. In this case, after the onMessage method completes, the message will be confirmed in this case as long as the method throws an exception, the message will not be acknowledged. So the question is, if a message can not be processed and will be returned to the server reassigned, if there is only one consumer, the message will be re-acquire and re-throw an exception. Even if there are multiple consumers, often on a news server can not handle, still can not be processed on another server. Is it such a return - get - given the cycle of death yet?
After retry six times, ActiveMQ believe this message is "toxic", will be thrown into the dead-letter message queue. If your message is gone, look inside to ActiveMQ.DLQ, maybe just lying there.
----------------
Disclaimer: This article is CSDN bloggers 'bright _' original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source and this link statement.
Original link: https: //blog.csdn.net/soulmate_p/article/details/81380937