About problems encountered in the project - Import optimization ------- ******************************* ************************

1. The cause of the incident:

JD import data project is too slow, so help us to see the real reason Mr. Wu found vehicle ID data exists more than ten million pieces found after one-time memory, from memory and then compare whether there is, in this way

It causes the server to get stuck, will request a timeout.

2. Solution:

Data over tens of thousands, have not found all of the comparison into memory, this approach will lead to service micro-hang (you may want to restart the micro-services), mainly because the imported data to 500, and the amount of data for the model ID 12W

Bar, all the way models ID or get to go to the database each cycle.

Guess you like

Origin www.cnblogs.com/yunliu0603/p/11097147.html