Cut the nonsense, look at the code first
def add_end(L=[]):
L.append('END')
return L
Does this python code look wrong? It seems that there is no problem. L is an input parameter in the form of a list. If it is not passed, there is an empty list as the default value. The content of L is added to the business body, and the added L object is returned. business is simple
But it is such a seemingly harmless code that made a website suffer for a month, and lost a large number of users at the same time. Let us take a look at the memories of those who experienced it.
Digg’s v4 launch: an optimism born of necessity.
Students who do not want to read English can translate the full text or read the key content of the excerpt below:
Background: digg is a technology-based news site. Users can submit news to digg, which will be displayed on the homepage of digg through the digg mechanism. More than ten years ago, the wave of web2.0, the monthly visit volume at its peak was more than 3000W, but it declined in 2012, and it is still in operation today. This incident happened in 18 years, which made this not rich website worse
From Digg to the launch of V4 in 2018, it has been two years of continuous development from V3.5 to V4. The entire company hopes to bring Digg back to the first echelon of the Internet through V4. They even prepared celebratory champagne in advance (champagne opened at halftime, traditional performance)
They have an upgrade plan, but no rollback plan (this is the flag, keeping in mind Murphy's Law)
The upgraded website did not display as planned, but most pages failed to load because of the high load in the background. Halfway through, the Digg team observed a high load on the database (Cassandra) and cache (memcache), and expanded the capacity once, but the problem was still not completely resolved. Especially the most important MyNews page in V4, which is the most eye-catching feature in this version, and the user's default page, but this page keeps reporting errors
The Digg team later changed the initial page to TopNews to ensure that users would not see an error page when logging in, but the problem was still not resolved
Two days later, the Digg team migrated the data from Cassandra to redis to improve the access performance of the database. The problem has been alleviated, but the service still needs to be restarted every 4 hours
A month later, the problem was finally found out. Digg's API service is a Python service, similar to the gateway layer. As a gateway, user information needs to be queried. The parameter form of the user information query function is the above list form (of course, the specific We don’t know what the code looks like, we only know the form), it is this code form that caused this month’s problem
OK, let's execute the above function once
>>> add_end()
['END']
It looks okay, let’s execute another one with parameters
>>> add_end([1, 2, 3])
[1, 2, 3, 'END']
It’s okay to look at it, let’s execute it again without parameters
>>> add_end()
['END', 'END']
>>> add_end()
['END', 'END', 'END']
Um? why? The default parameter is [], but the function seems to "remember" the list after adding 'END' last time
The reason is explained as follows:
When the Python function is defined, the value of the default parameter L is calculated, that is, [], because the default parameter L is also a variable, which points to the object [], each time the function is called, if the content of L is changed, then The next time it is called, the content of the default parameter will change, and it will no longer be [] when the function is defined
Therefore, when a large number of users visit Digg, the default list in L will become larger and larger, and with concurrent access, this is a polynomial growth of more than two degrees, which will soon fill up the memory and cause the service to be unavailable
Code and explanation reference, which also talks about how to solve it: Liao Xuefeng: function parameters
Such problems are indeed very hidden, and sometimes counter-intuitive (maybe for those who are not familiar with them). At this time, it is especially necessary to use some static code checking tools proficiently. Many of these pitfalls that others have stepped on have been summarized in these tools. Even if he cannot be sure that this must be a BUG, he will give the developer some hints about dangerous coding methods.