Produced | OSC Open Source Community (ID: oschina2013)
Recently, Mozilla has detected a large number of crashes in Firefox, mainly on users who use Linux systems, especially those based on older versions of Debian.
After investigation, Mozilla finally found that the problem was not caused by Firefox itself, but by the Linux kernel and Google's JavaScript code.
These crashes happened quite suddenly, when Mozilla first detected that thousands of users of a Debian distribution called Huayra were affected, specifically Huayra 5 (based on Debian 10). Subsequent ongoing testing found that this problem affects almost all distributions based on older versions of Debian.
Crash events do not happen randomly, but are 100% reproducible. Firefox crashes whenever a user searches for an image on Google, a problem that affects all versions of the Firefox browser -- from the latest to very old versions.
Combining the above two conditions, Mozilla researchers determined that the problem was not caused by Firefox, but by Google and the old version of Linux, and began to analyze the cause of the problem.
Mozilla then began analyzing Firefox's behavior when it crashed and found that the crash occurred during stack probing. The JIT touched the area where variables are saved for the next JavaScript call, and somehow caused an overflow.
The first oddity is that Mozilla discovered that Google recently made changes to its image search page, which now has a JS function, and Google assigns 20,000 variables in this single function. Further analysis revealed that this function may be code generated by AI.
Although such a problem was discovered, in theory Firefox should still not crash, because Linux will automatically expand the stack, and the Mozilla team has reserved enough space, and then they confirmed it by looking at the memory of the affected process at this point.
Before doing this, we did a stack inspection and verified that the amount of extra stack memory we allocated would not overflow the native stack limit we had set for ourselves. So there seems to be a divide between our self-imposed limits and operating system limits. This is somewhat distro-dependent, but confusing: for example, it affects Debian 10 but not Debian 11.
The Mozilla team then focused the inspection on the Linux kernel, and it turned out that the Linux kernel used to have a check that prevented access to the stack from being too far from the stack pointer. In particular, accesses beyond 64KiB+256 bytes will crash instead of expanding the stack. This problem was fixed in Linux 4.20, so users of newer distributions are not affected.
According to testing, Google seems to have fixed this issue in image search, but Mozilla is still working on a solution to see if it can be solved once and for all for users who are still using the old system, so that the same situation does not happen in the future.
Related link: https://hg.mozilla.org/mozilla-central/rev/304d01f5488b
------
We have created a high-quality technical exchange group. When you are with excellent people, you will become excellent yourself. Hurry up and click to join the group and enjoy the joy of growing together. In addition, if you want to change jobs recently, I spent 2 weeks a year ago collecting a wave of face-to-face experience from big factories. If you plan to change jobs after the festival, you can click here to claim them !
recommended reading
Grafana 10 is officially released: cooler, more efficient, easier to analyze and collaborate!
How to optimize the performance of Elasticsearch 8.X? It is enough to read this article!
··································
Hello, I am DD, a programmer. I have been developing a veteran driver for 10 years, MVP of Alibaba Cloud, TVP of Tencent Cloud. From general development to architect to partner. Along the way, my deepest feeling is that we must keep learning and pay attention to the frontier. As long as you can persevere, think more, complain less, and work hard, it will be easy to overtake on corners! So don't ask me if it's too late to do what I do now. If you are optimistic about something, you must persevere to see hope, not to persevere only when you see hope. Believe me, as long as you stick to it, you will be better than now! If you have no direction yet, you can follow me first, and I will often share some cutting-edge information here to help you accumulate capital for cornering and overtaking.