How to improve the ability to write code

Author | Bi Xuan
Source | Alibaba Cloud Native Official Account

For programmers, I always believe that code is the key to display ability. It is easy to see the difference between code written by a good programmer and code written by an ordinary programmer. Code is the hard power and business card of programmers. Show, how to improve the ability to write code is always a key topic, but unfortunately this article is not actually about specific steps, silver bullet methods, martial arts secrets, etc. This article talks about writing to me in my own impression Four experiences with a relatively large improvement in code ability may be for reference.

The first paragraph: For the first time experience the challenge of a billion-level system every day

In 2008, the second version of HSF was launched on Taobao's most important trading center at the time. On the day of launch, the Taobao website was visited very slowly and transaction pages could hardly be opened. Finally, it was restored by offline HSF.

After going offline, I started to check the problem. The second version of HSF is based on jboss-remoting. In the current version, the timeout period of remote synchronization calls is 60s, which is hard-coded in the code, and the called service does have Some phenomena that took more than 10 seconds appeared, causing the thread pool of web applications to process web requests to be gradually occupied by these slow requests, the requests accumulate, and finally the page opening is very slow.

After investigating the reasons, I decided to rewrite the entire HSF communication based on Mina at that time. The two months of rewriting have greatly improved my ability to write code, whether it is in-depth learning of network IO processing or In-depth learning on high-concurrency systems, now the way to think about learning is to look through various types of network IO popular science materials, and then read the source code of Mina and Java network IO. The study of concurrency is mainly based on the classic book "Java Concurrent Programming Practice", and reading the code in the Java JUC, the biggest difference between this period of learning compared to the previous Think in Java is that after learning, put it into practice, with the new rewrite of HSF The release of the version is basically a real mastery of the code capabilities of these parts.

In addition to the improvement of code capabilities, the other biggest lesson learned is that for a system with a billion-level and long-running system, many seemingly small problems will definitely become serious problems, which is why writing high-concurrency systems The difficulty of this requires that you have to be very clear about the code you write and the implementation of the various APIs called by your own code, so as to truly ensure the robustness of the final code.

Second paragraph: The story of the folk "fire brigade"

The second paragraph of my experience of improving my own code writing ability was particularly in the days when I was in the civilian "fire brigade". Taobao had a lot of failures in 2009, but there is no standard system and organization for handling failures, which leads to many problems. There was no one to deal with the faults, or the processing efficiency was not high, so at that time, a classmate of the operation and maintenance team pulled some people to form a group, the name of the group is Taobao Fire Brigade, to deal with various faults in Taobao , I happened to also join this group. There is another super technical god recognized by Ali: Duolong.

When I first saw various faults, I didn’t know how to do it. Handling faults usually requires not only the ability to write code, but also a certain grasp of the overall picture of a system, such as an article in the past few years. Hot articles, when you click on the article about what happened behind the search, you need to be particularly familiar with the processing flow of a system. This is very important when dealing with faults. After you have the fault, it is very The important thing is to control the details of the code operation mechanism in this link. At this time, various tools are usually very important, which can effectively help you know what is happening, such as top -H at the system level, java level Btrace and so on, can let you locate the problem according to the running situation.

During this period of time, I feel that my improvement is relying on a lot of practice. There are indeed a lot of faults. At the beginning, I depended on how people deal with it. I learned from Duolong. Then I tried to solve some faults by myself. After that, my proficiency gradually increased. In addition to improving the ability to solve faults, because I have seen a lot of faults caused by the code level, it is very helpful to how to better ensure robustness when writing code to avoid faults. It helps. For example, I have seen many cases where a large number of threads are created due to abuse of the thread pool, and the thread cannot be created eventually. I will understand that I must be very clear in the scenario of using the thread pool to control the maximum number, including accumulation For example, I have seen N many OOM cases caused by self-increasing capacity data structures, and I will understand that when writing code, it cannot be assumed that the data structure will not grow to a super large size, so I don’t do anything. The case of protection, what I understand at this time is that it is not difficult to write a code that can work and meet the requirements, but it is really not easy to write a code that can run stably for a long time under various conditions. I think this is a profession The biggest difference between programmers who write business systems and just write programs for fun.

The third paragraph: rewrite the communication framework

In 2010, I left the middleware team and started working on HBase. At that time, the communication in HBase was still implemented in a very simple way. I thought about porting what I used in HSF to HBase at this time. Duolong just used c to write a general communication framework libeasy for all kinds of c applications, so there was a test. I remember the first test result that the high concurrency of the communication framework in the original HSF is different from that of libeasy. It's extremely huge. Duolong and I discussed how he realized it. I saw if I could learn and change it in the Java version, so I had this experience of rewriting the communication framework.

I thought that the previous years of writing HSF should be regarded as a good mastery of the code-related abilities of the communication framework. During the process of rewriting with Duolong, I found that the gap is still very large, Duolong I have taught a lot of details. The core of the NIO-based communication framework is to use very few IO threads to process IO events (too many is useless, because some parts can only be serialized), so how to use these efficiently An IO thread is very critical. It is necessary to minimize the processing of some irrelevant actions by these IO threads. Another point is to minimize the switching between IO threads and business processing threads. For example, it is common to batch multiple requests in a stream later. Throw it to the business processing thread once.

This experience is very helpful for me to have a deeper grasp of the overall details of the code logic. This is very important for writing systems with high requirements. After all, for a super large-scale system, 1% improvement is still Considerable.

The fourth paragraph: learning JVM

I used to deal with a lot of faults. For a while, I began to share with my colleagues how to deal with faults. Later, I found that some problems were not clear to me or how to deal with them. I had to learn JVM in depth. I can't figure it out at all, and I don't know where to start the JVM code.

Fortunately, I came across a classmate who has the same hobby and is much stronger than me, Sakya, who is usually called R in the circle. Sakya and I have made appointments to watch the JVM code in the company for several weekends. Sakya gave me advice. It's finally getting started. I know how to read it, and two people look at the code together, share and discuss with each other, the efficiency is very high.

With this experience, coupled with continuing to deal with some faults, I have basically gradually gained a more understanding of the code implementation of the JVM, and finally I can better understand it when I do fault sharing and problem solving. Knowing the reason, it is also very helpful for the ability to deal with faults and the ability to write code. For example, you will better understand what the so-called GC-friendly code means before, and you will have a deeper feeling. In fact, Java code is usually not written too badly, because the JVM will do a lot of optimizations as much as possible during the runtime, and pull it to an average line, but it is very difficult to write, because it needs to understand JVM, understand the OS under JVM.

to sum up

In fact, I can’t sum up anything, because everyone’s environment is different, and there are methods suitable for their improvement. Looking at my own experience, I think:

• If the environment is not available, give yourself a challenge proposition. For example, if you want to learn high-concurrency communication, you can try to write a pk that compares with others and does performance. This usually improves a lot, and you have to learn GC , You can try to give yourself a few questions to control the behavior of GC, etc. If the environment has it, it will indeed be more beneficial.

• Together with excellent programmers, I have learned a lot from Duolong and Sakya, and learned a lot from many excellent open source codes, such as Netty and OpenJDK, so it is also a good idea to participate in some excellent open source projects. A good way to improve, reading excellent books (such as the Java Concurrent Programming Practice in Concurrency, Oracle JRockit: The Definitive Guide in JVM, in-depth understanding of the Java Virtual Machine, etc.) is also a way to learn from excellent programmers Great way.

• Try to solve problems/faults a lot. This is definitely a very good way to improve your code synthesis ability. If there are few opportunities in your own work, there are a lot of them on the Internet, such as stackoverflow, which are good practice areas.

In the end, I still want to say that as a programmer's hard business card, code ability is always the most effective thing to distinguish programmer ability. I think the phrase "talk is cheap, show me the code" is always true.