Didi Taxi's Architecture Changes

http://www.csdn.net/article/2015-06-25/2825058-didi

Established in 2012, Didi Taxi was written in only three years: covering 300 cities, the number of users increased from 22 million To 150 million, the monthly active users have increased by more than 600 times (on Christmas Eve in 2014, the number of people using Didi taxis in China exceeded 30 million), and the success rate of taxis is higher than 90%... These will always be remembered in the mobile Internet. Fantastic record in history. What is not known is the architecture that supports such a large number of users of Didi Taxi, as well as the technical partners who have responded to the challenges without sleepless nights countless times.

Zhang Bo, the CTO of Didi Taxi, known as "Master Bo", described it this way: "For the past three years, we have been 'fighting' every day. Every day when I open my eyes, I have to think about life and death. Keep it, whoever wins. We race with technology and time. After the speed of life and death, we are happy to find that we have accumulated a lot of valuable experience and cultivated a large number of excellent practical talents."



Didi Taxi CTO Zhang Bo is often called by friends

The structural changes behind the three

products , which we affectionately call “Master Bo” , Didi Dache was founded to solve the problem of information asymmetry between drivers and passengers, and to break down information barriers through mobile Internet and smartphones. From taxi to private car to free ride, behind the three products of Didi Taxi are architectural challenges and system changes.

CSDN: From a product perspective, how many stages does Didi Taxi develop? What are your priorities as CTO?

Zhang Bo: Didi Taxi was founded to solve the problem of information asymmetry between drivers and passengers. Passengers and drivers are often unable to achieve information connection through traditional means. One is unable to get a car or is rejected; the other is empty and time-consuming, labor-intensive and resource-consuming. This is the barrier of information, which can be completely broken through the mobile Internet and smart phones. In the follow-up operation, we found that even if the empty taxis are reduced to O, it cannot meet the travel demand of passengers during the morning and evening peak hours. Taking Beijing as an example, there are 106,000 taxis, but the population has 20 million permanent residents and 10 million floating population. To meet such demand, it is necessary to concentrate social efforts and dispatch social resources. This is the second product of Didi Taxi - Didi Chuxing. At present, the number of special car orders has approached the number of taxi orders, but the number of car rental orders has not declined. Obviously, it is the new incremental market that is leveraging. The third product is a hitchhiker, and the scene of satisfaction is very representative. Through the big data analysis system of Didi, it is found that many people live and work close to each other, have similar income levels, and have similar industry attributes. These people often use taxis or special cars. Low-cost travel, or saving social resources, are valuable. What's more interesting is that with the addition of social attributes, people can make new friends and travel becomes more interesting. As CTO, my work has three main parts: the first is product, the second is engineering technology, and the third is big data.

CSDN: The three products represent the different development stages of Didi Taxi. Believe that the architecture behind each stage has obvious characteristics?

Zhang Bo: When Didi Taxi was first established in 2012, the traffic was very small and no architecture was needed. Two servers could solve all problems. With the rapid development, the performance bottleneck was first discovered in the "subsidy war" in early 2014. Our order volume increased by 50 times within a week. The forecast at the time was for a 10% increase. 500% to 10%, the results can be imagined. Network and storage failures continued, and Webserver and MySQL also experienced frequent problems. The challenge the team faced was enormous. What's more stressful is that relying on traditional procurement machines to achieve expansion is obviously completely unable to meet business needs. Through analysis and comparison, we finally decided to move to Tencent Cloud as a whole. This is the first very large adjustment of the architecture in the face of high concurrency and massive data challenges. However, the relocation was not all smooth sailing, and the code needed a lot of refactoring to solve technical single-point problems.

The second architectural change was when the second product, Didi Chuxing, was originally designed to support one product, but now the architecture needs to support multiple products at the same time. There are similarities and differences between products. To this end, we have set up a technical architecture department to sink general-purpose services into the architecture, avoid duplication of wheels, and put personalized services on the business layer to realize service development. For example, payment, account system, computing, storage, etc. are all general-purpose services that can be placed in the architecture. This architectural refactoring is relatively successful, and it is precisely this kind of organizational and technical changes, so when the third product hits the road, and when we release more new products soon, we can get smooth support and support. Serve.

CSDN: The industry was deeply impressed by the challenges that Didi Taxi encountered at the time. What kind of technical cooperation did you have with Tencent Cloud at that time?

Zhang Bo: At that time, Didi Dache had already accepted investment from Tencent, and it was naturally closer to Tencent Cloud. Moreover, Tencent Cloud has also experienced the high concurrency test of massive users of QQ and WeChat, such as the Spring Festival red envelopes. There is no doubt about its stability and experience. The cooperation with Tencent Cloud has three directions in terms of technology: the first is security. During the development of Didi Taxi, it has suffered many hacker attacks, so security is very important to us, and Tencent Cloud's Dayu security system can be very Help us well; followed by network services such as CDN, some technologies are difficult for startups to achieve. For example, network layer optimization, when a user's mobile phone sends an order, it needs to access the nearest base station and find the best quality link. These can only be achieved by companies that invest a lot of manpower and material resources. And these services are extremely important to the user experience, so start-ups can go faster and better through cloud services; the third is to save a lot of manpower in basic platform construction, such as hardware procurement, hardware operation and maintenance, etc. , but can focus more energy on the application layer and business layer, as well as other more valuable data analysis and other new technologies. In fact, at the time of the subsidy war, as a start-up company that was only one and a half years old, it was simply unbearable in the face of such a large amount of traffic. It was also at that time that Tencent's senior architect was directly stationed there and established a brotherly friendship with us.

CSDN: Tencent Cloud's new products are released very quickly. What new services will Didi Taxi use in the future?

Zhang Bo: Didi Taxi and Tencent Cloud have in-depth cooperation. In addition to security, network, and operation and maintenance, there are also overall system optimization, partial tuning, and new product practice. Didi Taxi will even go deep into Tencent’s new businesses, such as the credit reporting system. The number of no-shows and the frequency of no-shows of passengers/drivers collected through the Didi taxi platform will become a data source. Another example is in the future ride-hailing, based on big data analysis, how to give priority to matching an Internet goddess for an Internet diaosi, these interesting scenarios require data exchange at the bottom. More new technologies and new services need to be applied to Tencent Cloud. Both of us are very open and hope to create greater value for the society.

CSDN: After the Ctrip and eLong incidents, CTOs paid more attention to data security. How does Didi Taxi plan in terms of safety?

Zhang Bo: At present, most of Didi Taxi's services are on the cloud, and some services are operated and maintained by renting Tencent's computer room. The overall architecture is a hybrid cloud model. Security is private to every company. Now there are strict security controls inside, and the outside is mainly through Tencent Cloud, such as anti-hacking attacks, such as DDos attacks. There is no end to security protection, and there is still a lot of work we need to do.

Open source optimization to build technical core competitiveness

DiDi Dache is building its own technical core competitiveness in the areas of Dimi system, user portrait system, precision marketing, intelligent matching, demand forecasting system and transportation capacity forecasting system. On the road of research and development, big data and machine learning are not only the heart of Didi taxi products, but also the heart of Didi taxi business.

CSDN: In 2015, Didi's big data technology gradually deepened in the aspects of Dimi system, user portrait system, precision marketing, intelligent matching, demand forecasting system and transportation capacity forecasting system. So what is the overall idea of ​​R&D?

Zhang Bo: Yes. Big data is the heart of Didi Taxi. Not only the heart of Didi taxi products, but also the heart of Didi taxi business. The basic principle of our research and development is to find ways to match passengers and drivers, meet their needs and ensure their experience. For example, at a certain moment in Zhongguancun, there are many orders and there are many drivers around. The decision we have to make is: send the order to the right driver. Because the driver can only hear one of the simultaneous burst orders at any one time. Therefore, if the matching is accurate, then the recommendation algorithm should be accurate, the matching efficiency should be high, the calculation should be fast, and the push should be timely. This is not enough. Before we push an order to this driver, we should first predict his level of interest in the order. The advertising field is called CTR, and we call it STR. In the posterior process, Didi can achieve 80% accuracy. Among them, not only the personal characteristics of the driver should be calculated, but also the decision-making system should be combined, such as preference, whether it is tip-sensitive, long or short-distance sensitive, time-sensitive, or direction-sensitive and other static characteristics and the positional relationship between the driver and the order, time Comprehensive analysis of dynamic characteristics such as relationships. In addition, there are subsidies, what kind of subsidies are given to passengers, what kind of subsidies are given to drivers, who is more sensitive, and how much affects more positively. Behind these strategies, big data is at work. We hope to maximize the quality and activity of users with limited resources, which cannot be achieved through human flesh, only technology can achieve this. In the process of implementation, there are great challenges to architecture, operation, and products.

CSDN: Does Didi Taxi prefer open source technology optimization or independent research and development in terms of big data?

Zhang Bo: Our business is divided into several layers. Basically open source technology is used. The basic level is the data platform, mainly for big data computing and storage, using a relatively mature open source system in the industry, Hadoop. On the base layer is a self-built data warehouse, followed by a strategy architecture. The experimental platform makes strategy iteration more agile, such as rapid feature extraction to complete model training, and a small flow test to verify the model, and then go online quickly after passing. These are all engineering developments. On top of that is machine learning. Didi Dache is now flooding nearly 10TB of data every day. It optimizes the machine learning model by continuously collecting user standard data features. For example, the order is pushed to the driver, whether the driver grabs the order or not, this is a natural mark. And through these annotations, the learning system can be optimized. At the top is the entire big data system, which supports new product development and strategic decision-making. In my opinion, many open source projects are relatively stable in evolution and have good performance. There is no need to build building blocks from scratch. The most important thing is to serve the technology to the application as soon as possible. In continuous practice, through the improvement and optimization of open source technology, and feedback to the community, we are willing to do so.

CSDN: Artificial intelligence is developing rapidly. How far has Didi come in machine learning?

Zhang Bo: We will have a heavyweight product release soon. The head of Didi's big data field is He Xiaofei, a world-class scholar in the field of machine learning. Our investment in machine learning is huge.

CSDN: What valuable experiences can be shared?

Zhang Bo: The first is how to deal with the challenges of large traffic and high concurrency. For example, how to design the access frequency of each interface, how many times the cache is accessed behind, how many times the database will be read and written, what level of instantaneous concurrency can be achieved for each service in the backend, and how much read and write concurrency should each layer of stress test withstand. , 10 machines can withstand how much read and write concurrent, etc., must be aware of. Another example is that before each marketing activity, a physical examination of the system is required to evaluate how much volume it can undertake. The second is to have very good operation and maintenance tools, which can monitor the load of each back-end service module on the line in real time, and be able to detect problems and alarm in time. The third is to set up multiple sets of contingency plans. When a problem occurs, the team can respond as soon as possible and make prepared actions. The fourth is to have a downgrade strategy. Under the impact of large traffic, priority should be given to ensuring the mainstream layer. For Didi, it is the issuance of orders. User issuance and driver grabbing are the main process. In such a special case, the Points Mall, etc. are non-mainstream layers and can be discarded.

"The happiest thing for me is that everyone is growing rapidly."

Every battle of life and death is a desperate race against time. And after each victory in the battle, it has cultivated excellent practical talents for Didi Taxi. What makes Zhang Bo happiest is that while the team members are growing rapidly, not a single one is lost.

CSDN: Didi Taxi has only been established for three years, but its impact on the industry is obvious to all. As a tech lead, what do you see as the biggest challenge?

Zhang Bo: In the past three years, wars have been fought almost every day. Every day is a matter of life and death as soon as you open your eyes. Just like in the subsidy war, the data rose 50 times in a week, but we were completely unable to deal with it. At that time, there was even such a scene, every peak, we went down. When we go down, our opponents go down. Because the users have gone to their side. Of course, they were down, and so were we. At that time, it was a competition of who stabilized first and who could keep users. At that time, we were desperately racing against time. Our team of more than 60 people did not sleep for 7 days and 7 nights, eating and living in the company. After going through it and looking back, I found that we have accumulated a lot of valuable experience, and we have also cultivated more talents with actual combat. Perhaps, it is difficult to say that a person has changed within a week, but for the past three years, on average, every few months have to pass through life and death. Under such pressure, you will be amazed at how fast you grow. A fast-growing industry, a fast-growing enterprise, your rapid growth is inevitable. Many people know that before I started my business, I was doing search technology in Baidu. At that time, I only needed to do a good job in strategy, and the underlying infrastructure layer, such as the computing and storage layer, should not be considered. DiDi is responsible for almost all levels, building systems, responding to business challenges, and doing more product design. There is basically no way to enumerate more, so the biggest challenge is to respond to challenges and accelerate growth.

CSDN: How to evaluate the team that "lives and dies together" with you?

Zhang Bo: My team is very good. Because they have experienced a lot of life and death battles, each of them will grow 2-3 times faster than a large company. Maybe 2-3 years ago, they were just unknown members of the technical team of a big company, but now they are the backbone of Didi Taxi and undertake very important work. For example, an iOS engineer who was once responsible for Didi’s public products and technologies has become a full-stack player. There are many more such examples. That's what I'm most happy about, seeing them grow rapidly, and from the beginning of their entry, not a single one is lost. In the next step, we will establish Didi Technical College. Each role has a technical growth roadmap, mentoring, and regular training. We will also introduce high-caliber talents, such as how to teach, and ask these top experts to lead everyone down the road.

CSDN: After the merger of Didi Kuaidi, the joint CEO system will be implemented, and the personnel structure will remain unchanged. But will there be a merger in terms of technical teams?

Zhang Bo: The technology platform and the corresponding architecture are being merged. We hope to become a different professional team, to deepen and refine the areas we are in charge of, and to help the business run faster.

CSDN: How is the future technical planning considered?

Zhang Bo: Increase the continuous investment in big data and other aspects, and increase the investment in the infrastructure level. We see that many companies in Silicon Valley have invested heavily in large-scale distributed computing, storage and machine learning platforms. For example, Google's optimization of MapReduce makes its computing far better than other platforms. We should continue to learn in these areas and improve our core competitiveness in technology.

CSDN: Your transformation is very successful. Do you have any good suggestions for the two different development paths from CIO to CTO or from technical backbone to CTO?

Zhang Bo: From CIO to CTO, we must clearly understand that technology serves products and products serve business. CTOs must have a business vision. Going backwards, CIOs should start with technology, support product development, and then support the company's current and future business plans. To put it simply: from the technical backbone to the product backbone, you must understand Internet products, understand user needs, know how to make a product that meets user needs, you must know what the business model is after these users come, and how does the enterprise create profit. From the technical backbone to the CTO, it is even more difficult. Technologists tend to be deep in one area, but to be an all-rounder, you need to understand each area, find the real top talent in the field, and then organize your team and steer the team to achieve your goals.

Written at the end:

When it was written, it happened that Shanghai had become "the sea", and the circle of friends was swiped, calling for "Didi Special Ship". We look forward to working with Didi Dache, which has completed the optimization of its architecture, to launch more innovative products that meet sub-scenarios more quickly.