introduction
Today, when Internet digital companies have ended their "barbaric expansion" and pursued high-quality growth, R&D efficiency has become the core proposition that companies pay attention to. With the cloud-native concept taking root in the software field, cloud-native is driving great changes in software application design, implementation, deployment, and operation and maintenance, bringing new challenges and opportunities to R&D efficiency governance, and software efficiency will usher in a new era. The era of cloud native revolution.
On March 25, 2023, the Techo TVP Developer Summit "Cloud as the Core, Efficiency Fusion" officially came to an end. 11 well-known technical leaders and experts from the field of efficiency, from efficiency governance, cloud native, DevOps, observability, etc. In this aspect, the best practices and future trends of R&D efficiency improvement are discussed.
W
The summit was hosted by Lekaiser CTO and Tencent Cloud TVP Huang Daoyong. At the beginning of the meeting, Ms. Huang first shared the original intention and purpose of the Techo TVP Developer Summit - aiming to present a "most informative and interesting" event for developers and friends through the most attentive sharing, cutting-edge insight, and most practical views. , and useful” developer summit.
Teacher Huang said that in the world of martial arts, the only thing that can't be broken is fast. In the era of digital economy dominated by information technology, R&D efficiency determines how "fast" you are. However, with the continuous improvement of software scale and complexity, the traditional software R&D model of "human sea tactics" is no longer feasible. We need to re-examine the traditional software R&D model, R&D process, R&D tools and other aspects, and improve the engineering team one by one. R&D efficiency, under the premise of ensuring quality, continuously deliver value as efficiently as possible.
guest speech
Huang Junhong, vice president of Tencent Cloud, delivered a speech for the summit. He first reviewed the history of Tencent's continuous internal R&D efficiency governance. Starting from Tencent's internal cloud migration in 2019, DevOps of the R&D process is a very important task. Through R&D data asset governance and continuous delivery pipeline transformation, combined with containerization and microservice framework transformation, Tencent's self-developed business will be fully cloud-enabled in 2022.
He also mentioned several data in the just released "Tencent R&D Big Data Report": More than 90% of business R&D is built and deployed through Tencent CI cloud-native pipeline, the construction time is shortened by 60%, and the deployment efficiency is increased by 75%.
Tencent Cloud also condenses Tencent's internal research efficiency management experience into technical capabilities to help more companies improve research efficiency. At the infrastructure layer, the Tencent Cloud Native team actively explores serverless, builds a future-oriented application development and delivery system and cloud experience, and realizes the development of cloud-native resources as a service; at the basic software layer, the Tencent Cloud database team developed and polished TDSQL, a large Significantly improved database performance to meet the needs of elasticity, high availability, and scalability; in the R&D process, launched a cloud IDE for remote collaboration—Tencent Cloud Studio, and opened up WebIDE and DevOps capabilities to connect the development resource environment on the cloud. Optimize the cloud development and collaboration experience; at the operation and maintenance level, Tencent Cloud Monitoring has launched an integrated observable platform to break through the data islands of monitoring products, covering different scenarios such as App, distributed applications, containers, etc., effectively improving the efficiency of monitoring operation and maintenance, and will Its observation capabilities are integrated with cloud-native application management capabilities, allowing technical teams to fully enjoy the benefits of cloud-native technologies; Tencent's security team has subversively innovated traditional development security tools to provide data asset security prevention and control capabilities on the cloud, forming a comprehensive DevSecOps system construction.
In the end, he concluded: "With the continuous development of technology and the emergence of more and more new technologies and new tools, the improvement of R&D efficiency will become more and more scientific and intelligent, just like the recent explosive emergence of AI. Similarly, I believe it will bring revolutionary innovation to the development of the entire industry."
Tencent Cloud R&D Efficiency Improvement and Business Practice
Zhang Yu, head of the Tencent Cloud R&D Efficiency Improvement Working Group and Vice President of Tencent Health, shared the theme of "Tencent Cloud R&D Efficiency Improvement and Business Practice".
He first shared the pain points Tencent Cloud encountered in R&D efficiency and the main solutions. The pain points can be summed up in three points: standard specifications, tool platforms, and cultural promotion. Specifically, in terms of standards and specifications, from the perspective of Tencent Cloud or the upper layer, there is a lack of higher-level unified standards and specifications; on the tool platform, there are many tools, but they do not form a unified whole; In terms of cultural promotion, it is hoped that managers and front-line employees will reach a consensus on the importance of R&D efficiency.
The Research Efficiency Improvement Working Group has done a lot of work on these three pain points: one is to formulate unified standards and specifications, so that everyone can gradually return to the recommended mainstream standards; the other is to connect the tool platforms in one-stop series to form a unified The research platform shared by the entire Tencent Cloud; the third is to strengthen publicity and guidance, so that everyone can pay attention to and participate.
He used the pyramid model to share the concept and thinking of the entire Tencent Cloud research platform. The ultimate goal is to realize the whole process from requirements, development, construction to final operation and maintenance, and complete the whole process on the research efficiency platform in an integrated manner, so as to improve the happiness of front-line R&D and operation and maintenance. The first stage is the improvement of the capabilities of various tools at the bottom of the pyramid, selecting the existing mainstream tools, and then improving their capabilities, especially the enhancement of automation. Further up is to connect the tools in series through a one-stop portal. From requirement proposal to code management to the final online release and monitoring operation and maintenance, all are completed on this platform. This is the second stage. The third stage is to practice the DevOps concept on top of this, especially to run through the process from an application perspective. The top layer is value expression, which is also the ultimate goal of the research efficiency work, that is, through data measurement and tracking, it is hoped that all the research efficiency improvement work is indeed in line with expectations, and it is indeed moving forward according to the envisioned track.
Afterwards, he took Tencent's medical business R&D effectiveness practice as a case to share his thinking during the construction process. To sum up, in the development domain, the development model, development specification, and scaffolding for automatically creating services are unified; in the construction domain, continuous integration is improved, automatic pipeline creation, continuous integration increases quality access control, and build quality is improved; in the test domain, the test The efficiency has been significantly improved; in the deployment domain, the continuous release has been improved; in the operation domain, the observability system has been improved.
Finally, he introduced the future planning of Tencent Cloud's research and efficiency work. Continuously strengthening the interconnection within the project is the focus of the next step, and he also hopes to further improve the efficiency of the entire cross-BG aspect.
R&D cloud-native efficiency "singularity"
Everyone is familiar with the word "cloud native", so what kind of sparks can "cloud native" and "efficiency" collide with?
Xiao Ran, General Manager of Thoughtworks Innovation Technology and Tencent Cloud TVP, shared the topic of "Singularity" in R&D Cloud Native Efficiency. Mr. Xiao believes that the current digital challenges faced by large R&D organizations mainly include four aspects: R&D costs are rising rapidly, as long as the productivity is increased by 20%, it is equivalent to reducing R&D costs by 10-50 million US dollars per year; talent recruitment and retention challenge, scarce talent votes with their feet; in terms of competitiveness and time to value, disruptive companies have less historical baggage and can grow 10 times faster than most entrenched market leaders; Increasingly frustrated with better 2-3 year program outcomes and consistent timelines.
He focused on the six pillars of R&D cloud native management: value-driven, integrated collaboration, distributed autonomy, rapid response, built-in quality, and continuous improvement.
Teacher Xiao put forward three slogans: First, we must work hard on the architecture. Cloud Native does not allow everyone to move existing things directly into cloud containers. This is called Shangyun, not Cloud Native. Therefore, we must work hard on the structure. The second is that the team must be value-oriented. Next, the team will be more fragmented and smaller, but it must be value-oriented, otherwise problems will arise. The third is to reduce costs and increase efficiency in project implementation. If you really want to enter cloud native, you must reduce costs and increase efficiency. Don’t be ashamed to mention that if you spend 1 million this year, you will win if you spend 900,000 next year. This is real money for enterprises.
From Monitor to Observability Tencent Cloud's development and practice in the field of observability
Huang Jie, Technical Director of Tencent Cloud Observable Platform, brought a sharing titled "From Monitor to Observability, Tencent Cloud's Development and Practice in the Field of Observability", discussing how to establish an end-to-end integrated observable platform to improve monitoring efficiency , reducing the workload of operation and maintenance.
He first introduced the difference between observability and traditional monitoring. He believes that traditional monitoring is mainly to solve individual problems, such as whether the system is working normally, whether a certain service is normal, and solves relatively fixed single problems; while observability is more about solving why the system does not work, the reason What is, is a systemic problem.
After introducing the three pillars of observability - Metrics, Logging and Tracing -, he explained in detail how to build an observable platform, and introduced the Tencent Cloud integrated observable platform.
In the end, Mr. Huang Jie made some prospects. One is to hope that observability can also be standardized, so that the data will be more standardized, more convenient to process, and business migration is also very convenient; the other is how to pave the way for AI and let machines help us Identify business problems.
Building a New Paradigm for Enterprise Cloud Development and Deployment
As the "cloud-native" development model has gradually become a new industry trend, the development tools that integrate the underlying cloud resources and the online development environment will reshape the new development paradigm around the modern process. Wang Shengjie, product director of Tencent Cloud Cloud Studio, shared his sharing on "Building a New Paradigm for Enterprise Cloud Development and Deployment".
He believes that the opportunity for an enterprise-level cloud development environment is the development model change under the background of cost reduction and efficiency increase. model. Based on container isolation, it combines with the R&D standard process in the enterprise, empowers enterprise developers and managers, and optimizes and controls R&D costs.
The cloud development environment can bring four aspects of value: improve efficiency, develop one-click operation, take advantage of the elasticity of cloud computing, and start whatever resources are needed; improve productivity, work in parallel through containerization; can closely integrate DevOps ; It is beneficial to supervise and develop resources and reduce costs.
Elasticity is one of the advantages of the cloud development environment. Another advantage is that the code on the cloud can not be implemented.
Some companies prefer that the code is an enterprise's asset, not to enter the local computer, and hope that the cloud development environment has higher security requirements. Tencent Cloud's Cloud Studio has also released such functions, such as digital watermarking, and some encryption methods. The code of the containerized storage cloud development environment will undergo the highest level of encryption.
The cloud development environment also brings a new working mode of collaborative development and collaborative investigation on the cloud. In Cloud Studio, everyone enters the same workspace and starts multi-cursor. Here, a lot of concepts of documents are used, and code follow is used. At the same time, I also did a terminal follow-up. When I need to debug the problem of the terminal, I locate this place, and another person receives it. If I encounter a problem here, I can ask others to help me solve it. There are also functions such as message boards. Architecture discussions and reviews can also be done.
Teacher Wang Shengjie also introduced some solutions for the cloud development environment. For example, Cloud Studio can directly empower the mobile development ecosystem. Taking small programs as an example, it can quickly pull up all the environments of small programs, rely on packages, preview them at the same time, and publish them directly to Multi-terminal mini program ecology. Scenarios such as teaching and interviews can directly create a development environment for the target group. Combined with the recent popular ChatGPT, he believes that AI will further empower the cloud development environment, and the future is promising!
From Push to Pull——First Principles of Software R&D Effectiveness
What is the nature of the R&D effectiveness problem? How to achieve efficient R&D management and ensure product delivery efficiency and quality? Wu Qiong, Chief Consultant of Agilean and TVP of Tencent Cloud, presented a presentation titled "From Push to Pull - The First Principles of Software R&D Efficiency".
Teacher Wu Qiong made it clear what is the first principle of software performance—that is, when we think about the problem of performance, what is our thinking point, where the problem comes from, and how to solve it.
What are the root causes of ineffective software development? Teacher Wu Qiong thinks it is congestion - in a state of congestion, most improvement measures will have little effect. Congestion is characterized by less movement and more stops, accompanied by low flow efficiency and high resource saturation. In congested scenarios, the team is usually forced to adopt multi-task parallelism, and task switching under high concurrency is often accompanied by great performance loss. Therefore, the key to improving R&D efficiency is to reduce parallelism and waiting, and accelerate delivery.
Next, he analyzed the causes of congestion: one is the deadline scheduling mechanism of the push process, and the other is the abuse of the push workflow. Under the deadline scheduling mechanism, frequent queue jumps and tight schedules lead to chaotic work rhythm of the R&D team, and the R&D team is exhausted in the "firefighting" work; in the push workflow, nodes with capacity bottlenecks will soon be upstreamed The transferred work items are full, and the employees at the bottleneck point will face huge workload and delivery pressure.
To solve the congestion problem, we can use the pulling concept of lean thinking to establish a pulling value flow. Let the bottleneck points in the R&D process establish their own delivery rhythm, so that the R&D work can pass through the bottleneck points more efficiently and orderly, thereby solving congestion and improving the overall delivery speed and delivery quality of the delivery team.
Teacher Wu Qiong finally concluded: "Identify the degree of team congestion through demand lead time and flow efficiency; implement demand optimization and version trains to establish a pulling R&D process; use kanban to transparent the R&D process and promote demand flow. The R&D team can refer to such an implementation The path alleviates the congestion in the R&D process and realizes the global optimization of R&D efficiency."
Platform Engineering platform engineering: based on cloud native Serverless to create the ultimate research effect
Serverless technology has many advantages such as reduced development costs, automatic expansion and contraction on demand, and no operation and maintenance. It has been widely used by developers to build cloud applications faster. How will serverless create a future-oriented application development delivery system and cloud experience? He Shiyou, product manager of Tencent Cloud Functions, shared "Platform Engineering: Creating Ultimate Research Effects Based on Cloud Native Serverless".
As mentioned in the sharing, the market size of serverless architecture continues to grow, and Tencent Cloud has also created a serverless product matrix with rich content.
The market size of the serverless architecture continues to grow. The function SCF has served 2 million developers in total, and the average call volume of the next day has exceeded 30 billion. Teacher He also introduced in detail the research and operation platform built with Serverless products to meet the needs of enterprises and developers.
He concluded in his sharing: "We pay more attention to the intermediary process in the next ten years, especially now that many have embarked on a combination of cross-cloud, hybrid cloud, IDC, and cloud-on-cloud pull-through solutions. , how to introduce advanced development experience into the enterprise. So we are trying to take another path. This path is compatible with the demands of the enterprise and the demands of the developers in the enterprise, so as to meet the cost and budget of the enterprise. , management demands, while meeting the ultimate development needs of developers, such an overall solution.”
Tencent Development Security DevSecOps Landing Practice Sharing
During the transition to DevOps, traditional development security tools encountered great challenges in the DevOps mode. How to build a complete DevSecOps system to ensure security prevention and control capabilities on the cloud? Liu Xianlei, Senior Solution Architect of Tencent Cloud Development Security, delivered a speech titled "Tencent Development Security DevSecOps Practice Sharing".
Defects are born, loopholes are inevitable, and the security of the code itself is the foundation of security. Teacher Liu Xianlei first analyzed some reasons for the safety problem. In the past two years, there have been some new problems in the whole process of development security, such as supply chain problems, problems like purchasing external supplier supply chain problems, problems like third-party development components introduced in the construction phase, and so on.
The evolution of the R&D model to DevOps requires new application security development and construction. The idea of DevSecOps is to shift security to the left: the earlier security is involved, the lower the cost of vulnerability repair.
He reviewed several stages of Tencent DevSecOps construction: from outside to inside, from point to surface, from problem solving to efficiency.
Mr. Liu Xianlei concluded that although security is an old proposition with more than 20 years of development process, the development of security is faced with new scenarios, new models, and new threats. In response to these security issues, Tencent has experienced deep pains, and finally found that using our This tool can be more in line with the current development mode of people in the DevSecOps mode.
Analysis and best practice of new generation cloud native database technology
In the era of cloud computing, cloud-native databases are becoming more and more important due to the growing demand for elasticity, high availability, scalability, and on-demand usage of applications from different business domains. Liu Di, Product Director of Tencent Cloud Database, shared "Analysis and Best Practices of New Generation Cloud Native Database Technology".
In his sharing, he introduced the core technology of Tencent Cloud's native database. The first is the cloud-native software design, which separates computing and storage. Computing nodes and storage nodes are completely independently decoupled and deployed, making computing nodes completely stateless, and their high availability and high reliability are much better than traditional databases. The second is the self-developed database kernel—the enterprise-level custom kernel TXSQL, which has done a lot of work on enterprise-level features, performance acceleration, and data security at the kernel level. The speedup ratio can reach 20. The third is to do a lot of optimization in the combination of software and hardware. Through the RDMA network upgrade, compiler optimization and the secondary cache capability realized by persistent memory, the database performance can be improved by more than 2 times. The fourth is to explore the integration of AI, realize "unmanned driving" database autonomy and AI intelligent parameter adjustment based on deep reinforcement learning algorithms, and further improve the stability and performance limit of cloud-native databases.
Mr. Liu Di also shared three cases: After Ruixing Coffee replaced the transaction system and the core ordering system of the mall with a cloud-native database, when encountering the scene of ordering explosive products, the cloud-native database reached a certain load After that, automatic second-level elastic processing can be performed; after blued migrated to the cloud-native database, the delay dropped by 80%, which was lower than the original running delay alone, which solved the performance and operation and maintenance complexity problems; finally, combined with the new generation of serverless architecture in A large number of application cases in the background of WeChat applets highlight that the realization of serverless technology through the underlying architecture of the cloud-native database can realize self-elasticity and self-expansion capabilities for users, and help users reduce database usage costs by more than 80%.
Roundtable Dialogue: From management to tools, the whole process of improving quality and efficiency is delivered
After the wonderful sharing session, the summit entered into a round table forum where ideas and viewpoints collided. The theme of the roundtable session is "From management to tools, the whole process of improving quality and efficiency". The session was still presided over by Mr. Huang Daoyong, and four guests including Wu Qiong, He Shiyou, Wang Shengjie and Zhao Jiuzhou, product director of Tencent Cloud Observable Platform participated in the exchange.
On the issue of how to understand the essence of R&D efficiency, Mr. He Shiyou said that R&D efficiency emphasizes that there is a closed-loop pipeline from project approval to final launch and subsequent iterations, and it is a cycle that is always iterative and spiraling. Let each role stay in the field of expertise to focus on the core work, find the weak link and make the weakest link efficient, and the entire link will be more efficient.
When discussing how observability can help R&D efficiency, Mr. Zhao Jiuzhou made an analogy that R&D efficiency is to better help companies reach their destinations, and observability is navigation, and the two are inseparable. Specifically, it can be reflected in three aspects: helping better troubleshooting and prevention in the R&D process, helping R&D performance to do better measurement and planning, and helping to improve the efficiency of cooperation with the operation and maintenance department. Tencent Cloud's observable platform includes multiple functions such as stress testing, multi-testing, and performance monitoring. It can observe performance problems such as freezes and crashes at the user level; it can also help software by analyzing related data such as logs, links, and indicators. Locate faults and defects, and through historical data analysis, predict potential risks and discover resource utilization improvement points.
When discussing how efficiency governance can avoid efficiency silos and horizontally integrate various single-point capabilities to achieve overall optimization, Mr. Wang Shengjie analyzed from the company level. The most important thing for the company is the issue of information value flow. First of all, a unified perspective is needed to make the overall structure more transparent; secondly, there must be a good value stream, the so-called value stream is data, and it is necessary to define a good overall view and a good data model. When it is difficult for the system to see the overall situation, it depends on what you can empower the enterprise and how much ability you can add to other teams, so as to make the entire business, value stream, and corporate culture better.
When discussing the fatal misunderstandings of agile project practice at different stages, Mr. Wu Qiong pointed out that the biggest misunderstanding of agile practice is to copy other people's experience, not to objectively analyze one's own situation, and to formulate an evolution method suitable for one's own. The requirements of each organization are different. For example, the main problem of your current business is that it cannot be implemented quickly, so although you are not innovative enough at this time, we may feel that this is not your short-term focus. An organization cannot do everything. That doesn't exist, and the cost is unbearable.
In the process of R&D efficiency governance, many obstacles will be encountered, such as conflicts of interest, imperfect infrastructure, insufficient awareness of the team and even management. In the joint discussion session, the guests shared their views on issues such as whether management systems or platform tools should be given priority in research efficiency governance, and how small and medium-sized enterprises can improve research efficiency.
Mr. Wang Shengjie believes that the platform should go first. Never expect the boss to understand the details before the employees. In terms of R&D efficiency, the boss pays more attention to cost-effectiveness. After the platform is first, make a best practice, or have figures to prove how much you have improved, and the boss is willing to pay for you.
Teacher He Shiyou believes that management needs to go first, but the proportion of platform tools will be greater. Management should be as light and thin as possible, only focus on the core, only set the basic development norms and processes, do not involve management, do not involve performance, more as a traction, let everyone achieve cooperation. In the end, we still rely on tools to help you execute, so that engineers can automate the process as much as possible.
Teacher Wu Qiong believes that the future trend will be to strengthen management and use tools at the same time. All organizations need to be managed. The Internet didn't pay much attention to management before, but the next three years will be three years of strong management. To be effective, you must first have data, and only when you have data and a baseline can you manage. How did the baseline come about? This reflects the importance of tools. Without tools, it is difficult to achieve large-scale management. Therefore, SaaS-based and public cloud tools will definitely be used more and more, and self-developed tools will become less and less. In particular, avoid using it as a tool instead of management.
Mr. Zhao Jiuzhou believes that it depends on the nature of the company, small companies are engaged in management, and large companies are engaged in tools. People are uncontrollable. In large companies or complex organizations, it is too inefficient to drive people through management, and tools and platforms are needed for management. In small-scale startups or technology-based companies, it may still be necessary to focus on management, and it is enough to use some basic tools/services that meet actual business needs. Many cloud vendors on the market are also providing corresponding capabilities to provide targeted support for small and medium-sized enterprises. Many tools can be used, so don't waste energy on self-development. Mr. Zhao also gave an example of the team's cooperation with the Tencent conference some time ago. Programmers from different places opened the Tencent conference to cast screens to change the code, and continued to communicate with the conference on programming and research results.
The host Mr. Huang Daoyong concluded that whether the tool comes first or the system comes first cannot be generalized, it depends on everyone's environment. The cost of management in a large company is very high, and it requires extremely high authority or consensus to change the work behavior of many people, so it may be easier to produce results by using tools. If it is a small team, the cost of making tools is too high, and the benefits of management are faster. Maybe everyone holds a meeting, synchronizes, and solves it after one or two training sessions. It is not so easy for a large team to reach a consensus. This also requires each developer to judge according to their own environment, and which one comes first must be based on their actual business.
epilogue
The exchange event with frequent technological sparks and thought collisions is always memorable. Amid the applause of the participants, the summit came to an end. The summit was rich in content. Eleven experts shared insights on how cloud-native drives R&D efficiency and accelerates change. There are ideas, practices, methods, tools, methodologies from a strategic height, tools that can be implemented in practice, and more. Feature-rich cloud functions. From the sharing of experts, we can feel the profound changes that cloud native brings to R&D efficiency, and we can also feel from their speeches how AI technology will further change our R&D process. I hope that the exciting content of this summit can bring you some inspiration and promote the R&D efficiency change of your technical team.
The future has come, Techo TVP Developer Summit will walk with enterprises and developers all the way!