Hello, I am Guo Dongbai. Architects have four main roles in architectural activities, namely building consensus, controlling risks, ensuring delivery and accumulating knowledge. We talked about the first two in the last class, and this class will talk about guaranteeing delivery and accumulating knowledge.
guaranteed delivery
Guaranteed delivery means that architects can reduce the uncertainty and complexity of large-scale architectural activities, minimize architectural solutions, and ultimately ensure high-quality delivery. There are three key actions: reducing uncertainty, controlling complexity, and minimizing architectural solutions.
In fact, uncertainty and complexity are two of the difficulties in delivering architectural activities. After all, the competitive environment, technological environment, regulatory environment, and business environment in the Internet age make all R&D activities very uncertain. In addition, due to the large number of participants, the uncertainty brought by each participant will also be transmitted to the architecture activities.
Uncertainty will bring complexity, coupled with the natural communication barriers of Internet companies, the complexity will naturally be further magnified. Therefore, reducing uncertainty and control complexity is the main difficulty that architects need to overcome in ensuring quality delivery.
reduce uncertainty
There are several sources of uncertainty. The first is the uncertainty of the target. This is mainly caused by the sponsor's uncertainty about the target. Regarding this point, I will talk about systematic coping methods in Lecture 18.
The second is the uncertainty of resources. In my opinion, this is the greatest challenge facing architects in the Internet age. Whether it is domestic or foreign Internet companies, they often stimulate the output of the team through a plan similar to the overselling of virtual machines.
Enterprises often have multiple R&D activities at the same time, and the overall R&D resources of the enterprise are not enough to complete the tasks assigned to all R&D activities. Then architects must compete for shares in the limited R&D resource pool to ensure the delivery of architectural activities.
Take the international e-commerce project as an example. On the one hand, the project requires the cooperation of operations, product, R&D, and testing personnel in multiple countries around the world, and these roles have their own KPIs and guaranteed items. On the other hand, the leaders of other projects are also enlisting the support and cooperation of these roles. So how do we get our due share of resources?
The two survival rules we highlighted earlier are imperatives: ensuring that architectural activities are properly targeted, and maximizing the business value of the project. This is benevolent. Of course, there are other rules to follow.
For example, respecting human nature, discovering the interests of participants, and maximizing the personal input of participants. Otherwise, when the architectural goals are not completely consistent with the interests of the individual participants and their teams, additional incentives need to be invested to ensure their commitment. I have seen a company pay half a year's salary to all participants as a bonus in order to ensure the success of a half-year project. If you hear that the working hours of Internet companies are 9/11/7, don't be surprised, there must be a brave man under the reward.
In addition to technical resources, there are also limited resources such as operational resources and office environment. We need to cooperate with project managers to ensure that these resources are sufficient. After the project is launched, the entrance traffic guarantee of related functions in multiple scenarios (home page, search, recommendation, big promotion acceptance page, detail page, etc.) is also crucial to the success of the project.
In fact, in the research and development environment of the Internet, as long as there are limited resources, they will eventually become scarce resources. Release windows, physical machines, computing resources, algorithms, A/B testing, can all become unexpected obstacles to delivery. Just like coordinating time window resources where most people participate, it is often the bottleneck of architecture projects in large enterprises.
In e-commerce projects, coordinating time windows is extremely difficult. Every country has fixed holidays, and users all over the country have different work and vacation habits. And each country's business has its own big promotion time and project delivery time. Then the time window really left for the architect to coordinate the team and participate in large-scale integration testing and acceptance is very small, generally only one or two per month. This kind of time window is bound to be the target of major projects, so it must be locked and taken care of in advance.
Not only international projects face the challenge of time windows, but also domestic projects. Assuming that an e-commerce project is launched in the fourth quarter, it will inevitably hit the Double Eleven and Double Twelve promotions, and the time window is even more scarce. In short, inventorying and ensuring the supply of scarce resources are critical to the success of the project.
Third, the uncertainty of business and technology environment. In view of this, we have made a detailed description of the countermeasures in the fifth rule. In addition, I also want to talk about the solution from the perspective of delivery certainty.
The best way is to increase the abstraction of technical solutions while shortening the phased delivery cycle. How to shorten the staged delivery cycle? In the early stages of a large project, both architects and other participants have limited understanding of the project. If you split the architectural activities into multiple phased delivery points, watch the feedback online. Then we can look at the impact of changes in the business or technical environment on architectural goals and business effects based on online data, instead of guessing out of thin air.
Increasing the abstraction of technical solutions refers to improving the robustness of API design to technology selection as much as possible, that is, improving the abstraction of interface and model design. Then in the subsequent delivery stage, we can correct or upgrade the suboptimal technology selection.
Fourth, the uncertainty of user needs. Refers to user needs that are inconsistent with our expectations, or user needs have changed significantly over time.
In addition to design thinking based on human nature, the solution can also deliver units based on incremental value. Use the real behavior of online users to judge whether the user's needs are consistent with the previous research, and at the same time, according to the expected behavior deviation and effect analysis, decide whether to make adjustments to the user experience.
In addition to the above-mentioned factors, there are other uncertain factors such as cultural environment and organizational structure, but they seldom change dramatically during architectural activities, so we don't need to make corresponding coping strategies.
complexity control
Complexity and uncertainty seem to be the same thing, but they are very different:
- Uncertainty refers to the discontinuous and unpredictable changes in the problem over time.
- Complexity emphasizes problems or solutions, which are difficult to describe in a few simple dimensions.
So how to control the complexity? First, decompose the architectural planning and delivery plan from the problem domain level.
Specifically, it is to decompose the entire architecture activity into sub-problems in different domains according to the problem domain. Then, in each problem domain, it is decomposed step by step into fine-grained execution plans from coarse to fine.
In fact, the daily task division of the R&D team is often disassembled according to the problem domain. So you can maximize the use of the daily communication and collaboration mechanism that has been formed to minimize delivery risks.
Let's look at an example. Suppose we are working on a large-scale e-commerce system content upgrade project, which needs to be divided according to daily vertical fields-traffic, shopping guide, search recommendation, transaction, capital, evaluation, commodity, merchant, service, performance, logistics, etc. Assign overall tasks to various areas. As for each field, the planning of each field will be determined according to the needs of contentization. For example, in the field of traffic, there is a need for customization, delivery, and generation of the content undertaking part. The field of contract fulfillment and logistics has almost nothing to do with this project.
However, in the process of division, you may find that the previous domain division is not optimal. Still this example. After the further refinement of the plan is completed in each field, we may find that there are common modules in traffic, shopping guides, commodities, merchants, services and other fields, that is, content generation and other content operations. If each domain developed these modules separately, it would cause a lot of waste. At this time, these content-related issues across multiple vertical fields can be combined into a new content domain. All content-related planning and delivery is done by this new vertical, from generation to review, encoding, delivery, playback, tracking, monitoring and analysis, and more.
Second, increase the structural nature of the architecture design itself.
Structural refers to a unified model that runs through all software implementation solutions in an enterprise. Reflected in the design, it is structured design (Structured Design), which means that the designs of different fields and different modules are isomorphic.
A very simple example. If all parties involved adopt microservice design, responsive design or front-end low-code design. So what are the advantages of this structured design? In addition to the advantages of operation and maintenance testing, its biggest advantage is that the cost of future changes is relatively low. We never do architecture for the present, but for the future. Only a design that is easy to adapt to the future environment can maximize the survival of the enterprise. This is what we emphasized in Rule 5.
For example, if we are working on an e-commerce system, we can divide the entire technical architecture into three layers: the business layer, the middle platform, and the infrastructure layer without business semantics. As we deepen our understanding of the business, we will find that the middle platform needs to be divided into two layers-the upper layer is the business middle platform, and the lower layer is the data middle platform layer and the shared technology layer. Both steps are split horizontally.
Then we can further switch the modules of the business layer into multiple vertical fields according to the business domain, and further refine the scheme in each field. Although the business logic of each domain is different, they use the same programming language, microservice framework, testing framework and release process. This kind of macro structure at the enterprise level is the structured design we want to pursue.
It should be noted that the process of structured design is not just verbal as an architect. This macroscopic structure comes from the pursuit of structure in the decision-making of architects and other participants in each field.
For example, the transaction module needs to classify the business requirements of different countries, and abstract the common requirements to the transaction platform. As for individual needs, it is related to the country's special business form or regulatory environment, and needs to be kept at the business layer to avoid complexity from intruding into the business center.
Regardless of whether this process will affect the entities related to the platform and transactions in the data, such as the abstraction of entities such as marketing activities, orders, and fund accounts. Therefore, the modeling of entities will definitely affect the marketing middle platform and capital middle platform through the data middle platform, as well as the front-end business that relies on these two middle platforms. At this time, only by delving into the domain model level of all related modules can we form a complete understanding of the complexity of the entire architecture activity.
If both you and the team are pursuing this kind of macro structure, then in the process of vertical segmentation, as you and the team sort out the subdivision scenarios, such as API design, domain model, message mechanism, etc. will gradually become clearer and more reasonable , more structured. Conversely, if a person tries to destroy the structural rationality of his own field out of greed, then this behavior will be passed on to the middle office and spread to other fields. This is why I have repeatedly emphasized in Rule 6 that a culture of seeking truth and having a conscience is extremely important for an enterprise.
This combing process, if done more seriously, will also help you reduce the risk of delivery. And as you sort out, your plan will become more and more complete, and the certainty of the delivery time of each module will continue to improve. The research and development tasks also range from the segmentation of the macro domain model, to the segmentation of database tables, to the segmentation of fields and messages, and the granularity is getting finer and finer. The risk has also shifted from the relatively macro risk at the beginning to the specific design level and the delivery date of the subdivided modules.
In fact, the sole goal of many architecture activities is to improve the structure of enterprise software architecture. You must have participated in projects such as technology stack unification, data model unification, and site-wide standardization. Their goals are all to improve the structure of the software system. However, the cost of reconstructing a chaotic system into a structured system is very high. Therefore, we should avoid structural degradation of software through the self-discipline of daily architectural activities. It can be said that any behavior that destroys the overall software structure of the enterprise is shameful!
Of course, the primary goal of some architecture activities is to construct new business models or maximize business value. Such large business model changes often cover multiple fields and multiple layers of software architecture at the same time, and the time is very tight. In this kind of architecture activity, it is more difficult to improve the structure of software design. Because such projects are generally destructive to the structure.
At this time, if you can find a way to limit the explosion radius of architectural activities, you can indirectly reduce the structural damage to the software. On this point, you can refer to the performance optimization project case I gave in Lecture 5. Through the abstraction and accurate measurement of the business value of performance optimization, that is, the concept of performance loss, I constrain the scope and depth of exploration of participants in the entire architecture activity, avoiding the waste of meaningless R&D resources by participants, and also avoiding structural damage to the system. This kind of restricting the scope of architectural solutions by emphasizing a single result indicator can be said to be tried and tested.
Third, segment the delivery modules in various ways.
The idea of splitting delivery modules is similar to running in small steps. The more common method is to deliver in installments, such as by department, by domain, or by architectural layers.
Delivery by department or domain means that the delivery of architecture projects is completed separately according to vertical domains. Layered delivery by architecture refers to completing the changes at the lowest level first, and then gradually working upwards. The delivery of each layer maintains backward compatibility, thereby minimizing the intrusion to the upper layer business. In future courses, we will also mention the method of delivering by the smallest value unit, which refers to delivering modules according to user scenarios and requirements.
Regardless of the delivery scheme, the advantage is that the entire delivery process is similar to a loosely coupled grayscale release process, so that you can try and make mistakes in a small area and find problems in advance. Another benefit is to reduce the complexity of joint debugging and acceptance. In addition, the problems of uncertainty and complexity we mentioned at the beginning can basically be solved. Almost all technical projects on my team are delivered this way.
Minimal Architecture Solution
Whether delivered in stages, or constrained in scope by a single goal, these are examples of minimizing architectural activity. This minimal and necessary principle is the most important way to improve the success rate of delivery.
However, in actual projects, most of the behaviors we see are the complete opposite, which is especially common in large factories. Some project leaders can't wait to spread the project to the entire enterprise. Even in order to expand the influence, the working radius of the project is deliberately enlarged. This kind of behavior is a no-no in project delivery, and you can read it in any introductory project management book.
It is a truism that most architecture activities fail. Although many people in charge of architecture activities don't say this to the outside world, the real situation must be that there are more defeats than victories. If there was only one recipe for success in architectural activities, it would be to minimize it!
There is a very important user experience index of architectural activities related to minimization, that is, the proportion of business volume affected by architectural activities. The lower the ratio, the better. The most perfect situation is zero, that is, the architecture activities are completely transparent to the upper-level business. During the delivery process, the upper-level business can continue to iterate without any impact.
Conversely, the delivery experience of the worst architecture activity is to completely block the upper layer business iteration. I have seen an international middle-Taiwan project, which was originally expected to be delivered in three months, and then gradually postponed to 6 months, 9 months, and finally finished for a year, and it was still unfinished. The most frightening thing is that the person in charge of this project did not listen to dissuasion and chose a very extreme blocking delivery mode: During the progress of the project, all related upper-level businesses cannot make any changes in the transaction and marketing models. In this way, the upper-level business is like the generals of the Yang family fighting a war. It is already very difficult to have no food for three months. After being on a hunger strike for a year, there is no need for opponents to pick him off the horse, and he will starve to death.
So please remember to minimize the impact of the project on the upper module!
Precipitate knowledge
In architecture activities, architects have a macro perspective different from other participants. Therefore, it is necessary to ensure the quality of thinking and decision-making in architecture activities through effective knowledge accumulation, and it is also necessary to provide valuable experience and methodologies for future architecture activities of enterprises. On the one hand, architects need to deposit complete and authentic process records. On the other hand, it is also necessary to inject logical thinking into the enterprise and guide the enterprise to make correct decisions.
Some people mistakenly think that accumulating knowledge is collecting, organizing and documenting. Indeed, it is important to continuously accumulate digital or documented records of the entire life cycle of architectural activities. In a way, this record is a digital mirror image of the entire architectural activity. Not only can it provide complete and comprehensive information for participants in architecture activities, it can also provide valuable experience for architects and relying parties of other projects. For example, Confluence, a commercial documentation tool, has been very successful.
However, this is not the whole of precipitation knowledge. What we need to do more is to inject rational thinking into architectural activities through various documentation tools, design tools, communication tools, and review tools.
You can spend a few tens of seconds thinking about the difference between the two:
- Collecting, organizing and writing documents is a process of digital mirroring. The real-world behavior happens before, and the digital process happens after. This is a passive process.
- The process of injecting rational thinking is the process of driving rational thinking by the rigorous logic in the document. Documentation and design happen in front of us, driving rational, fact-based thinking and decision-making by us and other participants in order to change the real world. This is an active process.
In the passive process, the architect is like an automatic burial and log collection system, faithfully recording all behaviors, phenomena and results during the project process.
In the active process, the architect is more like a computing device, driving the thought experiments of the participants in the architecture activities through documents, and improving the quality of thinking through theoretical deduction, rather than writing codes, publishing, and online trial and error. Complete the iteration of the architecture proposal.
Remember the case of performance optimization I mentioned in the fifth lecture? Before we started to change the code, I defined the concept of performance loss, deduced and proved the formula of performance loss on the document, and discussed the actual measurement methods in different pages and scenarios on the whiteboard, etc. . This document-driven thought experiment made us know the value it could generate before we made a move.
You might say that this case is too special. Not! As long as the project is complex enough, the cost is high enough, and the risk is high enough, I will definitely ask myself or the project leader to do a similar deduction. This is the best way I have practiced to improve the success rate of large projects, and it is also a habit I have persisted as an architect for many years. Deduction is not necessarily a formula, it can also be a document, or even a stigmatized PPT. The form is not important, the key lies in "active thinking".
Maybe you will also say that in the Internet age, everyone should try and make mistakes. Is it worth spending so much time on deduction? My answer is: not only value, but value.
Or my own experience. The topic of my doctoral dissertation is related to the application of computer vision in the medical field. Generally speaking, doctoral dissertations should have specific formula derivation and results presentation. But one of the derivations is wrong, and I didn't find it when I wrote the code, and I didn't realize it until I wrote the paper. In the end, you can only go back and change the code.
Moreover, this error occurred in the feature modeling process before image segmentation, so I was forced to change all the codes that I spent two years writing, from feature extraction to feature calculation, to image segmentation, 3D visualization, 3D modeling, and stress analysis. calculate.
In order to complete the dissertation, it took about half a year, from 8:00 am on Friday to 7:00 pm on Sunday, and I worked continuously for more than 50 hours. Fell asleep at least four times while driving on the highway. Looking back now, I am really scared!
In fact, it took me just over a day to find and prove this derivation wrong after I rewrote the paper. But I almost ruined my young life because of the carelessness of this day! Since then, I have developed a habit: as long as it is a sufficiently complex project and work, I must write down the complete formula, thinking logic and decision-making logic on paper, and scrutinize it repeatedly. And share it with colleagues around, let everyone help me find problems together.
No matter how many times the logic is deduced on paper, the cost is far less than the cost of having dozens of people chanting slogans and rushing towards a cliff with flesh and blood. I think it is this habit that makes me go further and better on the professional track of architect.
Thought experiments can help not only businesses, but ourselves. You can find an introduction to Amazon's Six Paper on the Internet. This is also a thought experiment. This process is very time-consuming and labor-intensive. I once wrote a document that had to be revised 13 times before passing the review. These are not minor revisions. There are more than a dozen people participating in the review for each revision, including a technical VP.
Think about how much this thought experiment cost! One of Amazon's corporate creeds is "Bias for action". The full expression of this creed should be: "Be brave to try and make mistakes after sufficient thought experiments."
To sum up: an ideal process of knowledge precipitation includes not only a passive process of recording activity history through documents, but also an active process of creating history through documents to drive thought experiments.
summary
Architects who want to continuously guarantee delivery throughout the entire architectural activity must continuously control uncertainty and complexity while minimizing architectural solutions. The source of uncertainty comes from goals, resources, environment, and user needs, and these external factors will change discontinuously over time. To deal with the discontinuity of change, on the one hand, we need to confirm these influencing factors early and continuously. On the other hand, it is also necessary to look for invariants, to look for a higher and deeper stable abstraction.
The complexity comes from the complexity of the business model of Internet companies themselves. You can control this through partitioning of the problem domain, continual pursuit of structured solutions, and reasonable partitioning of deliverables. But these are all "techniques", the most important thing is the principle of minimum necessity. Don't be overjoyed, let alone put yourself in front of the business and block the iteration of the upper-level business.
Finally, we also talked about knowledge precipitation. An ideal knowledge precipitation process includes not only a passive process of recording activity history, but also an active thought experiment process of driving value creation. The accumulation of knowledge in the virtual world and the structural activities in the real world are like the two strands of DNA. Knowledge guides activities, and in activities, knowledge can be continuously created and verified. The two are processes of mutual motivation, mutual creation and mutual influence.
This is the most beautiful part of the architect's work! Our work can always make our thinking more perfect.
thinking questions
Three reflection questions, choose one:
- What is the biggest uncertainty you have experienced in a project? What countermeasures have been taken? How is the result?
- Our two classes have taught the four basic skills of building consensus, controlling risks, ensuring delivery, and accumulating knowledge. In addition, what other skills do you think an architect must have? Why?
- What is the most successful thought experiment you have ever heard of? Why can it succeed? Precipitate knowledge