cp : https://www.csdn.net/article/2011-02-11/291667
Abstract: Projects that have been actively developed for more than one year often fall into some common problems in the later stage: The construction speed is slow, and it often takes more than an hour to generate a final output product; The architecture is complex: Although the architecture itself can be used similar to MVC/Service General purpose like Bus
Projects that have been actively developed for more than a year often fall into some common problems at a later stage:
The construction speed is slow, and it often takes more than an hour to generate the final output product once;
Complex architecture: Although the architecture itself can be described by something like MVC/Service Bus, it is often the business itself that complicates the architecture;
The development speed is slow, and the construction speed is one of the factors, which makes the feedback of continuous integration much lower than expected; however, such large projects are often layered and divided into projects through various technical means. You have to face May not be a project, but a group of projects. In the projects I have participated in and consulted before, developers have to face dozens of projects when they open the IDE, and as many as hundreds. Even with the most powerful development machine at present, it still seems powerless in the face of hundreds of thousands of lines of code.
And a series of problems arising from the above: such as new talent training, knowledge transfer and so on.
Before coming up with solutions to these problems, let's look at how these problems arise. It usually takes a long time for these problems to become problems, and often when they first appear, there are always some quick and effective solutions to cover up, which further aggravates the escalation of the problem and eventually becomes a protracted problem that requires a lot of manpower to solve. .
the creation of the project
New project is coming. Team members are excited to introduce the latest MVC technology frameworks (such as SpringMVC/ASP.NET MVC), persistence frameworks, dependency injection frameworks, and more. The now popular iterative development method was also introduced. So the first few iterations passed, Domain, Service, Web, etc., well-layered applications were produced. The needs were quickly fulfilled. The code is very healthy. Build is very fast. All are happy.
2 months have passed. Deliberate team members are constantly refactoring the code to ensure that duplicate logic and duplicate code are eliminated. New people join the team. New business needs also come. These constantly refactored codes are further refactored: finally caused some problems: Since there is only one main line: Domain -> DAO -> Service -> Web, under parallel development (such as 5-8 parallel jobs at the same time) Then the publicly used line will continue to generate code merge conflicts/or business logic conflicts.
This is not a serious problem. However, this problem has restricted the expansion of the team. For example, when more people are required to join the project, the time spent on communication will greatly increase, and the effective productivity of new members will not be improved.
Not too difficult problem to solve. The team is not that big right now. The team's architectural role only needs to spend a weekend to divide the existing code vertically according to business logic and divide it into different small projects. The problem is basically solved.
Here comes the problem
More code is submitted. Someone complained when the build speed went from 2 minutes to 6 minutes, so took some time to optimize the build script and the time was reduced to 5 minutes. The code continues to grow - this is an inevitable trend - the build time continues to increase, from 5 minutes to 11 minutes, everyone's work habits start to change a little: once you start building, start chatting with your partner next to you , or take the opportunity to grab some coffee. A local commit is a bit different than a continuous integration server at this point - the local may only run a few build steps, the necessary tests, and the server runs everything.
When it went from 11 minutes to 23 minutes, everyone felt that something had to be done. Upgraded the development machines of all developers, the latest quad-core 8G memory machine, cool. Distributed build clusters were also introduced. It used to take 23 minutes, but now the time has dropped to less than 10 minutes after passing through the distribution.
more questions
Demand is constantly expanding. The size of the code swelled with it. Build times are increasing unnoticed. Until one day a few years later, I suddenly found:
1. Even if distributed, build takes an hour
2. Open the IDE and face 72 projects
3. Although it can be tolerated, it is a bit slow to do everything
4. What about the architecture? What about the architecture?
Solutions
Most of the ideas for solving these kinds of problems are still at the surface level: adding machines (improving build speed), enhancing pair programming (improving communication), writing more wikis (increasing consensus on code). However, one fact is gradually ignored, that is:
Such a huge "business requirement" cannot be carried by a project at all.
Let's start at the code level.
A large project needs to open dozens of projects in the IDE. There are inextricably linked between these projects - no matter how well the dependencies are managed, no one can clearly know how they are depended on. More importantly - most of the time you won't touch more than 60% of the project and 80% of the code. So what's the point of these codes?
Because you are in a team, others will use it.
The reference then becomes the code reference with the strongest and most vulnerable dependencies.
So what if we turn these project references into binary references (like JARs, DLLs)? Since the dependent projects are already built, the compilation time can be reduced. You also only need to focus on your own projects.
It sounds too lightweight. indeed so. How to get these binary references? For JARs, assuming a Maven dependency repository is a must; for DLLs there doesn't seem to be a mature solution but it's not too difficult.
There are a lot of implementation details in this process, and it is likely that most teams are stunned in the first step: analyzing project dependencies. It is very challenging to disassemble so many projects. Under the parallel pressure of business requirements, teams lacking courage are likely to stop there.
Are these dependencies components?
In the process of making binary references, you should keep asking yourself this question: is this dependency a component? Or is it just a simple zip package?
To evaluate whether a project is a component, there are several constraints in my opinion:
1. Are there more than 2 projects that depend on it? Note that the dependencies here are not the dependencies you specified in the IDE, but the real dependencies of API calls. For teams with poor awareness of componentization, such projects often become temporary code piles, and only useful components can be extracted through identification and migration.
2. Is it stable? The so-called stability means that in the past period of time (such as a month), this module has not undergone major adjustments, the API is basically stable, and the future changes are only to increase the number of APIs rather than adjust the structure of the API.
3. Self-reliance on the outside is less than enough.
Through this step, you can often identify the public components, public APIs, etc. used in the project. Componentize them, manage them with Maven or your own dependency library, tag them with versions, and use binary references for everyone. With this step, the build time should be drastically reduced. Through this process, it should be possible to have a clearer understanding of which are the core business logic and which are the third-party auxiliary libraries that can be considered independently. More importantly, these components can be independently developed, upgraded, and optimized without affecting the development process of the mainline.
Is a component a library or a service?
After the above step, there may still be some project dependencies in the project. These projects are often public and called through the API. For example, in a certain banking business, the payment module is depended on by many other businesses. The payment module has a lot of code and also needs to be deployed in the main process along with other modules. But the payment module is too independent, although there are some simple interactions with other domains.
One of the reasons the above approach is not suitable is that it is a runtime-only dependency - it is a service as a whole, not a static library. At this time, you can consider making it completely independent and become an independent service. Its form can be an operating system service, or an independently deployed application. Then write a set of standard lightweight APIs such as REST/WebService to interact with it. This part is also independent.
important consideration
The seemingly lightweight process above actually takes a long, long, long time in operation. One reason is that components are hard to identify. However, the reason why it is difficult to identify is not that the process is difficult, but that we tend to put everything together in the process of completing a project, at most differentiated by project but still lacks real physical isolation. This is a cognitive hurdle, especially when we're dealing with "projects" rather than "products." The word "project" itself has a short-term, purposeful meaning. The identified components themselves will not bring much benefit to the team in the short term, but will increase the workload. Like all knowledge accumulation efforts, their benefits are not causally continuous with their inputs.
What do we get?
This is not an ideal description, what we end up with is:
1. A truly physically isolated set of projects: able to independently build, develop, deploy, and upgrade
2. Dependency repository
3. Components and services with a clear division of labor
4. Product-specific versions and deployment strategies
Original link: http://michael.nona.name/archives/359