Abstract: Ali has many R&D teams. The release process and branching strategy used by different business units are not uniform, but they are relatively regular in general. Among them, there is a mainstream release mode and the corresponding branch usage method, called "AoneFlow". This work mode has a unique idea and is rarely seen outside of Alibaba. This article focuses on these practices and discusses the topic of branch management.
introduction
Inside Ali, there are many interesting engineering practices. Some practices are embedded in the general environment of the group through tools and processes, which are not easy to copy outside, while some practices are revealed in everyone's daily habits and are silently followed. For example, branch management is actually a tool and a habit, and it is quite a component of Ali's characteristics, which is suitable as an example. Ali has many R&D teams, and the release process and branching strategy used by different business units are not uniform, but they are relatively regular in general. Among them, there is a mainstream release mode and the corresponding branch usage method, called "AoneFlow". This work mode has a unique idea and is rarely seen outside of Alibaba. This article focuses on these practices and discusses the topic of branch management.
fine-grained branching mode
When it comes to branch management mode, we are most familiar with TrunkBased and GitFlow.
The TrunkBased mode is a working method advocated by the idea of continuous integration. It consists of a single trunk branch and many release branches. Each release branch is created from the trunk at the commit point of a specific version for online deployment and Hotfix. In TrunkBased mode, there are no explicit feature branches. Of course, in fact, the distributed feature of Git inherently allows everyone to have local branches, and TrunkBased does not exclude the existence of short-term feature branches, but when talking about this mode, people usually do not explicitly emphasize it.
While there have been many good examples in recent years, the TrunkBased model has not ruled the world. Its shortcomings are obvious, too many teams are working on the trunk at the same time, and disaster may occur when it is released (especially when multiple versions are developed in parallel). The compensating measure is FeatureToggle, frequent integration and sufficient test coverage, which puts higher requirements on the ability of the development team. At present, the TrunkBased model is mainly used for SaaS projects that do not need to maintain multiple historical versions at the same time, especially for various small services transformed by microservices.
There are two common evolutions of the TrunkBased pattern. The OneFlow mode refers to many ideas of TrunkBased, defines the operation process more strictly, and adds the Hotfix branch and so on. The multi-trunk mode (usually dual trunks, fixed development branches and fixed release branches) is a special case of TrunkBased using fixed release branches. It was introduced in the article on improving the team's ability to implement microservices and will not be repeated here.
The GitFlow pattern is the culmination of several patterns, including a trunk branch, a development branch, many feature branches, many release branches and hotfix branches, and many tedious merge rules. It has a Git plugin, but it's long since unmaintained. Due to the clear definition of each operation at each stage, it used to be a sweet potato in the eyes of many process-oriented companies. But it is not very easy to use, a lot of merge conflicts and unfriendly integration tests are also the places where it is most criticized.
Yes, there is also the GithubFlow mode, but this strategy is nothing more than adding the operation of merging code in a personal repository and Pull Request on the basis of TrunkBased, which is similar to adding a personal branch in the same repository. In a practical sense , it is more suitable for distributed teams. There are also evolutions of GithubFlow, such as the GitlabFlow pattern that emphasizes multi-environment deployment and associating repositories or branches with environments.
Either as simple and crude as TrunkBased, or as complicated as GitFlow. Is there really no other choice?
A different approach to AoneFlow
You can see shadows of many other branching patterns on AoneFlow. It basically takes into account the "ease of continuous integration" of TrunkBased and the "ease of management requirements" of GitFlow, while avoiding the red tape of GitFlow.
Look at the specific routine. AoneFlow uses only three branch types: trunk branch, feature branch, release branch, and three basic rules.
Rule number one, create a feature branch from trunk before starting work.
The feature branch of AoneFlow is basically borrowed from GitFlow, and there is nothing special about it. Whenever you start a new work item (such as a new feature or an open problem), create a feature branch, usually named with the feature/ prefix, from the trunk representing the latest released version, and then on this branch Submit code changes. That is to say, each work item (which can be completed by one person or by multiple people) corresponds to a feature branch, and all modifications are not allowed to be directly submitted to the trunk.
Rule 2: By merging feature branches, a release branch is formed.
The release branch design of AoneFlow is very clever, which can be described as the essence of the whole system. GitFlow first merges the completed feature branch back into the public mainline (i.e. the development branch), and then pulls the release branch from the public mainline. TrunkBased also waits for all required features to be developed on the trunk branch, and then pulls the release branch from a specific location on the trunk branch. The idea of AoneFlow is to pull a new branch from the trunk, and merge all the feature branches to be integrated or released this time in turn to obtain the release branch. Release branches are usually named with the release/ prefix.
This rule is very simple, but the actual gameplay is quite rich.
First, the purpose of release branches can be very flexible. The basic gameplay is to correspond each release branch to a specific environment, such as the release/test branch corresponding to the deployment test environment, the release/prod branch corresponding to the online formal environment, etc., and combined with the pipeline tool to connect the code in each environment Quality scanning and automated testing levels, and publish the resulting deployment package directly to the corresponding environment. The gameplay of the advanced point is to correspond a release branch to multiple environments, such as stringing together the grayscale release and the official release, and adding manual acceptance steps in the middle. For advanced gameplay, if you associate feature branches according to the iterative plan, create a fixed release branch that evolves iteratively, and then string a series of environments on the pipeline of this release branch, it will taste a bit like a classic continuous integration pipeline. Or make a release branch that links all feature branches together, and is dedicated to integration testing of all commits, and the effect of TrunkBased is played. Of course, these fancy advanced gameplays are my imagination, and Ali's release branches are generally quite satisfactory.
Second, the feature composition of a release branch is dynamic, making it particularly easy to adjust. In some Internet companies whose market is changing rapidly, as well as Party B's companies adopting "agile operation", this situation is often encountered. The demand that has been completed and is waiting to go online may be due to market strategy adjustment or a temporary decision of Party A at any time. A feature suddenly requires a delayed release or no longer at all. Or a certain feature is found to have serious development problems before it goes live, which needs to be eliminated. According to the usual practice, at this time, it is necessary to manually "tick the code", and remove the relevant commits that have been merged into the development branch or the main branch one by one. Students who have done it know that it is very troublesome. In the AoneFlow mode, rebuilding the release branch is a matter of minutes. Delete the original release branch, pull out a new release branch with the same name from the trunk, and merge the feature branches that need to be retained. This series of actions can be automated to a large extent, and it will not leave a bunch of records of culling code in the warehouse, which is clean and pollution-free.
In addition, the release branches are loosely coupled, so that there can be multiple integration environments for integration testing of different feature combinations, and it is also convenient to manage the timing of deploying each feature into different environments. Loose coupling does not mean that there is no correlation. Since the release processes such as the test environment, integration environment, pre-release environment, grayscale environment, and online formal environment are usually carried out sequentially, the process can only require features that have passed the verification of the previous environment. , it can be passed to the next environment for deployment, forming a funnel-shaped feature release flow. Ali has a unified platform to automate the migration of feature combinations between release branches, which will be introduced in the tool section below.
Rule 3: After publishing to the online official environment, merge the corresponding release branch into the trunk, add tags to the trunk, and delete the feature branch associated with the release branch.
When a pipeline on a release branch completes the deployment of an online formal environment, it means that the corresponding function is actually released. At this time, the release branch should be merged into the trunk. In order to avoid accumulating a large number of historical feature branches in the code repository, you should also clean up some feature branches that have been launched. Similar to GitFlow, the latest version on the trunk branch is always the same as the online version. If you want to go back to the historical version, you only need to find the corresponding version tag on the trunk branch.
In addition to the basic rules, there are some unwritten tips in practice. For example, for a hotfix after going online, the normal processing method should be to create a new release branch corresponding to the online environment (equivalent to the Hotfix branch), and create a temporary pipeline for this branch to ensure necessary pre-release inspections and smoke tests can be executed automatically. But in fact, there is a simple way to clear all the feature branches associated with the release branch corresponding to the online official environment, make changes directly on this release branch, and use the ready-made pipeline to automatically release the changes. What if you have to fix a bug in a historical version? Then honestly find the version label location in the trunk branch, and then create a Hotfix branch from that location. However, since most of Alibaba's products are online SaaS businesses, such scenarios are rare.
It is these simple rules that make up the unique core routine of AoneFlow.
Every seemingly simple step in AoneFlow is not imaginary, but the experience accumulated by a large number of product teams repeatedly tempered. Next, I will talk about the technical threshold of AoneFlow and Alibaba's internal response.
Experience optimization of AoneFlow
Those who are familiar with martial arts know that to master the housekeeping martial arts of a sect, in addition to being able to move, you must also have deep internal skills and handy weapons. Otherwise, if you take the Evil Exorcism Sword Manual, you can only look at the spectrum and sigh.
The internal strength and weapons of the Ali team are actually good coding habits and complete supporting tools.
The habits mentioned here, in addition to the development process and the management of code branches, also include some conventions in daily development. Many of Alibaba's development protocols are recorded in "documents", which are mainly included in the "Alibaba Java Development Manual". Its contents have now been made public, so it has long been considered a secret.
Give a concrete example. In the AoneFlow process, each time the release branch is rebuilt, the code is re-merged and then compiled to generate a new deployment package. However, even if the content of the code is the same, if the project depends on some third-party packages that will change, it may still lead to inconsistent behavior of the packaged product. Therefore, in Ali's code specification, it is clearly pointed out that the code used for online release cannot use the dependency package that contains the "SNAPSHOT version" (ie, the version that has not been officially released), so as to ensure that every time the built product is built is consistent. There are many details like this, and good development habits are a necessary prerequisite to ensure software quality.
Tools can make team collaboration smoother. Although as long as you understand the principle, each branch creation, merge, and change step in AoneFlow can be played with simple Git commands. But some of these operations (such as picking the right combination of feature branches for each release branch to merge in) are extremely error-prone to perform manually, and it's not pleasant to have individuals on the team repeat the commands of these daily chores.
Within Alibaba, teams using the AoneFlow process basically do not need to run Git themselves to handle branching matters, but are taken over by the collaborative R&D platform (hereinafter referred to as the platform) called Aone within Alibaba Group. This platform, which undertakes the complete R&D process of 80% of the group's products from the proposal of requirements and user stories to the deployment and launch, has built-in many R&D efficiency tools embedded in the form of service components, among which the publishing components add a lot to the user experience of AoneFlow. The more significant auxiliary "effects" include the following aspects.
The first is the automation of the overall process.
Being an internal tool, the functionality of the platform is highly cohesive. For the project, from the original requirement, the requirement is divided into tasks, and then the feature branch is created online according to the task, and then the release branch is generated by aggregation, and the test environment is automatically created according to the template, until the later operation and maintenance guarantee can be one-stop. get it done.
This process has gone far beyond the scope of code branch management. But it is precisely because of this that for AoneFlow, the platform associates feature branches with requirements items forward to ensure the naming standard of feature branches; backwards associates release branches with deployment behaviors to ensure that each environment version reliability of the source. It has opened up the two veins of Ren and Du of end-to-end delivery.
Second is the pipeline of release branches.
As a means of process automation, CI/CD pipelines are a common standard practice in many modern delivery teams. There are many branches involved in AoneFlow's code life cycle, and when these branches are created or updated, a series of other actions are often required. Pipelines can connect these code branches in the daily development process to the deep intentions they express (such as committing code for integration testing). Especially the release branch, each release branch of AoneFlow is usually associated with a specific deployment environment. When new code is merged into the branch, the code should be checked and deployed in time.
Ideally, each distinct branch should have a pipeline that matches its role to serve it. The release branch of AoneFlow is relatively fixed, so it is easier to do continuous integration than GitFlow. In theory, any pipeline tool can be used with AoneFlow. However, Ali's unified platform provides pipeline integration of code review, security inspection, online deployment and other functions, which has greatly improved the use and optimization of AoneFlow's internal team.
Another useful aid is the management of branch associations.
The maintenance of the relationship between feature branches and release branches is an AoneFlow-specific problem. Remembering which feature branch each release branch came from makes sense when you need to make changes based on an existing feature mix. For example, when a feature needs to be withdrawn from a specific release branch, the branches where other features except the feature are located are usually merged once to replace the original release branch. It is not easy to manually record this information, and it will be much more convenient to display and assist through the platform.
When some functional combinations are verified in a low-level release environment (such as an integration test environment), we want to migrate its content directly to the corresponding release branch in a high-level environment (such as a pre-release environment). This ensures that the online version must be pre-release verified, the pre-release version must be integrated verification, and so on, so that each release branch forms a series. Again, this can be done using normal Git commands, but using a visual tool will make the process more intuitive.
In addition, the platform provides a unified display of the status of each branch of the code warehouse, including the machine information and operation records of the deployment environment corresponding to the branch at a glance. It is these "high value-added" aids that enable AoneFlow to take advantage of its strengths and circumvent weaknesses, and become the first choice for the Ali team to support complex projects.
write at the end
There is no absolute right or wrong in choosing a code branching mode, the key is to match the scale and release cadence of the project. After many practical experiences, Alibaba's collaborative R&D platform has summed up a set of original branch management methods. Through a flexible, efficient and simple and practical process, the delivery of many products under Alibaba is guaranteed. When you are still hesitating about the dazzling branch mode, not only reluctant to develop the parallel features of GitFlow, but also to the continuous integration friendliness of TrunkBased, AoneFlow may be a choice worth considering.