As we all know, the competition of apps has reached a white-hot stage where user experience is king and quality is the top priority. Users are very picky. If a company's promotion team finally spends a lot of money to promote an APP, and finally has some users, due to an online bug, a batch of users have flashback bugs in use. All the money was wasted, and the most important thing is that the reputation is not good, and the number of users will not be able to increase in the future. Calm down and analyze the cause of the problem. It is nothing more than that the quality has not passed the test and went online. Apart from some subjective factors, I think most of the objective factors can be prevented by us. According to a set of development specifications proposed by the great gods, CI + FDD can help us solve the objective factors to a great extent. This article focuses on Continuous Integration Continuous Integration (CI for short)
content
1. Why we need continuous integration
2. Continuous Integration Tool - Jenkins
3. iOS automated packaging commands - xcodebuild + xcrun and fastlane - gym command
4. Packaging completes automatic upload to fir / Dandelion third-party platform
5. Complete continuous integration process
6.Jenkins + Docker
1. Why do we need continuous integration
When it comes to why we need it, we need to start with what is. So what is continuous integration?
Continuous integration is a software development practice: many teams integrate their work frequently, each member typically doing daily integrations, and then multiple integrations per day. Each integration is automatically built (including testing ) to detect errors as quickly as possible. Many teams find this approach can significantly reduce integration problems and make team development faster.
CI is a development practice. Practice should consist of 3 basic modules, a process that can automatically build, automatically compile code, and can automatically distribute, deploy and test. A code repository, SVN or Git . The last one is a continuous integration server. Through continuous integration, we can obtain and respond to product feedback with high frequency through automation and other means.
So what benefits can continuous integration bring us? Here is an article recommended , which divides Continuous integration (CI) and test-driven development (TDD) into 12 steps. However the benefits are multiplied and there are 24 benefits.
Let me talk about some of the advantages that I have experienced after using CI.
1. Shorten the development cycle and quickly iterate versions
The development cycle is estimated at the beginning of each version, but it is always delayed due to various things. This includes some objective factors. Due to the increase in product lines, the iteration speed is getting faster and faster, and the pressure on testing is also increasing. If the tests are all tested after the development is fully developed, it will affect for a long time. At this time, the rhythm of the project will be seriously slowed down due to the late integration. If we can achieve continuous integration as soon as possible and enter the 12-step iterative cycle as shown in the figure above, we can expose problems as soon as possible, solve them early, and try our best to complete the task within the specified time.
2. Efficiency brought by automated assembly line operations
In fact, packaging is a very time-consuming and non-technical work for developers. If there are more developers, the chances of conflicting code changes will be greater. In addition, there is no production line management mechanism, and it is difficult to guarantee the code quality of the code warehouse. The team will spend some time to resolve the conflict, and after resolving the conflict, you need to manually package it yourself. At this time, if the certificate is incorrect, it will be delayed for a long time. This time can actually be saved with continuous integration. One or two days is not much, but it can save a lot of time by calculating it in units of years!
3. Ready to deploy
With continuous integration, we can package by day. The biggest advantage of this high-frequency integration is that it can be deployed online at any time. In this way, it will not lead to the imminent launch, with loopholes and bugs everywhere, and it will not be able to be deployed after messing around, which will seriously affect the launch time.
4. Minimize low-level mistakes
We can make mistakes, but making low-level mistakes is bad. The low-level errors referred to here include the following: compilation errors, installation problems, interface problems, and performance problems.
Continuous integration in units of days can quickly find compilation problems, and automatic packaging cannot pass directly. After the package is finished, the test scan code cannot be installed, and this problem will be exposed immediately. Interface problems and performance problems are found with automated test scripts. These low-level problems are exposed by continuous integration, reminding us to avoid low-level errors.
2. Continuous Integration Tool - Jenkins
Jenkins is an open source project that provides an easy-to-use continuous integration system that frees developers from complicated integration and focuses on more important business logic implementations. At the same time, Jenkins can monitor the errors existing in the integration, provide detailed log files and reminder functions, and visualize the trend and stability of project construction in the form of charts.
According to the official definition, Jenkins has the following uses:
-
Build the project
-
Run test cases to detect bugs
-
Static code detection
-
deploy
Regarding these 4 points, it is more convenient in actual use:
1. Automatic packaging of construction projects can save developers a lot of time. Importantly, Jenkins maintains a set of high-quality and usable code for us, and ensures a pure environment. We often encounter packaging failures due to local configuration errors. Now Jenkins is a fair judge. It can't compile the ipa correctly, that is, there is a compilation error or a configuration problem. There is no need for developers to argue that it can be run locally, and after pulling the code of so-and-so, it cannot run. Work together to maintain the normal compilation of Jenkins, because the compilation environment of Jenkins is much simpler than our local, and it is the most pure and pollution-free compilation environment. Developers just have to focus on coding. This is a convenience for developers.
2. This can be used for automated testing. Generate large batches of test cases locally. Use the server to run these use cases continuously every day. Run each interface every day. It may seem unnecessary, but in fact a system that works fine today may very well have problems tomorrow due to code changes made today. With Jenkins, regression testing can be performed in units of days. As long as the code is changed, Jenkins will run all the regression test use cases. In the case of a tight project schedule, in many cases, the test does not pay much attention to the regression test. After all, it is likely to be a futile "useless effort" after one test. However, due to the untimely regression testing, the system is unavailable when the final version is released. At this time, it is time-consuming to go back and find the reason. Checking the submission records and seeing hundreds of submission records is also a headache to troubleshoot. Day-by-day regression testing finds problems immediately. Testers can focus on unit testing every day and manually regression testing once a week. This is a convenience for testers.
3. This is static code analysis, which can detect many code problems, such as potential memory leaks. Due to the purity of the environment where Jenkins is located, some problems that cannot be found in our local complex environment can still be found, and the code quality can be further improved. This is a convenience for quality inspection.
4. Deploy at any time. Jenkins can set subsequent operations after the packaging is completed. At this time, it is often to submit the app to the system that runs the test case, or deploy it to the internal testing platform to generate a QR code. Some low-level problems such as inability to install during deployment are immediately exposed. Testers also only need to scan the QR code to install, which is very convenient. This is also a convenience for testing.
The following example uses Weekly Release 2.15 on 2016-07-24 22:35 as an example.
Let's start installing Jenkins. Download the latest pkg installation package from the official website https://jenkins.io/ .
You can also download jenkins.war, and then run java -jar jenkins.war to install it.
After the installation is complete, Safari may open automatically. If it does not open automatically, open the browser and enter http://localhost:8080
An error may be reported at this time. If this problem occurs. The reason for this problem is that there is a problem with the Java environment, and the Java environment can be reset.
At this time, if you restart the computer, you will find that Jenkins has added a user for you, the name is Jenkins, but you do not know the password at this time. You may try the password, which is definitely not right, because the initial password is very complicated. The correct way at this time is to open http://localhost:8080 and the interface to reset the initial password will appear as shown in the figure below.
Follow the prompts and find the /Users/Shared/Jenkins/Home/ directory. Although this directory is a shared directory, it has permissions. Non-Jenkins users have no read and write permissions in the /secrets/ directory.
Open the initialAdminPassword file, copy the password, and then fill in the webpage to reset the password. As shown below
Install it all the way, enter the user name, password, and email, even if the installation is complete.
Or continue to log in to localhost:8080, select "System Management" - "Manage Plugins", we have to install some auxiliary plugins first.
Install the GitLab plugin
Because we use GitLab to manage source code, Jenkins itself does not come with a GitLab plugin, so we need to select System Management -> Manage Plugins in turn, and select "GitLab Plugin" and "Gitlab Hook Plugin" in "Optional Plugins" These two items are then installed.
Install the Xcode plugin
The same as the steps for installing the GitLab plugin, we select System Administration -> Manage Plugins in turn, and select "Xcode integration" in "Optional Plugins" to install.
With this installed, we can configure a build project.
Click on the newly created project to configure the General parameters.
Here you can set the number of days and days to keep the package.
Then set up source management .
Since I am using GitLab now, first configure the SSH Key and add SSH in the certificate management of Jenkins. On the Jenkins management page, select "Credentials", then select "Global credentials (unrestricted)", click "Add Credentials", as shown in the figure below, we fill in our SSH information, and then click "Save" to add SSH to the in the global domain of Jenkins.
If the normal configuration is correct, the red warning in the figure below will not appear. If there is a prompt in the figure below, it means that Jenkins has not connected to GitLab or SVN, then please check whether the SSH Key is configured correctly.
Build Trigger Settings This is where the automated tests are set up. There is a lot of content involved here, and I haven't studied it in depth for the time being, so I won't set it here for the time being. Those with automated testing needs can study the settings here.
However, there are two configurations that still need to be configured
Poll SCM (poll source code management) Poll source code management
You need to set the path of the source code to have the effect of polling. Typically set to something like: 0/5 poll every 5 minutes
Build periodically (定时build)
The general setting is similar to: 00 20 * Execute scheduled build at 20:00 every day. Of course, the settings for both are the same.
The format is like this
Minute (0-59) Hour (0-23) Date (1-31) Month (1-12) Day of the week (0-7, 0 and 7 are both Sundays) ( see here for more detailed settings )
Build environment settings
iOS packaging requires signature files and certificates, so in this section we check "Keychains and Code Signing Identities" and "Mobile Provisioning Profiles".
Here we need to use the Jenkins plugin again. On the system management page, select "Keychains and Provisioning Profiles Management".
Go to the Keychains and Provisioning Profiles Management page, click the "Browse" button, and upload your own keychain and certificate respectively. After the upload is successful, we specify the name of the signature file for the keychain. Click "Add Code Signing Identity", and finally add it successfully, as shown in the following figure:
Note: The first time I imported the certificate and Provisioning Profiles file, I encountered a small "pit". I thought I needed a certificate, but the Keychain required here is not a cer certificate file. This Keychain is actually in /Users/admin username/Library/keychains/login.keychain. After setting this Keychain, Jenkins will copy this Keychain to /Users/Shared/Jenkins/Library/keychains, (Library is a hidden file). The Provisioning Profiles file is also copied directly to the /Users/Shared/Jenkins/Library/MobileDevice file directory.
In this way, the Adhoc certificate and signature files are configured in Jenkins, and then we only need to specify the relevant files in the item settings.
Go back to our newly created item, find the build environment , and select your own relevant certificate and signature file as shown below.
Next in the build settings
We choose to execute a package script here. Scripts are explained in detail in the next chapter.
Post-build actions
Here we choose Execute a set of scripts, here is also a script, this script is used to upload the automatically packaged ipa file. Scripts are explained in detail in Chapter 4.
At this point, our Jenkins setup is complete. Click Build to start building the project.
Build once, and the meanings of each color are as follows:
The weather barometer represents the quality of the project, which is also a feature of Jenkins.
If the build fails, you can go to the Console Output to view the log log.
Three, iOS automation packaging commands - xcodebuild + xcrun and fastlane - gym command
In daily development, packaging is an indispensable part of the final launch. If you need to package the project into an ipa file, the usual method is to click "Product -> Archive" in Xcode, and when the entire project is archived, then automatically pop up " Organizer", export ad hoc, enterprise type ipa package as needed. Although Xcode can do the packaging perfectly, it still requires us to manually click 5 or 6 times. Coupled with the fact that we now need continuous integration, automating execution with packaging commands is a natural need.
1. xcodebuild + xcrun command
Xcode provides a set of build and package commands for our developers, which is xcodebuild
and the xcrun command. xcodebuild packages our specified projects into .app files, and xcrun converts the specified .app files into corresponding .ipa files.
The specific documents are as follows: xcodebuild official document , xcrun official document
| 1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
The most important of the above 10 commands are the first 3.
Next, let's explain the parameters:
-project -workspace: These two correspond to the name of the project. If there are multiple projects, and it is not specified here, the default is the first project.
-target: package the corresponding targets, if not specified, this defaults to the first one.
-configuration: If this configuration is not modified, the default is the Debug and Release versions, and the default is the Release version if not specified.
-buildsetting=value ...: Use this command to modify the configuration of the project.
-scheme: Specifies the packaged scheme.
These are the most basic commands.
上面10个命令的第一个和第二个里面的参数，其中 -target
和 -configuration 参数可以使用 xcodebuild -list
获得，-sdk 参数可由 xcodebuild -showsdks
获得，[buildsetting=value ...] 用来覆盖工程中已有的配置。可覆盖的参数参考官方文档：Xcode Build Setting Reference。
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
上面第3个命令就是专门用来打带有Cocopods的项目，因为这个时候项目工程文件不再是xcodeproj了，而是变成了xcworkspace了。
再来说说xcrun命令。
| 1 2 3 4 5 6 7 8 9 |
|
参数不多，使用方法也很简单，xcrun -sdk iphoneos -v PackageApplication + 上述一些参数。
参数都了解之后，我们就来看看该如何用了。下面这个是使用了xcodebuild + xcrun命令写的自动化打包脚本
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
2. gym 命令
说到gym，就要先说一下fastlane。
fastlane是一套自动化打包的工具集，用 Ruby 写的，用于 iOS 和 Android 的自动化打包和发布等工作。gym是其中的打包命令。
fastlane的官网看这里, fastlane的github看这里。
要想使用gym，先要安装fastlane。
| 1 |
|
-
fastlane包含了我们日常编码之后要上线时候进行操作的所有命令。
-
deliver：上传屏幕截图、二进制程序数据和应用程序到AppStore
-
snapshot：自动截取你的程序在每个设备上的图片
-
frameit：应用截屏外添加设备框架
-
pem：可以自动化地生成和更新应用推送通知描述文件
-
sigh：生成下载开发商店的配置文件
-
produce：利用命令行在iTunes Connect创建一个新的iOS app
-
cert：自动创建iOS证书
-
pilot：最好的在终端管理测试和建立的文件
-
boarding：很容易的方式邀请beta测试
-
gym：建立新的发布的版本，打包
-
match：使用git同步你成员间的开发者证书和文件配置
-
scan：在iOS和Mac app上执行测试用例
整个发布过程可以用fastlane描述成下面这样
| 1 2 3 4 5 6 7 8 9 10 11 |
|
Ps：这里可能大家还会听过一个命令叫 xctool
xctool是官方xcodebuild命令的一个增强实现，输出的内容比xcodebuild直观可读得多。通过brew即可安装。
| 1 |
|
使用gym自动化打包，脚本如下：
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
|
四、打包完成自动化上传 fir / 蒲公英 第三方平台
要上传到 fir / 蒲公英 第三方平台，都需要注册一个账号，获得token，之后才能进行脚本化操作。
1. 自动化上传fir
安装fir-clifir的命令行工具
需要先装好ruby再执行
| 1 2 3 |
|
2.自动化上传蒲公英
| 1 2 3 4 5 6 7 8 9 10 |
|
五、完整的持续集成流程
经过上面的持续化集成，现在我们就拥有了如下完整持续集成的流程
六、Jenkins + Docker
关于Jenkins的部署，其实是分以下两种：
单节点（Master）部署
这种部署适用于大多数项目，其构建任务较轻，数量较少，单个节点就足以满足日常开发所需。
多节点(Master-Slave)部署
Projects with large scale, frequent code submissions (meaning frequent builds), and high automated testing pressure will adopt this deployment structure. Under this deployment structure, the Master usually only acts as a manager, responsible for task scheduling, slave node management, and task status collection, and specific construction tasks are assigned to slave nodes. There is no upper limit on the number of slave nodes that a Master node can manage in theory, but usually with the increase of the number, its performance and stability will decrease to varying degrees, and the specific impact will vary depending on the performance of the Master hardware.
However, multi-node deployment will have some defects. When the test cases become massive, it will cause some problems, so someone designed the following deployment structure, Jenkins + Docker
Since the author's current project is still in single-node (Master) deployment, there is no practical experience in multi-node (Master-Slave) deployment, and the improved version of Docker has not been touched. However, if there are such a large number of test cases, high pressure For a large number of complex regression testing needs, I recommend everyone to read this article.