Foreword: With the emergence of cross-end frameworks such as Flutter, business development students often need to cross-end business development and problem location on Android/IOS. The construction of a new unfamiliar environment will always encounter various problems, resulting in failure of the construction, especially the IOS development environment, which is the most complicated. Not only is the environment cumbersome to set up, but the packaging speed after branching is very slow, so We designed and implemented two tools to optimize Xianyu IOS development experience.
Problems with iOS development experience
▐ Development environment is difficult to set up
Development environment depends on specific software version and complicated configuration
Xianyu iOS project not only relies on XCode, but also relies on two package management tools, taobaoenv 1.2.0 and cocoapods 1.2.0. According to everyone's experience, these two tools have fewer problems when running ruby2.3.x. Specific software versions, system built-in software version conflicts, environment variable settings, etc. series of operation steps lead to complicated environment setup, which requires assistance from iOS development students to get it done.
Difficult to maintain
After the Mac system was upgraded, cocoapod was prone to problems and had to rebuild the development environment. The specific reasons are also various: system environment variables have changed, resulting in no specific version of ruby; ruby is upgraded with the system and cocoapod cannot be used and needs to be reinstalled; Gem version problems; Ruby source problems, etc. This has also led to many developers who dare not easily upgrade the system and cannot experience the features of the new system in time.
Pod relies on large downloads
Due to the working principle of cocoapod itself, when pod updates and downloads project dependencies, it will download the file information of each version, which is very large. Taking the Xianyu iOS project as an example, a total of nearly 20G of cache files need to be downloaded, and most of them are small files of a few K. The download time may last more than ten hours, resulting in a very long time span from the establishment of a new environment to the first experience.
▐ App packaging is slow after segmentation
When development students develop in multiple branches/versions, they often need to switch branches to develop debugging and bugfixes. But after switching branches, the packaging time of the entire iOS project is about 30-40 minutes. Sometimes in order to fix a version of the bug, you have to switch branches and repackage for debugging. It may only take five minutes to fix and verify the bug, but it takes more than 30 minutes to package, and the input and output are disproportionate.
In order to solve these existing problems, we have carried out a series of explorations, and share them with you, and we welcome better solutions.
iOS environment construction
The continuous development of virtualization technology has provided us with new ideas for a unified end-side development environment. We imagine that if the iOS development environment can be decoupled from the Mac, can be transplanted, and everyone can reuse it easily, then the first and second problems will be solved. . For this we have made several attempts:
▐ Virtual machine solution
Build a virtual machine locally in MacOS, and install the MacOS system. Set up the iOS development environment in the virtual machine, and then implement the transplantation of the iOS development environment through the virtual machine image copy to solve the problem of environment construction.
This solution has the following problems:
Performance issues: The compilation process of iOS is an IO-intensive and CPU-intensive operation. The performance of the virtual machine through the disk and CPU of the virtual HOST system will be greatly reduced, resulting in a longer compilation time and affecting the development experience.
Security issue: Installing a virtual machine on a Mac work machine needs to pass the company's security audit.
Black Apple problem: The Mac system in the virtual machine is not authorized, which will bring the risk of piracy.
Virtual machine is a relatively heavy virtualization technology, so we turned to the more lightweight docker technology.
▐ Complete Docker
Dockerize all the software and environment variables that iOS development depends on. Realize the transplantation of iOS development environment through docker image. For ruby tools such as cocoapod, taobaoenv, etc., given the cross-platform nature of ruby, they can be easily migrated to docker. But for XCode that strongly relies on MacOS, we try to replace it with xcbuild produced by Facebook. This is a compilation tool compatible with xcodebuild, and there are indeed netizens using this software to build an iOS compilation environment.
This solution has the following problems:
The compatibility of Xcbuild and Xcodebuild cannot be evaluated
After xcode is upgraded, xcbuild can only fall back to the original development plan and switch the development environment back and forth during the period of time when xcbuild is following up the compatibility.
The above two solutions did not solve the problem of porting and decoupling the iOS development environment. However, in a complete dockerization attempt, we found that the most complex cocoapod and ruby installation and configuration parts can be dockerized. After xcode is installed, No special configuration is required, so we designed and implemented a compromise solution: Host development (partially dockerized)
▐ Host development (partially dockerized)
In this scheme: the development, compilation and debugging work is still in the local MacOS, using xcode; and cocoapod and taobaoenv related software and environment variable configuration are dockerized. This not only follows the development experience of the development students, but also takes into account the portability of the development environment.
In order to enable the dependency files pulled by cocoapod and the generated pod project in Docker to be recognized by the local XCode, we mount the local pod cache directory to docker, so that the dependencies pulled by the Pod can be updated in docker or in Accessed by XCode in MacOS, as shown in the following figure (unified programming plane + Faas software architecture diagram):
This not only simplifies the complexity of setting up the development environment, it is convenient for students who want to try iOS development to quickly set up the environment, and it can also give developers an undifferentiated experience. And through this solution, our iOS development environment can be easily migrated in the development environment of each student, and it can also be upgraded uniformly.
This solution migrates Pod-related dependencies to Docker and decouples it from MacOS. Therefore, iOS development students can freely upgrade the Mac system without worrying about the development environment being damaged, which solves the problem of difficult maintenance.
In order to solve the problem that the newly built environment needs to pull a large number of pod dependencies, we upload the local intermediate file of the pod to the OSS cloud disk (the blue OSS cloud disk in the above picture), and the development students only need to download the compressed package once and unzip it locally , And then incremental update is fine.
App packaging speed problem after branching
Client development students often need to develop services on multiple branches (versions), and often need to switch back and forth for service development and problem location. This brings about a problem: when the development students switch from the A branch to the B branch, the APP needs to be repackaged. The whole process takes about 30-40 minutes.
After analyzing the packaging process of Xianyu iOS project, we locked the time-consuming in two stages: Pod operation and XCode compilation. Packing speed optimization will also be divided into two stages:
▐ Pod operation acceleration
The main work of Pod install/update is to read Podfile, perform dependency version control and conflict resolution, and generate Pod projects. The generated related files are stored in the Pods directory and Pods.xcodeproj. When switching back to the previous branch, the Podfile often does not change, so regenerating the pod project is a waste.
After testing, if we save these intermediate files and switch branches multiple times, these intermediate files can still restore the previous Pod project, thus avoiding the step of regenerating the Pod project after branching and saving about 10 minutes of overhead.
▐ XCode compilation speed optimization
For the optimization of XCode compilation speed, there are many solutions on the Internet, which can be roughly divided into three categories:
Cocopods relies on compilation acceleration:
For example, cocoapods-packager, which can package pod dependencies into static libraries, and IOS projects introduce pod dependencies in the form of static libraries, saving time for repeated compilation.
However, this solution also has some problems; it is very troublesome to update private libraries and third-party libraries, each time you need to repackage the static library and upload it to the code warehouse; and it is difficult to debug the source code
Distributed compilation: such as distcc
The principle of distributed compilation is to distribute the files that need to be compiled to other machines in the compilation cluster, and then transfer the compiled binary files back. The native compiler then links these binaries together. Distributed compilation speeds up large projects obviously, but for small projects, it will slow down compilation speed.
Cache the intermediate results of compilation: CCache, BUCK
More extensive acceleration schemes are to cache the intermediate results of compilation, such as CCache, Buck, etc. These schemes have detailed information on the Internet, so I will not repeat them one by one. However, the introduction of these solutions requires the transformation of the current iOS project, and even changes the user's development habits, so it does not meet our requirements. But the solution of caching intermediate compilation results gives us some inspiration:
We know that XCode is capable of incremental compilation. This actually uses the intermediate product of the last compilation. When compiling again locally, if the file is found to be unchanged, then this file will be ignored. If the source file timestamp is updated, then Recompile this file, because the source code changes each time a small amount, so that you can achieve the purpose of speeding up the compilation.
For the Xianyu iOS project, if we save the intermediate product compiled by the current iOS branch before switching to the branch, and then restore the previously saved intermediate product when switching back to the current branch, can XCode incremental compilation be triggered? This is indeed the case.
specific plan:
1. Cache the Pods Project, Flutter Project and compiled intermediate products, Podfile.lock, linkmap and other related files of the current branch before splitting.
2. Switch branches
3. Restore the intermediate products cached before the new branch
4. Repackage the iOS APP.
Through these two steps of optimization, we reduced the packaging time of Xianyu iOS project after branching from the original 30-40 minutes to less than five minutes, and the efficiency has been increased by nearly six times.
to sum up
The complex and time-consuming steps in building the iOS environment are greatly reduced after docker mirroring and cache optimization, and the iOS novice can basically complete it within three hours.
At the same time, by caching and reusing the intermediate products generated in the packaging process, the packaging time after switching branches is controlled within five minutes, which is reduced to one-sixth of the original, which improves development efficiency.