How to speed up compilation of C++ code

  C++ code has always faced the world with its high-performance runtime, but when it comes to compilation speed, it is only low-key. For example, the source code I am currently working on, even if I use Incredibuild to mobilize nearly 100 machines, a complete build will take four hours. Horror! ! ! Although it is usually not necessary to do a complete build locally, compiling a few related projects is enough for you to wait for a long time (foreigners call this monkey around, which is quite vivid). Think of the scene of working on a single-core 2.8GHZ for a few years - put a book in front of you, click the build button, and read the book for a while~~~ The past is unbearable.

It is conceivable that if you do not pay attention to it, the compilation speed is very likely to become a bottleneck in the development process. So why does C++ compile so slowly?

I think one of the most important reasons should be the basic "header-source" compilation model of C++:

1. As a compilation unit, each source file may contain hundreds or even thousands of header files, and in each compilation unit, these header files will be read from the hard disk once and then parsed. 

2. Each compilation unit will generate an obj file, and then these obj files will be linked together, and this process is difficult to parallelize. 
The problem here is the repeated loading and parsing of countless header files, and the intensive disk operations.


The following are some ways to speed up the compilation from various angles, mainly for the key issue raised above.

First, the code angle

1. Use forward declarations in header files instead of including header files directly.

Don't think that you just added an extra header file, the effect may be magnified infinitely due to the "included" nature of header files. So, do everything possible to keep the header file as compact as possible. In many cases, pre-declaring a class in a namespace will be more painful, and direct include will be much more convenient. Be sure to resist this temptation; class members, function parameters, etc. should also use references and pointers as much as possible to create pre-declaration. condition.

2. Use Pimpl mode

Pimpl is called Private Implementation. The interface and implementation of the traditional C++ class are confused, and Pimpl makes the interface and implementation of the class completely separated. In this way, as long as the public interface of the class remains unchanged, the modification to the implementation of the class always only needs to compile the cpp; at the same time, the header file provided by the class to the outside world will be much simplified.

3. Highly modular

Modularity is low coupling, which means reducing interdependence as much as possible. There are actually two levels of meaning here. One is that the change of a header file between files should not cause the recompilation of other files; the second is that between projects, the modification of one project should not cause the compilation of too many other projects. This requires that the header file or the content of the project must be single, and don't stuff everything into it, causing unnecessary dependencies. This can also be called cohesion.

Take the header file as an example, don't put two unrelated classes or macro definitions that are not related into one header file. The content should be as simple as possible so that the file containing them does not contain unwanted content. I remember we once did such a thing, find out the most "hot" header files in the code, and then divide them into multiple independent small files, the effect is quite impressive.

In fact, the refactoring we did last year, separating many DLLs into two parts, UI and Core, also has the same effect - improving development efficiency.

4. Remove redundant header files

Some codes have been developed and maintained for the past ten years and have been handled by countless people. It is very likely that useless header files may be included, or the phenomenon of repeated inclusion will occur. It is quite necessary to remove these redundant includes. Of course, this is mainly for cpp, because for a header file, it is difficult to define whether a certain include is redundant, it depends on whether it is used in the final compilation unit, and this may appear in a compilation unit. , but not used in another compilation unit.

Before, I wrote a Perl script to automatically remove these redundant header files. In a certain project, more than 5,000 includes were removed.

5. Pay special attention to inline and template

These are two of the more "advanced" mechanisms in C++, but they force us to include the implementation in the header file, which contributes a lot to increasing the content of the header file and thus slowing down the compilation. Before using it, weigh it.

2. Comprehensive skills

1、预编译头文件(PCH)

把一些常用但不常改动的头文件放在预编译头文件中。这样,至少在单个工程中你不需要在每个编译单元里一遍又一遍的load与解析同一个头文件了。

2、Unity Build

Unity Build做法很简单,把所有的cpp包含到一个cpp中(all.cpp) ,然后只编译all.cpp。这样我们就只有一个编译单元,这意味着不需要重复load与解析同一个头文件了,同时因为只产生一个obj文件,在链接的时候也不需要那么密集的磁盘操作了,估计能有10x的提高,看看这个视频感受一下其做法与速度吧。

3、ccache

compiler cache, 通过cache上一次编译的结果,使rebuild在保持结果相同的情况下,极大的提高速度。我们知道如果是build,系统会对比源代码与目标代码的时间来决定是否要重新编译某个文件,这个方法其实并不完全可靠(比如从svn上拿了上个版本的代码),而ccache判断的原则则是文件的内容,相对来讲要可靠的多。

很可惜的是,Visual Studio现在还不支持这个功能 - 其实完全可以加一个新的命令,比如cache build,介于build与rebuild之间,这样,rebuild就可以基本不用了。

4、不要有太多的Additional Include Directories

编译器定位你include的头文件,是根据你提供的include directories进行搜索的。可以想象,如果你提供了100个包含目录,而某个头文件是在第100个目录下,定位它的过程是非常痛苦的。组织好你的包含目录,并尽量保持简洁。

三、编译资源

要提高速度,要么减少任务,要么加派人手,前面两个方面讲得都是减少任务,而事实上,在提高编译速度这块,加派人手还是有着非常重要的作用的。

1、并行编译

买个4核的,或者8核的cpu,每次一build,就是8个文件并行着编,那速度,看着都爽。 要是你们老板不同意,让他读读这篇文章:Hardware is Cheap, Programmers are Expensive

2、更好的磁盘

我们知道,编译速度慢很大一部分原因是磁盘操作,那么除了尽可能的减少磁盘操作,我们还可以做的就是加快磁盘速度。比如上面8个核一块工作的时候,磁盘极有可能成为最大的瓶颈。买个15000转的磁盘,或者SSD,或者RAID0的,总之,越快越好。

3、分布式编译

一台机子的性能始终是有限的,利用网络中空闲的cpu资源,以及专门用来编译的build server来帮助你编译才能从根本上解决我们编译速度的问题,想想原来要build 1个多小时工程的在2分钟内就能搞定,你就知道你一定不能没有它 - Incredibuild。

4、并行,其实还可以这么做。

这是一个比较极端的情况,如果你用了Incredibuild,对最终的编译速度还是不满意,怎么办?其实只要跳出思维的框架,编译速度还是可以有质的飞跃的 - 前提是你有足够多的机器:

假设你有solution A和solution B,B依赖于A,所以必须在A之后Build B。其中A,B Build各需要1个小时,那么总共要2个小时。可是B一定要在A之后build吗?跳出这个思维框架,你就有了下述方案:

◦同时开始build A和B 。 

◦A的build成功,这里虽然B的build失败了,但都只是失败在最后的link上。

◦重新link B中的project。

这样,通过让A的build与B的编译并行,最后link一下B中的project,整个编译速度应该能够控制在1个小时15分钟之内。


另外,这本书谈了很多这方面的内容:大规模C++程序设计


参考:

转载自:

http://blog.csdn.net/linear_luo/article/details/52699119

http://www.codeweblog.com/%E5%A6%82%E4%BD%95%E5%8A%A0%E5%BF%ABc-%E4%BB%A3%E7%A0%81%E7%9A%84%E7%BC%96%E8%AF%91%E9%80%9F%E5%BA%A6/



Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325724756&siteId=291194637