With RAM disk technology, accelerate the speed of front-end engineering package

 

Background
with Jenkins servers, for example, when building internal project, CE deploy once every service, the fastest six minutes, most
slowly around for nearly 13 minutes. Encounter multiple concurrent projects will be packaged as resource consumption and other issues will be extended, and even there have been
more than a few 20-minute situation.

 

I often receive some Tips: for example, like this shot, the other often only made a map, but it does not say:

 

The reason
prior to understand why, let's look at history, that is, then why should Yarn. From this history, we
can analyze the reasons out slow.

Before Yarn tool does not launch, usually using NPM dependency management, early NPM it has a fatal
flaw; need to wait for a package, when fully installed , making processing of the next packet; it has no concurrency, so it not
related to the problem of high IO, the expense of latency.

When NPM encountered the same version of dependent libraries, will re-download the package again ; because it does not have the native ability to cache, so
only download it again. It also does not involve IO problem, because it sacrifices the download time and body node_modules folders
product (here quoted earlier concept, the fourth has the following description).

 

So then, we refer to the Yarn tool to improve installation speed dependent.

(1) It is the most special is that, with the lock (lock) file , where did you download a file, your team of people where to download files, avoid
free package versioning issues. I can solve the problem offline so that ah, how to you it can not work like that.

(2) 会缓存它下载的每个包,所以无需重复下载。

(3) 具备离线模式,之前安装过的包会被写入缓存目录,以后安装就直接从缓存中复制过来
这样做的本质是减少重复下载,减少不必要的网络请求。

(4) 为了减少 node_modules 体积,会对依赖次数做选举,把引用频率高的类库放到顶级目录。

 

我们已经了解,它是依赖文件缓存的方式,提升了效率问题;仅仅是做选举,就已经引发了
高频 IO 的操作,因为它要分析引用次数来回的移动目录。Webpack 打包构建情况也是类似,
具体就不在展开细说。

 

现象
所以我们有了一个印象,前端工程下载后,会做很多 IO 操作。这对 SSD 磁盘很有效。如果
是机械硬盘,遇到高频读写,性能就会很慢。这是 Yarn 在做依赖库选举而优化磁盘占用时,
机械硬盘所消耗的时间耗时 13 分钟:


这种情况我们很难避免,只有几种选择:
1. 后退,使用 NPM 工具,选择等待牺牲网络下载时间,这条路走不通。
2. Jenkins 更换 SSD 磁盘,更换硬件实际上是最好的方案,短期内也走不通。
3. 优化 Yarn 依赖库的选举方案,Yarn PnP 还不太成熟,我们还在调研中,有坑,也走不通。


解决方案
当时的情况是,正常的方案都无法走通了。直到有一天我的同事提供一个思路,说他当年在 Windows
下片时,为了加快 Copy 速度把内存当磁盘用了,Windows 设置这个东西很简单。你们看看
Linux 支不支持。随后我们就开始调研,本机测试后发现可行。

然后我们以线下的的环境做试点,部署脚本改好了测试近一周,发现可行。

 

优化前,最高 23 分钟(开篇第一张图),现在平均 3 分钟

另一个项目优化前,平均 8 分钟:

优化后,平均 49 秒

 

本次主要是给大家,提供一个解决 IO 问题的思路。我们使用的是 RAM disk 中的 temfs
方案。技术对比看下方链接的文章就行,很简单:

 

参考链接
https://baike.baidu.com/item/tmpfs/1476960?fr=aladdin
https://blog.csdn.net/wz947324/article/details/80007122

Guess you like

Origin www.cnblogs.com/wubaiqing/p/11460766.html