treeshaking
The essence of treeshaking is to remove redundant code algorithms. In front-end performance optimization, es6 introduces the tree shaking mechanism. Tree shaking means that when we introduce other modules into the project, it will automatically shake out the code that we don't use, or the code that will never be executed. In Uglify Stage detected, not packaged into the bundle.
The purpose of learning treeshaking is also to lay the groundwork for learning the principle of rollup packaging later. In rollup, a file is a module. Each module will generate an AST syntax abstraction tree according to the code of the file, and rollup needs to analyze each AST node. To analyze an AST node is to see if the node calls a function or method. If so, check to see if the called function or method is in the current scope, if not, look up until you find the top-level scope of the module. If this module is not found, it means that this function and method depend on other modules and need to be imported from other modules. rollup only processes functions and top-level import/export variables.
0 pre-knowledge - installation related dependencies
Install webpack
npm install webpack webpack-cli --save-dev
Initialize the project
npm init -y
View the packaged content of webpack
npx webpack ./test1.js
install nodemon
nodemon is a tool to assist node.js development. When the files in the directory change, it will automatically restart the node application
npm i nodemon -g
Test code command: nodemon ./index.js
Parameters: watch
install acorn
npm i acorn -d -s
const acorn = require('acorn');The default usage of acorn is very simple. Just parse the code string and get the AST structure:
let acorn = require("acorn");
console.log(acorn.parse("for(let i=0;i<10;i+=1){console.log(i);}", {ecmaVersion: 2020}));Parsed AST syntax tree
Node {
type: 'Program',
start: 0,
end: 39,
body: [
Node {
type: 'ForStatement',
start: 0,
end: 39,
init: [Node],
test: [Node],
update: [Node],
body: [Node]
}
],
sourceType: 'script'
}
It can be seen that the type of this AST is program, indicating that this is a program. body contains the AST child nodes corresponding to all the following statements of the program. Each node has a type type, such as Identifier, indicating that this node is an identifier;
install magic-string
magic-string is a tool for manipulating strings and generating source-maps. magic-string is a library about string manipulation written by the author of rollup.
Install command:
npm i magic-string -D -S
Here's an example from github:
var MagicString = require('magic-string');
var magicString = new MagicString('export var name = "beijing"');
//类似于截取字符串
console.log(magicString.snip(0,6).toString()); // export
//从开始到结束删除字符串(索引永远是基于原始的字符串，而非改变后的)
console.log(magicString.remove(0,7).toString()); // var name = "beijing"
//很多模块，把它们打包在一个文件里，需要把很多文件的源代码合并在一起
let bundleString = new MagicString.Bundle();
bundleString.addSource({
content:'var a = 1;',
separator:'\n'
});
bundleString.addSource({
content:'var b = 2;',
separator:'\n'
});
/* let str = '';
str += 'var a = 1;\n'
str += 'var b = 2;\n'
console.log(str); */
console.log(bundleString.toString());
// var a = 1;
//var b = 2;which introduces the method
const MagicString = require('magic-string');The advantage of magic-string is that sourcemap will be generated
1. First understanding of AST syntax analysis
Abstract Syntax Tree (AST), or Syntax tree for short, is an abstract representation of the grammatical structure of source code. It represents the grammatical structure of the programming language in the form of a tree, and each node on the tree represents a structure in the source code. By manipulating this tree, we can accurately locate declaration statements, assignment statements, operation statements, etc., and implement operations such as code analysis, optimization, and changes.
You can simply understand that it is a tree-structured representation of the code you write. The cores of tools such as webpack, UglifyJs, and lint are all implemented through the ast abstract syntax book, which enables code inspection and analysis. The bottom layer is called js parser to generate abstract syntax tree.
AST workflow
- Parse (analysis) converts the source code into an abstract syntax tree, and there are many estree nodes on the tree
- Transform (transformation) transforms the abstract syntax tree
- Generate (code generation) Generate new code from the converted abstract syntax tree in the previous step
New file source.js
const a = ()=>'a888888'
const b = ()=>'b'
a()
Preview the results on the AST Spanning Tree website
Variable information can be analyzed layer by layer based on the data generated by AST
{
"type": "Program",
"start": 0,
"end": 46,
"body": [
{
"type": "VariableDeclaration",
"start": 0,
"end": 23,
"declarations": [
{
"type": "VariableDeclarator",
"start": 6,
"end": 23,
"id": {
"type": "Identifier",
"start": 6,
"end": 7,
"name": "a"
},
"init": {
"type": "ArrowFunctionExpression",
"start": 10,
"end": 23,
"id": null,
"expression": true,
"generator": false,
"async": false,
"params": [],
"body": {
"type": "Literal",
"start": 14,
"end": 23,
"value": "a888888",
"raw": "'a888888'"
}
}
}
],
"kind": "const"
},
{
"type": "VariableDeclaration",
"start": 24,
"end": 41,
"declarations": [
{
"type": "VariableDeclarator",
"start": 30,
"end": 41,
"id": {
"type": "Identifier",
"start": 30,
"end": 31,
"name": "b"
},
"init": {
"type": "ArrowFunctionExpression",
"start": 34,
"end": 41,
"id": null,
"expression": true,
"generator": false,
"async": false,
"params": [],
"body": {
"type": "Literal",
"start": 38,
"end": 41,
"value": "b",
"raw": "'b'"
}
}
}
],
"kind": "const"
},
{
"type": "ExpressionStatement",
"start": 42,
"end": 45,
"expression": {
"type": "CallExpression",
"start": 42,
"end": 45,
"callee": {
"type": "Identifier",
"start": 42,
"end": 43,
"name": "a"
},
"arguments": [],
"optional": false
}
}
],
"sourceType": "module"
}
You can see that in the disassembly above, each block has several fields such as type, start, end, and body. Where type expresses the type of the current block. For example, FunctionDeclaration means function definition, Identifier means identifier, BlockStatement means block statement, ReturnStatement means return statement, etc. start indicates where the block begins, end indicates where the block ends, and body indicates the sub-block. Other fields vary according to the nature of the block.
Add index.js file
//文件形式读取source.js
//fs模块 node.js中文件处理工具
const fs = require('fs');//引入
//读文件 readFileSync方法是同步读取文件，第一个参数表示文件路径，第二个参数表示读文件的编码方式（可省略）
const code = fs.readFileSync('./source.js').toString();
console.log('-------------source code----------------')
console.log(code);
//acorn 一个将代码解析为AST语法树的工具
const acorn = require('acorn');
const ast = acorn.parse(code,{ecmaVersion:'7'});//指定解析的js ECMAScript版本
console.log('--------------parsed as AST-------------')
ast.body.map(node=>{
console.log(node)
})
//MagicString 一个处理字符串的工具
const MagicString = require('magic-string');
const m = new MagicString(code);
console.log('--------------output node-------------')
console.log('index info')
ast.body.map((node,index)=>{
console.log(index+' ',m.snip(node.start,node.end).toString())//打印每个节点的信息
})
//分离声明和调用类型
const VariableDeclaration = []
const ExpressionStatement =[]
//对象当做map用 key是变量名，value是变量对应的节点
const VariableObj ={}
//statement数组 存放变量的声明和使用
const statementArr = []
ast.body.map(node=>{
if(node.type == 'VariableDeclaration') {
VariableDeclaration.push(node);//声明节点数组
//取声明数组的变量名key和节点value
const key = node.declarations[0].id.name
VariableObj[key] = node
} else if (node.type == 'ExpressionStatement') {
//对于引用的数组
ExpressionStatement.push(node);//引用节点数组
}
})
//取变量名
console.log('---------variableDeclaration name--------------')
VariableDeclaration.map(node=>{
console.log(node.declarations[0].id.name)
})
console.log('----------expressionStatement --------------')
ExpressionStatement.map(node=>{
// console.log(node.expression.callee.name)
console.log(node)
})
ExpressionStatement.map(node=>{
statementArr.push(VariableObj[node.expression.callee.name])//把表达式中使用的变量名的定义语句push到数组中
statementArr.push(node)//将表达式也push到数组中,未在表达式中调用的变量将不会遍历其VariableObj数组，也即过滤掉
})
console.log('------------treeshaking result----------')
// console.log(statementArr)
statementArr.map((node,index)=>{
console.log(index,m.snip(node.start,node.end).toString())
})Execute the statement nodemon ./index.js
output
-------------source code----------------
const a = ()=>'a888888'
const b = ()=>'b'
a()
--------------parsed as AST-------------
Node {
type: 'VariableDeclaration',
start: 0,
end: 23,
declarations: [
Node {
type: 'VariableDeclarator',
start: 6,
end: 23,
id: [Node],
init: [Node]
}
],
kind: 'const'
}
Node {
type: 'VariableDeclaration',
start: 25,
end: 42,
declarations: [
Node {
type: 'VariableDeclarator',
start: 31,
end: 42,
id: [Node],
init: [Node]
}
],
kind: 'const'
}
Node {
type: 'ExpressionStatement',
start: 44,
end: 47,
expression: Node {
type: 'CallExpression',
start: 44,
end: 47,
callee: Node { type: 'Identifier', start: 44, end: 45, name: 'a' },
arguments: []
}
}
--------------output node-------------
index info
0 const a = ()=>'a888888'
1 const b = ()=>'b'
2 a()
---------variableDeclaration name--------------
a
b
----------expressionStatement --------------
Node {
type: 'ExpressionStatement',
start: 44,
end: 47,
expression: Node {
type: 'CallExpression',
start: 44,
end: 47,
callee: Node { type: 'Identifier', start: 44, end: 45, name: 'a' },
arguments: []
}
}
------------treeshaking result----------
0 const a = ()=>'a888888'
1 a()Two, treeShaking node traversal method
Using TDD test-driven development
Added walk.js function
First, test the entry and exit functions
const walk = (ast, callObj)=>{
callObj.enter(ast)
callObj.leave(ast)
}
module.exports = walkWrite the walk.spec.js function
Test the case where the ast is an object
//walk.spec.js
//测试walk函数
describe('walk函数',()=>{
test('单个节点',()=>{
const ast = {
a:1,
// child:[{b:2}]
}
const walk = require('../walk')
const mockEnter = jest.fn()//fn方法是jest工厂方法
const mockLeave = jest.fn()
//walk函数遍历ast对象，对于单个节点，进入时调用enter函数，退出时调用leave函数
walk(ast,{
enter:mockEnter,
leave:mockLeave
})
//判断mockEnter是否被调用
let calls = mockEnter.mock.calls //calls是数组，每调用一次增加一项
expect(calls.length).toBe(1)//断言，ast={a:1}
expect(calls[0][0]).toEqual({a:1})
calls = mockLeave.mock.calls //在对leave是否调用进行判断
expect(calls.length).toBe(1)//断言，ast={a:1}
expect(calls[0][0]).toEqual({a:1})
})
})test
It can be seen from the test results that the walk function can test {a:1}
Use --watchAll to monitor jest in real time
jest --watchAllRealize printing the variable names of all variables
Implementation process: call the walk node variable method, customize the input parameter enter function in walk: take out all variables according to the variable name type attribute of the ast syntax tree in node as VariableDeclaration
New test.js
//文件形式读取source.js
//fs模块 node.js中文件处理工具
const fs = require('fs');//引入
//读文件 readFileSync方法是同步读取文件，第一个参数表示文件路径，第二个参数表示读文件的编码方式（可省略）
const code = fs.readFileSync('./source.js').toString();
console.log('-------------source code----------------')
console.log(code);
//引入walk函数
const walk = require('./src/walk')
//acorn 一个将代码解析为AST语法树的工具
const acorn = require('acorn');
const ast = acorn.parse(code,{ecmaVersion:'7'});//指定解析的js ECMAScript版本
console.log('--------------walk ast-------------')
ast.body.map(node=>{
walk(node,{
enter:(node)=>{
// console.log('enter---------------lalala')
if(node && typeof node === 'object') {
if(node.type === 'VariableDeclaration'){
console.log(node.declarations[0].id.name)
// console.log(JSON.stringify(node.declarations[0].id.name,'\t','\t'))
}
}
},
leave:(node) =>{
// console.log('leave----------------lalala')
}
})
})
ast.body is the node, and the printed results are as follows
boost: find variable names at any level
Example:
输入
const a,b =1
if(true) {
const c ='123'
}
function fn1() {
const d =1
}
const e =3
-----------------------------------
输出
a
b
c
fn1 =>d
e
-------------------------------------Add source2.js
Save the original program
const a=2
const b =1
if(true) {
const c ='123'
}
function fn1() {
const d =1
}Add test.js
//test.js
//文件形式读取source2.js
//fs模块 node.js中文件处理工具
const fs = require('fs');//引入
//读文件 readFileSync方法是同步读取文件，第一个参数表示文件路径，第二个参数表示读文件的编码方式（可省略）
const code = fs.readFileSync('./source2.js').toString();
console.log('-------------source code----------------')
console.log(code);
//引入walk函数
const walk = require('./src/walk')
//acorn 一个将代码解析为AST语法树的工具
const acorn = require('acorn');
const ast = acorn.parse(code,{ecmaVersion:'7'});//指定解析的js ECMAScript版本
console.log('--------------walk ast-------------')
const statements = []
const parseAST = (ast)=>{
ast.body.map(node=>{
// console.log(node)
walk(node,{
enter:(node)=>{
if(node.type === 'VariableDeclaration'){
console.log(node.declarations[0].id.name)
}
//是个函数
if(node.type === 'FunctionDeclaration'){
console.log('=>'+node.id.name)
}
},
leave:(node) =>{
}
})
})
}
parseAST(ast)
Test Results
nodemon test.js
3. Scope simulation
Create a new scope scope to achieve a result similar to the following
const a = '1'
function(){
const b = 2
}Add src/scope.js file
module.exports = class Scope{
//定义构造方法
constructor(option){
//初始化names
this.names=[]
if(option){
this.parent = option.parent
}
}
//新增方法，每次添加一个变量名到作用域中
add(name){
this.names.push(name)
}
//判断对象中是否包含某个变量名，谁调用，this指向谁
contains(name){
return this.names.includes(name) || this.parent && this.parent.contains(name)
}
findDefiningScope(name){
//如果调用方的names中包含name,返回调用方本身，否则沿着调用方的作用域链逐层向上寻找
if(this.names.includes(name)){
return this
} else if(this.parent){//如果存在父作用域，递归寻找父作用域中是否含有该方法
return this.parent.findDefiningScope(name)
} else {
return null
}
}
}
Upgrade contains(name) method
The findDefiningScope method can get the calling object. If an object is found, the object contains the name attribute
module.exports = class Scope{
//定义构造方法
constructor(option){
//初始化names
this.names=[]
if(option){
this.parent = option.parent
}
}
//新增方法，每次添加一个变量名到作用域中
add(name){
this.names.push(name)
}
//判断对象中是否包含某个变量名，谁调用，this指向谁
contains(name){
// return this.names.includes(name) || this.parent && this.parent.contains(name)
//等价与下面 其中！！表示取字符串的布尔类型表示
return !! this.findDefiningScope(name)
}
//返回实际的作用域对象
findDefiningScope(name){
//如果调用方的names中包含name,返回调用方本身，否则沿着调用方的作用域链逐层向上寻找
if(this.names.includes(name)){
return this
} else if(this.parent){//如果存在父作用域，递归寻找父作用域中是否含有该方法
return this.parent.findDefiningScope(name)
} else {
return null
}
}
}Add src/__test__/scope.spec.js file
describe('scope',()=>{
test('scope',()=>{
const Scope = require('../scope')
//实例化Scope，命名为route
const root = new Scope()
root.add('a')
//定义一个child子作用域，嵌套在Parent父作用域
const child = new Scope({'parent':root})
child.add('b')
//编写断言
expect(child.findDefiningScope('a')).toBe(root)//'child中是否有a',a在父作用域链上，按作用域规则可以找到
expect(child.contains('a')).toEqual(true)//toEqual比较每项每个的值
expect(child.findDefiningScope('b')).toBe(child)//toBe比较的是地址
expect(child.contains('b')).toBe(true)
// expect(child.findDefiningScope('c')).toBe(null)
// expect(child.contains('c')).toEqual(false)
})
})Add three-level scope, test
describe('scope',()=>{
test('scope',()=>{
const Scope = require('../scope')
//实例化Scope，命名为route
const root = new Scope()
root.add('a')
//定义一个child子作用域，嵌套在Parent父作用域
const child = new Scope({'parent':root})
child.add('b')
//定义一个三级作用域，孙子节点
const childChild = new Scope({'parent':child})
childChild.add('c')
//编写断言
expect(child.findDefiningScope('a')).toBe(root)//'child中是否有a',a在父作用域链上，按作用域规则可以找到
expect(child.contains('a')).toEqual(true)//toEqual比较每项每个的值
expect(child.findDefiningScope('b')).toBe(child)//toBe比较的是地址
expect(child.contains('b')).toBe(true)
expect(child.findDefiningScope('c')).toBe(null)
expect(child.contains('c')).toEqual(false)
expect(childChild.findDefiningScope('b')).toBe(child)
expect(childChild.contains('b')).toEqual(true)
})
})4. Integrate node traversal and scope functions
Create a new analyze.js file
//analyze.js
//输入是一个ast 输出是已经分析好的scope
const acorn = require('acorn')
const fs = require('fs')
const { node } = require('webpack')
const {walk} = require('../walk')
const code = fs.readFileSync('../source.js').toString()
const ast = acorn.parse(code,{ecmaVersion:7})
const Scope = require('./scope')
module.exports = function analyze(ast) {
const root = new Scope()
ast.body.map(node=>{
walk(node,{
enter:(node)=>{
if(node.type === 'Identifier'){
root.add(node.name)
// console.log(node.Identifier.name)
}
//是个函数 如果存在函数，函数在子作用域里
if(node.type === 'FunctionDeclaration'){
// console.log('=>'+node.id.name)
}
},
leave:(node) =>{
}
})
})
return new Scope()
}Added analyze.spec.js file
describe('analyze',()=>{
test('analyze 1',()=>{
const analyze = require('../analyze')
//测试ast到scope
const acorn = require('acorn')
const fs = require('fs')
const {walk} = require('../walk')
const code = `
const a =1`
const ast = acorn.parse(code,{ecmaVersion:7})
const root = analyze(ast)
expect(root.findDefiningScope('a')).toBe(root)
expect(root.contains('a')).toEqual(true)
})
})