AST abstract syntax tree Javascript version

In the javascript world, you can think of an abstract syntax tree (AST) is the lowest level. Further down, it is about the conversion and compilation of "black magic" in the field of.

Now, we are dismantling a simple add function

function add(a, b) {
    return a + b
}

First, we get this syntax block, is a FunctionDeclaration (function definition) objects.

Forced open, it became three:

• A id, is its name, that add
• Two params, is its parameters, i.e. [a, b]
• A body, that is, a bunch of things in curly braces

add no way to continue to dismantle go, it is one of the most basic Identifier (flag) objects, used to uniquely identify a function.

{
    name: 'add'
    type: 'identifier'
    ...
}

params continue to dismantle it, in fact, it is an array consisting of two Identifier. After the split there is no way any longer.

[
    {
        name: 'a'
        type: 'identifier'
        ...
    },
    {
        name: 'b'
        type: 'identifier'
        ...
    }
]

Next, we continue to open the body
we found, in fact, it is a body BlockStatement (block domain) object that is used to represent{return a + b}

Open Blockstatement, hidden inside a ReturnStatement (Return domain) objects used to represent thereturn a + b

ReturnStatement continues to open, there is a BinaryExpression that (binomial) objects used to represent thea + b

Continue to open BinaryExpression, it became three leftparts, operator, ,right

• operator which is+
• left Was inside, the object is Identifier a
• right Was inside, the object is Identifer b

In this way, we add a simple function of dismantling is completed.

AST (Abstract Syntax Tree), is indeed a standard tree structure.

Well, we mentioned above Identifier, Blockstatement, ReturnStatement, BinaryExpression, a small part of this specification, go check?

See the AST objects in the document

recast

input the command:npm i recast -S

You can get a screwdriver to manipulate the syntax tree

Next, you can manipulate it in any screwdriver js file, we create a new parse.js schematic:

Create a file parse.js

// 给你一把"螺丝刀"——recast
const recast = require("recast");

// 你的"机器"——一段代码
// 我们使用了很奇怪格式的代码，想测试是否能维持代码结构
const code =
  `
  function add(a, b) {
    return a +
      // 有什么奇怪的东西混进来了
      b
  }
  `
// 用螺丝刀解析机器
const ast = recast.parse(code);

// ast可以处理很巨大的代码文件
// 但我们现在只需要代码块的第一个body，即add函数
const add  = ast.program.body[0]

console.log(add)

Enter node parse.jsyou can view the structure of the add function, which is consistent with previously by AST objects in the document can be found on its specific attributes:

FunctionDeclaration{
    type: 'FunctionDeclaration',
    id: ...
    params: ...
    body: ...
}

recast.types.builders making molds

recast.types.builders which provides a lot of "die" so that you can easily spliced ​​into a new machine.

In the simplest case, before we want to function add(a, b){...}declare an anonymous function into a style statementconst add = function(a ,b){...}

How to modify?

First, we create a VariableDeclaration declare an object variable, declared head for the const, content VariableDeclarator object that will be created.

Next, create a VariableDeclarator, add.id placed on the left, the right is the creation of objects FunctionDeclaration

The third step, we create a FunctionDeclaration, the three components described above, id params body, because the anonymous function id is set to null, params use add.params, body use add.body.

In this way, it has been created const add = function(){}in the AST objects.

After the previous parse.js the code, add the following code

// 引入变量声明，变量符号，函数声明三种“模具”
const {variableDeclaration, variableDeclarator, functionExpression} = recast.types.builders

// 将准备好的组件置入模具，并组装回原来的ast对象。
ast.program.body[0] = variableDeclaration("const", [
  variableDeclarator(add.id, functionExpression(
    null, // 匿名化函数表达式.
    add.params,
    add.body
  ))
]);

//将AST对象重新转回可以阅读的代码
//这一行其实是recast.parse的逆向过程，具体公式为
//recast.print(recast.parse(source)).code === source
const output = recast.print(ast).code;

console.log(output)

Print function contents still preserved "original", even the comments have not changed.

In fact, we can also print out the code segment beautification format:

const output = recast.prettyPrint(ast, { tabWidth: 2 }).code

//输出为
const add = function(a, b) {
  return a + b;
};

Advanced combat: js file modify the command line

In addition to parse / print / builder, Recast three main functions:

• run: js files read from the command line, and converted to ast for processing.
• tnt: By assert () and check (), you can verify the type of object ast.
• visit: ast tree traversal, obtain a valid AST objects and make changes.

To learn all of the recast tool library through a series of small business:

demo.js

function add(a, b) {
  return a + b
}

function sub(a, b) {
  return a - b
}

function commonDivision(a, b) {
  while (b !== 0) {
    if (a > b) {
      a = sub(a, b)
    } else {
      b = sub(b, a)
    }
  }
  return a
}

recast.run command line file read

Called a new read.jsfile, write

read.js

recast.run( function(ast, printSource){
    printSource(ast)
})

Command line input

node read demo.js

We checked to see js file content printed on the console.

We know, node readyou can read the demo.jsfile and demo.js content into ast object.

It also provides a printSourcefunction, at any time ast content can be converted back to the source code to facilitate debugging.

recast.visit AST node traversal

read.js

#!/usr/bin/env node
const recast  = require('recast')

recast.run(function(ast, printSource) {
  recast.visit(ast, {
      visitExpressionStatement: function({node}) {
        console.log(node)
        return false
      }
    });
});

The nodes in the AST recast.visit objects one by one traverse.

note

• You want to operate the function declaration, use visitFunctionDelaration traversal want to operate assignment expression, use visitExpressionStatement. As long as the AST objects document object is defined, preceded visit, you can traverse.
• AST can take the object through the node
• Traversal function must be added after each return false, or select the following wording, otherwise an error:

#!/usr/bin/env node
const recast  = require('recast')

recast.run(function(ast, printSource) {
  recast.visit(ast, {
      visitExpressionStatement: function(path) {
        const node = path.node
        printSource(node)
        this.traverse(path)
      }
    })
});

When debugging, if you want to output AST object,console.log(node)

If you want to output AST objects corresponding source code, you canprintSource(node)

Command line input node read demo.jsfor testing.

#!/usr/bin/env nodeIn all uses recast.run()top of the file will need to add this line, its meaning and finally we discuss.

TNT object type determination AST

TNT, i.e. recast.types.namedTypes, it is an object to determine whether the specified type AST.

TNT.Node.assert (), just like in the good machine to destroy buried device, when the machine can not be operated in good condition (type mismatch), to destroy the machine (error exit)

TNT.Node.check (), it can be determined whether the same type, and outputs False and True

Node above objects can be replaced by any AST

例如:TNT.ExpressionStatement.check(),TNT.FunctionDeclaration.assert()

read.js

#!/usr/bin/env node
const recast = require("recast");
const TNT = recast.types.namedTypes

recast.run(function(ast, printSource) {
  recast.visit(ast, {
      visitExpressionStatement: function(path) {
        const node = path.value
        // 判断是否为ExpressionStatement，正确则输出一行字。
        if(TNT.ExpressionStatement.check(node)){
          console.log('这是一个ExpressionStatement')
        }
        this.traverse(path);
      }
    });
});

read.js

#!/usr/bin/env node
const recast = require("recast");
const TNT = recast.types.namedTypes

recast.run(function(ast, printSource) {
  recast.visit(ast, {
      visitExpressionStatement: function(path) {
        const node = path.node
        // 判断是否为ExpressionStatement，正确不输出，错误则全局报错
        TNT.ExpressionStatement.assert(node)
        this.traverse(path);
      }
    });
});

Combat: AST modify the source code with the Export all methods

exportific.js

Now, we want the function to rewrite this file to be able to export all forms, such as

function add (a, b) {
    return a + b
}

I want to change

exports.add = (a, b) => {
  return a + b
}

First, we will start with builders out of thin air to achieve a key function head

exportific.js

#!/usr/bin/env node
const recast = require("recast");
const {
  identifier:id,
  expressionStatement,
  memberExpression,
  assignmentExpression,
  arrowFunctionExpression,
  blockStatement
} = recast.types.builders

recast.run(function(ast, printSource) {
  // 一个块级域 {}
  console.log('\n\nstep1:')
  printSource(blockStatement([]))

  // 一个键头函数 ()=>{}
  console.log('\n\nstep2:')
  printSource(arrowFunctionExpression([],blockStatement([])))

  // add赋值为键头函数  add = ()=>{}
  console.log('\n\nstep3:')
  printSource(assignmentExpression('=',id('add'),arrowFunctionExpression([],blockStatement([]))))

  // exports.add赋值为键头函数  exports.add = ()=>{}
  console.log('\n\nstep4:')
  printSource(expressionStatement(assignmentExpression('=',memberExpression(id('exports'),id('add')),
    arrowFunctionExpression([],blockStatement([])))))
});

And he wrote a step by step we infer that exports.add = ()=>{}the process to obtain specific AST structure.

Use node exportific demo.jsRun to view the results.

Subsequently, only in the final expression obtained, the id ( 'add') to replace the function name obtained traversal, the traversal function parameters to replace the parameters obtained, the blockStatement ([]) is replaced traversal obtained function block-level scope, successfully rewrite all the functions!

In addition, we need to pay attention, in the commonDivision function, citing sub function should be rewritten as exports.sub

exportific.js

#!/usr/bin/env node
const recast = require("recast");
const {
  identifier: id,
  expressionStatement,
  memberExpression,
  assignmentExpression,
  arrowFunctionExpression
} = recast.types.builders

recast.run(function (ast, printSource) {
  // 用来保存遍历到的全部函数名
  let funcIds = []
  recast.types.visit(ast, {
    // 遍历所有的函数定义
    visitFunctionDeclaration(path) {
      //获取遍历到的函数名、参数、块级域
      const node = path.node
      const funcName = node.id
      const params = node.params
      const body = node.body

      // 保存函数名
      funcIds.push(funcName.name)
      // 这是上一步推导出来的ast结构体
      const rep = expressionStatement(assignmentExpression('=', memberExpression(id('exports'), funcName),
        arrowFunctionExpression(params, body)))
      // 将原来函数的ast结构体，替换成推导ast结构体
      path.replace(rep)
      // 停止遍历
      return false
    }
  })


  recast.types.visit(ast, {
    // 遍历所有的函数调用
    visitCallExpression(path){
      const node = path.node;
      // 如果函数调用出现在函数定义中，则修改ast结构
      if (funcIds.includes(node.callee.name)) {
        node.callee = memberExpression(id('exports'), node.callee)
      }
      // 停止遍历
      return false
    }
  })
  // 打印修改后的ast源码
  printSource(ast)
})

One step, send a simple front-end tools exportific

Add the following code made two minor changes

1. Add manual --help, and added --rewrite mode, you can directly overwrite the file or default file for the exported * .export.js.
2. The code before the final printSource (ast) replaced writeASTFile (ast, filename, rewriteMode)

exportific.js

#!/usr/bin/env node
const recast = require("recast");
const {
  identifier: id,
  expressionStatement,
  memberExpression,
  assignmentExpression,
  arrowFunctionExpression
} = recast.types.builders

const fs = require('fs')
const path = require('path')
// 截取参数
const options = process.argv.slice(2)

//如果没有参数，或提供了-h 或--help选项，则打印帮助
if(options.length===0 || options.includes('-h') || options.includes('--help')){
  console.log(`
    采用commonjs规则，将.js文件内所有函数修改为导出形式。

    选项： -r  或 --rewrite 可直接覆盖原有文件
    `)
  process.exit(0)
}

// 只要有-r 或--rewrite参数，则rewriteMode为true
let rewriteMode = options.includes('-r') || options.includes('--rewrite')

// 获取文件名
const clearFileArg = options.filter((item)=>{
  return !['-r','--rewrite','-h','--help'].includes(item)
})

// 只处理一个文件
let filename = clearFileArg[0]

const writeASTFile = function(ast, filename, rewriteMode){
  const newCode = recast.print(ast).code
  if(!rewriteMode){
    // 非覆盖模式下，将新文件写入*.export.js下
    filename = filename.split('.').slice(0,-1).concat(['export','js']).join('.')
  }
  // 将新代码写入文件
  fs.writeFileSync(path.join(process.cwd(),filename),newCode)
}


recast.run(function (ast, printSource) {
  let funcIds = []
  recast.types.visit(ast, {
    visitFunctionDeclaration(path) {
      //获取遍历到的函数名、参数、块级域
      const node = path.node
      const funcName = node.id
      const params = node.params
      const body = node.body

      funcIds.push(funcName.name)
      const rep = expressionStatement(assignmentExpression('=', memberExpression(id('exports'), funcName),
        arrowFunctionExpression(params, body)))
      path.replace(rep)
      return false
    }
  })


  recast.types.visit(ast, {
    visitCallExpression(path){
      const node = path.node;
      if (funcIds.includes(node.callee.name)) {
        node.callee = memberExpression(id('exports'), node.callee)
      }
      return false
    }
  })

  writeASTFile(ast,filename,rewriteMode)
})

Now try

node exportific demo.js

Already can be found in the source code changes in the current directory demo.export.jsfiles.

npm contract

Edit package.json file

{
  "name": "exportific",
  "version": "0.0.1",
  "description": "改写源码中的函数为可exports.XXX形式",
  "main": "exportific.js",
  "bin": {
    "exportific": "./exportific.js"
  },
  "keywords": [],
  "author": "wanthering",
  "license": "ISC",
  "dependencies": {
    "recast": "^0.15.3"
  }
}

Note bin option, which means that the global command exportificpoints to the current directoryexportific.js

In this case, enter npm linkin locally generated a exportificcommand.

After that, as long as the guide which js file you want to use it, on exportific XXX.jsit.

We must pay attention to exportific.js header has

#!/usr/bin/env node

Next, released npm package!

Npm if you already have an account, please use the npm loginlog in

If you do not have an account npm https://www.npmjs.com/signup very simple to register npm

Then, enter
npm publish

No tedious steps, the audit did not, you just released a practical front-end gadget exportific. Anyone can

npm i exportific -g

This global installed a plugin.

Tip: When test tutorial, and I do not pack the same name, modify contract name.

#!/usr/bin/env node

Different users or different script interpreter is possible to install in a different directory, how the system know where to go to find your interpreter do? /usr/bin/envIt is to tell the system can be found in PATH directory. So configured #!/usr/bin/env node, it is to solve the various different user node routing problem, allowing the system to dynamically locate node to execute your script file.

If there is No such file or directoryan error? Because your node installation path is not added to the PATH system. Therefore, to carry out an environment variable node configuration on it.

If you just want a simple test, you can which nodebe found on your local node installation path command, /usr/bin/envinstead you find the path to the node.

---

Reference article: https://segmentfault.com/a/1190000016231512?utm_source=tag-newest#comment-area