foreword
In the current front-end environment, front-end engineering is gradually derived from the previous html, css, , from simple to complex, and more and more complicated. The most complicated one belongs to us . Engineers have appeared to specialize configuration .jswebpackwebpackwebpack
There are tens of thousands of front-end engineering packaging tools, who is yours NO.One.
This article implements a simple javaScriptpackaging tool, which does not involve non javaScript-packaging, such as: css, html, 静态文件and so on.
environment
Our computer needs to be equipped with nodean environment.
Required Parts Tools
fs
fsModules are used to manipulate files. This module can only nodebe used in the environment, not in the browser.
path
pathA module is a module for working with files and file paths.
@babel/parser
@babel/parserThe module is used to receive source code, perform lexical analysis, syntax analysis, and generate 抽象语法树（Abstract Syntax Tree）, referred to as ast.
@babel/traverse
@babel/traverseModules for traversal updates we use @babel/parsergenerated AST. Operate on specific nodes.
@babel/core
@babel/coretransformThe code used to compile our module in the module can be converted to the first version of the code to make it more compatible. In this article transformFromAstSync, the effect is the same.
@babel/preset-env
@babel/preset-enves规范A module is a preset tool module for an intelligent environment, allowing us to write code using the latest , without having to manage all the tedious details of which syntax transformations are required for the target environment.
Write a bundler
We will combine the above tool modules to write our own jspackaging tool. If we need to package non- jscontent, we need other module tools.
What this article achieves can only be packaged js, let us do it together.
There is a small detail to remind my friends
The content packaged in macthe system is the same as that printed by the terminal, but the download is hidden.终端windowmac
Environment build
First we need to create a new folder, then execute npm init/ pnpm initto generate package.jsonfiles. Then install the above modules.
//npm
npm i @babel/core @babel/parser @babel/preset-env @babel/traverse
//pnpm
pnpm i @babel/core @babel/parser @babel/preset-env @babel/traverse
new main.jsfile
We create a new main.jsfile to write our code, of course you can use other file names.
new srcdirectory
Here we need to create a new srcdirectory to hold the code we wrote.
In srcthe directory, we create two new jsfiles, as follows:
// foo.js
const foo = () => {
console.log('我是foo');
}
export {
foo
}
Let's create a new index.jsfile, import it foo.js, and index.jsexecute foothe method in the method. Then we perform indexdefense.
// index.js
import {
foo } from "./foo.js"
const index = () => {
foo()
console.log('我是index');
for(let item of [1, 2]){
console.log(item);
}
}
index()
writemain.js
Now it's main.jstime for us to start writing.
Introduce the tool module we just needed, here we need to use requirethe form of reference,
const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
// 这里需要添加.default来引入@babel/traverse模块，因为require不支持default的导出，故添加此内容
const traverse = require('@babel/traverse').default
const babel = require('@babel/core')
read file content
We add readFilea method to read the content of the file we wrote js, here we use the method fsin the module readFileSync, and set the content format to utf-8.
Then we pass in index.jsthe path to the file and execute the method.
const readFile = (fileName) => {
const content = fs.readFileSync(fileName, 'utf-8')
console.log(content);
}
readFile('./src/index.js')
We execute in the terminal node main.js.
We see the terminal print out index.jsthe contents of our file. The content is exactly the same as ours index.js, the difference is that the printed string is a string with a \nnewline character added inside.
import {
foo } from "./foo.js"
const index = () => {
foo()
console.log('我是index');
for(let item of [1, 2]){
console.log(item);
}
}
index()
Generate asta syntax tree from the content of the obtained file
Above we have got the code we wrote, now we need to @babel/parsergenerate ours through the tool ast.
We readFileadd it in the method @babel/parserand set it sourceTypeto module. And still execute in the terminal node main.js.
const ast = parser.parse(content, {
sourceType: 'module'
})
console.log(ast);
The printed result is as follows, which is a nodeformat node, and the content of our code is in program-> body.
Node {
type: 'File',
start: 0,
end: 165,
loc: SourceLocation {
start: Position {
line: 1, column: 0, index: 0 },
end: Position {
line: 12, column: 7, index: 165 },
filename: undefined,
identifierName: undefined
},
errors: [],
program: Node {
type: 'Program',
start: 0,
end: 165,
loc: SourceLocation {
start: [Position],
end: [Position],
filename: undefined,
identifierName: undefined
},
sourceType: 'module',
interpreter: null,
body: [ [Node], [Node], [Node] ],
directives: []
},
comments: []
}
@babel/traverseUpdate our traversal usingast
Here we use @babel/traversethe tool to iterate over what we just generated ast.
In this environment, we need to create a new dependenciesobject named , which is used to install astthe dependencies we have processed.
We will just astpass in, optionand ImportDeclarationadd a formal parameter to receive the path of each file for one of the functions.
const dependencies = {
}
traverse(ast, {
ImportDeclaration: ({
node }) => {
}
})
We pathhandle paths to our files through modules.
const dirName = path.dirname(fileName)
We need to do some further processing with our filenames and paths. And regex it to replace backslashes.
const dir = './' + path.join(dirName, node.source.value).replace('\\', '/')
In the above code node.source.valueare all the file names and paths we astobtained according to.
We store the file path we got dependenciesinto the object.
dependencies[node.source.value] = dir
Finally we execute in the terminal node main.jsand print our dependenciesobject. The printed content is consistent with the file path we need to compile.
{
'./foo.js': './src/foo.js' }
@babel/coreTranscode our code using
Here we need to use the solution @babel/corein the tool transformto translate our code, so that our code can also run normally in the first version of the browser.
Here we use the latest api transformFromAstSyncto transpile our code.
transformFromAstSyncWhat it does: Recompile the transcode we just modified astback into our code.
We only need its transformed code, we don't need anything else, so we destructure its result to get only its code.
const {
code } = babel.transformFromAstSync(ast, null, {
})
We need to use it here @babel/preset-envto downgrade our code, that is to say, the new version of the specification we use is written, and we need to transfer it back to the old version of the specification. If we don't handle it here, our code will not handle it, and the output will be as it is.
So we need to add a presetsproperty to it and put it in our @babel/preset-envtool. Here we modulesset the property falseto let it output esmthe formatted code.
Other property extensions: commonjs, amd, umd, systemjs,auto
const {
code } = babel.transformFromAstSync(ast, null, {
presets: [
[
"@babel/preset-env",
{
modules: false
}
]
]
})
We execute it in the terminal node main.js, and the printed content is as follows:
import {
foo } from "./foo.js";
var index = function index() {
foo();
console.log('我是index');
for (var _i = 0, _arr = [1, 2]; _i < _arr.length; _i++) {
var item = _arr[_i];
console.log(item);
}
};
index();
The complete code of the readFile method:
const readFile = (fileName) => {
const content = fs.readFileSync(fileName, 'utf-8')
const ast = parser.parse(content, {
sourceType: 'module'
})
const dependencies = {
}
traverse(ast, {
ImportDeclaration: ({
node }) => {
const dirName = path.dirname(fileName)
const dir = './' + path.join(dirName, node.source.value).replace('\\', '/')
dependencies[node.source.value] = dir
}
})
const {
code } = babel.transformFromAstSync(ast, null, {
presets: [
[
"@babel/preset-env",
{
modules: false
}
]
]
})
return {
fileName,
dependencies,
code
}
}
It has successfully downgraded our code, and we return our , 文件名/文件路径, 依赖关系（dependencies）, for our later use.代码（code）return
Write a dependency generator
We need to create a new createDependciesGraphmethod named , which will collect our file dependencies. Add a formal parameter to receive the filename we pass in.
const createDependciesGraph = entry => {
}
Create an graphListarray named to hold the return value from our readFilemethod return.
const graphList = [readFile(entry)]
We need recursive processing here graphListto prevent multiple dependencies in it.
for(let i = 0;i < graphList.length; i++){
}
We need to stage each item in the loop, so we declare one itemto hold.
const item = graphList[i]
We also need to temporarily store the dependencies of each item here.
const {
dependencies } = item
Here we need to add a judgment. If there is a dependency relationship, we continue to loop its dependency layer again and insert it, graphListso as to recursively nest the loop and read the file content until the loop ends.
if(dependencies){
for(let j in dependencies){
graphList.push( readFile( dependencies[j] ) )
}
}
Complete code for this part:
const createDependciesGraph = entry => {
const graphList = [readFile(entry)]
for(let i = 0;i < graphList.length; i++){
const item = graphList[i]
const {
dependencies } = item
if(dependencies){
for(let j in dependencies){
graphList.push(
readFile(dependencies[j])
)
}
}
}
console.log(graphList);
}
Let's print what has been processed graphList, terminal input node main.js, the result is as follows:
[
{
fileName: './src/index.js',
dependencies: {
'./foo.js': './src/foo.js' },
code: import {
foo } from "./foo.js";
var index = function index() {
foo();
console.log('我是index');
for (var _i = 0, _arr = [1, 2]; _i < _arr.length; _i++) {
var item = _arr[_i];
console.log(item);
}
};
index();
},
{
fileName: './src/foo.js',
dependencies: {
},
code: var foo = function foo() {
console.log('我是foo');};export {
foo };
}
]
Relationship layer combing
We have seen that our relationship layer has been completely printed just now, and we need to sort it out now.
We create a new object to hold our combed relationship layer.
const graph = {
}
Here we loop through graphListthe array and graphwrite our detailed dependency layer into it.
for(let item of graphList){
const {
dependencies, code} = item
graph[item.fileName] = {
dependencies,
code
}
}
Let's print the sorted out content just now, still typing in the terminal node main.js:
{
'./src/index.js': {
dependencies: {
'./foo.js': './src/foo.js' },
code: import {
foo } from "./foo.js";
var index = function index() {
foo();\n' +
console.log('我是index');
for (var _i = 0, _arr = [1, 2]; _i < _arr.length; _i++) {
var item = _arr[_i];
console.log(item);
}
};
index();
},
'./src/foo.js': {
dependencies: {
},
code: var foo = function foo() {
console.log('我是foo');};export {
foo };
}
}
We need to return the sorted out relationship layer returnfor our later use.
return graph
The complete code of the createDependenciesGraph method:
const createDependciesGraph = entry => {
const graphList = [readFile(entry)]
for(let i = 0;i < graphList.length; i++){
const item = graphList[i]
const {
dependencies } = item
if(dependencies){
for(let j in dependencies){
graphList.push(
readFile(dependencies[j])
)
}
}
}
const graph = {
}
for(let item of graphList){
const {
dependencies, code} = item
graph[item.fileName] = {
dependencies,
code
}
}
return graph
}
Let's create a file management method first
In this step, we first create a folder management method, which is used to clear the directory and recreate it every time we pack.
We declare a rmdirmethod called, that manages our packaging directory folder
const rmdir = async () => {
}
Let's give it returnan internal new Promiseinstance. As for the reason, we will use it later and understand it, which is convenient for us to use later.
const rmdir = async () => {
return new Promise(async (resolve, reject) => {
})
}
We declare one errto get errors when we operate folders and files,
let err = null
We read the state of our current packaging folder, empty and delete it if it exists. recursiveIndicates whether to delete the folder, truewhich is delete.
const isDir = fs.existsSync('dist')
if(isDir){
fs.rmdir('dist', {
recursive: true}, error => {
if(error){
err = error
}
})
}
Here we make a wrong judgment, and when errit is true we throw an error and returngo out.
if(err){
reject(err)
return
}
Here we use it setTimeoutto delay the notification of success to avoid deleting folders and creating folders at the same time, resulting in unsuccessful creation.
setTimeout(()=>{
resolve()
}, 1000)
rmdir complete code:
const rmdir = async () => {
return new Promise(async (resolve, reject) => {
let err = null
const isDir = fs.existsSync('dist')
if(isDir){
fs.rmdir('dist', {
recursive: true}, error => {
if(error){
err = error
}
})
}
if(err){
reject(err)
return
}
setTimeout(()=>{
resolve()
}, 1000)
})
}
code generator method
Here I am using esbuilda packaging output mode, that is, the packaged files are generated synchronously according to the directory rules when the project was created.
Here we create a generateCodemethod called , make our code generation entry call, and write the generated file.
const generateCode = entry => {
}
Call the method inside it createDependciesGraphand entry（打包的入口文件）pass it in. And declare codeInfoto accept.
const codeInfo = createDependciesGraph(entry)
We can print it first to see codeInfowhat it looks like.
{
'./src/index.js': {
dependencies: {
'./foo.js': './src/foo.js' },
code: import {
foo } from "./foo.js";
var index = function index() {
foo();
console.log('我是index');
for (var _i = 0, _arr = [1, 2]; _i < _arr.length; _i++) {
var item = _arr[_i];
console.log(item);
}
};
index();
},
'./src/foo.js': {
dependencies: {
},
code: var foo = function foo() {
console.log('我是foo');};export {
foo };
}
}
Now we create folders and write files based on dependencies.
At this moment, we have lined up rmdirthe method just now, we call rmdirthe method, and .thenwrite our file creation process in it. This is the reason why rmdirone was returned when Promiseit was created just now. After the package directory is deleted and emptied, the package folder and files are created, so that we avoid the problem of creating and deleting folders and files at the same time.
rmdir().then(()=>{
})
Now let's create the packaging directory folder.
fs.mkdir('dist', () => {
})
We loop our dependencies in the callback to create the packaged folder, because codeInfoit is an object, we can't use it for..of..., and we use the es6new one in for..in...
for(let key in codeInfo){
}
Here we create a folder with the same name and write the specified code into a file with the same name. Here we get splitthe current file name by the method we use, and take the last item, because the last item is our file name.
let value = key.split('/')
value = value[value.length - 1]
We create an asking price based on the file name obtained above, and write the corresponding code into the file.
let value = key.split('/')
value = value[value.length - 1]
fs.writeFile(`./dist/${
value}`, codeInfo[key]['code'], [], () => {
})
generateCode method complete code
const generateCode = entry => {
const codeInfo = createDependciesGraph(entry)
console.log(codeInfo);
rmdir().then(()=>{
fs.mkdir('dist', () => {
for(let key in codeInfo){
let value = key.split('/')
value = value[value.length - 1]
fs.writeFile(`./dist/${
value}`, codeInfo[key]['code'], [], () => {
})
}
})
})
}
We need to main.jscall our generateCodecode generator's method in . We need to pass in the entry file of the package file while calling.
generateCode('./src/index.js')
We're done writing, now let's run it, enter it in the terminal node main.jsand allow it.
We will find that our project directory has generated a directory, which contains distour files.srcjs
Let's see foo.jsif index.jsthe file contains srcthe contents of the directory. ,
foo.js
var foo = function foo() {
console.log('我是foo');
};
export {
foo };
index.js
import {
foo } from "./foo.js";
var index = function index() {
foo();
console.log('我是index');
for (var _i = 0, _arr = [1, 2]; _i < _arr.length; _i++) {
var item = _arr[_i];
console.log(item);
}
};
index();
Verify that packaged files allow
We create a new one index.html, and import the files distin the directory index.js.
<script src="./dist/index.js" type="module"></script>
The effect is as follows:
Our packaged files can be run normally.
full code
const fs = require('fs')
const path = require('path')
const parser = require('@babel/parser')
const traverse = require('@babel/traverse').default
const babel = require('@babel/core')
const readFile = (fileName) => {
const content = fs.readFileSync(fileName, 'utf-8')
const ast = parser.parse(content, {
sourceType: 'module'
})
const dependencies = {
}
traverse(ast, {
ImportDeclaration: ({
node }) => {
const dirName = path.dirname(fileName)
const dir = './' + path.join(dirName, node.source.value).replace('\\', '/')
dependencies[node.source.value] = dir
}
})
const {
code } = babel.transformFromAstSync(ast, null, {
presets: [
[
"@babel/preset-env",
{
modules: false
}
]
]
})
return {
fileName,
dependencies,
code
}
}
const createDependciesGraph = entry => {
const graphList = [readFile(entry)]
for(let i = 0;i < graphList.length; i++){
const item = graphList[i]
const {
dependencies } = item
if(dependencies){
for(let j in dependencies){
graphList.push(
readFile(dependencies[j])
)
}
}
}
const graph = {
}
for(let item of graphList){
const {
dependencies, code} = item
graph[item.fileName] = {
dependencies,
code
}
}
return graph
}
const generateCode = entry => {
const codeInfo = createDependciesGraph(entry)
rmdir().then(()=>{
fs.mkdir('dist', () => {
for(let key in codeInfo){
let value = key.split('/')
value = value[value.length - 1]
fs.writeFile(`./dist/${
value}`, codeInfo[key]['code'], [], () => {
})
}
})
})
}
const rmdir = async () => {
return new Promise(async (resolve, reject) => {
let err = null
const isDir = fs.existsSync('dist')
if(isDir){
fs.rmdir('dist', {
recursive: true}, error => {
if(error){
err = error
}
})
}
if(err){
reject(err)
return
}
setTimeout(()=>{
resolve()
}, 1000)
})
}
generateCode('./src/index.js')
Summarize
At this point, our simple JavaScript打包器implementation is over. The implementation of this simple packager is only used to understand and understand the principle of the mainstream packager.
Our current packager still has some flaws:
- Nested directory files cannot be packaged.
- can only pack
jsfiles