Dachang Technology Advanced Front-end Node Advanced
Click on the top programmer's growth guide, pay attention to the public number
Reply 1, join the advanced Node exchange group
Recently, I wrote an automated deployment npm package zuo-deploy [1] . With the click of a button, the server deployment script can be executed to complete the function update iteration. Client uses Vue + ElementUI, service koa + socket + koa-session etc. The basic function code is less than 300 lines, and it has been open sourced on github. zuoxiaobai/zuo-deploy welcome Star, Fork. The specific implementation details and ideas are introduced here.
Directory Structure
├── bin # 命令行工具命令
│ ├── start.js # zuodeploy start 执行入口
│ └── zuodeploy.js # zuodeploy 命令入口，在 package.json 的 bin 属性中配置
├── docImages # README.md 文档图片
├── frontend # 客户端页面/前端操作页面（koa-static 静态服务指定目录）
│ └── index.html # Vue + ElementUI + axios + socket.io
├── server # 服务端
│ ├── utils
│ │ ├── logger.js # log4js
│ │ └── runCmd.js # node child_process spawn（执行 shell 脚本、pm2 服务开启）
│ └── index.js # 主服务（koa 接口、静态服务 + socket + 执行 shell 脚本）
├── .eslintrc.cjs # eslint 配置文件 + prettier
├── args.json # 用于 pm2 改造后，跨文件传递端口、密码参数
├── CHANGELOG.md # release 版本功能迭代记录
├── deploy-master.sh # 用于测试，当前目录开启服务偶，点击部署按钮，执行该脚本
├── index.js # zuodeploy start 执行文件，用于执行 pm2 start server/index.js 主服务
├── package.json # 项目描述文件，npm 包名、版本号、cli 命令名称、
├── publish.sh # npm publish（npm包） 发布脚本
└── README.md # 使用文档
复制代码Front-end and back-end technology stacks, related dependencies
Frontend/Client
Static html + css, non-front-end engineering, the library is imported in the form of cdn, and used through the global variables exposed by the library in UMD packaging
vue3, MVVM framework, no need to operate dom
element-plus, the basic form style is unified and beautified
axios, request interface
socket.io, receive real-time deployment logs
Server
Ordinary interface, you may need to wait until it is fully deployed before you can get the result
Based on Node.js technology stack, no database
commander, used to generate the command zuodeploy runtime help documentation, tips, zuodeploy start execution entry
prompts, refer to vue-create, guide the user to enter the port and password
koa, http server, provides interfaces, static services to run containers (similar to nginx, tomcat, etc.)
koa-bodyparser, used to parse post request parameters (required by the login authentication interface)
koa-router, used for different interfaces (paths, such as /login, /deploy, etc.) to execute different methods
koa-session, used for interface authentication, to prevent others from crazy request for deployment after obtaining the deployment interface
koa-static, static server, similar to nginx to start static services
socket.io, socket server, when git pull, npm run build take a long time to deploy, send logs to the front end in real time
log4js, log output with timestamp
pm2, execute directly, when the terminal ends the service will be turned off, use pm2 to execute silently in the background
Basic function realization ideas
Initial goal: Click the deploy button on the front-end page to directly let the server execute the deployment and return the deployment log to the front-end
How to achieve this?
1. There must be a front-end page that gives the deployment button and the log display area.
2. For the front-end page to interact with the server, there must be a server-side server
2.1 Provide an interface, click on the front-end page to deploy, request the interface, and know when to execute the deployment,
2.2 After the back-end interface receives the request, how to execute the deployment task,
2.3 How to collect and send the log of shell script execution to the front end. Same as above, spawn supports log output
Technology stack determination:
1. Vue + ElementUI basic page layout + basic logic, axios request interface data
2. Use node technology stack to provide server server
2.1 Implementing the interface using koa/koa-router
2.2 Deployment is generally to execute shell scripts. Node uses the built-in subprocess spawn to execute shell script files and run commands running under the terminal.
2.3 When spawn is executed, the subprocess stdout and stderr can obtain the script execution log, and return it to the front end after collection
Considering the deployment of front-end pages, it can be put together with the koa server service, and use koa-static to open static file services to support front-end page access
Front-end engineering is not used here @vue/cli, static html is used directly, vue is introduced through cdn, etc.
1. Client client Vue + ElementUI + axios
We put the front-end service in frontend/index.html, and the koa-static static service directly points to the frontend directory to access the page
The core code is as follows:
Note: The cdn links are all // relative paths, you need to use the http service to open the page, and cannot be opened in the form of ordinary File files! You can wait until the koa is written later, open the service and then access
<head>
<title>zuo-deploy</title>
<!-- 导入样式 -->
<link rel="stylesheet" href="//unpkg.com/element-plus/dist/index.css" />
<!-- 导入 Vue 3 -->
<script src="//unpkg.com/vue@next"></script>
<!-- 导入组件库 -->
<script src="//unpkg.com/element-plus"></script>
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
</head>
<body>
<div id="app" style="margin:0 20px;">
<el-button type="primary" @click="deploy">部署</el-button>
<div>
<p>部署日志:</p>
<div class="text-log-wrap">
<pre>{
{ deployLog }}</pre>
</div>
</div>
</div>
<script>
const app = {
data() {
return {
deployLog: '点击按钮进行部署',
}
},
methods: {
deploy() {
this.deployLog = '后端部署中，请稍等...'
axios.post('/deploy')
.then((res) => {
// 部署完成，返回 log
console.log(res.data);
this.deployLog = res.data.msg
})
.catch(function (err) {
console.log(err);
})
}
}
}
Vue.createApp(app).use(ElementPlus).mount('#app')
</script>
</body>
复制代码2. Service end koa + koa-router + koa-static
koa starts the http server and writes the deploy interface for processing. koa-static enable static service
// server/index.js
const Koa = require("koa");
const KoaStatic = require("koa-static");
const KoaRouter = require("koa-router");
const path = require("path");
const app = new Koa();
const router = new KoaRouter();
router.post("/deploy", async (ctx) => {
// 执行部署脚本
let execFunc = () => {};
try {
let res = await execFunc();
ctx.body = {
code: 0,
msg: res,
};
} catch (e) {
ctx.body = {
code: -1,
msg: e.message,
};
}
});
app.use(new KoaStatic(path.resolve(__dirname, "../frontend")));
app.use(router.routes()).use(router.allowedMethods());
app.listen(7777, () => console.log(`服务监听 ${7777} 端口`));
复制代码run the project
In the current project directory, execute
npm initInitialize package.jsonnpm install koa koa-router koa-static --saveInstall dependenciesnode server/index.jsRun the project, note that if port 7777 is occupied, you need to change a port
Visit http://127.0.0.1:7777 to access the page, click Deploy to request success
3.Node executes the shell script and outputs the log to the front end
Under the built-in module child_process of node, spawn executes terminal commands, including sh 脚本文件.shcommands
Let's take a look at a demo, create a new testExecShell test directory, and test the effect
// testExecShell/runCmd.js
const { spawn } = require('child_process');
const ls = spawn('ls', ['-lh', '/usr']); // 执行 ls -lh /usr 命令
ls.stdout.on('data', (data) => {
// ls 产生的 terminal log 在这里 console
console.log(`stdout: ${data}`);
});
ls.stderr.on('data', (data) => {
// 如果发生错误，错误从这里输出
console.error(`stderr: ${data}`);
});
ls.on('close', (code) => {
// 执行完成后正常退出就是 0
console.log(`child process exited with code ${code}`);
});
复制代码node testExecShell/runCmd.jsRunning can use node to execute ls \-lh /usr, and receive log information through ls.stdout and print
Back to the topic, here you need to execute the shell script, you can ls \-lh /usrreplace it with sh 脚本文件.sh. Let's try
// testExecShell/runShell.js
const { spawn } = require('child_process');
const child = spawn('sh', ['testExecShell/deploy.sh']); // 执行 sh deploy.sh 命令
child.stdout.on('data', (data) => {
// shell 执行的 log 在这里搜集，可以通过接口返回给前端
console.log(`stdout: ${data}`);
});
child.stderr.on('data', (data) => {
// 如果发生错误，错误从这里输出
console.error(`stderr: ${data}`);
});
child.on('close', (code) => {
// 执行完成后正常退出就是 0
console.log(`child process exited with code ${code}`);
});
复制代码To create a shell script to execute, you can first sh estExecShell/deploy.shtry to see if it is executable, if not, add it (chmod +x file name)
# /testExecShell/deploy.sh
echo '执行 pwd'
pwd
echo '执行 git pull'
git pull
复制代码Run node testExecShell/runShell.jsto let node execute the deploy.sh script, as shown below
Reference: child\_process \- Node.js built-in module notes [2]
4. The deploy interface integrates the function of executing shell scripts
Modify the previous deploy interface, add a runCmd method, and execute the deploy.sh deployment script in the current directory. After completion, the interface will execute the log response to the front end
// 新建 server/indexExecShell.js，将 server/index.js 内容拷贝进来，并做如下修改
const rumCmd = () => {
return new Promise((resolve, reject) => {
const { spawn } = require('child_process');
const child = spawn('sh', ['deploy.sh']); // 执行 sh deploy.sh 命令
let msg = ''
child.stdout.on('data', (data) => {
// shell 执行的 log 在这里搜集，可以通过接口返回给前端
console.log(`stdout: ${data}`);
// 普通接口仅能返回一次，需要把 log 都搜集到一次，在 end 时 返回给前端
msg += `${data}`
});
child.stdout.on('end', (data) => {
resolve(msg) // 执行完毕后，接口 resolve，返回给前端
});
child.stderr.on('data', (data) => {
// 如果发生错误，错误从这里输出
console.error(`stderr: ${data}`);
msg += `${data}`
});
child.on('close', (code) => {
// 执行完成后正常退出就是 0
console.log(`child process exited with code ${code}`);
});
})
}
router.post("/deploy", async (ctx) => {
try {
let res = await rumCmd(); // 执行部署脚本
ctx.body = {
code: 0,
msg: res,
};
} catch (e) {
ctx.body = {
code: -1,
msg: e.message,
};
}
});
复制代码After the modification is completed, run node server/indexExecShell.jsStart the latest service, click Deploy, and the interface executes normally, as shown in the following figure
The deploy.sh in the current directory is executed, and there is no corresponding file. Put the above testExeclShell/deploy.sh in the current directory and click Deploy
In this way, the basic functions of automatic deployment are basically completed.
Function optimization
1. Use socket to output log in real time
In the above example, the common interface needs to wait for the deployment script to be executed before responding to the front-end. If the script contains git pull, npm run build and other time-consuming commands, there will be no log information on the front-end page, as shown in the following figure
test shell
echo '执行 pwd'
pwd
echo '执行 git pull'
git pull
git clone [email protected]:zuoxiaobai/zuo11.com.git # 耗时较长的命令
echo '部署完成'
复制代码
Here we transform and use socket.io [3] to send the deployment log to the front end in real time
socket.io is divided into two parts: client and server
client code
<!-- frontend/indexSocket.html -->
<script src="https://cdn.socket.io/4.4.1/socket.io.min.js"></script>
<script>
// vue mounted 钩子里面链接 socket 服务端
mounted() {
this.socket = io() // 链接到 socket 服务器，发一个 http 请求，成功后转 101 ws 协议
// 订阅部署日志，拿到日志，就一点点 push 到数组，显示到前端
this.socket.on('deploy-log', (msg) => {
console.log(msg)
this.msgList.push(msg)
})
},
</script>
复制代码Introduce socket.io code in backend koa
// server/indexSoket.js
// npm install socket.io --save
const app = new Koa();
const router = new KoaRouter();
// 开启 socket 服务
let socketList = [];
const server = require("http").Server(app.callback());
const socketIo = require("socket.io")(server);
socketIo.on("connection", (socket) => {
socketList.push(socket);
console.log("a user connected"); // 前端调用 io()，即可连接成功
});
// 返回的 socketIo 对象可以用来给前端广播消息
runCmd() {
// 部分核心代码
let msg = ''
child.stdout.on('data', (data) => {
// shell 执行的 log 在这里搜集，可以通过接口返回给前端
console.log(`stdout: ${data}`);
socketIo.emit('deploy-log', `${data}`) //socket 实时发送给前端
// 普通接口仅能返回一次，需要把 log 都搜集到一次，在 end 时 返回给前端
msg += `${data}`
});
// ...
child.stderr.on('data', (data) => {
// 如果发生错误，错误从这里输出
console.error(`stderr: ${data}`);
socketIo.emit('deploy-log', `${data}`) // socket 实时发送给前端
msg += `${data}`
});
}
// app.listen 需要改为上面加入了 socket 服务的 server 对象
server.listen(7777, () => console.log(`服务监听 ${7777} 端口`));
复制代码We add the above code to the previous demo to complete the socket transformation, node server/indexSocket.js, open 127.0.0.1:7777/indexSocket.html, click deploy, and you can see the following effect. Complete the demo access address [4]
Related questions
Regarding the http to ws protocol, we can open the F12 NetWork panel to see the front-end socket related connection steps
GET
http://127.0.0.1:7777/socket.io/?EIO=4&transport=polling&t=Nz5mBZkget sidPOST
http://127.0.0.1:7777/socket.io/?EIO=4&transport=polling&t=Nz5mBaY&sid=DKQAS0fxzXUutg0wAAAGGET
http://127.0.0.1:7777/socket.io/?EIO=4&transport=polling&t=Nz5mBav&sid=DKQAS0fxzXUutg0wAAAGws://127.0.0.1:7777/socket.io/?EIO=4&transport=websocket&sid=DKQAS0fxzXUutg0wAAAG
In ws, you can see the data transmitted by the socket
HTTP request success status code is generally 200, ws Status Code is 101 Switching Protocols
2. Deploy the interface to add authentication
The above is only the function implemented by the interface, and there is no permission control. After anyone knows the interface address, they can request the interface through postman to trigger the deployment. As shown below
For the sake of security, we add authentication to the interface here, and add a password login function to the front end. Here, koa-session is used for authentication, and only the login state can request success
// server/indexAuth.js
// npm install koa-session koa-bodyparser --save
// ..
const session = require("koa-session");
const bodyParser = require("koa-bodyparser"); // post 请求参数解析
const app = new Koa();
const router = new KoaRouter();
app.use(bodyParser()); // 处理 post 请求参数
// 集成 session
app.keys = [`自定义安全字符串`]; // 'some secret hurr'
const CONFIG = {
key: "koa:sess" /** (string) cookie key (default is koa:sess) */,
/** (number || 'session') maxAge in ms (default is 1 days) */
/** 'session' will result in a cookie that expires when session/browser is closed */
/** Warning: If a session cookie is stolen, this cookie will never expire */
maxAge: 0.5 * 3600 * 1000, // 0.5h
overwrite: true /** (boolean) can overwrite or not (default true) */,
httpOnly: true /** (boolean) httpOnly or not (default true) */,
signed: true /** (boolean) signed or not (default true) */,
rolling: false /** (boolean) Force a session identifier cookie to be set on every response. The expiration is reset to the original maxAge, resetting the expiration countdown. (default is false) */,
renew: false /** (boolean) renew session when session is nearly expired, so we can always keep user logged in. (default is false)*/,
};
app.use(session(CONFIG, app));
router.post("/login", async (ctx) => {
let code = 0;
let msg = "登录成功";
let { password } = ctx.request.body;
if (password === `888888`) { // 888888 为设置的密码
ctx.session.isLogin = true;
} else {
code = -1;
msg = "密码错误";
}
ctx.body = {
code,
msg,
};
});
router.post("/deploy", async (ctx) => {
if (!ctx.session.isLogin) {
ctx.body = {
code: -2,
msg: "未登录",
};
return;
}
// 有登录态，执行部署
})
复制代码Front-end related changes, add a password input box and a login button
<!-- frontend/indexAuth.html 注意id="app"包裹 -->
<div class="login-area">
<div v-if="!isLogin">
<el-input v-model="password" type="password" style="width: 200px;"></el-input>
<el-button type="primary" @click="login">登录</el-button>
</div>
<div v-else>已登录</div>
</div>
<script>
data() {
return {
isLogin: false,
password: ''
}
},
methods: {
login() {
if (!this.password) {
this.$message.warning('请输入密码')
return
}
axios.post('/login', { password: this.password })
.then((response) => {
console.log(response.data);
let { code, msg } = response.data
if (code === 0) {
this.isLogin = true
} else {
this.$message.error(msg)
}
})
.catch(function (err) {
console.log(err);
this.$message.error(err.message)
})
}
}
</script>
复制代码node server/indexAuth.js, open 127.0.0.1:7777/indexAuth.html, and deploy after successful login
3. Packaged into an npm package cli tool
Why package it into an npm package and use a command line tool to start the service. It is mainly simple and easy to use. If you do not use the command line tool form, you need three steps:
Download the code to the server first
npm install
node index.js or pm2 start index.js -n xxx to start the service
Changing to npm package command-line tool form only requires the following two steps, and it saves time
npm install zuo-deploy pm2 -g
Running zuodeploy start will automatically start the service with pm2
Let's first look at a simple example, create an npm package and upload it to the official npm library steps
You need an npm account. If you don't have one, you can go to www.npmjs.com/[5 ] to register one. My username is 'guoqzuo'
Create a folder to store npm package content, such as npmPackage
In this directory, run npm init to initialize a package.json, the input name is the npm package name, here I set the name to 'zuoxiaobai-test'
There are two types of package names, ordinary package vue-cli and scoped package @vue/cli. For the difference, see what does \@ mean in front of the npm package\(The difference between vue-cli and \@vue/cli\) [6]
Generally, the default entry is index.js, which exposes a variable and a method
// index.js
module.exports = {
name: '写一个npm包',
doSomething() {
console.log('这个npm暴露一个方法')
}
}
复制代码In this way, you can publish directly, create a publish script, and execute it (chmod +x publish.sh;./publish.sh; under linux)
# publish.sh
npm config set registry=https://registry.npmjs.org
npm login # 登陆 ，如果有 OTP, 邮箱会接收到验证码，输入即可
# 登录成功后，短时间内会保存状态，可以直接 npm pubish
npm publish # 可能会提示名称已存在，换个名字，获取使用作用域包（@xxx/xxx）
npm config set registry=https://registry.npm.taobao.org # 还原淘宝镜像
复制代码
Go to npmjs.org to search for the corresponding package and you can see it
Using the npm package, create testNpm/index.js
const packageInfo = require('zuoxiaobai-test')
console.log(packageInfo)
packageInfo.doSomething()
复制代码In the testNpm directory, npm init initializes package.json, then npm install zuoxiaobai-test --save; then node index.js, the execution is as shown below, and the npm package is called normally
In this way, we know how to write an npm package and upload it to the official npm library.
Next, let's see how to integrate cli commands in npm packages. For example: npm install @vue/cli \-gAfter , a vue command will be added to the environment variable. Use vue create xx to initialize a project. Usually this form is the cli tool.
Generally, there is a bin attribute in package.json, which is used to create custom commands for the npm package
// package.json
"bin": {
"zuodeploy": "./bin/zuodeploy.js"
},
复制代码The configuration above means: After installing npm install xx -g globally, the zuodeploy command is generated, and when the command is run, bin/zuodeploy.js will be executed
When developing locally, after the configuration is complete, run sudo npm link in the current directory to link the zuodeploy command to the local environment variable. Running zuodeploy in any terminal will execute the file under the current project. Unlink can use npm unlink
Generally, cli will use commander to generate help documents and manage command logic. The code is as follows
// bin/zuodeploy.js
#!/usr/bin/env node
const { program } = require("commander");
const prompts = require("prompts");
program.version(require("../package.json").version);
program
.command("start")
.description("开启部署监听服务") // description + action 可防止查找 command拼接文件
.action(async () => {
const args = await prompts([
{
type: "number",
name: "port",
initial: 7777,
message: "请指定部署服务监听端口：",
validate: (value) =>
value !== "" && (value < 3000 || value > 10000)
? `端口号必须在 3000 - 10000 之间`
: true,
},
{
type: "password",
name: "password",
initial: "888888",
message: "请设置登录密码（默认：888888）",
validate: (value) => (value.length < 6 ? `密码需要 6 位以上` : true),
},
]);
require("./start")(args); // args 为 { port: 7777, password: '888888' }
});
program.parse();
复制代码Use commander to quickly manage and generate help documents, and assign the execution logic of specific instructions
In the above code, the start command is specified. When zuodeploy start executes, it first collects parameters by asking questions through prompts, and then executes bin/start.js
In start.js, we can copy all the code of server/index.js to complete zuodeploy start to start the service, click the deploy function
4. Improved stability - pm2 transformation
In order to improve stability, we can execute pm2 src/index.js in code in start.js, so that the service is more stable and reliable. In addition, log4js can be added to output logs with timestamps, which is helpful for troubleshooting.
Specific code reference: zuo-deploy-github [7]
All test demo addresses: zuo-deploy implementation demo - fedemo -github [8]
finally
Bringing together the above fragmented knowledge points is the implementation of zuo-deploy. The code is written casually. Welcome to star, fork, and improve PR!
other problems
Why is there only one html on the front end/client side without engineering
Front-end engineering way to organize code is relatively heavy, unnecessary
The function here is relatively simple, only the deployment button, deployment log viewing area, authentication (password input) area
Easy to deploy, you can access the static service directly by koa-static, no need to package and build
Why change from type: module to plain CommonJS
After configuring type: module in package.json, ES Modules are used by default, and some node methods will have some problems
Although it can be solved by modifying the file suffix to .cjs, but there are too many files, it is better to directly remove the type: module and use the default package form of node
__dirnamereport an error.__dirnameVery important for cli projects. When you need to use the files in the current project instead of the files in the directory where zuodeploy start is executed, you need to use __dirnamerequire("../package.json") is changed to import xx from '../package.json', there will be an error when importing JSON files
About this article
Author: Zuo Xiaobai, the front-end
https://juejin.cn/post/7070921715492061214
Node 社群
我组建了一个氛围特别好的 Node.js 社群，里面有很多 Node.js小伙伴，如果你对Node.js学习感兴趣的话（后续有计划也可以），我们可以一起进行Node.js相关的交流、学习、共建。下方加 考拉 好友回复「Node」即可。
如果你觉得这篇内容对你有帮助，我想请你帮我2个小忙：
1. 点个「在看」，让更多人也能看到这篇文章2. 订阅官方博客 www.inode.club 让我们一起成长
点赞和在看就是最大的支持❤️