nodejs also realize worker_threads a thread pool, but a static configuration, actually a lot of use we need to be dynamic, poolifier is a very good implementation of
the use of simple, flexible support fixed thread is and dynamic thread pool, the following is a simple learn to use (needs attention node version 12.x and above)
Project Preparation
- Project structure
├── README.md
├── package.json
├── src
│ ├── index.js
│ └── myworker. js
└── yarn.lock
- Code Description
package.json node and npm script defined dependent
{
"name": "node-thread-pool",
"version": "1.0.0",
"main": "src/index.js",
"license": "MIT",
"dependencies": {
"poolifier": "^1.0.0"
},
"scripts": {
"start": "node src/index.js",
"test:standard":"standard",
"test:standard-fix":"standard --fix"
},
"devDependencies": {
"standard": "^14.3.1"
}
}
src / index.js node inlet applications
'use strict'
// fix (node:2381) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 26 error listeners added. Use emitter.setMaxListeners() to increase limit
require('events').EventEmitter.defaultMaxListeners = 30
const { FixedThreadPool, DynamicThreadPool } = require('poolifier')
// a fixed thread pool
const pool = new FixedThreadPool(15,
'./src/myWorker.js',
{ errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker1 is online') })
// or a dynamic thread pool
const pool2 = new DynamicThreadPool(10, 100,
'./src/myWorker.js',
{ errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker2 is online') })
pool2. emitter. one ( 'Full Pool', () => console. log ( 'pool is full'))
// the execute method signature is the same for both implementations,
// so you can easy switch from one to another
pool.execute({ username: 'dalong', userage: 333 }).then(res => {
console.log(res)
}).catch(err => {
console.log(err)
})
pool2.execute({ username: 'dalong2', userage: 555 }).then(res => {
console.log(res)
}).catch(err => {
console.log(err)
})
src / myworker.js worker code defines
'use strict'
const { ThreadWorker } = require('poolifier')
function myFunction (data) {
return { ok: 1, data }
}
module.exports = new ThreadWorker(myFunction, { maxInactiveTime: 60000 })
Start && effect
- start up
yarn start
- effect
yarn start
yarn run v1.21.1
$ node src/index.js
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker1 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
worker2 is online
{ ok: 1, data: { username: 'dalong', userage: 333 } }
{ ok: 1, data: { username: 'dalong2', userage: 555 } }
Explanation
poolifier simplified cost-based nodejs worker_threads develop multithreaded applications, the use of simple, flexible
Reference material
https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads
https://github.com/pioardi/poolifier
https://github.com/rongfengliang/node-thread-pool-learning