I am building an express server to receive request (a dict with 10 items) from my react front end and then save the data to database. Below is my code. I found that my code is work and the query does save the record back to Db. But in each for loop, this error is returned in server. What cause this error and the MaxListenersExceededWarning?
The request data:
{{.....}, {.....}, {.....}, {.....}, {.....}} #10 item
Code:
connection.js:
const p = mysql.createPool({
"connectionLimit" : 100,
"host": "example.org",
"user": "test",
"password": "test",
"database": "test",
"multipleStatements": true
});
const getConnection = function(callback) {
p.getConnection(function(err, connection) {
callback(err, connection)
})
};
module.exports = getConnection
routers.js
router.post('/test', (req, res) => {
getConnection(function(err, conn){
if (err) {
return res.json({ success: false, error: err })
} else {
const dict = req.body;
Object.keys(dict).forEach(function(r){
#putting dict's value to query
query = "UPDATE ......;"
conn.query(query, function (err, result, fields) {
conn.release()
console.log(query)
if (err) {
console.log("err")
return res.json({ success: false, error: err });
}
});
});
}
});
return res.json({ success: true });
});
Error:
error: uncaughtException: Connection already released
Error: Connection already released
at Pool.releaseConnection (/home/node_modules/mysql/lib/Pool.js:138:13)
at PoolConnection.release (/home/node_modules/mysql/lib/PoolConnection.js:35:15)
at Query.<anonymous> (/home/routes/test.js:276:22)
at Query.<anonymous> (/home/node_modules/mysql/lib/Connection.js:526:10)
at Query._callback (/home/node_modules/mysql/lib/Connection.js:488:16)
at Query.Sequence.end (/home/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
at Query._handleFinalResultPacket (/home//node_modules/mysql/lib/protocol/sequences/Query.js:149:8)
at Query.OkPacket (/home//node_modules/mysql/lib/protocol/sequences/Query.js:74:10)
at Protocol._parsePacket (/home//node_modules/mysql/lib/protocol/Protocol.js:291:23)
at Parser._parsePacket (/home//node_modules/mysql/lib/protocol/Parser.js:433:10)
(node:15881) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added. Use emitter.setMaxListeners() to increase limit
One connection (conn) is being retrieved from the pool, and is used to launch 10 queries in the forEach loop.
When the first query finishes to run, the first step of its callback is: conn.release(). The connection is released.
When the second query finishes to run, its callback also tries to release the connection,causing the error.
This problem might be solved in multiple ways:
Solve using a counter
In the callback of the database query, before calling call.release, check the number of queries already processed, and only close the connection when the last product is being processed.
const dict = req.body;
// initialize counter
let itemCount = 0
, errors = []
Object.keys(dict).forEach(function(r){
#putting dict's value to query
query = "UPDATE ......;"
conn.query(query, function (err, result, fields) {
// check whether this is the last callback
if (itemCount === dict.length-1) {
conn.release()
let result = errors.length ? { success: false, error: errors } : { success: true }
res.json(result)
}
// increment counter
itemCount++
console.log(query)
if (err) {
console.log("err")
errors.push(err)
}
});
});
Edit: There is also an issue with the res.json calls: inside the code in the question, res.json({ success: true }) is always executed, without waiting for the queries' execution results. The modified code sample above calls res.json only once after the execution of all queries, this is the only place where res.json should be called. This implies modifying the client-side code so that it can handle an array of errors, rather than only one error.
Solve by using a recursive function instead of for loop.
It is not a good practice to use for loops for the execution of asynchronous code. You might run into Maximum call stack size exceeded errors whenever the data volume gets too large.
Instead, create a recursive function (e.g. updateDictItem) to process one update query at a time. Read more about the asynchronous patterns in node.js in this article.
Other possible enhancements
Rather than firing ten database queries, it is worth considering grouping all the updates in one MERGE update statement, otherwise doing all the updates in a TRANSACTION.