How to use promises with reduce and how to choose between serial and parallel processing
本文译自How to use async functions with Array.reduce in Javascript - Tamás Sallai 。
In the first article , we introduced how async / await helps to execute asynchronous commands, but it does not help when processing collections asynchronously. In this article, we will study reducefunctions, which are the most versatile aggregate functions because it can simulate all other functions.
1. Array.reduce
Reduce iteratively constructs a value and returns it, it is not necessarily a collection. This is where the name comes from, because it reduces the value collected.
The iterative function gets the previous result ( memo called in the example below) and the current value e.
The following function sums the elements, starting at 0 (the second parameter reduce):
const arr = [1, 2, 3];
const syncRes = arr.reduce((memo, e) => {
return memo + e;
}, 0);
console.log(syncRes);
// 6
| memo | e | result |
|---|---|---|
| 0 (initial) | 1 | 1 |
| 1 | 2 | 3 |
| 3 | 3 | (Final result) 6 |
2. Asynchronous reduce
Asynchronous version is almost the same, but each iteration returns a Promise, and therefore memowill be the result of previously Promise. Iterated function requires awaitit to calculate the next result:
// utility function for sleeping
const sleep = (n) => new Promise((res) => setTimeout(res, n));
const arr = [1, 2, 3];
const asyncRes = await arr.reduce(async (memo, e) => {
await sleep(10);
return (await memo) + e;
}, 0);
console.log(asyncRes);
// 6
| memo | e | result |
|---|---|---|
| 0 (initial) | 1 | Promise (1) |
| Promise (1) | 2 | Promise (3) |
| Promise (3) | 3 | (Final result) Promise (6) |
The structure used async (memo, e) => await memo, reducemay be any asynchronous processing functions, and can be awaitedited.
3. Timing
When reducethere is an interesting property in concurrent. In the synchronous version, the elements are processed one-to-one, which is not surprising, because they depend on previous results. However, when the asynchronous reduceoperation, all iterated function will start to run in parallel, await memowaiting on a result only when needed.
3.1 await memo last
In the example above, all sleepperformed in parallel, since await memosuch a function is waiting to be executed after the function is completed.
const arr = [1, 2, 3];
const startTime = new Date().getTime();
const asyncRes = await arr.reduce(async (memo, e) => {
await sleep(10);
return (await memo) + e;
}, 0);
console.log(`Took ${new Date().getTime() - startTime} ms`);
// Took 11-13 ms
3.2 await memo first
But when they await memoappear first, these functions run in order:
const arr = [1, 2, 3];
const startTime = new Date().getTime();
const asyncRes = await arr.reduce(async (memo, e) => {
await memo;
await sleep(10);
return (await memo) + e;
}, 0);
console.log(`Took ${new Date().getTime() - startTime} ms`);
// Took 36-38 ms
This behavior is usually not a problem, which means that everything that does not depend on the previous result will be calculated immediately, and only the dependent part is waiting for the previous value.
3.3 When parallelism is important
But in some cases, it may not be feasible to do something in advance.
For example, I have a piece of code to print a different PDF, and use the pdf-liblibrary splice them into one file.
Achieve printPDFparallel running resource-intensive functions:
const result = await printingPages.reduce(async (memo, page) => {
const pdf = await PDFDocument.load(await printPDF(page));
const pdfDoc = await memo;
(await pdfDoc.copyPages(pdf, pdf.getPageIndices()))
.forEach((page) => pdfDoc.addPage(page));
return pdfDoc;
}, PDFDocument.create());
I noticed that when I have many pages to print, it will take up too much memory and slow down the whole process.
A simple change to make printPDFcalls on a waiting call is completed:
const result = await printingPages.reduce(async (memo, page) => {
const pdfDoc = await memo;
const pdf = await PDFDocument.load(await printPDF(page));
(await pdfDoc.copyPages(pdf, pdf.getPageIndices()))
.forEach((page) => pdfDoc.addPage(page));
return pdfDoc;
}, PDFDocument.create());
4 Conclusion
reduceFunctions can be easily converted to asynchronous functions, but it can be tricky to figure out parallelism. Fortunately, it rarely destroys anything, but in some resource-intensive or rate-limited operations, it is important to understand how to call the function.
Recommended reading
