Converting a javascript loop to be asynchronous and functional

John Harrison :

I'm trying to make the following code more functional and asynchronous, but not having a lot of luck!

So the api calls can only take 5 id's at a time, which is why I created splitListOfIds and am looping through it.The second api call is dependent on the first api call

The code below works, but is slow and I get ESLINT errors with this code though and want to make it more functional and asynchronous. I've been trying to use Promises but not sure how to do it. Ideally want to get rid of the for loop.

const splitListOfIds = [
    [1,2,3,4,5],
    [6,7,8,9,10]
]

const allPolicies = []

for (let i = 0; i < splitListOfIds.length; i++) {
    const people = await peopleApi.getPeople(splitListOfIds[i])
    const policyIds = people.map(p => p.policyId)
    const policies = await policyApi.getPolicyDetails(policyIds)
    allPolicies.push(policies)
}

Jacob :

If this API allows you to do requests in parallel, you could take this kind of approach:

const allPolicies = await Promise.all(splitListOfIds.map(async peopleIds => {
  const people = await peopleApi.getPeople(peopleIds)
  const policyIds = people.map(p => p.policyId)
  const policies = await policyApi.getPolicyDetails(policyIds)
  return policies;
}));

Depending on what the APIs are doing in the background, this can get you in trouble if you've got too many things in parallel. If that's the case, then you need to implement some kind of max parallelism mechanism, something like this (untested, though there are probably libraries for this already):

async function processInParallel(maxParallelism, data, taskFn) {
  const results = [];
  const inFlight = new Set();  

  for (let i = 0; i < data.length; i++) {
    while (inFlight.size >= maxParallelism) {
      // Wait for at least one to complete
      await Promise.race([...inFlight]);
    } 

    const task = taskFn(data[i]).then(result => {
      results.push(result);
      inFlight.delete(task);
    });
    inFlight.add(task);
  }

  await Promise.all([...inFlight]);

  return results;
}

// Usage
const allPolicies = await processInParallel(10, splitListOfIds, async peopleIds => {
  const people = await peopleApi.getPeople(peopleIds)
  const policyIds = people.map(p => p.policyId)
  const policies = await policyApi.getPolicyDetails(policyIds)
  return policies;
}));