Created
January 26, 2024 04:14
-
-
Save onmax/3550417272e8c7d0cabb984ca2d90474 to your computer and use it in GitHub Desktop.
Effective approach to processing items in batches asynchronously in JavaScript. This function processByBatches allows parallel processing of items, with a customizable batch size. There's room for enhancements, like incorporating Promise.allSettled or adding more options. Feedback and improvements are welcome!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
interface ProcessByBatchesOptions { | |
/** | |
* The number of items to process in parallel. | |
* @default 10 | |
*/ | |
batchSize?: number | |
} | |
async function processByBatches<ItemType, ResultType>( | |
items: ItemType[], | |
callback: (item: ItemType) => Promise<ResultType>, | |
options: ProcessByBatchesOptions = {}, | |
): Promise<ResultType[]> { | |
const { batchSize = 10 } = options | |
const batches = [] | |
for (let i = 0; i < items.length; i += batchSize) | |
batches.push(items.slice(i, i + batchSize)) | |
const maybePromise: Promise<ResultType[]>[] = [] | |
for (const batch of batches) | |
maybePromise.push(Promise.all(batch.map(callback))) | |
return Promise.all(maybePromise).then(results => results.flat()) | |
} | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment