Skip to content

Instantly share code, notes, and snippets.

@magnetikonline
Last active January 29, 2024 23:26
Show Gist options
  • Star 10 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save magnetikonline/bfaf2ada33c4922b1a7b0dc876b9aef4 to your computer and use it in GitHub Desktop.
Save magnetikonline/bfaf2ada33c4922b1a7b0dc876b9aef4 to your computer and use it in GitHub Desktop.
An example of recursion with Promises - reading recursive directories.

An example of recursion with Promises

A pattern for recursion with Promises - in this example, walking a directory structure.

Function flow

  • readDirRecursive() is called with a starting directory and will itself return a Promise.
  • Internally readDir() is called and passed starting directory to read from.
  • A list of directory items is returned by getItemList() as a Promise, which in turn is chained to getItemListStat() to stat each item to determine if file or directory.
  • Finalised list then passed to processItemList():
    • Files are added to accumulating fileList array.
    • Directories are added to readDirQueue.
    • If directories exist on readDirQueue, next directory is shifted off to another readDir() call and returned to parent Promise (this is the recursion).
  • Process continues until readDirQueue is exhausted at which point the final Promise will resolve by returning accumulated fileList from readDirRecursive().

Example

$ nodejs readdirrecursive.js
[ '/path/to/README.md',
  '/path/to/readdirrecursive.js',
  '/path/to/.git/COMMIT_EDITMSG',
  '/path/to/.git/HEAD',
  '/path/to/.git/config',
  '/path/to/.git/description',
  '/path/to/.git/index',
  '/path/to/.git/packed-refs',
  '/path/to/.git/hooks/applypatch-msg.sample',
  '/path/to/.git/hooks/commit-msg.sample',
  '/path/to/.git/hooks/post-update.sample',
  '/path/to/.git/hooks/pre-applypatch.sample',
  '/path/to/.git/hooks/pre-commit.sample',
  '/path/to/.git/hooks/pre-push.sample',
  '/path/to/.git/hooks/pre-rebase.sample',
  '/path/to/.git/hooks/pre-receive.sample',
  '/path/to/.git/hooks/prepare-commit-msg.sample',
  '/path/to/.git/hooks/update.sample',
  '/path/to/.git/info/exclude',
  '/path/to/.git/info/refs',
  '/path/to/.git/logs/HEAD',
  '/path/to/.git/objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391',
  '/path/to/.git/objects/info/packs',
  '/path/to/.git/objects/pack/pack-d781ac0f9e797c9aa5683e5071e8db3f7af8e5f8.idx',
  '/path/to/.git/objects/pack/pack-d781ac0f9e797c9aa5683e5071e8db3f7af8e5f8.pack',
  '/path/to/.git/logs/refs/heads/master',
  '/path/to/.git/refs/remotes/origin/HEAD',
  '/path/to/.git/logs/refs/remotes/origin/HEAD',
  '/path/to/.git/logs/refs/remotes/origin/master' ]
'use strict';
const fs = require('fs'),
path = require('path');
function readDirRecursive(startDir) {
const readDirQueue = [],
fileList = [];
function readDir(dir) {
function getItemList(readDir) {
return new Promise((resolve,reject) => {
fs.readdir(readDir,(err,itemList) => {
if (err) {
return reject(err);
}
// resolve with parent path added to each item
resolve(itemList.map((item) => path.resolve(readDir,item)));
});
});
}
function getItemListStat(itemList) {
function getStat(itemPath) {
return new Promise((resolve,reject) => {
fs.stat(itemPath,(err,stat) => {
if (err) {
return reject(err);
}
// resolve with item path and if directory
resolve({ itemPath,isDirectory: stat.isDirectory() });
});
});
}
// stat all items in list
return Promise.all(itemList.map(getStat));
}
function processItemList(itemList) {
for (const { itemPath,isDirectory } of itemList) {
// if directory add to queue
if (isDirectory) {
readDirQueue.push(itemPath);
continue;
}
// add file to list
fileList.push(itemPath);
}
// if queue, process next item recursive
if (readDirQueue.length > 0) {
return readDir(readDirQueue.shift());
}
// finished - return file list
return fileList;
}
// read item list from directory, stat each item then walk result
return getItemList(dir)
.then(getItemListStat)
.then(processItemList);
}
// commence reading at the top
return readDir(startDir);
}
readDirRecursive('.')
.then((itemList) => console.log(itemList))
.catch((err) => console.log(err));
@songfarm-david
Copy link

// resolve with parent path added to each item
  resolve(itemList.map((item) => path.resolve(readDir,item)));

This line seems to be significant but I'm not clear if path.resolve is a method native to path and not the resolve executor of Promise. I'm not familiar with the details of using require to import modules.

@magnetikonline
Copy link
Author

Hello @songfarm-david - it's the resolve() function of the Node.js path module:

https://nodejs.org/api/path.html#path_path_resolve_paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment