Skip to content

Commit

Permalink
add more tests for filter, update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@manidlou
manidlou committed Apr 23, 2017
1 parent 739a6e3 commit e56432d
Show file tree
Hide file tree
Showing 2 changed files with 66 additions and 122 deletions.
74 changes: 42 additions & 32 deletions README.md
View file
@@ -1,5 +1,5 @@
klaw-sync
=========
node-klaw-sync
==============

[![npm Package](https://img.shields.io/npm/v/klaw-sync.svg?style=flat-square)](https://www.npmjs.com/package/klaw-sync)
[![Build Status](https://travis-ci.org/manidlou/node-klaw-sync.svg?branch=master)](https://travis-ci.org/manidlou/node-klaw-sync)
Expand All @@ -9,6 +9,10 @@ klaw-sync

`klaw-sync` is a Node.js recursive file system walker, which is the synchronous counterpart of [klaw](https://github.com/jprichardson/node-klaw). It lists all files and directories inside a directory recursively and returns an array of objects that each object has two properties: `path` and `stats`. `path` is the full path of the file or directory and `stats` is an instance of [fs.Stats](https://nodejs.org/api/fs.html#fs_class_fs_stats).

Breaking Change
---------------
`ignore` option is no longer supported (as of `v2.0.0`). Instead, you can use `filter` option to achieve the same functionality. Please see [examples](#examples).

Install
-------

Expand All @@ -19,29 +23,30 @@ Usage

### klawSync(directory[, options])

- `directory` `{String}`
- `options` `{Object}` *optional* (all options are `false` by default)
- `ignore` `{String | Array<String>}` any paths or [micromatch](https://github.com/jonschlinkert/micromatch#features) patterns to ignore (can be string or an array of strings)
- `nodir` `{Boolean}` return only files (ignore directories)
- `nofile` `{Boolean}` return only directories (ignore files)
- `directory` `<String>`
- `options` `<Object>` *optional* (all options are `false` by default)
- `nodir` `<Boolean>` return only files (ignore directories)
- `nofile` `<Boolean>` return only directories (ignore files)
- `noRecursiveOnFilter` `<Boolean>` if your `filter` function applies to **directories only**, set `true` to prevent unnecessary traversal of unwanted directories
- `filter` `<Function>` function that gets one argument `fn({path: '', stats: {}})` and returns true to include or false to exclude the item

- return: `{Array<Object>}` `[{path: '', stats: {}}]`
- return: `<Array<Object>>` `[{path: '', stats: {}}]`

Examples
--------

```js
var klawSync = require('klaw-sync')
var paths = klawSync('/some/dir')
const klawSync = require('klaw-sync')
const paths = klawSync('/some/dir')
// paths = [{path: '/some/dir/dir1', stats: {}}, {path: '/some/dir/file1', stats: {}}]
```

_**catch error**_

```js
var klawSync = require('klaw-sync')
const klawSync = require('klaw-sync')

var paths
let paths
try {
paths = klawSync('/some/dir')
} catch (er) {
Expand All @@ -53,38 +58,48 @@ console.dir(paths)
_**files only**_

```js
var klawSync = require('klaw-sync')
var files = klawSync('/some/dir', {nodir: true})
const klawSync = require('klaw-sync')
const files = klawSync('/some/dir', {nodir: true})
// files = [{path: '/some/dir/file1', stats: {}}, {path: '/some/dir/file2', stats: {}}]
```

_**directories only**_

```js
var klawSync = require('klaw-sync')
var dirs = klawSync('/some/dir', {nofile: true})
const klawSync = require('klaw-sync')
const dirs = klawSync('/some/dir', {nofile: true})
// dirs = [{path: '/some/dir/dir1', stats: {}}, {path: '/some/dir/dir2', stats: {}}]
```

_**ignore `node_modules`**_

Notice here because the `filter` function is applied to directories only, `noRecursiveOnFilter: true` is used since we don't want anything from `node_modules` in this case.

```js
var klawSync = require('klaw-sync')
var paths = klawSync('/some/dir', {ignore: 'node_modules'})
const klawSync = require('klaw-sync')

const filterFn = item => item.path.indexOf('node_modules') < 0
const paths = klawSync('/some/dir', { filter: filterFn, noRecursiveOnFilter: true })
```

_**ignore `node_modules` and `.git` using [micromatch](https://github.com/jonschlinkert/micromatch#features) patterns**_
_**ignore `node_modules` and `.git`**_

```js
var klawSync = require('klaw-sync')
var paths = klawSync('/some/dir', {ignore: '{node_modules,.git}'})
const klawSync = require('klaw-sync')

const filterFn = item => item.path.indexOf('node_modules') < 0 && item.path.indexOf('.git') < 0
const paths = klawSync('/some/dir', { filter: filterFn, noRecursiveOnFilter: true })
```

_**ignore `node_modules`, `.git` and all `*.js` files using [micromatch](https://github.com/jonschlinkert/micromatch#features) patterns**_
_**get all `js` files**_

Notice here `noRecursiveOnFilter` is not required since we are interested in all `js` files. In other words, although the `filter` function doesn't pass for directories, we still want to read them and see if they have any `js` files.

```js
var klawSync = require('klaw-sync')
var paths = klawSync('/some/dir', {ignore: ['{node_modules,.git}', '*.js']})
const klawSync = require('klaw-sync')

const filterFn = item => path.extname(item.path) === '.js'
const paths = klawSync('/some/dir', { filter: filterFn })
```

Run tests
Expand All @@ -100,24 +115,19 @@ lint & unit: `npm test`
Performance compare to other similar modules
-----------------------------------------------

The `bm.js` runs some basic [benchmark](https://github.com/bestiejs/benchmark.js) tests for two cases, `without --ignore` (basic usage) and `with --ignore`, on these modules:
The `bm.js` runs some basic [benchmark](https://github.com/bestiejs/benchmark.js) tests for two cases: basic usage and with `--nodir=true` (get files only), on these modules:

- `klaw-sync`
- [walk-sync](https://github.com/joliss/node-walk-sync)
- [glob.sync](https://github.com/isaacs/node-glob#globsyncpattern-options)

Just for fun, it turned out (as of January 25, 2017) for the most cases `klaw-sync` is faster than other modules!

#####run benchmark

To run benchmark, just specify the root `--dir=`. To ignore paths or patterns, use `-i` flag.
##### run benchmark

`npm run benchmark -- --dir=/some/dir`

`npm run benchmark -- --dir=/some/dir -i "node_modules"`

`npm run benchmark -- --dir=/some/dir -i "node_modules" -i "*.js"`

`npm run benchmark -- --dir=/some/dir --nodir=true`

Credit
------
Expand Down
114 changes: 24 additions & 90 deletions test/test.js
View file
Expand Up @@ -87,96 +87,6 @@ describe('klaw-sync', () => {
assert.deepEqual(dir.stats, dirsOnly[i].stats)
})
})
/*
it('should ignore if opts.ignore is path name', () => {
const dirToIgnore = path.join(TEST_DIR, 'node_modules')
fs.ensureDirSync(dirToIgnore)
const paths = [
{path: DIRS[0], stats: fs.statSync(DIRS[0])},
{path: FILES[0], stats: fs.statSync(FILES[0])},
{path: DIRS[1], stats: fs.statSync(DIRS[1])},
{path: DIRS[2], stats: fs.statSync(DIRS[2])},
{path: DIRS[3], stats: fs.statSync(DIRS[3])},
{path: FILES[1], stats: fs.statSync(FILES[1])},
{path: FILES[2], stats: fs.statSync(FILES[2])}
]
const items = klawSync(TEST_DIR, {ignore: 'node_modules'})
assert.equal(items.length, paths.length)
items.forEach((p, i) => {
assert.deepEqual(p, paths[i])
assert.strictEqual(p.path, paths[i].path)
assert.deepEqual(p.stats, paths[i].stats)
})
})
it('should ignore if opts.ignore is glob pattern', () => {
const dirToIgnore1 = path.join(TEST_DIR, 'node_modules')
const dirToIgnore2 = path.join(TEST_DIR, '.git')
fs.ensureDirSync(dirToIgnore1)
fs.ensureDirSync(dirToIgnore2)
const paths = [
{path: DIRS[0], stats: fs.statSync(DIRS[0])},
{path: FILES[0], stats: fs.statSync(FILES[0])},
{path: DIRS[1], stats: fs.statSync(DIRS[1])},
{path: DIRS[2], stats: fs.statSync(DIRS[2])},
{path: DIRS[3], stats: fs.statSync(DIRS[3])},
{path: FILES[1], stats: fs.statSync(FILES[1])},
{path: FILES[2], stats: fs.statSync(FILES[2])}
]
const items = klawSync(TEST_DIR, {ignore: '{node_modules,.git}'})
assert.equal(items.length, paths.length)
items.forEach((p, i) => {
assert.deepEqual(p, paths[i])
assert.strictEqual(p.path, paths[i].path)
assert.deepEqual(p.stats, paths[i].stats)
})
})
it('should ignore if opts.ignore is array of patterns', () => {
const dirToIgnore1 = path.join(TEST_DIR, 'node_modules')
const dirToIgnore2 = path.join(TEST_DIR, '.git')
const fileToIgnore1 = path.join(TEST_DIR, 'dir1', 'somefile.md')
const fileToIgnore2 = path.join(TEST_DIR, 'dir2', 'dir2_1', 'someotherfile.md')
fs.ensureDirSync(dirToIgnore1)
fs.ensureDirSync(dirToIgnore2)
fs.ensureFileSync(fileToIgnore1)
fs.ensureFileSync(fileToIgnore2)
const paths = [
{path: DIRS[0], stats: fs.statSync(DIRS[0])},
{path: FILES[0], stats: fs.statSync(FILES[0])},
{path: DIRS[1], stats: fs.statSync(DIRS[1])},
{path: DIRS[2], stats: fs.statSync(DIRS[2])},
{path: DIRS[3], stats: fs.statSync(DIRS[3])},
{path: FILES[1], stats: fs.statSync(FILES[1])},
{path: FILES[2], stats: fs.statSync(FILES[2])}
]
const items = klawSync(TEST_DIR, {ignore: ['node_modules', '.git', '.md']})
assert.equal(items.length, paths.length)
items.forEach((p, i) => {
assert.deepEqual(p, paths[i])
assert.strictEqual(p.path, paths[i].path)
assert.deepEqual(p.stats, paths[i].stats)
})
})
it('should ignore if opts.ignore is glob pattern with negation', () => {
const f1 = path.join(TEST_DIR, 'dir1', 'foo.js')
const f2 = path.join(TEST_DIR, 'dir2', 'dir2_1', 'bar.js')
fs.ensureFileSync(f1)
fs.ensureFileSync(f2)
const paths = [
{path: f1, stats: fs.statSync(f1)},
{path: f2, stats: fs.statSync(f2)}
]
const items = klawSync(TEST_DIR, {ignore: '.js'})
assert.equal(items.length, paths.length)
items.forEach((p, i) => {
assert.deepEqual(p, paths[i])
assert.strictEqual(p.path, paths[i].path)
assert.deepEqual(p.stats, paths[i].stats)
})
})
*/

describe('when opts.filter is true', () => {
it('should filter based on path', () => {
Expand Down Expand Up @@ -258,5 +168,29 @@ describe('klaw-sync', () => {
assert.deepEqual(p.stats, paths[i].stats)
})
})

it('should filter when it is used to ignore items', () => {
const dirToIgnore1 = path.join(TEST_DIR, 'node_modules')
const dirToIgnore2 = path.join(TEST_DIR, '.git')
fs.ensureDirSync(dirToIgnore1)
fs.ensureDirSync(dirToIgnore2)
const paths = [
{path: DIRS[0], stats: fs.statSync(DIRS[0])},
{path: FILES[0], stats: fs.statSync(FILES[0])},
{path: DIRS[1], stats: fs.statSync(DIRS[1])},
{path: DIRS[2], stats: fs.statSync(DIRS[2])},
{path: DIRS[3], stats: fs.statSync(DIRS[3])},
{path: FILES[1], stats: fs.statSync(FILES[1])},
{path: FILES[2], stats: fs.statSync(FILES[2])}
]
const filterFunc = i => i.path.indexOf('node_modules') < 0 && i.path.indexOf('.git') < 0
const items = klawSync(TEST_DIR, {filter: filterFunc, noRecursiveOnFilter: true})
assert.equal(items.length, paths.length)
items.forEach((p, i) => {
assert.deepEqual(p, paths[i])
assert.strictEqual(p.path, paths[i].path)
assert.deepEqual(p.stats, paths[i].stats)
})
})
})
})

0 comments on commit e56432d

Please sign in to comment.