Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: tapjs/tapjs
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: e41b6737650a8d4559082ed608c591245a7fa49c
Choose a base ref
...
head repository: tapjs/tapjs
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: eaa008498b556ca8e6ca1ba7a55b877fe7032593
Choose a head ref

Commits on Jun 24, 2016

  1. Copy the full SHA
    4b73279 View commit details
  2. Copy the full SHA
    ca6b5b0 View commit details
  3. Update issues link URL

    Close #262
    dotproto authored and isaacs committed Jun 24, 2016
    Copy the full SHA
    6e0247e View commit details
  4. Clean up argument ordering logic in t.throws

    Also add a test for this uniquely flexible assertion method
    isaacs committed Jun 24, 2016
    Copy the full SHA
    6e637a5 View commit details
  5. make coverage piping error non-fatal

    If the coveralls.io API returns an error, such as an unrecognized token
    or some temporary service glitch, the test run should probably not end
    in a failure if it was otherwise successful at running all the tests.
    rmg authored and isaacs committed Jun 24, 2016
    Copy the full SHA
    afdd65a View commit details
  6. update nyc version

    isaacs committed Jun 24, 2016
    Copy the full SHA
    2a9ba20 View commit details
  7. v5.8.0

    isaacs committed Jun 24, 2016
    Copy the full SHA
    77727e6 View commit details

Commits on Jun 29, 2016

  1. root: Only produce output if requested

    This prevents require('tap') from necessarily producing the minimum TAP
    output.  If the user explicitly pipes somewhere, then it'll produce this
    minimum correct TAP, or if the user does anything that would produce
    output, the root TAP object will be piped to stdout.
    
    However, this means that a user can do something like:
    
        var t = require('tap')
        var myTest = new t.Test()
    
    without also hooking onto the process, writing to stdout, etc.
    
    Breaking change, but in my opinion, a correct one.
    isaacs committed Jun 29, 2016
    Copy the full SHA
    6b24d67 View commit details

Commits on Jun 30, 2016

  1. Remove built-in codecov.io support

    See #270
    isaacs committed Jun 30, 2016
    Copy the full SHA
    f8964a0 View commit details
  2. adds .taprc file support #228

    a few changes to .taprc file support #234
    
    includes:
    - moves parsing of `process.env.TAP` back to where it was
    - restructures dotfile tests
    - updates a test that was copied from another, but not updated
    kgraves authored and isaacs committed Jun 30, 2016
    Copy the full SHA
    54e73f4 View commit details
  3. remove deep-extension for rc file handling

    Deep extension isn't actually necessary, just plain old extension
    isaacs committed Jun 30, 2016
    Copy the full SHA
    b1c99b3 View commit details

Commits on Jul 1, 2016

  1. Copy the full SHA
    b7e048f View commit details
  2. Copy the full SHA
    ca5be4f View commit details
  3. shim home directory in rcfile test

    In case you have some configs at ~/.taprc
    isaacs committed Jul 1, 2016
    Copy the full SHA
    4d3b0e8 View commit details
  4. Document rc files

    isaacs committed Jul 1, 2016
    Copy the full SHA
    d87a857 View commit details
  5. Copy the full SHA
    f085de5 View commit details
  6. hard-code TAP_TIMEOUT in rcfiles test

    Otherwise running with coverage fails, because of bigger default
    isaacs committed Jul 1, 2016
    Copy the full SHA
    ec90a96 View commit details
  7. Child tests should inherit bail-on-fail from parent

    Unless explicitly specified in options object.
    isaacs committed Jul 1, 2016
    Copy the full SHA
    1f88f4a View commit details
  8. v6.0.0

    isaacs committed Jul 1, 2016
    Copy the full SHA
    a5d8cee View commit details
  9. adds 'diagnostic' flag to include diags always

    Passing tests typically don't get a yaml diagnostic block.
    
    With this, you can pass `diagnostic:true` to include a yaml block
    regardless of whether the test is a pass or fail.
    
    Fix #243
    isaacs committed Jul 1, 2016
    Copy the full SHA
    840fd12 View commit details
  10. v6.1.0

    isaacs committed Jul 1, 2016
    Copy the full SHA
    191eeed View commit details
  11. remove unused codecov.io dep

    isaacs committed Jul 1, 2016
    Copy the full SHA
    500f8ad View commit details
  12. v6.1.1

    isaacs committed Jul 1, 2016
    Copy the full SHA
    a976db2 View commit details
  13. signal-exit@3

    isaacs committed Jul 1, 2016
    Copy the full SHA
    3ace105 View commit details

Commits on Jul 15, 2016

  1. Add --test-arg=<argument> option

    Close #273
    isaacs committed Jul 15, 2016
    Copy the full SHA
    f745959 View commit details
  2. v6.2.0

    isaacs committed Jul 15, 2016
    Copy the full SHA
    8faa184 View commit details

Commits on Jul 18, 2016

  1. Copy the full SHA
    6eb21aa View commit details
  2. Copy the full SHA
    f068025 View commit details

Commits on Jul 30, 2016

  1. upgrade nyc to v7

    isaacs committed Jul 30, 2016
    Copy the full SHA
    f5b1b30 View commit details
  2. Copy the full SHA
    49354cd View commit details
  3. Copy the full SHA
    137a2bb View commit details
  4. v6.3.0

    isaacs committed Jul 30, 2016
    Copy the full SHA
    3cdc354 View commit details

Commits on Aug 10, 2016

  1. tap-parse@1.3.1

    This provides support for Test2-style buffered child tests.
    isaacs committed Aug 10, 2016
    Copy the full SHA
    8eb912f View commit details
  2. Move Subtest: comments to parent level

    This follows the approach used by Test2 in Perl land, and is
    quite a bit more readable.
    
    Both ways still parse just fine, but I think this is going to be a
    breaking change, because if anyone's piping TAP output to another
    parser, then it could upset things.
    isaacs committed Aug 10, 2016
    Copy the full SHA
    1d3f655 View commit details

Commits on Aug 11, 2016

  1. Revert "Move Subtest: comments to parent level"

    This reverts commit 1d3f655.
    
    Will do later, once tap-parser regression is dealt with.
    isaacs committed Aug 11, 2016
    Copy the full SHA
    0d870a8 View commit details

Commits on Aug 18, 2016

  1. Use tap-parser v2

    isaacs committed Aug 18, 2016
    Copy the full SHA
    d470907 View commit details
  2. Move Subtest: comments to parent level

    This follows the approach used by Test2 in Perl land, and is quite a bit
    more readable.
    
    Both ways still parse just fine, but I think this is going to be a
    breaking change, because if anyone's piping TAP output to another
    parser, then it could upset things.
    isaacs committed Aug 18, 2016
    Copy the full SHA
    f208773 View commit details
  3. tap-mocha-reporter v2

    isaacs committed Aug 18, 2016
    Copy the full SHA
    736e476 View commit details

Commits on Aug 19, 2016

  1. Support use of save file with --bail

    Fix #279
    isaacs committed Aug 19, 2016
    Copy the full SHA
    632f935 View commit details
  2. Don't nextTick root bailout exit

    But, still allow 'bailout' event handlers.
    
    Do this by overriding the tap.bailout function, rather than by setting
    an event handler before any user code has a chance to run.
    isaacs committed Aug 19, 2016
    1
    Copy the full SHA
    14cbf2e View commit details

Commits on Aug 23, 2016

  1. Don't rely on system 'head' exe for EPIPE test

    Node can do this, and it works more consistently.
    isaacs committed Aug 23, 2016
    Copy the full SHA
    3e4f6da View commit details
  2. Copy the full SHA
    4f50bf0 View commit details
  3. Handing timeout SIGTERMs after explicit root test end

    Previously, it'd just throw a 'test after end' exception because the
    onTimeout method emits a fail().  This prints something at least a
    little useful to stderr before exiting in failure.
    isaacs committed Aug 23, 2016
    Copy the full SHA
    fbc83b0 View commit details
  4. epipe test: just use js piping instead of fd sharing

    For some reason, that's only connecting like half the time.
    
    Really weird.
    isaacs committed Aug 23, 2016
    Copy the full SHA
    4a2011f View commit details

Commits on Aug 24, 2016

  1. keep for..in loops from iterating inherited keys

    patch: add hasOwnProperty checks into for...in loops to prevent iteration over unexpected members
    
    resolve conflicts
    
    use `Object.keys(...).forEach(...` everywhere instead of `hasOwnProperty(...)`
    
    - Bind `this` in `forEach()` iterations if necessary,
    - remove useless property checks and
    - simplify the corresponding test.
    
    fix #268
    schwarzkopfb authored and isaacs committed Aug 24, 2016
    Copy the full SHA
    ca686f6 View commit details
  2. Copy the full SHA
    14bec9f View commit details

Commits on Aug 26, 2016

  1. Remove test that blows up when coverage is engaged

    The same things are still tested, but this test file makes `npm test`
    spew tons of errors.  Revert this commit when istanbul is resilient
    against Object.prototype mutations.
    
    See istanbuljs-archived-repos/istanbul-lib-instrument#18
    isaacs committed Aug 26, 2016
    Copy the full SHA
    ccce5a6 View commit details
  2. add the --100 flag for 100% coverage check

    I keep having to add 4 flags to module tests for full coverage,
    why not just have a single option for it?
    isaacs committed Aug 26, 2016
    Copy the full SHA
    e49616d View commit details

Commits on Aug 27, 2016

  1. Copy the full SHA
    31f5467 View commit details
  2. allow SIGKILL in runner timeout tests

    Especially when coverage is engaged, this test can run slow, resulting in
    a SIGTERM not killing the process in time and a SIGKILL being issued.
    isaacs committed Aug 27, 2016
    Copy the full SHA
    1c7d987 View commit details
Showing 491 changed files with 41,064 additions and 9,920 deletions.
11 changes: 11 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Set the default behavior, in case people don't have core.autocrlf set
* text=auto

# Require Unix line endings
* text eol=lf

# Denote all files that are truly binary and should not be modified.
*.png binary
*.jpg binary
*.ico binary
*.gif binary
3 changes: 3 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# These are supported funding model platforms

github: [isaacs]
29 changes: 25 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,25 @@
node_modules/
coverage/
.nyc_output/
nyc_output/
/*
/.*

!bin/
!lib/
!docs/
/docs/public
/docs/node_modules
/docs/.cache
!package.json
!package-lock.json
!README.md
!CONTRIBUTING.md
!LICENSE
!CHANGELOG.md
!example/
!scripts/
!tap-snapshots/
!test/
!.travis.yml
!.gitignore
!.gitattributes
!coverage-map.js
!postpublish.sh
!netlify.toml
20 changes: 14 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
sudo: false
language: node_js

node_js:
- '0.10'
- '0.12'
- '4'
- '5'
- '6'
- 12
- 10
- 8

os:
- linux

cache:
directories:
- $HOME/.npm

notifications:
email: false
140 changes: 2 additions & 138 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,138 +1,2 @@
## 5.4

Support passing in a class to `t.throws`, rather than an Error
sample object.

## 5.3

Return a `Promise` object from `t.test()`, `t.spawn()`, and
`t.stdin()`.

## 5.2

Adds `t.beforeEach()` and `t.afterEach()`.

## 5.1

All about the cli flags!

Support `--node-arg=...` and `--nyc-arg=...` command line flags.

Add support for coverage checking using `--statements=95` etc.

Test for executable-ness more consistently across platforms.

## 5.0

Make it impossible to `try/catch` out of plan/end abuses. Calling
`t.end()` more than once, or having a number of tests that doesn't
match the `plan()` number, is always a failure.

Push thrown errors to the front of the action queue. This means that,
even if other things are pending, an uncaught exception or a plan/end
bug, will always take the highest priority in what gets output.

Many updates to nyc, spawn-wrap, and foreground-child, so that tap now
reliably works on Windows (and a [ci to prove
it](https://ci.appveyor.com/project/isaacs/node-tap).)

Moved into the [tapjs org](https://github.com/tapjs).

## 4.0

Raise an error if `t.end()` is explicitly called more than once. This
is a breaking change, because it can cause previously-passing tests to
fail, if they did `t.end()` in multiple places.

Support promises returned by mochalike functions.

## 3.1

Support sending coverage output to both codecov.io and coveralls.io.

## 3.0

Upgrade to nyc 5. This means that `config.nyc.exclude` arrays in
`package.json` now take globs instead of regular expressions.

## 2.3

Use the name of the function supplied to `t.test(fn)` as the test name
if a string name is not provided.

Better support for sparse arrays.

## 2.2

Add support for Codecov.io as well as Coveralls.io.

Catch failures that come after an otherwise successful test set.

Fix timing of `t.on('end')` so that event fires *before* the next
child test is started, instead of immediately after it.

`t.throws()` can now be supplied a regexp for the expected Error
message.

## 2.1

Exit in failure on root test bailout.

Support promises returned by `t.test(fn)` function.

## 2.0

Update matching behavior using [tmatch](http://npm.im/tmatch). This
is a breaking change to `t.match`, `t.similar`, `t.has`, etc., but
brings them more in line with what people epirically seem to expect
these functions to do.

Deal with pending handles left open when a child process gets a
`SIGTERM` on timeout.

Remove domains in favor of more reliable and less invasive state and
error-catching bookkeeping.

## 1.4

Add `t.contains()` alias for `t.match()`.

Use `deeper` for deep object similarity testing.

Treat unfinished tests as failures.

Add support for pragmas in TAP output.

## 1.3

Bind all Test methods to object.

Add `t.tearDown()`, `t.autoend()`, so that the root export is Just
Another Test Object, which just happens to be piping to stdout.

Support getting an error object in bailout()

## 1.2

Better support for exit status codes.

## 1.1

Add coverage using nyc.

If a `COVERALLS_REPO_TOKEN` is provided, then run tests with coverage,
and pipe to coveralls.

## 1.0

Complete rewrite from 0.x.

Child tests implemented as nested TAP output, similar to Perl's `Test::More`.

## 0.x

The 0.x versions used a "flattened" approach to child tests, which
requires some bookkeeping.

It worked, mostly, but its primary success was inspiring
[tape](http://npm.im/tape) and tap v1 and beyond.
Please see [the tap website](http://www.node-tap.org/changelog/) for
the curated changelog.
22 changes: 10 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
- Check the [issues](https://github.com/taprc/node-tap/issues) to see
Please consider signing [the neveragain.tech pledge](http://neveragain.tech/)

- Check the [issues](https://github.com/tapjs/node-tap/issues) to see
stuff that is likely to be accepted.
- Every patch should have a new test that fails without the patch and
passes with the patch.
- All tests should pass on Node 0.8 and above. If some tests have to
be skipped for very old Node versions that's fine, but the
functionality should still work as intended.
- Run `npm run regen-fixtures` to re-generate the output tests
whenever output is changed. However, when you do this, make sure to
check the change to ensure that it's what you intended, and that it
didn't cause any other inadvertent changes.
- Prefer adding cases to an existing test rather than writing a new
one from scratch. For example, add a new test in `test/test/*.js`
rather than create a new test that validates test output.
- Docs should be changed on the `gh-pages` branch
- All tests should pass on Node 8 and above. If some tests have to be
skipped for very old Node versions that's fine, but the functionality
should still work as intended.
- Run `npm run snap` to re-generate the output tests whenever output is
changed. However, when you do this, make sure to check the change to
ensure that it's what you intended, and that it didn't cause any other
inadvertent changes.
150 changes: 143 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -3,10 +3,12 @@
A <abbr title="Test Anything Protocol">TAP</abbr> test framework for
Node.js.

[![Build Status](https://travis-ci.org/tapjs/node-tap.svg?branch=master)](https://travis-ci.org/tapjs/node-tap) [![Build Status](https://ci.appveyor.com/api/projects/status/913p1ypf21gf4leu?svg=true)](https://ci.appveyor.com/project/isaacs/node-tap) [![Coverage Status](https://coveralls.io/repos/tapjs/node-tap/badge.svg?branch=master&service=github)](https://coveralls.io/github/tapjs/node-tap?branch=master)
[![Build Status](https://travis-ci.org/tapjs/node-tap.svg?branch=master)](https://travis-ci.org/tapjs/node-tap)

It includes a command line test runner for consuming TAP-generating
test scripts, and a JavaScript framework for writing such scripts.
_Just wanna see some code? [Get started!](http://www.node-tap.org/basics/)_

It includes a command line test runner for consuming TAP-generating test
scripts, and a JavaScript framework for writing such scripts.

* [Getting started guide](http://www.node-tap.org/basics/)
* Built-in [test coverage](http://www.node-tap.org/coverage/)
@@ -15,8 +17,142 @@ test scripts, and a JavaScript framework for writing such scripts.
* Great [promise support](http://www.node-tap.org/promises/)
* Comprehensive [assert library](http://www.node-tap.org/asserts/)
* Other [advanced stuff](http://www.node-tap.org/advanced/)
* [Command-line interface](http://www.node-tap.org/cli/) for running
tests (whether they use node-tap or not)
* Mocha-like [BDD DSL](http://www.node-tap.org/mochalike/)
* [Parallel Testing](http://www.node-tap.org/parallel/)
* [Command-line interface](http://www.node-tap.org/cli/) for running tests
(whether they use node-tap or not)

See [the changelog](http://www.node-tap.org/changelog/) for recent updates,
or just get started with [the basics](http://www.node-tap.org/basics/).

All this is too much to manage in a single README file, so head over to
[the website](http://www.node-tap.org/) to learn more.

## Why TAP?

Why should you use this thing!? **LET ME TELL YOU!**

Just kidding.

Most frameworks spend a lot of their documentation telling you why they're
the greatest. I'm not going to do that.

### <i lang="it" title="all tastes are tastes">tutti i gusti sono gusti</i>

Software testing is a software and user experience design challenge that
balances on the intersection of many conflicting demands.

Node-tap is based on [my](http://izs.me) opinions about how a test
framework should work, and what it should let you do. I do _not_ have any
opinion about whether or not you share those opinions. If you do share
them, you will probably enjoy this test library.

1. **Test files should be "normal" programs that can be run directly.**

That means that it can't require a special runner that puts magic
functions into a global space. `node test.js` is a perfectly ok way to
run a test, and it ought to function exactly the same as when it's run
by the fancy runner with reporting and such. JavaScript tests should be
JavaScript programs; not english-language poems with weird punctuation.

2. **Test output should be connected to the structure of the test file in a
way that is easy to determine.**

That means not unnecessarily deferring test functions until `nextTick`,
because that would shift the order of `console.log` output. Synchronous
tests should be synchronous.

3. **Test files should be run in separate processes.**

That means that it can't use `require()` to load test files. Doing
`node ./test.js` must be the exact same sort of environment for the test
as doing `test-runner ./test.js`. Doing `node test/1.js; node
test/2.js` should be equivalent (from the test's point of view) to doing
`test-runner test/*.js`. This prevents tests from becoming implicitly
dependent on one anothers' globals.

4. **Assertions should not normally throw (but throws MUST be handled
nicely).**

I frequently write programs that have many hundreds of assertions based
on some list of test cases. If the first failure throws, then I don't
know if I've failed 100 tests or 1, without wrapping everything in a
try-catch. Furthermore, I usually want to see some kind of output or
reporting to verify that each one actually ran.

Basically, it should be your decision whether you want to throw or not.
The test framework shouldn't force that on you, and should make either
case easy.

5. **Test reporting should be separate from the test process, included in
the framework, and enabled by default for humans.**

The [raw test output](https://www.node-tap.org/tap-format/) should be
machine-parseable and human-intelligible, and a separate process should
consume test output and turn it into a [pretty summarized
report](https://www.node-tap.org/reporting/). This means that test data
can be stored and parsed later, dug into for additional details, and so
on. Also: nyan cat.

6. **Writing tests should be easy, maybe even fun.**

The lower the barrier to entry for writing new tests, the more tests get
written. That means that there should be a relatively small vocabulary
of actions that I need to remember as a test author. There is no
benefit to having a distinction between a "suite" and a "subtest".
Fancy DSLs are pretty, but more to remember.

That being said, if you return a Promise, or use a DSL that throws a
decorated error, then the test framework should Just Work in a way that
helps a human being understand the situation.

7. **Tests should output enough data to diagnose a failure, and no more or
less.**

Stack traces pointing at JS internals or the guts of the test framework
itself are not helpful. A test framework is a serious UX challenge, and
should be treated with care.

8. **Test coverage should be included.**

Running tests with coverage changes the way that you think about your
programs, and provides much deeper insight. Node-tap bundles
[NYC](https://istanbul.js.org/) for this.

It _does_ necessarily change the nature of the environment a little bit.
But in this case, it's worth it, and NYC has come a long way towards
maintaining this promise.

Coverage _enforcement_ is not on by default, but I strongly encourage
it. You can put `"tap":{"check-coverage":true}` in your package.json,
or pass [`--100`](https://www.node-tap.org/100/) on the command line.
In a future version, it will likely be enabled by default.

9. **Tests should not require more building than your code.**

Babel and Webpack are lovely and fine. But if your code doesn't require
compilation, then I think your tests shouldn't either. Tap is extremely
[promise-aware](https://www.node-tap.org/promises/). JSX, TypeScript,
Flow, and ES-Modules are
[built-in](https://www.node-tap.org/using-with/) when tests are run by
the tap CLI.

10. **Tests should run as fast as possible, given all the prior
considerations.**

As of version 10, tap supports [parallel
tests](https://www.node-tap.org/parallel/). As of version 13, the test
runner defaults to running the same number of parallel tests as there
are CPUs on the system.

This makes tests significantly faster in almost every case, on any machine
with multiple cores.

Software testing should help you build software. It should be a security
blanket and a quality ratchet, giving you the support to undertake massive
refactoring and fix bugs without worrying. It shouldn't be a purification
rite or a hazing ritual.

All this is too much to manage in a single README file, so head over
to [the website](http://www.node-tap.org/) to learn more.
There are many opinions left off of this list! Reasonable people can
disagree. But if you find yourself nodding along, [maybe tap is for
you](https://www.node-tap.org/basics/).
Loading