Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: tapjs/tapjs
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 81f08e3eebea1e3e59aa4e40b2a610e785e0c119
Choose a base ref
...
head repository: tapjs/tapjs
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: bc49fb76608d4dc49dfed760797a1a93d00e2835
Choose a head ref

Commits on Dec 15, 2016

  1. neveragain.tech pledge request

    isaacs committed Dec 15, 2016
    Copy the full SHA
    0c8b426 View commit details

Commits on Dec 30, 2016

  1. wip: buffered test output

    re #305
    isaacs committed Dec 30, 2016
    Copy the full SHA
    b41618c View commit details
  2. Copy the full SHA
    0dcedc1 View commit details
  3. Copy the full SHA
    951f7ba View commit details
  4. Copy the full SHA
    81d35e1 View commit details
  5. Copy the full SHA
    00dae08 View commit details
  6. plan excess buffer test output

    isaacs committed Dec 30, 2016
    Copy the full SHA
    4ac5866 View commit details
  7. Copy the full SHA
    157ffdd View commit details
  8. Copy the full SHA
    86e82fa View commit details
  9. Support empty buffered tests as skips

    This requires an update to the parser, which also allows removing a lot
    of dead code.
    isaacs committed Dec 30, 2016
    Copy the full SHA
    1b2caa4 View commit details

Commits on Jan 3, 2017

  1. Handle buffered tests much more simply

    This also updates to make buffered child bailouts less clunky
    isaacs committed Jan 3, 2017
    Copy the full SHA
    4d481a3 View commit details
  2. Copy the full SHA
    7ac36fd View commit details
  3. Copy the full SHA
    966a1d2 View commit details
  4. tap-parser@3.0.5

    isaacs committed Jan 3, 2017
    Copy the full SHA
    1469b9d View commit details
  5. split up output tests by type, not name

    Juggling the alphabet is too hard as the number of tests and cases
    increases.  Passing two booleans to a function feels like it should
    probably be an options object, but whatever.  We can cross that bridge
    later.
    
    Also, this allows running a single case in an ad-hoc manner which is a
    bit more convenient in development.
    isaacs committed Jan 3, 2017
    Copy the full SHA
    9ea5a52 View commit details
  6. output tests for consuming static child test output

    Simulates cases where the 'TAP_etc' environs aren't handled in any sort
    of way in the child process test.
    
    Particularly, the output for bail-on-failure and buffered parent when
    the non-buffered non-bail-on-fail child has a failure midstream, is not
    very ideal.  Printing the extra data is debatably valuable, but the fact
    that it lists the test name as the bailout reason rather than the original
    failure is very unfortunate.
    
    Ideally, it'd be treated just like when a non-child-process test bails out.
    isaacs committed Jan 3, 2017
    Copy the full SHA
    c40ed06 View commit details

Commits on Jan 5, 2017

  1. Copy the full SHA
    2ed6355 View commit details
  2. Copy the full SHA
    8aeb886 View commit details

Commits on Jan 7, 2017

  1. Prune some excessive diagnostic output

    Also, move the code around a little bit so that it'll be more
    straightforward to have yaml diagnostics ahead of buffered child test
    output once that's supported by the parser.
    isaacs committed Jan 7, 2017
    Copy the full SHA
    55c23c5 View commit details
  2. Always put a \n after a yaml diag block

    Otherwise it's pretty crowded and noisy.
    
    However, to prevent double-spacing, also change some of the logic around
    how and when subtest asserts get spaced (always putting a space after a
    child test point).
    isaacs committed Jan 7, 2017
    Copy the full SHA
    67ae57e View commit details

Commits on Jan 8, 2017

  1. Copy the full SHA
    6f727c6 View commit details
  2. Copy the full SHA
    60efe95 View commit details
  3. Merge branch 'buffered-tests'

    isaacs committed Jan 8, 2017
    Copy the full SHA
    5d60097 View commit details
  4. bump tap-mocha-reporter dep

    isaacs committed Jan 8, 2017
    Copy the full SHA
    a3bb777 View commit details
  5. Copy the full SHA
    750b63d View commit details
  6. tap-parser@4.2.3

    isaacs committed Jan 8, 2017
    Copy the full SHA
    6f1e771 View commit details
  7. nyc@10

    isaacs committed Jan 8, 2017
    Copy the full SHA
    8b3b996 View commit details
  8. v9.0.0

    isaacs committed Jan 8, 2017
    Copy the full SHA
    aec8ece View commit details
  9. Copy the full SHA
    e354064 View commit details
  10. v9.0.1

    isaacs committed Jan 8, 2017
    Copy the full SHA
    c82c752 View commit details

Commits on Jan 9, 2017

  1. Copy the full SHA
    642fada View commit details
  2. refactoring and code organization

    Nudging slightly towards having a single 'main' method for a child test.
    Seriously considering making spawn test children, in-process subtest,
    and stdin subtests all inherit from some root class, just to get a
    consistent interface.
    
    Once the 'run the subtest' and 'collect the results' logic is fully
    separated, it'll be possible to have two run at once and collect them
    both up later, making parallel jobs possible.
    
    Having an enormous amount of tests makes this work even possible.
    Another helpful step may be to push for 100% coverage (or as close as is
    feasible.)
    isaacs committed Jan 9, 2017
    Copy the full SHA
    da59c8e View commit details
  3. v9.0.2

    isaacs committed Jan 9, 2017
    3
    Copy the full SHA
    05b9d9c View commit details
  4. Make tests pass on 0.10

    isaacs committed Jan 9, 2017
    Copy the full SHA
    5c820a8 View commit details
  5. Avoid slashes in output tests

    causing test failures on windows.
    isaacs committed Jan 9, 2017
    Copy the full SHA
    cddf1db View commit details
  6. v9.0.3

    isaacs committed Jan 9, 2017
    Copy the full SHA
    1a603cd View commit details

Commits on Jan 11, 2017

  1. Copy the full SHA
    6d1501e View commit details
  2. move 'test-point.js' to 'point.js'

    My hands REALLY want 'lib/t<tab>' to expand to 'lib/test.js'
    isaacs committed Jan 11, 2017
    Copy the full SHA
    f5962e9 View commit details

Commits on Jan 12, 2017

  1. Clean up queue management, loads of debugging

    It'd be good to move the debugging into a function so that I can turn it
    on and off with a NODE_DEBUG=tap env setting.
    
    Spawn still isn't working, but child Tests are good with buffering and
    parallelizing in the job pool.
    isaacs committed Jan 12, 2017
    Copy the full SHA
    ea7ee51 View commit details

Commits on Jan 16, 2017

  1. Getting closer

    - Implemented a root test handler (uncaught exceptions still not
      handled)
    - Promises can be returned by the cb
    - subtest methods return a promise
    - Fix an issue where an empty buffered test would not be handled
    isaacs committed Jan 16, 2017
    Copy the full SHA
    9a4ab01 View commit details

Commits on Jan 17, 2017

  1. Copy the full SHA
    eda82cd View commit details
  2. Copy the full SHA
    0503927 View commit details
  3. Timeout handling

    isaacs committed Jan 17, 2017
    Copy the full SHA
    0c68213 View commit details

Commits on Jan 18, 2017

  1. Copy the full SHA
    8121612 View commit details
  2. Copy the full SHA
    82d4553 View commit details
  3. Copy the full SHA
    adbcd73 View commit details
  4. add beforeEach/afterEach

    isaacs committed Jan 18, 2017
    Copy the full SHA
    bb37283 View commit details
  5. autoend and tearDown()

    isaacs committed Jan 18, 2017
    Copy the full SHA
    45723c4 View commit details
  6. Copy the full SHA
    c65eb55 View commit details
  7. Copy the full SHA
    179d713 View commit details
Showing 474 changed files with 37,248 additions and 11,160 deletions.
11 changes: 11 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Set the default behavior, in case people don't have core.autocrlf set
* text=auto

# Require Unix line endings
* text eol=lf

# Denote all files that are truly binary and should not be modified.
*.png binary
*.jpg binary
*.ico binary
*.gif binary
3 changes: 3 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# These are supported funding model platforms

github: [isaacs]
36 changes: 36 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: ci

on:
push:
paths-ignore:
- 'docs/**'
- 'docs-content/**'
- '*.md'
pull_request:
paths-ignore:
- 'docs/**'
- 'docs-content/**'
- '*.md'

jobs:
test:
runs-on: ${{ matrix.os }}

strategy:
matrix:
node-version: [10.0.x, 10.x, 12.0.x, 12.x, 14.0.x, 14.x, 15.x]
os: [ubuntu-latest]

steps:
- uses: actions/checkout@v2

- name: Use Node.js
uses: actions/setup-node@v2.1.4
with:
node-version: ${{ matrix.node-version }}
- name: Install
run: |
npm install --ignore-scripts
- name: Run tests
run: |
npm test --color=always -- -c -t0
30 changes: 26 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,26 @@
node_modules/
coverage/
.nyc_output/
nyc_output/
/*
/.*

!bin/
!lib/
!settings.js
!docs/
/docs/public
/docs/node_modules
/docs/.cache
!package.json
!package-lock.json
!README.md
!CONTRIBUTING.md
!LICENSE
!CHANGELOG.md
!example/
!scripts/
!tap-snapshots/
!test/
!.travis.yml
!.gitignore
!.gitattributes
!coverage-map.js
!postpublish.sh
!netlify.toml
7 changes: 0 additions & 7 deletions .travis.yml

This file was deleted.

20 changes: 9 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
Please consider signing [the neveragain.tech pledge](http://neveragain.tech/)

- Check the [issues](https://github.com/tapjs/node-tap/issues) to see
stuff that is likely to be accepted.
- Every patch should have a new test that fails without the patch and
passes with the patch.
- All tests should pass on Node 0.8 and above. If some tests have to
be skipped for very old Node versions that's fine, but the
functionality should still work as intended.
- Run `npm run regen-fixtures` to re-generate the output tests
whenever output is changed. However, when you do this, make sure to
check the change to ensure that it's what you intended, and that it
didn't cause any other inadvertent changes.
- Prefer adding cases to an existing test rather than writing a new
one from scratch. For example, add a new test in `test/test/*.js`
rather than create a new test that validates test output.
- Docs should be changed on the `gh-pages` branch
- All tests should pass on Node 8 and above. If some tests have to be
skipped for very old Node versions that's fine, but the functionality
should still work as intended.
- Run `npm run snap` to re-generate the output tests whenever output is
changed. However, when you do this, make sure to check the change to
ensure that it's what you intended, and that it didn't cause any other
inadvertent changes.
150 changes: 143 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -3,10 +3,12 @@
A <abbr title="Test Anything Protocol">TAP</abbr> test framework for
Node.js.

[![Build Status](https://travis-ci.org/tapjs/node-tap.svg?branch=master)](https://travis-ci.org/tapjs/node-tap) [![Build Status](https://ci.appveyor.com/api/projects/status/913p1ypf21gf4leu?svg=true)](https://ci.appveyor.com/project/isaacs/node-tap) [![Coverage Status](https://coveralls.io/repos/tapjs/node-tap/badge.svg?branch=master&service=github)](https://coveralls.io/github/tapjs/node-tap?branch=master)
![Build Status](https://github.com/tapjs/node-tap/workflows/ci/badge.svg)

It includes a command line test runner for consuming TAP-generating
test scripts, and a JavaScript framework for writing such scripts.
_Just wanna see some code? [Get started!](http://www.node-tap.org/basics/)_

It includes a command line test runner for consuming TAP-generating test
scripts, and a JavaScript framework for writing such scripts.

* [Getting started guide](http://www.node-tap.org/basics/)
* Built-in [test coverage](http://www.node-tap.org/coverage/)
@@ -15,8 +17,142 @@ test scripts, and a JavaScript framework for writing such scripts.
* Great [promise support](http://www.node-tap.org/promises/)
* Comprehensive [assert library](http://www.node-tap.org/asserts/)
* Other [advanced stuff](http://www.node-tap.org/advanced/)
* [Command-line interface](http://www.node-tap.org/cli/) for running
tests (whether they use node-tap or not)
* Mocha-like [BDD DSL](http://www.node-tap.org/mochalike/)
* [Parallel Testing](http://www.node-tap.org/parallel/)
* [Command-line interface](http://www.node-tap.org/cli/) for running tests
(whether they use node-tap or not)

See [the changelog](http://www.node-tap.org/changelog/) for recent updates,
or just get started with [the basics](http://www.node-tap.org/basics/).

All this is too much to manage in a single README file, so head over to
[the website](http://www.node-tap.org/) to learn more.

## Why TAP?

Why should you use this thing!? **LET ME TELL YOU!**

Just kidding.

Most frameworks spend a lot of their documentation telling you why they're
the greatest. I'm not going to do that.

### <i lang="it" title="all tastes are tastes">tutti i gusti sono gusti</i>

Software testing is a software and user experience design challenge that
balances on the intersection of many conflicting demands.

Node-tap is based on [my](http://izs.me) opinions about how a test
framework should work, and what it should let you do. I do _not_ have any
opinion about whether or not you share those opinions. If you do share
them, you will probably enjoy this test library.

1. **Test files should be "normal" programs that can be run directly.**

That means that it can't require a special runner that puts magic
functions into a global space. `node test.js` is a perfectly ok way to
run a test, and it ought to function exactly the same as when it's run
by the fancy runner with reporting and such. JavaScript tests should be
JavaScript programs; not english-language poems with weird punctuation.

2. **Test output should be connected to the structure of the test file in a
way that is easy to determine.**

That means not unnecessarily deferring test functions until `nextTick`,
because that would shift the order of `console.log` output. Synchronous
tests should be synchronous.

3. **Test files should be run in separate processes.**

That means that it can't use `require()` to load test files. Doing
`node ./test.js` must be the exact same sort of environment for the test
as doing `test-runner ./test.js`. Doing `node test/1.js; node
test/2.js` should be equivalent (from the test's point of view) to doing
`test-runner test/*.js`. This prevents tests from becoming implicitly
dependent on one anothers' globals.

4. **Assertions should not normally throw (but throws MUST be handled
nicely).**

I frequently write programs that have many hundreds of assertions based
on some list of test cases. If the first failure throws, then I don't
know if I've failed 100 tests or 1, without wrapping everything in a
try-catch. Furthermore, I usually want to see some kind of output or
reporting to verify that each one actually ran.

Basically, it should be your decision whether you want to throw or not.
The test framework shouldn't force that on you, and should make either
case easy.

5. **Test reporting should be separate from the test process, included in
the framework, and enabled by default for humans.**

The [raw test output](https://www.node-tap.org/tap-format/) should be
machine-parseable and human-intelligible, and a separate process should
consume test output and turn it into a [pretty summarized
report](https://www.node-tap.org/reporting/). This means that test data
can be stored and parsed later, dug into for additional details, and so
on. Also: nyan cat.

6. **Writing tests should be easy, maybe even fun.**

The lower the barrier to entry for writing new tests, the more tests get
written. That means that there should be a relatively small vocabulary
of actions that I need to remember as a test author. There is no
benefit to having a distinction between a "suite" and a "subtest".
Fancy DSLs are pretty, but more to remember.

That being said, if you return a Promise, or use a DSL that throws a
decorated error, then the test framework should Just Work in a way that
helps a human being understand the situation.

7. **Tests should output enough data to diagnose a failure, and no more or
less.**

Stack traces pointing at JS internals or the guts of the test framework
itself are not helpful. A test framework is a serious UX challenge, and
should be treated with care.

8. **Test coverage should be included.**

Running tests with coverage changes the way that you think about your
programs, and provides much deeper insight. Node-tap bundles
[NYC](https://istanbul.js.org/) for this.

It _does_ necessarily change the nature of the environment a little bit.
But in this case, it's worth it, and NYC has come a long way towards
maintaining this promise.

Coverage _enforcement_ is not on by default, but I strongly encourage
it. You can put `"tap":{"check-coverage":true}` in your package.json,
or pass [`--100`](https://www.node-tap.org/100/) on the command line.
In a future version, it will likely be enabled by default.

9. **Tests should not require more building than your code.**

Babel and Webpack are lovely and fine. But if your code doesn't require
compilation, then I think your tests shouldn't either. Tap is extremely
[promise-aware](https://www.node-tap.org/promises/). JSX, TypeScript,
Flow, and ES-Modules are
[built-in](https://www.node-tap.org/using-with/) when tests are run by
the tap CLI.

10. **Tests should run as fast as possible, given all the prior
considerations.**

As of version 10, tap supports [parallel
tests](https://www.node-tap.org/parallel/). As of version 13, the test
runner defaults to running the same number of parallel tests as there
are CPUs on the system.

This makes tests significantly faster in almost every case, on any machine
with multiple cores.

Software testing should help you build software. It should be a security
blanket and a quality ratchet, giving you the support to undertake massive
refactoring and fix bugs without worrying. It shouldn't be a purification
rite or a hazing ritual.

All this is too much to manage in a single README file, so head over
to [the website](http://www.node-tap.org/) to learn more.
There are many opinions left off of this list! Reasonable people can
disagree. But if you find yourself nodding along, [maybe tap is for
you](https://www.node-tap.org/basics/).
19 changes: 0 additions & 19 deletions appveyor.yml

This file was deleted.

Loading