Why did is-promise happen and what can we learn from it

On the 25th of April 2020, version 2.2.0 of is-promise library on npm was released by JavaScript developer and maintainer Forbes Lindesay. Reportedly, this release caused failures in popular developer build tools used for scaffolding new projects, such as Facebook’s create-react-app, Google’s firebase-tools, angular-cli, and others. Forbes promptly addressed the problems associated with the 2.2.0 version and ultimately released 2.2.2 some 3 hours later. 

Left-pad all over again? Not quite!

Contrary to what seems to be a popular opinion amongst those sharing this story, this isn’t a software supply-chain problem and is not similar to the case we’ve seen in 2016 with the left-pad incident.

Moreover, the issue is rooted in an innocent error that could’ve happened to anyone and Forbes proved to be a responsible maintainer with a quick fix just shy of hours away from the incident. He also wrote a post-mortem of the incident, which I highly recommend you read.

In this article I’d like to lay out what led to the incident with is-promise and how we can learn from it as maintainers and users.

Who is affected from the is-promise incident?

The is-promise npm package is used by some 500 direct dependencies and gets 12,000,000 downloads a week so, it is fair to say, it is a popular package. However it didn’t specifically break thousands of builds for teams—it mostly impacted users who created new projects using developer tooling.

500 direct dependencies may sound as not so terribly popular but if you know how convoluted the npm ecosystem is with nested dependencies, then GitHub’s stat of 3,500,000 million projects dependending on is-promise, should give you a better estimation of its popularity.

Moreover, this would have only caused an issue for those using Node.js 12.16 and higher— an LTS release—but most of the reported issues I’ve seen on GitHub (e.g: [1], [2]) actually cite users with Node.js 13 or 14 which are not stable versions of Node.js and aren’t recommended unless you’re looking for the bleeding edge.

So what actually happened?

The maintainer sought out to add ES Modules (ESM) support for the library and make it simultaneously use both Node.js’s long-time CommonJS (CJS) module system, as well as the relatively new standardized JavaScript module system.

This happened with the first version in 5 years time—2.1.0—which shows the diff below. (Find the original is-promise comparison on GitHub) In short, the updates included:

• an explicit default export

• making use of the relatively new way to specify the dual-module system in package.json with the exports key, as well as defining the ESM type key

• update on the TypeScript definitions

"isWhat has actually gone wrong here, though? The exports field was not properly defined which led to the following exception thrown and which drove users to report issues on GitHub:

Error [ERR_INVALID_PACKAGE_TARGET]: Invalid "exports" main target "index.js" defined in the package config 

What are the takeaways of this incident?

I believe there are lessons to learn both as maintainers and as users that can help us prepare better for future incidents like this one. In fact, Forbes has already addressed these by adding some controls as a maintainer to catch any such issues from happening in the future.

Is this a software supply-chain issue?

This honest mistake could have happened to anyone and isn’t rooted in the fact that is-promise is a one-liner library, nor is it rooted in a software supply-chain issue by itself.

The left-pad incident revealed weaknesses in how the npm registry was handling package ownership, life-cycle, and so on, and the result of which was policies and measures to prevent such a case in the future. What did the npm registry do to fix the issue with is-promise? Nothing—because it didn’t need to. As I have already stressed out, this isn’t a software supply-chain issue.

Semantic versioning matters

Adding or switching support from CommonJS to ESM, through the use of the type and exports key in package.json, is actually documented to require a breaking change. However, is-promise@2.2.0 was published as a minor change which was resolved as the latest version for many packages that depend on it.

Therefore, our second take-away is to properly give thought to semantic versioning—its impact in our ecosystem is significant.

Package end-to-end testing

One of the ways in which this issue could’ve been caught before releasing it, is by adding end-to-end package testing with the aim of testing the package as a consumer and having sanity tests that ensure it works as expected.

Forbes added these end-to-end tests in the latest version. I have taken a slightly different but similar in concept approach with end-to-end tests for a package called Pie My Vulns. In my CircleCI tests, I spin up Verdaccio as a local package registry, publish the package, and ensure that an npm install, as well as some sanity APIs or CLI commands, work as expected.

Use Node.js LTS

You should always be using Node.js LTS which, at the time of writing this article, resolves to version 12. Using bleeding edge versions of Node.js such as 13 and 14 would have, undoubtedly, positioned you for failure with the is-promise change.

Please keep in mind, however, that in this case, this is not entirely true, since the issue did impact anyone on Node.js 12.16 and above, due to the fact that it shipped with the esm experimental support unflagged. If you were still on 12.15 or earlier, this issue would not have impacted you.

This provides us with another good takeaway—make sure all of your tests pass for Node.js LTS versions, even though adding support for later versions such as Node.js 14 would have been helpful here to catch the issue.

Are you upgrading too early ?

If you had received an auto-upgrade pull request to automatically upgrade to the new is-promise@2.2.0 version and you were on the same impact profile as laid out above, you were also vulnerable to the issue.

Moving forward, as good practice, hold your horses on swift upgrades in order to avoid such issues, as well as malicious packages! That’s why Snyk auto upgrades don’t rush to upgrade your versions and have a 21-days delay until new versions are proposed for your projects.

Do you know how npx works ?

I noticed that many of the reported issues were referring to the use of npx and I wanted to clear some of the confusion—lockfiles, specifically package-lock.json or yarn.lock, won’t be of any help here since these aren’t generally published with the package. Even if they were, they aren’t consumed on install, which is what npx does as part of its execution.

You may choose, however, to use npm’s shrinkwrap lockfile (npm-shrinkwrap.json) as a way to pin your dependencies the way I did with is-website-vulnerable, to ensure that I am always shipping the same tree—the one I always test, regardless of new versions being released in the registry.

If you’re keen to learn more on how lockfiles work, I also wrote about making sense of package lockfiles in the npm ecosystem.

Summary

In closing, these issues happen, and we should be grateful for the community stepping up and the work that has been put by Forbes Lindesay to address the issues in a quick turnaround. There are also many takeaways for us, developers and maintainers, to learn, as we’re all citizens of the open source country.

Want to learn more? Here’s another invite to read Forbes’s post-mortem and the lessons he learned from this incident.

The fixed version for is-promise is 2.2.2 for the 2.x branch, and there’s also a new major version — 4.0.0 — if you are seeking an upgrade for the ES Modules or TypeScript support.