Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: isaacs/node-tar
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: a044a87c6c7fb3ace4ea9bf903c63f0f15965398
Choose a base ref
...
head repository: isaacs/node-tar
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 3302cf7330052982ad7d7e9f85e823fa1bb945a4
Choose a head ref
  • 9 commits
  • 12 files changed
  • 5 contributors

Commits on Dec 10, 2022

  1. chore: bump @npmcli/template-oss from 4.10.0 to 4.11.0

    Bumps [@npmcli/template-oss](https://github.com/npm/template-oss) from 4.10.0 to 4.11.0.
    - [Release notes](https://github.com/npm/template-oss/releases)
    - [Changelog](https://github.com/npm/template-oss/blob/main/CHANGELOG.md)
    - [Commits](npm/template-oss@v4.10.0...v4.11.0)
    
    ---
    updated-dependencies:
    - dependency-name: @npmcli/template-oss
      dependency-type: direct:development
      update-type: version-update:semver-minor
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    dependabot[bot] authored and lukekarrys committed Dec 10, 2022

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    5f31636 View commit details
  2. Copy the full SHA
    82bb328 View commit details

Commits on Apr 12, 2023

  1. Copy the full SHA
    75d3081 View commit details

Commits on May 2, 2023

  1. deps: minipass@5.0.0

    PR-URL: #381
    Credit: @wraithgar
    Close: #381
    Reviewed-by: @isaacs
    wraithgar authored and isaacs committed May 2, 2023
    Copy the full SHA
    4cbdd67 View commit details
  2. 6.1.14

    isaacs committed May 2, 2023
    Copy the full SHA
    4aaffc8 View commit details

Commits on May 12, 2023

  1. move mutateFS reset out of t.teardown

    Not sure why this was failing on my system, but this fixes it.
    isaacs committed May 12, 2023
    Copy the full SHA
    8cd8139 View commit details
  2. Copy the full SHA
    24efc74 View commit details

Commits on May 17, 2023

  1. Normalize unicode internally using NFD

    Previously, the path reservation system, which defends against unicode
    path name collisions (the subject of a handful of past CVE issues), was
    using NFKD normalization internally to determine of two paths would be
    likely to reference the same file on disk.
    
    This has the weird effect of normalizing things like `℀` into simple
    decomposed character strings, for example `a/c`. These can contain
    slashes and double-dot sections, which means that the path reservations
    may end up reserving more (or different) paths than intended.
    
    Thankfully, tar was already *extracting* properly, even if the path
    reservations collided, and these collisions resulted in tar being *more*
    aggressive than it should be in restricting parallel extraction, rather
    than less. That's a good direction to err in, for security, but also,
    made tar less efficient than it could be in some edge cases.
    
    Using NFD normalization, unicode characters are not decomposed in
    compatibility mode, but still result in matching path reservation keys
    as intended.
    
    This does not cause any change in observed behavior, other than allowing
    some files to be extracted in parallel where it is provably safe to do
    so.
    
    Credit: discovered by @Sim4n6. This did not result in a juicy security
    vulnerability, but it sure looked like one at first. They were extremely
    patient, thorough, and persistent in trying to pin this down to a POC
    and CVE. There is very little reward or visibility when a security
    researcher finds a bug that doesn't result in a security disclosure, but
    the attempt often results in improvements to the project.
    isaacs committed May 17, 2023
    Copy the full SHA
    4501bdb View commit details
  2. 6.1.15

    isaacs committed May 17, 2023
    Copy the full SHA
    3302cf7 View commit details
Showing with 85 additions and 25 deletions.
  1. +1 −1 lib/normalize-unicode.js
  2. +2 −2 lib/pack.js
  3. +1 −1 lib/path-reservations.js
  4. +2 −2 lib/read-entry.js
  5. +1 −1 lib/unpack.js
  6. +3 −3 lib/write-entry.js
  7. +5 −5 package.json
  8. +30 −0 tap-snapshots/test/normalize-unicode.js.test.cjs
  9. +27 −0 test/normalize-unicode.js
  10. +4 −3 test/pack.js
  11. +3 −3 test/parse.js
  12. +6 −4 test/unpack.js
2 changes: 1 addition & 1 deletion lib/normalize-unicode.js
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ const normalizeCache = Object.create(null)
const { hasOwnProperty } = Object.prototype
module.exports = s => {
if (!hasOwnProperty.call(normalizeCache, s)) {
normalizeCache[s] = s.normalize('NFKD')
normalizeCache[s] = s.normalize('NFD')
}
return normalizeCache[s]
}
4 changes: 2 additions & 2 deletions lib/pack.js
Original file line number Diff line number Diff line change
@@ -22,7 +22,7 @@ class PackJob {
}
}

const MiniPass = require('minipass')
const { Minipass } = require('minipass')
const zlib = require('minizlib')
const ReadEntry = require('./read-entry.js')
const WriteEntry = require('./write-entry.js')
@@ -56,7 +56,7 @@ const path = require('path')
const warner = require('./warn-mixin.js')
const normPath = require('./normalize-windows-path.js')

const Pack = warner(class Pack extends MiniPass {
const Pack = warner(class Pack extends Minipass {
constructor (opt) {
super(opt)
opt = opt || Object.create(null)
2 changes: 1 addition & 1 deletion lib/path-reservations.js
Original file line number Diff line number Diff line change
@@ -123,7 +123,7 @@ module.exports = () => {
// effectively removing all parallelization on windows.
paths = isWindows ? ['win32 parallelization disabled'] : paths.map(p => {
// don't need normPath, because we skip this entirely for windows
return normalize(stripSlashes(join(p))).toLowerCase()
return stripSlashes(join(normalize(p))).toLowerCase()
})

const dirs = new Set(
4 changes: 2 additions & 2 deletions lib/read-entry.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
'use strict'
const MiniPass = require('minipass')
const { Minipass } = require('minipass')
const normPath = require('./normalize-windows-path.js')

const SLURP = Symbol('slurp')
module.exports = class ReadEntry extends MiniPass {
module.exports = class ReadEntry extends Minipass {
constructor (header, ex, gex) {
super()
// read entries always start life paused. this is to avoid the
2 changes: 1 addition & 1 deletion lib/unpack.js
Original file line number Diff line number Diff line change
@@ -105,7 +105,7 @@ const uint32 = (a, b, c) =>
// Note that on windows, we always drop the entire cache whenever a
// symbolic link is encountered, because 8.3 filenames are impossible
// to reason about, and collisions are hazards rather than just failures.
const cacheKeyNormalize = path => normalize(stripSlash(normPath(path)))
const cacheKeyNormalize = path => stripSlash(normPath(normalize(path)))
.toLowerCase()

const pruneCache = (cache, abs) => {
6 changes: 3 additions & 3 deletions lib/write-entry.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
'use strict'
const MiniPass = require('minipass')
const { Minipass } = require('minipass')
const Pax = require('./pax.js')
const Header = require('./header.js')
const fs = require('fs')
@@ -41,7 +41,7 @@ const stripAbsolutePath = require('./strip-absolute-path.js')

const modeFix = require('./mode-fix.js')

const WriteEntry = warner(class WriteEntry extends MiniPass {
const WriteEntry = warner(class WriteEntry extends Minipass {
constructor (p, opt) {
opt = opt || {}
super(opt)
@@ -417,7 +417,7 @@ class WriteEntrySync extends WriteEntry {
}
}

const WriteEntryTar = warner(class WriteEntryTar extends MiniPass {
const WriteEntryTar = warner(class WriteEntryTar extends Minipass {
constructor (readEntry, opt) {
opt = opt || {}
super(opt)
10 changes: 5 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -2,10 +2,10 @@
"author": "GitHub Inc.",
"name": "tar",
"description": "tar for node",
"version": "6.1.13",
"version": "6.1.15",
"repository": {
"type": "git",
"url": "https://github.com/npm/node-tar.git"
"url": "https://github.com/isaacs/node-tar.git"
},
"scripts": {
"genparse": "node scripts/generate-parse-fixtures.js",
@@ -20,14 +20,14 @@
"dependencies": {
"chownr": "^2.0.0",
"fs-minipass": "^2.0.0",
"minipass": "^4.0.0",
"minipass": "^5.0.0",
"minizlib": "^2.1.1",
"mkdirp": "^1.0.3",
"yallist": "^4.0.0"
},
"devDependencies": {
"@npmcli/eslint-config": "^4.0.0",
"@npmcli/template-oss": "4.10.0",
"@npmcli/template-oss": "4.11.0",
"chmodr": "^1.2.0",
"end-of-stream": "^1.4.3",
"events-to-array": "^2.0.3",
@@ -55,7 +55,7 @@
},
"templateOSS": {
"//@npmcli/template-oss": "This file is partially managed by @npmcli/template-oss. Edits may be overwritten.",
"version": "4.10.0",
"version": "4.11.0",
"content": "scripts/template-oss",
"engines": ">=10",
"distPaths": [
30 changes: 30 additions & 0 deletions tap-snapshots/test/normalize-unicode.js.test.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/* IMPORTANT
* This snapshot file is auto-generated, but designed for humans.
* It should be checked into source control and tracked carefully.
* Re-generate by setting TAP_SNAPSHOT=1 and running tests.
* Make sure to inspect the output below. Do not ignore changes!
*/
'use strict'
exports[`test/normalize-unicode.js TAP normalize with strip slashes "1/4foo.txt" > normalized 1`] = `
1/4foo.txt
`

exports[`test/normalize-unicode.js TAP normalize with strip slashes "\\\\a\\\\b\\\\c\\\\d\\\\" > normalized 1`] = `
/a/b/c/d
`

exports[`test/normalize-unicode.js TAP normalize with strip slashes "¼foo.txt" > normalized 1`] = `
¼foo.txt
`

exports[`test/normalize-unicode.js TAP normalize with strip slashes "﹨aaaa﹨dddd﹨" > normalized 1`] = `
﹨aaaa﹨dddd﹨
`

exports[`test/normalize-unicode.js TAP normalize with strip slashes "\bbb\eee\" > normalized 1`] = `
\bbb\eee\
`

exports[`test/normalize-unicode.js TAP normalize with strip slashes "\\\\\eee\\\\\\" > normalized 1`] = `
\\\\\eee\\\\\\
`
27 changes: 27 additions & 0 deletions test/normalize-unicode.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
process.env.TESTING_TAR_FAKE_PLATFORM = 'win32'
const t = require('tap')
const normalize = require('../lib/normalize-unicode.js')
const stripSlash = require('../lib/strip-trailing-slashes.js')
const normPath = require('../lib/normalize-windows-path.js')

// café
const cafe1 = Buffer.from([0x63, 0x61, 0x66, 0xc3, 0xa9]).toString()
@@ -10,3 +13,27 @@ const cafe2 = Buffer.from([0x63, 0x61, 0x66, 0x65, 0xcc, 0x81]).toString()
t.equal(normalize(cafe1), normalize(cafe2), 'matching unicodes')
t.equal(normalize(cafe1), normalize(cafe2), 'cached')
t.equal(normalize('foo'), 'foo', 'non-unicode string')

t.test('normalize with strip slashes', t => {
const paths = [
'\\a\\b\\c\\d\\',
'﹨aaaa﹨dddd﹨',
'\bbb\eee\',
'\\\\\eee\\\\\\',
'¼foo.txt',
'1/4foo.txt',
]

t.plan(paths.length)

for (const path of paths) {
t.test(JSON.stringify(path), t => {
const a = normalize(stripSlash(normPath(path)))
const b = stripSlash(normPath(normalize(path)))
t.matchSnapshot(a, 'normalized')
t.equal(a, b, 'order should not matter')
t.end()
})
}
t.end()
})
7 changes: 4 additions & 3 deletions test/pack.js
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ const Header = require('../lib/header.js')
const zlib = require('zlib')
const miniz = require('minizlib')
const mutateFS = require('mutate-fs')
const MiniPass = require('minipass')
const { Minipass } = require('minipass')
process.env.USER = 'isaacs'
const EE = require('events').EventEmitter
const rimraf = require('rimraf')
@@ -563,8 +563,8 @@ t.test('readdir fail', t => {

t.test('pipe into a slow reader', t => {
const out = []
const mp = new MiniPass()
const mp2 = new MiniPass()
const mp = new Minipass()
const mp2 = new Minipass()
const p = new Pack({ cwd: files }).add('long-path').end()
p.pause()
p.pipe(mp).pipe(mp2)
@@ -1135,6 +1135,7 @@ t.test('prefix and hard links', t => {
cwd: dir + '/in',
prefix: 'out/x',
noDirRecurse: true,
jobs: 1,
})
const out = []
p.on('data', d => out.push(d))
6 changes: 3 additions & 3 deletions test/parse.js
Original file line number Diff line number Diff line change
@@ -7,12 +7,12 @@ const fs = require('fs')
const path = require('path')
const tardir = path.resolve(__dirname, 'fixtures/tars')
const zlib = require('zlib')
const MiniPass = require('minipass')
const { Minipass } = require('minipass')
const Header = require('../lib/header.js')
const EE = require('events').EventEmitter

t.test('fixture tests', t => {
class ByteStream extends MiniPass {
class ByteStream extends Minipass {
write (chunk) {
for (let i = 0; i < chunk.length - 1; i++) {
super.write(chunk.slice(i, i + 1))
@@ -604,7 +604,7 @@ t.test('end while consuming', t => {
'package/node_modules/b/package.json',
]

const mp = new MiniPass()
const mp = new Minipass()
const p = new Parse({
onentry: entry => {
actual.push(entry.path)
10 changes: 6 additions & 4 deletions test/unpack.js
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ process.umask(0o022)
const Unpack = require('../lib/unpack.js')
const UnpackSync = Unpack.Sync
const t = require('tap')
const MiniPass = require('minipass')
const { Minipass } = require('minipass')

const makeTar = require('./make-tar.js')
const Header = require('../lib/header.js')
@@ -514,7 +514,8 @@ t.test('symlink in dir path', {

t.test('clobber through symlink with busted unlink', t => {
const poop = new Error('poop')
t.teardown(mutateFS.fail('unlink', poop))
// for some reason, resetting fs.unlink in the teardown was breaking
const reset = mutateFS.fail('unlink', poop)
const warnings = []
const u = new Unpack({
cwd: dir,
@@ -523,6 +524,7 @@ t.test('symlink in dir path', {
})
u.on('close', _ => {
t.same(warnings, [['TAR_ENTRY_ERROR', 'poop', poop]])
reset()
t.end()
})
u.end(data)
@@ -2252,7 +2254,7 @@ t.test('transform', t => {
}
}

class Bracer extends MiniPass {
class Bracer extends Minipass {
write (data) {
const d = data.toString().split('').map(c => '[' + c + ']').join('')
return super.write(d)
@@ -2324,7 +2326,7 @@ t.test('transform error', t => {
const poop = new Error('poop')

const txFn = () => {
const tx = new MiniPass()
const tx = new Minipass()
tx.write = () => tx.emit('error', poop)
tx.resume()
return tx