prepare v0.8.0

101arrowz · May 16, 2023 · 52da76e · 52da76e
1 parent 038dff5
commit 52da76e
Show file tree

Hide file tree

Showing 26 changed files with 493 additions and 51 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,10 @@
+## 0.8.0
+- BREAKING: synchronous decompression functions now take an options object rather than an output buffer as a second parameter
+  - `inflateSync(compressed, outBuf)` is now `inflateSync(compressed, { out: outBuf })`
+- Support dictionaries in compression and decompression
+- Support multi-member files in GZIP streaming decompression
+- Dramatically improved streaming performance
+- Fixed missing error on certain malformed GZIP files
 ## 0.7.3
 - Fix folder creation for certain operating system
   - Create 0-length "files" for each directory specified with "object" syntax"

diff --git a/README.md b/README.md
@@ -4,20 +4,21 @@ High performance (de)compression in an 8kB package
 ## Why fflate?
 `fflate` (short for fast flate) is the **fastest, smallest, and most versatile** pure JavaScript compression and decompression library in existence, handily beating [`pako`](https://npmjs.com/package/pako), [`tiny-inflate`](https://npmjs.com/package/tiny-inflate), and [`UZIP.js`](https://github.com/photopea/UZIP.js) in performance benchmarks while being multiple times more lightweight. Its compression ratios are often better than even the original Zlib C library. It includes support for DEFLATE, GZIP, and Zlib data. Data compressed by `fflate` can be decompressed by other tools, and vice versa.
 
-In addition to the base decompression and compression APIs, `fflate` supports high-speed ZIP file archiving for an extra 3 kB. In fact, the compressor, in synchronous mode, compresses both more quickly and with a higher compression ratio than most compression software (even Info-ZIP, a C program), and in asynchronous mode it can utilize multiple threads to achieve over 3x the performance of any other utility.
+In addition to the base decompression and compression APIs, `fflate` supports high-speed ZIP file archiving for an extra 3 kB. In fact, the compressor, in synchronous mode, compresses both more quickly and with a higher compression ratio than most compression software (even Info-ZIP, a C program), and in asynchronous mode it can utilize multiple threads to achieve over 3x the performance of virtually any other utility.
 
 |                             | `pako` | `tiny-inflate`         | `UZIP.js`             | `fflate`                       |
 |-----------------------------|--------|------------------------|-----------------------|--------------------------------|
 | Decompression performance   | 1x     | Up to 40% slower       | **Up to 40% faster**  | **Up to 40% faster**           |
-| Compression performance     | 1x     | N/A                    | Up to 5% faster       | **Up to 50% faster**           |
+| Compression performance     | 1x     | N/A                    | Up to 25% faster      | **Up to 50% faster**           |
 | Base bundle size (minified) | 45.6kB | **3kB (inflate only)** | 14.2kB                | 8kB **(3kB for inflate only)** |
+| Decompression support       | ✅     | ✅                      | ✅                    | ✅                             |
 | Compression support         | ✅     | ❌                      | ✅                    | ✅                             |
-| Thread/Worker safe          | ✅     | ✅                      | ❌                    | ✅                             |
 | ZIP support                 | ❌     | ❌                      | ✅                    | ✅                             |
 | Streaming support           | ✅     | ❌                      | ❌                    | ✅                             |
-| GZIP/Zlib support           | ✅     | ❌                      | ❌                    | ✅                             |
+| GZIP support                | ✅     | ❌                      | ❌                    | ✅                             |
 | Supports files up to 4GB    | ✅     | ❌                      | ❌                    | ✅                             |
 | Doesn't hang on error       | ✅     | ❌                      | ❌                    | ✅                             |
+| Dictionary support          | ✅     | ❌                      | ❌                    | ✅                             |
 | Multi-thread/Asynchronous   | ❌     | ❌                      | ❌                    | ✅                             |
 | Streaming ZIP support       | ❌     | ❌                      | ❌                    | ✅                             |
 | Uses ES Modules             | ❌     | ❌                      | ❌                    | ✅                             |
@@ -506,20 +507,22 @@ See the [documentation](https://github.com/101arrowz/fflate/blob/master/docs/REA
 
 ## Bundle size estimates
 
-Since `fflate` uses ES Modules, this table should give you a general idea of `fflate`'s bundle size for the features you need. The maximum bundle size that is possible with `fflate` is about 30kB if you use every single feature, but feature parity with `pako` is only around 10kB (as opposed to 45kB from `pako`). If your bundle size increases dramatically after adding `fflate`, please [create an issue](https://github.com/101arrowz/fflate/issues/new).
-
-| Feature                 | Bundle size (minified)         | Nearest competitor     |
-|-------------------------|--------------------------------|------------------------|
-| Decompression           | 3kB                            | `tiny-inflate`         |
-| Compression             | 5kB                            | `UZIP.js`, 184% larger |
-| Async decompression     | 4kB (1kB + raw decompression)  | N/A                    |
-| Async compression       | 6kB (1kB + raw compression)    | N/A                    |
-| ZIP decompression       | 5kB (2kB + raw decompression)  | `UZIP.js`, 184% larger |
-| ZIP compression         | 7kB (2kB + raw compression)    | `UZIP.js`, 103% larger |
-| GZIP/Zlib decompression | 4kB (1kB + raw decompression)  | `pako`, 1040% larger   |
-| GZIP/Zlib compression   | 5kB (1kB + raw compression)    | `pako`, 812% larger    |
-| Streaming decompression | 4kB (1kB + raw decompression)  | `pako`, 1040% larger   |
-| Streaming compression   | 5kB (1kB + raw compression)    | `pako`, 812% larger    |
+The bundle size measurements for `fflate` on sites like Bundlephobia include every feature of the library and should be seen as an upper bound. As long as you are using tree shaking or dead code elimination, this table should give you a general idea of `fflate`'s bundle size for the features you need.
+
+The maximum bundle size that is possible with `fflate` is about 31kB if you use every single feature, but feature parity with `pako` is only around 10kB (as opposed to 45kB from `pako`). If your bundle size increases dramatically after adding `fflate`, please [create an issue](https://github.com/101arrowz/fflate/issues/new).
+
+| Feature                 | Bundle size (minified)         | Nearest competitor      |
+|-------------------------|--------------------------------|-------------------------|
+| Decompression           | 3kB                            | `tiny-inflate`          |
+| Compression             | 5kB                            | `UZIP.js`, 2.84x larger |
+| Async decompression     | 4kB (1kB + raw decompression)  | N/A                     |
+| Async compression       | 6kB (1kB + raw compression)    | N/A                     |
+| ZIP decompression       | 5kB (2kB + raw decompression)  | `UZIP.js`, 2.84x larger |
+| ZIP compression         | 7kB (2kB + raw compression)    | `UZIP.js`, 2.03x larger |
+| GZIP/Zlib decompression | 4kB (1kB + raw decompression)  | `pako`, 11.4x larger    |
+| GZIP/Zlib compression   | 5kB (1kB + raw compression)    | `pako`, 9.12x larger    |
+| Streaming decompression | 4kB (1kB + raw decompression)  | `pako`, 11.4x larger    |
+| Streaming compression   | 5kB (1kB + raw compression)    | `pako`, 9.12x larger    |
 
 ## What makes `fflate` so fast?
 Many JavaScript compression/decompression libraries exist. However, the most popular one, [`pako`](https://npmjs.com/package/pako), is merely a clone of Zlib rewritten nearly line-for-line in JavaScript. Although it is by no means poorly made, `pako` doesn't recognize the many differences between JavaScript and C, and therefore is suboptimal for performance. Moreover, even when minified, the library is 45 kB; it may not seem like much, but for anyone concerned with optimizing bundle size (especially library authors), it's more weight than necessary.

diff --git a/docs/README.md b/docs/README.md
@@ -43,13 +43,19 @@
 * [AsyncZlibOptions](interfaces/asynczliboptions.md)
 * [DeflateOptions](interfaces/deflateoptions.md)
 * [FlateError](interfaces/flateerror.md)
+* [GunzipOptions](interfaces/gunzipoptions.md)
+* [GunzipStreamOptions](interfaces/gunzipstreamoptions.md)
 * [GzipOptions](interfaces/gzipoptions.md)
+* [InflateOptions](interfaces/inflateoptions.md)
+* [InflateStreamOptions](interfaces/inflatestreamoptions.md)
 * [UnzipDecoder](interfaces/unzipdecoder.md)
 * [UnzipDecoderConstructor](interfaces/unzipdecoderconstructor.md)
 * [UnzipFile](interfaces/unzipfile.md)
 * [UnzipFileInfo](interfaces/unzipfileinfo.md)
 * [UnzipOptions](interfaces/unzipoptions.md)
 * [Unzipped](interfaces/unzipped.md)
+* [UnzlibOptions](interfaces/unzliboptions.md)
+* [UnzlibStreamOptions](interfaces/unzlibstreamoptions.md)
 * [ZipAttributes](interfaces/zipattributes.md)
 * [ZipInputFile](interfaces/zipinputfile.md)
 * [ZipOptions](interfaces/zipoptions.md)
@@ -258,7 +264,7 @@ ___
 
 ### decompressSync
 
-▸ **decompressSync**(`data`: Uint8Array, `out?`: Uint8Array): Uint8Array
+▸ **decompressSync**(`data`: Uint8Array, `opts?`: [AsyncInflateOptions](interfaces/asyncinflateoptions.md)): Uint8Array
 
 Expands compressed GZIP, Zlib, or raw DEFLATE data, automatically detecting the format
 
@@ -267,7 +273,7 @@ Expands compressed GZIP, Zlib, or raw DEFLATE data, automatically detecting the
 Name | Type | Description |
 ------ | ------ | ------ |
 `data` | Uint8Array | The data to decompress |
-`out?` | Uint8Array | Where to write the data. Saves memory if you know the decompressed size and provide an output buffer of that length. |
+`opts?` | [AsyncInflateOptions](interfaces/asyncinflateoptions.md) | The decompression options |
 
 **Returns:** Uint8Array
 
@@ -325,7 +331,7 @@ ___
 
 ▸ **gunzip**(`data`: Uint8Array, `opts`: [AsyncGunzipOptions](interfaces/asyncgunzipoptions.md), `cb`: [FlateCallback](README.md#flatecallback)): [AsyncTerminable](interfaces/asyncterminable.md)
 
-Asynchronously expands single-member GZIP data
+Asynchronously expands GZIP data
 
 #### Parameters:
 
@@ -339,7 +345,7 @@ Name | Type | Description |
 
 ▸ **gunzip**(`data`: Uint8Array, `cb`: [FlateCallback](README.md#flatecallback)): [AsyncTerminable](interfaces/asyncterminable.md)
 
-Asynchronously expands single-member GZIP data
+Asynchronously expands GZIP data
 
 #### Parameters:
 
@@ -354,16 +360,16 @@ ___
 
 ### gunzipSync
 
-▸ **gunzipSync**(`data`: Uint8Array, `out?`: Uint8Array): Uint8Array
+▸ **gunzipSync**(`data`: Uint8Array, `opts?`: [GunzipOptions](interfaces/gunzipoptions.md)): Uint8Array
 
-Expands single-member GZIP data
+Expands GZIP data
 
 #### Parameters:
 
 Name | Type | Description |
 ------ | ------ | ------ |
 `data` | Uint8Array | The data to decompress |
-`out?` | Uint8Array | Where to write the data. GZIP already encodes the output size, so providing this doesn't save memory. |
+`opts?` | [GunzipOptions](interfaces/gunzipoptions.md) | The decompression options |
 
 **Returns:** Uint8Array
 
@@ -477,7 +483,7 @@ ___
 
 ### inflateSync
 
-▸ **inflateSync**(`data`: Uint8Array, `out?`: Uint8Array): Uint8Array
+▸ **inflateSync**(`data`: Uint8Array, `opts?`: [InflateOptions](interfaces/inflateoptions.md)): Uint8Array
 
 Expands DEFLATE data with no wrapper
 
@@ -486,7 +492,7 @@ Expands DEFLATE data with no wrapper
 Name | Type | Description |
 ------ | ------ | ------ |
 `data` | Uint8Array | The data to decompress |
-`out?` | Uint8Array | Where to write the data. Saves memory if you know the decompressed size and provide an output buffer of that length. |
+`opts?` | [InflateOptions](interfaces/inflateoptions.md) | The decompression options |
 
 **Returns:** Uint8Array
 
@@ -577,7 +583,7 @@ ___
 
 ### unzlib
 
-▸ **unzlib**(`data`: Uint8Array, `opts`: [AsyncGunzipOptions](interfaces/asyncgunzipoptions.md), `cb`: [FlateCallback](README.md#flatecallback)): [AsyncTerminable](interfaces/asyncterminable.md)
+▸ **unzlib**(`data`: Uint8Array, `opts`: [AsyncUnzlibOptions](interfaces/asyncunzliboptions.md), `cb`: [FlateCallback](README.md#flatecallback)): [AsyncTerminable](interfaces/asyncterminable.md)
 
 Asynchronously expands Zlib data
 
@@ -586,7 +592,7 @@ Asynchronously expands Zlib data
 Name | Type | Description |
 ------ | ------ | ------ |
 `data` | Uint8Array | The data to decompress |
-`opts` | [AsyncGunzipOptions](interfaces/asyncgunzipoptions.md) | The decompression options |
+`opts` | [AsyncUnzlibOptions](interfaces/asyncunzliboptions.md) | The decompression options |
 `cb` | [FlateCallback](README.md#flatecallback) | The function to be called upon decompression completion |
 
 **Returns:** [AsyncTerminable](interfaces/asyncterminable.md)
@@ -608,7 +614,7 @@ ___
 
 ### unzlibSync
 
-▸ **unzlibSync**(`data`: Uint8Array, `out?`: Uint8Array): Uint8Array
+▸ **unzlibSync**(`data`: Uint8Array, `opts?`: [UnzlibOptions](interfaces/unzliboptions.md)): Uint8Array
 
 Expands Zlib data
 
@@ -617,7 +623,7 @@ Expands Zlib data
 Name | Type | Description |
 ------ | ------ | ------ |
 `data` | Uint8Array | The data to decompress |
-`out?` | Uint8Array | Where to write the data. Saves memory if you know the decompressed size and provide an output buffer of that length. |
+`opts?` | [UnzlibOptions](interfaces/unzliboptions.md) | The decompression options |
 
 **Returns:** Uint8Array
 

diff --git a/docs/classes/asyncgunzip.md b/docs/classes/asyncgunzip.md
@@ -1,6 +1,6 @@
 # Class: AsyncGunzip
 
-Asynchronous streaming GZIP decompression
+Asynchronous streaming single or multi-member GZIP decompression
 
 ## Hierarchy
 

diff --git a/docs/classes/gunzip.md b/docs/classes/gunzip.md
@@ -1,6 +1,6 @@
 # Class: Gunzip
 
-Streaming GZIP decompression
+Streaming single or multi-member GZIP decompression
 
 ## Hierarchy
 
@@ -24,6 +24,19 @@ Streaming GZIP decompression
 
 ### constructor
 
+\+ **new Gunzip**(`opts`: [GunzipStreamOptions](../interfaces/gunzipstreamoptions.md), `cb?`: [FlateStreamHandler](../README.md#flatestreamhandler)): [Gunzip](gunzip.md)
+
+Creates a GUNZIP stream
+
+#### Parameters:
+
+Name | Type | Description |
+------ | ------ | ------ |
+`opts` | [GunzipStreamOptions](../interfaces/gunzipstreamoptions.md) | The decompression options |
+`cb?` | [FlateStreamHandler](../README.md#flatestreamhandler) | The callback to call whenever data is inflated  |
+
+**Returns:** [Gunzip](gunzip.md)
+
 \+ **new Gunzip**(`cb?`: [FlateStreamHandler](../README.md#flatestreamhandler)): [Gunzip](gunzip.md)
 
 Creates a GUNZIP stream

diff --git a/docs/classes/inflate.md b/docs/classes/inflate.md
@@ -24,9 +24,22 @@ Streaming DEFLATE decompression
 
 ### constructor
 
+\+ **new Inflate**(`opts`: [InflateStreamOptions](../interfaces/inflatestreamoptions.md), `cb?`: [FlateStreamHandler](../README.md#flatestreamhandler)): [Inflate](inflate.md)
+
+Creates a DEFLATE decompression stream
+
+#### Parameters:
+
+Name | Type | Description |
+------ | ------ | ------ |
+`opts` | [InflateStreamOptions](../interfaces/inflatestreamoptions.md) | The decompression options |
+`cb?` | [FlateStreamHandler](../README.md#flatestreamhandler) | The callback to call whenever data is inflated  |
+
+**Returns:** [Inflate](inflate.md)
+
 \+ **new Inflate**(`cb?`: [FlateStreamHandler](../README.md#flatestreamhandler)): [Inflate](inflate.md)
 
-Creates an inflation stream
+Creates a DEFLATE decompression stream
 
 #### Parameters:
 

diff --git a/docs/classes/unzlib.md b/docs/classes/unzlib.md
@@ -24,6 +24,19 @@ Streaming Zlib decompression
 
 ### constructor
 
+\+ **new Unzlib**(`opts`: [UnzlibStreamOptions](../interfaces/unzlibstreamoptions.md), `cb?`: [FlateStreamHandler](../README.md#flatestreamhandler)): [Unzlib](unzlib.md)
+
+Creates a Zlib decompression stream
+
+#### Parameters:
+
+Name | Type | Description |
+------ | ------ | ------ |
+`opts` | [UnzlibStreamOptions](../interfaces/unzlibstreamoptions.md) | The decompression options |
+`cb?` | [FlateStreamHandler](../README.md#flatestreamhandler) | The callback to call whenever data is inflated  |
+
+**Returns:** [Unzlib](unzlib.md)
+
 \+ **new Unzlib**(`cb?`: [FlateStreamHandler](../README.md#flatestreamhandler)): [Unzlib](unzlib.md)
 
 Creates a Zlib decompression stream

diff --git a/docs/interfaces/asyncdeflateoptions.md b/docs/interfaces/asyncdeflateoptions.md
@@ -17,6 +17,7 @@ Options for compressing data asynchronously into a DEFLATE format
 ### Properties
 
 * [consume](asyncdeflateoptions.md#consume)
+* [dictionary](asyncdeflateoptions.md#dictionary)
 * [level](asyncdeflateoptions.md#level)
 * [mem](asyncdeflateoptions.md#mem)
 
@@ -33,6 +34,23 @@ unusable but will increase performance and reduce memory usage.
 
 ___
 
+### dictionary
+
+• `Optional` **dictionary**: Uint8Array
+
+*Inherited from [DeflateOptions](deflateoptions.md).[dictionary](deflateoptions.md#dictionary)*
+
+A buffer containing common byte sequences in the input data that can be used to significantly improve compression ratios.
+
+Dictionaries should be 32kB or smaller and include strings or byte sequences likely to appear in the input.
+The decompressor must supply the same dictionary as the compressor to extract the original data.
+
+Dictionaries only improve aggregate compression ratio when reused across multiple small inputs. They should typically not be used otherwise.
+
+Avoid using dictionaries with GZIP and ZIP to maximize software compatibility.
+
+___
+
 ### level
 
 • `Optional` **level**: 0 \| 1 \| 2 \| 3 \| 4 \| 5 \| 6 \| 7 \| 8 \| 9

diff --git a/docs/interfaces/asyncgunzipoptions.md b/docs/interfaces/asyncgunzipoptions.md
@@ -6,13 +6,16 @@ Options for decompressing GZIP data asynchronously
 
 * AsyncOptions
 
+* [InflateStreamOptions](inflatestreamoptions.md)
+
   ↳ **AsyncGunzipOptions**
 
 ## Index
 
 ### Properties
 
 * [consume](asyncgunzipoptions.md#consume)
+* [dictionary](asyncgunzipoptions.md#dictionary)
 
 ## Properties
 
@@ -24,3 +27,15 @@ Options for decompressing GZIP data asynchronously
 
 Whether or not to "consume" the source data. This will make the typed array/buffer you pass in
 unusable but will increase performance and reduce memory usage.
+
+___
+
+### dictionary
+
+• `Optional` **dictionary**: Uint8Array
+
+*Inherited from [InflateStreamOptions](inflatestreamoptions.md).[dictionary](inflatestreamoptions.md#dictionary)*
+
+The dictionary used to compress the original data. If no dictionary was used during compression, this option has no effect.
+
+Supplying the wrong dictionary during decompression usually yields corrupt output or causes an invalid distance error.
diff --git a/docs/interfaces/asyncgzipoptions.md b/docs/interfaces/asyncgzipoptions.md
@@ -15,6 +15,7 @@ Options for compressing data asynchronously into a GZIP format
 ### Properties
 
 * [consume](asyncgzipoptions.md#consume)
+* [dictionary](asyncgzipoptions.md#dictionary)
 * [filename](asyncgzipoptions.md#filename)
 * [level](asyncgzipoptions.md#level)
 * [mem](asyncgzipoptions.md#mem)
@@ -33,6 +34,23 @@ unusable but will increase performance and reduce memory usage.
 
 ___
 
+### dictionary
+
+• `Optional` **dictionary**: Uint8Array
+
+*Inherited from [DeflateOptions](deflateoptions.md).[dictionary](deflateoptions.md#dictionary)*
+
+A buffer containing common byte sequences in the input data that can be used to significantly improve compression ratios.
+
+Dictionaries should be 32kB or smaller and include strings or byte sequences likely to appear in the input.
+The decompressor must supply the same dictionary as the compressor to extract the original data.
+
+Dictionaries only improve aggregate compression ratio when reused across multiple small inputs. They should typically not be used otherwise.
+
+Avoid using dictionaries with GZIP and ZIP to maximize software compatibility.
+
+___
+
 ### filename
 
 • `Optional` **filename**: string