How to use the stream-json/streamers/StreamArray.streamArray function in stream-json

To help you get started, we’ve selected a few stream-json examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github ekalinin / sitemap.js / examples / express.example.js View on Github external
app.get('/sitemap.xml', function(req, res) {
  res.header('Content-Type', 'application/xml');
  res.header('Content-Encoding', 'gzip');
  // if we have a cached entry send it
  if (sitemap) {
    res.send(sitemap)
    return
  }
  try {
    // this could just as easily be a db response
    const gzippedStream = fs.createReadStream(resolve(__dirname, '..', 'tests', 'mocks', 'perf-data.json'))
    .pipe(parser())
    .pipe(streamArray()) // replace with streamValues for JSONStream
    .pipe(map.obj(chunk => chunk.value))
    .pipe(new SitemapStream({ hostname: 'https://example.com/' }))
    .pipe(createGzip())

    // cache the response
    streamToPromise(gzippedStream).then(sm => sitemap = sm)
    // stream the response
    gzippedStream.pipe(res).on('error', (e) => {throw e})
  } catch (e) {
    console.error(e)
    res.status(500).end()
  }
});
github ravendb / ravendb-nodejs-client / src / Documents / Commands / FacetQueryCommand.ts View on Github external
public static async parseQueryResultResponseAsync(
        bodyStream: stream.Stream,
        conventions: DocumentConventions,
        fromCache: boolean,
        bodyCallback?: (body: string) => void): Promise {

        const resultsPromise = RavenCommandResponsePipeline.create()
            .collectBody(bodyCallback)
            .parseJsonAsync([
                pick({ filter: "Results" }),
                streamArray()
            ])
            .streamKeyCaseTransform("camel", "DOCUMENT_LOAD") // we don't care about the case in facets
            .collectResult((result, next) => [...result, next["value"]], [])
            .process(bodyStream);

        const includesPromise = parseDocumentIncludes(bodyStream, conventions);
        const restPromise = parseRestOfOutput(bodyStream, /^Results|Includes$/);

        const [results, includes, rest] = await Promise.all([resultsPromise, includesPromise, restPromise]);
        const rawResult = Object.assign({} as any, rest, { results, includes }) as QueryResult;
        const queryResult = conventions.objectMapper.fromObjectLiteral(rawResult, {
            typeName: QueryResult.name,
            nestedTypes: {
                indexTimestamp: "date",
                lastQueryTime: "date"
            }
github ravendb / ravendb-nodejs-client / src / Mapping / Json / Streams / Pipelines.ts View on Github external
export function getDocumentResultsPipeline(
    conventions: DocumentConventions): RavenCommandResponsePipeline {

    return RavenCommandResponsePipeline.create()
        .parseJsonAsync([
            pick({ filter: "Results" }),
            streamArray(),
        ])
        .streamKeyCaseTransform(conventions.entityFieldNameConvention, "DOCUMENT_LOAD");
}
github ravendb / ravendb-nodejs-client / src / Documents / Commands / MultiGet / MultiGetCommand.ts View on Github external
public async setResponseAsync(bodyStream: stream.Stream, fromCache: boolean): Promise {
        if (!bodyStream) {
            this._throwInvalidResponse();
        }

        const responsesPromise = this._pipeline()
            .parseJsonAsync([
                ignore({ filter: /^Results\.\d+\.Result/ }),
                pick({ filter: "Results" }),
                streamArray()
            ])
            .streamKeyCaseTransform({
                defaultTransform: "camel",
                ignorePaths: [/\./],
            })
            .collectResult({
                initResult: [] as GetResponse[],
                reduceResults: (result: GetResponse[], next) => {
                    return [...result, next["value"]];
                }
            })
            .process(bodyStream);

        const responsesResultsPromise = this._pipeline()
            .parseJsonAsync([
                pick({ filter: "Results" }),

stream-json

stream-json is the micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API. I

BSD-3-Clause
Latest version published 9 days ago

Package Health Score

80 / 100
Full package analysis