How to use the scramjet.StringStream.SPLIT_LINE function in scramjet

To help you get started, we’ve selected a few scramjet examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github openaq / openaq-fetch / adapters / arpalazio.js View on Github external
export const getStream = function (cityName, url, averagingPeriod, source, orgUrl) {
  const { metadata } = source;
  const match = url.match(/[\w]{2}_([\w.]{2,})_([\d]{4})(?:_gg)?.txt/);
  const parameter = match[1].toLowerCase().replace('.', '');
  const year = match[2];
  const unit = getUnit(parameter);
  const dayPosition = averagingPeriod.value === 1 ? 0 : 1;

  const fewDaysAgo = +Number(moment.tz(timezone).subtract(4, 'days').format('DDD'));
  log.verbose(`Fetching data from ${url}`);

  const stations = {};
  return StringStream.from(
    request(url)
  )
    .lines(StringStream.SPLIT_LINE)
    .map(x => x.replace(/\s+/g, ' ').replace(/^\s+|\s+$/g, ''))
    .parse(
      row =>
        row
          .trim()
          .split(/\s+/g)
    )
    .shift(1, ([header]) => {
      header
        .slice(2)
        .forEach((x, i) => {
          if (+x) {
            stations[i] = Object.assign(metadata[x]);
          }
        });
    })

scramjet

Lightweight and real-time data functional stream programming framework like event-stream, written in ES6 using async await with multi-threading and typescript support

MIT
Latest version published 8 months ago

Package Health Score

62 / 100
Full package analysis