How to use the chardet.detectFileSync function in chardet

To help you get started, we’ve selected a few chardet examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github martpie / museeks / src / ui / utils / utils-m3u.ts View on Github external
export const parse = (filePath: string): string[] => {
  try {
    const baseDir = path.parse(filePath).dir;
    const encoding = chardet.detectFileSync(filePath);
    const content = fs.readFileSync(filePath);

    if (typeof encoding !== 'string') {
      throw (new Error(`could not guess the file encoding (${filePath})`));
    }

    const decodedContent = iconv.decode(content, encoding);

    const files = decodedContent
      .split('\n')
      .reduce((acc, line) => {
        if (line.length === 0) {
          return acc;
        }

        // If absolute path
github datopian / data.js / lib / index.js View on Github external
get encoding() {
    // When data is huge, we want to optimize performace (in tradeoff of less accuracy):
    // So we are using sample of first 100K bytes here:
    if (this.size > 1000000) {
      return chardet.detectFileSync(this.path, {
        sampleSize: 1000000
      });
    }

    return chardet.detectFileSync(this.path);
  }
github datopian / data.js / src / index.js View on Github external
get encoding() {
    // When data is huge, we want to optimize performace (in tradeoff of less accuracy):
    // So we are using sample of first 100K bytes here:
    if (this.size > 1000000) {
      return chardet.detectFileSync(this.path, {sampleSize: 1000000})
    }
    return chardet.detectFileSync(this.path)
  }
}
github datopian / data-cli / dist / utils / data.js View on Github external
get encoding() {
    return chardet.detectFileSync(this.path);
  }
}
github datopian / data-cli / lib / utils / data.js View on Github external
get encoding() {
    return chardet.detectFileSync(this.path)
  }
}
github betagouv / preuve-covoiturage / api / legacy / modules / importer / src / index.ts View on Github external
const detectEncoding = (path) => {
  const enc = chardet.detectFileSync(path);
  switch (enc) {
    case 'ISO-8859-1':
    case 'ISO-8859-2':
      return 'latin1';
    default:
      return 'utf8';
  }
};

chardet

Character encoding detector

MIT
Latest version published 7 months ago

Package Health Score

81 / 100
Full package analysis

Popular chardet functions