How to use the string-punctuation-tokenizer.occurrencesInString function in string-punctuation-tokenizer

To help you get started, we’ve selected a few string-punctuation-tokenizer examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github unfoldingWord / translationCore / src / js / helpers / VerseObjectHelpers.js View on Github external
stringHelpers.tokenizeWithPunctuation(_verseObject.text).map( text => {
        let verseObject;
        if (stringHelpers.word.test(text)) { // if the text has word characters, its a word object
          const wordIndex = verseObjects.length - nonWordVerseObjectCount;
          let occurrence = stringHelpers.occurrenceInString(_verseObjectsWithTextString, wordIndex, text);
          const occurrences = stringHelpers.occurrencesInString(_verseObjectsWithTextString, text);
          if (occurrence > occurrences) occurrence = occurrences;
          verseObject = {
            tag: "w",
            type: "word",
            text,
            occurrence,
            occurrences
          };
        } else { // the text does not have word characters
          nonWordVerseObjectCount ++;
          verseObject = {
            type: "text",
            text: text
          };
        }
        verseObjects.push(verseObject);

string-punctuation-tokenizer

Small library that provides functions to tokenize a string into an array of words with or without punctuation

MIT
Latest version published 3 years ago

Package Health Score

46 / 100
Full package analysis