View all blingfire analysis

How to use the blingfire.text_to_words function in blingfire

To help you get started, we’ve selected a few blingfire examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

soskek / bookcorpus / tokenize_sentlines.py View on Github

import sys
from blingfire import text_to_words


for l in sys.stdin:
    if l.strip():
        print(text_to_words(l.strip()))
    else:
        print('')