How to use the d2l.data.base.Vocab function in d2l

To help you get started, we’ve selected a few d2l examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github dsgiitr / d2l-pytorch / d2l / data / base.py View on Github external
def load_data_time_machine(num_examples=10000):
    """Load the time machine data set (available in the English book)."""
    with open('../data/timemachine.txt') as f:
        raw_text = f.read()
    lines = raw_text.split('\n')
    text = ' '.join(' '.join(lines).lower().split())[:num_examples]
    vocab = Vocab(text)
    corpus_indices = [vocab[char] for char in text]
    return corpus_indices, vocab