How to use the danlp.models.embeddings.AVAILABLE_EMBEDDINGS function in danlp

To help you get started, we’ve selected a few danlp examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github alexandrainst / danlp / tests / test_embeddings.py View on Github external
def setUp(self):
        # First we will add smaller test embeddings to the
        MODELS['wiki.da.small.wv'] = {
            'url': 'https://danlp.s3.eu-central-1.amazonaws.com/test-models/wiki.da.small.zip',
            'vocab_size': 5000,
            'dimensions': 300,
            'md5_checksum': 'fcaa981a613b325ae4dc61aba235aa82',
            'size': 5594508,
            'file_extension': '.bin'
        }

        AVAILABLE_EMBEDDINGS.append('wiki.da.small.wv')

        self.embeddings_for_testing = [
            'wiki.da.small.wv',
            'dslreddit.da.wv'
        ]
        # Lets download the models and unzip it
        for emb in self.embeddings_for_testing:
            download_model(emb, process_func=_unzip_process_func)
github alexandrainst / danlp / examples / benchmarks / wordembeddings_benchmarks.py View on Github external
def load_wv_models():
    for da_wv_model in AVAILABLE_EMBEDDINGS:
        yield da_wv_model, load_wv_with_gensim(da_wv_model)