View all mlprimitives analysis

How to use the mlprimitives.custom.text.TextCleaner function in mlprimitives

To help you get started, we’ve selected a few mlprimitives examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

HDI-Project / MLPrimitives / tests / custom / test_text.py View on Github

def test__remove_stopwords_not_empty(self):
        text_cleaner = TextCleaner()
        text_cleaner.language_code = 'en'

        text_cleaner.STOPWORDS['en'] = ['is', 'a', 'in']

        returned = text_cleaner._remove_stopwords('This is a text written in English')

        assert returned == 'This text written English'

HDI-Project / MLPrimitives / tests / custom / test_text.py View on Github

def test__remove_stopwords_empty(self):
        text_cleaner = TextCleaner()

        returned = text_cleaner._remove_stopwords('')

        assert returned == ''