How to use the snorkel.preprocess.preprocessor function in snorkel

To help you get started, we’ve selected a few snorkel examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github snorkel-team / snorkel / test / labeling / lf / test_core.py View on Github external
@preprocessor()
def returns_none(x: DataPoint) -> DataPoint:
    return None
github snorkel-team / snorkel / test / labeling / apply / test_spark.py View on Github external
@preprocessor()
def square(x: Row) -> Row:
    return Row(num=x.num, num_squared=x.num ** 2)
github snorkel-team / snorkel / test / labeling / test_convergence.py View on Github external
@preprocessor()
def copy_features(x: DataPoint) -> DataPoint:
    """Compute x2 + 0.25 for direct comparison to x1."""
    x.x3 = x.x2 + 0.25
    return x
github snorkel-team / snorkel / test / labeling / lf / test_nlp.py View on Github external
@preprocessor()
def combine_text(x: DataPoint) -> DataPoint:
    x.text = f"{x.title} {x.article}"
    return x
github snorkel-team / snorkel / test / labeling / apply / test_spark.py View on Github external
        @preprocessor(memoize=True)
        def square_memoize(x: DataPoint) -> DataPoint:
            return Row(num=x.num, num_squared=x.num ** 2)
github snorkel-team / snorkel-tutorials / drybell / drybell_lfs_spark.py View on Github external
@preprocessor()
def combine_text(x):
    return Row(title=x.title, body=x.body, article=f"{x.title} {x.body}")
github snorkel-team / snorkel-tutorials / drybell / drybell_lfs.py View on Github external
@preprocessor()
def combine_text(x):
    x.article = f"{x.title} {x.body}"
    return x
github snorkel-team / snorkel-tutorials / recsys / recsys_tutorial.py View on Github external
@preprocessor(memoize=True)
def textblob_polarity(x):
    if isinstance(x.review_text, str):
        x.blob = TextBlob(x.review_text)
    else:
        x.blob = None
    return x
github snorkel-team / snorkel-tutorials / spouse / spouse_demo.py View on Github external
@preprocessor()
def get_text_between(cand):
    """
    Returns the text between the two person mentions in the sentence for a candidate
    """
    start = cand.person1_word_idx[1] + 1
    end = cand.person2_word_idx[0]
    cand.text_between = " ".join(cand.tokens[start:end])
    return cand
github snorkel-team / snorkel-tutorials / spam / 01_spam_tutorial.py View on Github external
@preprocessor(memoize=True)
def textblob_sentiment(x):
    scores = TextBlob(x.text)
    x.polarity = scores.sentiment.polarity
    x.subjectivity = scores.sentiment.subjectivity
    return x