How to use the ruia.exceptions.IgnoreThisItem function in ruia

To help you get started, we’ve selected a few ruia examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github howie6879 / ruia / tests / test_item.py View on Github external
async def clean_title(self, title):
        raise IgnoreThisItem
github howie6879 / ruia / ruia / item.py View on Github external
item_ins = cls()
        fields_dict = getattr(item_ins, "__fields", {})
        for field_name, field_value in fields_dict.items():
            if not field_name.startswith("target_"):
                clean_method = getattr(item_ins, f"clean_{field_name}", None)
                value = field_value.extract(html_etree)
                if clean_method is not None and callable(clean_method):
                    try:
                        aws_clean_func = clean_method(value)
                        if isawaitable(aws_clean_func):
                            value = await aws_clean_func
                        else:
                            raise InvalidFuncType(
                                f""
                            )
                    except IgnoreThisItem:
                        item_ins.ignore_item = True

                setattr(item_ins, field_name, value)
                item_ins.results[field_name] = value
        return item_ins