View icrawler on Snyk Open Source Advisor

Package Health Score

72 / 100

security
Security review needed
popularity
Recognized
maintenance
Healthy
community
Sustainable

Explore Similar Packages

Popularity

Recognized

GitHub Stars: 855
Forks: 174
Contributors: 20

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package icrawler, we found that it has been starred 855 times.

Security

Security review needed

Security and license risk for latest version

Release Date: May 31, 2024
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
1
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

MIT

Security Policy

We found a way for you to contribute to the project! Looks like icrawler is missing a security policy.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Healthy

Commit Frequency

Open Issues: 32
Open PR: 2
Last Release: 6 months ago
Last Commit: 1 month ago

Further analysis of the maintenance status of icrawler based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that icrawler demonstrates a positive version release cadence with at least one new version released in the past 12 months.

In the past month we didn't find any pull request activity or change in issues status has been detected for the GitHub repository.

Community

Sustainable

Readme: No
Contributing.md: No
Code of Conduct: No
Contributors: 20
Funding: No

With more than 10 contributors for the icrawler repository, this is possibly a sign for a growing and inviting community.

We found a way for you to contribute to the project! Looks like icrawler is missing a Code of Conduct.

How about a good first contribution to this project? It seems that icrawler is missing a README file.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: >=3.7

Age: 9 years
Latest Release: 6 months ago
Dependencies: 7 Direct / 12 Total
Versions: 43
Maintainers: 2
Wheels: OS Independent

Not sure how to use icrawler?

To help you get started, we've collected the most common ways that icrawler is being used within popular public projects.

hellock / icrawler / icrawler / builtin / greedy.py View on Github

if len(tmp) > 1 and tmp[-1] not in [
                        'html', 'shtml', 'shtm', 'php', 'jsp', 'asp'
                ]:
                    continue
                # discard urls such as 'javascript:void(0)'
                elif href.find('javascript', 0, 10) == 0:
                    continue
                # discard urls such as 'android-app://xxxxxxxxx'
                elif urlsplit(href).scheme not in ['http', 'https', 'ftp']:
                    continue
                # urls of the same domain
                elif self.is_in_domain(href, domains):
                    yield href


class GreedyImageCrawler(Crawler):

    def __init__(self,
                 feeder_cls=GreedyFeeder,
                 parser_cls=GreedyParser,
                 downloader_cls=ImageDownloader,
                 *args,
                 **kwargs):
        super(GreedyImageCrawler, self).__init__(
            feeder_cls, parser_cls, downloader_cls, *args, **kwargs)

    def crawl(self,
              domains,
              max_num=0,
              min_size=None,
              max_size=None,
              file_idx_offset=0):

View more ways to use icrawler

icrawler dependencies

beautifulsoup4 bs4 lxml pillow pyyaml requests six

FAQs

What is icrawler?

A multi-thread crawler framework with many builtin image crawlers provided. Visit Snyk Advisor to see a full health score report for icrawler, including popularity, security, maintenance & community analysis.

Is icrawler popular?

The python package icrawler receives a total of 2,156 weekly downloads. As such, icrawler popularity was classified as a recognized. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is icrawler well maintained?

We found that icrawler demonstrated a healthy version release cadence and project activity. It has a community of 20 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is icrawler safe to use?

While scanning the latest version of icrawler, we found that a security review is needed. A total of 0 vulnerabilities or license issues were detected. See the full security scan results.

Last updated on 22 November-2024, at 02:41 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP

icrawler

A multi-thread crawler framework with many builtin image crawlers provided. For more information about how to use this package see README

Ensure you're using the healthiest python packages

Package Health Score

Explore Similar Packages

Popularity

Direct Usage Popularity

Security

Security and license risk for latest version

Maintenance

Commit Frequency

Community

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Package

Not sure how to use icrawler?

icrawler dependencies

FAQs

What is icrawler?

Is icrawler popular?

Is icrawler well maintained?

Is icrawler safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Fix identified vulnerabilities

Monitor for new issues

Package Health Score

Explore Similar Packages

Direct Usage Popularity

Security and license risk for latest version

Commit Frequency

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Not sure how to use icrawler?

icrawler dependencies

FAQs

What is icrawler?

Is icrawler popular?

Is icrawler well maintained?

Is icrawler safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Source Code

Open Source

Container

Config

Fix identified vulnerabilities

Monitor for new issues