sglang

v0.3.6

SGLang is yet another fast serving framework for large language models and vision language models. For more information about how to use this package see README

Latest version published 13 hours ago

License: Apache-2.0

PyPI

GitHub

Package Health Score

84 / 100

security
No known security issues
popularity
Recognized
maintenance
Healthy
community
Active

Explore Similar Packages

Popularity

Recognized

GitHub Stars: 6.1K
Forks: 510
Contributors: 160

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package sglang, we found that it has been starred 6,103 times.

Security

No known security issues

Security and license risk for latest version

Release Date: Nov 22, 2024
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

Apache-2.0

Security Policy

We found a way for you to contribute to the project! Looks like sglang is missing a security policy.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Healthy

Commit Frequency

Open Issues: 66
Open PR: 22
Last Release: 13 hours ago
Last Commit: 4 days ago

Further analysis of the maintenance status of sglang based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that sglang demonstrates a positive version release cadence with at least one new version released in the past 3 months.

As a healthy sign for on-going project maintenance, we found that the GitHub repository had at least 1 pull request or issue interacted with by the community.

Community

Active

Readme: Yes
Contributing.md: No
Code of Conduct: No
Contributors: 160
Funding: No

A good and healthy external contribution signal for sglang project, which invites more than one hundred open source maintainers to collaborate on the repository.

We found a way for you to contribute to the project! Looks like sglang is missing a Code of Conduct.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: >=3.8

Age: 10 months
Latest Release: 13 hours ago
Dependencies: 4 Direct
Versions: 56
Maintainers: 3
Wheels: OS Independent

Readme

PyPI - Downloads

News

[2024/10] 🔥 The First SGLang Online Meetup (slides).
[2024/09] SGLang v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision (blog).
[2024/07] Faster Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) (blog).

[2024/02] SGLang enables 3x faster JSON decoding with compressed finite state machine (blog).
[2024/04] SGLang is used by the official LLaVA-NeXT (video) release (blog).
[2024/01] SGLang provides up to 5x faster inference with RadixAttention (blog).
[2024/01] SGLang powers the serving of the official LLaVA v1.6 release demo (usage).

About

SGLang is a fast serving framework for large language models and vision language models. It makes your interaction with models faster and more controllable by co-designing the backend runtime and frontend language. The core features include:

Fast Backend Runtime: Provides efficient serving with RadixAttention for prefix caching, jump-forward constrained decoding, continuous batching, token attention (paged attention), tensor parallelism, FlashInfer kernels, chunked prefill, and quantization (INT4/FP8/AWQ/GPTQ).
Flexible Frontend Language: Offers an intuitive interface for programming LLM applications, including chained generation calls, advanced prompting, control flow, multi-modal inputs, parallelism, and external interactions.
Extensive Model Support: Supports a wide range of generative models (Llama, Gemma, Mistral, QWen, DeepSeek, LLaVA, etc.), embedding models (e5-mistral, gte, mcdse) and reward models (Skywork), with easy extensibility for integrating new models.
Active Community: SGLang is open-source and backed by an active community with industry adoption.

Getting Started

Install SGLang: See https://sgl-project.github.io/start/install.html

Send requests: See https://sgl-project.github.io/start/send_request.html

Backend: SGLang Runtime (SRT)

See https://sgl-project.github.io/backend/backend.html

Frontend: Structured Generation Language (SGLang)

See https://sgl-project.github.io/frontend/frontend.html

Benchmark And Performance

Learn more in our release blogs: v0.2 blog, v0.3 blog

Roadmap

Development Roadmap (2024 Q4)

Citation And Acknowledgment

Please cite our paper, SGLang: Efficient Execution of Structured Language Model Programs, if you find the project useful. We also learned from the design and reused code from the following projects: Guidance, vLLM, LightLLM, FlashInfer, Outlines, and LMQL.

sglang dependencies

ipython numpy requests tqdm

FAQs

What is sglang?

SGLang is yet another fast serving framework for large language models and vision language models. Visit Snyk Advisor to see a full health score report for sglang, including popularity, security, maintenance & community analysis.

Is sglang popular?

The python package sglang receives a total of 4,150 weekly downloads. As such, sglang popularity was classified as a recognized. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is sglang well maintained?

We found that sglang demonstrated a healthy version release cadence and project activity. It has a community of 160 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is sglang safe to use?

The python package sglang was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 22 November-2024, at 20:20 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP