Package Health Score

78 / 100

security
No known security issues
popularity
Small
maintenance
Healthy
community
Sustainable

Explore Similar Packages

Popularity

Small

GitHub Stars: 2.5K
Forks: 371
Contributors: 30

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package nexaai, we found that it has been starred 2,496 times.

Security

No known security issues

Security and license risk for latest version

Release Date: Nov 6, 2024
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

Apache-2.0

Security Policy

We found a way for you to contribute to the project! Looks like nexaai is missing a security policy.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Healthy

Commit Frequency

Open Issues: 14
Open PR: 1
Last Release: 9 days ago
Last Commit: 23 days ago

Further analysis of the maintenance status of nexaai based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that nexaai demonstrates a positive version release cadence with at least one new version released in the past 3 months.

As a healthy sign for on-going project maintenance, we found that the GitHub repository had at least 1 pull request or issue interacted with by the community.

Community

Sustainable

Readme: Yes
Contributing.md: No
Code of Conduct: No
Contributors: 30
Funding: No

With more than 10 contributors for the nexaai repository, this is possibly a sign for a growing and inviting community.

We found a way for you to contribute to the project! Looks like nexaai is missing a Code of Conduct.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: ==3.10.*, ==3.11.*, ==3.12.*, ==3.13.*, ==3.7.*, ==3.8.*, ==3.9.*

Age: 3 months
Latest Release: 9 days ago
Dependencies: 12 Direct
Versions: 12
Maintainers: 2
Wheels: No

Readme

Nexa SDK

GitHub License

On-device Model Hub / Nexa SDK Documentation

Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), and speech-to-text (ASR), and text-to-speech (TTS) capabilities. Additionally, it offers an OpenAI-compatible API server with JSON schema mode for function calling and streaming support, and a user-friendly Streamlit UI. Users can run Nexa SDK in any device with Python environment, and GPU acceleration is supported, including CUDA, Metal, and ROCm. An executable version is also available.

Latest News 🔥

[2024/11] Support Nexa AI's own vision language model (0.9B parameters): nexa run omnivision and audio language model (2.9B): nexa run omniaudio
[2024/11] Support audio language model: nexa run qwen2audio, we are the first open-source toolkit to support audio language model with GGML tensor library.
[2024/10] Support embedding model: nexa embed
[2024/10] Support pull and run supported Computer Vision models in GGUF format from HuggingFace: nexa run -hf -mt COMPUTER_VISION
[2024/10] Support VLM in local server.
[2024/10] Added option to customize maximum context window for NLP and VLM models.
[2024/10] Support running model from user's local path
[2024/10] Added LoRA support for NLP models.
[2024/10] Added support for whisper-large-v3-turbo: nexa run faster-whisper-large-turbo
[2024/10] Added support for AMD-Llama-135m: nexa run AMD-Llama-135m:fp16
[2024/09] Nexa now has executables for easy installation: Install Nexa SDK ✨
[2024/09] Added support for Llama 3.2 models: nexa run llama3.2
[2024/09] Added support for Qwen2.5, Qwen2.5-coder and Qwen2.5-Math models: nexa run qwen2.5
[2024/09] Support pull and run NLP models in GGUF format from HuggingFace: nexa run -hf -mt NLP
[2024/09] Added support for ROCm
[2024/09] Added support for Phi-3.5 models: nexa run phi3.5
[2024/09] Added support for OpenELM models: nexa run openelm
[2024/09] Introduced logits API support for more advanced model interactions
[2024/09] Added support for Flux models: nexa run flux
[2024/09] Added support for Stable Diffusion 3 model: nexa run sd3
[2024/09] Added support for Stable Diffusion 2.1 model: nexa run sd2-1

Welcome to submit your requests through issues, we ship weekly.

Installation - Executable

macOS

Download

Linux

curl -fsSL https://public-storage.nexa4ai.com/install.sh | sh

FAQ: cannot use executable with nexaai python package already installed

Try using nexa-exe instead:

nexa-exe

Windows

Coming soon. Install with Python package below 👇

Installation - Python Package

We have released pre-built wheels for various Python versions, platforms, and backends for convenient installation on our index page.

> [!NOTE] > > 1. If you want to use ONNX model, just replace pip install nexaai with pip install "nexaai[onnx]" in provided commands. > 2. If you want to convert and quantize huggingface models to GGUF models, just replace pip install nexaai with pip install "nexaai[nexa-gguf]". > 3. For Chinese developers, we recommend you to use Tsinghua Open Source Mirror as extra index url, just replace --extra-index-url https://pypi.org/simple with --extra-index-url https://pypi.tuna.tsinghua.edu.cn/simple in provided commands.

CPU

pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra-index-url https://pypi.org/simple --no-cache-dir

GPU (Metal)

For the GPU version supporting Metal (macOS):

CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

FAQ: cannot use Metal/GPU on M1

Try the following command:

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh
conda create -n nexasdk python=3.10
conda activate nexasdk
CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

GPU (CUDA)

To install with CUDA support, make sure you have CUDA Toolkit 12.0 or later installed.

For Linux:

CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows PowerShell:

$env:CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON"; pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows Command Prompt:

set CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" &amp; pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows Git Bash:

CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple --no-cache-dir

FAQ: Building Issues for llava

If you encounter the following issue while building:

try the following command:

CMAKE_ARGS="-DCMAKE_CXX_FLAGS=-fopenmp" pip install nexaai

GPU (ROCm)

To install with ROCm support, make sure you have ROCm 6.2.1 or later installed.

For Linux:

CMAKE_ARGS="-DGGML_HIPBLAS=on" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/rocm621 --extra-index-url https://pypi.org/simple --no-cache-dir

GPU (Vulkan)

To install with Vulkan support, make sure you have Vulkan SDK 1.3.261.1 or later installed.

For Windows PowerShell:

$env:CMAKE_ARGS="-DGGML_VULKAN=on"; pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/vulkan --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows Command Prompt:

set CMAKE_ARGS="-DGGML_VULKAN=on" &amp; pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/vulkan --extra-index-url https://pypi.org/simple --no-cache-dir

For Windows Git Bash:

CMAKE_ARGS="-DGGML_VULKAN=on" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/vulkan --extra-index-url https://pypi.org/simple --no-cache-dir

Local Build

How to clone this repo

git clone --recursive https://github.com/NexaAI/nexa-sdk

If you forget to use --recursive, you can use below command to add submodule

git submodule update --init --recursive

Then you can build and install the package

pip install -e .

Features

Model Support:
- ONNX & GGML models
- Conversion Engine
- Inference Engine:
  - Text Generation
  - Image Generation
  - Vision-Language Models (VLM)
  - Speech-to-Text (ASR)

Detailed API documentation is available here.

Server:
- OpenAI-compatible API
- JSON schema mode for function calling
- Streaming support
Streamlit UI for interactive model deployment and testing

Below is our differentiation from other similar tools:

Feature	Nexa SDK	ollama	Optimum	LM Studio
GGML Support	✅	✅	❌	✅
ONNX Support	✅	❌	✅	❌
Text Generation	✅	✅	✅	✅
Image Generation	✅	❌	❌	❌
Vision-Language Models	✅	✅	✅	✅
Text-to-Speech	✅	❌	✅	❌
Server Capability	✅	✅	✅	✅
User Interface	✅	❌	❌	✅

Supported Models & Model Hub

Our on-device model hub offers all types of quantized models (text, image, audio, multimodal) with filters for RAM, file size, Tasks, etc. to help you easily explore models with UI. Explore on-device models at On-device Model Hub

Supported models (full list at Model Hub):

Model	Type	Format	Command
omniaudio	AudioLM	GGUF	`nexa run omniaudio`
qwen2audio	AudioLM	GGUF	`nexa run qwen2audio`
octopus-v2	NLP	GGUF	`nexa run octopus-v2`
octo-net	NLP	GGUF	`nexa run octo-net`
omnivision	Multimodal	GGUF	`nexa run omnivision`
nanollava	Multimodal	GGUF	`nexa run nanollava`
llava-phi3	Multimodal	GGUF	`nexa run llava-phi3`
llava-llama3	Multimodal	GGUF	`nexa run llava-llama3`
llava1.6-mistral	Multimodal	GGUF	`nexa run llava1.6-mistral`
llava1.6-vicuna	Multimodal	GGUF	`nexa run llava1.6-vicuna`
gpt2	NLP	GGUF	`nexa run gpt2`
tinyllama	NLP	GGUF	`nexa run tinyllama`
llama2	NLP	GGUF/ONNX	`nexa run llama2`
llama2-uncensored	NLP	GGUF	`nexa run llama2-uncensored`
llama2-function-calling	NLP	GGUF	`nexa run llama2-function-calling`
llama3	NLP	GGUF/ONNX	`nexa run llama3`
llama3.1	NLP	GGUF/ONNX	`nexa run llama3.1`
llama3.2	NLP	GGUF	`nexa run llama3.2`
llama3-uncensored	NLP	GGUF	`nexa run llama3-uncensored`
gemma	NLP	GGUF/ONNX	`nexa run gemma`
gemma2	NLP	GGUF	`nexa run gemma2`
qwen1.5	NLP	GGUF	`nexa run qwen1.5`
qwen2	NLP	GGUF/ONNX	`nexa run qwen2`
qwen2.5	NLP	GGUF	`nexa run qwen2.5`
mathqwen	NLP	GGUF	`nexa run mathqwen`
codeqwen	NLP	GGUF	`nexa run codeqwen`
mistral	NLP	GGUF/ONNX	`nexa run mistral`
dolphin-mistral	NLP	GGUF	`nexa run dolphin-mistral`
codegemma	NLP	GGUF	`nexa run codegemma`
codellama	NLP	GGUF	`nexa run codellama`
deepseek-coder	NLP	GGUF	`nexa run deepseek-coder`
phi2	NLP	GGUF	`nexa run phi2`
phi3	NLP	GGUF/ONNX	`nexa run phi3`
phi3.5	NLP	GGUF	`nexa run phi3.5`
openelm	NLP	GGUF	`nexa run openelm`
AMD-Llama-135m	NLP	GGUF	`nexa run AMD-Llama-135m:fp16`
stable-diffusion-v1-4	Computer Vision	GGUF	`nexa run sd1-4`
stable-diffusion-v1-5	Computer Vision	GGUF/ONNX	`nexa run sd1-5`
stable-diffusion-v2-1	Computer Vision	GGUF	`nexa run sd2-1`
stable-diffusion-3-medium	Computer Vision	GGUF	`nexa run sd3`
FLUX.1-schnell	Computer Vision	GGUF	`nexa run flux`
lcm-dreamshaper	Computer Vision	GGUF/ONNX	`nexa run lcm-dreamshaper`
hassaku-lcm	Computer Vision	GGUF	`nexa run hassaku-lcm`
anything-lcm	Computer Vision	GGUF	`nexa run anything-lcm`
faster-whisper-tiny	Audio	BIN	`nexa run faster-whisper-tiny`
faster-whisper-small	Audio	BIN	`nexa run faster-whisper-small`
faster-whisper-medium	Audio	BIN	`nexa run faster-whisper-medium`
faster-whisper-base	Audio	BIN	`nexa run faster-whisper-base`
faster-whisper-large	Audio	BIN	`nexa run faster-whisper-large`
whisper-large-v3-turbo	Audio	BIN	`nexa run faster-whisper-large-turbo`
whisper-tiny.en	Audio	ONNX	`nexa run whisper-tiny.en`
whisper-tiny	Audio	ONNX	`nexa run whisper-tiny`
whisper-small.en	Audio	ONNX	`nexa run whisper-small.en`
whisper-small	Audio	ONNX	`nexa run whisper-small`
whisper-base.en	Audio	ONNX	`nexa run whisper-base.en`
whisper-base	Audio	ONNX	`nexa run whisper-base`
mxbai-embed-large-v1	Embedding	GGUF	`nexa embed mxbai`
nomic-embed-text-v1.5	Embedding	GGUF	`nexa embed nomic`
all-MiniLM-L6-v2	Embedding	GGUF	`nexa embed all-MiniLM-L6-v2:fp16`
all-MiniLM-L12-v2	Embedding	GGUF	`nexa embed all-MiniLM-L12-v2:fp16`

CLI Reference

Here's a brief overview of the main CLI commands:

nexa run: Run inference for various tasks using GGUF models.
nexa onnx: Run inference for various tasks using ONNX models.
nexa convert: Convert and quantize huggingface models to GGUF models.
nexa server: Run the Nexa AI Text Generation Service.
nexa eval: Run the Nexa AI Evaluation Tasks.
nexa pull: Pull a model from official or hub.
nexa remove: Remove a model from local machine.
nexa clean: Clean up all model files.
nexa list: List all models in the local machine.
nexa login: Login to Nexa API.
nexa whoami: Show current user information.
nexa logout: Logout from Nexa API.

For detailed information on CLI commands and usage, please refer to the CLI Reference document.

Start Local Server

To start a local server using models on your local computer, you can use the nexa server command. For detailed information on server setup, API endpoints, and usage examples, please refer to the Server Reference document.

Swift Package

Swift SDK: Provides a Swifty API, allowing Swift developers to easily integrate and use llama.cpp models in their projects.

Acknowledgements

We would like to thank the following projects:

nexaai dependencies

cmake fastapi faster-whisper huggingface-hub pillow prompt-toolkit pydantic python-multipart streamlit-audiorec tabulate tqdm uvicorn

FAQs

What is nexaai?

Nexa AI SDK. Visit Snyk Advisor to see a full health score report for nexaai, including popularity, security, maintenance & community analysis.

Is nexaai popular?

The python package nexaai receives a total of 18 weekly downloads. As such, nexaai popularity was classified as small. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is nexaai well maintained?

We found that nexaai demonstrated a healthy version release cadence and project activity. It has a community of 30 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is nexaai safe to use?

The python package nexaai was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 12 November-2024, at 13:15 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP