netspresso

Use NetsPresso for a seamless model optimization process. NetsPresso resolves AI-related constraints in business use cases and enables cost-efficiency and enhanced performance by removing the requirement for high-spec servers and network connectivity and preventing high latency and personal data breaches.

Easily compress various models with our resources. Please browse the Docs for details, and join our Discussion Forum for providing feedback or sharing your use cases.

To get started with NetsPresso, you'll need to sign up here.

We offer a comprehensive guide to walk you through the process of optimizing an AI model using NetsPresso. A full tutorial can be found Google Colab.

Installation

Prerequisites

• Python 3.8 | 3.9 | 3.10
• PyTorch 1.13.0 (recommended) (compatible with: 1.11.x - 1.13.x)
• TensorFlow 2.8.0 (recommended) (compatible with: 2.3.x - 2.8.x)

Install with PyPI (stable)

pip install netspresso

To use editable mode or docker, see INSTALLATION.md.

Getting started

Login

Log-in to your netspresso account. Please sign-up here if you need one.

from netspresso import NetsPresso

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

Trainer

Train

To start training a model, first select a task.

Then configure the dataset, model, augmentation, and hyperparameters.

Once setup is finished, enter the GPU number and project name for training.

from netspresso.enums import Task
from netspresso.trainer.optimizers import AdamW
from netspresso.trainer.schedulers import CosineAnnealingWarmRestartsWithCustomWarmUp
from netspresso.trainer.augmentations import Resize


# 1. Declare trainer
trainer = netspresso.trainer(task=Task.OBJECT_DETECTION)  # IMAGE_CLASSIFICATION, OBJECT_DETECTION, SEMANTIC_SEGMENTATION

# 2. Set config for training
# 2-1. Data
trainer.set_dataset_config(
    name="traffic_sign_config_example",
    root_path="/root/traffic-sign",
    train_image="images/train",
    train_label="labels/train",
    valid_image="images/valid",
    valid_label="labels/valid",
    id_mapping=["prohibitory", "danger", "mandatory", "other"],
)

# 2-2. Model
print(trainer.available_models)  # ['EfficientFormer', 'YOLOX-S', 'ResNet', 'MobileNetV3', 'MixNetL', 'MixNetM', 'MixNetS']
trainer.set_model_config(model_name="YOLOX-S", img_size=512)

# 2-3. Augmentation
trainer.set_augmentation_config(
    train_transforms=[Resize()],
    inference_transforms=[Resize()],
)

# 2-4. Training
optimizer = AdamW(lr=6e-3)
scheduler = CosineAnnealingWarmRestartsWithCustomWarmUp(warmup_epochs=10)
trainer.set_training_config(
    epochs=40,
    batch_size=16,
    optimizer=optimizer,
    scheduler=scheduler,
)

# 3. Train
training_result = trainer.train(gpus="0, 1", project_name="PROJECT_TRAIN_SAMPLE")

Retrain

To start retraining a model, use hparams.yaml file which is one of the artifacts generated during the training of the original model.

Then, enter the compressed model path, which is an artifact of the compressor in fx_model_path.

Adjust the training hyperparameters as needed. (See 2-2. for detailed code.)

from netspresso.trainer.optimizers import AdamW

# 1. Declare trainer
trainer = netspresso.trainer(yaml_path="./temp/hparams.yaml")

# 2. Set config for retraining
# 2-1. FX Model
trainer.set_fx_model(fx_model_path="./temp/FX_MODEL_PATH.pt")

# 2-2. Training
optimizer = AdamW(lr=6e-3)
trainer.set_training_config(
    epochs=30,
    batch_size=16,
    optimizer=optimizer,
)

# 3. Train
retraining_result = trainer.train(gpus="0, 1", project_name="PROJECT_RETRAIN_SAMPLE")

Compressor

Compress (Automatic compression)

To start compressing a model, enter the model path to compress and the appropriate compression ratio.

The compressed model will be saved in the specified output directory (output_dir).

# 1. Declare compressor
compressor = netspresso.compressor()

# 2. Run automatic compression
compression_result = compressor.automatic_compression(
    input_shapes=[{"batch": 1, "channel": 3, "dimension": [224, 224]}],
    input_model_path="./examples/sample_models/graphmodule.pt",
    output_dir="./outputs/compressed/pytorch_automatic_compression",
    compression_ratio=0.5,
)

Converter

Convert

To start converting a model, enter the model path to convert and the target framework and device name.

For NVIDIA GPUs and Jetson devices, enter the software version additionally due to the jetpack version.

The converted model will be saved in the specified output directory (output_dir).

from netspresso.enums import DeviceName, Framework, SoftwareVersion

# 1. Declare converter
converter = netspresso.converter()

# 2. Run convert
conversion_result = converter.convert_model(
    input_model_path="./examples/sample_models/test.onnx",
    output_dir="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1",
    target_framework=Framework.TENSORRT,
    target_device_name=DeviceName.JETSON_AGX_ORIN,
    target_software_version=SoftwareVersion.JETPACK_5_0_1,
)

Benchmarker

Benchmark

To start benchmarking a model, enter the model path to benchmark and the target device name.

For NVIDIA GPUs and Jetson devices, device name and software version have to be matched with the target device of the conversion.

TensorRT Model has strong dependency with the device type and its jetpack version.

from netspresso.enums import DeviceName, SoftwareVersion

# 1. Declare benchmarker
benchmarker = netspresso.benchmarker()

# 2. Run benchmark
benchmark_result = benchmarker.benchmark_model(
    input_model_path="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1.trt",
    target_device_name=DeviceName.JETSON_AGX_ORIN,
    target_software_version=SoftwareVersion.JETPACK_5_0_1,
)
print(f"model inference latency: {benchmark_result['result']['latency']} ms")
print(f"model gpu memory footprint: {benchmark_result['result']['memory_footprint_gpu']} MB")
print(f"model cpu memory footprint: {benchmark_result['result']['memory_footprint_cpu']} MB")

conversion_result = converter.convert_model(
    input_model_path=INPUT_MODEL_PATH,
    output_dir=OUTPUT_DIR,
    target_framework=Framework.TENSORRT,
    target_device_name=DeviceName.JETSON_AGX_ORIN,
    target_software_version=SoftwareVersion.JETPACK_5_0_1,
)
benchmark_result = benchmarker.benchmark_model(
    input_model_path=CONVERTED_MODEL_PATH,
    target_device_name=DeviceName.JETSON_AGX_ORIN,
    target_software_version=SoftwareVersion.JETPACK_5_0_1,
)

benchmark_result = benchmarker.benchmark_model(
    input_model_path=CONVERTED_MODEL_PATH,
    target_device_name=DeviceName.RENESAS_RA8D1,
    target_data_type=DataType.INT8,
    target_hardware_type=HardwareType.HELIUM
)

Guide to Credit Consumption by Module

Contact

Join our Discussion Forum for providing feedback or sharing your use cases, and if you want to talk more with Nota, please contact us here.
Or you can also do it via email(netspresso@nota.ai) or phone(+82 2-555-8659)!