Polygraphy: A Deep Learning Inference Prototyping and Debugging Toolkit

Table of Contents

Introduction
Installation
Command-line Toolkit
Python API
Examples
How-To Guides
Contributing

Introduction

Polygraphy is a toolkit designed to assist in running and debugging deep learning models
in various frameworks. It includes a Python API and
a command-line interface (CLI) built using this API.

Among other things, Polygraphy lets you:

Run inference among multiple backends, like TensorRT and ONNX-Runtime, and compare results
(example: API, CLI)
Convert models to various formats, e.g. TensorRT engines with post-training quantization
(example: API, CLI)
View information about various types of models
(example: CLI)
Modify ONNX models on the command-line:
- Extract subgraphs (example: CLI)
- Simplify and sanitize (example: CLI)
Isolate faulty tactics in TensorRT
(example: CLI)

Installation

IMPORTANT: Polygraphy supports only Python 3.6 and later.
Before following the instructions below, please ensure you are using a supported version of Python.

Installing Prebuilt Wheels

python -m pip install colored polygraphy --extra-index-url https://pypi.ngc.nvidia.com

NOTE: On Linux, the command-line toolkit is usually installed to ${HOME}/.local/bin by default.
Make sure to add this directory to your PATH environment variable.

Building From Source

Using Make Targets (Linux)

make install

Using Powershell Script (Windows)

Make sure you are allowed to execute scripts on your system then run:

.\install.ps1

Building Manually

Install prerequisites:

python -m pip install wheel

Build a wheel:

python setup.py bdist_wheel

Install the wheel manually from outside the repository:

On Linux, run:
```
python -m pip install Polygraphy/dist/polygraphy-*-py2.py3-none-any.whl
```
On Windows, using Powershell, run:
```
$wheel_path = gci -Name Polygraphy\dist
python -m pip install Polygraphy\dist\$wheel_path
```
NOTE: It is strongly recommended to install the colored module for colored output
from Polygraphy, as this can greatly improve readability:
```
python -m pip install colored
```

Installing Dependencies

Polygraphy has no hard-dependencies on other Python packages. However, much of the functionality included
does require other Python packages.

Automatically Installing Dependencies

It's non-trivial to determine all the packages that will be required ahead of time,
since it depends on exactly what functionality is being used.

To make this easier, Polygraphy can optionally automatically install or upgrade dependencies at runtime, as they are needed.
To enable this behavior, set the POLYGRAPHY_AUTOINSTALL_DEPS environment variable to 1 or
polygraphy.config.AUTOINSTALL_DEPS = True using the Python API.

NOTE: By default, dependencies will be installed using the current interpreter, and may overwrite existing
packages. The default installation command, which is python -m pip install, can be overriden by setting
the POLYGRAPHY_INSTALL_CMD environment variable, or setting polygraphy.config.INSTALL_CMD using the Python API.

If you'd like Polygraphy to prompt you before automatically installing or
upgrading pacakges, set the POLYGRAPHY_ASK_BEFORE_INSTALL environment variable to 1
or polygraphy.config.ASK_BEFORE_INSTALL = True using the Python API.

Installing Manually

Each backend directory includes a requirements.txt file that specifies the minimum set of packages
it depends on. This does not necessarily include all packages required for all the functionality provided
by the backend, but does serve as a good starting point.

You can install the requirements for whichever backends you're interested in with:

python -m pip install -r polygraphy/backend/<name>/requirements.txt

If additional packages are required, warnings or errors will be logged.
You can install the additional packages manually with:

python -m pip install <package_name>

Command-line Toolkit

For details on the various tools included in the Polygraphy toolkit,
see the CLI User Guide.

Python API

For more information on the Polygraphy Python API, including a high-level overview and the
Python API reference documentation, see the API directory.

Examples

For examples of both the CLI and Python API, see the examples directory.

How-To Guides

For how-to guides, see the how-to guides directory.

Contributing

For information on how you can contribute to this project, see CONTRIBUTING.md

Package profile

polygraphy

Releases

Release table

PyPI Downloads

Dependencies

Details

Project URLs

Dependencies

Package description