Torq Compiler

• User Manual
  • Introduction
    • Technical Background
    • Distribution Contents
      • Release Package
      • Docker Image
    • System Requirements
    • Getting Started
    • Support and Contact
  • Quickstart
    • Setup
      • Release Package Ubuntu 24.04
      • Docker Image
    • Compile and Run the Model
  • Model Preparation
    • Overview
    • Convert a TFLite model to TOSA and MLIR
    • Convert Torch Model to MLIR
    • Convert ONNX Model to MLIR
    • References
  • Input Guide
    • Supported model formats for torq-compile
    • Input Structure for torq-run-module
      • Literal Inputs
      • Multiple Inputs
      • Feeding Input from Files
      • Converting Images to .npy or .bin
  • Step-by-Step Model Deployment Examples
    • Prerequisites
    • Prepare Model (Convert to MLIR)
    • Compile for Device
      • Example: TFLite model (MobileNetV2)
      • Example: ONNX model
    • Run Inference on Torq
      • Using the Python Bindings
    • Compiling for Host (Simulation)
    • Handling Unsupported Operations
  • Torq Runtime Python API (Beta)
    • Installation
    • Quick Start
    • API Reference
      • VMFBInferenceRunner
        • infer(inputs)
      • profile_vmfb_inference_time
      • run_vmfb
      • TensorInfo
      • Utility Functions
        • random_inputs_from_info(inputs_info)
    • Examples
      • Inspecting Model Inputs and Outputs
      • Profiling Inference Latency
      • Running with Custom Inputs
  • Supported Operators
    • Activation related Operators
    • Element-Wise Operators
    • Composite Operators
    • Other Operators
  • Profiling
    • Compile-Time Profiling
    • Runtime Profiling
    • Profiling with the Test Framework
      • Wall-time measurement
  • Handling Unsupported Ops
    • CSS Fallback
    • Host Fallback
  • Compiling Models for Custom Synaptics SoCs
    • NSS Subsystem Overview
    • Custom Hardware Configuration
      • Example Configurations
  • Debug tips and tricks
  • References
  • Glossary
  • Release Notes
    • Version 1.5.1 (2026-04-20)
    • Version 1.5.0 (2026-04-03)
    • Version 1.1.0 to 1.4.0
    • Version 1.0.0
    • Version 0.9.0
      • Key Features & Enhancements
      • Fixed Issues
      • Known Issues
      • Breaking Changes in This Release
      • Deprecated Features
      • Documentation
  • What is SyNAP?
    • SyNAP - Torq Backend
    • Torq Model to Synap model conversion
    • Runtime Integration
  • SyNAP - Torq Tests
    • synap_cli Examples
    • GStreamer Pipelines and AI Examples
    • OOBE AI Demo
• Developer Manual
  • Introduction
  • Build and setup
    • Prerequisites
    • Download the code
    • Install required system packages
    • Build compiler and runtime for host
    • Build runtime for target
  • Compile and run a model
  • Design
    • Torq compiler backend
      • Overview
      • Dialects
      • Passes
      • Data Layout
      • Memory Allocation and Planning
      • Strided Convolutions
      • Torq HAL driver
  • TOSA to Torq Conversion
    • Example
  • Kernel Development
    • NPU HW Overview
    • Programming a Slice
    • EasyKernel
    • Tutorial
      • Vector Multiplication (basic)
      • Vector Multiplication (vectorized)
      • Tensor multiplication (vectorized)
      • Tensor multiplication (optimized)
      • Matrix Reduce
    • Reference
      • Kernel Structure
      • ALU Instructions
      • Vectorization
      • Writing output results
  • Adding a New Pass
    • Example
  • The Super Tiling Pass
    • Fuse groups
    • Memory footprint check
    • Finding a good tiling factor
  • Adding a new compute optimization
  • Recommended submission process
    • Coding Style
    • Updating this Documentation
  • Debug tips and tricks
  • Troubleshooting
  • References
    • Glossary
  • Testing
    • Running tests
    • Profiling of tests
    • How test code works
      • Fixtures
      • Parametrization
      • Multiple parameters
      • Caching
  • Tools
    • Template Profiling Tool
      • Overview
      • Prerequisites
        • Hardware Setup for Remote Testing
      • Quick Start
        • Tool Output
      • Architecture
      • How It Works
        • Template MLIR Files
        • Shape Generation Algorithm
        • Compiler Variants
      • Features and Capabilities
        • What It Can Do
          • 1. Multi-Rank Testing
          • 2. Multi-Dtype Support
          • 3. Dynamic Shape Generation
          • 4. Performance Profiling
          • 5. Selective Test Execution
          • 6. Custom Compiler Options
      • Understanding Test Output
        • Profiling Output
      • Limitations and Current Constraints
        • What It Currently Cannot Do
          • 1. Arbitrary Shape Relationships
          • 3. Complex Dtype Interactions
          • 4. Memory Constraint Validation
          • 5. Multi-Input Shape Broadcasting
  • Model testing and debugging workflow
    • Running test suite for a model
      • Initial setup
    • Collect all the tests for a model
    • What the tests do
    • Executing the tests
    • Investigating ERORRs
    • Investigating FAILUREs
    • Debugging a test
    • Performance debugging
    • Executing tests using real hardware
      • Auto-deploying the runtime to the board
      • Deploying a custom NPU kernel module
      • Finding boards on the network
    • Using the results dashboard
    • Advanced: Test framework internals and PyTest fixtures
    • TFLite model testing
      • Initial setup
      • Collecting tests
      • What the tests do
      • Environment variables
      • Running specific subsets
      • How layer extraction works
      • Caching
      • Investigating errors and failures
      • Key TFLite fixtures