Tools

• Template Profiling Tool
  • Overview
  • Prerequisites
    • Hardware Setup for Remote Testing
  • Quick Start
    • Tool Output
  • Architecture
  • How It Works
    • Template MLIR Files
    • Shape Generation Algorithm
    • Compiler Variants
  • Features and Capabilities
    • What It Can Do
      • 1. Multi-Rank Testing
      • 2. Multi-Dtype Support
      • 3. Dynamic Shape Generation
      • 4. Performance Profiling
      • 5. Selective Test Execution
      • 6. Custom Compiler Options
  • Understanding Test Output
    • Profiling Output
  • Limitations and Current Constraints
    • What It Currently Cannot Do
      • 1. Arbitrary Shape Relationships
      • 3. Complex Dtype Interactions
      • 4. Memory Constraint Validation
      • 5. Multi-Input Shape Broadcasting
• torq-mlir-query
  • Building
  • Usage
    • Interactive use
    • Non-interactive use (file / stdin)
  • Registered matchers
  • Composing matchers
  • Extracting matches into a function
  • Tab completion
  • Registering a new matcher
• torq.lab developer guide
  • Source layout
  • Core API
    • Driving the pipeline
    • Running the CLI in-process
  • Packaging
  • Adding a command
• Gen-Config Architecture
  • Table of Contents
  • 1. Overview
  • 2. Module Map
    • Responsibility split
    • Key helpers in core.py
  • 3. Dependency Graph
  • 4. Two-JSON Design
    • Report JSON (ops format)
    • Compiler JSON (op_assignments format)
    • Generation rules
  • 5. Quantization Support
    • Discovery flow with quantization
    • Mapping quantized MLIR to original layers
    • Quantization options
  • 6. CLI Subcommand Flow
    • discover
    • run
    • view
    • edit
  • 7. How to Add a New Model Format
    • Step 1: Create _<format>.py extraction/conversion helpers
    • Step 2: Add the engine hooks in _runner.py (or _runner_<format>.py)
    • Step 3: Wire the format into cli.py
    • Key principle
  • 8. How to Add a New CLI Subcommand
    • Step 1: Add the handler in cli.py
    • Step 2: Register in main()
    • Step 3: Add tests
    • Convention
  • 9. Testing Architecture
    • Test files
    • CLI test pattern
    • Running tests
  • 10. Standalone CLI Options
  • 11. TFLite Discovery (Removed)
• TORQ ONNX Quantization
  • Table of Contents
  • 1. Overview
    • High-level lowering flow
  • 2. ONNX Quantization Formats
    • QDQ
    • QOperator
    • Hybrid
  • 3. torq-gen-config Quantization Workflow
    • Supported Quantization Options
    • Per-Layer vs. Full-Model Quantization
    • CLI Usage
  • 4. Quantized Convolution Zero-Point Correction
    • 4.1 Operation definition
    • 4.2 Mechanical rewriting
    • 4.3 Sub-component definitions
      • 4.3.1 Weight zero-point folding
      • 4.3.2 Input zero-point correction
      • 4.3.3 Bias addition
      • 4.3.4 Output rescale and clamp
    • 4.4 Lowering each sub-component
      • 4.4.1 QConv2DPattern.cpp
      • 4.4.2 QuantizePattern.cpp (fallback)
        • Quantize generic
        • Dequantize generic
    • 4.5 Special cases
      • 4.5.1 weight_zp == 0
      • 4.5.2 input_zp == 0
      • 4.5.3 Both zero-points are zero
      • 4.5.4 Dynamic weights
    • 4.6 Why the other expanded terms disappear
    • 4.7 Generalization to other weighted ops
  • 5. References
    • ONNX Runtime quantization
    • Quantization theory and best-practice papers