Snapdragon NPU: Windows on Arm

Dec 30, 2025 • 18 min read

Apple Silicon's Neural Engine gave Apple laptops a massive advantage for on-device AI — up to 38 TOPS (Tera Operations Per Second) of dedicated ML accelerator performance. Qualcomm's Snapdragon X Elite closes this gap: the Hexagon NPU delivers 45 TOPS, making it the most powerful NPU in any Windows laptop. The catch: unlike Apple's Core ML framework, Qualcomm's toolchain requires navigating the Qualcomm AI Stack, AI Hub compilation service, and understanding when to use platform-specific vs. cross-vendor (DirectML) approaches.

1. The Snapdragon AI Hardware Stack

2. Qualcomm AI Hub: Automated Model Compilation

The Hexagon NPU requires models to be compiled to a specific binary format targeting the exact NPU revision. Qualcomm AI Hub does this automatically:

pip install qai-hub torch torchvision

import qai_hub
import torch
import torchvision

# Step 1: Load your PyTorch model
model = torchvision.models.efficientnet_b0(pretrained=True)
model.eval()

# Step 2: Trace the model (convert to TorchScript)
sample_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(model, sample_input)

# Step 3: Submit to AI Hub for NPU compilation
# AI Hub compiles, optimizes, and quantizes for the target device
compile_job = qai_hub.submit_compile_job(
    model=traced_model,
    input_specs={"image": ((1, 3, 224, 224), "float32")},
    device=qai_hub.Device("Snapdragon X Elite CRD"),  # Target device
    options="--target_runtime qnn_lib_aarch64_android",  # NPU runtime
)

# Wait for compilation (usually 5-15 minutes)
compile_job.wait()
assert compile_job.get_status().success

# Download optimized model binary
target_model = compile_job.get_target_model()
target_model.download("efficientnet_b0_hexagon.bin")
print(f"Model compiled successfully for Snapdragon X Elite NPU")

3. Profiling Model Performance on Device

# Profile before deploying — AI Hub runs the model on real device hardware
profile_job = qai_hub.submit_profile_job(
    model=target_model,
    device=qai_hub.Device("Snapdragon X Elite CRD"),
    input_specs={"image": ((1, 3, 224, 224), "float32")},
)

profile_job.wait()
profile = profile_job.download_profile()

# Profile output:
print(f"Inference time: {profile.inference_time_ms:.2f}ms")
print(f"Peak memory: {profile.peak_memory_bytes / 1e6:.1f} MB")
print(f"Layer breakdown:")
for layer in profile.top_N_layers_by_latency(5):
    print(f"  {layer.name}: {layer.latency_ms:.2f}ms ({layer.runtime})")

# Example output for EfficientNet-B0:
# Inference time: 3.2ms
# Peak memory: 18.3 MB
# Layer breakdown:
#   conv_stem: 0.8ms (NPU)
#   blocks.0.0.depthwise_conv: 0.4ms (NPU)
#   ... all layers on NPU ✓

4. Deploying to Windows Application with ONNX Runtime

# For Windows applications: use ONNX Runtime with QNN Execution Provider
pip install onnxruntime-qnn  # Qualcomm QNN execution provider

import onnxruntime as ort
import numpy as np

# First convert model to ONNX format
import torch.onnx
sample_input = torch.rand(1, 3, 224, 224)
torch.onnx.export(
    model, sample_input, "model.onnx",
    input_names=["image"],
    output_names=["logits"],
    opset_version=17,
    dynamic_axes={"image": {0: "batch"}}
)

# Load with QNN Execution Provider (targets Hexagon NPU)
session_options = ort.SessionOptions()
session = ort.InferenceSession(
    "model.onnx",
    providers=["QNNExecutionProvider"],
    provider_options=[{
        "backend_path": "QnnHtp.dll",  # HTP = Hexagon Tensor Processor
        "enable_htp_fp16_precision": "1",
        "profiling_level": "off",
    }],
    sess_options=session_options,
)

# Run inference on NPU
input_array = np.random.randn(1, 3, 224, 224).astype(np.float32)
outputs = session.run(["logits"], {"image": input_array})
predictions = outputs[0]  # Shape: (1, 1000) for ImageNet classification

5. DirectML: Cross-Vendor NPU Access

# DirectML works across Qualcomm, Intel, AMD, and NVIDIA hardware
# Ideal when you want one binary targeting all Windows AI PCs

pip install onnxruntime-directml

import onnxruntime as ort

# DirectML automatically routes to the best available accelerator:
# - Snapdragon X Elite → Hexagon NPU via DirectML
# - Intel Core Ultra   → Intel NPU via DirectML  
# - AMD Ryzen AI       → AMD AI accelerator via DirectML
# - NVIDIA GPU         → GPU via DirectML (slower than CUDA, but works)

session = ort.InferenceSession(
    "model.onnx",
    providers=["DmlExecutionProvider", "CPUExecutionProvider"],
    # DmlExecutionProvider is tried first, falls back to CPU
)

# Check which execution provider was selected
provider = session.get_providers()[0]
print(f"Running on: {provider}")
# → "DmlExecutionProvider" on Snapdragon X Elite Windows laptop

# Performance comparison (EfficientNet-B0 on Snapdragon X Elite):
# CPU only:    ~45ms per inference 
# DirectML:    ~8ms per inference
# QNN (NPU):   ~3ms per inference (fastest, but Qualcomm-only)
# Tradeoff: QNN = best performance, DirectML = cross-platform compatibility

6. On-Device LLM with AI Hub

# Qualcomm provides pre-compiled versions of popular LLMs via AI Hub
# No compilation needed — download and run directly

import qai_hub

# List available pre-compiled models
models = qai_hub.get_models(device=qai_hub.Device("Snapdragon X Elite CRD"))
llm_models = [m for m in models if 'llama' in m.name.lower() or 'phi' in m.name.lower()]

# Llama 3.2 1B Instruct — fits on Hexagon NPU, ~10 tokens/sec
model = qai_hub.get_model("llama_v3_2_1b_instruct_quantized")
model.download("llama3.2-1b-hexagon.bin")

# Performance on Snapdragon X Elite (measured):
# Llama 3.2 1B:  ~40 tokens/sec on Hexagon NPU (fast, small)
# Phi-3 Mini:    ~25 tokens/sec on NPU (good quality)  
# Llama 3 8B:    ~8 tokens/sec on NPU (high quality, slower)
# Note: 70B+ models cannot fit in NPU memory — require cloud API

Frequently Asked Questions

Is Qualcomm NPU support better or worse than Apple Neural Engine?

Apple's Core ML ecosystem is more mature — it has been available since 2017 and most ML frameworks (PyTorch, TensorFlow, Scikit-learn) support Core ML export. Qualcomm's toolchain requires the AI Hub compilation step and the QNN execution provider, which adds complexity. However, Qualcomm's NPU raw performance (45 TOPS on Snapdragon X Elite) is competitive with or exceeds Apple M3 Pro's Neural Engine (18 TOPS unified), making the more complex toolchain worthwhile for Windows developers targeting AI PCs.

Can I run Qualcomm-compiled models without Windows on Arm?

Qualcomm's Hexagon NPU models (.bin files from AI Hub) require Snapdragon hardware. However, Qualcomm also supports Android (Snapdragon phones and tablets) with the same AI Hub workflow. The ONNX models (pre-QNN compilation) run anywhere ONNX Runtime runs — the QNN execution provider just routes them to the NPU when Snapdragon hardware is detected.

Conclusion

The Qualcomm AI Stack — AI Hub for automated NPU compilation, QNN Execution Provider for Qualcomm-specific performance, and DirectML for cross-vendor compatibility — gives Windows developers a complete toolkit for on-device AI. For applications targeting the AI PC market (Copilot+ PCs with Snapdragon X, Intel NPUs, or AMD AI accelerators), DirectML provides the lowest-friction path to hardware-accelerated inference. For Snapdragon-exclusive applications (mobile or Windows on Arm), the QNN path provides the best performance with AI Hub handling the complex compilation automatically.