Hugging Face's logo

qualcomm
/
ResNet-Mixed-Convolution

Video Classification
PyTorch
backbone
android
Model card Files
xet
Community
1
ResNet-Mixed-Convolution
File size: 9,675 Bytes
1cbfac7
 
76be976
1cbfac7
 
 
4b6cff5
1cbfac7
 
 
 
 
7be4d2d
1cbfac7
 
 
7be4d2d
 
 
 
 
 
 
 
 
 
 
 
 
 
1aa5be8
 
 
 
 
7be4d2d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1aa5be8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1cbfac7
 
a6d3324
 
1cbfac7
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
 
---
library_name: pytorch
license: other
tags:
- backbone
- android
pipeline_tag: video-classification

---

![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/web-assets/model_demo.png)

# ResNet-Mixed-Convolution: Optimized for Qualcomm Devices

ResNet Mixed Convolutions is a network with a mixture of 2D and 3D convolutions used for video understanding.

This is based on the implementation of ResNet-Mixed-Convolution found [here](https://github.com/pytorch/vision/blob/main/torchvision/models/video/resnet.py).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/resnet_mixed) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

## Getting Started
There are two ways to deploy this model on your device:

### Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.47.0/resnet_mixed-onnx-float.zip)
| ONNX | w8a16 | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.47.0/resnet_mixed-onnx-w8a16.zip)
| QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.47.0/resnet_mixed-qnn_dlc-float.zip)
| QNN_DLC | w8a16 | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.47.0/resnet_mixed-qnn_dlc-w8a16.zip)
| TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/resnet_mixed/releases/v0.47.0/resnet_mixed-tflite-float.zip)

For more device-specific assets and performance metrics, visit **[ResNet-Mixed-Convolution on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/resnet_mixed)**.


### Option 2: Export with Custom Configurations

Use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/resnet_mixed) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for [ResNet-Mixed-Convolution on GitHub](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/resnet_mixed) for usage instructions.

## Model Details

**Model Type:** Model_use_case.video_classification

**Model Stats:**
- Model checkpoint: Kinetics-400
- Input resolution: 112x112
- Number of parameters: 11.7M
- Model size (float): 44.6 MB
- Model size (w8a16): 11.5 MB

## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® X Elite | 13.83 ms | 22 - 22 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 9.602 ms | 2 - 276 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Qualcomm® QCS8550 (Proxy) | 13.162 ms | 0 - 56 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Qualcomm® QCS9075 | 26.529 ms | 2 - 5 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.818 ms | 0 - 215 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.558 ms | 2 - 221 MB | NPU
| ResNet-Mixed-Convolution | ONNX | float | Snapdragon® X2 Elite | 7.095 ms | 22 - 22 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® X Elite | 9.075 ms | 11 - 11 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 6.385 ms | 1 - 257 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCS6490 | 1728.003 ms | 47 - 64 MB | CPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 8.508 ms | 0 - 18 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCS9075 | 9.052 ms | 1 - 4 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Qualcomm® QCM6690 | 903.451 ms | 103 - 109 MB | CPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 5.215 ms | 1 - 192 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 871.987 ms | 136 - 142 MB | CPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 3.512 ms | 0 - 191 MB | NPU
| ResNet-Mixed-Convolution | ONNX | w8a16 | Snapdragon® X2 Elite | 4.646 ms | 12 - 12 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® X Elite | 13.998 ms | 2 - 2 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 9.545 ms | 0 - 284 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 97.144 ms | 1 - 223 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 13.208 ms | 2 - 5 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA8775P | 25.07 ms | 1 - 223 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS9075 | 27.827 ms | 4 - 8 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 27.657 ms | 0 - 240 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA7255P | 97.144 ms | 1 - 223 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Qualcomm® SA8295P | 26.762 ms | 0 - 194 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.622 ms | 2 - 227 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.733 ms | 2 - 233 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | float | Snapdragon® X2 Elite | 7.684 ms | 2 - 2 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® X Elite | 9.806 ms | 1 - 1 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Gen 3 Mobile | 6.638 ms | 1 - 256 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS6490 | 37.23 ms | 3 - 6 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS8275 (Proxy) | 29.752 ms | 1 - 190 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS8550 (Proxy) | 9.152 ms | 1 - 4 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA8775P | 9.413 ms | 1 - 191 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS9075 | 10.394 ms | 1 - 4 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCM6690 | 183.549 ms | 1 - 209 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® QCS8450 (Proxy) | 13.258 ms | 1 - 253 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA7255P | 29.752 ms | 1 - 190 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Qualcomm® SA8295P | 16.301 ms | 1 - 192 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 5.263 ms | 1 - 189 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 7 Gen 4 Mobile | 15.829 ms | 1 - 197 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4.008 ms | 1 - 189 MB | NPU
| ResNet-Mixed-Convolution | QNN_DLC | w8a16 | Snapdragon® X2 Elite | 5.374 ms | 1 - 1 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 216.262 ms | 0 - 314 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 585.629 ms | 0 - 248 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 309.699 ms | 0 - 3 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA8775P | 293.466 ms | 0 - 249 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 334.839 ms | 0 - 281 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA7255P | 585.629 ms | 0 - 248 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Qualcomm® SA8295P | 358.637 ms | 0 - 232 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 201.617 ms | 0 - 255 MB | NPU
| ResNet-Mixed-Convolution | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 198.394 ms | 0 - 251 MB | NPU

## License
* The license for the original implementation of ResNet-Mixed-Convolution can be found
  [here](https://github.com/pytorch/vision/blob/main/LICENSE).

## References
* [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://arxiv.org/abs/1711.11248)
* [Source Model Implementation](https://github.com/pytorch/vision/blob/main/torchvision/models/video/resnet.py)

## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).