Ir directamente a la navegación principal Ir directamente a la búsqueda Ir directamente al contenido principal

H4H: Hybrid Convolution-Transformer Architecture Search for NPU-CIM Heterogeneous Systems for AR/VR Applications

  • Yiwei Zhao*
  • , Jinhui Chen
  • , Sai Qian Zhang
  • , Syed Shakib Sarwar
  • , Kleber Hugo Stangherlin
  • , Jorge Tomas Gomez
  • , Jae Sun Seo
  • , Barbara De Salvo
  • , Chiao Liu
  • , Phillip B. Gibbons
  • , Ziyun Li
  • *Autor correspondiente de este trabajo

Producción científica: Capítulo del libro/informe/acta de congreso › Contribución a la conferencia › revisión exhaustiva

4 Citas (Scopus)

Resumen

Low-latency and low-power edge AI is crucial for Augmented/Virtual Reality applications. Recent advances demonstrate that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve a superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can present system challenges for latency and energy efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and explore diverse execution schemas for efficient hybrid model executions. We introduce H4H-NAS, a two-stage Neural Architecture Search (NAS) framework to automate the design of hybrid CNN/ViT models for heterogeneous edge systems featuring both NPU and CIM. We propose a two-phase incremental supernet training in our NAS to resolve gradient conflicts between sampled subnets caused by different block types in a hybrid model search space. Our H4H-NAS approach is also powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN-ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet-1k. Moreover, results from our algorithm/hardware co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing heterogeneous computing over baseline solutions. Overall, our framework guides the design of hybrid network architectures and system architectures for NPU+CIM heterogeneous systems.

Idioma originalInglés
Título de la publicación alojadaASP-DAC 2025 - 30th Asia and South Pacific Design Automation Conference, Proceedings
EditorialInstitute of Electrical and Electronics Engineers Inc.
Páginas1133-1141
Número de páginas9
ISBN (versión digital)9798400706356
DOI
EstadoPublicada - 4 mar 2025
Publicado de forma externaSí
Evento30th Asia and South Pacific Design Automation Conference, ASP-DAC 2025 - Tokyo, Japón
Duración: 20 ene 2025 → 23 ene 2025

Serie de la publicación

NombreProceedings of the Asia and South Pacific Design Automation Conference, ASP-DAC
ISSN (versión impresa)2153-6961
ISSN (versión digital)2153-697X

Conferencia o congreso

Conferencia o congreso30th Asia and South Pacific Design Automation Conference, ASP-DAC 2025
País/TerritorioJapón
CiudadTokyo
Período20/01/25 → 23/01/25

Nota bibliográfica

Publisher Copyright:
© 2025 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.

ODS de las Naciones Unidas

Este resultado contribuye a los siguientes Objetivos de Desarrollo Sostenible

  1. ODS 7: Energía asequible y no contaminante
    ODS 7: Energía asequible y no contaminante

Huella

Profundice en los temas de investigación de 'H4H: Hybrid Convolution-Transformer Architecture Search for NPU-CIM Heterogeneous Systems for AR/VR Applications'. En conjunto forman una huella única.

Citar esto