tinyML Research Symposium: Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural...
https://www.tinyml.org/event/research-symposium-2023/#schedule
Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference
Rafael STAHL , Doctoral candidate, Technical University of Munich
Memory optimization for deep neural network (DNN) inference gains high relevance with the emergence of TinyML, which refers to the deployment of DNN inference tasks on tiny, low-power microcontrollers. Applications such as audio keyword detection or radar-based gesture recognition are heavily constrained by the limited memory on such tiny devices because DNN inference requires large intermediate run-time buffers to store activations and other intermediate data, which leads to high memory usage. In this paper, we propose a new Fused Depthwise Tiling (FDT) method for the memory optimization of DNNs, which, compared to existing tiling methods, reduces memory usage without inducing any run time overhead. FDT applies to a larger variety of network layers than existing tiling methods that focus on convolutions. It improves TinyML memory optimization significantly by reducing memory of models where this was not possible before and additionally providing alternative design points for models that show high run time overhead with existing methods. In order to identify the best tiling configuration, an end-to-end flow with a new path discovery method is proposed, which applies FDT and existing tiling methods in a fully automated way, including the scheduling of the operations and planning of the layout of buffers in memory. Out of seven evaluated models, FDT achieved significant memory reduction for two models by 76.2% and 18.1% where existing tiling methods could not be applied. Two other models showed a significant run time overhead with existing methods and FDT provided alternative design points with no overhead but reduced memory savings.
Видео tinyML Research Symposium: Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural... канала The tinyML Foundation
Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference
Rafael STAHL , Doctoral candidate, Technical University of Munich
Memory optimization for deep neural network (DNN) inference gains high relevance with the emergence of TinyML, which refers to the deployment of DNN inference tasks on tiny, low-power microcontrollers. Applications such as audio keyword detection or radar-based gesture recognition are heavily constrained by the limited memory on such tiny devices because DNN inference requires large intermediate run-time buffers to store activations and other intermediate data, which leads to high memory usage. In this paper, we propose a new Fused Depthwise Tiling (FDT) method for the memory optimization of DNNs, which, compared to existing tiling methods, reduces memory usage without inducing any run time overhead. FDT applies to a larger variety of network layers than existing tiling methods that focus on convolutions. It improves TinyML memory optimization significantly by reducing memory of models where this was not possible before and additionally providing alternative design points for models that show high run time overhead with existing methods. In order to identify the best tiling configuration, an end-to-end flow with a new path discovery method is proposed, which applies FDT and existing tiling methods in a fully automated way, including the scheduling of the operations and planning of the layout of buffers in memory. Out of seven evaluated models, FDT achieved significant memory reduction for two models by 76.2% and 18.1% where existing tiling methods could not be applied. Two other models showed a significant run time overhead with existing methods and FDT provided alternative design points with no overhead but reduced memory savings.
Видео tinyML Research Symposium: Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural... канала The tinyML Foundation
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
![tinyML Neuromorphic Engineering Forum - Sensors Session](https://i.ytimg.com/vi/Mt9X0ALdCpI/default.jpg)
![tinyML Vision Challenge - Himax & Edge Impulse](https://i.ytimg.com/vi/6tmCEzNSIas/default.jpg)
![tinyML Talks Chao Xu: Enabling Neural network at the low power edge: A neural network compiler...](https://i.ytimg.com/vi/Rs1yeTSuZLA/default.jpg)
![SensMACH 2020 Daniel Situnayake: Embedded machine learning in the real world](https://i.ytimg.com/vi/a67hWPT1NLE/default.jpg)
![tinyML Talks: Empowering the Edge: Practical Applications of Embedded Machine Learning on MCUs](https://i.ytimg.com/vi/tkqNS611cRc/default.jpg)
![tinyML Talks: Efficient AI for Wildlife Conservation](https://i.ytimg.com/vi/FfvcZEMn2l0/default.jpg)
![tinyML Research Symposium 2022: Towards Agile Design of Neural Processing Units with Chisel](https://i.ytimg.com/vi/xlP1xdKRrqc/default.jpg)
![tinyML Talks Phoenix: Novel Device and Materials in Emerging Memory for Neuromorphic Computing](https://i.ytimg.com/vi/_apkQF1ZL6A/default.jpg)
![tinyML Talks - Phoenix meetup: Analog TinyML for health management using intelligent wearables](https://i.ytimg.com/vi/bCzg8y6aRi8/default.jpg)
![tinyML Talks India: Single Lead ECG Classification On Wearable and Implantable Devices](https://i.ytimg.com/vi/uHywaYleCtA/default.jpg)
![tinyML Summit 2023:Personal Computing devices use-case and applications enabled by Smart Sensors](https://i.ytimg.com/vi/9hvz6ZB5G8A/default.jpg)
![tinyML Talks: From the lab to the edge: Post-Training Compression](https://i.ytimg.com/vi/Ada9Tq8JAX8/default.jpg)
![tinyML Talks: State of Hardware & Software Ecosystem for Low-Power ML Applications on RISC-V](https://i.ytimg.com/vi/Rcbrc2rnXlk/default.jpg)
![tinyML Talks: Meetup Italy with small-medium industries](https://i.ytimg.com/vi/sAmRSm-tdd4/default.jpg)
![tinyML Hackathon Challenge 2023 - Infineon XENSIV 60GHz Radar Sensor and devkit explanation](https://i.ytimg.com/vi/yL6f61MKzFo/default.jpg)
![tinyML Auto ML Tutorial with Qeexo](https://i.ytimg.com/vi/qo0JTM6gaIE/default.jpg)
![tinyML On Device Learning Forum - Warren Gross: On-Device Learning For Natural Language Processing..](https://i.ytimg.com/vi/ERLFRluwRjA/default.jpg)
![EMEA 2021 tiny Talks: Building Heterogeneous TinyML Pipelines](https://i.ytimg.com/vi/p-Rtnvj4L4I/default.jpg)
![tinyML EMEA 2022- Eran Treister: Wavelet Feature Maps Compression for Image-to-Image CNNs](https://i.ytimg.com/vi/fPmvwecx7TY/default.jpg)
![tinyML Talks Pakistan: SuperSlash: Unifying Design Space Exploration and Model Compression...](https://i.ytimg.com/vi/eHDDbr0XDeY/default.jpg)
![tinyML Trailblazer Success Stories with Marian Verhelst](https://i.ytimg.com/vi/COmoXOSQALY/default.jpg)