Modeling Optimal Tile Size for Enhancing Data Reuse in Convolutional Neural Networks
S. Seydi
1
(
)
M. Salehi
2
(
University of Tehran
)
Keywords: Convolutional neural networks, energy consumption, off-chip memory, data reuse, tiling.,
Abstract :
Artificial neural networks are a class of computational models inspired by the structure and functionality of biological neural networks in the human brain. Convolutional Neural Networks (CNNs), as a prominent type of these models, are widely applied in various domains such as image classification, object detection, natural language processing, and healthcare.
As CNN architectures grow in size, the number of parameters and the volume of data movement increase, leading to higher dependence on off-chip memory, which in turn significantly raises energy consumption. A primary strategy for reducing both energy usage and off-chip memory accesses is to maximize data reuse at every level of the memory hierarchy. Data reuse can be exploited at three levels: (1) data flow and processing elements, (2) loop and computation scheduling, and (3) inter-layer and network-level operations.
Tiling is one of the key techniques for improving data reuse at the scheduling level. In this work, we present a precise mathematical formulation for modeling the number of data reuses. We then formulate an optimization problem to determine the optimal parameters for maximizing data reuse for each network configuration. Furthermore, we investigate the relationship between network structural parameters, such as kernel size and stride, and the optimal tile size. Our analysis shows that, in 70% of the network layers examined, the optimal tile size is smaller than four times the kernel size.