Algorithm/Hardware Co-Design for In-Memory Computing Accelerators for Binary Neural Networks

Abstract

While Deep Neural Networks (DNNs) have shown cutting-edge performance on various applications, state-of-the-art DNN models are usually very deep and large. The increased compute cost and memory requirement of such large DNN models pose a burden on resource-constrained environments such as mobile devices. To mitigate the problem, in-memory computing has been proposed as a promising platform for DNN computations. Unlike the conventional Von Neumann computing architecture, in-memory computing enables computing with minimal data transfer from memory to other processing units. Since the memory devices serve as storage element for the weights of DNN models as well as compute element on which vector-matrix multiplication is performed, memory bottleneck due to the large amount of data transfer can be avoided. Meanwhile, quantization of DNN models has been actively studied to reduce the compute cost and memory requirement of large DNN models. By reducing the bitwidth of weights and/or activations, energy consumed to run a DNN model can also be saved. Recent researches have demonstrated that both weight and activation can be quantized to even 1-bit precision in Binary Neural Networks (BNNs). In BNNs, high precision multiplication and accumulation can be replaced by simple XNOR logic operation and popcount operation. Due to the lightweight nature, BNNs are becoming popular for various applications on edge devices. When in-memory computing and BNN are combined together, they show great synergy. Since in-memory computing is based on analog computing, it requires analog-to-digital and/or digital-to-analog conversion of data. The peripheral circuits for those conversions are one of the main source of energy and area consumption of in-memory computing. When BNNs are computed within an in-memory computing platform, high precision peripheral circuits can be simplified. Therefore, in-memory BNN computing has great potential as an energy efficient platform for mobile applications. In this dissertation, several methods are proposed to improve the energy efficiency of in-memory BNN accelerators and to improve the accuracy of BNNs. To minimize the overhead of high resolution peripheral circuits, in-memory batch-normalization (BN) technique is proposed. While most BNNs use BN layers, implementation of BN layer on in-memory computing architecture has not been discussed in depth yet. In this work, a technique to compute the BN layer on memory array is proposed so that the overhead of peripheral circuit can be reduced. Signed-to-unsigned input conversion technique is also proposed to reduce the energy consumption of memory array. The technique allows one to use input 0's for BNN so that energy consumption of in-memory computing can be saved. In addition, the ratio regularization technique that maximizes the number of input 0's during training is proposed to fully exploit the energy saving from input 0's. While the proposed techniques can improve the energy efficiency of in-memory BNN accelerators, accuracy degradation due to aggressive binarization limits the deployment of in-memory neural network accelerators in real-world applications. To mitigate the problem, BitSplit-Net, in which multi-bit activations are approximated by multiple binary activations, is proposed. Furthermore, an in-memory computing architecture on which the BitSplit-Net can be mapped with minimal peripheral circuit overhead is proposed. In addition, a training scheme that can improve the accuracy of BNNs is also proposed.