I. INTRODUCTION

Analog synaptic devices are being investigated as one of the most important parts of neuromorphic systems because biological synapses play a role in signal transmission and memory effect ⁽¹⁾. Recently, several types of emerging electronic synapse devices such as phase-change memory (PCRAM) ⁽²⁾, resistive change memory (RRAM) ^(3,4), ferroelectric devices ⁽⁵⁾, and FET-based devices ^(6,7,8) have been proposed to biological synapses. These synaptic devices, such as RRAM, MRAM, and NOR-FLASH, each have their advantages and disadvantages. The high current operation, a poor characteristic variation, reliability, and a requirement of selectors on RRAM and scalability on MRAM and NOR-FLASH are known issues. But The devices which currently used in mass production are NAND flash memory and logic transistors. We believe that we need a way to realize analog synapses with best known process and design capabilities based on those mass-production experiences. I think that it would be good to make NAND-flash with a similar structure to the standard cell of the logic CMOS device. Therefore, in this paper, we propose an analog switch type synapse using a NAND flash device and a pMOSFET.

II. NAND FLASH MEMORY FOR SYNAPSTIC DEVICE

1. NAND Array for VMM

A method of constructing a synapse array using NAND cell strings has been steadily proposed due to its scalability and mass productivity. For VMM operation, all cells except the target cell are connected with the corresponding NAND cell string by applying pass voltage(V$_{\mathrm{pass}}$) as shown in Fig. 1.

Fig. 1. V$_{\mathrm{Read}}$ is applied for the output of synapse #0, otherwise V$_{\mathrm{Ppass}}$

For the weighted sum operation of the output neuron, a complex circuit is required to sequentially apply V$_{\mathrm{read}}$ to all cells in the string and finally calculate the weighted sum. This approach also requires an additional circuit in complex neural networks that do not know when the input signal will come as Fig. 2.

Fig. 2. V$_{\mathrm{Read}}$ is applied for the output of synapse #0, otherwise V$_{\mathrm{pass}}$

The pre-synaptic signal is applied to V$_{\mathrm{read}}$ and V$_{\mathrm{pass}}$ according to the system clock and compares the amount of current flowing through the string in the sense amp. The value corresponding to the entire synapse cell needs to be accumulated through the adder before going to the postsynaptic line. This operation method is inefficient because it requires the additional signal formation circuits to generate pre-synaptic signals and compare and calculate string out currents. A different algorithm is required to use NAND flash memory as a synaptic device.

III. ANALOG SWITCH FOR SYNAPSE

1. NAND String with pMOSFET for Connecting

Accumulated charge of synapse array with NAND can represent the weighted sum of neuron input in Fig. 3(a). The NAND array is broken when one cell is turned off, so it is necessary to preserve the current. If a specific route is connected by pMOSFET, a current path can be formed Fig. 3(b). If a specific resistance is used instead of pMOSFET, it can make a disturbance on NAND cell output.

Fig. 3. proposed NAND string for using synaptic array.

We can derive the total current of this series approximately. For the derivation of a relation between synapse series current and each synapse weight. We assume binary weight system, weight=0 or 1. k$^{\mathrm{th}}$ synapse cell is on state for total n cells series otherwise off state. Equations describing total NAND string current (I$_{\mathrm{total}}$) with applied the pre-synaptic input voltage (V$_{\mathrm{GS}}$) and bit line voltage (V$_{\mathrm{BL}}$) are listed below:

(1)

$$I_{D S}=\mu_{n} \frac{\varepsilon_{o x}}{t_{o x}} \frac{W}{L}\left(V_{G S}-V_{T H}\right) V_{D S} \text { for } V_{G S} \gg V_{D S}$$

(2)

$$I_{\text {total}}=\frac{V_{B L}}{2 R_{o n} n-k}=V_{B L}\left(2 R_{o n} n-k\right)^{-1}$$

(3)

$$\begin{array}{l} I_{\text {total}} \cong \frac{V_{B L}}{R_{o f f} n}+\frac{V_{B L}}{4 R_{o n}^{2} n^{2}} k \\ \text { at } R_{o n}=\frac{1}{\mu_{n}} \frac{t_{o x}}{\varepsilon_{o x}} \frac{L}{W} V_{o v} \end{array}$$

Where R$_{\mathrm{off}}$ denotes off-state resistance value for no pre-synaptic input signal case, R$_{\mathrm{on}}$ is for having a pre-synaptic input signal (V$_{\mathrm{ov}}$). I$_{\mathrm{DS}}$ is current for each synapse cell and Vov is the difference between V$_{\mathrm{GS}}$ and threshold voltage(V$_{\mathrm{th}}$). We can define total resistance R$_{\mathrm{total }}$and total current I$_{\mathrm{total}}$.

There is interference due to each cell V$_{\mathrm{DS}}$ differences which can occur depending on the position of each cell in the array. But we can assume V$_{\mathrm{DS}}$ is the same at each synapse cell. It because V$_{\mathrm{DS}}$ is very small value due to bit line voltage(V$_{\mathrm{BL}}$) dividing by number of synapse cells. And R$_{\mathrm{off}}$ value is not big different from R$_{\mathrm{on}}$ also.

Eq. (3) is Taylor approximation to 1$^{\mathrm{st}}$ order. We assume total number of synapse cells is big enough to the number of on-state cells (n{\textgreater}{\textgreater}k).

2. Characteristics of Synapse

Fig. 4(a) is schematic of a proposed synapse cell. The gates of NAND and pMOSFET are connected to the same input. When current flows through the NAND cell, there is no current distortion because the pMOSFET is off. At the same time, if the synapse input is 0, the pMOSFET will operate to provide the current. Fig. 4(b) is the layout of the proposed synapse. This is similar to the form of a CMOS logic circuit. The similarity of the layout is a big advantage in the fabrication process in terms of process variation when the logic cell and this synapse cell process at the same time. The input value is represented by the number of spikes. In addition, V$_{th}$ change according to the amount of charge trap of NAND cell can represent the weight in Fig. 4(c).

Fig. 4. single synapse cell structure (a) Schematic of 1 synapse cell with 1 floating gate (b) Design layout of 1 synapse cell (c) synapse cell characteristic by V$_{th}$

IV. CIRCUIT OPERATION ANALYSIS

In the circuit operation analysis for VMM: Using the 10 cells synapse devices, we perform the spice simulation of VMM operation. As in Fig. 5(a), excitatory synapse and inhibitory synapse is made using 20 synapses with current mirrors, and the voltage of membrane capacitance of Integrate-and-Fire (I&F) system is measured.

To do this, we randomly generate a weight value between -1 and 1 and an input signal between 0 and 100. The input pulse uses the rate coding method and V$_{th}$ is adjusted to the second decimal point. Then weighted sum value calculation is listed below:

And V$_{th}$ value is listed for the inference below:

As a result of Fig. 5(b), the weighted sum and membrane potential has a 99% correlation value. For the perfect definition of synapse weight, membrane voltage have to be exact linear relation. But there is some variation. This comes from assumptions in Eqs. (1)-(3). VDS cannot be exact same at each synapse node. And Itotal is not perfect linear function even in region n>>k. These two assumption can lead an accuracy degradation. Even though existing variation, membrane voltage and weighted sum value has a good correlation.

Fig. 5. VMM SPICE simulation setup and result (a) 10 excitatory and 10 inhibitory synapses are implemented with current mirrors and membrane capacitor for neuron input, (b) R$^{2}$ value is near 0.99 for a randomly generated weighted sum.

The precision of the NAND cell has a big impact on VMM accuracy. To check the VMM accuracy, we perform the simulation by the number of V$_{th}$ offering of NAND cell. The R$^{2}$ value is 95% with 3bit NAND cell. There is a significant accuracy degradation for under 2 bits NAND cell in Fig. 6. In order to secure accuracy, it is necessary to make a sufficient V$_{th}$ range.

Fig. 6. (a) VMM correlation chart between membrane potential which is output of synapse and weighted sum value by # of weights on NAND, (b) VMM correlation value by # of weights on NAND.

V. BENCH MARKING RESULT WITH MNIST

Based on this synapse array, we examine the difference in recognition rate according to ANN's accuracy and cell variation of the proposed synapse. For this purpose, 8×8 hand-written digit data are used as shown in Table 1 ⁽⁹⁾.

Fig. 7 is the entire VMM architecture for MNIST benchmarking. Program and erase operation for weight projection is the same as usual NAND memory architecture. The Only capacitor is added for current integration. This membrane capacitor will connect to the neuron circuit to fire postsynaptic spike.

Fig. 7. Entire VMM architecture for MNIST bench marking.

First, 1-layer fully-connected-network, which is a reference, performs learning using SciKit-Learn sample code ⁽¹⁰⁾. The recognition rate was 86%. The obtained weight value infers to each NAND cell by value of threshold voltage based on Eq. (3). And the 1,797 rate coded input data is converted into an input signal by modulated input frequency. SPICE simulation is performed by HSPICE simulator. After performing spice simulation, we pick up the largest voltage value of the membrane capacitor. It is the result of model recognition. We complete the correlation plot as shown in Fig. 8(a) and (b). The recognition rate has a difference by 3% compared with ANN's.

Fig. 8. Correlation plot of 8×8 MNIST by spice simulation (a) result of 1-layer FCN by ANN, (b) result of 1-layer FCN by SPICE simulation with weight projection which is from ANN result.

VI. CONCLUSIONS

The analog switch type synapse using the proposed NAND array and pMOSFET has the advantage that the existing NAND program / erase mechanism can be used without any change in the mass production process. At the same time, it has the advantage of using same architecture of standard cell unit on an industry process. At the same time having a chance of smaller area than silicon-based synapse due to processing with logic and NAND synapse unit. And SPICE simulation of this synapse inference verification of FCN network resulted in 83% recognition rate, which is 3% lower than reference value of ANN.