阅读说明：本技术 脉冲神经网络的忆阻器交叉开关阵列实施方案中的脉冲检测 (Pulse detection in memristor crossbar array implementations of pulsed neural networks ) 是由 A·瓦桑塔库玛里巴布 P·卡莱 于 2020-10-29 设计创作，主要内容包括：本申请涉及脉冲神经网络的忆阻器交叉开关阵列实施方案中的脉冲检测。例如,集成电路包含连接在字线和位线之间的第一忆阻器的交叉开关阵列。所述第一忆阻器被配置成将施加在所述字线上的电压转换成所述位线中的电流。具有阈值的第二忆阻器分别连接到所述位线。当流过所述第二忆阻器中的每个相应忆阻器的电流达到所述相应忆阻器的所述阈值时,所述相应忆阻器可以减小其电阻以引起所述电流中的脉冲。电流电平检测器连接到所述第二忆阻器,以确定所述位线中的所述电流是否具有对应于达到所述第二忆阻器的阈值的电平,并因此在不使用模数转换器来测量所述位线中的所述电流的情况下,生成脉冲神经元的输出脉冲。(The application relates to pulse detection in memristor crossbar array implementations of pulsed neural networks. For example, an integrated circuit includes a crossbar array of first memristors connected between word lines and bit lines. The first memristor is configured to convert a voltage applied on the word line into a current in the bit line. Second memristors having thresholds are respectively connected to the bit lines. When the current flowing through each respective memristor in the second memristor reaches the threshold of the respective memristor, the respective memristor may reduce its resistance to cause a pulse in the current. A current level detector is connected to the second memristor to determine whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor, and thus to generate an output pulse of a pulsing neuron without using an analog-to-digital converter to measure the current in the bit line.)

1. An integrated circuit, comprising:

a crossbar array having first memristors of word lines and bit lines, the first memristors configured to convert a voltage applied on the word lines into a current in the bit lines;

second memristors respectively connected to the bit lines, the second memristors configured to control pulses in the electrical currents in the bit lines, wherein the second memristors respectively have thresholds, and wherein when an electrical current flowing through each respective memristor in the second memristors reaches the threshold of the respective memristor, the respective memristor reduces a resistance of the respective memristor to cause a pulse in the electrical current; and

a current level detector connected to the second memristor and configured to determine whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor, and to generate an output pulse of a pulsing neuron based on determining whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor.

2. The integrated circuit of claim 1, wherein digital data representing the output pulse is generated without using an analog-to-digital converter to measure a current in the bit line.

3. The integrated circuit of claim 2, wherein the threshold of the second memristor is programmed according to an activation level threshold of a pulsing neuron.

4. The integrated circuit of claim 3, wherein a resistance of the first memristor is programmed according to a pulsing neuron.

5. The integrated circuit of claim 4, wherein each of the first memristors is connected between a respective one of the word lines and a respective one of the bit lines to convert a voltage on the respective word line to a portion of a current flowing through the respective bit line.

6. The integrated circuit of claim 4, further comprising:

a pulse encoder configured to generate an input pulse according to input data and to apply the voltage on the word line of the crossbar array according to the input pulse.

7. The integrated circuit of claim 4, further comprising:

routing logic coupled to a digital communication network and configured to receive input pulse data and provide the digital data representing the output pulse.

8. A method, comprising:

converting, by a crossbar array of first memristors, a plurality of voltages applied on word lines of the crossbar array into currents in bit lines of the crossbar array;

controlling pulses in the electrical current in the bit lines by second memristors respectively connected to the bit lines, wherein the second memristors respectively have thresholds, and wherein when an electrical current flowing through each respective memristor in the second memristors reaches the threshold of the respective memristor, the respective memristor reduces the resistance of the respective memristor to cause a pulse in the electrical current;

determining, by a current level detector connected to the second memristor, whether a current in the bit line has a level corresponding to reaching a threshold of the second memristor; and

generating an output pulse of a pulsing neuron based on determining whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor.

9. The method of claim 8, wherein digital data representing the output pulse is generated without using an analog-to-digital converter to measure current in the bit line.

10. The method of claim 9, further comprising:

programming the threshold of the second memristor according to an activation level threshold of a pulsing neuron.

11. The method of claim 10, further comprising:

programming a resistance of the first memristor according to a pulsing neuron.

12. The method of claim 11, wherein each respective pair of the bit lines of the crossbar array is summed by currents generated by a subset of the first memristors coupled between the respective bit line of the crossbar array and the word line.

13. The method of claim 11, further comprising:

generating an input pulse from the input data; and

applying the voltage on the word line of the crossbar array according to the input pulse.

14. The method of claim 11, wherein the crossbar array is a first crossbar array; and the method further comprises:

the digital data representing the output pulse is delivered as an input to a second crossbar array of third memristors.

15. An apparatus, comprising:

an integrated circuit die;

a digital communication network disposed on the integrated circuit die;

a plurality of tiles configured on the integrated circuit die, each of the tiles comprising:

a plurality of word lines;

a plurality of bit lines;

first memristors each coupled between a respective word line of the plurality of word lines and a respective bit line of the plurality of bit lines to convert a voltage applied on the respective word line into a portion of a current collected in the respective bit line;

second memristors respectively connected to the bit lines, the second memristors configured to control pulses in the electrical currents in the bit lines, wherein the second memristors respectively have thresholds, and wherein when an electrical current flowing through each respective memristor in the second memristors reaches the threshold of the respective memristor, the respective memristor reduces a resistance of the respective memristor to cause a pulse in the electrical current;

current level detectors respectively connected to the second memristors and configured to determine whether the current in the bit lines has a level corresponding to reaching a threshold of the second memristor, and to generate an output pulse of a pulsing neuron based on determining whether the current in the bit lines has a level corresponding to reaching a threshold of the second memristor; and

routing logic coupled to the digital communication network to receive input data specifying a voltage to be applied on the word line and to provide output data representing the output pulse.

16. The device of claim 15, wherein each of the tiles is free of an analog-to-digital converter configured to measure current in the bit line.

17. The apparatus of claim 16, in which the threshold of the second memristor is programmed according to an activation level threshold of a pulsing neuron; and programming a resistance of the first memristor according to a pulsing neuron.

18. The apparatus of claim 17, wherein the output data identifies a plurality of pulse trains, each of the pulse trains identifying a presence or absence of a pulse in a bit line at a plurality of times.

19. The device of claim 17, further comprising:

a pulse encoder configured to generate pulse trains from input data, wherein each of the pulse trains identifies the presence or absence of a pulse at a plurality of time instances.

20. The device of claim 17, further comprising:

a router, disposed on the integrated circuit die, for routing pulse trains between the tiles through the digital communication network.

Technical Field

At least some embodiments disclosed herein relate generally to artificial neural networks, and more particularly, but not by way of limitation, memristor crossbar array implementations of pulsed neural networks.

Background

Generally, an Artificial Neural Network (ANN) uses an artificial neural network to process inputs to the network and generate outputs from the network.

For example, each neuron in the network receives a set of inputs. Some inputs to the neurons may be the outputs of some of the neurons in the network; and some of the inputs to the neurons may be inputs provided to a neural network. The input/output relationships between neurons in the network represent the neuron connectivity in the network.

For example, each neuron may have a bias, an activation function, and a set of synaptic weights to its input, respectively. The activation function may be in the form of a step function, a linear function, a logarithmic sigmoid function, or the like. Different neurons in the network may have different activation functions.

For example, each neuron may generate a weighted sum of its inputs and its bias, and then produce an output of a function of the weighted sum, calculated using the neuron's activation function.

The relationship between the inputs and outputs of an ANN is typically defined by an ANN model that includes data representing the connectivity of neurons in the network, as well as the bias, activation function, and synaptic weights for each neuron. Using a given ANN model, the computing device computes the output of the network from a given set of inputs to the network.

For example, input to the ANN network may be generated based on camera input; and the output from the ANN network may be an identification of an item such as an event or object.

The Spiking Neural Network (SNN) is a type of ANN that closely resembles a natural neural network. When the level of activation of a neuron is high enough, the SNN neuron generates a pulse as an output. The level of activation of SNN neurons mimics the membrane potential of natural neurons. The output/pulse of an SNN neuron may alter the activation level of other neurons receiving the output. The current activation level of the SNN neuron as a function of time can be modeled using differential equations and taking into account the state of the SNN neuron. Incoming pulses from other neurons may push the activation level of the neuron higher to reach the threshold of the pulse. Once a neuron is pulsed, its activation level is reset. Prior to forming the pulse, the activation level of the SNN neuron may decay over time, which is governed by a differential equation. The temporal elements in the behavior of SNN neurons make SNNs suitable for processing spatiotemporal data. The connectivity of SNNs is typically sparse, which is beneficial to reduce the computational effort.

In general, an ANN may be trained using a supervised approach, where parameters in the ANN are adjusted to minimize or reduce errors between known outputs produced by respective inputs and computed outputs generated by applying the inputs to the ANN. Examples of supervised learning/training methods include reinforcement learning and error correction learning.

Alternatively or in combination, the ANN may be trained using an unsupervised approach, wherein the exact output produced by a given set of inputs is unknown until training is complete. The ANN may be trained to classify items into multiple categories, or to classify data points into clusters.

A variety of training algorithms may be employed for complex machine learning/training paradigms.

Disclosure of Invention

In one aspect, the present application provides an integrated circuit comprising: a crossbar array having first memristors of word lines and bit lines, the first memristors configured to convert a voltage applied on a word line into a current in a bit line; second memristors respectively connected to the bit lines, the second memristors configured to control pulses in electrical currents in the bit lines, wherein the second memristors respectively have thresholds, and wherein when an electrical current flowing through each respective memristor of the second memristors reaches the threshold of the respective memristor, the respective memristor reduces the resistance of the respective memristor to cause a pulse in the electrical current; and a current level detector connected to the second memristor and configured to determine whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor, and to generate an output pulse of the pulsing neuron based on determining whether the current in the bit line has a level corresponding to reaching the threshold of the second memristor.

In another aspect, the present application also provides a method comprising: converting, by a crossbar array of first memristors, a plurality of voltages applied on word lines of the crossbar array into currents in bit lines of the crossbar array; controlling a pulse in a current in the bit lines by second memristors respectively connected to the bit lines, wherein the second memristors respectively have thresholds, and wherein when a current flowing through each respective memristor in the second memristors reaches the threshold of the respective memristor, the respective memristor reduces the resistance of the respective memristor to cause the pulse in the current; determining, by a current level detector connected to the second memristor, whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor; and generating an output pulse of the pulsing neuron based on determining whether the current in the bit line has a level corresponding to reaching a threshold of the second memristor.

In yet another aspect, the present application also provides an apparatus comprising: an integrated circuit die; a digital communication network disposed on the integrated circuit die; a plurality of tiles configured on an integrated circuit die, each of the tiles comprising: a plurality of word lines; a plurality of bit lines; first memristors each coupled between a respective word line of a plurality of word lines and a respective bit line of a plurality of bit lines to convert a voltage applied on the respective word line into a portion of a current collected in the respective bit line; second memristors respectively connected to the bit lines, the second memristors configured to control pulses in electrical currents in the bit lines, wherein the second memristors respectively have thresholds, and wherein when the electrical current flowing through each respective memristor of the second memristors reaches the threshold of the respective memristor, the respective memristor reduces the resistance of the respective memristor to cause the pulses in the electrical current; current level detectors respectively connected to the second memristors and configured to determine whether the current in the bit lines has a level corresponding to reaching a threshold of the second memristors, and to generate output pulses of the pulsing neurons based on determining whether the current in the bit lines has a level corresponding to reaching the threshold of the second memristors; and routing logic coupled to the digital communication network for receiving input data specifying a voltage to be applied on the word line and providing output data representing the output pulse.

Drawings

Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.

FIG. 1 shows a tile of a spiking neural network implemented using memristors, according to one embodiment.

FIG. 2 illustrates detecting pulses in a pulsed neural network implemented using memristors using current pulses through a memristor array, according to one embodiment.

Figure 3 illustrates an impulse neural network implemented using a network of impulse neural network tiles, according to one embodiment.

FIG. 4 illustrates processing a pulsed neural network by detecting pulses using memristors and routing the detected pulses between memristor tiles.

FIG. 5 illustrates a method of implementing a spiking neural network, according to one embodiment.

Detailed Description

At least some embodiments disclosed herein provide systems, methods, and apparatus to implement a pulsed neural network using one or more memristor crossbar arrays without using an analog-to-digital converter to detect pulses.

Resistive random access memory (ReRAM or RRAM) operates by changing the resistance across a dielectric solid state material, which may be referred to as a memristor. Generally, a memristor is a two-terminal electrical component with a non-constant resistance. Its resistance may vary based on the history of the current flowing through the memristor and/or the history of the voltage applied across the memristor. When the power supply of the memristor is turned off, the memristor may remember its most recent resistance. Thus, memristors may be used to implement non-volatile memories.

At least some embodiments disclosed herein provide efficient implementations for multiply-accumulate (MAC) operations of a Spiking Neural Network (SNN). A crossbar array of memristors may be configured to perform a multiply-accumulate (MAC) operation through analog circuitry. Currents through a set of memristors in the crossbar array, through the word lines, to the bit lines are summed in the bit lines, which corresponds to an accumulation operation. The current corresponds to the product of the voltage applied on the word line and a parameter related to the resistance of the memristor, which corresponds to a product operation. The current in the bit line may be compared to a pulse threshold to determine whether the activation level of the neuron represented by the bit line has reached a form pulse level. The memristor arrays may be respectively connected to bit lines and programmed to have thresholds corresponding to activation level thresholds of the pulsing neurons. The current detector may be configured as each memristor for connection to an output of a bit line to determine whether a level of a current in the bit line corresponds to a level exceeding a threshold of the memristor. The detection result of the current detector may represent an impulse response in the output of the impulse neuron represented by the bit line. The impulse response may be routed to another portion of the SNN or to the output of the SNN.

Alternatively, an analog-to-digital converter (ADC) may be used to measure the current in the bit line and convert it to digital data for pulse detection in a comparator. However, ADCs consume more space and energy than memristors and current detectors. Thus, using memristor arrays and current detectors in detecting pulses to eliminate the need for ADCs can improve the spatial and energy efficiency of SNN implementations.

FIG. 1 shows a tile of a spiking neural network implemented using memristors, according to one embodiment.

In fig. 1, a memristor crossbar array 101 receives an input voltage pulse 103 to generate a current through a memristor array 105, which may function as a switch array. Each memristor in the array 105 is programmed to have a threshold for pulse detection. When a current flowing through the memristor is below a threshold, the memristor has substantially the same resistance; the switch implemented by the memristor is considered to be in an off state. However, when the current flowing through the memristor reaches its threshold, the resistance of the memristor drops (e.g., approaches zero), causing the current flowing through the memristor to increase significantly and form a pulse; and in such cases, the switch implemented by the memristor is considered to be in a conducting state. Once the switch implemented by the memristor is in the on-state, the current flowing through the memristor may increase to at least a predetermined level that is independent of the threshold of the memristor and higher than the current that may flow through the memristor in the array 101 before the threshold of the memristor in the array 101 is reached.

The current level detector array 107 may be configured to detect whether a current flowing through a memristor in the array 105 has reached a level corresponding to an on-state of the memristor in the array 105. The detection may be performed based on detecting whether at least a predetermined level of current flows through the respective memristors in the array 105. Thus, the current detection operation of the current level detector 107 is independent of the threshold of the memristors in the array 105. Based on whether there is at least a predetermined level of current in the memristors in the array, the respective detectors in the current level detector array 107 generate digital signals that indicate whether current pulses of the memristors in the array 105 are detected. The set of outputs from the current level detector 107 provides an output pulse 109.

FIG. 2 illustrates detecting pulses in a pulsed neural network implemented using memristors using current pulses through a memristor array, according to one embodiment. For example, the crossbar array 101 and memristor array 105 of fig. 1 may be implemented in the manner shown in fig. 2.

In fig. 2, each memristor in crossbar array 101 is connected between a word line (e.g., 131) and a bit line (e.g., 141). The word line 131, …, 133, 135, …, 137 is configured to receive an input voltage; the bit lines 141, 143, …, 145 are configured to provide an output current; and in accordance with the activation level threshold of the pulsing neuron, the memristor array 105 is configured to generate output current pulses corresponding to the pulses of the neurons represented by the bit lines 141, 143, …, 145.

For example, when a voltage is applied on the word line 131, the voltage generates a current that flows to the bit lines 141, 143, …, 145 through the memristors 111, 121, …, respectively. The contribution to the current by the voltage applied to word line 131 in bit lines 141, 143, …, 145 is proportional to the weight and response of the neuron to the input represented by the voltage applied to word line 131. The weight and response of the neuron may be achieved by programming the resistance of the resistors 111, 121, …, respectively.

The bit lines 141, 143, …, 145 sum the currents contributed by the voltages applied to the word lines 131, …, 133, 135, …, 137 to the bit lines 141, 143, …, 145. Thus, the current in the bit lines 141, 143, …, 145 corresponds to the sum of the product of the weight and the response of the neuron, which is achieved by the programmed resistance of the crossbar array 101 with the voltage of the word line 131, …, 133, 135, …, 137 representing the neuron input.

For example, the contributions to the bit line 141 from the voltages on the word lines 131, …, 133, 135, …, 137 are summed by the currents flowing from the word lines 131, …, 133, 135, …, 137 through the memristors 111, …, 113, 115, …, 117 to the bit line 141; and the contributions to the bit line 143 from the voltages on the word lines 131, …, 133, 135, …, 137 are summed by the current flowing from the word lines 131, …, 133, 135, …, 137 through the memristors 121, …, 123, 125, …, 127 to the bit line 143; and the like.

Thus, crossbar array 101 performs a multiply-accumulate (MAC) operation by converting the voltages on word lines 131, …, 133, 135, …, 137 into currents on bit lines 141, 143, …, 145.

In fig. 2, rather than using ADCs to measure the current flowing through the bit lines 141, 143, …, 145, the current pulses are generated using the memristor array 105 based on the thresholds of the memristors 119, 129, … connected to the bit lines 141, 143, …, respectively.

For example, the memristor 119 is connected to the bit line 141 and is programmed to have a threshold value corresponding to a threshold value of the activation level of the pulsing neuron. When the current on the bit line 141 is less than the current threshold, the memristor 119 has a first resistance corresponding to an off state; and when the current on the bit line 141 has reached the current threshold, the memristor 119 has a second resistance corresponding to the off-state, which is significantly lower than the first resistance.

The current level detector 151 is configured to detect whether the current on the bit line 141 is at a level corresponding to the on state of the memristor 119, which indicates whether a pulsing neuron corresponding to the bit line 141 produces a pulse.

Thus, the outputs of the current level detectors 151, 153, …, 155 correspond to the output pulses 109 of the bit lines 141, 143, …, 145.

For example, a pulse train may be applied on word line 131. The pulse train is the timing of the voltage applied to the word line 131. The voltages in the pulse train may be arranged at fixed time intervals. The voltage in the pulse train may be selected from two predetermined voltages. Wherein one voltage corresponds to the presence of a pulse at one instant in the pulse train; and the other corresponds to the absence of a pulse at one instant in the pulse train.

For example, a data item containing several bits may be converted into a pulse train at the same number of times. The voltage applied at each instant is based on the value of the corresponding bit in the data item.

When a set of pulse trains is applied to the set of word lines 131, …, 133, 135, …, 137, respectively, the outputs of the current level detectors 151, 153, …, 155 generate a set of output pulse trains that may be further processed by another tile of the pulsed neural network.

The resistance values of the memristors in the crossbar array 101 may be set to model the behavior of the pulsed neural network. By selectively applying current and/or voltage to program a memristor, the resistance value of the memristor in the crossbar array 101 may be changed. For example, after a memristor (e.g., 111) is selected for programming, a voltage applied across the memristor (e.g., 111) may be raised to place the memristor (e.g., 111) in an on-state; a programming pulse may then be applied across the memristor (e.g., 111) to adjust the resistance of the memristor (e.g., 111) to a desired value. After programming, the memristor (e.g., 111) remembers its programming resistance during its off-state until the memristor (e.g., 111) is again selected for programming in the on-state. Separate circuitry may be used to select a memristor (e.g., 111) and program the resistance of the memristor (e.g., 111).

Figure 3 illustrates an impulse neural network implemented using a network of impulse neural network tiles, according to one embodiment.

In fig. 3, a plurality of SNN tiles 161, 163, 165, … are configured on an integrated circuit die. Each SNN tile (e.g., 161, 163, or 165) may be implemented in the manner shown in fig. 1 and/or fig. 2. Each SNN tile (e.g., 161, 163, or 165) has associated routing logic (e.g., 162, 164, or 166) configured to provide input pulse data (e.g., 103) from communication network 173 to the SNN tile and transmit its output pulse data (e.g., 109) to other SNN tiles via communication network 173.

For example, each SNN tile (e.g., 161, 163, or 165) and router 171 may have a unique address for data transfer over communication network 173. An input pulse train address to an SNN tile (e.g., 161) is received by routing logic (e.g., 162) of the SNN tile, thereby generating respective voltages for word lines (e.g., 131, …, 133, 135, …, 137) of the tile (e.g., 101), causing the tile (e.g., 101) to generate output pulses 109 through current level detectors (e.g., 151, 153, …, 155) connected to bit lines (e.g., 141, 143, …, 145) of the SNN tile (e.g., 101). Routing logic (e.g., 162) then delivers the output pulse to router 171. Router 171 is configured to route the output pulse to the next tile (e.g., 163, and/or 165) according to neural connectivity in the spiking neural network.

On the integrated circuit die (e.g., 161, 163, and/or 165), router 171 is configured (e.g., according to a routing table configured according to neural connectivity in a spiking neural network) to route input/output pulses to and from the SNN tiles. Thus, the SNN tiles (e.g., 161, 163, or 165) on the integrated circuit die may be reconfigured by router 171 to model different spiking neural networks of different neuron connectivity configurations.

FIG. 4 illustrates processing a pulsed neural network by detecting pulses using memristors and routing the detected pulses between memristor tiles.

In fig. 4, input data 181 to the impulse neural network is converted by an impulse encoder 183 to generate impulses 185 as input to one or more SNN tiles 187. Each SNN tile 187 may be implemented in the manner shown in fig. 1 or 2, where an output pulse in an electron current in a bit line (e.g., 141, 143, …, 145) is detected by the memristor array 105 having a threshold configured according to a threshold of an activation level of a pulsing neuron. The current level detectors 151, 153, …, 155 generate digital data of the output pulses 109, which can be supplied as input pulses 189 to further SNN tiles 191. Additional layers of SNN tiles may be added in a manner similar to the use of the second layer of tiles 191.

The tiles used in fig. 4 (e.g., 187 and 191) may be tiles 161, 163, 165, … configured on an integrated circuit die. Certain tiles 161, 163, 165, … on the integrated circuit die may be reconfigured to model different portions of the spiking neurons in different time networks.

For example, by programming the resistance of memristors (e.g., 111-117, 121-127, …) in crossbar array 101 of each tile (e.g., 101), and by programming the threshold of memristors (e.g., 119, 129, …) for pulse detection, first layer SNN tile 187 may be initially configured on tiles 161, 163, 165. Some of the tiles 161, 163, 165 may then be reprogrammed into SNN tiles 191 for second layer processing.

FIG. 5 illustrates a method of implementing a spiking neural network, according to one embodiment. For example, the method of fig. 5 may be implemented in the integrated circuit shown in fig. 1, 2, and/or 3 in conjunction with the process flow of fig. 4.

At block 201, the crossbar array 101 of first memristors (e.g., 111-117, 121-127) may convert a plurality of voltages applied on word lines (e.g., 131, …, 133, 135, …, 137) of the crossbar array 101 into currents on bit lines (e.g., 141, 143, …, 145) of the crossbar array 101.

At block 203, second memristors (e.g., 119, 128, …) having thresholds and connected to bit lines (e.g., 119, 129, …), respectively, control pulses in currents in the bit lines (e.g., 119, 129, …).

For example, once the current flowing through each respective memristor of the second memristors (e.g., 119, 129, …) increases to reach the threshold of the respective memristor, the respective memristor may substantially reduce its resistance to cause a pulse in the current of the respective memristor.

At block 205, a current level detector (e.g., 151, 153, …, 155) connected to the second memristor (e.g., 119, 129, …) determines whether a respective current in the bit line (e.g., 141, 143, …, 145) has a level corresponding to reaching a respective threshold of the second memristor (e.g., 119, 129, …).

For example, the current in the memristor 119 reaches its threshold, the resistance of the memristor 119 decreases significantly to enter the on-state, causing the current to increase and form a pulse. Current level detector 151 detects the added and formed pulses and generates a digital signal representative of the pulses.

At block 207, a current level detector (e.g., 151, 153, …, 155) generates an output pulse 109 of the pulsing neuron based on determining whether the current in the bit line (e.g., 141, 143, …, 145) has a level corresponding to reaching a threshold of a second memristor (e.g., 119, 129, …).

Since the digital signal of the output pulse 109 is generated based on detecting the predetermined current level corresponding to reaching the threshold of the second memristor (e.g., 119, 129, …), data representative of the output pulse 109 may be generated without using an analog-to-digital converter to measure the current in the bit line (e.g., 141, 143, …, 145).

For example, the threshold of the second memristor (e.g., 119, 129, …) may be programmed according to an activation level threshold of a pulsing neuron implemented using the first memristor (e.g., 111-117, 121-127, …) in the crossbar array. A resistance of a first memristor (e.g., 111-117, 121-127, …) from a pulsing neuron implemented using the first memristor (e.g., 111-117, 121-127, …). The voltage on the word line (e.g., 131, …, 133, 135, …, 137) corresponds to the input of the pulsing neuron and the current on the bit line (e.g., 141, 143, …, 145) corresponds to the activation level of the pulsing neuron in response to the input. The inverse of the resistance of each first memristor (e.g., 111) corresponds to a response coefficient of the pulsing neuron to an input represented by a voltage applied on a word line (e.g., 131); also, the bit line (e.g., 141) sums the currents from the memristors (e.g., 111, …, 113, 135, …, 117) and thus sums the responses of the pulsing neurons to the different inputs (e.g., 131, …, 133, 135, …, 137) connected to the pulsing neurons to provide the activation levels of the pulsing neurons.

Pulse encoder 183 may be used to generate input pulses 103 or 185 from input data 181. The voltage on the word lines (e.g., 131, …, 133, 135, …, 137) of crossbar array 101 may be applied in accordance with input pulse 103.

For example, the input of a word line (e.g., 131) may be in the form of a pulse train specifying the presence or absence of pulses at various times. The plurality of time instants may be separated by a predetermined fixed time interval. At each instant, the pulse train indicates whether a pulse is present. If a pulse is present, a first voltage (e.g., a positive high voltage) is applied on the word line (e.g., 131); if no pulse is present, a second voltage (e.g., ground or a positive low voltage) is applied on the word line (e.g., 131).

Similarly, output data from the tile may identify a plurality of pulse trains generated by the pulsed neurons implemented by the tile. Each output pulse train identifies whether a pulse is present in the bit line (e.g., 141, 143, …, 145) at multiple times.

A digital communication network and a plurality of tiles may be configured on an integrated circuit die. Each tile may be used to implement a set of pulsed neurons corresponding to bit lines (e.g., 141, 143, …, 145) in the tile. Each tile may have a crossbar array of memristors and routing logic (e.g., 162, 164, or 166) to receive input data specifying a voltage to be applied to one word line (e.g., 131, …, 133, 135, …, 137) and to provide output data representing output pulses 109 corresponding to the pulse state of current in a bit line (e.g., 141, 143, …, 145).

A router 171 may be configured on the integrated circuit die to route pulse trains between tiles through a digital communication network 173.

The present disclosure encompasses methods and apparatus for performing the above-described methods, including data processing systems that perform these methods, and computer-readable media containing instructions that, when executed on a data processing system, cause the system to perform these methods.

An integrated circuit implementing a Spiking Neural Network (SNN) may be used in a data processing system.

A typical data processing system may contain interconnects (e.g., buses and system core logic) that interconnect a microprocessor and a memory. The microprocessor is typically coupled to a cache memory.

An interconnect interconnects the microprocessor and the memory together and also interconnects them to an input/output (I/O) device through an I/O controller. The I/O devices may include display devices and/or peripheral devices such as mice, keyboards, modems, network interfaces, printers, scanners, cameras, and other devices known in the art. In one embodiment, when the data processing system is a server system, some I/O devices such as printers, scanners, mice, and/or keyboards are optional.

The interconnect may include one or more buses connected to each other through various bridges, controllers, and/or adapters. In one embodiment, the I/O controller includes a USB (Universal Serial bus) adapter for controlling USB peripheral devices, and/or an IEEE-1394 bus adapter for controlling IEEE-1394 peripheral devices.

The memory may include one or more of the following: ROM (read only memory), volatile RAM (random access memory) and non-volatile memory, such as a hard disk drive, flash memory, etc.

Volatile RAM is typically implemented as dynamic RAM (dram), which requires power continually in order to refresh or maintain the data in the memory. The non-volatile memory is typically a magnetic hard drive, a magnetic optical drive, an optical drive (e.g., DVD RAM), or other type of memory system that maintains data even after power is removed from the system. The non-volatile memory may also be a random access memory.

The non-volatile memory may be a local device coupled directly to the rest of the components in the data processing system. Non-volatile storage remote from the system, such as a network storage device coupled to the data processing system through a network interface such as a modem or Ethernet interface, may also be used.

In the present disclosure, some functions and operations are described as being performed by or caused by software code to simplify description. However, such expressions are also used to specify functions resulting from execution of the code/instructions by a processor, such as a microprocessor.

Alternatively or in combination, the functions and operations described herein may be implemented using special purpose circuits, with or without software instructions, such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA). Embodiments may be implemented using hardwired circuitry in the absence of or in combination with software instructions. Thus, the techniques are not limited to any specific combination of hardware circuitry and software nor to any particular source for the instructions executed by the data processing system.

While one embodiment may be implemented in fully functioning computers and computer systems, the various embodiments are capable of being distributed as a computing article of manufacture in a variety of forms, and of being applied regardless of the particular type of machine or computer readable media used to actually effect the distribution.

At least some aspects of the disclosure may be presented, at least in part, in software. That is, the techniques may be performed in a computer system or other data processing system that is responsive to its processor, such as a microprocessor, that executes sequences of instructions contained in a memory (e.g., ROM, volatile RAM, non-volatile memory, cache, or remote storage device).

The routines executed to implement the embodiments, may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as a "computer program". The computer programs generally include one or more instructions that are set at various times in various memories and storage devices in the computer, and when read and executed by one or more processors in the computer, cause the computer to perform the operations necessary to execute elements relating to the various aspects.

A machine-readable medium may be used to store software and data which, when executed by a data processing system, cause the system to perform various methods. Executable software and data may be stored in various locations including, for example, ROM, volatile RAM, non-volatile memory, and/or cache. Portions of this software and/or data may be stored in any of these storage devices. Further, the data and instructions may be obtained from a centralized server or a peer-to-peer network. Different portions of the data and instructions may be obtained from different centralized servers and/or peer-to-peer networks at different times and in different communication sessions or the same communication session. The data and instructions may all be obtained prior to execution of the application. Alternatively, portions of the data and instructions may be dynamically obtained in time when execution is required. Thus, it is not required that the data and instructions be entirely on machine-readable media at a particular time.

Examples of computer readable media include, but are not limited to, non-transitory, recordable and non-recordable type media (e.g., volatile and non-volatile memory devices, Read Only Memory (ROM), Random Access Memory (RAM), flash memory devices, floppy and other removable disks), magnetic disk storage media, optical storage media (e.g., compact disk read only memory (CD ROM), Digital Versatile Disks (DVDs), etc.), among others. A computer readable medium may store instructions.

The instructions may also be present in digital and analog communications links for electrical, optical, acoustical or other forms of propagated signals, e.g., carrier waves, infrared signals, digital signals, etc., however, propagated signals, e.g., carrier waves, infrared signals, digital signals, etc., are not a tangible, machine-readable medium and are not configured to store the instructions.

Generally, a machine-readable medium includes any mechanism that provides (i.e., stores and/or transmits) information in a form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.).

In various embodiments, hardwired circuitry may be used in combination with software instructions to implement the techniques. Thus, the techniques are not limited to any specific combination of hardware circuitry and software nor to any particular source for the instructions executed by the data processing system.

The above description and drawings are illustrative and not restrictive. Numerous specific details are described to provide a thorough understanding. However, in certain instances, well-known or conventional details are not described in order to avoid obscuring the description. References to one embodiment or an embodiment in this disclosure are not necessarily references to the same embodiment; and, such references mean at least one.

In the foregoing specification, the disclosure has been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.