阅读说明：本技术 光电处理系统 (Photoelectric processing system ) 是由 J.邓 A.霍塞因扎德 Y.徐 Y.白 孟怀宇 R.加格农 卢正观 J.特里 M.斯 于 2021-04-29 设计创作，主要内容包括：一种光电处理系统,包括：至少一个输入光学波导,被配置以接收光波；至少一个数字输入端口,被配置以接收一系列数字输入值,每个数字输入值包括两个或更多个比特；以及光学调制器,耦合至所述输入光学波导。所述光学调制器包括光学波导部分,所述光学波导部分包括多个光学波导片段,所述光学波导片段与沿着所述光学片段定位的二极管区段相关联,其中所述二极管区段被配置为对通过所述光学波导片段传播的光波施加不同的相应调制贡献。(An electro-optical processing system, comprising: at least one input optical waveguide configured to receive a light wave; at least one digital input port configured to receive a series of digital input values, each digital input value comprising two or more bits; and an optical modulator coupled to the input optical waveguide. The optical modulator includes an optical waveguide portion including a plurality of optical waveguide segments associated with diode segments positioned along the optical segments, wherein the diode segments are configured to impose different respective modulation contributions to light waves propagating through the optical waveguide segments.)

1. An electro-optical processing system, comprising:

at least one input optical waveguide configured to receive a light wave;

At least one digital input port configured to receive a series of digital input values in successive time intervals, each digital input value comprising two or more bits; and

an optical modulator coupled to the input optical waveguide, the optical modulator comprising:

an optical waveguide portion comprising a plurality of optical waveguide segments associated with a plurality of diode segments positioned along corresponding optical segments, the optical waveguide segments being part of a continuous optical waveguide, wherein the diode segments are configured to impose different respective modulation contributions to light waves propagating through the optical waveguide segments, and each respective diode segment comprises: a semiconductor diode associated with an optical path length of less than about 1 millimeter, and an electrical contact for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of an optical waveguide segment associated with the diode segment is modulated in response to a value of a corresponding bit in the digital input values, and

a signal conditioning circuit configured to shape changes in the amplitude of the electrical signal applied to at least one of the semiconductor diodes in association with corresponding changes between successive ones of the series of digital input values.

2. The electro-optical processing system of claim 1, wherein the signal conditioning circuit is configured to shape the amplitude change by increasing a magnitude of the amplitude change between a first electrical signal level associated with a first time interval and a second electrical signal level associated with the second time interval for a beginning portion of a second time interval.

3. The electro-optical processing system of claim 2, wherein the signal conditioning circuit is configured to shape the amplitude change by reducing a magnitude of the amplitude change between the first electrical signal level and the second electrical signal level for a final portion of the second time interval.

4. The electro-optical processing system of claim 1, wherein the signal conditioning circuit is configured to shape the amplitude change by applying an electrical signal to the semiconductor diode through a matching circuit configured to match an impedance associated with the semiconductor diode without significantly changing an amplitude of the applied electrical signal.

5. The electro-optical processing system of claim 4, wherein the matching circuit comprises a passive circuit.

6. The electro-optical processing system of claim 5, wherein the matching circuit consists essentially of an inductor.

7. The electro-optical processing system of claim 1, wherein the signal conditioning circuit is configured to shape the amplitude change by applying an electrical signal to the semiconductor diode by a circuit configured to pump a current between the semiconductor diode and a capacitor connected in series between the semiconductor diode and a circuit providing the series of digital input values, wherein an amount of charge transferred by the pumped current is determined based at least in part on a voltage that is constant over a plurality of consecutive time intervals providing the series of digital input values.

8. The electro-optical processing system of claim 1, wherein the optical modulator comprises an interferometric optical modulator further comprising an optical interference portion configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on a cumulative modulation contribution of the optical waveguide segments associated with the diode segments.

9. The electro-optical processing system of claim 8, wherein the optical interference section comprises an optical combiner.

10. The electro-optical processing system of claim 9, wherein the optical waveguide section comprises: at least two optical waveguide arms, each optical waveguide arm: receiving an optical wave dropped from the same optical splitter coupled to the input optical waveguide, and providing an optical wave to the optical combiner.

11. The electro-optical processing system of claim 10, wherein the optical characteristic of the optical waveguide segment associated with the diode segment comprises an effective index of refraction of the optical waveguide segment, and the different respective modulation contributions comprise different respective phase shifts.

12. The electro-optical processing system of claim 11, wherein an optical path length of a first waveguide segment associated with the semiconductor diode of the first diode section is about twice an optical path length of a second waveguide segment associated with the semiconductor diode of the second diode section.

13. The electro-optical processing system of claim 12, wherein the signal conditioning circuitry is configured to shape the amplitude change by: applying a first predetermined shape of electrical signal amplitude to the electrical contacts of the first diode segment and the electrical contacts of the second diode segment in response to a change in the value of the corresponding bit from 0 to 1, and applying a second predetermined shape of electrical signal amplitude to the electrical contacts of the first diode segment and the electrical contacts of the second diode segment in response to a change in the value of the corresponding bit from 1 to 0.

14. The electro-optical processing system of claim 12, wherein the signal conditioning circuitry is configured to shape the amplitude change by: applying an electrical signal to an electrical contact of the first diode section through a first matching circuit configured to match an impedance associated with the first semiconductor diode, and applying an electrical signal to an electrical contact of the second diode section through a second matching circuit configured to match an impedance associated with the second semiconductor diode.

15. The electro-optical processing system of claim 14, wherein the first and second matching circuits each consist essentially of an inductor.

16. The electro-optical processing system of claim 10, wherein the input optical waveguide is coupled to an optical demultiplexer that separates light waves of at least two different wavelengths.

17. The electro-optical processing system of claim 1, wherein the optical modulator comprises an absorbing optical modulator configured to provide a degree of absorbance that results in a predetermined amplitude reduction based on a cumulative modulation contribution of the optical waveguide segments associated with the diode segments.

18. The electro-optical processing system of claim 17, wherein the optical characteristic of the optical waveguide segment associated with the diode segment comprises an absorption coefficient of the optical waveguide segment, and the different respective modulation contributions comprise different respective absorbances.

19. The electro-optical processing system of claim 1, wherein the at least one input optical waveguide comprises a plurality of input optical waveguides, a plurality of optical modulators are each coupled with a different respective input optical waveguide of the plurality of input optical waveguides, and outputs from the plurality of optical modulators are combined to provide a result of the vector-matrix multiplication.

20. The electro-optical processing system of claim 1, wherein the optical modulator comprises at least one calibration phase shifter configured to compensate for an imbalance in optical phase shifts imparted by two waveguide arms of the optical modulator.

21. The electro-optical processing system of claim 1, comprising an Artificial Neural Network (ANN) computing system, the ANN computing system comprising:

a first unit configured to generate a plurality of modulator control signals comprising a series of digital input values over successive time intervals, each digital input value comprising two or more bits;

a light source configured to provide a plurality of light outputs;

a plurality of optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector by modulating the plurality of light outputs provided by the light source based on the plurality of modulator control signals, the optical input vector comprising a plurality of optical signals;

wherein the plurality of optical modulators includes an optical modulator comprising the optical waveguide portion and the signal conditioning circuit; and

a matrix processing unit coupled to the plurality of optical modulators and the first unit, the matrix processing unit configured to transform the optical input vector into an analog output vector based on a plurality of weight control signals.

22. The electro-optical processing system of claim 21, wherein the ANN computing system comprises:

a second unit coupled to the matrix processing unit and configured to convert the analog output vector into a digitized output vector; and

a controller comprising an integrated circuit configured to perform operations comprising:

receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set, the input data set comprising a first numeric input vector;

receiving a first plurality of neural network weights; and

generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

23. The electro-optical processing system of claim 21, wherein the matrix processing unit comprises:

a plurality of replica modules, wherein each replica module corresponds to a subset of one or more optical signals of the optical input vector and is configured to split the subset of one or more optical signals into two or more replicas of the optical signal;

A plurality of multiplication modules, wherein each multiplication module corresponds to a subset of one or more optical signals and is configured to multiply the one or more optical signals of the subset by one or more matrix element values using optical amplitude modulation; and

one or more summation modules, wherein each summation module is configured to produce an electrical signal representing a sum of results of two or more of the multiplication modules.

24. The electro-optical processing system of any one of claims 1-23, comprising at least one of a personal computer, a server computer, a vehicle computer, or a flight computer, wherein the at least one input optical waveguide, the at least one digital input port, and the optical modulator are part of the personal computer, the server computer, the vehicle computer, or the flight computer.

25. An electro-optical processing system, comprising:

at least one input optical waveguide configured to receive a light wave;

at least one digital input port configured to receive a series of digital input values in successive time intervals, each digital input value comprising two or more bits; and

An optical modulator coupled to the input optical waveguide, the optical modulator comprising:

an optical waveguide portion comprising a plurality of diode segments positioned along the optical waveguide portion, wherein the diode segments impose different respective modulation contributions to light waves propagating through the optical waveguide portion, and each respective diode segment comprises: a semiconductor diode having an optical path length of less than about 1 millimeter, and an electrical contact for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit in the digital input value, and

a signal conditioning circuit configured to shape changes in the amplitude of the electrical signal applied to at least one of the semiconductor diodes in association with corresponding changes between successive ones of the series of digital input values.

26. The photo-processing system of claim 25, wherein shaping the amplitude change of an electrical signal applied to the semiconductor diode in association with successive ones of the series of digital input values comprises: for a beginning portion of a second time interval, a magnitude of a change in amplitude between a first electrical signal level associated with the first time interval and a second electrical signal level associated with the second time interval is increased.

27. The electro-optical processing system of claim 26, wherein shaping the amplitude change of an electrical signal applied to the semiconductor diode in association with successive ones of the series of digital input values further comprises: for a final portion of the second time interval, reducing a magnitude of a change in amplitude between the first electrical signal level and the second electrical signal level.

28. The photo-processing system of claim 25, wherein shaping the amplitude change of an electrical signal applied to the semiconductor diode in association with successive ones of the series of digital input values comprises: applying an electrical signal to the semiconductor diode through a matching circuit configured to match an impedance associated with the semiconductor diode without significantly changing an amplitude of the applied electrical signal.

29. The electro-optical processing system of claim 28, wherein the matching circuit comprises a passive circuit.

30. The photo-processing system of claim 29, wherein the matching circuit consists essentially of an inductor.

31. The photo-processing system of claim 25, the shaping the change in the amplitude of the electrical signal applied to the semiconductor diode for successive ones of the series of digital input values comprising: applying an electrical signal to the semiconductor diode through a circuit configured to pump a current between the semiconductor diode and a capacitor connected in series between the semiconductor diode and the circuit providing the series of digital input values, wherein an amount of charge transferred by the pumped current is determined based at least in part on a voltage that is constant over a plurality of consecutive time intervals providing the series of digital input values.

32. The electro-optical processing system of claim 25, wherein the optical modulator comprises an interferometric optical modulator further comprising an optical interference portion configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on a cumulative modulation contribution of the diode segments.

33. The electro-optical processing system of claim 32, wherein the optical interference section comprises an optical combiner.

34. The electro-optical processing system of claim 33, wherein the optical waveguide section comprises: at least two optical waveguide segments, each optical waveguide segment: receiving an optical wave dropped from the same optical splitter coupled to the input optical waveguide, and providing an optical wave to the optical combiner.

35. The photo-processing system of claim 34, wherein the optical characteristic of the diode section comprises an effective index of refraction of the diode section, and the different respective modulation contributions comprise different respective phase shifts.

36. The electro-optical processing system of claim 35, wherein the optical path length of the first semiconductor diode of the first diode section is about twice the optical path length of the second semiconductor diode of the second diode section.

37. The electro-optical processing system of claim 36, wherein shaping the change in the amplitude of the electrical signal applied to the semiconductor diode for successive ones of the series of digital input values comprises: applying a first predetermined shape of electrical signal amplitude to the electrical contacts of the first diode segment and the electrical contacts of the second diode segment in response to a change in the value of the corresponding bit from 0 to 1, and applying a second predetermined shape of electrical signal amplitude to the electrical contacts of the first diode segment and the electrical contacts of the second diode segment in response to a change in the value of the corresponding bit from 1 to 0.

38. The electro-optical processing system of claim 36, wherein shaping the change in the amplitude of the electrical signal applied to the semiconductor diode for successive ones of the series of digital input values comprises: applying an electrical signal to an electrical contact of the first diode section through a first matching circuit configured to match an impedance associated with the first semiconductor diode, and applying an electrical signal to an electrical contact of the second diode section through a second matching circuit configured to match an impedance associated with the second semiconductor diode.

39. The electro-optical processing system of claim 38, wherein the first and second matching circuits each consist essentially of an inductor.

40. The electro-optical processing system of claim 34, wherein the input optical waveguide is coupled to an optical demultiplexer that separates light waves of at least two different wavelengths.

41. The photo-processing system of claim 25, wherein the optical modulator comprises an absorbing optical modulator configured to provide a degree of absorbance that results in a predetermined amplitude reduction based on a cumulative modulation contribution of the diode segments.

42. The electro-optical processing system of claim 41, wherein the optical characteristic of the diode section comprises an absorption coefficient of the diode section, and the different respective modulation contributions comprise different respective absorbances.

43. The optoelectronic processing system of claim 25, wherein the at least one input optical waveguide comprises a plurality of input optical waveguides, a plurality of optical modulators are each coupled to a different respective input optical waveguide of the plurality of input optical waveguides, and outputs from the plurality of optical modulators are combined to provide a result of the vector-matrix multiplication.

44. An electro-optical processing system, comprising:

at least one input optical waveguide configured to receive a light wave;

at least one digital input port configured to receive a series of digital input values in successive time intervals, each digital input value comprising two or more bits; and

an interferometric optical modulator coupled to the at least one input optical waveguide, the interferometric optical modulator comprising:

an optical waveguide portion comprising a diode section along the optical waveguide portion, wherein the diode section modulates light waves propagating through the optical waveguide portion, and the diode section comprises: a semiconductor diode having an optical path length of less than about 1 millimeter, and an electrical contact for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of the diode section is modulated in response to the digital input value,

a signal conditioning circuit configured to shape changes in the amplitude of an electrical signal applied to the semiconductor diode in association with corresponding changes between successive ones of the series of digital input values, wherein the signal conditioning circuit comprises: a first signal conditioning path providing an unconditioned electrical signal corresponding to said series of digital input values, a second signal conditioning path providing a delayed, scaled and/or inverted version of said unconditioned electrical signal, and a third signal conditioning path providing a delayed, scaled and/or inverted version of said unconditioned electrical signal, and

An optical interference section configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on modulation of an optical characteristic of the diode segment.

45. The electro-optical processing system of claim 44, wherein the optical interference section comprises a coupling section of each optical waveguide of a pair of optical waveguides, wherein the coupling sections of each optical waveguide are in proximity to each other.

46. The electro-optical processing system of claim 45, wherein the optical waveguide section comprises a first optical waveguide of the pair of optical waveguides formed in a closed path.

47. The electro-optical processing system of claim 44, comprising an Artificial Neural Network (ANN) computing system, the ANN computing system comprising:

a first unit configured to generate a plurality of modulator control signals comprising a series of digital input values in successive time intervals, each digital input value comprising two or more bits;

a light source configured to provide a plurality of light outputs;

a plurality of interferometric optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector by modulating the plurality of optical outputs provided by the light source based on the plurality of modulator control signals, the optical input vector comprising a plurality of optical signals;

Wherein the plurality of optical modulators includes an optical modulator comprising the optical waveguide portion and the signal conditioning circuit; and

a matrix processing unit coupled to the plurality of optical modulators and the first unit, the matrix processing unit configured to transform the optical input vector into an analog output vector based on a plurality of weight control signals.

48. The photo-electric processing system of claim 47, wherein the ANN computing system comprises:

a second unit coupled to the matrix processing unit and configured to convert the analog output vector into a digitized output vector; and

a controller comprising an integrated circuit configured to perform operations comprising:

receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set, the input data set comprising a first numeric input vector;

receiving a first plurality of neural network weights; and

generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

49. The photo-processing system of claim 47, wherein the matrix processing unit comprises:

a plurality of replica modules, wherein each replica module corresponds to a subset of one or more optical signals of the optical input vector and is configured to split the subset of one or more optical signals into two or more replicas of the optical signal;

a plurality of multiplication modules, wherein each multiplication module corresponds to a subset of one or more optical signals and is configured to multiply the one or more optical signals of the subset by one or more matrix element values using optical amplitude modulation; and

one or more summation modules, wherein each summation module is configured to produce an electrical signal representing a sum of results of two or more of the multiplication modules.

50. The electro-optical processing system of any one of claims 44-49, comprising at least one of a personal computer, a server computer, a vehicle computer, or a flight computer, wherein the at least one input optical waveguide, the at least one digital input port, and the optical modulator are part of the personal computer, the server computer, the vehicle computer, or the flight computer.

51. An electro-optical processing system, comprising:

at least one input optical waveguide configured to receive a light wave;

at least one input port configured to receive a series of modulator control signals; and

an optical modulator coupled to the input optical waveguide, configured to provide pulse amplitude modulation having four or more amplitude levels, the optical modulator comprising a first modulator arm and a second modulator arm, at least one of the first modulator arm or the second modulator arm comprising an optical waveguide and a plurality of phase shifters positioned along the optical waveguide, the phase shifters configured to impose different respective modulation contributions to light waves propagating through the optical waveguide, each phase shifter coupled to a respective signal conditioning circuit configured to provide enhanced bandwidth for binary modulation, and different phase shifters coupled to different signal conditioning circuits;

wherein each phase shifter comprises a semiconductor diode or capacitor associated with an optical path length of less than about 1 millimeter; and

a control circuit configured to perform at least one of the following for each phase shifter: (i) providing an electrical signal to the semiconductor diode such that the semiconductor diode is in a forward-biased state in which an optical characteristic of an optical waveguide associated with the phase shifter is modulated in response to the modulator control signal, or (ii) providing an electrical signal to the capacitor such that charge is accumulated at the capacitor, wherein the optical characteristic of the optical waveguide associated with the phase shifter is modulated in response to the modulator control signal.

52. The electro-optical processing system of claim 51, wherein the modulator control signals comprise digital control signals, each digital control signal comprising two or more bits, and

the number of phase shifters in the first modulator arm is equal to the number of bits in the digital control signal.

53. The electro-optical processing system of claim 51, wherein the signal conditioning circuit is configured to shape changes in amplitude of the electrical signal applied to at least one of the phase shifters in association with corresponding changes between successive modulator control signals in the series of modulator control signals.

54. The electro-optical processing system of claim 51, wherein each phase shifter comprises a semiconductor diode, and

the optical modulator includes a control circuit configured to, for each phase shifter, provide an electrical signal to the semiconductor diode such that the semiconductor diode is in a forward biased state in which an optical characteristic of an optical waveguide associated with the phase shifter is modulated in response to a value of a corresponding modulator control signal.

55. The electro-optic processing system of claim 51, wherein each phase shifter comprises a capacitor configured to achieve free carrier based modulation by carrier accumulation, and

The optical modulator includes a control circuit configured to, for each phase shifter, provide an electrical signal to the capacitor to cause a charge to accumulate at the capacitor, wherein an optical characteristic of an optical waveguide associated with the phase shifter is modulated in response to a value of a corresponding modulator control signal.

56. The electro-optical processing system of claim 51, wherein the optical modulator comprises at least one calibration phase shifter configured to compensate for an imbalance in optical phase shifts imparted by the first and second modulator arms.

57. The electro-optical processing system of claim 56, comprising:

a plurality of 1-bit DACs, each 1-bit DAC configured to drive a corresponding phase shifter, wherein different phase shifters are driven by different 1-bit DACs, an

At least one m-bit DAC configured to drive the at least one calibration phase shifter,

wherein m is more than or equal to 2.

58. The electro-optical processing system of claim 57, wherein m ≧ 4.

59. The electro-optical processing system of claim 58, wherein m ≧ 8.

60. The electro-optical processing system of claim 51, wherein the first modulator arm comprises at least a first phase shifter, a second phase shifter, and a third phase shifter;

The first phase shifter comprises a first diode section;

the second phase shifter comprises a second diode section;

the third phase shifter comprises a third diode section;

the second diode section is associated with an optical path length that is longer than an optical path length associated with the first diode section;

the third diode section is associated with an optical path length that is longer than the optical path length associated with the second diode section;

the first diode segment is coupled to a first signal conditioning circuit configured to provide an enhanced bandwidth for modulating the first diode segment;

the second diode section is coupled to a second signal conditioning circuit configured to provide an enhanced bandwidth for modulating the second diode section; and

the third diode section is coupled to a third signal conditioning circuit configured to provide an enhanced bandwidth for modulating the third diode section.

61. The electro-optical processing system of claim 60, wherein each of the first, second, and third signal conditioning circuits is configured to shape changes in amplitude of electrical signals respectively applied to the first, second, and third diode sections in association with corresponding changes between successive ones of the series of modulator control signals,

Each of the first, second, and third diode sections comprises a semiconductor diode associated with an optical path length of less than about 1 millimeter,

each of the first, second, and third signal conditioning circuits is configured to shape the amplitude change by applying an electrical signal to first, second, and third semiconductor diodes, respectively, via a circuit configured to pump a current between the semiconductor diode and a capacitor connected in series between the semiconductor diode and a circuit providing the series of modulator control signals, wherein an amount of charge transferred by the pumped current is determined based at least in part on a voltage that is constant over a plurality of consecutive time intervals in which the series of modulator control signals is provided.

62. The electro-optical processing system of claim 61, wherein the first diode section includes electrical contacts for applying a first electrical signal to the semiconductor diodes in the first diode section in a forward-biased state, and an optical characteristic of the first optical waveguide segment is modulated in response to a value of a first portion of the modulator control signal,

The second diode section comprising an electrical contact for applying a second electrical signal to the semiconductor diode in the second diode section in a forward-biased state, and the optical characteristic of the second optical waveguide segment being modulated in response to the value of the second portion of the modulator control signal,

the third diode section includes an electrical contact for applying a third electrical signal to the semiconductor diode in the third diode section in a forward-biased state, and an optical characteristic of the third optical waveguide segment is modulated in response to a value of a third portion of the modulator control signal.

63. The electro-optical processing system of claim 51, wherein said modulator control signal is derived from an n-bit digital input value, n being an integer greater than or equal to 4,

the number of phase shifters in the first modulator arm is equal to n/2,

the electro-optical processing system includes a modulator control module comprising n/2-bit DACs, each 2-bit DAC configured to convert 2 bits of a digital input value into a modulator control signal capable of having four possible values.

64. The electro-optical processing system of claim 51, wherein said modulator control signal is derived from an n-bit digital input value, n being an integer greater than or equal to 6,

The number of phase shifters in the first modulator arm is equal to n/m, m is less than n, m is an integer value greater than 2,

the electro-optical processing system includes a modulator control module including n/m-bit DACs, each m-bit DAC configured to convert an m-bit digital input value to be capable of having 2^mModulator control signals of possible values.

65. The electro-optical processing system of claim 51 comprising an Artificial Neural Network (ANN) computing system, the ANN computing system comprising:

a first unit configured to generate a series of modulator control signals;

a light source configured to provide a plurality of light outputs;

a plurality of optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector by modulating the plurality of light outputs provided by the light source based on the modulator control signal, the optical input vector comprising a plurality of optical signals;

wherein the plurality of optical modulators includes an optical modulator comprising at least one modulator arm comprising an optical waveguide and a plurality of phase shifters positioned along the optical waveguide; and

A matrix processing unit coupled to the plurality of optical modulators and the first unit, the matrix processing unit configured to transform the optical input vector into an analog output vector based on a plurality of weight control signals.

66. The electro-optical processing system of claim 65, wherein the ANN computing system comprises:

a second unit coupled to the matrix processing unit and configured to convert the analog output vector into a digitized output vector; and

a controller comprising an integrated circuit configured to perform operations comprising:

receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set, the input data set comprising a first numeric input vector;

receiving a first plurality of neural network weights; and

generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

67. The electro-optical processing system of claim 65, wherein the matrix processing unit comprises:

a plurality of replica modules, wherein each replica module corresponds to a subset of one or more optical signals of the optical input vector and is configured to split the subset of one or more optical signals into two or more replicas of the optical signal;

A plurality of multiplication modules, wherein each multiplication module corresponds to a subset of one or more optical signals and is configured to multiply the one or more optical signals of the subset by one or more matrix element values using optical amplitude modulation; and

one or more summation modules, wherein each summation module is configured to produce an electrical signal representing a sum of results of two or more of the multiplication modules.

68. The electro-optical processing system of any one of claims 51-67, comprising at least one of a personal computer, a server computer, a vehicle computer, or a flight computer, wherein the at least one input optical waveguide, the at least one digital input port, and the optical modulator are part of the personal computer, the server computer, the vehicle computer, or the flight computer.

69. An electro-optical processing system, comprising:

at least one input optical waveguide configured to receive a light wave;

at least one digital input port configured to receive a series of digital input values, each digital input value comprising two or more bits; and

An interferometric optical modulator coupled to the at least one input optical waveguide, the interferometric optical modulator comprising:

an optical waveguide portion comprising a diode segment positioned along the optical waveguide portion, wherein the diode segment is configured to modulate a light wave propagating through the optical waveguide portion, an

A signal conditioning circuit configured to shape changes in the amplitude of the electrical signal applied to the diode segment in association with corresponding changes between successive ones of the series of digital input values, wherein the signal conditioning circuit comprises:

a first signal conditioning path providing an unconditioned electrical signal corresponding to the series of digital input values,

a second signal conditioning path providing a delayed, scaled and/or inverted version of the unregulated electrical signal, an

A third signal conditioning path providing a delayed, scaled and/or inverted version of the unconditioned electrical signal.

70. The electro-optical processing system of claim 69, wherein the diode section comprises a semiconductor diode having an optical path length of less than about 1 millimeter.

71. The electro-optical processing system of claim 69, wherein the interferometric optical modulator comprises an optical interference section configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on modulation of the optical characteristics of the diode segments.

72. The electro-optical processing system of claim 69, wherein the diode section comprises:

a semiconductor diode, and

a control circuit configured to apply an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of the diode section is modulated in response to the digital input value.

73. The electro-optical processing system of claim 72, wherein the semiconductor diode has an optical path length of less than about 1 millimeter.

74. The electro-optical processing system of claim 69, comprising an Artificial Neural Network (ANN) computing system, the ANN computing system comprising:

a first unit configured to generate a plurality of modulator control signals comprising a series of digital input values, each digital input value comprising two or more bits;

a light source configured to provide a plurality of light outputs;

a plurality of interferometric optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector by modulating the plurality of optical outputs provided by the light source based on the plurality of modulator control signals, the optical input vector comprising a plurality of optical signals;

Wherein the plurality of interferometric optical modulators comprises an interferometric optical modulator comprising the optical waveguide portion and the signal conditioning circuit; and

a matrix processing unit coupled to the plurality of interferometric optical modulators and the first unit, the matrix processing unit configured to transform the optical input vector into an analog output vector based on a plurality of weight control signals.

75. The electro-optical processing system of claim 74, wherein the ANN computing system comprises:

a second unit coupled to the matrix processing unit and configured to convert the analog output vector into a digitized output vector; and

a controller comprising an integrated circuit configured to perform operations comprising:

receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set, the input data set comprising a first numeric input vector;

receiving a first plurality of neural network weights; and

generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

76. The electro-optical processing system of claim 74, wherein the matrix processing unit comprises:

a plurality of replica modules, wherein each replica module corresponds to a subset of one or more optical signals of the optical input vector and is configured to split the subset of one or more optical signals into two or more replicas of the optical signal;

a plurality of multiplication modules, wherein each multiplication module corresponds to a subset of one or more optical signals and is configured to multiply the one or more optical signals of the subset by one or more matrix element values using optical amplitude modulation; and

one or more summation modules, wherein each summation module is configured to produce an electrical signal representing a sum of results of two or more of the multiplication modules.

77. The electro-optical processing system of any one of claims 69-76, comprising at least one of a personal computer, a server computer, a vehicle computer, or a flight computer, wherein the at least one input optical waveguide, the at least one digital input port, and the optical modulator are part of the personal computer, the server computer, the vehicle computer, or the flight computer.

78. An electro-optical processing system, comprising:

an optical modulator configured to modulate a light wave representing a series of digital input values, each digital input value comprising two or more bits, the optical modulator comprising an optical waveguide portion comprising a plurality of diode segments positioned along the optical waveguide portion, wherein the diode segments are configured to impose a different respective modulation contribution to the light wave as it propagates through the optical waveguide portion;

wherein each diode segment comprises a semiconductor diode configured to operate in a forward biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit of the digital input value, and

wherein the optical modulator comprises a plurality of signal conditioning circuits configured to shape changes in the amplitude of the electrical signal applied to each diode segment in association with corresponding changes between successive ones of the series of digital input values, wherein each signal conditioning circuit is associated with one of the diode segments and different diode segments are associated with different signal conditioning circuits.

79. The electro-optical processing system of claim 78, wherein the semiconductor diode has an optical path length of less than about 1 millimeter.

80. The electro-optical processing system of claim 78 comprising an Artificial Neural Network (ANN) computing system, the ANN computing system comprising:

a first unit configured to generate a plurality of modulator control signals comprising a series of digital input values;

a light source configured to provide a plurality of light outputs;

a plurality of optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector by modulating the plurality of light outputs provided by the light source based on the plurality of modulator control signals, the optical input vector comprising a plurality of optical signals;

wherein the plurality of optical modulators includes an optical modulator comprising the optical waveguide portion and the signal conditioning circuitry, the optical waveguide portion including a plurality of diode segments positioned along the optical waveguide portion; and

a matrix processing unit coupled to the plurality of optical modulators and the first unit, the matrix processing unit configured to transform the optical input vector into an analog output vector based on a plurality of weight control signals.

81. The electro-optical processing system of claim 80, wherein the ANN computing system comprises:

a second unit coupled to the matrix processing unit and configured to convert the analog output vector into a digitized output vector; and

a controller comprising an integrated circuit configured to perform operations comprising:

receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set, the input data set comprising a first numeric input vector;

receiving a first plurality of neural network weights; and

generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

82. The electro-optical processing system of claim 81, wherein the matrix processing unit comprises:

a plurality of replica modules, wherein each replica module corresponds to a subset of one or more optical signals of the optical input vector and is configured to split the subset of one or more optical signals into two or more replicas of the optical signal;

A plurality of multiplication modules, wherein each multiplication module corresponds to a subset of one or more optical signals and is configured to multiply the one or more optical signals of the subset by one or more matrix element values using optical amplitude modulation; and

one or more summation modules, wherein each summation module is configured to produce an electrical signal representing a sum of results of two or more of the multiplication modules.

83. The electro-optical processing system of any one of claims 78 to 82, comprising at least one of a personal computer, a server computer, a vehicle computer, or a flight computer, wherein the optical modulator is part of the personal computer, the server computer, the vehicle computer, or the flight computer, the optical modulator comprising the optical waveguide portion and the signal conditioning circuitry, the optical waveguide portion comprising a plurality of diode segments positioned along the optical waveguide portion.

84. An electro-optical processing system, comprising:

a first unit configured to generate a first set of modulator control signals;

a second unit configured to generate a second set of modulator control signals; and

A processor unit, the processor unit comprising:

a light source or light port configured to provide a plurality of light outputs; and

a first set of optical modulators coupled to the light source or light port and the first unit;

the first set of optical modulators is configured to generate an optical input vector by modulating the plurality of optical outputs provided by the optical source or optical port based on digital input values corresponding to the first set of modulator control signals, the optical input vector comprising a plurality of optical signals;

the processor unit further includes a matrix processing unit comprising a second set of optical modulators, the matrix processing unit coupled to the second unit and configured to convert the optical input vector to an analog output vector based on a plurality of digital weight values corresponding to the second set of modulator control signals;

at least one optical modulator in the first set of optical modulators comprises a first optical waveguide portion comprising a plurality of diode segments positioned along the first optical waveguide portion;

The diode segments are configured to impose different respective modulation contributions to a light wave propagating through the first optical waveguide portion based on different respective single bits of one of the digital input values;

at least one optical modulator in the second set of optical modulators includes a second optical waveguide portion comprising a single diode segment configured to apply modulation to light waves propagating through the second optical waveguide portion based on a plurality of bits of one of the digital weight values.

85. The electro-optical processing system of claim 84, each respective diode segment comprising a semiconductor diode configured to operate in a forward biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit of one of the digital input values.

86. An electro-optical processing system, comprising:

a first set of optical modulators configured to generate an optical input vector by modulating a plurality of input optical signals based on a modulator control signal, the optical input vector comprising a plurality of optical signals; and

a matrix processing unit comprising a second set of optical modulators, the matrix processing unit coupled to the first set of optical modulators and configured to convert the optical input vector into an analog output vector based on a plurality of weight values;

Wherein at least one optical modulator of the first set of optical modulators comprises a first modulator arm and a second modulator arm, at least one of the first modulator arm or the second modulator arm comprising an optical waveguide and at least two phase shifters positioned along the optical waveguide, each phase shifter configured to apply a modulation contribution to an optical wave propagating through the optical waveguide based on one of the modulation control signals, a different phase shifter configured to apply a different respective modulation contribution to the optical wave propagating through the optical waveguide in response to the same modulation control signal level;

wherein at least one optical modulator in the second set of optical modulators comprises a first modulator arm and a second modulator arm, the first modulator arm and the second modulator arm each comprising an optical waveguide, at least one of the first modulator arm and the second modulator arm comprising a single phase shifter positioned along the optical waveguide, the first modulator arm and the second modulator arm each comprising at most one phase shifter.

87. The electro-optical processing system of claim 86, wherein each optical modulator of the first set of optical modulators includes a first modulator arm and a second modulator arm, at least one of the first modulator arm or the second modulator arm including an optical waveguide and at least two phase shifters positioned along the optical waveguide, the phase shifters configured to impose different respective modulation contributions to light waves propagating through the optical waveguide;

Each optical modulator in the second set of optical modulators includes a first modulator arm and a second modulator arm, each of the first modulator arm and the second modulator arm including an optical waveguide and at most one phase shifter positioned along the optical waveguide.

88. The electro-optical processing system of claim 87, wherein at least one of the first modulator arm or the second modulator arm of each optical modulator in the first set of optical modulators includes an optical waveguide and at least three phase shifters positioned along the optical waveguide, the phase shifters configured to impose different respective modulation contributions on light waves propagating through the optical waveguide.

89. The optoelectronic processing system of claim 88, wherein at least one of the first or second modulator arms of each optical modulator in the first set of optical modulators includes an optical waveguide and at least four phase shifters positioned along the optical waveguide, the phase shifters configured to impose different respective modulation contributions on light waves propagating through the optical waveguide.

90. The electro-optical processing system of claim 89, wherein at least one of the first modulator arm or the second modulator arm of each optical modulator in the first set of optical modulators includes an optical waveguide and at least eight phase shifters positioned along the optical waveguide, the phase shifters configured to impose different respective modulation contributions on light waves propagating through the optical waveguide.

91. The electro-optical processing system of claim 86, comprising a first modulator control unit configured to receive a digital input value and to generate the modulator control signal based on the digital input value,

wherein for each digital input value, the first modulator control unit is configured to generate the modulator control signal and apply the modulator control signal to the at least two phase shifters of the at least one optical modulator, and the at least one optical modulator is configured to generate a modulated optical signal that is an analog representation of the digital input value.

92. The electro-optical processing system of claim 91, wherein the first modulator control unit is configured to receive an n-bit digital input value, the modulated optical signal is an analog representation of the n-bit digital input value, and the modulated optical signal has a value that is 2ⁿA signal level of one of the possible signal levels.

93. The electro-optical processing system of claim 92, wherein the first modulator control unit comprises n 1-bit DACs for driving the at least one optical modulator, n being a positive integer greater than 1, the at least one optical modulator in the first set of optical modulators comprising a first modulator arm comprising n phase shifters, each of the 1-bit DACs being configured to generate a modulator control signal for controlling a corresponding phase shifter.

94. The electro-optical processing system of claim 92, wherein the first modulator control unit comprises n/2-bit DACs for driving the at least one optical modulator, n being an even positive integer greater than 3, the at least one optical modulator in the first set of optical modulators comprising a first modulator arm comprising n/2 phase shifters, each of the 2-bit DACs being configured to generate a modulator control signal for controlling a corresponding phase shifter.

95. The electro-optical processing system of claim 92, wherein the first modulator control unit comprises n/3-bit DACs, n being a positive integer greater than 5, n/3 being an integer greater than 1, the at least one optical modulator in the first set of optical modulators comprising a first modulator arm comprising n/3 phase shifters, each of the 3-bit DACs being configured to generate a modulator control signal for controlling a corresponding phase shifter.

96. The electro-optical processing system of claim 92, wherein the first modulator control unit comprises n/4-bit DACs, n being a positive integer greater than 7, n/4 being an integer greater than 1, the at least one optical modulator in the first set of optical modulators comprising a first modulator arm comprising n/4 phase shifters, each of the 4-bit DACs being configured to generate a modulator control signal for controlling a corresponding phase shifter.

97. The electro-optical processing system of claim 91, comprising a second modulator control unit configured to receive digital weight values and to generate analog weight values applied to a single phase shifter of a modulator arm of the at least one optical modulator in the second set of optical modulators,

wherein the first modulator control unit updates the modulator control signals at a first frequency, the first set of optical modulators is reconfigured at the first frequency, the second modulator control unit updates the analog weight values at a second frequency, the second set of optical modulators is reconfigured at the second frequency, and the first frequency is greater than the second frequency.

98. The photo-processing system of claim 97, wherein the first frequency is at least twice the second frequency.

99. The photo-processing system of claim 97, wherein the first frequency is at least four times the second frequency.

100. The photo-processing system of claim 97, wherein the first frequency is at least ten times the second frequency.

101. An electro-optical processing system, comprising:

a first unit configured to generate a plurality of modulator control signals;

a processor unit, comprising:

a light source configured to provide a plurality of light outputs;

a modulator array comprising a first set of a plurality of optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector comprising a plurality of optical signals by modulating a plurality of optical outputs provided by the light source based on the plurality of modulator control signals, wherein each of the plurality of optical modulators has a segmented design and comprises two or more phase shifters, and each of the phase shifters is associated with a signal conditioning circuit configured to implement pre-emphasis and de-emphasis of the phase shifter to enhance an operating bandwidth of the phase shifter; and

a matrix processing unit coupled to the array of modulators and the first unit, the matrix processing unit configured to convert the optical input vector to an analog output vector based on a plurality of weight control signals, wherein the matrix processing unit comprises a second set of a plurality of optical modulators having a non-fragmented design, each optical modulator having a single phase shifter in at least one modulator arm, each modulator arm having at most one phase shifter;

A second unit coupled to the matrix processing unit and configured to convert the analog output vector into a digitized output vector; and

a controller, comprising an integrated circuit, configured to perform operations comprising:

receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set, the input data set comprising a first numeric input vector;

receiving a first plurality of neural network weights; and

generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

Technical Field

The present disclosure relates to optical modulation for electro-optical processing.

Background

Neuromorphic computing (neuromorphic computing) is a method of approximating the operation of the brain in the field of electronics. One prominent method of neuromorphic computation is the Artificial Neural Network (ANN), which is a collection of artificial neurons interconnected in a specific manner to process information in a manner similar to brain function. ANN has found use in a variety of applications including artificial intelligence, speech recognition, text recognition, natural language processing, and various forms of pattern recognition.

The ANN has an input layer, one or more hidden layers, and an output layer. Each layer has nodes or artificial neurons, and the nodes are interconnected between layers. Each node of the hidden layer performs a weighted sum of the signals received from nodes of the previous layer and performs a nonlinear transformation ("activation") of the weighted sum to produce an output. The weighted sum may be calculated by performing a matrix multiplication step. Therefore, computing an ANN typically involves multiple matrix multiplication steps, which are typically performed using electronic integrated circuits.

Computations performed on electronic data encoded in analog or digital form on an electronic signal, such as a voltage or current, are typically implemented using electronic computing hardware, such as an analog or digital electronic device (e.g., a processor, an application-specific integrated circuit (ASIC), or a system on a chip (SoC)) implemented in an integrated circuit, an electronic circuit board, or other electronic circuit. Optical signals have been used to transmit data over long and short distances (e.g., within data centers). Operations performed on such optical signals are typically performed in the context of optical data transmission, such as within a device used to switch or filter the optical signals in a network. The use of optical signals in computing platforms has been more limited. Various components and systems for all-optical (all-optical) computing have been proposed. For example, a system may include conversion from and to electrical signals at the input and output, respectively, but neither type (electrical or optical) of signal may be used for important operations performed in the computation.

Disclosure of Invention

In general, in a first aspect, a system includes at least one input optical waveguide (optical waveguide) configured to receive a light wave; at least one digital input port configured to receive a series of digital input values in successive time intervals, each digital input value comprising two or more bits; and an optical modulator (optical modulator) coupled to the input optical waveguide. The optical modulator includes an optical waveguide portion including a plurality of diode segments positioned along the optical waveguide portion. The diode segments impose different respective modulation contributions (distributions) on light waves propagating through the optical waveguide section, each diode segment comprising a semiconductor diode having an optical path length (optical path length) of less than about 1 millimeter, and electrical contacts for applying an electrical signal to the semiconductor diode in a forward-biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit of the digital input value. The optical modulator includes signal conditioning (shaping) circuitry configured to shape (shape) changes in amplitude of an electrical signal applied to at least one of the semiconductor diodes in association with corresponding changes between successive ones of a series of digital input values.

Aspects can include one or more of the following features. Shaping the change in the amplitude of the electrical signal applied to the semiconductor diode in association with successive ones of a series of digital input values may comprise: for a beginning portion of the second time interval, a magnitude of a change in amplitude between a first electrical signal level associated with the first time interval and a second electrical signal level associated with the second time interval is increased.

Shaping the change in the amplitude of the electrical signal applied to the semiconductor diode in association with successive ones of a series of digital input values may further comprise: for a final portion of the second time interval, the magnitude of the change in amplitude between the first electrical signal level and the second electrical signal level is reduced.

Shaping the change in the amplitude of the electrical signal applied to the semiconductor diode in association with successive ones of a series of digital input values may include: an electrical signal is applied to the semiconductor diode through a matching circuit configured to match an impedance associated with the semiconductor diode without significantly changing an amplitude of the applied electrical signal.

The matching circuit may comprise a passive circuit.

The matching circuit may consist essentially of an inductor.

Shaping the change in the amplitude of the electrical signal applied to the semiconductor diode for successive ones of a series of digital input values may comprise: applying an electrical signal to the semiconductor diode by a circuit configured to pump a current between the semiconductor diode and a capacitor connected in series between the semiconductor diode and the circuit providing the series of digital input values, wherein an amount of charge transferred by the pumped current is determined based at least in part on a voltage that is constant over a plurality of consecutive time intervals providing the series of digital input values.

The optical modulator may comprise an interferometric optical modulator (interferometric optical modulator) further comprising an optical interference portion configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on the cumulative modulation contribution of the diode segments.

The optical interference portion may include an optical combiner (optical combiner).

The optical waveguide section may include: at least two optical waveguide segments, each optical waveguide segment receiving a light wave split from the same optical splitter (optical splitter) coupled to the input optical waveguide and providing a light wave to the optical combiner.

The optical characteristic of the diode section may comprise an effective refractive index of the diode section, and the different respective modulation contributions may comprise different respective phase shifts.

The optical path length of the first semiconductor diode of the first diode section may be about twice the optical path length of the second semiconductor diode of the second diode section.

Shaping the change in the amplitude of the electrical signal applied to the semiconductor diode for successive ones of a series of digital input values may comprise: applying a first predetermined shape of an electrical signal amplitude to the electrical contacts of the first diode segment and the electrical contacts of the second diode segment in response to a change in the value of the corresponding bit from 0 to 1, and applying a second predetermined shape of an electrical signal amplitude to the electrical contacts of the first diode segment and the electrical contacts of the second diode segment in response to a change in the value of the corresponding bit from 1 to 0.

Shaping the change in the amplitude of the electrical signal applied to the semiconductor diode for successive ones of a series of digital input values may comprise: applying an electrical signal to an electrical contact of the first diode section through a first matching circuit configured to match an impedance associated with the first semiconductor diode, and applying an electrical signal to an electrical contact of the second diode section through a second matching circuit configured to match an impedance associated with the second semiconductor diode.

The first matching circuit and the second matching circuit may each consist essentially of an inductor.

The input optical waveguide may be coupled to an optical demultiplexer (optical demultiplexer) that separates light waves of at least two different wavelengths.

The optical modulator may comprise an absorbing optical modulator (absorption optical modulator) configured to provide a degree of absorbance (absorbance) resulting in a predetermined amplitude reduction based on the cumulative modulation contribution of the diode segments.

The optical characteristic of the diode section may comprise an absorption coefficient of the diode section, and the different respective modulation contributions may comprise different respective absorbances.

The at least one input optical waveguide may include a plurality of input optical waveguides, a plurality of optical modulators may each be coupled with a different respective input optical waveguide of the plurality of input optical waveguides, and outputs from the plurality of optical modulators may be combined to provide a result of the vector-matrix multiplication.

In another general aspect, a system includes: at least one input optical waveguide configured to receive optical waves; at least one digital input port configured to receive a series of digital input values in successive time intervals, each digital input value comprising two or more bits; and an interferometric optical modulator coupled to the input optical waveguide. The interferometric optical modulator includes an optical waveguide portion including a diode section along the optical waveguide portion. The diode section modulates a light wave propagating through the optical waveguide portion, the diode section comprising: a semiconductor diode having an optical path length of less than about 1 millimeter, and electrical contacts for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of the diode section is modulated in response to the digital input value. The interferometric optical modulator includes a signal conditioning circuit configured to shape changes in the amplitude of an electrical signal applied to the semiconductor diode in association with corresponding changes between successive ones of a series of digital input values. The signal conditioning circuit includes: a first signal conditioning path providing an unconditioned electrical signal corresponding to the series of digital input values; a second signal conditioning path providing a delayed, scaled and/or inverted version of the unconditioned electrical signal; and a third signal conditioning path providing delayed, scaled and/or inverted versions of the unconditioned electrical signal. The interferometric optical modulator further includes an optical interference section configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on the modulation of the optical characteristic of the diode segment.

Aspects can include one or more of the following features. The optical interference section may include a coupling section of each of a pair of optical waveguides, the coupling sections of each of the optical waveguides being in proximity to each other.

The optical waveguide portion may include a first optical waveguide of the pair of optical waveguides formed in a closed path.

In another general aspect, a system includes: at least one input optical waveguide configured to receive optical waves; at least one digital input port configured to receive a series of digital input values, each digital input value comprising two or more bits; and an optical modulator coupled to the input optical waveguide, the optical modulator comprising an optical waveguide portion comprising a plurality of diode segments positioned along the optical waveguide portion, wherein the diode segments are configured to impose different respective modulation contributions to light waves propagating through the optical waveguide portion.

Aspects can include one or more of the following features. The optical modulator may include a signal conditioning circuit configured to shape a change in amplitude of the electrical signal applied to at least one of the diode segments in association with a corresponding change between successive ones of a series of digital input values.

Each diode segment may include: a semiconductor diode, and an electrical contact for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of the diode section is modulated in response to a value of a corresponding bit in the digital input value.

Each semiconductor diode may have an optical path length of less than about 1 millimeter.

In another general aspect, a system includes: at least one input optical waveguide configured to receive optical waves; at least one digital input port configured to receive a series of digital input values, each digital input value comprising two or more bits; and an interferometric optical modulator coupled to the at least one input optical waveguide. The interferometric optical modulator includes an optical waveguide portion including a diode segment positioned along the optical waveguide portion, wherein the diode segment is configured to modulate a light wave propagating through the optical waveguide portion. The interferometric optical modulator includes a signal conditioning circuit configured to shape changes in the amplitude of the electrical signal applied to the diode segment in association with corresponding changes between successive ones of a series of digital input values. The signal conditioning circuit includes: a first signal conditioning path providing an unconditioned electrical signal corresponding to the series of digital input values; a second signal conditioning path providing a delayed, scaled and/or inverted version of the unconditioned electrical signal; and a third signal conditioning path providing delayed, scaled and/or inverted versions of the unconditioned electrical signal.

Aspects can include one or more of the following features. The diode section may include a semiconductor diode having an optical path length of less than about 1 millimeter.

The interferometric optical modulator may include an optical interference section configured to provide a degree of destructive optical interference resulting in a predetermined amplitude reduction based on the modulation of the optical characteristics of the diode section.

The diode section may include: a semiconductor diode, and an electrical contact for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of the diode section is modulated in response to the digital input value.

The semiconductor diode may have an optical path length of less than about 1 millimeter.

In another general aspect, a system includes an optical modulator configured to modulate a light wave representing a series of digital input values, each digital input value including two or more bits. The optical modulator includes an optical waveguide portion including a plurality of diode sections positioned along the optical waveguide portion. The diode sections are configured to apply different respective modulation contributions to the light waves as they propagate through the optical waveguide portion. Each diode segment includes a semiconductor diode configured to operate in a forward biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit in the digital input value. The optical modulator includes a signal conditioning circuit configured to shape a change in amplitude of an electrical signal applied to at least one of the diode segments in association with a corresponding change between successive ones of a series of digital input values.

Aspects can include the following features. Each semiconductor diode may have an optical path length of less than about 1 millimeter.

In another general aspect, a system includes: a first unit configured to generate a first set of modulator control signals; a second unit configured to generate a second set of modulator control signals, and a processor unit. The processor unit includes a light source or light port configured to provide a plurality of light outputs; and a first set of optical modulators coupled to the light source or light port and the first unit. The first set of optical modulators is configured to generate an optical input vector comprising a plurality of optical signals by modulating the plurality of optical outputs provided by the light source or light port based on digital input values corresponding to the first set of modulator control signals. The processor unit also includes a matrix multiplication unit that includes a second set of optical modulators. The matrix multiplication unit is coupled to the second unit and configured to convert the optical input vector to an analog output vector based on a plurality of digital weight values corresponding to the second set of modulator control signals. At least one optical modulator in the first set of optical modulators includes a first optical waveguide portion including a plurality of diode segments positioned along the first optical waveguide portion. The diode segments are configured to impose different respective modulation contributions to the light wave propagating through the first optical waveguide portion based on different respective single bits of one of the digital input values. At least one optical modulator in the second set of optical modulators includes a second optical waveguide portion comprising a single diode segment configured to apply modulation to an optical wave propagating through the second optical waveguide portion based on a plurality of bits in which one of the digital weight values is based.

Aspects can include the following features. Each respective diode segment includes a semiconductor diode configured to operate in a forward biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit of one of the digital input values.

In another general aspect, a system includes: at least one input optical waveguide configured to receive optical waves; at least one digital input port configured to receive a series of digital input values in successive time intervals, each digital input value comprising two or more bits; and an optical modulator coupled to the input optical waveguide. The optical modulator includes: an optical waveguide portion comprising a plurality of optical waveguide segments associated with a plurality of diode segments positioned along corresponding optical waveguide segments, the optical waveguide segments being part of a continuous optical waveguide in which the diode segments are configured to impose different respective modulation contributions to light waves propagating through the optical waveguide segments. Each respective diode segment includes: a semiconductor diode having an optical path length less than about 1 millimeter, and electrical contacts for applying an electrical signal to the semiconductor diode in a forward biased state in which an optical characteristic of an optical waveguide segment associated with the diode segment is modulated in response to a value of a corresponding bit of the digital input values, and a signal conditioning circuit configured to shape an amplitude change of the electrical signal applied to at least one of the semiconductor diodes in association with a corresponding change between successive ones of a series of digital input values.

In another general aspect, a system includes: at least one input optical waveguide configured to receive optical waves; at least one input port configured to receive a series of modulator control signals; and an optical modulator, coupled to the input optical waveguide, configured to provide pulse amplitude modulation having four or more amplitude levels. The optical modulator includes a first modulator arm (modulator arm) and a second modulator arm, at least one of the first modulator arm and the second modulator arm including an optical waveguide and a plurality of phase shifters (phase shifters) positioned along the optical waveguide, the phase shifters configured to impose different respective modulation contributions to a light wave propagating through the optical waveguide, each phase shifter coupled to a respective signal conditioning circuit configured to provide an enhanced bandwidth for binary modulation, and the different phase shifters coupled to different signal conditioning circuits. Each phase shifter includes a semiconductor diode or capacitor associated with an optical path length of less than about 1 millimeter. The system includes a control circuit configured to perform at least one of the following for each phase shifter: (i) providing an electrical signal to the semiconductor diode such that the semiconductor diode is in a forward biased state in which an optical characteristic of an optical waveguide associated with the phase shifter is modulated in response to the modulator control signal, or (ii) providing an electrical signal to the capacitor such that charge is accumulated at the capacitor, wherein the optical characteristic of the optical waveguide associated with the phase shifter is modulated in response to the modulator control signal.

In another general aspect, a system includes: an optical modulator configured to modulate a light wave representing a series of digital input values, each digital input value comprising two or more bits, the optical modulator comprising an optical waveguide portion comprising a plurality of diode segments positioned along the optical waveguide portion, wherein the diode segments are configured to impose a different respective modulation contribution to the light wave as the light wave propagates through the optical waveguide portion. Each diode segment includes a semiconductor diode configured to operate in a forward biased state in which an optical characteristic of the diode segment is modulated in response to a value of a corresponding bit of the digital input value. The optical modulator includes a plurality of signal conditioning circuits configured to shape changes in the amplitude of the electrical signal applied to each diode segment in association with corresponding changes between successive ones of a series of digital input values, wherein each signal conditioning circuit is associated with one of the diode segments and different diode segments are associated with different signal conditioning circuits.

In another general aspect, a system includes: a first set of optical modulators configured to generate an optical input vector by modulating a plurality of input optical signals based on a modulator control signal, the optical input vector comprising a plurality of optical signals; and a matrix processing unit comprising a second set of optical modulators, the matrix processing unit coupled to the array of modulators and configured to convert the optical input vector into an analog output vector based on a plurality of weight values. At least one optical modulator of the first set of optical modulators includes a first modulator arm and a second modulator arm, at least one of the first modulator arm and the second modulator arm including an optical waveguide and at least two phase shifters positioned along the optical waveguide, the phase shifters configured to apply a modulation contribution to an optical wave propagating through the optical waveguide based on one of the modulation control signals, different phase shifters configured to apply different respective modulation contributions to the optical wave propagating through the optical waveguide in response to the same modulation control signal level. At least one optical modulator in the second set of optical modulators includes a first modulator arm and a second modulator arm, each of the first modulator arm and the second modulator arm including an optical waveguide, at least one of the first modulator arm and the second modulator arm including a single phase shifter positioned along the optical waveguide, each of the first modulator arm and the second modulator arm including at most one phase shifter.

In another general aspect, a system includes: a first unit configured to generate a plurality of modulator control signals; and a processor unit. The processor unit includes: a light source configured to provide a plurality of light outputs, a modulator array, and a matrix processing unit. The modulator array includes a first set of a plurality of optical modulators coupled to the light source and the first unit, the plurality of optical modulators configured to generate an optical input vector comprising a plurality of optical signals by modulating a plurality of optical outputs provided by the light source based on the plurality of modulator control signals, wherein each of the plurality of optical modulators has a segmented design and includes two or more phase shifters, each of the phase shifters associated with a signal conditioning circuit configured to implement pre-emphasis (pre-emphasis) and de-emphasis (de-emphasis) of the phase shifter to enhance an operating bandwidth of the phase shifter. The matrix processing unit is coupled to the array of modulators and the first unit, the matrix processing unit configured to convert the optical input vector to an analog output vector based on a plurality of weight control signals, wherein the matrix processing unit comprises a second set of a plurality of optical modulators of non-fragmented design, each optical modulator having a single phase shifter in at least one modulator arm, each modulator arm having at most one phase shifter. The system includes a second unit coupled to the matrix multiplication unit and configured to convert the analog output vector to a digitized output vector; and a controller comprising an integrated circuit configured to perform operations comprising: receiving an artificial neural network computation request, the artificial neural network computation request comprising an input data set (dataset) comprising a first digital input vector; receiving a first plurality of neural network weights; and generating, by the first unit, a first plurality of modulator control signals based on the first digital input vector and a first plurality of weight control signals based on the first plurality of neural network weights.

Aspects can include one or more of the following advantages.

In certain embodiments described herein, the types of components used to perform optical modulation, and/or the characteristics of these components, are selected to provide performance improvements or other design advantages in the performance of the resulting optoelectronic computing system. In systems fabricated using silicon photonics (silicon photonics) technology, optical waveguides may be formed in silicon, and there are various types of semiconductor diode structures that may be formed by doping silicon in the vicinity of a waveguide to achieve modulation of light waves propagating in the waveguide. For example, a PIN diode structure or a metal-oxide-semiconductor (MOS) capacitor may be designed to implement free-carrier-based modulation by carrier injection (carrier injection), carrier depletion (carrier depletion), or carrier accumulation (carrier accumulation) using different doping profiles. Carrier injection uses a forward biased PIN diode structure, which is typically of relatively small size (e.g., less than 1 mm) due to its high modulation efficiency, but provides lower speed modulation (e.g., less than about 1Gb/s), while carrier depletion uses a reverse biased PIN diode structure, which is typically of larger size (e.g., greater than 1 mm) but provides higher speed modulation (e.g., greater than about 1 Gb/s). In some examples, carrier accumulation uses a capacitor to store charge, where the capacitor includes a thin vertical insulating layer. For some semiconductor processes, it may be difficult to fabricate a thin vertical insulating layer. Thus, for certain semiconductor processes, a forward biased PIN diode structure for achieving carrier injection may be easier to manufacture.

Carrier injection provides both small size and low power characteristics, which are useful in large-scale photovoltaic computing systems with dense arrays of modulators. As described in more detail below, using a segmented modulator design for multi-bit modulation (e.g., PAM modulation) in combination with a carrier-injected PIN diode structure, bandwidth enhancement techniques can be used to overcome bandwidth limitations (e.g., increase bandwidth by a factor of about 10) with a simple, compact design that fits the dense arrangement of modulators in the system. Since the resulting enhancement bandwidth does not necessarily need to be as high as the bandwidth achievable by a carrier-depleting or carrier-accumulating modulator, the combination of simple bandwidth enhancement achieved by the segmented modulator with the compact size of the carrier-injecting modulator synergistically provides advantages that are particularly useful in some system embodiments described herein. Some forms of bandwidth enhancement, such as pre-emphasis, may be difficult to achieve when there are more than two amplitude levels. However, as described in more detail below, with a segmented modulator, separate binary pre-emphasis can be implemented for the two amplitude levels of each bit of the multi-bit modulation.

Furthermore, since the calculation is performed using a modulated optical signal, the number of analog amplitude levels of the calculated result (e.g., an 8-bit signal modulated using PAM-256) sampled when converted to a digital signal may be greater than the number of levels of the input to the calculation (e.g., a 4-bit signal modulated using PAM-16). For example, multiplication of a 4-bit value with another 4-bit value may result in a value suitable for representation in 8 bits. Therefore, in the case where the resolution needs to be improved after the input signal is processed by the optoelectronic computing system, bandwidth enhancement is promoted to be used to improve the signal to noise ratio (signal to noise ratio) because efficient analog to digital conversion is required.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In case of conflict with a patent application or patent application publication incorporated by reference herein, the present disclosure, including definitions, will control.

The details of one or more embodiments of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the disclosure will become apparent from the description, the drawings, and the claims.

Drawings

The disclosure is best understood from the following detailed description when read with the accompanying drawing figures. It is emphasized that, according to common practice, the various features of the drawing are not to scale. On the contrary, the dimensions of the various features are arbitrarily expanded or reduced for clarity.

FIG. 1 is a schematic diagram of an example of an Artificial Neural Network (ANN) computing system.

FIG. 2 is a schematic diagram of an example of an MZI.

Fig. 3 is a diagram of an example of a charge-pump bandwidth boosting circuit.

Fig. 4 is a flow chart illustrating an example of a method for performing an ANN calculation.

FIG. 5 is a diagram illustrating one aspect of the method of FIG. 4.

Fig. 6 is a schematic diagram of an example of a wavelength division multiplexing ann (wavelength division multiplexed ann) computing system.

Fig. 7 is a diagram of an example of a Mach-Zehnder (Mach-Zehnder) modulator.

Fig. 8 is a diagram of an example of a mach-zehnder modulator using a segmented design.

Fig. 9 is a diagram of the mach-zehnder modulator of fig. 8 and a corresponding drive circuit.

Fig. 10 to 16 are diagrams illustrating a mach-zehnder modulator.

Fig. 17 is a timing diagram for the charge pumping bandwidth enhancement circuit of fig. 3.

Fig. 18 is a graph showing an intensity-voltage curve of the mach-zehnder modulator of fig. 7.

FIG. 19 is a schematic diagram of an example optoelectronic computing system.

Fig. 20 and 21 are schematic diagrams of example system configurations.

Fig. 22 is an explanatory diagram of an example of a symmetric differential configuration (symmetric differential configuration).

Fig. 23 and 24 are circuit diagrams of examples of the system module.

Fig. 25 is a schematic diagram of an example of a symmetric differential configuration.

Fig. 26 is a schematic diagram of an example of the system configuration.

Fig. 27 is a schematic diagram of an example optical amplitude modulator.

Fig. 28 to 30 are schematic diagrams of examples of optical amplitude modulators using optical detection in a symmetric difference distribution configuration.

Fig. 31 to 33 are photoelectric circuit diagrams of example system configurations.

34-38 are schematic diagrams of example computing systems using multiple optoelectronic systems.

Fig. 39 is a schematic diagram of an example of a wavelength division multiplexing ANN computing system using an optoelectronic processor.

Fig. 40 and 41 are schematic diagrams of examples of wavelength division multiplexing photoelectric matrix multiplication units.

FIG. 42 is a schematic diagram of a homodyne detector.

FIG. 43 is a schematic diagram of a computing system including optical fibers, each carrying a signal having a plurality of wavelengths.

Fig. 44 is a schematic diagram of an example of an optical matrix multiplication unit.

Fig. 45 and 46 are schematic diagrams of example configurations of interconnected mach-zehnder interferometers (MZIs).

Fig. 47 is a schematic diagram of a segmented MZI modulator including a calibration phase shifter.

Fig. 48 is a schematic diagram of the segmented MZI modulator of fig. 47 and associated drive circuitry.

Fig. 49 shows a schematic diagram of a segmented MZI modulator including a calibration phase shifter.

Fig. 50 shows an example of a modulator array having optical modulators of a segmented design.

FIG. 51 is a schematic diagram of an example of an Artificial Neural Network (ANN) computing system.

Fig. 52 is a schematic diagram of an example of a wavelength division multiplexing photoelectric matrix multiplication unit.

Fig. 53 is a flowchart illustrating an example of a method for performing an ANN calculation.

Fig. 54 is a diagram of an example of a ring resonator (ring resonator) modulator having a segmented design.

Like reference numbers and designations in the various drawings indicate like elements.

Detailed Description

Fig. 1 shows a schematic diagram of an example of an Artificial Neural Network (ANN) computing system 100. The system 100 includes a controller 110, a memory unit 120, a Modulator Control (MC) unit 130, a photo processor 140, and an analog-to-digital conversion (ADC) unit 160. The controller 110 is coupled to the computer 102, the memory unit 120, the MC unit 130, and the ADC unit 160. The controller 110 includes an integrated circuit configured to control the operation of the ANN computing system 100 to perform ANN computations.

The integrated circuit of the controller 110 may be an application specific integrated circuit specifically configured to perform the steps of the ANN calculation process. For example, the integrated circuit may implement microcode or firmware specific to performing ANN computation processing. As such, the controller 110 may have a reduced instruction set relative to general-purpose processors used in conventional computers (e.g., the computer 102). In some embodiments, the integrated circuit of the controller 110 may include two or more circuits configured to perform different steps of the ANN calculation process.

In an example operation of the ANN computing system 100, the computer 102 may issue an artificial neural network computation request to the ANN computing system 100. The ANN calculation request may include neural network weights defining the ANN, and an input data set processed by the provided ANN. The controller 110 receives the ANN calculation request and stores the input data set and the neural network weights in the memory unit 120.

The input data set may correspond to various digital information to be processed by the ANN. Examples of input data sets include image files, audio (audio) files, LiDAR (LiDAR) point clouds, biometric data files, and GPS coordinate sequences, and the operation of the ANN computing system 100 will be described based on receiving the image files as input data sets. Typically, the size of the input data set can vary widely, from hundreds of data points (data points) to millions of data points or more. For example, a digital image file having a 1 million pixels (megapixel) resolution has approximately one million pixels, and each of the one million pixels may be a data point processed by the ANN. Due to the large number of data points in a typical input data set, the input data set is typically divided into a plurality of digital input vectors of smaller size to be processed separately by the photoelectric processor 140. As an example, for a gray-scale digital image (greyscale digital image), the elements of the digital input vector may be 8-bit values representing the image intensity, and the digital input vector may have a length ranging from tens of elements (e.g., 32 elements, 64 elements) to hundreds of elements (e.g., 256 elements, 512 elements). In general, an input data set of arbitrary size may be divided into digital input vectors of a size suitable for processing by the photo-electric processor 140. In case the number of elements of the input data set is not evenly divisible by the length of the digital input vector, zero padding (zero padding) may be used to pad the data set such that it is evenly divisible by the length of the digital input vector. The processed output of each digital input vector may be processed to reconstruct the complete output, which is the result of processing the input data set through the ANN. In some embodiments, the splitting of the input data set into a plurality of input vectors and subsequent vector-level processing may be implemented using block matrix multiplication techniques (block matrix multiplication techniques).

Neural network weights are a set of values that define the connectivity (connectivity) of the artificial neurons of the ANN, including the relative importance or weight of those connections. The ANN may include one or more hidden layers with corresponding node sets. In the case of an ANN with a single hidden layer, the ANN may be defined by two sets of neural network weights, one set corresponding to connectivity between the input nodes and the nodes of the hidden layer, and a second set corresponding to connectivity between the hidden layer and the output nodes. The neural network weights describing each set of connectivity correspond to a matrix to be implemented by the optoelectronic processor 140. For ANN's with two or more hidden layers, an additional set of neural network weights are needed to define the connectivity between the additional hidden layers. As such, in a typical case, the neural network weights included in the ANN calculation request may include a set of multiple neural network weights that represent connectivity between respective layers of the ANN.

Since the input data set to be processed is typically divided into a plurality of smaller digital input vectors for separate processing, the input data set is typically stored in a digital memory. However, the speed of memory operations between the memory and the processor of the computer 102 is significantly slower than the rate at which the ANN computing system 100 can perform ANN calculations. For example, the ANN computing system 100 may perform tens to hundreds of ANN computations during a typical memory read cycle of the computer 102. As such, during the process of processing the ANN calculation request, if the ANN calculation of the ANN calculation system 100 involves multiple data transfers between the system 100 and the computer 102, the rate of ANN calculations that can be performed by the ANN calculation system 100 may be limited below its full processing rate. For example, if the computer 102 were to access an input data set from its own memory and provide a digital input vector to the controller 110 upon request, the operation of the ANN computing system 100 may be significantly slowed by the time required for the series of data transfers required between the computer 102 and the controller 110. Notably, the memory access latency (latency) of the computer 102 is typically non-deterministic, which further complicates and slows the speed at which the digital input vector may be provided to the ANN computing system 100. Furthermore, processor cycles of the computer 102 may be wasted in managing data transfers between the computer 102 and the ANN computing system 100.

Rather, in some embodiments, the ANN computing system 100 stores the entire input data set in the memory unit 120, the memory unit 120 being part of the ANN computing system 100 and dedicated to the ANN computing system 100. The dedicated memory unit 120 allows transactions (transactions) between the memory unit 120 and the controller 110 to be particularly suitable for allowing a smooth and uninterrupted data flow between the memory unit 120 and the controller 110. This uninterrupted data flow can significantly improve the overall throughput of the ANN computing system 100 by allowing the electro-optical processor 140 to perform matrix multiplication at its full processing rate without being limited by the slow memory operations of a conventional computer (e.g., computer 102). Furthermore, because all of the data needed in performing the ANN calculation is provided by the computer 102 to the ANN computing system 100 in a single transaction, the ANN computing system 100 is able to perform its ANN calculation in an exclusive manner independent of the computer 102. This unique operation of the ANN computing system 100 reduces the computational burden on the computer 102 and eliminates external dependencies in the operation of the ANN computing system 100, improving the performance of both the system 100 and the computer 102.

Example embodiments of the photo processor 140 and the MC unit 130 will now be described. In some embodiments, the optoelectronic processor 140 includes a laser unit 142, a modulator array 144, an Optoelectronic Matrix Multiplication (OMM) unit 150, and an analog electronics unit 146. The modulator array 144 modulates the optical waves received from the laser unit 142 to provide an optical input vector of length N based on the encoded input data (e.g., digital input data) provided by the MC unit 130, which propagates to the OMM unit 150. The OMM unit 150 receives the optical input vector of length N and performs matrix multiplication on the received optical input vector in the optical domain. The matrix multiplication may be an N × N matrix multiplication determined by the internal configuration of the OMM unit 150. The internal configuration of the OMM unit 150 may be controlled by electrical signals, such as those generated by the MC unit 130.

The OMM unit 150 may be implemented in various ways. For example, the optical modulation used by the modulator array 144 and/or the OMM unit 150 may include a plurality of interconnected Mach-Zehnder interferometers (MZIs). FIG. 2 shows a schematic diagram of an example of MZI 170. The MZI 170 includes a first input waveguide 171, a second input waveguide 172, a first output waveguide 178, and a second output waveguide 179. Further, each MZI 170 of the plurality of interconnected MZIs includes a first phase shifter 174 in at least one of the arms, the first phase shifter 174 configured to change a splitting ratio (splitting ratio) of the MZI 170, and in some examples may include phase shifters in both arms, the phase shifters configured to perform a push-pull operation. In some embodiments, the second phase shifter 176 is configured to shift the phase of one output of the MZI 170, such as the light exiting the MZI 170 through a second output waveguide 179. The first phase shifter 174 and the second phase shifter 176 of the MZI 170 are coupled to a plurality of weight control signals generated by the MC unit 130. The first phase shifter 174 and the second phase shifter 176 are examples of reconfigurable elements of the OMM unit 150. Examples of the reconfiguration element include a thermo-optical phase shifter (thermo-optical phase shifter) or an electro-optical phase shifter (electro-optical phase shifter). Thermo-optic phase shifters operate by heating the waveguide to change the refractive index of the waveguide and cladding material, which translates into a change in phase. The electro-optic phase shifter operates by applying an electric field (e.g., lithium niobate (LiNbO3), reverse-biasing a PN junction) or a current (e.g., forward-biasing a PIN junction), which changes the refractive index of the waveguide material. By changing the weight control signals, the phase delay of the first 174 and second 176 phase shifters of each interconnected MZI 170 may be changed, which reconfigures the optical interference unit 154 of the OMM unit 150 to achieve a particular matrix multiplication determined by the phase delay disposed over the entire optical interference unit 154. Additional embodiments of the OMM unit 150 AND OPTICAL interference unit 154 are disclosed in U.S. patent publication No. US 2017/0351293A1 entitled "APPARATUS AND METHODS FOR OPTICAL NEURAL NETWORKS", U.S. patent publication No. US 20190370652A1 entitled "OPTOELECTRONIC COMPUTING SYSTEMS", U.S. patent publication No. US 20200110992A1 entitled "OPTOELECTRONIC COMPUTING SYSTEMS", AND PCT publication No. WO 2020191217A 1 entitled "OPTOELECTRONIC COMPUTING SYSTEMS", which are fully incorporated herein by reference.

An optical input vector is generated by the laser unit 142 and the modulator array 144. The length-N optical input vector has N independent optical signals, each optical signal having an intensity corresponding to the value of a respective element of the length-N digital input vector. As an example, the laser unit 142 may produce N optical outputs. In this example, the N light outputs have the same wavelength and are coherent (coherent). The optical coherence of the light outputs allows the light outputs to optically interfere with each other, a characteristic that is exploited by the OMM unit 150 (e.g., in the operation of the MZI). Further, the optical outputs of the laser units 142 may be substantially the same as each other. For example, the N light outputs may be substantially uniform in their intensities (e.g., within 5%, within 3%, within 1%, within 0.5%, within 0.1%, or within 0.01%) and in their relative phases (e.g., within 10 degrees, within 5 degrees, within 3 degrees, within 1 degree, within 0.1 degrees). The uniformity of the light output may improve the fidelity (faithfull) of the optical input vector to the digital input vector, thereby improving the overall accuracy of the photo processor 140. In some embodiments, the light output of the laser unit 142 may have an optical power of 0.1 to 50mW each, a wavelength in the near infrared light range (e.g., between 900 nm and 1600 nm), and a linewidth of less than 1 nm. The optical output of the laser unit 142 may be a single transverse-mode (transverse-mode) optical output.

In some embodiments, the laser unit 142 includes a single laser source and an optical power splitter (optical power splitter). A single laser source is configured to generate laser light. The optical power splitter is configured to split light generated by the laser source into N optical outputs having substantially the same intensity and phase. By splitting a single laser output into multiple outputs, optical coherence of the multiple light outputs can be achieved. For example, the single laser source may be a semiconductor laser diode, a vertical-cavity surface-emitting laser (VCSEL), a Distributed Feedback (DFB) laser, or a Distributed Bragg Reflector (DBR) laser. For example, the optical power splitter may be 1: an N multimode interference (MMI) splitter comprising a plurality of 1: 2MMI splitter or multi-stage splitter of directional coupler (multi-stage splitter), or star coupler (star coupler). In some other embodiments, a master-slave laser configuration (master-slave laser configuration) may be used, in which the slave laser is injection locked (injection locked) by the master laser to have a stable phase relationship to the master laser.

The optical output of the laser unit 142 is coupled to a modulator array 144. The modulator array 144 is configured to receive an optical input from the laser unit 142 and modulate the intensity of the received optical input based on a modulator control signal (which is an electrical signal). Examples of the modulator include a mach-zehnder interferometer (MZI) modulator, a ring resonator modulator (ring resonator), and an electro-absorption modulator (electro-absorption modulator). For example, an electro-absorption modulator includes electrodes that apply an electric field to a waveguide segment to modulate the absorption spectrum of the waveguide segment, thereby modulating the intensity of light propagating in the waveguide segment. The modulator array 144 has N modulators, each modulator receiving one of the N optical outputs of the laser unit 142. The modulator receives control signals corresponding to elements of the digital input vector and modulates the intensity of the light. The control signal may be generated by the MC unit 130.

The MC unit 130 is configured to generate a plurality of modulator control signals and generate a plurality of weight control signals under the control of the controller 110. For example, MC unit 130 receives a first modulator control signal from controller 110, the first modulator control signal corresponding to a digital input vector to be processed by photo-electric processor 140. The MC unit 130 generates a modulator control signal, which is an analog signal suitable for driving the modulator array 144 and the OMM 150, based on the first modulator control signal. For example, the analog signal may be a voltage or a current, depending on the technology and design of the modulators of the array 144 and the OMM 150. The voltage may have an amplitude ranging from, for example, ± 0.1V to ± 10V, and the current may have an amplitude ranging from, for example, 100 μ a to 100 mA. In some embodiments, the MC unit 130 may include a modulator driver configured to buffer, amplify, or condition the analog signals so that the modulators of the array 144 and the OMM 150 may be fully driven. For example, certain types of modulators may be driven with differential control signals. In this case, the modulator driver may be a differential driver that generates a differential electrical output based on a single-ended (single-ended) input signal.

In some embodiments, the optical-to-electrical processor 140 may include some type of modulator having a 3dB bandwidth, which is less than the desired processing rate of the optical-to-electrical processor 140. In this case, the modulator driver may include a pre-emphasis circuit (pre-emphasis circuit) or other bandwidth enhancement circuit designed to extend the operating bandwidth of the modulator. For example, such bandwidth enhancement may be useful for modulators based on PIN diode structures that are forward biased to modulate the refractive index of a portion of a waveguide guiding a modulated optical wave using carrier injection. For example, if the modulator is an MZI modulator, a PIN diode structure may be used to implement a phase shifter in one or both waveguide arms of the MZI modulator. Configuring the phase shifter to operate in forward bias facilitates shorter modulator lengths and a more compact overall design. Either or both of modulator array 144 and OMM unit 150 may include a PIN diode structured modulator based on forward biasing to modulate using carrier injection. This helps make the modulator array 144 and/or the OMM unit 150 (which may have a large number of modulators) more compact.

In some embodiments, either or both of the modulator array 144 and the OMM unit 150 may use modulators that utilize a carrier accumulation design and incorporate MOS-type capacitor structures into the optical waveguide, such as MOS capacitor-based MZI modulators. The selection of which type of modulator to use, e.g., whether carrier injection or carrier accumulation is used, may be based in part on the overall semiconductor fabrication process used to fabricate the photo processor 140.

For example, in a pre-emphasis form (pre-emphasis form) with bandwidth enhancement, the analog electrical signal (e.g., voltage or current) driving the modulator may be shaped to include transient pulses (overshoots) that overshoot changes in analog signal levels representing modulation in a series of digital data values (digital data values)The controller controls a given digital data value of the signal. Each digital data value may have any number of bits, including a single 1-bit data value, as assumed for the remainder of this example. Thus, if the value of a bit is the same as the previous value, the analog electrical signal driving the modulator is maintained at a steady-state level (e.g., signal level X with a bit value of 0) ₀And an upper signal level X of bit value 1₁). However, if a bit changes from 0 to 1, the corresponding analog electrical signal used to drive the modulator may include transient pulses that stabilize at a steady-state value of X₁Previously, there was a peak X at the beginning of the bit transition (bit transition)₁+(X₁- X₀). Likewise, if a bit changes from 1 to 0, the corresponding analog electrical signal used to drive the modulator may include a transient pulse that stabilizes at a steady-state value X₀Previously, there was a peak X at the beginning of the bit transition₀+(X₀-X₁). The size and length of the temporal pulse may be selected to optimize bandwidth enhancement (e.g., to maximize the open area of the eye diagram of the non-return-to-zero (NRZ) modulation mode).

In a bandwidth enhanced charge pumping version, the analog current signal driving the modulator can be shaped to include momentary pulses that move a precisely determined amount of charge. Fig. 3 illustrates an example implementation of a charge pumping bandwidth enhancement circuit 4416, the charge pumping bandwidth enhancement circuit 4416 using a capacitor connected in series between a voltage source and a modulator to precisely control charge flow. A portion of the circuit shown in fig. 3 may be included in the modulator driver described above. In this example, the modulator is represented by a modulator circuit 4400, which modulator circuit 4400 models the electrical characteristics of the phase shifter of the modulator as a PIN diode. The modulator circuit 4400 includes an ideal diode having a capacitance C _dAnd a resistor having a resistance R. The pumping capacitor (pump capacitor)4402 has a capacitance C_p. The control voltage waveform 4404 is provided to an inverter circuit 4405 to generate a drive voltage waveform 4406 whose amplitude can be accurately calibrated to pass through the pumping capacitorThe modulator 4402 moves a predetermined amount of charge into or out of the modulator circuit 4400. The PIN diode modeling the modulator circuit 4400 is forward biased by applying a constant voltage VDD _ IO at terminal 4408. A charge pumping control voltage VCP is applied at terminal 4410 of the inverter 4405 to control the amount of charge pumped at the transitions of the drive voltage waveform 4406, as well as the corresponding optical phase shift applied by the modulator.

The value of voltage VCP may be adjusted prior to operation such that the nominal charge Q stored in charge pumping capacitor 4402 is based on capacitance C_pThe measured value (e.g., due to uncertainty during manufacturing, there may be some variability) is accurately calibrated. For example, voltage VCP may be equal to nominal charge Q divided by capacitance C_p. The resulting change in the index of refraction of the portion of the waveguide that intersects the PIN diode may then provide a phase shift of the guided light wave that is associated with the PIN diode (e.g., via internal capacitance C) _dStorage) and the amount of charge Q moved between the charge pumping capacitor 4402 is linearly proportional. If the drive voltage changes from a low value to a high value, the current inflow from the charge pumping capacitor 4402 to the PIN diode will deliver a predetermined amount of charge (i.e., the integral of the positive current over time) in a short time. If the drive voltage changes from a high value to a low value, the current flow from the PIN diode to the charge pumping capacitor 4402 removes a predetermined amount of charge (i.e., the integration of the negative current over time) in a short period of time. After this relatively short switching time, a steady state current is provided by current source 4412, and current source 4412 is controlled by switch 4414 to replace the charge lost due to the internal capacitor losing current through internal resistance R while the drive voltage is held (e.g., during the hold time for a particular digital value). Using such a charge pumping configuration may have advantages such as better accuracy than other techniques, including some pre-emphasis techniques, because the amount of charge moved within a short switching time depends on a constant physical parameter (C)_p) And a steady state control Value (VCP) and is therefore precisely controllable and repeatable.

Other forms of bandwidth enhancement may also be used. For example, matching circuits may be used to shape the amplitude changes of the electrical signal applied to the semiconductor diode that modulates the light wave. The control electrical signal may be applied to the semiconductor diode through a matching circuit configured to match an impedance associated with the semiconductor diode without significantly changing the amplitude of the applied electrical signal (e.g., without introducing pre-emphasis/de-emphasis amplitude changes).

In some embodiments, the matching circuit is a passive circuit, such as a circuit consisting essentially of an inductor. For example, one end of the inductor is connected to a voltage source, and the other end of the inductor is connected to one end of the modulator circuit 4400. An advantage of the inductor is manufacturing consistency compared to passive matching circuits using RC networks such as resistors and capacitors. The design of a suitable RC network typically relies on cancellation between poles (poles) and their corresponding zeros (zeros) in a transfer function (transfer function) to approach the transfer function to 1, but the design tolerances required to achieve cancellation make the device more susceptible to manufacturing errors. For example, cancellation may require trimming (fine-tuning) the products of different resistance and capacitance values to be equal to each other. Conversely, by using an inductor with an appropriate inductance value (e.g., 1 to 10nH), matching can be achieved without such trimming, and thus manufacturing errors are more tolerable.

In some cases, the modulators of the array 144 and/or the OMM 150 may have non-linear transfer functions. For example, MZI optical modulators may have a non-linear relationship (e.g., sinusoidal dependence) between the applied control voltage and its transmission. In this case, the first modulator control signal may be adjusted or compensated based on the nonlinear transfer function of the modulator such that a linear relationship between the digital input vector and the generated optical input vector may be maintained. Maintaining such linearity is generally important to ensure that the input to the OMM unit 150 is an accurate representation of the digital input vector. In some embodiments, the compensation of the first modulator control signal may be performed by the controller 110 through a look-up table that maps the values of the digital input vector to the values to be output by the MC unit 130, such that the resulting modulated optical signal is linearly proportional to the elements of the digital input vector. The look-up table may be generated by characterizing the nonlinear transfer function of the (characterizing) modulator and calculating the inverse function of the nonlinear transfer function.

In some embodiments, the nonlinearity of the modulator and the resulting nonlinearity in the generated optical input vector may be compensated by an ANN calculation algorithm.

The optical input vectors generated by the modulator array 144 are input to the OMM unit 150. The optical input vector may be N spatially separated optical signals, each optical signal having an optical power corresponding to an element of the digital input vector. For example, the optical power of the optical signal is typically in the range of 1 μ W to 10 mW. The OMM unit 150 receives the optical input vector and performs matrix multiplication based on its internal configuration. The internal configuration is controlled by the electrical signals generated by the MC unit 130. For example, the MC unit 130 receives a second modulator control signal from the controller 110, the second modulator control signal corresponding to the neural network weights to be implemented by the OMM unit 150. The MC unit 130 generates a weight control signal, which is an analog signal adapted to control the reconfigurable meta-components within the OMM unit 150, based on the second modulator control signal. For example, the analog signal may be a voltage or a current, depending on the type of reconfiguration element of the OMM unit 150. The voltage may have an amplitude ranging from, for example, 0.1V to 10V, and the current may have an amplitude ranging from, for example, 100 μ a to 10 mA.

The modulator array 144 may operate at a modulation rate different from the reconfiguration rate of the reconfigurable OMM unit 150. The optical input vector produced by the modulator array 144 propagates through the OMM unit at an approximate proportion of the speed of light (e.g., 80%, 50%, or 25% of the speed of light), depending on the optical characteristics (e.g., effective index) of the OMM unit 150. For a typical OMM unit 150, the propagation time of the optical input vector is in the range of 1 to several 10 picoseconds, which corresponds to a processing rate of several 10 to several 100 GHz. As such, the rate at which the matrix multiplication operations can be performed by the optoelectronic processor 140 is limited in part by the rate at which optical input vectors can be generated. Modulators with bandwidths of several 10GHz are readily available, and modulators with bandwidths exceeding 100GHz are under development. As such, for example, the modulation rate of the modulator array 144 may be in the range of 5GHz, 8GHz, or several 10GHz to several 100 GHz. To maintain operation of modulator array 144 at such modulation rates, the integrated circuit of controller 110 may be configured to output control signals for MC unit 130 at a rate greater than or equal to, for example, 5GHz, 8GHz, 10GHz, 20GHz, 25GHz, 50GHz, or 100 GHz.

Depending on the type of reconfigurable elements implemented by the OMM unit 150, the reconfiguration rate of the OMM unit 150 may be significantly slower than the modulation rate. For example, the reconfigurable elements of the OMM unit 150 may be thermo-optic types that use micro-heaters to adjust the temperature of the optical waveguides of the OMM unit 150, which in turn affects the phase of the optical signals within the OMM unit 150 and results in matrix multiplication. Due to the thermal time constant associated with heating and cooling of the structure, the reconfiguration rate may be limited to, for example, several 100kHz to several 10 MHz. As such, the modulator control signals used to control the modulator array 144 and the weight control signals used to reconfigure the OMM unit 150 may have significantly different speed requirements. Furthermore, the electrical characteristics of the modulator array 144 may differ significantly from the electrical characteristics of the reconfigurable elements of the OMM unit 150.

To accommodate the different characteristics of the modulator control signals and the weight control signals, in some embodiments, the MC unit 130 may include a first MC subunit 132 and a second MC subunit 134. The first MC subunit 132 may be specifically configured to generate modulator control signals for the input vectors, and the second MC subunit 134 may be specifically configured to generate weight control signals for the matrix multiplication. For example, the modulation rate of the modulator array 144 may be 25GHz, and the first MC subunit 132 may have a per-channel output update rate of 25 gigasamples per second (GSPS) and a resolution of 8 bits or higher. The reconfiguration rate of the OMM unit 150 may be 1MHz, and the second MC subunit 134 may have an output update rate of 1 mega-samples per second (MSPS) and a resolution of 10 bits. Implementing separate first MC subunit 132 and second MC subunit 134 allows the MC subunits to be optimized independently for the respective signals, which may reduce overall power consumption, complexity, cost, or a combination thereof of MC unit 130. It is noted that although the first MC sub-unit 132 and the second MC sub-unit 134 are described as sub-elements of the MC unit 130, in general, the first MC sub-unit 132 and the second MC sub-unit 134 may be integrated on a common chip or may be implemented as separate chips.

Based on the different characteristics of first MC subcell 132 and second MC subcell 134, in some embodiments, memory cell 120 may include a first memory subcell and a second memory subcell. The first memory subunit may be a memory dedicated to storing the input data set and the digital input vector and may have an operating speed sufficient to support the modulation rate. The second memory subunit may be a memory dedicated to storing neural network weights and may have an operating speed sufficient to support a reconfiguration rate of the OMM unit 150. In some embodiments, the first memory sub-unit may be implemented using SRAM and the second memory sub-unit may be implemented using DRAM. In some embodiments, the first memory sub-unit and the second memory sub-unit may be implemented using DRAMs. In some embodiments, the first memory subunit may be implemented as part of the controller 110 or as a cache (cache) of the controller 110. In some embodiments, the first and second memory subunits may be implemented by a single physical memory device as different address spaces.

The OMM unit 150 outputs an output vector of length N, which corresponds to the result of an N × N matrix multiplication of the optical input vector and the neural network weights. In some embodiments, the output vector may comprise an electrical signal (e.g., a voltage or current), and in other embodiments, the output vector may comprise an optical signal. The OMM unit 150 is coupled to the analog electronics unit 146, the analog electronics unit 146 being configured to perform any analog electronic processing for embodiments in which the output vector is an optical output vector, and may also be configured to perform photoelectric conversion. For example, the analog electronics unit 146 may include an array of N photodetectors configured to absorb the optical signal and generate the photocurrent, and an array of N transimpedance amplifiers (transimpedance amplifiers) configured to convert the photocurrent to an output voltage. Alternatively, if an electrical output vector is received from the OMM unit 150, a photodetector and transimpedance amplifier may be present within the OMM unit 150. The bandwidth of the photodetectors and transimpedance amplifiers can be set based on the modulation rate of the modulator array 144. The photodetector may be formed of various materials based on the wavelength of the detected optical output vector. Examples of materials for photodetectors include germanium, silicon germanium alloys, and indium gallium arsenide (InGaAs).

The analog electronics unit 146 is coupled to the ADC unit 160. The ADC unit 160 is configured to convert the N electrical signals output from the analog electronics unit 146 into N digitized optical outputs, which are quantized digital representations of the output voltage. For example, the ADC unit 160 may be an N-channel ADC. The controller 110 may derive N digitized optical outputs from the ADC unit 160 corresponding to the optical output vector of the optical matrix multiplication unit 150. Controller 110 may form a length-N digital output vector from the N digitized optical outputs that corresponds to the result of an N × N matrix multiplication of the length-N input digital vector. In some embodiments, if analog electronic processing is not required and the OMM unit 150 provides an electrical output signal, the analog electronic unit 146 may be omitted and the OMM unit 150 may be directly connected to the ADC unit 160.

The various electrical components of the ANN computing system 100 may be integrated in various ways. For example, the controller 110 may be an application specific integrated circuit fabricated on a semiconductor die. Other electrical components (e.g., memory cell 120, MC cell 130, ADC cell 160, or a combination thereof) may be monolithically integrated on the semiconductor die on which controller 110 is fabricated. As another example, two or more electrical components may be integrated into a System-on-Chip (SoC). In an embodiment of the SoC, the controller 110, the memory unit 120, the MC unit 130, and the ADC unit 160 may be fabricated on respective dies, and the respective dies may be integrated on a common platform (e.g., an interposer) that provides electrical connections between integrated components. Such SoC approaches may allow for faster data transfer between electronic components of the ANN computing system 100, thereby increasing the operating speed of the ANN computing system 100 relative to approaches that separately place and route components on a Printed Circuit Board (PCB). Furthermore, SoC approaches may allow the use of different fabrication techniques optimized for different electrical components, which may improve the performance of the different components and reduce overall cost compared to monolithically integrated approaches. Although integration of the controller 110, the memory unit 120, the MC unit 130, and the ADC unit 160 has been described, in general, a subset of the components may be integrated, while other components are implemented as separate components for various reasons (e.g., performance or cost). For example, in some embodiments, memory unit 120 may be integrated with controller 110 as a functional block (functional block) within controller 110.

The various optical components of the ANN computing system 100 may also be integrated in various ways. Examples of the optical components of the ANN computing system 100 include a laser unit 142, a modulator array 144, an OMM unit 150, and photodetectors of an analog electronics unit 146. These optical components may be integrated in various ways to improve performance and/or reduce cost. For example, the laser unit 142, the modulator array 144, the OMM unit 150, and the photodetector may be monolithically integrated on a common semiconductor substrate as a Photonic Integrated Circuit (PIC). On photonic integrated circuits formed based on compound semiconductor material systems, such as group III-V compound semiconductors (e.g., indium phosphide (InP)), lasers, modulators (e.g., electro-absorption modulators), waveguides, and photodetectors may be monolithically integrated on a single die. This monolithic integration approach can reduce the complexity of aligning the inputs and outputs of various discrete optical components, which can require alignment accuracy ranging from sub-micron to several microns. As another example, the laser source of the laser unit 142 may be fabricated on a compound semiconductor die, while the optical power splitter of the laser unit 142, the modulator array 144, the OMM unit 150, and the photodetector of the analog electronics unit 146 may be fabricated on a silicon die. PICs fabricated on silicon wafers, which may be referred to as silicon photonics technology, typically have greater integration density, higher lithographic (litholographic) resolution, and lower cost relative to group III-V based PICs. This greater integration density may be beneficial in the manufacture of the OMM unit 150, since the OMM unit 150 typically includes 10 to 100 optical components, such as power splitters and phase shifters. Furthermore, the higher lithographic resolution of silicon photonics may reduce the manufacturing variation of the OMM unit 150, thereby improving the accuracy of the OMM unit 150.

The ANN computing system 100 may be implemented in various form factors. For example, the ANN computing system 100 may be implemented as a co-processor (co-processor) that plugs into a host computer (host computer). Such an ANN computing system 100 may have a form factor such as a pci express (pci express) card and communicate with host computers over a PCIe bus. The host computer may carry (host) a plurality of co-processor type ANN computing systems 100 and be connected to a computer 102 via a network. Embodiments of this type may be applicable to cloud data centers where server racks may be dedicated to processing ANN computing requests received from other computers or servers. As another example, the coprocessor type ANN computing system 100 may be directly plugged into the computer 102 that issued the ANN computation request.

In some embodiments, controller 110, memory unit 120, modulator control unit 130, ADC unit 160, and microprocessor may be monolithically integrated on a semiconductor die. In some embodiments, the controller 110, the memory unit 120, the modulator control unit 130, the ADC unit 160, the microprocessor, and the system main memory may be integrated as a system on chip. This allows, for example, the artificial neural network computing system 100 to be used in a portable device, such as a laptop computer, tablet computer, or mobile phone. The microprocessor may include, for example, a plurality of high performance processor cores, a plurality of high efficiency processor cores, a plurality of graphics processors, a plurality of electronic neural engine cores, a level 1 cache, and a level 2 cache. The microprocessor may use the electronic neural engine core to execute artificial neural network computational instructions optimized for conventional electronic neural engine cores and may use the ANN computing system 100 to execute artificial neural network computational instructions optimized for optical processing performed by the optoelectronic processor 140. The microprocessor may be, for example, a reduced instruction set computer or a complex instruction set computer. The operating system may be designed to take into account the ANN computing system 100, for example, powering on the ANN computing system 100 to perform certain tasks more suitable for execution by the ANN computing system 100, and powering off the ANN computing system 100 or placing the ANN computing system 100 in a standby mode when such tasks are not being performed, thereby achieving overall higher computing performance and lower power consumption.

In some embodiments, the ANN computing system 100 may be integrated onto a physical system that requires real-time ANN computing capability. For example, systems that rely heavily on real-time artificial intelligence tasks (real-time intelligence conference tasks), such as autonomous vehicles, autonomous drones (unmanned planes), object or face recognition security cameras, and various Internet of Things (IoT) devices, may benefit from having the ANN computing system 100 directly integrated with other subsystems of such systems. The ANN computing system 100 with direct integration may enable real-time artificial intelligence in devices with poor or no network connectivity and enhance the reliability and availability of mission-critical artificial intelligence systems.

Although the MC unit 130 and the ADC unit 160 are shown coupled to the controller 110, in some embodiments, the MC unit 130, the ADC unit 160, or both may alternatively or additionally be coupled to the memory unit 120. For example, a Direct Memory Access (DMA) operation of the MC unit 130 or the ADC unit 160 may reduce the computational burden on the controller 110 and reduce the latency of reading and writing to the memory unit 120, thereby further increasing the operating speed of the ANN computing system 100.

Fig. 4 shows a flow diagram of an example of a process 200 for performing an ANN calculation. The steps of process 200 may be performed by controller 110. In some embodiments, the respective steps of process 200 may be executed in parallel, combined, in a loop, or in any order.

At step 210, an Artificial Neural Network (ANN) computation request including an input data set and a first plurality of neural network weights is received. The input data set includes a first numeric input vector. The first digital input vector is a subset of the input data set. For example, it may be a sub-region of the image. The ANN calculation request may be generated by various entities (e.g., computer 102). The computer may include one or more of various types of computing devices, such as a personal computer, a server computer, a vehicle computer (vehicle computer), and a flight computer (flight computer). The ANN calculation request generally refers to an electrical signal informing or informing the ANN calculation system 100 that an ANN calculation is to be performed. In some embodiments, the ANN calculation request may be split into two or more signals. For example, a first signal may query the (query) ANN computing system 100 to check whether the system 100 is ready to receive the input data set and the first plurality of neural network weights. In response to an acknowledgement by the system 100, the computer may transmit a second signal comprising the input data set and the first plurality of neural network weights.

In step 220, the input data set and the first plurality of neural network weights are stored. Controller 110 may store the input data set and the first plurality of neural network weights in memory unit 120. Storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow flexibility in the operation of the ANN computing system 100, which may improve the overall performance of the system, for example. For example, the input data set may be divided into digital input vectors of set size and format by taking (retrieve) the desired portion of the input data set from the memory unit 120. Different portions of the input data set may be processed in various orders or shuffled (shredded) to allow various types of ANN calculations to be performed. For example, shuffling may allow matrix multiplication to be performed by block matrix multiplication techniques where the input and output matrix sizes are different. As another example, storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow for queuing of the plurality of ANN computation requests by the ANN computation system 100, which may allow the ANN computation system 100 to maintain operation at its full speed without periods of inactivity.

In some embodiments, the input data set may be stored in a first memory subunit and the first plurality of neural network weights may be stored in a second memory subunit.

In step 230, a first plurality of modulator control signals is generated based on the first digital input vector and a first plurality of weight control signals is generated based on the first plurality of neural network weights. The controller 110 may send a first modulator control signal to the MC unit 130 to generate a first plurality of modulator control signals. MC unit 130 generates a first plurality of modulator control signals based on the first modulator control signals and modulator array 144 generates an optical input vector that represents a first digital input vector.

The first modulator control signal may include a plurality of digital values to be converted by the MC unit 130 into a first plurality of modulator control signals. The plurality of digital values generally corresponds to the first digital input vector and may be associated by various mathematical relationships or look-up tables. For example, the plurality of digital values may be linearly proportional to the values of the elements of the first digital input vector. As another example, the plurality of digital values may be associated with elements of the first digital input vector through a lookup table configured to maintain a linear relationship between the digital input vector and the optical input vector generated by the modulator array 144.

The controller 110 may send a second modulator control signal to the MC unit 130 to generate a first plurality of weight control signals. The MC unit 130 generates a first plurality of weight control signals based on the second modulator control signals, and the OMM unit 150 is reconfigured according to the first plurality of weight control signals to implement a matrix corresponding to the first plurality of neural network weights.

The second modulator control signal may include a plurality of digital values to be converted into the first plurality of weight control signals by the MC unit 130. The plurality of digital values generally correspond to the first plurality of neural network weights and may be associated by various mathematical relationships or look-up tables. For example, the plurality of digital values may be linearly proportional to the first plurality of neural network weights. As another example, the plurality of digital values may be calculated by performing various mathematical operations on the first plurality of neural network weights to generate weight control signals, which may configure the OMM unit 150 to perform matrix multiplication corresponding to the first plurality of neural network weights.

In step 240, a first plurality of digitized optical outputs corresponding to optical output vectors of the optical matrix multiplication unit is obtained. The optical input vectors produced by the modulator array 144 are processed by the OMM unit 150 and converted into optical or electrical output vectors. If the output vector is an optical output vector, the optical output vector is detected by the analog electronics unit 146 and converted to an electrical signal, which may be converted to a digitized value by the ADC unit 160. The controller 110 may, for example, send a conversion request to the ADC unit 160 to begin converting the voltage output by the analog electronics unit 146 to a digitized optical output. Once the conversion is completed, the ADC unit 160 may send the conversion result to the controller 110. Alternatively, the controller 110 may retrieve the conversion result from the ADC unit 160. The controller 110 may form a digital output vector from the digitized optical output, the digital output vector corresponding to a result of a matrix multiplication of the input digital vector. For example, the digitized optical outputs may be organized or concatenated (concatenated) to have a vector format.

In some embodiments, ADC unit 160 may be set or controlled to perform ADC conversion based on the modulator control signal being issued by controller 110 to MC unit 130. For example, the ADC conversion may be set to start at a preset time after the MC unit 130 generates the modulation control signal. Such control of the ADC conversion can simplify the operation of the controller 110 and reduce the number of necessary control operations.

In step 250, a non-linear transformation is performed on the first digital output vector to produce a first transformed digital output vector. The nodes or artificial neurons of the ANN operate by first performing a weighted sum of the signals received from the nodes of the previous layer, and then performing a nonlinear transformation ("activation") of the weighted sum to produce an output. Various types of ANN may implement various types of differentiable nonlinear transformations. Examples of non-linear transformation functions include modified linear units (restified li)A near unit; RELU) function, S-type (Sigmoid) function, hyperbolic tangent function (hyperbolic tangent function), X²A function and a | X | function. This non-linear transformation is performed on the first digital output by the controller 110 to produce a first transformed digital output vector. In some embodiments, the non-linear transformation may be performed by a dedicated digital integrated circuit within the controller 110. For example, controller 110 may include one or more modules or circuit blocks specifically adapted to accelerate the calculation of one or more types of non-linear transformations.

In step 260, the first transformed digital output vector is stored. The controller 110 may store the first transformed digital output vector in the memory unit 120. In the case where the input data set is divided into a plurality of digital input vectors, the first transformed digital output vector corresponds to the result of an ANN calculation of a portion of the input data set (e.g., the first digital input vector). As such, storing the first transformed digital output vector allows the ANN computing system 100 to perform and store additional computations on other digital input vectors of the input data set to be later aggregated into a single ANN output.

In step 270, an artificial neural network output generated based on the first transformed digital output vector is output. The controller 110 generates an ANN output that is a result of processing the input data set through an ANN defined by the first plurality of neural network weights. Where the input data set is divided into a plurality of digital input vectors, the resultant ANN output is an aggregated output comprising the first converted digital output, but may further comprise additional converted digital outputs corresponding to other portions of the input data set. Once the ANN output is generated, the generated output is sent to the computer (e.g., computer 102) that initiated the ANN computation request.

Various performance metrics may be defined for the ANN computing system 100 implementing the process 200. Defining the performance index may allow the performance of the ANN computation system 100 implementing the optoelectronic processor 140 to be compared with the performance of other systems for ANN computation that instead implement an electronic matrix multiplication unit. In one aspect, the rate at which the ANN calculations may be performed may be indicated in part by a first cycle period defined as the time elapsed between the step 220 of storing the input data set and the first plurality of neural network weights in the memory unit and the step 260 of storing the first transformed digital output vector in the memory unit. Thus, the first cycle period includes the time it takes to convert the electrical signal to an optical signal (e.g., step 230), perform a matrix multiplication in the optical domain, and convert the result back to the electrical domain (e.g., step 240). Both steps 220 and 260 involve storing data into the memory unit 120, a step shared between the ANN computing system 100 and a conventional ANN computing system without the photo processor 140. As such, measuring the first cycle period of the memory-to-memory transaction time may allow for an actual or fair comparison of the ANN computation throughput between the ANN computation system 100 and an ANN computation system without the optoelectronic processor 140 (e.g., a system implementing an electrical matrix multiplication unit).

Because of the rate at which the modulator array 144 may produce optical input vectors (e.g., at 25GHz) and the processing rate of the OMM unit 150 (e.g., >100GHz), the first cycle period of the ANN computation system 100 to perform a single ANN computation of a single digital input vector may be close to the inverse of the speed of the modulator array 144 (e.g., 40 ps). The first cycle period may be, for example, less than or equal to 100ps, less than or equal to 200ps, less than or equal to 500ps, less than or equal to 1ns, less than or equal to 2ns, less than or equal to 5ns, or less than or equal to 10ns, after taking into account the delay associated with the signal generation of the MC unit 130 and the ADC conversion of the ADC unit 160.

In comparison, the multiplication times of the M × 1 vector and the M × M matrix of an electrical matrix multiplication unit are usually equal to M²Proportional to-1 processor clock cycle. For M-32, this multiplication would take about 1024 cycles, which results in a run time of over 300ns at 3GHz clock speed, which is orders of magnitude slower than the first cycle period of the ANN computing system 100.

In some embodiments, process 200 further includes the step of generating a second plurality of modulator control signals based on the first transformed digital output vector. In some types of ANN calculations, a single digital input vector may be repeatedly propagated through or processed by the same ANN. An ANN that implements multi-pass processing may be referred to as a Recurrent Neural Network (RNN). The RNN is a neural network in which the output of the network is recycled back to the input of the neural network during the (k) th pass through the neural network and used as input during the (k +1) th pass. The RNN may have various applications in pattern recognition tasks, such as speech or handwriting recognition. Once the second plurality of modulator control signals are generated, process 200 may proceed to steps 240 through 260 to complete the second pass of the first digital input vector through the ANN. Generally speaking, recycling of the transformed digital output into the digital input vector may be repeated for a predetermined number of cycles, depending on the characteristics of the RNN received in the ANN calculation request.

In some embodiments, process 200 further includes the step of generating a second plurality of weight control signals based on the second plurality of neural network weights. In some cases, the artificial neural network computation request further includes a second plurality of neural network weights. Typically, an ANN has one or more hidden layers in addition to the input and output layers. For an ANN having two hidden layers, the second plurality of neural network weights may correspond to connectivity between a first layer of the ANN and a second layer of the ANN. To process the first digital input vector through the two hidden layers of the ANN, the first digital input vector may first be processed according to process 200 until step 260, wherein the result of processing the first digital input vector through the first hidden layer of the ANN at step 260 is stored in memory unit 120. The controller 110 then reconfigures the OMM unit 150 to perform a matrix multiplication corresponding to a second plurality of neural network weights associated with a second hidden layer of the ANN. Once the OMM unit 150 is reconfigured, the process 200 may generate a plurality of modulator control signals based on the first transformed digital output vector, which generates an updated optical input vector corresponding to the output of the first hidden layer. The updated optical input vector is then processed by the reconfigured OMM unit 150, the OMM unit 150 corresponding to the second hidden layer of the ANN. In general, the steps described may be repeated until the digital input vector has been processed through all hidden layers of the ANN.

As described above, in some embodiments of the OMM unit 150, the reconfiguration rate of the OMM unit 150 may be significantly slower than the modulation rate of the modulator array 144. In this case, the throughput of the ANN computing system 100 may be adversely affected by the amount of time it takes to reconfigure the OMM unit 150 during which ANN computations cannot be performed. To mitigate the effects of the relatively slow reconfiguration time of the OMM unit 150, batch processing techniques may be utilized in which two or more digital input vectors are propagated through the OMM unit 150 without configuration changes to amortize reconfiguration time over a larger number of digital input vectors.

Fig. 5 shows a diagram 290 illustrating aspects of process 200 of fig. 4. For an ANN with two hidden layers, instead of processing a first digital input vector through a first hidden layer, reconfiguring the OMM unit 150 for a second hidden layer, processing the first digital input vector (from a previous hidden layer) through the reconfigured OMM unit 150, and repeating the same operations for the remaining digital input vectors, all digital input vectors of the input data set may be processed first through the OMM unit 150 configured for the first hidden layer (configuration #1), as shown in the upper part of diagram 290. Once all digital input vectors have been processed by the OMM unit 150 with configuration #1, the OMM unit 150 is reconfigured to configuration #2, which corresponds to the second hidden layer of the ANN. This reconfiguration may be significantly slower than the rate at which the OMM unit 150 may process the input vectors. Once the OMM unit 150 is reconfigured for the second hidden layer, output vectors from previous hidden layers may be batched by the OMM unit 150. For large input data sets with tens or hundreds of thousands of digital input vectors, the impact of reconfiguration time may be reduced by approximately the same factors, which may significantly reduce the portion of time the ANN computing system 100 spends in reconfiguration.

To implement batch processing, in some embodiments, process 200 further includes the steps of: generating, by the MC unit, a second plurality of modulator control signals based on a second digital input vector; obtaining a second plurality of digitized optical outputs from the ADC unit corresponding to the optical output vector of the optical matrix multiplication unit, the second plurality of digitized optical outputs forming a second digital output vector; performing a non-linear transformation on the second digital output vector to produce a second transformed digital output vector; and storing the second transformed digital output vector in the memory unit. For example, generating the second plurality of modulator control signals may follow step 260. Furthermore, the ANN output of step 270 in this case is now based on both the first transformed digital output vector and the second transformed digital output vector. The acquiring, executing, and storing steps are similar to steps 240 through 260.

Batch processing techniques are one of many techniques for improving the throughput of the ANN computing system 100. Another technique for increasing the throughput of the ANN computing system 100 is by processing multiple digital input vectors in parallel using Wavelength Division Multiplexing (WDM). WDM is a technology of simultaneously propagating a plurality of optical signals of different wavelengths through a common propagation channel (e.g., a waveguide of the OMM unit 150). Unlike electrical signals, optical signals of different wavelengths can propagate through a common channel without affecting other optical signals of different wavelengths on the same channel. Furthermore, well-known structures such as optical multiplexers (multiplexers) and demultiplexers (demultiplexers) can be used to add (multiplex) or drop (demultiplex) optical signals from a common propagation channel.

In the context of the ANN computing system 100, multiple optical input vectors of different wavelengths may be independently generated, simultaneously propagated through the OMM unit 150, and independently detected to enhance the throughput of the ANN computing system 100. Referring to fig. 6, a schematic diagram of an example of a Wavelength Division Multiplexing (WDM) Artificial Neural Network (ANN) computing system 104 is shown. The WDM ANN computing system 104 is similar to the ANN computing system 100, unless described otherwise. To implement WDM techniques, in some embodiments of the ANN computing system 104, the laser unit 142 is configured to generate a plurality of wavelengths, such as λ 1, λ 2, and λ 3. The multiple wavelengths may preferably be separated by a wavelength spacing large enough to allow easy multiplexing and demultiplexing onto a common propagation channel. For example, wavelength intervals greater than 0.5nm, 1.0nm, 2.0nm, 3.0nm, or 5.0nm may allow for simple multiplexing and demultiplexing. On the other hand, the range between the shortest and longest wavelengths of the plurality of wavelengths ("WDM bandwidth") may preferably be small enough such that the characteristics or performance of the OMM unit 150 remain substantially the same over the plurality of wavelengths. Optical components are typically dispersive (meaning that their optical properties vary with wavelength). For example, the power split ratio of the MZI may vary with wavelength. However, by designing the OMM unit 150 to have a sufficiently large operating wavelength window (operating wavelength window), and by limiting the wavelengths within the operating wavelength window, the optical output vector output by the OMM unit 150 at each wavelength may be a sufficiently accurate result of the matrix multiplication implemented by the OMM unit 150. The operating wavelength window may be, for example, 1nm, 2nm, 3nm, 4nm, 5nm, 10nm, or 20 nm.

Fig. 7 shows a diagram of an example of a mach-zehnder modulator 3900 that may be used to modulate the amplitude of an optical signal. The mach-zehnder modulator 3900 includes two 1x 2-port multimode interference couplers (MMI _1x2)3902a and 3902b, two balanced arms (arm)3904a and 3904b, and a phase shifter 3906 in one arm (or one phase shifter in each arm). When a voltage is applied to the phase shifter in one arm through the signal line 3908, there will be a phase difference between the two arms 3904a and 3904b that will be converted to amplitude modulation. The 1 × 2-port multimode interference couplers 3902a and 3902b and the phase shifter 3906 are configured as broadband (broadband) photonic components, and the optical path lengths of the two arms 3904a and 3904b are configured to be equal. This enables the mach-zehnder modulator 3900 to operate over a wide range of wavelengths.

Fig. 8 shows a diagram of another example of a mach-zehnder modulator 3950 that may be used to modulate the amplitude of an optical signal. The mach-zehnder modulator 3950 includes two 1x 2-port multi-mode interference couplers (MMI _1x2)3952a and 3952b, two balanced arms 3954a and 3954b, and a set of phase shifters 3956a, 3956b, and 3956c on one arm (or a set of phase shifters on each arm). The modulator 3950 uses a segmented design to provide optical digital-to-analog conversion using a separate phase shifter for each bit of the digital input signal. For example, a digital input port may be configured to receive a series of digital input values in successive time intervals, each digital input value comprising three bits. For example, the digital input port is also configured to receive digital input values in parallel. Multiple phase shifters 3956a, 3956b, and 3956c may be implemented by forming different respective diode sections along the optical waveguide of arm 3954 a. In this example, the diode section of each respective phase shifter 3956a, 3956b, and 3956c includes a semiconductor diode and electrical contacts for applying an electrical signal to the semiconductor diode in a forward biased state to provide carrier injection to modulate the index of refraction of the waveguide segment corresponding to the phase shifter. This enables the refractive index of each diode segment to be modulated in response to the value of a different corresponding bit of the three-bit digital input value. The relative lengths of the diode sections are selected to correspond to the bit positions of each bit. Thus, in an example having three bits per digital input value, the most significant bit (most-significant bit) is used to apply a binary (0 or 1) modulation to the phase shifter 3956a of length 4L, the next bit is used to apply a binary (0 or 1) modulation to the phase shifter 3956b of length 2L, and the least significant bit (least-significant bit) is used to apply a binary (0 or 1) modulation to the phase shifter 3956c of length L. Together, the diode segments apply different respective modulation contributions to the light waves propagating through the optical waveguide portion to provide appropriate digital-to-analog conversion.

The phase shifter is an optical modulator. In this specification, the term "modulator" may refer to, for example, the entire modulator 3950, the phase shifter 3956, or an optical amplitude modulator, depending on the context.

Referring to fig. 9, in some embodiments, modulator 3950 includes a first ridge waveguide (rib waveguide)3964a in first arm 3954a and a second ridge waveguide 3964b in second arm 3954 b. The first ridge waveguide 3964a and the second ridge waveguide 3964b are formed on a semiconductor substrate (e.g., a silicon substrate). The phase shifter 3956a includes a heavily doped p + region 3960a and a heavily doped n + region 3960b, which have a length of 4L and are formed on the substrate on both sides of the first segment of the first ridge waveguide 3964 a. The p + region, the intrinsic (inrinsic) region between the p + and n + regions, and the n + region form a p + -i-n + (PIN) diode. A first electrode (anode) 3962a is electrically coupled to the p + region 3960a and a second electrode (cathode) 3962b is electrically coupled to the n + region 3960 b.

The first electrode 3962a and the second electrode 3962b may be driven by a driving circuit 3966a, the driving circuit 3966a being similar to the circuit shown in fig. 3. The drive circuit 3966a is configured to move a predetermined amount of charge to or from the phase shifter 3956a to control the corresponding optical phase shift applied by the phase shifter 3956 a. The driving circuit 3966a includes a capacitor C _{p_a}The pumping capacitor 4402 a. The control voltage waveform 4404a is provided to the inverter circuit 4405a to generate a drive voltage waveform 4406a, the amplitude of which can be precisely calibrated to move a predetermined amount of charge to or from the phase shifter 3956a via the pumping capacitor 4402 a. The PIN diode of phase shifter 3956a is forward biased by applying a constant voltage VDD _ IO at terminal 4408 a. The charge pumping control voltage VCP is applied to the terminal 4410a of the inverter 4405a to control the amount of charge pumped (pump) when the drive voltage waveform 4406a transitions (transitions), and the corresponding optical phase shift applied by the phase shifter 3956 a. For clarity of illustration, detailed connections between the respective drive circuits 3966 and the phase shifter 3956 are omitted from the drawing.

The phase shifter 3956b includes a heavily doped p + region 3968a and a heavily doped n + region 3968b, which has a length of 2L, and is formed on the substrate on both sides of the second segment of the first ridge waveguide 3964 a. The p + region, the intrinsic (inrinsic) region between the p + and n + regions, and the n + region form a p + -i-n + (PIN) diode. A third electrode (anode) 3970a is electrically coupled to the p + region 3968a and a fourth electrode (cathode) 3970b is electrically coupled to the n + region 3968 b. The third electrode 3970a and the fourth electrode 3970b may be driven by a driving circuit 3966b, which driving circuit 3966b is similar to driving circuit 3966 a. The control voltage waveform 4404b is provided to an inverter circuit to generate a drive voltage waveform whose amplitude can be precisely calibrated to move a predetermined amount of charge to or from the phase shifter 3956b via the pumping capacitor. The drive circuit 3966b is configured to move a predetermined amount of charge to or from the phase shifter 3956b to control the corresponding optical phase shift applied by the phase shifter 3956 b.

The phase shifter 3956c includes a heavily doped p + region 3972a and a heavily doped n + region 3972b, which have a length L, and are formed on the substrate on both sides of the third segment of the first ridge waveguide 3964 a. The p + region, the intrinsic (inrinsic) region between the p + and n + regions, and the n + region form a p + -i-n + (PIN) diode. A fifth electrode (anode) 3974a is electrically coupled to p + region 3972a, and a sixth electrode (cathode) 3974b is electrically coupled to n + region 3972 b. The fifth electrode 3974a and the sixth electrode 3974b may be driven by a driving circuit 3966c, the driving circuit 3966c being similar to the driving circuit 3966 a. The control voltage waveform 4404c is provided to an inverter circuit to generate a drive voltage waveform whose amplitude can be precisely calibrated to move a predetermined amount of charge to or from the phase shifter 3956c via the pumping capacitor. The drive circuit 3966c is configured to move a predetermined amount of charge to or from the phase shifter 3956c to control the corresponding optical phase shift applied by the phase shifter 3956 c.

The doped p + and n + regions of phase shifters 3956a, 3956b, and 3956c have lengths of 4L, 2L, and L, respectively. The driving circuits 4404a, 4404b, and 4404c supply binary signals representing 3-bit digital input values to the phase shifters 3956a, 3956b, and 3956c, and the phase shifters 3956a, 3956b, and 3956c impart (impart) optical phase shifts weighted according to a ratio of 4:2:1 to light propagating in the first, second, and third waveguide segments. For example, when a binary value of "1" is applied to phase shifter 3956a and phase shifter 3956c, the amount of optical phase shift imparted by phase shifter 3956a will be four times the amount of optical phase shift imparted by phase shifter 3956 c. Similarly, when a binary value of "1" is applied to phase shifter 3956b and phase shifter 3956c, the amount of optical phase shift imparted by phase shifter 3956b will be twice the amount of optical phase shift imparted by phase shifter 3956 c. In this way, the MZI 3950 converts the 3-bit digital input value into an analog output signal with 3-bit accuracy without using a digital-to-analog converter circuit; if MZI 3900 of FIG. 7 is used, it may be necessary to use a digital-to-analog converter circuit. Because high-speed digital-to-analog converters are difficult to implement, MZI 3950 with the segmented design allows digital electrical signals to be converted to analog optical signals at a faster rate than using MZI 3900.

As described above, in some examples of ANN calculation, such as when using batch processing techniques, the reconfiguration rate of the modulator array 144 may be faster than the reconfiguration rate of the OMM unit 150. For example, modulator array 144 may modulate optical waves received from laser unit 142 using a plurality of mach-zehnder modulators 3950 having a segmented design to provide an optical input vector of length N based on encoded input data (e.g., digital input data) provided by first MC subunit 132, and the optical input vector propagates to OMM unit 150. Since the reconfiguration rate of the OMM unit 150 may be slow, in some examples the OMM unit 150 includes a plurality of mach-zehnder modulators 3900 that do not use a segmented design. The circuit driving MZI 3900 may be simpler than the circuit driving multiple segments of MZI 3950. The OMM unit 150 may have a large number of MZIs, and thus using MZI 3900 in the OMM unit 150 may reduce the complexity of the driving circuitry in the OMM unit 150.

In some embodiments, the electro-optical processor 140 is configured to perform an ANN calculation that requires the OMM unit 150 to also be updated at a fast rate comparable to the rate of the modulator array 144. In this case, the OMM unit 150 may also use a plurality of MZI 3950 using a segmented design.

In some embodiments, modulator array 144 includes a plurality of MZIs 3900 that do not use a segmented design, and OMM unit 150 includes a plurality of MZIs 3950 that use a segmented design.

Referring to fig. 10, in some embodiments, a mach-zehnder modulator 3950 is used in an ANN system having two bits per digital input. In this example, arm 3954a is configured with a set of two phase shifters, including a first phase shifter 3956b of length 2L and a second phase shifter 3956c of length L.

Referring to fig. 11, in some embodiments, mach-zehnder modulator 3950 is used in an ANN system having four bits per digital input. In this example, arm 3954a is configured with a set of four phase shifters, including a first phase shifter 3956d of length 8L, a second phase shifter 3956a of length 4L, a third phase shifter 3956b of length 2L, and a fourth phase shifter 3956c of length L.

In some embodiments, each digital input value has N bits, and arm 3954a is configured as a set of N phase shifters, including a length of 2^N-1First phase shifter of length 2^N-2L second phase shifter, …, and nth phase shifter of length L.

In the examples shown in fig. 8 to 11, the phase shifters are substantially aligned in line. In some examples, the waveguide in arm 3954a may have one or more bends or flexures to reduce the overall length of MZI 3950.

In some embodiments, an MZI with a segmented design may have phase shifters on both arms (e.g., 3954a and 3954 b). FIG. 12 is a diagram of an example MZI 3980 including a set of two phase shifters on each of two arms. FIG. 13 is a diagram of an example MZI 3990 that includes a set of three phase shifters on each of two arms. FIG. 14 is a diagram of an example MZI 3995 including sets of four phase shifters on each of two arms.

Fig. 15 is a diagram showing an example of a circuit for driving the phase shifters of the MZI 3990 of fig. 13, the MZI 3990 including a set of three phase shifters on each arm. Arm 3964b includes three phase shifters 3956d, 3956e, and 3956f, respectively configured similarly to phase shifters 3956a, 3956b, and 3956 c. The phase shifters 3956d, 3956e, and 3956f are driven by driving circuits 3966d, 3966e, and 3966f, respectively. The driver circuits 3966d, 3966e, and 3966f are similar to the driver circuits 3966a, 3966b, and 3966c, respectively. The inputs to the driving circuits 3966a and 3966d are driven in a push-pull (push pull) manner so that the phase shifters 3956a and 3956d impart optical phase shifts complementary to each other. For example, if drive circuit 3966a drives phase shifter 3956a to impart a phase shift of θ to light propagating in arm 3964a, drive circuit 3966d drives phase shifter 3956d to impart a phase shift of- θ to light propagating in arm 3964 b. The inputs to the driving circuits 3966b and 3966e are driven in a push-pull manner so that the phase shifters 3956b and 3956e give optical phase shifts complementary to each other. Similarly, the inputs to the driving circuits 3966c and 3966f are driven in a push-pull manner so that the phase shifters 3956c and 3956f impart optical phase shifts complementary to each other. For clarity of illustration, detailed connections between the respective drive circuits 3966 and the phase shifter 3956 are omitted from the drawing.

In some embodiments, the number of phase shifters in arm 3954a is different from the number of bits in each digital input value. In this case, the diode segments apply different respective modulation contributions to the light waves propagating through the optical waveguide portion to provide partial digital-to-analog conversion. Additional circuitry may be used to perform the digital-to-analog conversion.

Referring to fig. 16, in some embodiments, a mach-zehnder modulator 4010 is configured to convert a 6-bit digital electrical signal into an analog optical signal having 6-bit accuracy. Mach-zehnder modulator 4010 is similar to mach-zehnder modulator 3950 in fig. 9 except that control voltage waveforms 4404a, 4404b, and 4404c are provided by 2-bit digital-to-analog converters 4012a, 4012b, and 4012c (collectively 4012), respectively. For clarity of illustration, the detailed connections between the charge pumping bandwidth enhancement circuit and the phase shifter are omitted from the figures. For example, using the same digital-to-analog conversion architecture, a 2-bit digital-to-analog converter (DAC) may operate faster than a 6-bit DAC. Thus, an ANN system using a 2-bit DAC 4012 to drive an MZI 4010 with three phase shifters 3956 may operate faster than an ANN system using a 6-bit DAC to drive an MZI 3900 without a segmented design. The MZI with a set of six phase shifters (each phase shifter driven by a corresponding drive circuit) on one arm is longer than the MZI with a set of three phase shifters. Thus, an ANN system that uses a 2-bit DAC 4012 to drive an MZI 4010 having three phase shifters 3956 can be shorter in length than an ANN system that uses a binary signal to drive an MZI that includes a set of six phase shifters on one arm.

In general, a module is required that converts an N-bit digital electrical signal to an analog optical signal with N-bit accuracy, a p-bit DAC can be used to drive an MZI with a set of q phase shifters on one arm, where p × q ═ N. The choice of p and q depends on the speed at which the MZI needs to be reconfigured and how much space is available to accommodate the multiple phase shifters of the MZI. A smaller p results in a faster reconfiguration rate and a smaller q results in a shorter MZI.

In some embodiments, the semiconductor diode is operated in a forward biased state to take advantage of the index modulation effect of carrier injection, which can be implemented using a relatively short optical path length (e.g., less than about 0.1mm, or less than about 0.5mm, or less than about 0.1 mm). This short optical path length enables the modulator to achieve a compact modulator arrangement within an integrated optical device that uses multiple MZI or other types of modulators.

In addition to interferometric optical modulators such as MZI 3950, non-interferometric optical modulators may also be used to implement a segmented design for digital-to-analog conversion. For example, an absorbing optical modulator may be used for one or more arms of the MZI along a single optical waveguide instead of a phase shifter. The fragmented design also facilitates the use of signal conditioning for bandwidth enhancement, such as pre-emphasis and/or de-emphasis, which can be implemented independently for each fragment.

Fig. 17 shows timing diagrams for pre-emphasis and de-emphasis for the charge pumping bandwidth enhancement circuit 4416 of fig. 3. In some embodiments, the inverter circuit 4405 is driven by input data (waveform 4404 shown in fig. 1700). Capacitor C_pThe bottom plate (bottom plate) of 4402 will track the output of the inverter 4405, i.e., the drive voltage waveform 4406 (as shown in fig. 1702), at a level between 0V and VCP. The drive voltage waveform 4406 will remain on the capacitor C as the waveform 4406 changes_pThe charge within 4402 pushes or pulls to modulate the diode. Capacitor C_pThe voltage at the top plate (top plate) of 4402 is shown in graph 1704. Capacitor C_pThe voltage waveform at the top plate of 4402 can swing between VDD _ IO-Von + VCp and VDD _ IO-Von-VCp, which can be much higher or lower than the voltage driven by the current mode DAC or current source 4412 only. Von is the turn-on voltage of the diode. Control of the amount of charge pumped to or pulled from modulator circuit 4400 via pumping capacitor 4402, as shown in diagram 1704The voltage waveform 4404 (fig. 3, 9, 16) is pre-emphasized or de-emphasized to quickly move the desired amount of charge to or from the modulator circuit 4400.

The term "pre-emphasis" refers to the aforementioned action of the charge pumping circuit 4416 via capacitor C_p4402 rapidly pump charge to the modulator circuit 4400 to charge the capacitor C_pThe waveform at the top plate of 4402 rises quickly (e.g., 1706) to a portion of the level above the steady state voltage. The term "de-emphasis" refers to the aforementioned in-operation charge pumping circuit 4416 via capacitor C_p4402 quickly remove charge from the modulator circuit 4400 to enable the capacitor C_pThe waveform at the top plate of 4402 drops rapidly (e.g., 1708) to a portion below the level of the steady state voltage.

The charge pumping bandwidth enhancement circuit 4416 has several advantages over conventional pre-emphasis circuits using voltage driving. For example, some pre-emphasis conventional circuits can only perform pre-emphasis, while the charge pump pass bandwidth enhancement circuit 4416 can perform both pre-emphasis and de-emphasis. For example, some conventional circuit pre-emphasis circuits using voltage driving require a higher voltage source from an external source and bias the modulator with half the supply voltage, or require complex boost (voltage boost) circuits. In contrast, the charge pumping bandwidth enhancement circuit 4416 may use a lower voltage source and does not require a complex boost circuit. Furthermore, since conventional pre-emphasis circuits are voltage driven, to ensure that the modulator is not driven at the target phase, the conventional circuits may need to implement an additional control phase to stop the emphasis circuit. In contrast, the charge pumping bandwidth enhancement circuit 4416 can move an accurate amount of charge to the capacitor C _p4402 or slave capacitor C_p4402 remove a precise amount of charge and can accurately control the voltage applied to the modulator, thus eliminating the need to implement additional control phases to stop the emphasis circuit.

Fig. 54 is a diagram of an example of a ring resonator (ring resonator) modulator 5400 with a segmented design. The ring resonator modulator 5400 includes a first optical waveguide 5402a, a second optical waveguide 5402b, and a closed loop optical waveguide 5402 c. Two phase shifters, including a first phase shifter 5404a and a second phase shifter 5404b, are provided to modulate optical waves propagating in the closed-loop optical waveguide 5402 c. The first phase shifter 5404a and the second phase shifter 5404b have different lengths. In this example, the length of the second phase shifter 5404b is twice that of the first phase shifter 5404 a. Phase shifters 5404a and 5404b apply different respective modulation contributions to light waves propagating through optical waveguide portions associated with phase shifters 5404a and 5404b to provide appropriate digital-to-analog conversion. The ring resonator modulator 5400 includes two phase shifters, which will be referred to as two-piece ring resonator modulators hereinafter.

For example, an input optical wave entering the optical waveguide 5402a from port a may propagate to port B of the optical waveguide 5402a or be coupled to port C of the optical waveguide 5402B through the ring waveguide 5402C. The modulation imparted by phase shifters 5404a and 5404B to the light wave propagating within annular waveguide 5402c modulates the amplitude of the light wave at port B of optical waveguide 5402 a. The two phase shifters 5404a and 5404b effectively perform 2-bit digital-to-analog conversion.

For example, the first phase shifter 5404a is driven by a first charge pumping bandwidth enhancement circuit 5406a and the second phase shifter 5404b is driven by a second charge pumping bandwidth enhancement circuit 5406 b. The charge pump bandwidth enhancement circuits 5406a and 5406b are similar to the charge pump bandwidth enhancement circuit shown in fig. 3. For clarity of illustration, some signal connections of the charge pumping bandwidth enhancement circuits 5406a and 5406b are omitted from the figure.

For example, the first charge pump bandwidth enhancement circuit 5406a is driven by the 2-bit DAC 5408a and the second charge pump bandwidth enhancement circuit 5406b is driven by the 2-bit DAC 5408 b. The combined use of the 2-bit DACs 5408a and 5408b and the two-segment ring resonator modulator 5400 allows a 4-bit digital input signal to be converted to an analog optical signal with 4-bit accuracy.

In some examples, there are n phase shifters that modulate optical waves propagating in the ring waveguide 5402c, which is referred to as an n-segment ring resonator modulator. If each phase shifter is driven by a charge pumping bandwidth enhancement circuit, which in turn is driven by an m-bit DAC (m ≧ 1), the combination of the m-bit DAC and the n-segment ring resonator modulator can convert an m × n-bit digital input signal to an analog optical signal with m × n-bit accuracy. In general, the smaller the number m, the faster the m-bit DAC operates and the faster the ring resonator modulator is reconfigured.

In some embodiments, ring resonator modulator 5400 may be used for modulator array 144 in fig. 1 and 6, and modulator array 3208 in fig. 39 and 51. In some embodiments, the ring resonator modulator 5400 may also be used for the OMM unit 150 of fig. 1 and 6, and the OMM unit 3520 of fig. 39 and 51. The number of m-bit DACs and the number of n-segment ring resonator modulators n may be selected based on a number of criteria, such as the desired reconfiguration frequency of the modulator array and the OMM unit, and the circuit complexity of driving the phase shifter.

In some embodiments, in optical modulators of non-segmented design driven by electrical DAC signals, a single phase shifter in the interferometric modulator may also utilize signal conditioning for bandwidth enhancement. For example, if the interferometric modulator is a ring resonator (e.g., ring resonator 2222 in FIG. 29), the small size of the ring may result in insufficient space to accommodate multiple segments of appropriate length. The ring resonator may be used as, for example, a modulator to provide wavelength selectivity, thereby avoiding the need for additional optical multiplexers and demultiplexers for other kinds of modulators.

When Pulse Amplitude Modulation (PAM) has more than two levels (i.e., for digital values having more than 2 bits), signal conditioning in non-fragmented designs may be more complex than in fragmented designs. In an interferometric modulator (e.g., the MZI modulator of FIG. 7), the optical interference portion of the modulator (e.g., coupler 3902b of MZI modulator 3900) provides a degree of destructive optical interference resulting in a predetermined reduction in amplitude based on the modulation of the refractive index of the phase shifter (e.g., 3906 of FIG. 7). For corresponding changes between successive digital input values in a series of digital input values, the electrical DAC signal driving the phase shifter may be processed using a circuit that shapes the amplitude change, e.g. using pre-emphasis for the initial amplitude and/or de-emphasis for the trailing amplitude. In a non-fragmented design, rather than applying a simple pre-emphasis/de-emphasis scheme to switch between two levels, there are more complex forms of pre-emphasis/de-emphasis for switching between 4 or more levels.

In some embodiments, non-linearity may occur in the modulator when different modulator input values are mapped to different resulting amplitude reductions of the modulator. Techniques exist for compensating for such non-linearities, but the non-linearity compensation process may be relatively slow and/or consume a relatively large amount of power. Techniques for faster and more power efficient pre-emphasis/de-emphasis signal conditioning with non-linear compensation can be achieved by combining different signal conditioning paths in forming an electrical signal that drives a single phase shifter in an interferometric modulator. A given number of bits (e.g., n bits) in a sequence of input values, or n-bit values in parallel, may be used for control 2ⁿEach of a single-level (single-level) DAC. For example, 2ⁿA single level DAC may be included in the first MC subunit 132 (if a MZI with a segmented design is used for the modulator array 144), the second MC subunit 134 (if a MZI with a segmented design is used for the OMM unit 150), or both (if a MZI with a segmented design is used for both the modulator array 144 and the OMM unit 150). Each single level DAC is configured to provide only 2 amplitudes ⁿAn electrical signal of one of the bit values and corresponding pre-and de-emphasis for the amplitude, the pre-and de-emphasis being appropriately adjusted for a given non-linearity of the modulator of the amplitude. The electrical signal providing the modulator input value is then selected from the appropriate one of the regulated single level DAC outputs.

For example, a first signal conditioning path from one of the single level DACs providing a given modulator input value at which the output can be appropriately scaled based on the known non-linearity of the modulator may be configured to provide an unregulated electrical signal corresponding to a series of digital input values driving the DAC, for example a current-steering DAC. The second signal conditioning path may be configured to provide delayed, scaled and/or inverted versions of the unconditioned electrical signal (to provide pre-emphasis). The third signal conditioning path may be configured to provide delayed, scaled and/or inverted versions of the unconditioned electrical signal (to provide de-emphasis). Any number of additional signal conditioning paths may be used, if desired, to provide the appropriate amount of pre-emphasis and/or de-emphasis when different versions of the unconditioned electrical signal are added to the unconditioned electrical signal. The resulting electrical signal, which has been scaled to compensate for the non-linearity, and which has been adjusted to provide the appropriate amount of pre-emphasis and de-emphasis, can then be used as a modulator input value to arrive at an equalized (equalized) and bandwidth enhanced optical signal having different optical amplitudes that are uniformly distributed over the 4 or more levels used.

FIG. 18 is a graph 3910 showing an intensity-voltage curve for a Mach-Zehnder modulator 3900 using the configuration shown in FIG. 7 for wavelengths 1530nm, 1550nm, and 1570 nm. Plot 3910 shows that mach-zehnder modulator 3900 has similar intensity-voltage characteristics for different wavelengths in the range of 1530nm to 1570 nm.

Referring back to fig. 6, the modulator array 144 of the WDM ANN computing system 104 includes a bank of optical modulators (optical modulators) configured to generate a plurality of optical input vectors, each of the optical modulator banks corresponding to one of the plurality of wavelengths and generating a respective optical input vector having a respective wavelength. For example, for a system having optical input vectors of lengths of 32 and 3 wavelengths (e.g., λ 1, λ 2, and λ 3), modulator array 144 may have 3 groups of 32 modulators each. In addition, the modulator array 144 also includes an optical multiplexer configured to combine the plurality of optical input vectors into a combined optical input vector that includes the plurality of wavelengths. For example, an optical multiplexer may combine the outputs of three modulator groups of three different wavelengths into a single propagation channel (e.g., waveguide) for each element of an optical input vector. As such, returning to the example above, the combined optical input vector will have 32 optical signals, each signal comprising 3 wavelengths.

In addition, the analog electronics unit 146 of the WDM ANN computing system 104 is further configured to demultiplex the plurality of wavelengths and generate a plurality of demultiplexed output voltages. For example, the analog electronics unit 146 may include a demultiplexer configured to demultiplex three wavelengths included in each of the 32 signals of the multi-wavelength optical output vector and route (route) 3 single-wavelength optical output vectors to three groups of photodetectors coupled to the three groups of transimpedance amplifiers.

Further, the ADC unit 160 of the WDM ANN computing system 104 includes a bank of ADCs configured to convert the plurality of demultiplexed output voltages of the analog electronics unit 146. Each group of ADCs corresponds to one of the plurality of wavelengths and produces a corresponding digitized demultiplexed optical output. For example, the set of ADCs may be coupled to a set of transimpedance amplifiers of the analog electronics unit 146.

Controller 110 may implement a method similar to process 200, but extended to support multi-wavelength operation. For example, the method may comprise the steps of: obtaining a plurality of digitized demultiplexed optical outputs from the ADC unit 160, the plurality of digitized demultiplexed optical outputs forming a plurality of first digital output vectors, wherein each of the plurality of first digital output vectors corresponds to one of the plurality of wavelengths; performing a non-linear transformation on each of the plurality of first digital output vectors to produce a plurality of transformed first digital output vectors; and storing the plurality of transformed first digital output vectors in a memory unit.

In some examples, the ANN may be specifically designed and the digital input vector may be specifically formed such that the multi-wavelength optical output vector may be detected without demultiplexing. In such an example, the analog electronics unit 146 may be a wavelength-insensitive (wavelegnth-insensitive) detection unit that does not demultiplex multiple wavelengths of the multi-wavelength optical output vector. As such, each photodetector of the analog electronics unit 146 effectively adds multiple wavelengths of the optical signal to a single photocurrent, and each voltage output by the analog electronics unit 146 corresponds to an element-by-element sum (element-by-element sum) of the matrix multiplication results of multiple digital input vectors.

Up to now, the non-linear transformation of the weighted sum performed as part of the ANN calculation is performed in the digital domain by the controller 110. In some examples, the non-linear transformation may be computationally intensive or power consuming, significantly enhancing the complexity of the controller 110, or limiting the performance of the ANN computing system 100 in terms of throughput or power efficiency. As such, in some embodiments of the ANN computing system 100 or 104, the non-linear transformation may be performed in an analog domain (analog domain) by analog electronics.

In some embodiments, the analog electronics unit 146 is configured to apply a non-linear transfer function and output a transformed output voltage to the ADC unit 160. When the ADC unit 160 receives the voltage that has been nonlinearly converted by the analog electronic unit 146, the controller 110 may obtain a converted digitized output voltage corresponding to the converted output voltage from the ADC unit 160. Since the digitized output voltage from the ADC unit 160 has been non-linearly transformed ("activated"), the non-linear transformation step of the controller 110 may be omitted, reducing the computational burden on the controller 110. The first converted voltage directly obtained from the ADC unit 160 may then be stored as a first converted digital output vector in the memory unit 120.

The analog electronic nonlinear transformation performed by the analog electronics unit 146 may be implemented in a variety of ways. For example, a high-gain amplifier (high-gain amplifier) having a feedback (feedback) configuration, a comparator (comparator) having an adjustable reference voltage, a nonlinear IV (current-voltage) characteristic of a diode, a breakdown (breakdown) characteristic of a diode, a nonlinear CV (capacitance-voltage) characteristic of a variable capacitor, or a nonlinear IV characteristic of a variable resistor may be used.

Using an analog nonlinear transformation may improve the performance of the ANN computing system 104, such as throughput or power efficiency, by reducing the steps to be performed in the digital domain. Moving the non-linear transformation step out of the digital domain may allow for additional flexibility and improvement in the operation of the ANN computing system. For example, in a recursive neural network, the output of the OMM unit 150 is activated and recycled back to the input of the OMM unit 150. In the example performed by the controller 110 in the ANN computing system 100, the output voltage of the analog electronics unit 146 needs to be digitized each time it passes through the OMM unit 150. In examples where activation is performed before the ADC unit 160 digitizes, the number of ADC conversions required in performing the recurrent neural network calculations may be reduced.

In some embodiments, analog nonlinear conversion may be performed by ADC unit 160. For example, the non-linear ADC unit may be a linear ADC unit having a non-linear look-up table that maps the linear digitized output of the linear ADC unit to a desired non-linear transformed digitized output.

Some embodiments of the optoelectronic processor 140 of the ANN computing system 100 in fig. 1 include a laser unit 142 that produces N optical outputs having the same wavelength and that are optically coherent. In these embodiments, the optical matrix multiplication unit 150 performs N × N matrix multiplication in the optical domain, where the optical signals remain coherent from the input of the OMM unit 150 to the output of the OMM unit 150. The advantages of the OMM unit 150 performing matrix multiplication in the optical domain have been described above. In other embodiments, the OMM unit 150 is capable of processing incoherent or low coherent optical signals when performing matrix calculations. The following describes an optoelectronic computing system that does not require that the optical signal be coherent throughout the matrix multiplication process, where a portion of the computation is performed in the optical domain and a portion of the computation is performed in the electrical domain.

Optoelectronic computing systems may use different types of operations to produce computational results, each operation being performed on a signal (e.g., an electrical or optical signal) that is best suited to the fundamental physical characteristics of the operation (e.g., in terms of energy consumption and/or speed). For example, the replication may be performed using optical power splitting, the summation may be performed using current-based summation, and the multiplication may be performed using optical amplitude modulation. An example of a calculation that may be performed using these three types of operations is to multiply a vector by a matrix (e.g., as employed by artificial neural network calculations). These operations may be used to perform various other computations, representing a set of general linear operations that may perform the various computations, including but not limited to: vector-vector dot product, vector-vector element-by-element multiplication, vector-scalar element-by-element multiplication, or matrix-matrix element-by-element multiplication.

Referring to FIG. 19, an example of an optoelectronic computing system 1800 includes a set of optical ports or light sources 1802A, 1802B, etc. that provide optical signals. For example, in some embodiments, optical port/light source 1802A may include an optical input coupler that provides an optical signal that is coupled to optical path 1803. In other embodiments, optical port/light source 1802A may comprise a modulated light source, such as a laser (e.g., for coherent-sensitive embodiments) or a Light Emitting Diode (LED) (e.g., for coherent-insensitive embodiments), that generates an optical signal that is coupled to optical path 1803. Some embodiments may include a combination of ports that couple optical signals into system 1800 and sources that generate optical signals within system 1800. The optical signal may comprise any light wave (e.g., an electromagnetic wave whose spectrum includes wavelengths in a range between about 100nm and about 1 mm) that has been or is being modulated with information using any of a variety of forms of modulation. Optical path 1803 may be defined, for example, based on a guided mode of an optical waveguide (e.g., a waveguide embedded in a Photonic Integrated Circuit (PIC) or an optical fiber), or based on a predetermined free-space path between optical port/light source 1802A and another module of system 1800.

In some embodiments, the optoelectronic computing system 1800 is configured to perform computations on an array of input values encoded on respective optical signals provided by the optical ports/light sources 1802A, 1802B, etc. For example, for various machine learning applications based on neural networks, the computation may implement vector-matrix multiplication (or vector-by-matrix multiplication), where a matrix is multiplied by an input vector to produce an output vector as a result. The optical signal may represent elements of a vector, possibly including only a subset of selected elements of the vector. For example, for some neural network models, the size of the matrix used in the computation may be larger than the size of the matrix that may be loaded into a hardware system (e.g., an engine or co-processor of a larger system) that performs the vector matrix-multiplication portion of the computation. Thus, performing a portion of the computation may involve dividing the matrix and vector into smaller segments (segments) that may be provided to the hardware system separately.

The modules shown in fig. 19 may be part of a larger system that performs vector-matrix multiplication on a relatively large matrix (or sub-matrix), such as a 64 x 64 matrix of elements. For purposes of illustration, however, the modules will be described in the context of an example calculation that performs vector-matrix multiplication using a 2 x 2 matrix of elements. The modules referenced in this example will include two copy modules 1804A and 1804B, four multiply modules 1806A, 1806B, 1806C, and 1806D, and two summing modules, of which only one summing module 1808 is shown in fig. 19. These modules will input vectors Multiplication matrixTo generate an output vectorFor the vector-matrix multiplicationOutput vectorEach of the two elements of (a) may be represented by a different equation, as shown below.

y_A＝M_Ax_A+M_Bx_B(equation 1)

y_B＝M_Cx_A+M_Dx_B(equation 2)

Equations (1) and (2) may be decomposed into separate steps that may be performed in system 1800 using a set of basic operations (copy, multiply, and sum operations). In these equations, each element of the input vector occurs twice, so there are two copy operations. There are also four multiplication operations and there are two summation operations. For systems that implement vector-matrix multiplication using larger matrices, the number of operations performed will be larger, and using matrices that are not square in shape (i.e., different numbers of columns and rows), the number of relative instances of each operation will be different.

In this example, the copy operation is performed by the copy modules 1804A and 1804B. Input vector x_AAnd x_BAre represented by values encoded on optical signals from optical ports/light sources 1802A and 1802B, respectively. Each of these values is used in two equations, so each value is replicated to provide two resulting copies to different respective multiplication modules. For example, as described in more detail below, values may be encoded in a particular time slot using light waves that have been modulated to have power from a set of multiple power levels, or light waves that have duty cycles from a set of multiple duty cycles. The value is copied by copying the optical signal on which the value is encoded. Is encoded with a representation element x _AIs replicated by a replication module 1804A and encoded with a representation element x_BThe optical signal of values of (a) is replicated by the replication module 1804B. Each replication module may be implemented, for example, using an optical power splitter, such as a waveguide splitter that couples guided modes in an input waveguide to each of two output waveguides on a Y-splitter that gradually (e.g., adiabatically) splits the power, or a free-space splitter that uses a dielectric interface or membrane with one or more layers to transmit and reflect two output beams, respectively, from an input beam.

In this disclosure, it is said that the copy is encoded with the representation element x by the copy module 1804A_AWhen the optical signal of (2) is an optical signal, it means that the element x is generated based on the input signal_AThe output signal of the replica module 1804AMust have the same amplitude as the input signal. For example, if the replication module 1804A evenly splits the input signal power between the two output signals, each of the two output signals will have a power equal to or less than 50% of the input signal power. The two output signals are copies of each other, and the amplitude of each output signal of the copy module 1804A is different from the amplitude of the input signal. Also, in some embodiments having a set of multiple replica modules for replicating a given optical signal or subset of optical signals, each individual replica module does not necessarily split power evenly between its generated replicas, but the set of replica modules may be collectively configured to provide replicas having substantially equal power as the input of a downstream module (e.g., a downstream multiplication module).

In this embodiment, the multiplication operations are performed by four multiplication modules 1806A, 1806B, 1806C, and 1806D. For each copy of an optical signal, a multiplication module multiplies the copy of the optical signal by matrix element values, which can be performed using optical amplitude modulation. For example, multiplication module 1806A will input vector element x_AMultiplication by matrix element M_A. Vector element x_ACan be encoded on the optical signal and the matrix element M_AMay be encoded as an amplitude modulation level (am) of the optical amplitude modulator.

Is encoded with vector elements x_AMay be encoded using different forms of amplitude modulation. The amplitude of the optical signal may correspond to a particular instantaneous power level P of the physical light wave within a particular time slot_AOr may correspond to a particular energy E of the physical light wave at a particular time slot_A(the power integrated over time yields the total energy). For example, the power of the laser source may be modulated to have a particular power level from a predetermined set of multiple power levels. In some embodiments, it may be useful to operate the electronic circuit near an optimized operating point, so instead of varying power over many possible power levels, an optimized "on" power level is used, wherein the power level is signaled The signs are modulated to be "on" and "off (at zero power) for a particular portion of the time slot. The portion of the time that the power is at the "on" level corresponds to a particular energy level. Any of these particular values of power or energy may be mapped to element x_AA specific value (using a linear or non-linear mapping relationship). The actual integration over time (actual integration over time) to produce a particular total energy level may occur downstream of the system 1800 after the signal is in the electrical domain, as described in more detail below.

In addition, the term "amplitude" may refer to the amplitude of a signal represented by the instantaneous or integrated power in an optical wave, or may equivalently refer to the "electromagnetic field amplitude" of an optical wave. This is because the electromagnetic field amplitude has a well-defined relationship to the signal amplitude (e.g., by integrating the electromagnetic field strength (proportional to the square of the electromagnetic field amplitude) over the transverse dimension of the guided mode or free space beam to produce instantaneous power). This results in a relationship between modulation values, since by a specific valueA modulator that modulates the amplitude of the electromagnetic field can also be considered to modulate the power-based signal amplitude by a corresponding value M (since the optical power is proportional to the square of the electromagnetic field amplitude).

Used by multiplication modules to encode matrix elements M_AThe optical amplitude modulator of (a) may operate by changing the amplitude of the optical signal (i.e., the power in the optical signal) using any of a variety of physical interactions. For example, the modulator may include a ring resonator, an electro-absorption modulator, a thermo-optic modulator (thermo-optic modulator), or a mach-zehnder interferometer (MZI) modulator. In some techniques, a portion of the power is absorbed as part of a physical interaction, and in other techniques, the power is transferred using a physical interaction that modifies other characteristics of the light wave than its power, such as its polarization or phase, or modifies the coupling of optical power between different optical structures (e.g., using a tunable resonator). For makingOptical amplitude modulators that operate with interference (e.g., destructive and/or constructive interference) between light waves that have traveled different paths may use coherent light sources (e.g., lasers). For optical amplitude modulators that operate using absorption, either coherent or incoherent or low coherent light sources, such as LEDs, may be used.

In one example of a waveguide 1 x 2 optical amplitude modulator, a phase modulator is used to modulate power in an optical wave by placing the phase modulator in one of a plurality of waveguides of the modulator. For example, a waveguide 1 × 2 optical amplitude modulator may split a light wave guided by an input optical waveguide into a first arm and a second arm. The first arm includes a phase shifter that imparts a relative phase shift with respect to the phase delay of the second arm. The modulator then combines the light waves from the first and second arms. In some embodiments, the different phase delay values multiply the power in the optical wave guided by the input optical waveguide by a value between 0 and 1 through constructive or destructive interference. In some embodiments, the first and second arms combine into each of two output waveguides, and the difference between the photocurrents produced by the respective photodetectors receiving the light waves from the two output waveguides provides a signed multiplication result (e.g., multiplied by a value between-1 and 1), as described in more detail below (see fig. 28). By appropriate choice of amplitude scaling of the encoded optical signal, the range of matrix element values can be mapped to any range of positive values (0 to M) or signed values (-M to M).

In this example, the summing operation is performed by two summing modules, where summing module 1808 (shown in FIG. 19) is used to compute output vector element y_BThe summation is performed in equation (2) of (1). A corresponding summing module (not shown) is used in the calculation of the output vector element y_AThe summation is performed in equation (1) of (a). The summing module 1808 produces an electrical signal representing the sum of the results of the two multiplication modules 1806C and 1806D. In this example, the electrical signal is a current i_sumProportional to the sum of the powers in the output optical signals produced by the multiplication modules 1806C and 1806D, respectively. In some implementationsIn the example, the current i is generated_sumThe summing operation of (a) is performed in the optical-electrical domain, and in other embodiments in the electrical domain. Alternatively, some embodiments may use optical-domain summing for some summing modules and electrical-domain summing for other summing modules.

In embodiments where the summation is performed in the electrical domain, the summation module 1808 may be implemented using: (1) two or more input conductors, each input conductor carrying an input current, the magnitude of the input current being representative of the result of one of the multiplication modules, and (2) at least one output conductor carrying a current that is the sum of the input currents. This can occur, for example, if the conductors are wires that meet at a junction. For example, and without being bound by theory, this relationship may be understood based on Kirchhoff's current law, which indicates that the current flowing into a node is equal to the current flowing out of the node. For these embodiments, the signals 1810A and 1810B provided to the summing module 1808 are input currents that may be generated by photodetectors that are part of a multiplication module that generates respective photocurrents whose amplitudes are proportional to the power in the received optical signal. The summing module 1808 then provides an output current i _sum. The instantaneous value of the output current (instant value) or the integrated value of the output current (integrated value) may then be used to represent the quantitative value of the sum (quantitative value).

In embodiments where the summing is performed in the photo-electric domain, the summing module 1808 may be implemented using photo-detectors (e.g., photo-diodes) that receive optical signals generated by different respective multiplication modules. For these embodiments, signals 1810A and 1810B provided to summing module 1808 are input optical signals, each of which includes a light wave whose power represents the result of one of the multiplying modules. Output current i in this embodiment_sumIs the photocurrent generated by the photodetector. Since the wavelengths of the optical waves are different (e.g., sufficiently different so that no significant constructive or destructive interference occurs between them), the photocurrent will be proportional to the sum of the powers of the received optical signals. Light (es)The current is also substantially equal to the sum of the individual currents that will result in the individual detected optical powers being detected by the separate equivalent photodetectors. The wavelengths of the light waves are different but close enough that the photodetectors have substantially the same response (e.g., wavelengths within the substantially flat detection bandwidth of the photodetectors). As described above, summing in the electrical domain using current summing can achieve a simpler system architecture by avoiding the need for multiple wavelengths.

Fig. 20 shows an example of a system configuration 1900 for an implementation of a system that performs vector-matrix multiplication using a 2 x 2 matrix of elements, where the summation operation is performed in the electrical domain. In this example, the input vector isAnd the matrix isEach element of the input vector is encoded on a different optical signal. Two different replication modules 1902a, 1902b (collectively 1902) perform optical replication operations to separate computations on different paths (e.g., "up" and "down" paths, where the terms "up" and "down" refer to the relative positions of the paths in the figure). There are four multiplication modules 1904a, 1904b, 1904c, and 1904d (collectively 1904), each multiplication module 1904 multiplying a different matrix element using optical amplitude modulation. At the output of each multiplication module 1904, there is an optical detection module 1906 (e.g., 1906a, 1906b, 1906c, 1906d) that converts the optical signal to an electrical signal in the form of a current. The two upper paths (e.g., including the outputs of 1906a and 1906 c) for different input vector elements are combined using a summation module 1908a, and the two lower paths (including the outputs of 1906b and 1906d) for different input vector elements are combined using a summation module 1908 b. Summing modules 1908a and 1908b (collectively 1908) perform the summing in the electrical domain. Thus, each element of the output vector is encoded on a different electrical signal. As shown in FIG. 20, as the calculation proceeds, each component of the output vector is incrementally generated to generate an upper path and a lower path, respectively The following results of the path.

M₁₁v₁+M₁₂v₂

M₂₁v₁+M₂₂v₂

Different optical powers may represent the same value in different parts of the system. For example, replica module 1902a receives an input signal on input waveguide 1914 and provides an output signal to output waveguides 1916a and 1916 b. The amplitude of the optical signal representative of value v1 on output waveguide 1916a or 1916b has an amplitude that is about half the amplitude of the optical signal representative of value v1 on input waveguide 1914.

In some embodiments, if the replication module performs an optical replication operation to separate the computations over the three paths, the amplitude of the optical signal representing the particular value on the output waveguide of the optical splitter is about one-third of the amplitude of the optical signal representing the particular value on the input waveguide of the optical splitter. Similarly, if the replication module performs an optical replication operation to separate the computations on the four paths, the amplitude of the optical signal representing a particular value on the output waveguide of the optical splitter is approximately one quarter of the amplitude of the optical signal representing the particular value on the input waveguide of the optical splitter, and so on.

In some embodiments, the photonic integrated circuit includes different types of replication modules, such as a first replication module that performs an optical replication operation to separate computations on two paths, a second replication module that performs an optical replication operation to separate computations on three paths, a third replication module that performs an optical replication operation to separate computations on four paths, and a fourth replication module that performs an optical replication operation to separate computations on eight paths. The signals derived from the outputs of the first, second, third and fourth replica modules are scaled prior to combining.

For example, assume that vout1 is the value of a vector resulting from a vector-matrix multiplication using a 2 x 2 matrix of elements, where a 1-to-2 splitter is used in an optical copy operation, and vout2 is the value of a vector resulting from a vector-matrix multiplication using a 4 x 4 matrix of elements, where a 1-to-4 splitter is used in an optical copy operation. If the photonic integrated circuit is configured to combine vout1 with vout2, vout2 is scaled to twice its value before combining with vout 1.

System configuration 1900 may be implemented using any of a variety of optoelectronic technologies. In some embodiments, there is a common substrate (e.g., semiconductor (e.g., silicon)) that can support both integrated optical and electronic components. The optical path may be implemented in a waveguide structure having a material with a higher optical index surrounded by a material with a lower optical index (optical index), the waveguide structure defining a waveguide for propagating light waves carrying the optical signal. The electrical path may be implemented by a conductive material for propagating electrical current carrying electrical signals. In fig. 20 to 22, 25 to 38, unless otherwise stated, the thickness of the line representing the path is used to distinguish between an optical path (represented by a thicker line) and an electrical path (represented by a thinner line or a broken line). Optical devices, such as splitters and optical amplitude modulators, as well as electrical devices, such as photodetectors and operational amplifiers (op-amps), may be fabricated on a common substrate. Alternatively, different portions of the system may be implemented using different devices having different substrates, and those devices may communicate over a communication channel. For example, optical fibers may be used to provide a communication channel to transmit optical signals between multiple devices used to implement an overall system. Those optical signals may represent different subsets of input vectors provided when performing vector-matrix multiplication, and/or different subsets of intermediate results calculated when performing vector-matrix multiplication, as described in more detail below.

In the present disclosure, the figures may show an optical waveguide passing through an electrical signal line, with the understanding that the optical waveguide does not intersect the electrical signal line. The electrical signal lines and the optical waveguide may be arranged in different layers of the device.

Fig. 21 shows an example of a system configuration 1920 for an implementation of a system that performs vector-matrix multiplication using a 2 x 2 matrix of elements, where the summation operation is performed in the optical-electrical domain. In this example, two different respective wavelengths λ are used₁And λ₂Different input vector elements are encoded on the optical signal. The optical output signals of the multiplication modules 1904a and 1904c are combined in an optical combiner module 1910a and the optical output signals of the multiplication modules 1904b and 1904d are combined in an optical combiner module 1910 b. The optical waveguide 1914a couples the two wavelengths λ from the optical combiner module 1910a₁And λ₂The optical signal above is directed to an opto-electronic summation module 1912 a. Similarly, the optical waveguide 1914b couples the two wavelengths λ from the optical combiner module 1910b₁And λ₂The optical signal on is directed to an opto-electronic summing module 1912 b. The photo-electric summing modules 1912a and 1912b may be implemented using photo-detectors, such as the optical detection module 1906 used in the example of fig. 20. In this example, the sum is represented by the photocurrent indicative of the power in the two wavelengths, rather than the current leaving the junction between the different conductors.

In the present disclosure, when the drawings show two optical waveguides crossing each other, it will be clear from the description whether the two optical waveguides are actually optically coupled to each other. For example, two waveguides that cross each other from the top view of the device display can be implemented in different layers and thus do not cross each other. For example, in FIG. 21, the optical signal λ is measured₂An optical path provided as an input to the replica module 1902b and an optical signal M provided from the multiplication module 1904a₁₁V₁The optical paths to the optical combiner module 1910a are not optically coupled to each other, although they may appear to cross each other in the figure. Similarly, an optical signal λ is provided from the replica module 1902b₂An optical path to the multiplication module 1904d and an optical signal M provided from the multiplication module 1904b₂₁V₁The optical paths to the optical combiner modules 1910b are not optically coupled to each other, although in the figures they may appear to cross each other.

The system configurations shown in fig. 20 and 21 may be extended to implement a system configuration for performing vector-matrix multiplication using an m × n element matrix. In this example, the input vector isAnd the matrix isFor example, the input vector element v ₁To v_nIs provided by n waveguides and each input vector element is processed by one or more copy modules to provide m copies of the input vector element to m respective paths. There are M x n multiplication modules, each multiplying a different matrix element using optical amplitude modulation to produce a representation M_ij·v_j(i-1 … m, j-1 … n)) or an optical signal. Using the ith summation module (i ═ 1 … M) to combine and represent M_ij·v_j(j-1 … n) to yield the following results for the m paths, respectively.

M₁₁v₁+M₁₂v₂+…+M_1nv_n

M₂₁v₁+M₂₂v₂+…+M_2nv_n

…

M_m1v₁+M_m2v₂+…+M_mnv_n

Since optical amplitude modulation can reduce the power in an optical signal from its full value (full value) to a lower value, to zero (or near zero) power, any value between 0 and 1 can be multiplied. However, some calculations may require multiplication by a value greater than 1 and/or multiplication by a signed (positive or negative) value. First, to extend the range to 0 to M_max(wherein M is_max>1) The original modulation of the optical signal may comprise passing through M_maxExplicit (explicit) or implicit (implicit) scaling of original vector element amplitudes (or equivalently, by 1/M)_maxScaling values mapped to particular vector element magnitudes in a linear mapping) such that a range of 0 to 1 of matrix element magnitudes quantitatively corresponds to a range of 0 to M in the calculation _max. Second, to fit a positive range of matrix element values from 0 to M_maxExtension to signed range-M_maxTo M_maxSymmetric differential configurations may be used, as described in more detail belowA description is given. Similarly, symmetric differential configurations may also be used to extend the positive range of values encoded on various signals to signed ranges of values.

Fig. 22 shows an example of a symmetric differential configuration 2000 for providing a signed range of values for values encoded on an optical signal. In this example, there are two correlated optical signals that encode unsigned values, designated as unsigned valuesAndwherein each value is assumed to be between 0 (e.g., corresponding to near zero optical power) and V_maxE.g., optical power corresponding to a maximum power level. The relationship between two optical signals is when one optical signal uses a "main" valueDuring encoding, the other optical signal is encoded by using the corresponding anti-symmetry valueEncoding, so that the main value is encoded on an optical signalFrom 0 to V, monotonically increasing (monotonically increasing)_maxAntisymmetric values encoded on paired optical signalsFrom V_maxMonotonically decreases (monotonically decrease) to 0. Or, conversely, when the principal value is encoded on an optical signal From V_maxAntisymmetric values encoded on the paired optical signals when monotonically decreasing to 0Monotonically increasing from 0 to V_max. After the optical signals in the upper and lower paths are converted into current signals by the respective optical detection modules 1906a and 1906b, the difference between the current signals may be generated by a current subtraction module 2002. EncodingAndleads to a useful signed value V₁The encoded current, given as:

with unsigned primary valuesMonotonically increasing from 0 to V_maxAnd the antisymmetric value paired therewithFrom V_maxMonotonically decreasing to 0, signed value V₁at-V_maxAnd V_maxMonotonically increases in between. There are various techniques that can be used to implement the symmetric differential configuration of fig. 22, as shown in fig. 23 and 24.

In fig. 23, the optical signal is detected in a common-terminal configuration, where two photodiode detectors are connected to a common terminal 2032 (e.g., inverting terminal) of an operational amplifier 2030. In this configuration, current 2010 generated from first photodiode detector 2012 and current 2014 generated from second photodiode detector 2016 are combined at junction 2018 between the three conductors to produce difference current 2020 between current 2010 and current 2014. Current 2010 and current 2014 are provided from opposite sides of the respective photodiode, The photodiodes are connected at the other end to provide the same amplitude v_biasBut a voltage source (not shown) of bias voltage of opposite sign as shown in fig. 23. In this configuration, a difference is generated due to the behavior of the current converged at the common node 2018. Difference current 2020 represents a signed value encoded on the electrical signal that corresponds to a difference between unsigned values encoded on the detected optical signal. The operational amplifier 2030 may be configured as a transimpedance amplifier (TIA) configuration in which the other terminal 2024 is connected to ground and the output terminal 2026 is fed back to the common terminal 2032 using a resistive element 2028, the resistive element 2028 providing a voltage proportional to the difference current 2020. Such a TIA configuration would provide the resulting value as an electrical signal in the form of a voltage signal.

In some embodiments of sensors that convert photocurrent to voltage using a TIA, techniques are used to mitigate the effect of the internal capacitance of the photodiodes used as photodetectors 2012 and 2016. Internal capacitance C of photodiode_dCan be modeled as a capacitor in parallel with an ideal photodiode. One consequence is that when the current changes rapidly, the capacitor can act as a short circuit, which can limit the bandwidth of the detector. In general, the time constant τ associated with the changes caused to the input current 2020 to the operational amplifier 2030 should be maintained small to avoid significant bandwidth limitations. Without mitigation, then this time constant would be approximately equal to the capacitance C _dMultiplied by the input resistance R of the operational amplifier 2030_in，R_inApproximately equal to the feedback resistance R due to the resistive element 2028_fDividing by the operational amplifier gain a yields: τ ═ C_dR_fand/A. This effect is particularly detrimental to systems having a large number of photodiodes connected in parallel with each other (as some of the systems described in this specification) because the capacitances connected in parallel with each other will sum to a large effective capacitance. However, implementing a large op amp gain a to reduce the time constant consumes a significant amount of power in a system using a large number of such TIA circuits. To alleviate bandwidth limitations without requiring so much power, a given photodiode may be powered down by connecting a given voltage follower (voltage follower) to the input and output of the given photodiode,a voltage follower circuit (also known as a unity-gain amplifier or buffer amplifier) is placed in parallel with each photodiode. The voltage follower supplies current as needed to maintain approximately equal voltages at its input and output, which has the effect of preventing the internal capacitance of the photodiode from acting as a short circuit when the current changes rapidly, thereby achieving a near zero time constant τ. The use of voltage followers may allow the operational amplifier gain a in the TIA circuit to be relatively low (e.g., below 1000 or below 100), thereby reducing the power requirements of the system.

In fig. 24, the optical signal is detected in a differential terminal configuration, where two photodiode detectors are connected to different terminals of an operational amplifier 2050. In this configuration, the current 2040 generated from the first photodiode detector 2042 is connected to the inverting terminal 2052, and the current 2044 generated from the second photodiode detector 2046 is connected to the non-inverting terminal 2054. The currents 2040 and 2044 are provided from the same end of the respective photodiodes, which are connected at the other end to provide the same magnitude v_biasAnd a bias voltage of the same sign (not shown) as shown in fig. 24. The output 2056 of the operational amplifier 2050 in this configuration provides a current that is proportional to the difference between the currents 2040 and 2044. In this configuration, a difference value is generated due to the behavior of the circuit of the operational amplifier 2050. The difference current flowing from output 2056 represents a signed value encoded on the electrical signal that corresponds to the difference between unsigned values encoded on the detected optical signal.

Fig. 25 shows an example of a symmetric differential configuration 2100 for providing a signed range of values for values encoded as modulation levels of an optical amplitude modulator implementing the multiplication module 1904. In this example, there are two related modulators configured to pass a signal designated as Andof (a) unsigned valueIs modulated assuming that each value is at 0 (e.g., corresponding to an optical power modulated down to near zero) and M_max(e.g., corresponding to optical power maintained near a maximum power level). The relationship between the two modulation levels is when one modulation level is configured at the "primary" valueWhile the other modulation level is configured at the corresponding "antisymmetric" valueSo that when the dominant value of a modulatorMonotonically increasing from 0 to M_maxWhile the antisymmetric value of the other modulatorFrom M_maxMonotonically decreases to 0. Or, conversely, when the principal value of a modulatorFrom M_maxWhen monotonically decreasing to 0, the antisymmetric value of the other modulatorMonotonically increasing from 0 to M_max. After the replication modules 1902 replicate the input optical signal encoded with the value V, each modulator provides a modulated output optical signal to a corresponding optical detection module 1906. The multiplication module 1904 in the upper path includes an ANDMultiply and provide a valueA modulator of an encoded optical signal. The multiply module 1904 in the lower path includes an ANDMultiply and provide the valueA modulator of an encoded optical signal. After the optical signals are converted into current signals by the corresponding optical detection modules 1906, the difference between them may be generated by the current subtraction module 2102. Encoding Andresults in multiplying the current encoded by V by a signed value M₁₁Given as:

with unsigned primary valuesMonotonically increasing from 0 to M_maxAnd the antisymmetric value paired therewithFrom M_maxMonotonically decreasing to 0, signed value M₁₁at-M_maxAnd M_maxMonotonically increases in between.

Fig. 26 shows an example of a system configuration 2110 for an implementation of a system 1800 for performing vector-matrix multiplication using a 2 x 2 matrix of elements, where the summation operation is performed in the electrical domain and has signed elements of the input vector and signed elements of the matrix. In this example, for each signed element of the input vector, there are two associated optical signals encoding unsigned values. For a first signed input vector element value V₁Two are assigned asAndand for a second signed input vector element value V₂Two are assigned asAndis set to zero. Each unsigned value encoded on the optical signal is received by the replication module 2112, and the replication module 2112 performs one or more optical replication operations that produce four replicas of the optical signal on four respective optical paths. In some embodiments of the replication module 2112, there are three different Y-waveguide splitters, each configured to split using a different power ratio (which may be achieved using any of a variety of photonic devices, for example). For example, the first separator may use a 1: a power ratio of 4 to divert 25% (1/4) of the power to the first path, the second splitter may use a 1: a power ratio of 3 to transfer 25% (1/4 ═ 1/3 × 3/4) of the power to the second path, and a third splitter may use a 1: the power ratio of 2 is split to transfer 25% (1/4 ═ 1/2 × 2/3 × 3/4) of the power to the third path and the remaining 25% of the power to the fourth path. For example, individual splitters that are part of the replication module 2112 can be arranged in different portions of the substrate to properly distribute different replicas to different paths within the system. In other embodiments of the replication module 2112, there may be a different number of paths, optionally separated by different separation ratios. For example, the first separator may use a 1: a power ratio of 2 is split to provide two intermediate optical signals. Next, a solution having 1: a second splitter of power ratio of 2 to split one of the intermediate optical signals to transfer 25% of the power to each of the first and second paths, and may Use was made of a catalyst having 1: a third splitter of 2 power ratios splits the other of the intermediate optical signals to transfer 25% of the power to each of the third and fourth paths.

The system configuration 2110 also includes other modules arranged as shown in fig. 26 to provide two different output electrical signals representing output vectors that are the result of the vector-matrix multiplication performed by the system 100. There are 16 different multiplication modules 1904 that modulate different copies of the optical signal representing the input vector, and 16 different optical detection modules 1906 to provide electrical signals representing intermediate results of the calculations. There are also two different summation modules 2114A and 2114B that calculate the overall summation of each output electrical signal. In the figure, the signal lines that electrically couple the optical detection module 1906 to the summing module 2114B are shown in dashed lines. Because each overall sum may include some antisymmetric terms (anti-symmetric terms) subtracted from the pairwise principal terms (paired main terms) from any symmetric differential configuration for vector elements and/or matrix elements, summing modules 2114A and 2114B may include mechanisms for adding some terms in the sum after being inverted (inverted) (equivalently, subtracted from the non-inverted terms). For example, in some embodiments, the summing modules 2114A and 2114B include both inverting and non-inverting input ports, such that the terms to be added in the overall sum of requirements may be connected to the non-inverting input ports and the terms to be subtracted in the overall sum of requirements may be connected to the inverting input ports. An exemplary embodiment of such a summing module is an operational amplifier, wherein the non-inverting terminal is connected to a conductor conducting a current representing the signal to be added and the inverting terminal is connected to a conductor conducting a current representing the signal to be subtracted. Alternatively, if inversion of the antisymmetric terms is performed by other means, an inverting input port may not be required on the summing block. The summation modules 2114A and 2114B generate the following summation results, respectively, to complete the vector-matrix multiplication.

In the present disclosure, when the drawings show two electrical signal lines crossing each other, it is clear from the description whether the two electrical signal lines are electrically coupled to each other. For example, bearer M₂₁ ⁺V1⁺The signal line of the signal is not electrically coupled to the carrier M₁₁ ⁺V₁ ^-Signal line or carrier M of signal₁₁ ^-V₁ ^-Signal lines for signals.

The system configuration shown in fig. 26 can be extended to implement a system configuration that performs vector-matrix multiplication using an m x n matrix of elements, where the input vector and matrix include signed elements.

There are various techniques that may be used to implement the symmetric differential configuration of fig. 26. Some of these techniques utilize a 1 x 2 optical amplitude modulator to implement the multiplication module 1904 and/or provide pairs of optical signals associated with the primary and anti-symmetric pairs. Fig. 27 shows an example of a 1 × 2 optical amplitude modulator 2200. In this example, the 1 x 2 optical amplitude modulator 2200 includes an input optical splitter 2202 that splits an input optical signal to provide 50% of the power to a first path that includes a phase modulator 2204 (also referred to as a phase shifter) and 50% of the power to a second path that does not include a phase modulator. The paths may be defined in different ways depending on whether the optical amplitude modulator is implemented as a free space interferometer or as a waveguide interferometer. For example, in a free space interferometer, one path is defined by the transmission of a wave through a beam splitter, and the other path is defined by the reflection of the wave from the beam splitter. In a waveguide interferometer, each path is defined by a different optical waveguide that has been coupled to an incoming waveguide (e.g., in a Y-splitter). The phase modulator 2204 may be configured to impart a phase shift such that the total phase delay of the first path differs from the total phase delay of the second path by a configurable phase shift value (e.g., may be set to a value of the phase shift somewhere between 0 degrees and 180 degrees).

The 1 x 2 optical amplitude modulator 2200 includes a 2 x 2 coupler 2206 that combines the light waves from the first and second input paths in a particular manner using optical interference or optical coupling to transfer power into the first and second output paths at different rates, depending on the phase shift. For example, in a free-space interferometer, a phase shift of 0 degrees causes substantially all of the input power split between the two paths to constructively interfere to exit from one output path of the beam splitter implementing coupler 2206, and a phase shift of 180 degrees causes substantially all of the input power split between the two paths to constructively interfere to exit from the other output path of the beam splitter implementing coupler 2206. In a waveguide interferometer, a phase shift of 0 degrees results in substantially all of the input power split between the two paths being coupled to one output waveguide (e.g., 2208a) of coupler 2206, and a phase shift of 180 degrees results in substantially all of the input power split between the two paths being coupled to the other output waveguide (e.g., 2208b) of coupler 2206. The phase shift between 0 and 180 degrees may then multiply the power in the light wave (and the value encoded on the light wave) by a value between 0 and 1 by partial constructive or destructive interference or partial waveguide coupling. Multiplication by any value between 0 and 1 may then be mapped to multiplication by 0 and M as described above _maxAny value between.

In addition, the relationship between the power in the two optical waves emitted from modulator 2200 follows the relationship between the power of the primary and anti-symmetric pairs described above. As the magnitude of the optical power of one signal increases, the magnitude of the optical power of the other signal decreases, and thus the difference between the detected photocurrents can produce a signed vector element, or multiplied by a signed matrix element, as described herein. For example, the pair of correlated optical signals may be provided from two output ports of modulator 2200 such that the difference between the amplitudes of the correlated optical signals corresponds to the result of multiplying the input value by the signed matrix element value. Fig. 28 shows a symmetric differential configuration 2210 of a 1 × 2 optical amplitude modulator 2200 with an optical signal arranged at the output to be detected in the common-terminal version of the symmetric differential configuration of fig. 23. The current signals corresponding to the photocurrents produced by the pair of photodetectors 2212 and 2214 are combined at junction point 2216 to provide an output current signal having a magnitude corresponding to the difference between the magnitudes of the associated optical signals. In other examples, such as in the symmetric difference distribution configuration of fig. 24, different circuits may be used to combine the photocurrents detected from the two optical signals at the outputs.

Other techniques may be used to construct a 1 x 2 optical amplitude modulator for implementing the multiplication module 1904 and/or to provide pairs of optical signals that are related as primary and anti-symmetric pairs. Fig. 29 shows another example of a symmetric differential configuration 2220 of another type of 1 x 2 optical amplitude modulator. In this example, the 1 × 2 optical amplitude modulator includes a ring resonator 2222 configured to split the optical power of an optical signal at an input port 2221 to two output ports. The ring resonator 2222 (also referred to as a "microring") may be fabricated, for example, by forming a circular waveguide on a substrate, where the circular waveguide is coupled to a straight waveguide (straight waveguide) corresponding to the input port 2221. As the wavelength of the optical signal approaches the resonant wavelength associated with ring resonator 2222, the optical waves coupled into the ring circulate around the ring on clockwise path 2226 and destructively interfere at the coupling location such that the reduced power optical waves exit to the first output port via path 2224. The circulating lightwave is also coupled out of the ring such that another lightwave exits on path 2228 through the curved waveguide, which directs the lightwave out of the second output port.

Since the time scale of the optical power cycling around the ring resonator 2222 is small compared to the time scale of the amplitude modulation of the optical signal, an anti-symmetric power relationship is quickly established between the two output ports such that the optical wave detected by the photodetector 2212 and the optical wave detected by the photodetector 2214 form a dominant and anti-symmetric pair. The resonant wavelength of ring resonator 2222 may be adjusted to monotonically decrease/enhance the dominant/antisymmetric signal to achieve a signed result, as described above. When the ring is completely non-resonant, all power leaves the first output port via path 2224, and when it is completely resonant, all power leaves the second output port via path 2228 with appropriate adjustment of certain other parameters (e.g., quality factor and coupling coefficient). In particular, in order to achieve full power transfer, the coupling coefficients characterizing the coupling efficiency between the (characterizing) waveguide and the ring resonator should be matched. In some embodiments, it may be useful to have a relatively shallow tuning curve (tuning curve), which may be achieved by decreasing the quality factor of the ring resonator 2222 (e.g., by increasing the loss) and correspondingly increasing the coupling coefficient into and out of the ring. The shallow tuning curve provides less amplitude sensitivity to the resonant wavelength. Techniques such as temperature control may also be used for tuning and/or stabilization of the resonant wavelength.

Fig. 30 shows another example of a symmetric differential configuration 2230 of another type of 1 x 2 optical amplitude modulator. In this example, the 1 × 2 optical amplitude modulator includes two ring resonators 2232 and 2234. The optical power of the optical signal at input port 2231 is split into two ports. As the wavelength of the optical signal approaches the resonant wavelengths associated with the two ring resonators 2232 and 2234, the reduced-power optical wave exits the first output port via path 2236. A portion of the optical waves are also coupled into ring resonator 2232, which circulates around the ring on clockwise path 2238, and are also coupled into ring resonator 2234, which circulates around the ring on counterclockwise path 2240. The circulating lightwave is then coupled out of the ring, causing another lightwave to exit the second output port via path 2242. In this example, the light wave detected by photodetector 2212 and the light wave detected by photodetector 2214 also form a primary and anti-symmetric pair.

Fig. 31 and 32 show different examples of the use of optical amplitude modulators, for example using a 1 x 2 optical amplitude modulator 2200, for implementing a system 1800 for performing vector-matrix multiplication on a 2 x 2 matrix of elements. Fig. 31 shows an example of an optoelectronic system configuration 2300A, which includes optical amplitude modulators 2302A and 2302B providing values representing signed vector elements of an input vector. Modulator 2302A provides a pair An optical signal encoding a pair of values for a first signed vector elementAndand modulator 2302B provides a pair of optical signals that encode a pair of values for the second signed vector elementAnda vector-matrix multiplier (VMM) subsystem 2310A receives the input optical signals, performs splitting, multiplying, and some summing operations as described above, and provides output current signals to be processed by additional circuitry. In some examples, the output current signal representation is further processed to produce a partial sum of a final sum, the final sum resulting in signed vector elements of the output vector. In this example, some final summing operations are performed as a subtraction between different partial sums represented by the current signals at the inverting and non-inverting terminals of operational amplifiers 2306A and 2306B. The subtraction is used to provide signed values as described above (e.g., with reference to fig. 26). This example also illustrates how certain elements become part of the plurality of modules. In particular, the optical replication performed by waveguide splitter 2303 may be considered to be part of a replication module (e.g., one of replication modules 2112 in FIG. 26) and part of a multiplication module (e.g., one of multiplication modules 1904 in FIG. 26). The optical amplitude modulator used within the VMM subsystem 2310A is configured for detection in a common-terminal configuration shown in fig. 23.

Fig. 32 shows an example of an optoelectronic system configuration 2300B similar to the example of the optoelectronic system configuration 2300A shown in fig. 31. However, the VMM subsystem 2310B includes an optical modulator configured to detect in the differential terminal configuration shown in fig. 24. In this example, the output current signal of VMM subsystem 2310B also represents a partial sum that is further processed to produce a final sum that results in signed vector elements of the output vector. The final summing operation performed as a subtraction between different partial sums represented by the current signals at the inverting and non-inverting terminals of the operational amplifiers 2306A and 2306B is different from the example of fig. 31. However, as described above (e.g., with reference to FIG. 26), the final subtraction still results in the signed value being provided.

Fig. 33 shows an example of an optoelectronic system configuration 2300C that uses an alternative arrangement of the VMM subsystem 2310C in the case of detection in a common terminal configuration (as in the VMM subsystem 2310A shown in fig. 31), but with the optical signal carrying the result of the multiplication module routed through a subsystem within the waveguide (e.g., within the semiconductor substrate) to a portion of the substrate that includes a detector arranged to convert the optical signal to an electrical signal. In some embodiments, the grouping of the detectors allows for shortening of the electrical path, potentially reducing electrical crosstalk or other damage due to long electrical paths that would otherwise be used. In some embodiments, the optical waveguides may be routed within one layer of the substrate (route), or within multiple layers of the substrate, to allow greater flexibility in routing paths that intersect in two dimensions of the substrate but not in a third dimension (of the depth in the substrate). Various other changes may be made in the system configuration, including changes to components included in the VMM subsystem. For example, optical amplitude modulators 2302A and 2302B may be included as part of a VMM subsystem. Alternatively, the VMM subsystem may include optical input ports for receiving paired primary and anti-symmetric optical signals generated by modules other than the optical amplitude modulator, or for interfacing with other types of subsystems.

Fig. 34 shows an example of a system configuration 2400A for an implementation of system 1800 in which there are multiple devices 2410 carrying (host) different multiplication modules (e.g., multiplication modules 1806A, 1806B, 1806C, and 1806D), each configured as a VMM subsystem to perform vector-matrix multiplication on different subsets of vector elements through different sub-matrices of a larger matrix. For example, rather than implementing the VMM subsystem using a 2 x 2 matrix of elements, each multiplication module may be configured to implement the VMM subsystem using a matrix having a size as large as can be efficiently manufactured on a single device having a common substrate for the modules within the device, which may be configured similarly to system configuration 2110 (fig. 26). For example, each multiplication module may implement the VMM subsystem using a 64 x 64 element matrix.

Different VMM subsystems are arranged to combine the results of each sub-matrix appropriately to produce the result of a larger combined matrix (e.g., the elements of a 128-element vector multiplied by a 128 x 128-element matrix). Each set of optical ports or light sources 2402 provides a set of optical signals that represent a different subset of the vector elements of a larger input vector. The replication module 2404 is configured to replicate all optical signals within a received set of optical signals (encoded on the guided light waves in the set of 64 optical waveguides 2403) and provide that set of optical signals to each of two different sets of optical waveguides, in this example a set of 64 optical waveguides 2405A and a set of 64 optical waveguides 2405B. For example, by performing this replication operation using an array of waveguide splitters, each splitter in the array replicates one element of a subset of input vector elements (e.g., a subset of 64 elements for each replication module 2404) by splitting the light waves in the set of optical waveguides 2403 into a first corresponding light wave in the set of optical waveguides 2405A and a second corresponding light wave in the set of optical waveguides 2405B.

If multiple wavelengths (e.g., W wavelengths) are used in some embodiments, the number of separate waveguides (and thus the number of separate ports or sources in 2402) may be reduced by, for example, 1/W. Each VMM subsystem 2410 performs vector-matrix multiplication, providing its partial results as a set of electrical signals (a subset of elements for the output vector), with corresponding partial result pairs from different devices 2410 added together by a summing module 2414 as shown in fig. 34, using any of the techniques described herein (e.g., summing currents at junctions between conductors). In this example, the output of the device 2410a is transmitted to the summing module 2414a over the electrical line 2416a, and the output of the device 2410b is transmitted to the summing module 2414a over the electrical line 2416 b.

In some embodiments, for any number of levels of recursion, vector-matrix multiplication using the desired matrix may be performed recursively by combining the results from the smaller sub-matrices, ending with the use of a single-element light amplitude modulator at the root level (root level) of the recursion. At different levels of recursion, the VMM subsystem devices may be more compact (e.g., different data centers connected by long haul fiber networks at one level, different multi-chip devices connected by fiber in data centers at another level, different chips in devices connected by fiber at another level, and different portions of modules on the same chip connected by on-chip waveguides at another level).

Fig. 35 shows another example of a system configuration 2400B, where additional devices are used for optical transmission and reception to each VMM subsystem 2410. In some embodiments, the different vector-matrix multiplier subsystems 2410 are carried by separate devices (host) and/or distributed at separate remote locations. In this example, at an output 2418 of each VMM subsystem 2410 (the output 2418 providing an electrical signal), an array of optical transmitters 2420 is used to convert the electrical signal to an optical signal and couple each optical signal to a channel within the optical transmission line (e.g., an optical fiber in a fiber bundle between the VMM subsystems 2410, the VMM subsystems 2410 may be carried by separate devices (host) and/or distributed at a remote location). The optical transmitter array 2420 may comprise an array of, for example, laser diodes that convert the electrical signals at the output of the vector-matrix multiplier subsystem 2410 into optical signals. In some embodiments, different vector-matrix multiplier subsystems 2410 are located at different areas on an integrated device (e.g., a system-on-a-chip) that carries vector-matrix multiplier subsystems 2410 on a common substrate. In this example, at the output 2418 of each vector-matrix multiplier subsystem 2410, the electrical signals at the output 2418 are converted to optical signals using an optical transmitter array 2420 and each optical signal is coupled to a channel within a waveguide in the set of waveguides 2416 between different regions on the integrated device. The optical receiver array 2422 is used to output each subset of vector elements to convert the optical signals to electrical signals before the corresponding pairs of partial results are summed by the summing module 2414.

Fig. 36 shows another example of a system configuration 2400C in which the VMM subsystem 2410 may be reconfigured to enable different vector-matrix multiplications for different sub-matrices to be rearranged in different ways. For example, the shape of a larger matrix formed by combining different sub-matrices may be configurable. The user can dynamically configure how different sub-matrices are combined based on computational requirements. This provides greater flexibility in the operation of the optical processor. In this example, two different optical signal subsets 2424a and 2424b are provided from each set of optical ports or light sources 2402 to the optical switch 2430. There is also an electrical switch 2440 that can rearrange a subset of the electrical signals that represent the partial results that are to be summed by the summing module 2414 to provide an output vector or a separate output vector for the desired computation. For example, instead of using vector-matrix multiplication of a 2m × 2n size matrix composed of four sub-matrices of size m × n, the VMM subsystem 2410 may be rearranged to use a 2m × n size matrix or a m × 2n size matrix.

Fig. 37 shows another example of a system configuration 2400D, in which the VMM subsystem 2410 may be reconfigured in additional ways. Optical switch 2430 may receive up to four separate sets of optical signals and may be configured to provide different sets of optical signals to different VMM subsystems 2410 or to replicate any set of optical signals to multiple VMM subsystems 2410. Also, the electrical switch 2440 may be configured to provide any combination of the set of received electrical signals to the summing module 2414. This greater reconfigurability enables a greater variety of different vector-matrix multiplication calculations, including multiplication using matrices of sizes m x 3n, 3m x n, m x 4n, 4m x n.

FIG. 38 shows another example of a system configuration 2400E that includes additional circuitry that can perform various operations (e.g., digital logic operations) to enable the system configuration 2400E (e.g., for a complete electro-optical computing system, or for an optoelectronic system for a larger computing platform) to be used to implement a computing technique, such as artificial neural networks or other forms of machine learning. Data storage subsystem 2450 may include volatile storage media (e.g., SRAM and/or DRAM) and/or non-volatile storage media (e.g., solid state drives and/or hard drives). The data storage subsystem 2450 may also include a hierarchical cache module. The stored data may include, for example, training data, intermediate result data, or production data for feeding to an online computing system (online computing system). The data storage subsystem 2450 may be configured to provide concurrent access (concurrent access) to input data to modulate on different optical signals provided by the optical port or light source 2402. The conversion of data stored in digital form to analog form available for modulation may be performed by circuitry (e.g., a digital-to-analog converter) included at the output of data storage subsystem 2450, or at the input of optical port or light source 2402, or split between the two. An auxiliary processing subsystem (auxiliary processing subsystem)2460 may be configured to perform auxiliary operations (e.g., non-linear operations, data shuffling, etc.) on data, which may be looped through multiple iterations of vector-matrix multiplication using the VMM subsystem 2410. The result data 2462 from those ancillary operations may be sent to the data storage subsystem 2450 in digital form. The data retrieved by the data storage subsystem 2450 may be used to modulate the optical signal with the appropriate input vector and to provide control signals (not shown) used to set the modulation level of the optical amplitude modulator in the VMM subsystem 2410. The conversion of data encoded in analog form on electrical signals to digital form may be performed by circuitry (e.g., an analog-to-digital converter) within auxiliary processing subsystem 2460.

In some embodiments, a digital controller (not shown in the figures) is provided to control the operation of the data storage subsystem 2450, the hierarchical cache module, various circuits (e.g., digital-to-analog and analog-to-digital converters), the VMM subsystem 2410, and the light source 2402. For example, a digital controller is configured to execute program code to implement a neural network having several hidden layers. The digital controller iteratively performs matrix processing associated with each layer of the neural network. The digital controller performs a first iteration of the matrix processing by retrieving first matrix data from the data storage subsystem 2450, wherein the first matrix data represents coefficients of a first layer of the neural network, and setting a modulation level of the optical amplitude modulator in the VMM subsystem 2410 based on the retrieved data. The digital controller takes a set of input data from the data storage subsystem and sets the modulation levels for the light source 2402 to produce a set of optical input signals representing the elements of the first input vector.

The VMM subsystem 2410 performs matrix processing, representing processing of signals by the first layer of the neural network, based on the first input vector and the first matrix data. After the auxiliary processing subsystem 2450 produces the first set of result data 2462, the digital controller performs a second iteration of matrix processing by retrieving second matrix data representing coefficients of a second layer of the neural network from the data storage subsystem, and setting modulation levels of the optical amplitude modulator in the VMM subsystem 2410 based on the second matrix data. The first set of resulting data 2462 is used as a second input vector to set the modulation level of the light source 2402. The VMM subsystem 2410 performs matrix processing, representing processing of signals by the second layer of the neural network, based on the second input vector and the second matrix data, and so on. In the last iteration, an output of the signal processed by the last layer of the neural network is produced.

In some embodiments, when performing calculations associated with the hidden layer of the neural network, the resulting data 2462 is not sent to the data storage subsystem 2450, but is used by the digital controller to directly control a digital-to-analog converter that generates control signals for setting the modulation level of the optical amplitude modulator in the VMM subsystem 2410. This reduces the time required to store data to and access data from data storage subsystem 2450.

Other processing techniques may be incorporated into other examples of system configurations. For example, various techniques used with other kinds of vector-matrix multiplication subsystems (e.g., subsystems that do not have the electrical summing or signed multiplication described herein but use optical interference) may be incorporated into some system configurations, such as some techniques described in U.S. patent publication No. US 2017/0351293, which is incorporated herein by reference.

Referring to fig. 51, in some embodiments, an Artificial Neural Network (ANN) computing system 3200 includes a modulator array 3208. The modulator array 3208 uses a segmented modulator design, such as a segmented MZI modulator, such as those shown in fig. 8-16 and 47-50. For example, when each digital input vector is 4 in length, modulator array 3208 may be configured similar to modulator array 5000 of fig. 50. The ANN calculation system 3200 includes a photoelectric matrix multiplication unit 3220, the photoelectric matrix multiplication unit 3220 having, for example, a replication module, a multiplication module, and a summation module shown in fig. 19 to 37, to realize processing of incoherent or low-coherence optical signals when performing matrix calculation. The artificial neural network computing system 3200 includes a controller 110, a memory unit 120 and an ADC unit 160, similar to those in the system 100 of fig. 1. The artificial neural network computing system 3200 includes a modulator control unit 3202, the modulator control unit 3202 including a first DAC subunit 3204 and a second DAC subunit 3206. The first DAC subunit 3204 includes high-speed 1-bit DACs, where each 1-bit DAC drives a phase shifter, such as 3956 in fig. 8, 10, 11, and 15, 3960, 3968, and 3972 in fig. 9, 4714 in fig. 47-49, or sub-modulators 5022, 5024, 5026, and 5028 in fig. 50. For example, each 1-bit DAC may generate a 1-bit modulator control signal. The 1-bit DAC may receive a binary digital output directly from the controller 110 and condition the binary signal to a two-level voltage or current output suitable for driving a corresponding phase shifter in the modulator array 3208. The first DAC subunit 3204 may include charge pumping drive circuits, such as those shown in fig. 3, 9, 15, and 48, for moving charge to or from the phase shifters. For example, the first DAC subunit 3204 may be configured to perform pre-emphasis and de-emphasis to enhance bandwidth, as described previously. The controller 110 receives requests from the computer 102 and sends computing output to the computer 102, similar to that shown in FIG. 1.

In some examples, the OMM unit 3220 includes an MZI without a segmented design, and the second DAC subunit 3206 may be similar to the second DAC subunit 134 in fig. 1. This configuration is useful for performing ANN calculations where the reconfiguration rate of the OMM unit 3220 is slower than the reconfiguration rate of the modulator array 3208.

In some examples, the OMM unit 3220 includes an MZI having a segmented design, and the second DAC subunit 3206 may include a 1-bit DAC similar to the first DAC subunit 3204. The second DAC subunit 3206 may also include charge pumping drive circuits similar to the first DAC subunit 3204 and perform pre-emphasis and de-emphasis to enhance bandwidth.

The optoelectronic processor 3210 includes a light source 3230, which may be similar to the laser unit 142 of fig. 1, wherein a plurality of output signals of the light source 3230 are coherent. The light source 3230 may also use light emitting diodes to generate multiple output signals that are incoherent or have low coherence. The modulator array 3208 receives modulator control signals generated by the first DAC subunit 3204 based on the input vectors. The output of the modulator array 3208 may be compared to the output of the optical port/light source 1802 in fig. 19. The optical signal from the modulator array 3208 is processed by the electro-optical matrix multiplication unit 3220 in a manner similar to the way the replication module 1804, multiplication module 1806, and summation module 1808 process the optical signal from the optical port/light source 1802 in fig. 19.

Referring to fig. 52, the photo matrix multiplication unit 3220 receives an input vectorAnd multiplying the input vector by the matrixTo generate outputOutput vector

The electro-optical matrix multiplication unit 3220 includes m optical paths 1803_1, 1803_2, … …, 1803_ m (collectively 1803) carrying optical signals representing input vectors. The replication module 1804_1 inputs the optical signal v₁Are provided to the multiplication modules 1806_11, 1806_21, … …, 1806_ m 1. The replication module 1804_2 inputs the optical signal v₂Are provided to the multiplication modules 1806_12, 1806_22, … …, 1806_ m 2. The replication module 1804_ n inputs the optical signal v_nAre provided to the multiplication modules 1806_1n, 1806_2n, … …, 1806_ mn.

As described above, the optical signal v provided by the replication module 1804_1₁Are the same (or substantially the same) in amplitude relative to each other, but are the same as the optical signal v provided by the modulator array 3208₁Are different in magnitude. For example, if the replication module 1804_1 evenly divides v provided by the modulator array 3208 among the m signals₁Then each of the m signals will have a v equal to or less than that provided by the modulator array 3208₁1/m of the power of (1).

The multiplication module 1806_11 inputs the signal v₁And matrix element M₁₁Multiplying to produce M₁₁·v₁. The multiplication module 1806_21 inputs the signal v₁And matrix element M₂₁Multiplying to produce M₂₁·v₁. The multiplication module 1806_ m1 inputs the signal v₁And matrix element M_m1Multiplying to produce M_m1·v₁. The multiplication module 1806_12 inputs the signal v₂And matrix element M₁₂Multiplying to produce M₁₂·v₂. The multiplication module 1806_22 inputs the signal v₂And matrix element M₂₂Multiplying to produce M₂₂·v₂. The multiplication module 1806_ m2 inputs the signal v₂And matrixElement M_m2Multiplying to produce M_m2·v₂. The multiplication module 1806_1n inputs the signal v_nAnd matrix element M_1nMultiplying to produce M_1n·v_n. The multiplication module 1806_2n inputs the signal v_nAnd matrix element M_2nMultiplying to produce M_2n·v_n. The multiplication module 1806_ mn inputs the signal v_nAnd matrix element M_mnMultiplying to produce M_mn·v_nAnd so on.

The second DAC subunit 134 generates control signals based on the values of the matrix elements and transmits the control signals to the multiplication module 1806 to enable the multiplication module 1806 to multiply the values of the input vector elements by the values of the matrix elements, for example using optical amplitude modulation. For example, the multiplication module 1806_11 may comprise an optical amplitude modulator and may be implemented by combining matrix elements M₁₁As applied to represent the input vector element v ₁By the level of amplitude modulation of the input optical signal, the input vector elements v can be realized₁Multiplication by matrix element M₁₁。

The summing module 1808_1 receives the outputs of the multiplication modules 1806_11, 1806_12, … …, 1806_1n and produces a signal equal to M₁₁v₁+M₁₂v₂+…+M_1nv_nSum of (a) y₁. The summing module 1808_2 receives the outputs of the multiplication modules 1806_21, 1806_22, … …, 1806_2n and produces a signal equal to M₂₁v₁+M₂₂v₂+…+M_2nv_nSum of (a) y₂. The summing module 1808_ M receives the outputs of the multiplication modules 1806_ M1, 1806_ M2, … …, 1806_ mn and produces an output equal to M_m1v₁+M_m2v₂+ …+M_mnv_nSum of (a) y_m。

In the system 3200, the output of the photo matrix multiplication unit 3220 is provided to the ADC unit 160 without passing through the analog electronics unit 146 as is the case in the system 100 of fig. 1. This is because the multiplication module 1806 or the summation module 1808 has already converted the optical signal into an electrical signal, so no separate analog electronic unit 146 is needed in the system 3200.

Fig. 53 illustrates a flow diagram of an example of a method 3300 of performing an ANN calculation using the ANN calculation system 3200 of fig. 51. The steps of method 3300 may be performed by controller 110 of system 3200. In some embodiments, the steps of method 3300 may be run in parallel, combined, looped, or in any order.

In step 3310, an Artificial Neural Network (ANN) computation request including an input data set and a first plurality of neural network weights is received. The input data set includes a first numeric input vector. The first digital input vector is a subset of the input data set. For example, it may be a sub-region of the image. The ANN calculation request may be generated by various entities (e.g., computer 102 of fig. 51). The computer 102 may include one or more of various types of computing devices, such as a personal computer, a server computer, a vehicle computer, and a flight computer. The ANN calculation request generally refers to an electrical signal informing or informing the ANN calculation system 3200 of the ANN calculation to be performed. In some embodiments, the ANN calculation request may be split into two or more signals. For example, a first signal may query (query) ANN computing system 3200 to check whether system 3200 is ready to receive the input data set and the first plurality of neural network weights. In response to an acknowledgement by system 3200, computer 102 may transmit a second signal comprising the input data set and the first plurality of neural network weights.

In step 3320, the input data set and the first plurality of neural network weights are stored. Controller 110 may store the input data set and the first plurality of neural network weights in memory unit 120. Storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow flexibility in the operation of the ANN computing system 3200, e.g., may improve the overall performance of the system. For example, the input data set may be divided into digital input vectors of a set size and format by taking (retrieve) the desired portion of the input data set from the memory unit 120. Different portions of the input data set may be processed in various orders or shuffled (shredded) to allow various types of ANN calculations to be performed. For example, shuffling may allow matrix multiplication to be performed by block matrix multiplication techniques where the input and output matrix sizes are different. As another example, storing the input data set and the first plurality of neural network weights in the memory unit 120 may allow for queuing of the plurality of ANN computation requests by the ANN computation system 3200, which may allow the system 3200 to maintain operation at its full speed without inactive time periods.

In some embodiments, memory unit 120 may include a first memory sub-unit and a second memory sub-unit. The first memory subunit may be a memory dedicated to storing input data sets and digital input vectors and may have an operating speed sufficient to support the modulation rate of the modulator array 3208. The second memory subunit may be a memory dedicated to storing neural network weights and may have an operating speed sufficient to support the reconfiguration rate of the OMM unit 3220. In some embodiments, the first memory sub-unit may be implemented using SRAM and the second memory sub-unit may be implemented using DRAM. In some embodiments, the first memory sub-unit and the second memory sub-unit may be implemented using DRAMs. In some embodiments, the first memory subunit may be implemented as part of the controller 110 or as a cache of the controller 110. In some embodiments, the first and second memory subunits may be implemented by a single physical memory device as different address spaces. In some embodiments, the input data set may be stored in a first memory subunit and the first plurality of neural network weights may be stored in a second memory subunit.

In step 3330, a first plurality of modulator control signals is generated based on the first digital input vector and a first plurality of weight control signals is generated based on the first plurality of neural network weights. The controller 110 may send the first DAC control signal to the modulator control unit 3202 to generate a first plurality of modulator control signals. The first DAC subunit 3204 generates a first plurality of modulator control signals based on the first DAC control signal, and the modulator array 3208 generates an optical input vector that represents a first digital input vector.

The first DAC control signal may include a plurality of digital values to be converted into a first plurality of modulator control signals by the 1-bit DAC in the first DAC subunit 3204. The plurality of digital values generally corresponds to the first digital input vector and may be associated by various mathematical relationships or look-up tables. For example, the plurality of digital values may be linearly proportional to the values of the elements of the first digital input vector. As another example, the plurality of digital values may be associated with elements of the first digital input vector through a lookup table configured to maintain a linear relationship between the digital input vector and an optical input vector generated by the modulator array 3208.

The controller 110 may send the second DAC control signal to the modulator control unit 3202 to generate a first plurality of weight control signals. The second DAC subunit 3206 generates a first plurality of weight control signals based on the second DAC control signal, and the photo matrix multiplication unit 3220 is reconfigured according to the first plurality of weight control signals to implement a matrix corresponding to the first plurality of neural network weights.

The second DAC control signal may include a plurality of digital values to be converted into a first plurality of weight control signals by the second DAC subunit 3206. The plurality of digital values generally correspond to the first plurality of neural network weights and may be associated by various mathematical relationships or look-up tables. For example, the plurality of digital values may be linearly proportional to the first plurality of neural network weights. As another example, the plurality of digital values may be calculated by performing various mathematical operations on the first plurality of neural network weights to generate weight control signals, which may configure the photoelectric matrix multiplication unit 3220 to perform matrix multiplication corresponding to the first plurality of neural network weights.

In step 3340, a first plurality of digitized outputs corresponding to the electrical output vector of the photo matrix multiplication unit 3220 is obtained. The optical input vector produced by the modulator array 3208 is processed by the photo matrix multiplication unit 3220 and converted to an electrical output vector. The electrical output vector is converted into digitized values by the ADC unit 160. The controller 110 may, for example, send a conversion request to the ADC unit 160 to start converting the voltage output by the photo matrix multiplication unit 3220 to a digitized output. Once the conversion is completed, the ADC unit 160 may send the conversion result to the controller 110. Alternatively, the controller 110 may retrieve the conversion result from the ADC unit 160. The controller 110 may form a digital output vector from the digitized output, the digital output vector corresponding to a result of a matrix multiplication of the input digital vector. For example, the digitized output may be organized or concatenated to have a vector format.

In some embodiments, ADC unit 160 may be set or controlled to perform ADC conversion based on the DAC control signal being issued by controller 110 to modulator control unit 3202. For example, the ADC conversion may be set to start at a preset time after the modulator control unit 3202 generates the modulation control signal. Such control of the ADC conversion may simplify the operation of the controller 110 and reduce the number of necessary control operations.

In step 3350, a non-linear transformation is performed on the first digital output vector to produce a first transformed digital output vector. The nodes or artificial neurons of the ANN operate by first performing a weighted sum of the signals received from the nodes of the previous layer, and then performing a nonlinear transformation ("activation") of the weighted sum to produce an output. Various types of ANN may implement various types of differentiable nonlinear transformations. Examples of the nonlinear transformation function include a modified linear unit (RELU) function, an S-type function, a hyperbolic tangent function, and X²A function and a | X | function. This non-linear transformation is performed on the first digital output by the controller 110 to produce a first transformed digital output vector. In some embodiments, the non-linear transformation may be performed by a dedicated digital integrated circuit within the controller 110. For example, controller 110 may include one or more modules or circuit blocks that are particularly suited to speed up the computation of one or more types of non-linear transformations.

In step 3360, the first transformed digital output vector is stored. The controller 110 may store the first transformed digital output vector in the memory unit 120. In the case where the input data set is divided into a plurality of digital input vectors, the first transformed digital output vector corresponds to the result of an ANN calculation of a portion of the input data set, e.g., the first digital input vector. As such, storing the first transformed digital output vector allows the ANN computing system 3200 to perform and store additional computations on other digital input vectors of the input data set to be later aggregated into a single ANN output.

In step 3370, an artificial neural network output generated based on the first transformed digital output vector is output. The controller 110 generates an ANN output that is a result of processing the input data set through an ANN defined by the first plurality of neural network weights. Where the input data set is divided into a plurality of digital input vectors, the resultant ANN output is an aggregated output comprising the first converted digital output, but may further comprise additional converted digital outputs corresponding to other portions of the input data set. Once the ANN output is generated, the generated output is sent to the computer (e.g., computer 102) that initiated the ANN computation request.

In some embodiments, the controller 110, the memory unit 120, the modulator control unit 3202, the ADC unit 160, and the microprocessor may be monolithically integrated on a semiconductor die. In some embodiments, the controller 110, the memory unit 120, the modulator control unit 3202, the ADC unit 160, the microprocessor, and the system main memory may be integrated as a system on chip. In such examples, two or more steps in process 3300 may be performed by individual modules in a monolithic integrated circuit or system-on-a-chip.

Various performance metrics may be defined for the ANN computing system 3200 implementing the method 3300. Defining performance metrics may allow the performance of the ANN computing system 3200 implementing the optoelectronic processor 3210 to be compared to the performance of other systems for ANN computation that instead implement an electronic matrix multiplication unit (electronic matrix multiplication unit). In one aspect, the rate at which the ANN computation can be performed can be indicated in part by a first cycle period defined as the elapsed time between the step 3320 of storing the input data set and the first plurality of neural network weights in the memory unit and the step 3360 of storing the first transformed digital output vector in the memory unit. Thus, the first cycle period includes the time it takes to convert the electrical signal to an optical signal (e.g., step 3330), perform a matrix multiplication (e.g., step 3340) in the optical and electrical domains. Both steps 3320 and 3360 involve storing data in the memory unit 120, which is a step shared between the ANN computing system 3200 and a conventional ANN computing system without the optoelectronic processor 3210. As such, measuring the first cycle period of the memory-to-memory transaction time may allow for an actual or fair comparison of ANN computation throughput between the ANN computation system 3200 and an ANN computation system without the optoelectronic processor 3210 (e.g., a system implementing an electrical matrix multiplication unit).

Because of the rate at which the modulator array 3208 may produce optical input vectors (e.g., at 25GHz) and the processing rate of the photo matrix multiplication unit 3220 (e.g., >25GHz), the first cycle period of the ANN computation system 3200 used to perform a single ANN computation of a single digital input vector may be close to the inverse of the speed of the modulator array 3208 (e.g., 40 ps). The first cycle period may be, for example, less than or equal to 100ps, less than or equal to 200ps, less than or equal to 500ps, less than or equal to 1ns, less than or equal to 2ns, less than or equal to 5ns, or less than or equal to 10ns, after taking into account the delay associated with signal generation by the modulator control unit 3202 and ADC conversion by the ADC unit 160.

In comparison, the multiplication times of the M × 1 vector and the M × M matrix of an electrical matrix multiplication unit are usually equal to M²Proportional to-1 processor clock cycle. For M-32, this multiplication will take about 1024 cycles, which results in a run time of over 300ns at 3GHz clock speed, which is orders of magnitude slower than the first cycle period of the ANN computing system 3200.

In some embodiments, the method 3300 further includes the step of generating a second plurality of modulator control signals based on the first converted digital output vector. In some types of ANN calculations, a single numerical input vector may be repeatedly propagated through or processed by the same ANN. As described above, ANNs that implement multi-pass processing (multi-pass processing) may be referred to as Recurrent Neural Networks (RNNs). The RNN is a neural network in which the output of the network is recycled back to the input of the neural network during the (k) th pass through the neural network and used as input during the (k +1) th pass. The RNN may have various applications in pattern recognition tasks, such as speech or handwriting recognition. Once the second plurality of modulator control signals are generated, method 3300 may proceed to steps 3340 through 3360 to complete the second pass of the first digital input vector through the ANN. In general, recycling of the transformed digital output into the digital input vector may be repeated for a predetermined number of cycles, depending on the characteristics of the RNN received in the ANN calculation request.

In some embodiments, the method 3300 further includes the step of generating a second plurality of weight control signals based on the second plurality of neural network weights. In some cases, the artificial neural network computation request further includes a second plurality of neural network weights. As noted above, generally, an ANN has one or more hidden layers in addition to the input and output layers. For an ANN having two hidden layers, the second plurality of neural network weights may correspond to, for example, connectivity between a first layer of the ANN and a second layer of the ANN. To process the first digital input vector through the two hidden layers of the ANN, the first digital input vector may first be processed according to method 3300 until step 3360, where the result of processing the first digital input vector through the first hidden layer of the ANN at step 3360 is stored in memory unit 120. The controller 110 then reconfigures the photoelectric matrix multiplication unit 3220 to perform a matrix multiplication corresponding to a second plurality of neural network weights associated with a second hidden layer of the ANN. Once the photo matrix multiplication unit 3220 is reconfigured, the method 3300 may generate a plurality of modulator control signals based on the first transformed digital output vector, which generates an updated optical input vector corresponding to the output of the first hidden layer. The updated optical input vector is then processed by the reconfigured photo matrix multiplication unit 3220, the photo matrix multiplication unit 3220 corresponding to the second hidden layer of the ANN. Generally, the steps described may be repeated until the digital input vector has been processed through all hidden layers of the ANN.

In some embodiments of the optoelectronic matrix multiplication unit 3220, the rate of reconfiguration of the optoelectronic matrix multiplication unit 3220 may be significantly slower than the modulation rate of the modulator array 3208. In this case, the throughput of the ANN calculation system 3200 may be adversely affected by the amount of time it takes to reconfigure the photo matrix multiplication unit 3220 during which the ANN calculation cannot be performed. To mitigate the effects of the relatively slow reconfiguration time of the electro-optical matrix multiplication unit 3220, batch processing techniques may be utilized in which two or more digital input vectors are propagated through the electro-optical matrix multiplication unit 3220 without configuration changes to spread the reconfiguration time (amortize) over a larger number of digital input vectors.

Referring to fig. 39, in some embodiments, a Wavelength Division Multiplexing (WDM) Artificial Neural Network (ANN) computing system 3500 includes an optoelectronic processor 3510, the optoelectronic processor 3510 including an optoelectronic matrix multiplication unit 3520, the optoelectronic matrix multiplication unit 3520 having a replication module, a multiplication module, and a summation module as shown in fig. 19-37 to enable processing of incoherent or low coherence optical signals when performing matrix calculations, wherein the optical signals are encoded at a plurality of wavelengths. The WDM ANN computing system 3500 is similar to the ANN computing system 3200 except that WDM technology is used therein, for some embodiments of the ANN computing system 3500, the light source 3230 is configured to generate a plurality of wavelengths, e.g., λ 1, λ 2, and λ 3, similar to the system 104 of fig. 6.

The multiple wavelengths may preferably be separated by a wavelength spacing large enough to allow easy multiplexing and demultiplexing onto a common propagation channel. For example, wavelength intervals greater than 0.5nm, 1.0nm, 2.0nm, 3.0nm, or 5.0nm may allow for simple multiplexing and demultiplexing. On the other hand, the range between the shortest and longest wavelengths of the plurality of wavelengths ("WDM bandwidth") may preferably be small enough such that the characteristics or performance of the electro-optical matrix multiplication unit 3520 remain substantially the same over the plurality of wavelengths. Optical components are typically dispersive (meaning that their optical properties vary with wavelength). For example, the power split ratio of the MZI may vary with wavelength. However, by designing the electro-optical matrix multiplication unit 3520 to have a sufficiently large operating wavelength window (operating wavelength window), and by limiting the wavelengths within the operating wavelength window, the electrical output vector output by the electro-optical matrix multiplication unit 3520 for each wavelength may be a sufficiently accurate result of the matrix multiplication implemented by the electro-optical matrix multiplication unit 3520. The operating wavelength window may be, for example, 1nm, 2nm, 3nm, 4nm, 5nm, 10nm, or 20 nm.

The modulator array 144 of the WDM ANN computing system 3500 includes a plurality of optical modulator sets (banks of optical modulators) configured to generate a plurality of optical input vectors, each of the optical modulator sets corresponding to one of the plurality of wavelengths and generating a respective optical input vector having a respective wavelength. For example, for a system having optical input vectors of lengths of 32 and 3 wavelengths (e.g., λ 1, λ 2, and λ 3), modulator array 144 may have 3 groups of 32 modulators per group. In addition, the modulator array 144 also includes an optical multiplexer configured to combine the plurality of optical input vectors into a combined optical input vector that includes the plurality of wavelengths. For example, an optical multiplexer may combine the outputs of three modulator groups of three different wavelengths into a single propagation channel (e.g., waveguide) for each element of an optical input vector. As such, returning to the example above, the combined optical input vector will have 32 optical signals, each signal comprising 3 wavelengths.

The optoelectronic processing component of the WDM ANN computing system 3500 is further configured to demultiplex the plurality of wavelengths and generate a plurality of demultiplexed output electrical signals. Referring to fig. 40, the electro-optical matrix multiplication unit 3520 includes an optical path 1803, the optical path 1803 configured to receive a combined optical input vector including a plurality of wavelengths from the modulator array 144. For example, optical path 1803_1 receives combined optical input vector element v at wavelengths λ 1, λ 2, and λ 3 ₁. Optical input vector element v at wavelengths λ 1, λ 2 and λ 3₁Are provided to the multiplication modules 3530_11, 3530_21, … …, and 3530_ m 1. In some embodiments where the multiplication module 3530 outputs an electrical signal, the multiplication module 3530_11 outputs a representation of M₁₁·v₁At wavelengths λ 1, λ 2 andinput vector element v of 3₁. Corresponding to input vector elements v at wavelengths λ 1, λ 2 and λ 3₁The output electrical signals of the multiplication module 3530_11 are shown as (λ 1), (λ 2), and (λ 3), respectively. Similar signs apply to the outputs of the other multiplication modules. The multiply module 3530_21 outputs a representation M₂₁·v₁Corresponding to the input vector elements v at the wavelengths λ 1, λ 2 and λ 3, respectively₁. The multiplication module 3530_ M1 outputs a representation M_m1·v₁Corresponding to the input vector elements v at the wavelengths λ 1, λ 2 and λ 3₁。

Optical input vector element v at wavelengths λ 1, λ 2 and λ 3₂Are provided to the multiplication modules 3530_12, 3530_22, … …, and 3530_ m 2. Multiplication module 3530_12 outputs a representation of M₁₂·v₂Corresponding to the input vector elements v at the wavelengths λ 1, λ 2 and λ 3₂. Multiplication module 3530_22 outputs a representation M ₂₂·v₂Corresponding to a wavelength lambda₁、λ₂And λ₃Input vector element v of₂. The multiplication module 3530_ M2 outputs a representation M_m2·v₂Corresponding to the input vector elements v at the wavelengths λ 1, λ 2 and λ 3₂。

Optical input vector element v comprising wavelengths λ 1, λ 2 and λ 3_nAre provided to the multiplication blocks 3530_1n, 3530_2n, … …, and 3530_ mn. The multiplication module 3530_1n outputs a representation M_1n·v_nCorresponding to a wavelength lambda₁、λ₂And λ₃Input vector element v of_n. The multiplication module 3530_2n outputs a representation M_2n·v_nCorresponding to the input vector elements v at the wavelengths λ 1, λ 2 and λ 3_n. The multiplication module 3530_ mn outputs a representation of M_mn·v_nCorresponding to the input vector elements v at the wavelengths λ 1, λ 2 and λ 3_nAnd so on.

For example, each multiplication module 3530 may include a demultiplexer configured to demultiplex three wavelengths contained in each of the 32 signals of the multi-wavelength optical vector and route (route) 3 single-wavelength optical output vectors to three photodetector groups (e.g., photodetectors 2012, 2016 (FIG. 23) or 2042, 2046 (FIG. 24)) coupled to three operational or transimpedance amplifier groups (e.g., operational amplifiers 2030 (FIG. 23) or 2050 (FIG. 24)).

Three groups of the summing modules 1808 receive the outputs from the multiplication module 3530 and produce sums y of the input vectors corresponding to the various wavelengths. For example, three summing modules 1808_1 receive the outputs of multiplication modules 3530_11, 3530_12, … …, 3530_1n and produce output vector elements y corresponding to wavelengths λ 1, λ 2, and λ 3, respectively₁Sum of (a) y₁(λ1)、y₁(λ2)、y₁(λ 3) where the sum y at each wavelength₁Is equal to M₁₁v₁+M₁₂v₂+…+M_1nv_n. Three summing modules 1808_2 receive the outputs of the multiplication modules 3530_21, 3530_22, … …, 3530_2n and generate output vector elements y corresponding to wavelengths λ 1, λ 2, and λ 3, respectively₂Sum of (a) y₂(λ1)、y₂(λ2)、 y₂(λ 3) where the sum y at each wavelength₂Is equal to M₂₁v₁+M₂₂v₂+…+M_2nv_n. The three summation modules 1808_ m receive the outputs of the multiplication modules 3530_ m1, 3530_ m2, … …, 3530_ mn and generate output vector elements y corresponding to wavelengths λ 1, λ 2 and λ 3, respectively_mSum of (a) y_m (λ1)、y_m(λ2)、y_m(λ 3) where the sum y at each wavelength_mIs equal to M_m1v₁+M_m2v₂+ …+M_mnv_n。

Referring again to fig. 39, the ADC unit 160 of the WDM ANN computing system 3500 includes a bank of ADCs (banks of ADCs) configured to convert a plurality of demultiplexed output voltages (demultiplexed output voltages) of the photo matrix multiplication unit 3520. Each ADC group corresponds to one of a plurality of wavelengths and produces a corresponding digitized demultiplexed output. For example, a group of ADCs 160 may be coupled to a group of summing modules 1808, for example.

Controller 110 may implement a method similar to method 200 (fig. 4), but extended to support multi-wavelength operation. For example, the method may comprise the steps of: obtaining a plurality of digitized demultiplexed outputs from the ADC unit 160, the plurality of digitized demultiplexed outputs forming a plurality of first digital output vectors, wherein each of the plurality of first digital output vectors corresponds to one of the plurality of wavelengths; performing a non-linear transformation on each of the plurality of first digital output vectors to produce a plurality of transformed first digital output vectors; and storing the plurality of transformed first digital output vectors in a memory unit.

In some cases, the ANN may be specially designed and the digital input vectors may be specifically formed such that the multi-wavelength product (multi-wavelength product) of the multiplication module 3530 may be added without demultiplexing. In this case, the multiplication module 3530 may be a wavelength-insensitive (wavelegnth-insensitive) multiplication module that does not demultiplex multiple wavelengths of the multi-wavelength product. As such, each photodetector of the multiplication module 3530 effectively sums multiple wavelengths of the optical signal into a single photocurrent, and each voltage output by the multiplication module 3530 corresponds to a sum of products of matrix elements and vector elements for the multiple wavelengths. The summation module 1808 (only one group is required) outputs an element-by-element sum (element-by-element sum) of matrix multiplication results of a plurality of digital input vectors.

Fig. 41 shows an example of a system configuration 3500 for an implementation of a wavelength division multiplexed optical-matrix multiplication unit 3520 that performs vector-matrix multiplication using a 2 x 2 matrix of elements, where the summing operation is performed in the electrical domain. In this example, the input vector isAnd the matrix isIn this example, the input vector has a plurality of wavelengths λ 1, λ 2 toAnd 3, and each element of the input vector is encoded on a different optical signal. Two different replication modules 1902 perform optical replication operations to separate computations on different paths (e.g., "up" and "down" paths). There are four multiplication modules 1904, each multiplication module 1904 multiplying a different matrix element using optical amplitude modulation. The output of each multiplication module 1904 is provided to a demultiplexer and a set of optical detection modules 3501, the optical detection modules 3501 convert the wavelength division multiplexed optical signals into electrical signals in the form of electrical currents associated with wavelengths λ 1, λ 2, and λ 3. The two upper paths of different input vector elements are combined using a set of summation modules 3503 associated with wavelengths λ 1, λ 2 and λ 3, and the two lower paths of different input vector elements are combined using a set of summation modules 3503 associated with wavelengths λ 1, λ 2 and λ 3, where the summation modules 3503 perform the summation in the electrical domain. Thus, for each wavelength, each element of the output vector is encoded on a different electrical signal. As shown in fig. 41, as the calculation progresses, each component of the output vector is incrementally produced to produce the following results for the upper and lower paths, respectively, for each wavelength.

M₁₁v₁+M₁₂v₂

M₂₁v₁+M₂₂v₂

System configuration 3500 may be implemented using any of a variety of opto-electronic technologies. In some embodiments, there is a common substrate (e.g., semiconductor (e.g., silicon)) that can support integrated optical and electronic components. The optical path may be implemented in a waveguide structure having a material with a higher optical index surrounded by a material with a lower optical index (optical index), the waveguide structure defining a waveguide for propagating light waves carrying the optical signal. The electrical path may be implemented by a conductive material for propagating electrical current carrying electrical signals. (in fig. 41, the thickness of the lines representing the paths are used to distinguish between optical paths (represented by thicker lines) and electrical paths (represented by thinner lines or dashed lines).). Alternatively, different portions of the system may be implemented using different devices having different substrates, and those devices may communicate over a communication channel. For example, optical fibers may be used to provide a communication channel to transmit optical signals among multiple devices used to implement an overall system. Those optical signals may represent different subsets of input vectors provided when performing vector-matrix multiplication, and/or different subsets of intermediate results calculated when performing vector-matrix multiplication, as described in more detail below.

Various alternative system configurations or signal processing techniques may be used with various embodiments of the different systems, subsystems and modules described herein.

In some embodiments, it may be useful for some or all of the VMM subsystems to be replaced with alternative subsystems, including subsystems using different embodiments of various replication, multiplication, and/or summation modules. For example, the VMM subsystems may include the optical replication module described herein and the electrical summation module described herein, but the multiplication module may be replaced with a subsystem that performs multiplication operations in the electrical domain rather than the optical-electrical domain. In this example, the optical amplitude modulator array may be replaced by a detector array to convert the optical signal to an electrical signal, followed by an electronic subsystem (e.g., an ASIC, processor, or SoC). Alternatively, if optical signal routing is to be used for a summing module configured to detect optical signals, the electronics subsystem may include, for example, electro-optical conversion using an array of electrically modulated optical sources.

In some embodiments, it may be useful to be able to use a single wavelength for some or all optical signals for some or all VMM computations. Alternatively, in some embodiments, to help reduce the number of optical input ports that may be needed, the input ports may receive multiplexed optical signals having different values encoded on different optical waves of different wavelengths. Those light waves may then be separated at appropriate locations in the system, depending on whether any of the replication, multiplication, and/or summation modules are configured to operate on multiple wavelengths. However, even in a multi-wavelength embodiment, it may be useful to use the same wavelength, for example, for different subsets of optical signals used in the same VMM subsystem.

In some embodiments, accumulators may be used to implement time-domain encoding of optical and electrical signals received by the various modules, thereby alleviating the need for electronic circuitry to operate efficiently over a large number of different power levels. For example, a signal encoded using binary (on-off) amplitude modulation with a specific duty cycle over N time slots of each symbol may be converted into a signal with N amplitude levels per symbol after the signal passes through an accumulator (an analog electronic accumulator that integrates the current or voltage of the electrical signal). Thus, if optical devices (e.g., phase modulators in optical amplitude modulators) are capable of operating at a symbol bandwidth (B), they may also operate at a symbol bandwidth of B/100, where each symbol value uses N-100 time slots. A 50% integrated amplitude has a 50% duty cycle (e.g., the first 50 time slots at a non-zero "on" level followed by 50 time slots at a zero or near-zero "off" level), while a 10% integrated amplitude has a 10% duty cycle (e.g., the first 10 time slots at a non-zero "on" level followed by 90 time slots at a zero "off" level). In the examples described herein, such an accumulator may be positioned on the path of each electrical signal anywhere within the VMM subsystem, consistent for each electrical signal, e.g., before or after the summing module for all electrical signals in the VMM subsystem. The VMM subsystem may also be configured so that there is no significant relative time offset between the different electrical signals that maintain the alignment of the different symbols.

Referring to fig. 42, in some embodiments, homodyne detection may be used to derive the phase and amplitude of the modulated signal. Homodyne detector 4000 includes a beam splitter including a 2 x 2 multimode interference (MMI) coupler4002. Two photodetectors 4004a and 4004b, and a subtractor 4006. Beam splitter 4002 receives input signal E₁And E₂The output of the beam splitter 402 is detected by photodetectors 4004a and 4004 b. For example, the input signal E₁May be the signal to be detected, and the input signal E₂May be generated by a local oscillator with constant laser power. The local oscillator signal E is split by the beam splitter 4002 before the signal is detected by the photodetectors 4004a and 4004b₂And an input signal E₁And (4) mixing. The subtractor 4006 outputs the difference between the outputs of the photodetectors 4004a and 4004 b. Output 4008 of subtractor 4006 and | E₁||E₂In (θ), where E₁I and I E₂I is the amplitude of the two input optical fields and θ is their relative phase. Since the output is related to the product of two optical fields, extremely weak optical signals can be detected even at the single photon level.

For example, the homodyne detector 4000 may be used in the systems shown in fig. 1, 6, 19 to 38, and 39 to 41. The homodyne detector 4000 provides gain on the signal and therefore a better signal to noise ratio (signal noise ratio). For coherent systems, homodyne detector 4000 provides the additional benefit of revealing phase information of the signal by detecting the polarity of the result.

In the example of fig. 21, system 1920 includes a 2 x 2 matrix of elements in which two different respective wavelengths λ are used₁And λ₂Two input vector elements are encoded on two optical signals. Two optical signals may be provided to system 1920 using, for example, two optical fibers. For example, a system performing matrix processing on a 4 x 4 matrix may receive four input optical signals carried on four optical fibers. Although more optical fibers may be used to carry more input optical signals for systems that handle larger matrices, it is difficult to couple a large number of optical fibers to an optoelectronic chip because the coupling between the optical fibers and the optoelectronic chip takes up considerable space.

One way to reduce the number of optical fibers required to carry the optical signal to the optoelectronic chip is to use wavelength division multiplexing. Can be used forA single optical fiber is used to multiplex and transmit a plurality of optical signals having different wavelengths. For example, referring to FIG. 43, in computing system 4100, there is a wavelength λ₁Is modulated by a first modulator 4104 to produce a first modulated optical signal 4120 representing a first input vector element V1. Having a wavelength λ₂Is modulated by a second modulator 4108 to produce a second modulated optical signal 4122 representing a second input vector element V2. The first and second modulated optical signals are combined by a multiplexer 4110 to generate a wavelength division multiplexed signal that is transmitted over an optical fiber 4112 to an opto-electronic chip 4114, the opto-electronic chip 4114 comprising a plurality of matrix multiplication modules 4116a, 4116b, 4116c, and 4116d (collectively 4116) and 4118a, 4118b, 4118c, and 4118d (collectively 4118).

Inside the optoelectronic chip 4114, the wavelength division multiplexed signal is demultiplexed by the demultiplexer 4150 to separate the optical signal 4120 and the optical signal 4122. In this example, the optical signal 4120 is replicated by the replication module 4152 to produce a replica of the optical signal sent to the matrix multiplication modules 4116a and 4118 a. The optical signal 4122 is replicated by a replication module 4154 to produce a replica of the optical signal sent to the matrix multiplication modules 4116b and 4118 b. The outputs of the matrix multiplication modules 4116a and 4116b are combined using optical coupler 4120a and the combined signal is detected by photodetector 4122 a.

Having a wavelength λ₁Is modulated by a third modulator 4128 to generate a third modulated optical signal 4132 representative of a third input vector element V3. Having a wavelength λ₂Is modulated by a fourth modulator 4130 to generate a fourth modulated optical signal 4134 representative of a fourth input vector element V4. The third and fourth modulated optical signals are combined by the multiplexer 4136 to produce a wavelength division multiplexed signal that is transmitted through the optical fiber 4138 to the opto-electronic chip 4114.

Inside the optoelectronic chip 4114, the wavelength division multiplexed signal provided by the optical fiber 4138 is demultiplexed by the demultiplexer 4140 to separate the optical signals 4132 and 4134. In this example, the optical signal 4132 is replicated by the replication module 4142 to produce a replica of the optical signal sent to the matrix multiplication modules 4116c and 4118 c. The optical signal 4134 is replicated by the replication module 4144 to produce a replica of the optical signal sent to the matrix multiplication modules 4116d and 4118 d. The outputs of the matrix multiplication units 4116c and 4116d are combined using an optical coupler 4120b, and the combined signal is detected by a photodetector 4122 b. The outputs of the matrix multiplication units 4118a and 4118b are combined using an optical coupler, and the combined signal is detected by a photodetector. The outputs of the matrix multiplication units 4118c and 4118d are combined using optical couplers, and the combined signal is detected by a photodetector.

In some examples, a multiplexer may multiplex optical signals having three or more (e.g., 10 or 100) wavelengths to produce a wavelength division multiplexed signal transmitted by a single optical fiber, and a demultiplexer internal to the photo die may demultiplex the wavelength division multiplexed signal to separate signals having different wavelengths. This allows more optical signals to be transmitted in parallel over the optical fiber to the optoelectronic chip, enhancing the data processing throughput of the optoelectronic chip.

In some examples, the laser unit 142 of fig. 1 includes a single laser that provides light waves that can be modulated with different optical signals. In that case, the light waves in the waveguides of the system have a common wavelength that is substantially the same as each other within the resolution of the linewidth of the laser. For example, the light waves may have wavelengths within 1nm of each other. However, the laser unit 142 may also include multiple lasers capable of wavelength division multiplexing using different optical signals modulated onto different respective optical waves (e.g., each having a linewidth of 1nm or less). The different light waves may have peak wavelengths that are separated from each other by a wavelength distance greater than the line width of the respective laser (e.g., greater than 1 nm). In some examples, wavelength division multiplexed systems may use optical signals modulated onto light waves having wavelengths a few nanometers apart (e.g., 3nm or more). However, if the demultiplexer has a better resolution, the difference between different wavelengths in a WDM system can also be less than 3 nm.

Fig. 44 shows a schematic diagram of an example of the OMM unit 150 of fig. 1 and 6. The OMM unit 150 may include an array of input waveguides 152 to receive the optical input vectors; an optical interference unit 154 in optical communication with the array of input waveguides 152; and an array of output waveguides 156 in optical communication with the optical interference unit 154. The optical interference unit 154 linearly transforms the optical input vector into a second array of optical signals. The array of output waveguides 156 guides a second array of optical signals output by the optical interference unit 154. At least one input waveguide of the array of input waveguides 152 is in optical communication with each output waveguide of the array of output waveguides 156 through an optical interference unit 154. For example, for an optical input vector of length N, the OMM unit 150 may include N input waveguides 152 and N output waveguides 156.

The optical interference unit 154 may include a plurality of interconnected mach-zehnder interferometers (MZIs). FIGS. 45 and 46 show schematic diagrams of example configurations 157 and 158 of an interconnect MZI. The MZIs may be interconnected in various ways (e.g., in configurations 157 or 158) to enable linear transformation of optical input vectors received through the array of input waveguides 152. The MZI may be implemented using the examples shown in fig. 2 and 7-16.

In the examples shown in fig. 7-16, each MZI includes two balanced arms 3904a and 3904 b. When the phase shifters (e.g., 3906 of fig. 7 and 3956 of fig. 8-11, 15, 16) do not impart an optical phase shift to light propagating in the arms 3904a, 3904b, the light propagating in the two balanced arms 3904a, 3904b should constructively interfere when combined at the multimode interference coupler (MMI _1x2)3902 b. However, due to manufacturing tolerances, the two arms 3904a and 3904b may impart different amounts of optical phase shift to light propagating in the arms 3904a, 3904b, resulting in errors in the signal output from the multi-mode interference coupler 3902 b. This inaccuracy can be compensated for by using calibrated phase shifters.

Fig. 47 shows a schematic diagram of a segmented MZI modulator 4700 including calibrated phase shifters 4702a and 4702b (collectively 4702) that can compensate for an imbalance between the two arms of MZI modulator 4700. The segmented MZI modulator 4700 includes an input waveguide 4704, an output waveguide 4706, an input 1 × 2MMI splitter 4708, an output 1 × 2MMI splitter 4710, an upper arm waveguide 4712a, a lower arm waveguide 4712b, and a plurality of electro-optical phase shifters 4714a, 4714b, 4714c, 4714d, 4714e, 4714f, 4714g, and 4714h (collectively 4714).

During system operation, light propagates from the input waveguide 4704 to the 1 × 2MMI splitter 4708, which produces two light portions. Each light portion is affected by an electro-optical phase shifter 4714, and the two light portions are then combined at the 1 x 2MMI 4710 to interfere so that the optical intensity at the output waveguide 4706 will be related to the relative phase difference between the two arms 4712a and 4712 b. By implementing different lengths of phase shifter 4714, the output light may encode multiple levels of electrical signals through several binary on-off keying digital inputs.

Due to the manufacturing process, the upper arm waveguide 4712a and the lower arm waveguide 4712b may differ in waveguide width, and the phase difference due to the waveguide width variation will accumulate along the direction of propagation between the two arms, so that the initial phase imbalance between the two arms 4712a, 4712b may move the initial operating point of the MZI away from constructive interference. In addition, the plurality of electro-optic phase shifters 4714 may experience mask misalignment during manufacturing, which may also cause an initial phase difference between the two arms 4712a, 4712 b. Two calibrated phase shifters 4702a, 4702b are used to eliminate manufacturing induced phase imbalance.

The calibration phase shifter 4702 may be implemented, for example, by thermo-optic phase effect (operating by heating a waveguide to change the refractive index) or electro-optic effect (operating by applying an electric field to change the carrier distribution to affect the refractive index). for example, the calibration phase shifter 4702 is independently controlled by two low-speed DACs that are flip-chip or wire bonded (wire bonding) to a photonic chip containing the MZI 4700.

For example, a tap waveguide (tap waveguide) is provided to guide a portion of the optical signal at the output waveguide 4706 to a monitor photodetector, which can provide a feedback signal indicative of the intensity of the light propagating in the output waveguide 4706. To calibrate the MZI 4700, the phase shifter 4714 is driven with a zero signal such that any imbalance between the phases of the light portions in the upper and lower arms is caused by the difference between the waveguides 4712a, 4712b in the upper and lower arms. Based on the feedback provided by the monitoring photodetectors, the control signals applied to the calibration phase shifters 4702a, 4702b are adjusted so that the light portions from the upper and lower arms that reach the output 1 × 2MMI splitter 4710 have the same phase and constructively interfere.

The phase difference between the upper and lower arms may be affected by ambient temperature, which may fluctuate over time, and therefore it is necessary to periodically perform monitoring measurements by the photodetectors and periodically update the control signals applied to the calibrated phase shifter 4702, for example, once every fraction of a second, once every 10 seconds, or once every minute. The calibrated phase shifter 4702 is configured to be able to compensate for phase imbalance in the two arms in the range of 0 to 2 pi radians. Because the control signals to calibration phase shifter 4702 may be updated at a low frequency (e.g., 10Hz or less), high-precision multi-level (level) DACs may be used to generate the control signals to calibration phase shifter 4702.

Referring to fig. 48, in some embodiments, in the lower arm of MZI 4700, electro-optical phase shifters 4714e, 4714f, 4714g, and 4714h are driven by driver circuits 3966e, 3966f, 3966g, and 3966h, respectively. The alignment phase shifter 4702b may be driven by a driver circuit 4716b similar to driver circuit 3966. For example, the driver circuits 3966e, 3966f, 3966g, 3966h may be driven by control signals provided by the high-speed 1-bit DACs 4718e, 4718f, 4718g, and 4718h, respectively. Driver circuit 4716b (for calibrating phase shifter 4702 b) may be driven by a control signal provided through low-speed multi-bit DAC 4720 b. In the upper arm, electro-optical phase shifters 4714a, 4714b, 4714c, 4714d and calibration phase shifters 4702a may be controlled in a manner similar to electro-optical phase shifters 4714e, 4714f, 4714g, 4714h and calibration phase shifters 4702b, respectively.

As shown in fig. 49, in some embodiments, MZI 4800 includes electro-optical phase shifters 4714a, 4714b, 4714c, 4714d in the upper arm, and calibration phase shifters 4702a, 4702b in both the upper and lower arms. In some examples, an MZI may have electro-optic phase shifters 4714a, 4714b, 4714c, 4714d in the upper arm, electro-optic phase shifters 4714e, 4714f, 4714g, 4714h in the lower arm, and alignment phase shifter 4702a in the upper arm, with no other alignment phase shifters in the lower arm.

Typically, an electro-optical phase shifter for converting a digital electrical signal to an analog optical signal may be placed on a single arm of the MZI, or on both arms of the MZI. The MZI may include a single calibrated phase shifter placed on a single arm, or two calibrated phase shifters placed on both arms, to compensate for phase imbalance between the two arms. In general, placing phase shifters in two arms of an MZI may make each phase shifter shorter than placing a phase shifter in only one arm of the MZI, because each phase shifter need only impart a smaller amount of optical phase shift.

For example, modulators using the segmented design in fig. 8-16 and 47-49 may be used in the modulator array 144 of fig. 1, 6 and 39 and the modulators 2302 of fig. 31-33. If modulator array 144 uses modulators with a segmented design, first MC subunit 132 is modified because it does not require a multi-bit digital-to-analog converter. In this case, the first MC subunit 132 includes circuitry to convert the digital input values from the controller 110 into appropriate voltage signals that are applied to the various segments of the modulator (this can be considered equivalent to a 1-bit DAC). An advantage of using a segmented design for the optical modulator is that by eliminating the digital-to-analog converter in the first MC subunit 132, the power consumption can be reduced for a given data rate or the data rate can be increased for a given power consumption. An advantage of using a modulator based on a forward biased PIN diode structure to modulate the refractive index of the waveguide segment using carrier injection is that the modulator is compatible with Complementary Metal Oxide Semiconductor (CMOS) technology.

Fig. 50 shows an example of a modulator array 5000 with optical modulators of the segmented design of fig. 47. The modulator array 5000 performs a function similar to that of the modulator array 144 of fig. 1. In this example, the modulator array 5000 is configured to process a length-4 digital input vector that includes four elements, and each element is a digital value having four bits. Modulator array 5000 includes four optical modulators 5002, 5004, 5006 and 5008 having a segmented design, where each optical modulator includes four sub-modulators or four phase shifters.

In this example, the laser unit 142 outputs four optical signals on four optical waveguides 4704a, 4704b, 4704c, and 4704 d. The optical signal on waveguide 4704a is modulated by optical modulator 5002 according to the first 4-bit value of the digital input vector. The optical signal on waveguide 4704b is modulated by optical modulator 5004 according to the second 4-bit value of the digital input vector. The optical signal on waveguide 4704c is modulated by optical modulator 5006 according to the third 4-bit value of the digital input vector. The optical signal on waveguide 4704d is modulated by optical modulator 5008 according to the fourth 4-bit value of the digital input vector.

The optical modulator 5002 includes two 1 × 2 port multimode interference couplers (MMI _1x2)4708a and 4710a, two balanced arms 5012a and 5012b, and eight sub-modulators: 5022a, 5022b, 5022c, 5022d in the upper arm and 5022e, 5022f, 5022g, 5022h in the lower arm. For example, each sub-modulator may include a phase shifter. The sub-modulators 5022d and 5022h receive the LSB (B11) of the first value in the input vector. The sub-modulators 5022c and 5022g receive the second bit (B12) of the first value in the input vector. The sub-modulators 5022B and 5022f receive the third bit (B13) of the first value in the input vector. The sub-modulators 5022a and 5022e receive the MSB (B14) of the first value in the input vector. The optical modulator 5002 also includes calibration phase shifters 5032a and 5032b driven by the multi-bit DAC.

The sub-modulator 5022d comprises a first waveguide segment, the sub-modulator 5022c comprises a second waveguide segment, the sub-modulator 5022b comprises a third waveguide segment, and the sub-modulator 5022a comprises a fourth waveguide segment. The second waveguide segment is approximately twice as long as the first waveguide segment, the third waveguide segment is approximately four times as long as the first waveguide segment, and the fourth waveguide segment is approximately eight times as long as the first waveguide segment.

Similarly, sub-modulator 5022h comprises a fifth waveguide segment, sub-modulator 5022g comprises a sixth waveguide segment, sub-modulator 5022f comprises a seventh waveguide segment, and sub-modulator 5022e comprises an eighth waveguide segment. The sixth waveguide segment is approximately twice as long as the fifth waveguide segment, the seventh waveguide segment is approximately four times as long as the fifth waveguide segment, and the eighth waveguide segment is approximately eight times as long as the fifth waveguide segment.

The optical modulator 5004 includes two 1 × 2 port multimode interference couplers (MMI _1x2)4708b and 4710b, two balanced arms 5014a and 5014b, and eight sub-modulators: 5024a, 5024b, 5024c, 5024d in the upper arm and 5024e, 5024f, 5024g, 5024h in the lower arm. For example, each sub-modulator may include a phase shifter. The sub-modulators 5024d and 5024h receive the LSB of the second value in the input vector (B21). The sub-modulators 5024c and 5024g receive the second bit (B22) of the second value in the input vector. The sub-modulators 5024B and 5024f receive the third bit (B23) of the second value in the input vector. The sub-modulators 5024a and 5024e receive the MSB (B24) of the second value in the input vector. The optical modulator 5004 also includes calibration phase shifters 5034a and 5034b driven by the multi-bit DAC.

Optical modulators 5006 and 5008 are configured in a similar manner as optical modulators 5002 and 5004.

The optical modulator 5002 modulates the optical signal in the input waveguide 4708a according to the four bits of the first value in the digital input vector and generates an analog optical signal 5042 representative of the first value of the digital input vector. Optical modulator 5002 converts the first 4-bit value of the digital input vector to a first analog optical signal. The optical modulator 5004 modulates the optical signal in the input waveguide 4708b according to the four bits of the second value in the digital input vector and generates an analog optical signal 5044 representing the second value of the digital input vector. Optical modulator 5004 converts the second 4-bit value of the digital input vector to a second analog optical signal.

The optical modulator 5006 modulates the optical signal in the input waveguide 4708c according to the four bits of the third value in the digital input vector and generates an analog optical signal 5046 representative of the third value of the digital input vector. Optical modulator 5006 converts the third 4-bit value of the digital input vector to a third analog optical signal. The optical modulator 5008 modulates the optical signal in the input waveguide 4708d according to the four bits of the fourth value in the digital input vector and generates an analog optical signal 5048 representative of the fourth value of the digital input vector. Optical modulator 5008 converts the fourth 4-bit value of the digital input vector to a fourth analog optical signal. The analog optical signals 5042, 5044, 5046, 5048 together form an analog optical input vector, which may be provided to, for example, the opto-electronic matrix multiplication unit 150 of fig. 1.

The digital controllers (e.g., for controlling the components shown in fig. 38) and functional operations described in this disclosure may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures in this disclosure and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this disclosure can be implemented using one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium may be an article of manufacture (e.g., a hard drive in a computer system or an optical disk sold through retail outlets) or an embedded system. The computer readable medium may be separately retrieved and then encoded with one or more modules of computer program instructions, for example, over a wired or wireless network. The computer readable medium can be a machine readable storage device, a machine readable storage substrate, a memory device, or a combination of one or more of them.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this disclosure can be performed by one or more programmable processors (programmable processors) executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC).

Some background information on the various systems described in this specification is disclosed in us application 16/431,167 filed on 6/4/2019, the entire disclosure of which is incorporated herein by reference.

While the disclosure has been described in connection with certain embodiments, it is to be understood that the disclosure is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures as is permitted under the law.