阅读说明：本技术 学习设备、学习方法、存储有控制程序的非暂时性计算机可读介质、以及估计设备 (Learning apparatus, learning method, non-transitory computer-readable medium storing control program, and estimation apparatus ) 是由 户泉贵裕 赤司竜一 荻野有加 于 2019-02-05 设计创作，主要内容包括：在学习设备(10)中,光学转换单元(11)接收来自学习对象的光,并且使用所接收的光以根据参数的配置值来输出光。感测单元(13)感测从光学转换单元(11)所输出的光。估计单元(15A)基于由感测单元(13)感测到的光来形成对配置问题的答案的估计结果。更新单元(15B)基于估计单元(15A)的估计结果来计算光学转换单元(11)的参数的更新值,并用计算出的更新值来更新光学转换单元(11)的参数的配置值。光学转换单元(11)包括相互独立地设定参数的配置值的多个光学器件。(In a learning apparatus (10), an optical conversion unit (11) receives light from a learning object and uses the received light to output light according to a configuration value of a parameter. The sensing unit (13) senses light output from the optical conversion unit (11). An estimation unit (15A) forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit (13). An update unit (15B) calculates an update value of the parameter of the optical conversion unit (11) based on the estimation result of the estimation unit (15A), and updates the configuration value of the parameter of the optical conversion unit (11) with the calculated update value. The optical conversion unit (11) includes a plurality of optical devices that set configuration values of parameters independently of each other.)

1. A learning device comprising:

an optical conversion device for receiving light from the learning object and outputting light according to the configuration value of the parameter using the received light;

a sensing device for sensing light output from the optical conversion device;

estimating means for forming an estimate of an answer to the configuration question based on the sensed light; and

updating means for calculating an updated value of the parameter based on the estimation result of the estimating means and updating the configuration value of the parameter with the calculated updated value,

wherein the optical conversion apparatus includes a plurality of optical devices in which the configuration values of the parameters are set independently of each other.

2. The learning device according to claim 1, further comprising:

learning control means for sequentially switching update subject devices among the plurality of optical devices, switching the learning subject according to switching of the update subject devices, and causing the updating means to update the configuration value of the parameter for each update subject device.

3. The learning device according to claim 2, wherein,

the learning control means controls batch learning for each update subject device, and randomly selects the update subject device from the plurality of optical devices.

4. The learning apparatus according to claim 2 or 3, wherein,

the updating means calculates a gradient by an error back propagation method using an objective function related to an error between the estimation result of the estimating means and a correct answer, calculates an updated value of the parameter based on the calculated gradient, and updates the configuration value of the parameter with the calculated updated value.

5. The learning device according to claim 1, wherein,

the updating means forms a perturbation using a random number, calculates a gradient using an objective function related to an error between the estimation result of the estimating means and a correct answer and the formed perturbation, calculates an updated value of the parameter based on the calculated gradient, and updates the configuration value of the parameter with the calculated updated value.

6. The learning apparatus according to any one of claims 1 to 5,

the plurality of optical devices include liquid crystal devices, and

the updating means updates the configuration values of the parameters on a pixel-by-pixel basis of the liquid crystal device.

7. The learning device according to claim 6, wherein,

the liquid crystal device outputs light in which optical characteristics according to the configuration values of the parameters are emphasized on a pixel-by-pixel basis.

8. The learning apparatus according to any one of claims 1 to 7,

the optical conversion device switches the configuration value of the parameter from a first configuration value to a second configuration value within an exposure time of the sensing device.

9. The learning apparatus according to any one of claims 1 to 8,

the estimation means comprises a neural network, and

the updating means further updates a parameter of the neural network.

10. The learning apparatus according to any one of claims 1 to 9,

using a parameter value obtained by optical simulation as an initial value of the configuration value of the parameter in the learning process.

11. The learning apparatus according to any one of claims 1 to 10,

the optical conversion apparatus performs, as processing for the received light, at least one of: attenuation processing, amplification processing, light condensation processing, diffusion processing, light wave strengthening and synthesizing processing, moire fringe generation processing, three-dimensional processing and polarization extraction processing.

12. The learning apparatus according to any one of claims 1 to 11,

the configuration problem is image recognition, object detection, segmentation, anomaly detection, image generation, image transformation, image compression, light field generation, or three-dimensional image generation.

13. An estimation device, comprising:

an optical conversion device for outputting light according to a configuration value of the parameter using the received light;

a sensing device for sensing light output from the optical conversion device; and

estimation means for forming an estimation result of an answer to the configuration question based on the sensed light,

wherein the optical conversion apparatus includes a plurality of optical devices in which the configuration values of the parameters are set independently of each other.

14. A learning method, comprising:

forming an estimation result of an answer to a configuration question based on light output from an optical conversion apparatus that receives light from a learning object and includes a plurality of optical devices and according to a configuration value of a parameter set in the optical conversion apparatus;

calculating an updated value of the parameter based on the estimation result; and

updating the configuration value of the parameter with the calculated update value.

15. A non-transitory computer-readable medium storing a control program configured to cause a learning device to execute:

forming an estimation result of an answer to a configuration question based on light output from an optical conversion apparatus that receives light from a learning object and includes a plurality of optical devices and according to a configuration value of a parameter set in the optical conversion apparatus;

calculating an updated value of the parameter based on the estimation result; and

updating the configuration value of the parameter with the calculated update value.

Technical Field

The present disclosure relates to a learning apparatus, a learning method, a non-transitory computer-readable medium having a control program stored thereon, and an estimation apparatus.

Background

An estimation apparatus that estimates answers to various configuration questions has been proposed (for example, patent document 1). The configuration problem of the device disclosed in patent document 1 is to acquire a light field. Specifically, the light field acquisition apparatus disclosed in patent document 1 controls the setting of the code aperture shape of the code aperture portion (i.e., the optical conversion unit) based on an evaluation value obtained by comparing the restored light field data restored from the image signal with the reference light field data. This allows the light field acquisition apparatus disclosed in patent document 1 to be able to acquire a light field of a scene whose data amount is smaller than the number of pixels of image elements with high resolution.

List of citations

Patent document

[ patent document 1] Japanese unexamined patent application publication No. 2016-157999-

Disclosure of Invention

Technical problem

The present inventors have found that by including a plurality of optical devices that set configuration values of parameters independently of each other in an optical conversion unit, an estimation apparatus capable of estimating answers to configuration questions more accurately can be realized. The present inventors have proposed a learning apparatus that learns configuration values of parameters set in a plurality of optical devices.

An object of the present disclosure is to provide a learning apparatus and a learning method capable of learning configuration values of parameters set in a plurality of optical devices in an estimation apparatus capable of more accurately estimating an answer to a configuration question, and to provide a non-transitory computer-readable medium and an estimation apparatus on which a control program is stored.

Problem solving scheme

The learning apparatus according to the first aspect includes:

an optical conversion device for receiving light from the learning object and outputting light according to the configuration value of the parameter using the received light;

a sensing device for sensing light output from the optical conversion device;

estimating means for forming an estimate of an answer to the configuration question based on the sensed light; and

updating means for calculating an updated value of the parameter based on the estimation result of the estimating device and updating the configuration value of the parameter with the calculated updated value,

wherein the optical conversion apparatus includes a plurality of optical devices in which configuration values of the parameters are set independently of each other.

The estimation device according to the second aspect includes:

an optical conversion device for outputting light according to a configuration value of the parameter using the inputted light;

a sensing device for sensing light output from the optical conversion device; and

estimation means for forming an estimation result of an answer to the configuration question based on the sensed light,

wherein the optical conversion apparatus includes a plurality of optical devices in which configuration values of the parameters are set independently of each other.

The learning method according to the third aspect includes:

forming an estimation result of an answer to the configuration question based on light output from an optical conversion apparatus that receives light from the learning object and includes a plurality of optical devices, and according to the configuration value of the parameter set in the optical conversion apparatus;

calculating an updated value of the parameter based on the estimation result; and

the configuration values of the parameters are updated using the calculated update values.

The non-transitory computer-readable medium according to the fourth aspect stores a control program configured to cause the learning device to execute:

forming an estimation result of an answer to the configuration question based on light output from an optical conversion apparatus that receives light from the learning object and includes a plurality of optical devices, and according to the configuration value of the parameter set in the optical conversion apparatus;

calculating an updated value of the parameter based on the estimation result; and

the configuration values of the parameters are updated using the calculated update values.

Advantageous effects of the invention

According to the present disclosure, it is possible to provide a learning apparatus and a learning method capable of learning configuration values of parameters set in a plurality of optical devices in an estimation apparatus capable of estimating answers to configuration questions more accurately, and to provide the above-described non-transitory computer-readable medium having the estimation apparatus and having the control program stored thereon.

Drawings

Fig. 1 is a block diagram showing an example of a learning apparatus in the first exemplary embodiment.

Fig. 2 is a block diagram showing an example of the learning apparatus in the second exemplary embodiment.

Fig. 3 is a flowchart showing an example of processing operation of the learning apparatus in the second exemplary embodiment.

Fig. 4 is a block diagram showing an example of the learning apparatus in the third exemplary embodiment.

Fig. 5 is a block diagram showing an example of an estimation device in the fourth exemplary embodiment.

Fig. 6 is a diagram showing an example of the hardware configuration of the control device.

Detailed Description

Example embodiments will be described below with reference to the accompanying drawings. In the exemplary embodiments, the same or equivalent elements are assigned the same reference numerals, and the detailed description is omitted.

< first exemplary embodiment >

Fig. 1 is a block diagram showing an example of a learning apparatus in the first exemplary embodiment. In fig. 1, the learning device 10 includes an optical conversion unit 11, a sensing unit 13, and a control unit (control device) 15.

The optical conversion unit 11 receives (inputs) light from a learning object (for example, a learning image), and outputs light using the received (input) light according to the configuration value of the parameter. For example, the optical conversion unit 11 includes optical devices 12-1 to 12-N (N is a natural number of 2 or more) that set configuration values of parameters independently of each other. Hereinafter, when the optical devices 12-1 to 12-N are not distinguished from each other, the optical devices 12-1 to 12-N may be collectively referred to as the optical devices 12. The optical devices 12-1 to 12-N may be the same type of optical device as each other, or may include multiple types of optical devices. For example, the optical conversion unit 11 performs at least one of the following as processing on the received (input) light: attenuation processing, amplification processing, light collection processing, diffusion processing, light wave strengthening and synthesizing processing, moire fringe generation processing, three-dimensional processing, and polarization extraction processing.

The sensing unit 13 senses light output from the optical conversion unit 11. The sensing unit 13 is, for example, an image sensor. That is, the sensing unit 13 converts the sensed light into an electric signal and outputs the obtained electric signal to the control unit 15. It should be noted that, in fig. 1, the optical conversion unit 11 and the sensing unit 13 are connected to each other by a broken line, and the broken line indicates an optical path.

The control unit 15 includes an estimation unit 15A and an update unit 15B.

The estimation unit 15A forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit 13. Configuration problems are for example image recognition, object detection, segmentation, anomaly detection, image generation, image conversion, image compression, light field generation or three-dimensional image generation. That is, the estimation unit 15A is a functional unit that performs image analysis processing according to a configuration problem. For example, when the configuration problem is that of recognizing a red object, the estimation unit 15A forms and outputs "1" as the estimation result if the learning image includes an image of a red object, and the estimation unit 15A forms and outputs "0" as the estimation result if the learning image does not include an image of a red object. Further, for example, when the configuration problem is generation of a light field, the estimation unit 15A forms and outputs the light field as the estimation result.

The updating unit 15B calculates an updated value of the parameter of the optical conversion unit 11 based on the estimation result of the estimating unit 15A, and updates the configuration value of the parameter of the optical conversion unit 11 with the calculated updated value.

As described above, according to the first exemplary embodiment, in the learning apparatus 10, the optical conversion unit 11 receives (inputs) light from the learning object and outputs light using the received (input) light according to the configuration value of the parameter. The sensing unit 13 senses light output from the optical conversion unit 11. The estimation unit 15A forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit 13. The updating unit 15B calculates an updated value of the parameter of the optical conversion unit 11 based on the estimation result of the estimating unit 15A, and updates the configuration value of the parameter of the optical conversion unit 11 with the calculated updated value. The optical conversion unit 11 includes optical devices 12-1 to 12-N (N is a natural number of 2 or more) that set configuration values of parameters independently of each other.

By learning the configuration of the apparatus 10, it is possible to learn the configuration values of parameters set in a plurality of optical devices in an estimation apparatus (not shown) including a plurality of optical devices identical to the optical devices 12-1 to 12-N (N is a natural number of 2 or more), a sensing unit, and an estimation unit. Since the estimating apparatus (not shown) can use light containing more information for the estimation process by the plurality of optical devices, the estimating apparatus (not shown) can estimate the answer to the configuration question more accurately than the case where the number of optical devices is one. That is, according to the learning apparatus 10, it is possible to learn the configuration values of the parameters set in the plurality of optical devices in the estimation apparatus capable of estimating the answer to the configuration question more accurately. It should be noted that in the estimation device (not shown), the hardware neural network can be implemented by a plurality of optical devices. Therefore, the calculation amount of the estimation unit of the estimation device (not shown) can be reduced, so that the estimation device (not shown) can also be applied to an edge terminal having poor processing capability.

< second exemplary embodiment >

The second exemplary embodiment relates to a more specific exemplary embodiment.

< example of arrangement of learning apparatus >

Fig. 2 is a block diagram showing an example of the learning apparatus in the second exemplary embodiment. In fig. 2, the learning device 20 includes an optical conversion unit 11, a sensing unit 13, a display unit 21, and a control unit (control device) 22. The control unit 22 includes an estimation unit 22A, an update unit 22B, and a learning control unit 22C.

The display unit 21 displays the learning image under the control of the learning control unit 22C. This causes light corresponding to the learning image to be input to the optical conversion unit 11. Note that, in fig. 2, the display unit 21 and the optical conversion unit 11 are connected to each other by a broken line, and the broken line indicates an optical path.

The optical conversion unit 11 includes optical devices 12-1 to 12-N (N is a natural number of 2 or more) as described in the first exemplary embodiment. Here, the optical devices 12-1 to 12-N include liquid crystal devices as an example, and will be described on the assumption that the optical device 12-1 is a liquid crystal device.

The optical device 12-1 includes, for example, a first polarizing plate, a second polarizing plate whose polarization direction is rotated by 90 ° from that of the first polarizing plate, a color filter, and a liquid crystal cell interposed between the first polarizing plate and the second polarizing plate. By controlling the electrical signal (i.e. the configuration value of the parameter) applied to the liquid crystal cell, for example, the light transmittance of the liquid crystal cell can be controlled. The control can be performed on a pixel-by-pixel basis of the liquid crystal cell. That is, the liquid crystal device is capable of outputting light in which optical characteristics according to the configuration values of the parameters are emphasized on a pixel-by-pixel basis.

Alternatively, the optical device 12-1 has a liquid crystal cell. By controlling the electrical signal (i.e. the configuration value of the parameter) applied to the liquid crystal cell, the helical structure of the liquid crystal in the liquid crystal cell can be controlled, and the wavelength of the selectively reflected light can be controlled. The control can be performed on a pixel-by-pixel basis of the liquid crystal cell. That is, the liquid crystal device is capable of outputting light in which optical characteristics according to the configuration values of the parameters are emphasized on a pixel-by-pixel basis.

In the optical device 12, for example, a liquid lens, a deformable mirror, a microchannel plate type photomultiplier tube, or the like may be used in addition to or instead of the liquid crystal device. A liquid lens is a device capable of adjusting a focal length by an electrical signal. A deformable mirror is a device capable of controlling the direction of reflection by an electrical signal. The micro channel plate type photomultiplier is a device that converts incident light into electrons inside the device and amplifies the electrons inside to output light stronger than the incident light.

The learning control unit 22C controls learning of the configuration values of the parameters in the optical conversion unit 11. For example, the learning control unit 22C switches the learning image displayed on the display unit 21. In the learning process of the configuration values of the parameters in the optical conversion unit 11, the learning control unit 22C may use parameter values obtained by optical simulation as initial values of the parameters. This allows fine-tuning of the configuration values of the parameters obtained by the optical simulation to be performed. In this case, for example, the learning control unit 22C may perform simulation using an optical model obtained by modeling each of the optical devices 12-1 to 12-N, calculate a gradient by an error back propagation method, and optimize parameters of the optical model. This parameter value is used as the initial value described above. Note that, when the estimation unit 22A includes a neural network, the learning control unit 22C may control learning of the neural network.

Just like the estimation unit 15A of the first exemplary embodiment, the estimation unit 22A forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit 13. The estimation unit 22A includes, for example, a neural network.

The updating unit 22B calculates a gradient using an objective function related to an error between the estimation result of the estimating unit 22A and the correct answer. Then, the updating unit 22B calculates an updated value of the parameter based on the calculated gradient, and updates the configuration value of the parameter of the optical conversion unit 11 with the calculated updated value. The updating unit 22B may update the parameters (weights or deviations) of the neural network included in the estimating unit 22A. For the gradient calculation in the update process of the parameters (weights or deviations) of the neural network included in the estimation unit 22A, an error back propagation method used in the learning of the neural network can be used.

Since the optical input/output phenomenon in the optical conversion unit 11 occurs outside the control unit 22 (i.e., the computer), the error back propagation method used in the learning of the neural network cannot be used. Therefore, the updating unit 22B forms a perturbation vector using a random number, and calculates a gradient using an objective function related to an error between the estimation result of the estimating unit 22A and the correct answer and the formed perturbation vector. Then, the updating unit 22B calculates an updated value of the parameter based on the calculated gradient, and updates the configuration value of the parameter of the optical conversion unit 11 with the calculated updated value.

In the gradient calculation, the parameter vector is denoted by "p" and its random number perturbation vector is denoted by "e". When the entire neural network of the optical conversion unit 11 is f (x, p), the updating unit 22B calculates f (x, p) and f (x, p + e) for the input image batch x. In general gradient calculation, since the gradient is obtained by independently performing calculation on the scalar value pi of each element of the parameter vector p, the same number of operations as the number of elements of the parameter is required. On the other hand, in the calculation of random number disturbance, since the disturbance e is calculated by a vector, the number of operations is only 2. In contrast, the updating unit 22B generates the perturbation vector e using a random number. For example, the updating unit 22B calculates a random number using the bernoulli distribution. As a result, each element of the perturbation vector e takes on a value of-1 or 1. The updating unit 22B can control the differential width of the gradient calculation by multiplying "e" by the hyperparameter a. For the random number, the same random number may be used in each iteration, or a different random number may be used in each iteration. Random number perturbation can be applied not only to vectors but also to matrices and tensors of three or more layers. For the update of the gradient, for example, a random gradient descent method, Adam, or the like can be used. For the calculation of random numbers, in addition to the bernoulli distribution, a uniform distribution with 0 as the average or a gaussian distribution may be used.

< example of operation of learning apparatus >

An example of the processing operation of the learning apparatus 20 having the above-described configuration will now be described. Fig. 3 is a flowchart showing an example of processing operation of the learning apparatus in the second exemplary embodiment.

The learning control unit 22C sets initial values of parameters for the optical conversion unit 11 (step S101). As described above, the parameter value obtained by the optical simulation may be used as the initial value of the parameter.

The learning control unit 22C causes the display unit 21 to display the learning image (step S102). This causes light corresponding to the learning image to be input to the optical conversion unit 11.

The optical conversion unit 11 receives (inputs) light from the learning image and outputs light using the received (input) light according to the configuration value of the parameter (step S103).

The sensing unit 13 senses the light output from the optical conversion unit 11 (step S104).

The estimation unit 15A forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit 13 (step S105).

The updating unit 22B calculates a gradient using an objective function relating to an error between the estimation result of the estimating unit 22A and the correct answer (step S106).

The learning control unit 22C determines whether or not the end condition is satisfied (step S107). For example, if the sign of the gradient calculated this time by the updating unit 22B is different from the sign of the gradient calculated last time, it is considered that the minimum value (or the local minimum value) of the objective function is reached, and the learning control unit 22C determines that the end condition is satisfied. When the end condition is satisfied (yes in step S107), the processing flow ends.

When the end condition is not satisfied (no at step S107), the learning control unit 22C causes the updating unit 22B to calculate an updated value of the parameter based on the calculated gradient, and updates the arrangement value of the parameter of the optical conversion unit 11 with the calculated updated value (step S108).

The learning control unit 22C switches the learning image and causes the display unit 21 to display the switched image (step S109).

It should be noted that, in the above description, description is given on the premise of learning for each individual image, but is not limited thereto, and batch learning or small-batch learning may be performed.

< third exemplary embodiment >

In the second exemplary embodiment described above, description has been made on the premise that all the optical devices 12-1 to 12-N are made parameter update subject devices at a certain timing. On the other hand, the third exemplary embodiment relates to an exemplary embodiment in which the update subject device is sequentially switched among the optical devices 12-1 to 12-N.

Fig. 4 is a block diagram showing an example of the learning apparatus in the third exemplary embodiment. In fig. 4, the learning device 30 includes a control unit (control device) 31. The control unit 31 includes an updating unit 31A and a learning control unit 31B.

The learning control unit 31B sequentially switches the update subject device among the optical devices 12-1 to 12-N, switches the learning image displayed on the display unit 21 according to the switching of the update subject device, and causes the update unit 31A to update the configuration value of the parameter for each update subject device.

Here, it is assumed that all of the optical devices 12-1 to 12-N are liquid crystal devices. Then, the learning control unit 31B selects an update subject device from among the optical devices 12-1 to 12-N. The learning control unit 31B sets the configuration values of the parameters of the optical devices 12-1 to 12-N other than the update subject device to the maximum light transmittance. Then, the learning control unit 31B causes the updating unit 31A to calculate the gradient of the update subject device by an error back propagation method using an objective function relating to the error between the estimation result of the estimating unit 22A and the correct answer. Here, the arrangement value of the parameter of the optical device 12 other than the update target device is set to the maximum transmittance, and therefore it can be considered that the light output from the optical conversion unit 11 is not affected by the optical device 12 other than the update target device. Thus, since the input and output of the update subject device can be quantized, the gradient can be calculated using the error back propagation method. The learning control unit 31B optimizes the parameters of one update subject device, then switches the update subject device, and advances learning of the parameters of the update subject device after switching. That is, the parameters of the optics 12-1 through 12-N are learned here in a manner that optimizes the stacked auto-encoder.

Note that the learning control unit 31B may advance batch learning for one update subject device, randomly select an optical device 12 that has not been learned as a next update subject device when the batch learning is completed, and advance learning for the selected update subject device.

< fourth exemplary embodiment >

A fourth exemplary embodiment relates to an estimating apparatus in which the parameter values learned by the learning apparatuses described in the first to third exemplary embodiments are set.

Fig. 5 is a block diagram showing an example of an estimation device in the fourth exemplary embodiment. In fig. 5, the estimation device 50 includes an optical conversion unit 51, a sensing unit 53, a control unit (control device) 55, and an acquisition unit 57.

The optical conversion unit 51 outputs light using the received (input) light according to the configuration value of the parameter. The optical conversion unit 51 includes optical devices 52-1 to 52-N (N is a natural number of 2 or more) that set configuration values of parameters independently of each other. The optical devices 52-1 to 52-N correspond to the optical devices 12-1 to 12-N of the first to third exemplary embodiments, respectively, and the parameter value learned by the learning apparatus 10, 20, or 30 is set. That is, the optical conversion unit 51 has the configuration as in the optical conversion unit 11 of the first to third exemplary embodiments.

The sensing unit 53 senses the light output from the optical conversion unit 51. That is, the sensing unit 53 has the configuration as in the sensing unit 13 of the first to third exemplary embodiments.

The control unit 55 includes an estimation unit 55A and a setting unit 55B.

The estimation unit 55A forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit 53. That is, the estimation unit 55A has the same configuration as the estimation units 15A and 22A of the first to third exemplary embodiments.

The setting unit 55B sets the parameter values acquired by the acquisition unit 57 for the optical devices 12-1 to 12-N of the optical conversion unit 51.

The acquisition unit 57 acquires the parameter value learned by the learning apparatus 10, 20, or 30. The acquisition unit 57 may be a communication unit that acquires the parameter value from the learning device 10, 20, or 30 by wired communication or wireless communication. Alternatively, the acquisition unit 57 may be an interface unit to which a cable connecting the estimation device 50 and the learning device 10, 20, or 30 is connected. The parameter values learned by the learning apparatus 10, 20, or 30 can be acquired by the acquisition unit 57, for example, the parameter values to be set for the optical devices 52-1 to 52-N can be downloaded. The optical device in this embodiment is external to the computer, so the parameter values are set at the time of shipment of the device, and direct access to the parameters at the edge terminal can be prohibited. Thus, it is not possible to intrude on the parameters of the edge termination.

As described above, according to the fourth exemplary embodiment, in the estimation apparatus 50, the optical conversion unit 51 outputs light using the received (input) light according to the configuration value of the parameter. The sensing unit 53 senses the light output from the optical conversion unit 51. The estimation unit 55A forms an estimation result of an answer to the configuration question based on the light sensed by the sensing unit 53. The optical conversion unit 51 includes optical devices 52-1 to 52-N (N is a natural number of 2 or more) that set configuration values of parameters independently of each other.

In the configuration of the estimation apparatus 50, the optical conversion unit 51 includes the optical devices 52-1 to 52-N that set the configuration values of the parameters independently of each other, and therefore light containing more information can be used for the estimation processing of a plurality of optical devices than the case where there is one optical device. As a result, the estimation device 50 can estimate the answer to the configuration question more accurately. Since the hardware neural network can be realized by the optical devices 52-1 to 52-N, the calculation amount of the estimation unit 55A can be reduced. This allows the estimation device 50 to be applied to an edge terminal having poor processing capability.

< other example embodiments >

<1> the optical conversion units 11 and 51 of the first to fourth exemplary embodiments may switch the configuration values of the parameters from the first configuration values to the second configuration values within the exposure time of the sensing units 13 and 53. For example, within the exposure time of the sensing unit 13, the optical conversion unit 11 switches between a state in which the parameter value updated by the updating unit 15B, 22B, or 31A is set for all the optical devices 12-1 to 12-N and a state in which the parameter value of some of the optical devices 12-1 to 12-N is switched to the maximum light transmittance. This allows the sensing unit 13 to express an image in which the images in the two states are added. That is, the added layer in the neural network can be expressed by this operation. Alternatively, instead of addition, the images in the above two states may be arranged side by side in the sensing unit 13. This allows the expression of a connection layer in a neural network. According to this method, a hopping connection can be used for U-Net or the like. The same applies to the optical conversion unit 51 and the sensing unit 53.

<2> fig. 6 is a diagram showing an example of a hardware configuration of the control device. In fig. 6, the control apparatus 100 includes a processor 101 and a memory 102. The processor 101 may be, for example, a microprocessor, an MPU (micro processing unit), or a CPU (central processing unit). Processor 101 may include multiple processors. The memory 102 is composed of a combination of volatile memory and non-volatile memory. Memory 102 may include storage that is remotely located from processor 101. In this case, the processor 101 may access the memory 102 via an I/O interface (not shown).

The control devices (control units) 15, 22, 31, and 55 of the first to fourth exemplary embodiments can each have a hardware configuration shown in fig. 6. The estimation units 15A, 22A, and 55A, the update units 15B, 22B, and 31A, the learning control units 22C and 31B, and the setting unit 55B of the control devices 15, 22, 31, and 55 of the first to fourth exemplary embodiments may be implemented by the processor 101 reading and executing programs stored in the memory 102. The program can be stored and provided to the control devices 15, 22, 31, and 55 using various types of non-transitory computer-readable media. Examples of non-transitory computer readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, and hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). In addition, examples of non-transitory computer readable media include CD-ROM (read only memory), CD-R, and CD-R/W. Additionally, an example of a non-transitory computer readable medium includes a semiconductor memory. The semiconductor memory includes, for example, mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash ROM, and RAM (random access memory). The program may be provided to the control devices 15, 22, 31, and 55 through various types of temporary computer-readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable medium can provide the program to the control devices 15, 22, 31, and 55 via a wired communication path such as an electric wire and an optical fiber or a wireless communication path.

Although the invention of the present application has been described above with reference to the exemplary embodiments, the invention of the present application is not limited to the above. Various modifications as will be understood by those skilled in the art may be made in the arrangement and details of the invention within the scope thereof.

Some or all of the above example embodiments may be described by, but are not limited to, the following notes.

(Note 1)

A learning device comprising:

an optical conversion device for receiving light from the learning object and outputting light according to the configuration value of the parameter using the received light;

a sensing device for sensing light output from the optical conversion device;

estimating means for forming an estimate of an answer to the configuration question based on the sensed light; and

updating means for calculating an updated value of the parameter based on the estimation result of the estimating device and updating the configuration value of the parameter with the calculated updated value,

wherein the optical conversion apparatus includes a plurality of optical devices in which the configuration values of the parameters are set independently of each other.

(Note 2)

The learning apparatus according to supplementary note 1, further comprising learning control means for sequentially switching update subject devices among the plurality of optical devices, switching the learning subject according to switching of the update subject devices, and causing the updating means to update the configuration value of the parameter for each update subject device.

(Note 3)

The learning apparatus according to supplementary note 2, wherein the learning control means controls batch learning of each update subject device, and randomly selects the update subject device from the plurality of optical devices.

(Note 4)

The learning apparatus according to supplementary note 2 or 3, wherein the updating means calculates a gradient by an error back propagation method using an objective function relating to an error between the estimation result of the estimating means and a correct answer, calculates an updated value of the parameter based on the calculated gradient, and updates the configuration value of the parameter with the calculated updated value.

(Note 5)

The learning apparatus according to supplementary note 1, wherein the updating means forms a perturbation using a random number, calculates a gradient using an objective function relating to an error between the estimation result of the estimating means and a correct answer and the formed perturbation, calculates an updated value of the parameter based on the calculated gradient, and updates the configuration value of the parameter with the calculated updated value.

(Note 6)

The learning apparatus according to any one of supplementary notes 1 to 5, wherein,

the plurality of optical devices include liquid crystal devices, and

the updating means updates the configuration values of the parameters on a pixel-by-pixel basis of the liquid crystal device.

(Note 7)

The learning apparatus according to supplementary note 6, wherein the liquid crystal device outputs light in which optical characteristics according to the configuration values of the parameters are emphasized on a pixel-by-pixel basis.

(Note 8)

The learning apparatus according to any one of supplementary notes 1 to 7, wherein the optical conversion means switches the configuration value of the parameter from a first configuration value to a second configuration value within an exposure time of the sensing means.

(Note 9)

The learning apparatus according to any one of supplementary notes 1 to 8, wherein,

the estimation means comprises a neural network, and

the updating means further updates a parameter of the neural network.

(Note 10)

The learning apparatus according to any one of supplementary notes 1 to 9, wherein a parameter value obtained by optical simulation is used as an initial value of the configuration value of the parameter in the learning process.

(Note 11)

The learning apparatus according to any one of supplementary notes 1 to 10, wherein the optical conversion device executes at least one of the following as processing on the received light: attenuation processing, amplification processing, light collection processing, diffusion processing, light wave strengthening and synthesizing processing, moire fringe generation processing, three-dimensional processing, and polarization extraction processing.

(Note 12)

The learning apparatus according to any one of supplementary notes 1 to 11, wherein the configuration problem is image recognition, object detection, segmentation, abnormality detection, image generation, image conversion, image compression, light field generation, or three-dimensional image generation.

(Note 13)

An estimation device, comprising:

an optical conversion device for outputting light according to a configuration value of the parameter using the received light;

a sensing device for sensing light output from the optical conversion device; and

estimation means for forming an estimation result of an answer to the configuration question based on the sensed light,

wherein the optical conversion apparatus includes a plurality of optical devices in which the configuration values of the parameters are set independently of each other.

(Note 14)

A learning method, comprising:

forming an estimation result of an answer to a configuration question based on light output from an optical conversion apparatus that receives light from a learning object and includes a plurality of optical devices, and according to a configuration value of a parameter set in the optical conversion apparatus;

calculating an updated value of the parameter based on the estimation result; and

updating the configuration value of the parameter using the calculated update value.

(Note 15)

A non-transitory computer-readable medium storing a control program configured to cause a learning device to execute:

forming an estimation result of an answer to a configuration question based on light output from an optical conversion apparatus that receives light from a learning object and includes a plurality of optical devices, and according to a configuration value of a parameter set in the optical conversion apparatus;

calculating an updated value of the parameter based on the estimation result; and

updating the configuration value of the parameter using the calculated update value.

List of reference numerals

10 learning device

11 optical conversion unit

12 optical device

13 sensing unit

15 control unit (control device)

15A estimation unit

15B update unit

20 learning device

21 display unit

22 control unit (control device)

22A estimation unit

22B update unit

22C learning control unit

30 learning device

31 control unit (control device)

31A update unit

31B learning control unit

50 estimating apparatus

51 optical conversion unit

53 sensing unit

55 control unit (control device)

55A estimation unit

55B setting unit

57 an acquisition unit.