阅读说明：本技术 电磁波相位振幅生成装置、方法和非临时记录介质 (Electromagnetic wave phase amplitude generating apparatus, method and non-transitory recording medium ) 是由 堀崎辽一 谷田纯 江上力贵 于 2017-08-10 设计创作，主要内容包括：一种电磁波相位振幅生成装置,具备：照射部,其包括照明件和散射板,该照明件照射向拍摄对象照射的电磁波,该散射板具有被分割的区域,按照该区域为单位对作为电磁波的状态的电磁波的强度、振幅以及相位中的至少一个进行调制,散射板接受来自照明件的电磁波的照射,按照区域为单位对所照射的电磁波的状态进行调整；拍摄部,其包括检测散射电磁波的拍摄元件,散射电磁波是照射部所照射的具有照射图案的电磁波被拍摄对象散射后的电磁波,拍摄部通过拍摄散射电磁波来生成拍摄图像；生成部,其基于拍摄部生成的拍摄图像和表示照射图案的信息,进行基于拍摄图像的稀疏性的稀疏约束运算,由此生成表示散射电磁波的至少相位和振幅的信息的复振幅信息。(An electromagnetic wave phase amplitude generation device is provided with: an irradiation unit including an illuminator that irradiates an electromagnetic wave to be irradiated on an imaging target, and a diffusion plate that has divided regions and modulates at least one of an intensity, an amplitude, and a phase of the electromagnetic wave as a state of the electromagnetic wave in units of the regions, the diffusion plate receiving the irradiation of the electromagnetic wave from the illuminator and adjusting the state of the irradiated electromagnetic wave in units of the regions; an imaging unit including an imaging element that detects a scattered electromagnetic wave, the scattered electromagnetic wave being an electromagnetic wave that is irradiated by the irradiation unit and that has been scattered by an object to be imaged, the imaging unit generating an image by imaging the scattered electromagnetic wave; and a generation unit that generates complex amplitude information indicating information of at least the phase and amplitude of the scattered electromagnetic wave by performing a sparsity constraint operation based on sparsity of the captured image, based on the captured image generated by the imaging unit and information indicating the irradiation pattern.)

1. An electromagnetic wave phase amplitude generating apparatus, wherein,

the electromagnetic wave phase/amplitude generating device includes:

an irradiation unit including an illuminator that irradiates an electromagnetic wave to be irradiated on an imaging target, and a diffusion plate that has a divided region and modulates at least one of an intensity, an amplitude, and a phase of the electromagnetic wave as a state of the electromagnetic wave in units of the region, the diffusion plate receiving irradiation of the electromagnetic wave from the illuminator and adjusting the state of the irradiated electromagnetic wave in units of the region, thereby irradiating the imaging target with an electromagnetic wave having a random irradiation pattern;

an imaging section including an imaging element that detects a scattered electromagnetic wave that is an electromagnetic wave irradiated by the irradiation section and having the irradiation pattern, the electromagnetic wave being scattered by the subject, the imaging section generating a captured image by capturing the scattered electromagnetic wave; and

and a generation unit configured to generate complex amplitude information indicating at least a phase and an amplitude of the scattered electromagnetic wave by performing a sparsity constraint operation based on sparsity of the captured image, based on the captured image generated by the imaging unit and information indicating the irradiation pattern.

2. The electromagnetic wave phase amplitude generation apparatus according to claim 1,

the generating unit generates the complex amplitude information by repeating newly generating the complex amplitude information using the complex amplitude information generated last time when generating the complex amplitude information based on the generated complex amplitude information and the captured image generated by the capturing unit.

3. The electromagnetic wave phase amplitude generation apparatus according to claim 1 or 2,

in the imaging section, the number of pixels of the imaging element of the imaging section representing a resolution of the imaging section is smaller than the number of the divided regions of the diffuser plate representing a resolution calculated based on the sparsity constraint of the generation section,

the generating unit further generates the complex amplitude information so that the resolution thereof is higher than the resolution of the imaging unit, based on a correspondence relationship between the number of pixels of the imaging element of the imaging unit and the number of divided regions of the diffusion plate.

4. The electromagnetic wave phase amplitude generation apparatus according to claim 1,

the random illumination pattern is a pattern that is spectrally uniformly spread over spatial frequencies.

5. The electromagnetic wave phase amplitude generation apparatus according to claim 1,

the electromagnetic wave is at least one of visible light, X-ray, ultraviolet ray, infrared ray, terahertz wave, millimeter wave, and microwave.

6. The electromagnetic wave phase amplitude generation apparatus according to claim 1,

the diffusion plate included in the irradiation unit changes the intensity of the electromagnetic wave irradiated in units of the region, and changes the intensity of the electromagnetic wave irradiated by the illuminating device to an intensity different in units of the region, thereby modulating the state of the electromagnetic wave in units of the region.

7. The electromagnetic wave phase amplitude generation apparatus according to claim 1,

the degree of scattering of the electromagnetic wave that is irradiated to the divided regions by the scattering plate included in the irradiation section differs for each of the regions, and the state of the electromagnetic wave is modulated for each of the regions by scattering the electromagnetic wave irradiated by the illuminator to a degree of scattering that differs for each of the regions.

8. The electromagnetic wave phase amplitude generation apparatus according to claim 1,

the diffusion plate included in the irradiation unit modulates the state of the electromagnetic wave in units of the region by making the phase of the electromagnetic wave irradiated by the illuminator random.

9. An electromagnetic wave phase amplitude generating method, wherein,

the electromagnetic wave phase amplitude generation method includes:

an irradiation step of irradiating an electromagnetic wave to be irradiated on an object to be photographed with an illuminator having a divided region and modulating at least one of an intensity, an amplitude, and a phase of the electromagnetic wave as a state of the electromagnetic wave in units of the region, and a scattering plate receiving the irradiation of the electromagnetic wave from the illuminator and modulating the state of the irradiated electromagnetic wave in units of the region to irradiate the object with an electromagnetic wave having a random irradiation pattern;

an imaging step of including an imaging element that detects a scattered electromagnetic wave that is an electromagnetic wave irradiated by the irradiation step and having the irradiation pattern and is scattered by the imaging object, and generating an imaging image by imaging the scattered electromagnetic wave; and

a generating step of generating complex amplitude information, which is information indicating at least a phase and an amplitude of the scattered electromagnetic wave, by performing a sparsity constraint operation based on sparsity of the captured image based on the captured image generated by the capturing step and information indicating the irradiation pattern.

10. A non-transitory recording medium, wherein,

the non-transitory recording medium stores an electromagnetic wave phase amplitude generation program for causing a computer to execute:

an irradiation step of irradiating an electromagnetic wave to be irradiated on an object to be photographed with an illuminator having a divided region and modulating at least one of an intensity, an amplitude, and a phase of the electromagnetic wave as a state of the electromagnetic wave in units of the region, and a scattering plate receiving the irradiation of the electromagnetic wave from the illuminator and modulating the state of the irradiated electromagnetic wave in units of the region to irradiate the object with an electromagnetic wave having a random irradiation pattern;

an imaging step of including an imaging element that detects a scattered electromagnetic wave that is an electromagnetic wave irradiated by the irradiation step and having the irradiation pattern and is scattered by the imaging object, and generating an imaging image by imaging the scattered electromagnetic wave; and

a generating step of generating complex amplitude information, which is information indicating at least a phase and an amplitude of the scattered electromagnetic wave, by performing a sparsity constraint operation based on sparsity of the captured image based on the captured image generated by the capturing step and information indicating the irradiation pattern.

Technical Field

The invention relates to an electromagnetic wave phase amplitude generating apparatus, method and non-transitory recording medium.

Background

Conventionally, the following techniques are known: an electromagnetic wave is irradiated to an object to be imaged, the irradiated electromagnetic wave is scattered by the object to be imaged, the scattered electromagnetic wave is imaged via a scattering plate having random patterns of different magnitudes that attenuate the electromagnetic wave to each region, and a complex amplitude indicating a phase and an amplitude of the electromagnetic wave scattered by the object is generated from the imaged image and the random patterns (for example, non-patent document 1).

Documents of the prior art

Non-patent document

Non-patent document 1: single-shot phase imaging with a coded adaptation (OPTICS LETTERS/Vol.39, No.22/November 15, 2014)

Disclosure of Invention

Problems to be solved by the invention

In the conventional technique, a complex amplitude indicating the phase and amplitude of an electromagnetic wave scattered by an imaging object can be generated by one imaging. However, the scattering plate attenuates the electromagnetic waves scattered by the photographic subject, the signal-to-noise ratio decreases, and the noise increases. Further, if the state of the electromagnetic wave irradiated to the object is strengthened to compensate for the weakened portion of the scattering plate, the invasion to the object becomes large, which causes a problem of adverse effect on the object.

The invention provides an electromagnetic wave phase amplitude generation device, an electromagnetic wave phase amplitude generation method, and an electromagnetic wave phase amplitude generation program, which improve the signal-to-noise ratio and have little aggressivity to an imaging object.

One aspect of the present invention is an electromagnetic wave phase amplitude generation device including: an irradiation unit that irradiates an object to be photographed with an electromagnetic wave having a random irradiation pattern at a spatial frequency that determines a state of the electromagnetic wave irradiated to each of the divided regions; an imaging unit that images a scattered electromagnetic wave to generate an imaged image, wherein the electromagnetic wave of the irradiation pattern irradiated by the irradiation unit is scattered by the imaging subject to form the scattered electromagnetic wave; and a generation unit configured to perform a sparsity constraint operation based on sparsity of the imaging target based on the captured image generated by the imaging unit, information indicating the irradiation pattern, and information indicating a signal of the imaging target, thereby generating information indicating at least a phase and an amplitude of the electromagnetic wave from the imaging target.

In the electromagnetic wave phase amplitude generation device, the information indicating the irradiation pattern includes distance wavefront pattern information indicating a state of each of a plurality of distances of the electromagnetic wave, and the generation unit performs a sparsity constraint operation based on sparsity of the imaging target based on the distance wavefront pattern information to generate information indicating a phase and an amplitude of a tomographic plane of the imaging target.

In the electromagnetic wave phase/amplitude generating apparatus, the generating unit may repeatedly generate information indicating at least the phase and amplitude of the electromagnetic wave based on the generated information indicating at least the phase and amplitude of the electromagnetic wave and the information indicating the signal of the imaging target, thereby generating information indicating at least the phase and amplitude of the electromagnetic wave.

In one aspect of the present invention, a first resolution that is a resolution of the imaging unit is lower than a second resolution that is a resolution of a sparsity constraint calculation performed by the generation unit, and the generation unit further repeatedly generates information indicating at least a phase and an amplitude of the electromagnetic wave based on a correspondence relationship between the first resolution and the second resolution to generate information indicating at least the phase and the amplitude of the electromagnetic wave so that the resolution is higher than the first resolution.

In the electromagnetic wave phase/amplitude generating apparatus, the irradiation pattern that is random at the spatial frequency may be a pattern that is spectrally uniformly spread at the spatial frequency.

In addition, according to an aspect of the present invention, in the electromagnetic wave phase amplitude generating apparatus, the electromagnetic wave is at least one of visible light, X-ray, electron beam, ultraviolet ray, infrared ray, terahertz wave, millimeter wave, and microwave.

Another aspect of the present invention is an electromagnetic wave phase amplitude generating method including: an irradiation step of irradiating an object with an electromagnetic wave of an irradiation pattern random in spatial frequency that determines a state of the electromagnetic wave irradiated to each divided region; a photographing step of photographing a scattered electromagnetic wave to generate a photographed image, wherein the electromagnetic wave of the irradiation pattern irradiated at the irradiation step is scattered by the photographic subject to form the scattered electromagnetic wave; a generation step of performing a sparsity constraint operation based on sparsity of the object based on the captured image generated by the capturing step, information indicating the irradiation pattern, and information indicating a signal of the object, thereby generating information indicating at least a phase and an amplitude of the electromagnetic wave from the object.

In addition, an aspect of the present invention is an electromagnetic wave phase amplitude generation program for executing, in a computer, the steps of:

an irradiation step of irradiating an object with an electromagnetic wave of an irradiation pattern random in spatial frequency that determines a state of the electromagnetic wave irradiated to each divided region; a photographing step of photographing a scattered electromagnetic wave to generate a photographed image, wherein the electromagnetic wave of the irradiation pattern irradiated at the irradiation step is scattered by the photographic subject to form the scattered electromagnetic wave; a generation step of performing a sparsity constraint operation based on sparsity of the object based on the captured image generated by the capturing step, information indicating the irradiation pattern, and information indicating a signal of the object, thereby generating information indicating at least a phase and an amplitude of the electromagnetic wave from the object.

Effects of the invention

According to the present invention, it is possible to provide an electromagnetic wave phase/amplitude generation device, an electromagnetic wave phase/amplitude generation method, and an electromagnetic wave phase/amplitude generation program that improve the signal-to-noise ratio and have low aggression to an imaging target.

Drawings

Fig. 1 is a diagram showing an example of an external configuration of an electromagnetic wave phase amplitude generating device.

Fig. 2 is a diagram showing an example of a functional configuration of an electromagnetic wave phase amplitude generating device.

Fig. 3 is a flowchart showing an example of the operation of the electromagnetic wave phase/amplitude generating device.

Fig. 4 is a diagram showing an example of information on the amplitude of scattered light and information on the phase of scattered light.

Fig. 5 is a diagram showing an example of the diffusion plate and the captured image.

Fig. 6 is a diagram showing an example of information generated by the generating unit from the complex amplitude information.

Fig. 7 is a diagram showing an example of the configuration of an electromagnetic wave phase amplitude generating device.

Fig. 8 is an example of the amplitude tomographic image of the imaging target generated by the generation unit.

Fig. 9 is an example of the phase tomographic image of the imaging target generated by the generation unit.

Fig. 10 is a diagram showing an example of the configuration of an electromagnetic wave phase amplitude generating device.

Fig. 11 is a diagram showing another example of the configuration of the electromagnetic wave phase amplitude generating device.

Fig. 12 is a diagram showing an example of comparison of the number of pixels of a captured image.

Fig. 13 is a diagram showing an example of an amplitude image and a phase image of scattered light in a case where the number of pixels of the imaging element matches the number of scattering areas of the scattering plate.

Fig. 14 is a diagram showing an example of an amplitude image and a phase image of scattered light generated by the generation unit in the case where the number of pixels of the imaging element is smaller than the number of scattering regions of the scattering plate.

Fig. 15 is a diagram showing an example of an amplitude image and a phase image of scattered light according to a conventional method.

Fig. 16 is a diagram showing an example of amplitude images and phase images generated in time series.

Fig. 17 is a diagram showing an example of a result of comparing the resolutions of amplitude images.

Fig. 18 is a diagram showing another example of comparison of the number of pixels of a captured image.

Fig. 19 is a diagram showing an example of an amplitude image and a phase image of scattered light in a case where the number of pixels of the imaging element matches the number of scattering areas of the scattering plate.

Fig. 20 is a diagram showing an example of an amplitude image and a phase image of scattered light generated by the generation unit in the case where the number of pixels of the imaging element is smaller than the number of scattering regions of the scattering plate.

Fig. 21 is a diagram showing an example of an amplitude image and a phase image of scattered light according to a conventional method.

Fig. 22 is a diagram showing another example of amplitude images and phase images generated in time series.

Fig. 23 is a diagram showing an example of a result of comparison of the resolutions of phase images.

Detailed Description

[ first embodiment ]

Hereinafter, an embodiment of an electromagnetic wave phase amplitude generating device will be described with reference to the drawings.

[ Structure of electromagnetic wave phase amplitude generating device ]

Fig. 1 is a diagram showing an example of an external configuration of an electromagnetic wave phase amplitude generation device 100.

The electromagnetic wave phase amplitude generation device 100 includes an irradiation unit RL and a terminal device 10.

The electromagnetic wave irradiated from the irradiation section RL is irradiated to the object OB. The object OB is a sample observed by the electromagnetic wave phase amplitude generator 100. Specifically, the object OB to be imaged is an opaque, colorless, transparent biological sample, a material or a raw material of a non-biological sample, or the like. Here, the electromagnetic wave means at least one of visible light, X-ray, electron beam, ultraviolet ray, infrared ray, terahertz wave, millimeter wave, and microwave. The electromagnetic wave is not limited to this, and may be an electromagnetic wave of an arbitrary wavelength. In this example, a case where the electromagnetic wave is visible light will be described. In the following description, visible light may be simply referred to as light. In the following description, the light irradiated from the irradiation portion RL is also referred to as irradiation light REW. The terminal device 10 captures the scattered light SL scattered by the object OB as a captured image. In this example, the terminal device 10 is a terminal including a camera, such as a smartphone.

The illumination unit RL includes an illumination L and a diffusion plate MP. The irradiation unit RL irradiates the object with an electromagnetic wave of an irradiation pattern that is random at a spatial frequency that determines a state of the electromagnetic wave irradiated to each of the divided regions. The state of the electromagnetic wave refers to the state of the intensity, amplitude, and phase of the electromagnetic wave. The state of the intensity and amplitude of the electromagnetic wave refers to the state of the intensity of the electromagnetic wave. The state of the phase of the electromagnetic wave refers to a state in which the wave of the electromagnetic wave is delayed or advanced.

The spatial frequency is a spatial frequency in a captured image captured by the terminal device 10. The light emitted from the illumination L hits the diffusion plate MP, and the diffusion light having an intensity corresponding to the light diffusion rate different for each region of the diffusion plate MP is irradiated from the diffusion plate MP to the object OB as the irradiation light REW.

Specifically, the illumination L emits light. The light emitted from the illumination L is irradiated to the subject OB via the diffusion plate MP. The illumination L is a light source that emits light having higher coherence than light whose phase and amplitude are randomly changed. That is, a light source with high coherence refers to a light source that is related to the phase and amplitude of light emitted from the light source. More specifically, the illumination L is a laser light source, a semiconductor laser light source, or an LED (LIGHT EMITTING DIODE) light source.

The light emitted from the illumination L is irradiated to the diffusion plate MP. The diffusion plate MP diffuses the light emitted from the illumination L. The diffusion plate MP irradiates the diffused light as irradiation light REW to the photographic object OB. The diffusion plate MP is a plate that changes the intensity of light irradiated from the illumination L by area. The diffusion plate MP is an optical element that modulates at least one of the intensity of an electromagnetic wave, the amplitude of the electromagnetic wave, and the phase of the electromagnetic wave. In this example, the diffuser plate MP is a spatial light modulator.

The diffusion plate MP has divided regions so that the degree of light diffusion varies. Here, the area is an area having a size corresponding to the size of the object OB. In this example, in the diffusion plate MP, the region is a region divided into squares. The regions divided into regions that scatter light to different extents refer to patterns that have different light scattering rates. Specifically, the scattering plate MP is a scattering plate in which only the intensity of the electromagnetic wave is made random and the phase of the electromagnetic wave is made constant. The scattering plate MP may be a scattering plate in which only the phase of the electromagnetic wave is randomized while the intensity of the electromagnetic wave is constant. The scattering plate MP may be a scattering plate for making the intensity of the electromagnetic wave and the phase of the electromagnetic wave random.

Here, in an example of the present embodiment, the pattern of the scattering plate MP having a different light scattering rate is a pattern in which a pattern of an area irradiated with light irradiated to the object OB is spectrally spread uniformly in spatial frequency. In other words, the pattern of the diffusion plate MP having different light diffusion rates is a pattern in which the pattern of the region irradiated with the light irradiated to the object OB does not have a peak at a spatial frequency other than the origin.

Specifically, the pattern of the scattering plate MP having different light scattering rates is a white noise-like pattern in spatial frequency. White noise samples are patterns whose periodicity is roughly difficult to observe. That is, the pattern of the scattering plate MP having different light scattering rates need not be a pattern having no peak at all in spatial frequency or a pattern having a spectrum spread uniformly in spatial frequency.

The terminal device 10 includes a display unit 13. The terminal device 10 images scattered light SL scattered by the object OB irradiated with the irradiation light REW. The scattered light SL scattered by the object OB is information indicating a signal of the object OB. The terminal device 10 generates information indicating at least the phase and amplitude of the scattered light SL from the object OB based on the captured image of the captured scattered light SL and the information indicating the random irradiation pattern of the scattering plate MP. In the following description, information indicating at least the phase and amplitude of the scattered light SL from the object OB may be described as complex amplitude information.

The display unit 13 displays the intensity, phase, and amplitude of the scattered light SL based on information indicating at least the phase and amplitude of the scattered light SL generated by the terminal device 10. Specifically, the display unit 13 displays the complex amplitude information generated by the generation unit 12. The display unit 13 displays information of the image reconstructed by the generation unit 12 based on the complex amplitude information. In this example, the display unit 13 is specifically a liquid crystal display.

[ example of the Structure of the electromagnetic wave phase amplitude generating device ]

Next, an example of the configuration of the electromagnetic wave phase/amplitude generating device 100 according to the present embodiment will be described with reference to fig. 2.

Fig. 2 is a diagram showing an example of the functional configuration of the electromagnetic wave phase amplitude generation device 100. The irradiation unit RL and the object OB are the same as described above.

The terminal device 10 includes an operation detection unit 14, an imaging unit 11, an image acquisition unit 15, a generation unit 12, a storage unit 16, and a display unit 13.

The operation detection unit 14 detects an operation from a user operating the electromagnetic wave phase amplitude generation device 100. Specifically, when the irradiation light REW is irradiated from the irradiation unit RL to the object OB, the operation detection unit 14 detects that the user has operated the irradiation command. The operation detection portion 14 that detects the irradiation command from the user outputs an instruction to irradiate the irradiation portion RL with the irradiation light REW.

The imaging unit 11 includes an imaging element (not shown). The imaging element images the scattered light SL scattered by the object OB. Specifically, the imaging element has a plurality of pixels. The image pickup element accumulates charges corresponding to the amplitude of the scattered light SL or the intensity of the scattered light SL in each pixel. The imaging unit 11 images the scattered light SL based on the electric charge accumulated in the imaging element. In the following description, the imaging element includes horizontal y pixels and vertical x pixels. The distance between the object OB and the imaging element included in the imaging unit 11 is a distance z. The imaging unit 11 generates an image IP obtained by imaging the scattered light SL. The imaging unit 11 outputs the captured image IP in which the scattered light SL is generated to the image acquisition unit 15. In the following description, the captured image IP is information indicating the intensity of the scattered light SL.

The image acquiring unit 15 acquires the captured image IP from the imaging unit 11. The image acquiring unit 15 outputs the captured image IP acquired from the imaging unit 11 to the generating unit 12.

The storage unit 16 stores pattern information RPI indicating a random irradiation pattern of the diffusion plate MP.

The generation unit 12 acquires the captured image IP from the imaging unit 11. The generation unit 12 acquires the pattern information RPI stored in the storage unit 16.

The generation unit 12 generates information indicating at least the phase and amplitude of the scattered light SL from the object OB based on the captured image IP generated by the imaging unit 11, the pattern information RPI, and the information indicating the scattered light SL. The generation unit 12 generates information indicating a phase and an amplitude by performing a sparsity constraint operation based on the sparsity of the object OB. The information indicating the phase and amplitude generated by the generation unit 12 is complex amplitude information of the scattered light SL. The generating unit 12 generates information P indicating the phase of the scattered light SL and information VA indicating the amplitude of the scattered light SL based on the generated complex amplitude information.

The generator 12 outputs the generated information indicating the phase and amplitude, the information P indicating the phase of the scattered light SL, and the information VA indicating the amplitude of the scattered light SL to the display 13.

Display unit 13 displays information indicating the phase and amplitude, information P indicating the phase of scattered light SL, and information VA indicating the amplitude of scattered light SL, which are acquired from generation unit 12.

[ outline of operation of electromagnetic wave phase amplitude generating device ]

Next, an outline of the operation of the electromagnetic wave phase amplitude generation device 100 will be described with reference to fig. 3.

Fig. 3 is a flowchart showing an example of the operation of the electromagnetic wave phase/amplitude generating apparatus 100.

The irradiation unit RL irradiates the object OB with electromagnetic waves of a random irradiation pattern (step S110). The imaging unit 11 images the scattered light SL scattered by the object OB as an imaging image IP (step S120).

The imaging unit 11 outputs the captured image IP to the generation unit 12. The generation unit 12 acquires a captured image IP. The generation unit 12 acquires pattern information RPI indicating an irradiation pattern from the storage unit 16. The generation unit 12 generates information indicating the phase and amplitude of the scattered light SL by performing a sparsity constraint operation based on the sparsity of the imaging target OB based on the captured image IP acquired from the imaging unit 11 and the pattern information RPI acquired from the storage unit 16.

Specifically, the generation unit 12 generates information indicating the phase and amplitude of the scattered light SL according to equations (1) and (2).

(number formula 1)

|g|²＝|P_zMf|²...(1)

Equation (1) is an equation representing a problem predicted by using a mathematical model, which is a positive problem.

(number formula 2)

Equation (2) is an equation representing a problem of estimating a mathematical model from data, which is an inverse problem.

X and y included in the expressions (1) and (2) are numbers corresponding to the number of pixels included in the imaging element, that is, vertical x pixels and horizontal y pixels. The same applies to x and y in the following numerical expressions.

(ii) G +contained in formula (1) and formula (2)²Is a photographed image IP photographed by the photographing element. Specifically, the captured image IP is information obtained by squaring the absolute value of the amplitude of the scattered light SL.

G included in equations (1) and (2) is complex amplitude information indicating the phase and amplitude of the scattered light SL. In the following description, complex amplitude information indicating the phase and amplitude of the scattered light SL may be simply described as complex amplitude information g. More specifically, g is a matrix represented by formula (3). The same applies to g in the following numerical expression.

(number type 3)

g∈C^(Nx×Ny)×1...(3)

P contained in the formulae (1) and (2)_ZIs a toeplitz matrix of fresnel propagation in the distance z of the object OB to the image pickup element. More specifically, P_ZIs a matrix represented by formula (4). P in the following numerical expression_ZAs well as so.

(number formula 4)

P_z∈C^{(Nx×Ny)×(Nx×Ny)}...(4)

M included in the formulas (1) and (2) is a matrix representing pattern information RPI of the diffusion plate MP. Specifically, M is a matrix represented by formula (5). The same applies to M in the following numerical expression. In this example, the pattern information RPI is information indicating the pattern of the diffusion plate MP by a numerical value from 0, which does not diffuse the irradiated light, to 1, which does diffuse the light with the intensity thereof being maintained.

(number type 5)

M∈C^{(Nx×Ny)×(Nx×Ny)}...(5)

F included in the expressions (1) and (2) is information indicating a signal of the object OB. More specifically, f is a matrix represented by formula (6). The same applies to f in the following numerical expression.

(number 6)

f∈C^(Nx×Ny)×1...(6)

Here, l contained in the formula (2)₂Is a₂And (4) norm. In the following numerical expression₂As well as so.

R (f) included in formula (2) is a sparsity constraint. Specifically, r (f) is regularization based on sparsity of information representing the signal of the object OB. τ included in expression (2) is a parameter for regularization. The same applies to R (f) and τ in the following formulae.

That is, the generation unit 12 generates the complex amplitude information g of the scattered light SL by performing sparsity constraint calculation based on the sparsity of the imaging object OB (step S130). The generation unit 12 generates the sparse constraint operation by a known method. For example, the generating unit 12 performs a sparse constraint operation using a known sparse matrix.

The generator 12 reconstructs the intensity of the scattered light SL based on the generated complex amplitude information g.

Specifically, the generator 12 reconstructs the information indicating the intensity of the scattered light SL from the square of the absolute value of the generated complex amplitude information g. The generator 12 compares the reconstructed information indicating the intensity of the scattered light SL with the captured image obtained by the imaging unit 11 capturing the object OB (step S140). When the reconstructed information indicating the intensity of the scattered light SL is similar to the captured image obtained by the imaging unit 11 capturing the image of the object OB, the process ends (step S140; yes). If the reconstructed information indicating the intensity of the scattered light SL is not similar to the captured image obtained by the imaging unit 11 capturing the image of the object OB, the generated complex amplitude information g is substituted for the equations (1) and (2), and the process of step S130 is repeated (step S140; no). The generating unit 12 may use a known technique for comparing the reconstructed information indicating the intensity of the scattered light SL with the captured image obtained by capturing the image of the object OB by the imaging unit 11. In addition, the method of comparing the reconstructed information indicating the intensity of the scattered light SL with the captured image obtained by the imaging unit 11 capturing the object OB may be a method of determining whether or not the intensity is approximate by visual observation of the user.

[ specific example of operation of electromagnetic wave phase amplitude generating device ]

The outline of the operation of the generation unit 12 is explained so far. The generation unit 12 generates complex amplitude information g by solving equations (1) and (2). In the case of solving equations (1) and (2), the electromagnetic wave phase/amplitude generation apparatus 100 may have a problem that the forward problem shown in equation (1) is a nonlinear problem and cannot be easily solved.

Here, an example of a method of generating the complex amplitude information g will be described.

[ solution method using auxiliary plane ]

In the following description, the generating unit 12 generates the complex amplitude information g by an alternate projection method.

The generation unit 12 sets the auxiliary plane a between the object OB and the imaging element. Assuming the auxiliary plane a, the expression (1) can be expressed by the expressions (7) and (8).

(number type 7)

Here, z contained in the formula (7)₁Refers to the distance between the subject OB and the auxiliary plane a.

Z contained in the formula (7)₂Is an auxiliary plane a and an imaging elementThe distance between them. If the distance z is to be₁And a distance z₂The sum is the distance z between the object OB and the imaging element. That is, the auxiliary plane a is assumed to be a distance z from the object OB₁And a distance z from the image pickup element₂Complex amplitude information of the location of (a).

(number type 8)

In the formula (8), the distance between the object OB and the auxiliary plane a is z₁Equation of time. As shown in equation (8), the auxiliary plane a is generated by solving the linearity problem.

The auxiliary plane a is generated by performing inverse fresnel transform on the temporarily stored g generated from equation (9) described later. That is, the generating unit 12 can generate the complex amplitude information g by solving the phase estimation problem.

(number type 9)

As an initial value, the generation unit 12 sets a temporary value for the complex amplitude information g. The temporary value of the complex amplitude information g as the initial value may be any value. The generation unit 12 substitutes the provisional value-set auxiliary plane g into equation (9). The generation unit 12 generates the temporarily stored G by the G-S method using equation (9). The G-S method refers to an iterative phase estimation method.

(number type 10)

The expression (10) is a modified expression of the expression (9). The generation unit 12 generates the temporarily stored g by equation (10). In addition, as shown in the formula (10), the generating section 12 uses the distance z₂The complex amplitude information at the position and the intensity of the auxiliary plane a divide the captured image IP by element.

The generator 12 multiplies a value obtained by dividing the element by 1/2 to the power, and the sum of the values at the distance z₂The complex amplitude information at the position of (a) and the auxiliary plane (a) are multiplied by element unit to generate a temporarily stored g. The generation unit 12 generates the temporarily stored auxiliary plane a by performing inverse fresnel conversion on the generated temporarily stored g.

(number formula 11)

The generation unit 12 substitutes the auxiliary plane a temporarily stored and generated according to equation (10) for equation (11). The generation unit 12 generates f that is temporarily stored by solving equation (11) by the TwIST method. The TwinT method refers to a general solution for compressed sensing.

The generation unit 12 propagates the f temporarily stored. The generation unit 12 generates the auxiliary plane a from equation (10) using the propagated temporarily stored f as an initial value.

That is, the generation unit 12 substitutes a random value into the initial value of the complex amplitude information G to generate the auxiliary plane a by the G-S method. The generation unit 12 substitutes the generated auxiliary plane a into equation (11) and generates temporarily stored f by the TwIST method. The generation unit 12 substitutes the generated temporarily stored f into expression (10) and generates a temporarily stored g with better accuracy than the random value.

The generation unit 12 repeats the above-described processing until the information indicating the intensity of the scattered light SL reconstructed from the temporarily stored g is similar to the captured image obtained by the imaging unit 11 imaging the object OB.

The method of solving equations (1) and (2) is not limited to the alternate projection method using the G-S method and the TwIST method.

[ example of information indicating the phase and amplitude of scattered light from an imaging subject, generated by an electromagnetic wave phase and amplitude generating device ]

Next, an example of the complex amplitude information g generated by the electromagnetic wave phase amplitude generation device 100 will be described with reference to fig. 4 to 6.

Fig. 4 is a diagram showing an example of information VA indicating the amplitude of the scattered light SL and information P indicating the phase of the scattered light SL.

Fig. 4(a) shows an example of the information VA indicating the amplitude of the scattered light SL.

Fig. 4(b) shows an example of information P indicating the phase of the scattered light SL. In this example, for the purpose of experiment, the information P indicating the phase is a phase obtained by rotating the original phase by 90 degrees.

Next, fig. 5 is a diagram showing an example of the diffusion plate MP and the captured image IP.

Fig. 5(a) shows an example of the diffusion plate MP and the diffusion plate MPE with a part of the diffusion plate MP enlarged. The diffuser plate MP is a random pattern in spatial frequency. That is, if the pattern of the diffusion plate MP is fourier-transformed, it is a pattern in which peaks at spatial frequencies do not occur periodically.

Fig. 5(b) is an example of an image IP obtained by the imaging unit 11 imaging the scattered light SL scattered by the imaging object OB.

Next, fig. 6 is a diagram showing an example of information generated by the information g indicating the phase and amplitude generated by the generation unit 12.

Fig. 6(a) is a diagram showing an example of the intensity RVA generated by the information indicating the amplitude generated by the generation unit 12. As can be seen by comparing fig. 6(a) and 4(a), the generator 12 generates amplitude information approximate to the information VA indicating the amplitude of the scattered light SL.

Fig. 6(b) is a diagram showing an example of the information RP showing the phase generated by the generation unit 12. As can be seen by comparing fig. 6(b) and fig. 4(b), the generator 12 generates phase information approximate to the information P indicating the phase of the scattered light SL.

[ conclusion ]

As described above, the electromagnetic wave phase amplitude generation device 100 includes the irradiation unit RL, the imaging unit 11, and the generation unit 12. The imaging unit 11 images the irradiation light REW irradiated from the irradiation unit RL and the scattered light SL scattered by the imaging object OB. The generation unit 12 generates complex amplitude information g by performing sparsity constraint calculation based on sparsity of the object OB based on the image IP captured by the imaging unit 11, the pattern information RPI, and information f indicating the signal of the object OB. The electromagnetic wave phase amplitude generation apparatus 100 can directly detect the scattered light SL by the imaging device, and can improve the signal-to-noise ratio. In addition, since the electromagnetic wave phase amplitude generation apparatus 100 can directly detect the scattered light SL by the image pickup device, the intensity of the light emitted from the irradiation portion RL can be suppressed as compared with the case where the scattered light SL is not directly detected by the image pickup device. That is, the electromagnetic wave phase amplitude generation apparatus 100 can reduce the invasiveness to the imaging target.

Since the electromagnetic wave phase amplitude generation device 100 can generate the complex amplitude information g, the thickness of the object OB and the refractive index distribution information of the electromagnetic wave can be obtained. Since the electromagnetic wave phase amplitude generation device 100 can obtain the thickness of the object OB and the refractive index distribution information of the electromagnetic wave, quantitative information can be calculated from the complex amplitude information g.

The electromagnetic wave phase amplitude generation apparatus 100 can generate the complex amplitude information g based on the captured image IP captured at one time and the pattern information RPI indicating the random irradiation pattern, and thus can generate the complex amplitude information g without damaging the object OB susceptible to the electromagnetic wave. Further, since the electromagnetic wave phase amplitude generating apparatus 100 can generate the complex amplitude information g based on the captured image IP captured at one time and the pattern information RPI indicating the random irradiation pattern, the complex amplitude information g can be generated even in the case of the moving object OB.

The generation unit 12 repeatedly generates information g indicating at least the phase and amplitude of the electromagnetic wave based on the generated information g indicating at least the phase and amplitude of the electromagnetic wave and the information g indicating the signal of the object OB. The generating unit 12 can change the nonlinear problem to the linear problem by repeatedly generating the complex amplitude information g, and can generate the information g indicating at least the phase and amplitude of the electromagnetic wave.

In the case where the random pattern irradiated by the irradiation unit RL is a pattern spectrally spread uniformly in spatial frequency, the electromagnetic wave phase amplitude generation device 100 can generate the complex amplitude information g of the scattered light SL satisfactorily on all the surfaces irradiated with the irradiation light REW from the irradiation unit RL.

In addition, the electromagnetic wave is at least one of visible light, X-ray, electron beam, ultraviolet ray, infrared ray, terahertz wave, millimeter wave, and microwave. Since the electromagnetic wave phase amplitude generating apparatus 100 is configured without a lens, it is possible to generate information indicating the phase and amplitude of an electromagnetic wave such as an X-ray, an electron beam, an ultraviolet ray, an infrared ray, and a terahertz wave, which has been difficult to manufacture a lens in the related art. Further, since the electromagnetic wave phase and amplitude generating device 100 is configured without a lens, the size of the housing of the electromagnetic wave phase and amplitude generating device 100 can be made small.

When there are a plurality of types of scattering plates MP for each object OB, the storage unit 16 stores pattern information RPI for each type of scattering plate MP. In this case, the generation unit 12 selects the pattern information RPI to be read out by the operation from the user detected by the operation detection unit 14.

In the above description, the case where the generating unit 12 generates the complex amplitude information g by performing the sparsity constraint operation has been described, but the complex amplitude information g may be calculated by another device. The other device is a Web service or the like operated by a server on the network. In this case, the generating unit 12 outputs information necessary for generating the complex amplitude information g to another device. The generating unit 12 may acquire the complex amplitude information g generated by another device.

[ second embodiment ]

An example of the configuration of the electromagnetic wave phase amplitude generation device 100-1 according to the present embodiment will be described with reference to fig. 7.

Fig. 7 is a diagram showing an example of the structure of the electromagnetic wave phase amplitude generation device 100-1. The electromagnetic wave phase amplitude generation device 100-1 of the present embodiment is different from the electromagnetic wave phase amplitude generation device 100 described above in that three-dimensional imaging of the object OB is possible. The same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

The electromagnetic wave phase amplitude generating device 100-1 generates information indicating the phase and amplitude of the scattered light SL by performing a sparsity constraint operation based on the sparsity of the imaging target OB based on the state of the wave surface at a plurality of distances of the scattered light SL. Here, the wave surfaces of the plurality of distances of the scattered light SL refer to a plurality of wave surfaces separated from each other in the traveling direction of the scattered light SL among the wave surfaces of the scattered light SL. For example, the wave surfaces at a plurality of distances of the scattered light SL are wave surfaces having different distances from the imaging unit 11 at a certain moment.

Specifically, the storage unit 16 stores pattern information RPI. The pattern information RPI of the present embodiment includes wave surface pattern information indicating the state of the wave surface at a plurality of distances of the scattered light SL at each distance. That is, the storage unit 16 stores pattern information RPI including wave surface pattern information of each distance.

The generation unit 12 generates information indicating the phase and amplitude of the scattered light SL by performing a sparsity constraint operation based on the sparsity of the imaging target OB based on the captured image IP acquired from the imaging unit 11 and the pattern information RPI acquired from the storage unit 16. Here, the generation unit 12 of the present embodiment performs a sparsity constraint operation based on the sparsity of the imaging object OB for each wavefront of the scattered light SL based on the wavefront pattern information of each distance included in the pattern information RPI.

Here, each wavefront of the scattered light SL imaged by the imaging unit 11 includes information of each tomographic plane of the imaging object OB. The generation unit 12 generates information indicating the phase and amplitude of each tomographic plane of the imaging target OB by performing sparse constraint calculation on each wavefront of the scattered light SL.

The electromagnetic wave phase/amplitude generating apparatus 100-1 can generate information indicating the phase and amplitude of each tomographic plane of the object OB generated by the generating unit 12, that is, information indicating the three-dimensional structure of the object OB.

Fig. 8 and 9 show an example of the results of information indicating the phase and amplitude of each tomographic plane of the object OB generated by the electromagnetic wave phase/amplitude generating apparatus 100-1.

Fig. 8 is an example of an amplitude tomographic image of the object OB generated by the generation unit 12.

Fig. 9 is an example of a phase tomographic image of the object OB generated by the generation unit 12.

Fig. 8 and 9 show an example of the case where the object OB is a clitocybe. As shown in fig. 8 and 9, according to the electromagnetic wave phase amplitude generating apparatus 100-1, it is possible to generate a tomographic image in which the position of the imaging unit 11 in the optical axis AX direction is variously changed. Further, according to the electromagnetic wave phase amplitude generation apparatus 100-1, by reconstructing the tomographic image, an image showing the three-dimensional structure of the object OB can be obtained.

[ third embodiment ]

An example of the structure of the electromagnetic wave phase and amplitude generation device 100-2 according to the present embodiment will be described with reference to fig. 10 to 23. The electromagnetic wave phase/amplitude generation device 100-2 of the present embodiment is different from the electromagnetic wave phase/amplitude generation device 100 and the electromagnetic wave phase/amplitude generation device 100-1 described above in that it can obtain information indicating the phase and amplitude with high resolution even when the resolution of the imaging unit 11 is low. The same components as those in the above embodiments are denoted by the same reference numerals, and descriptions thereof are omitted.

Fig. 10 is a diagram showing an example of the structure of the electromagnetic wave phase amplitude generation device 100-2.

Fig. 11 is a diagram showing another example of the structure of the electromagnetic wave phase amplitude generation device 100-2.

In an example of the present embodiment, the fact that the resolution of the imaging unit 11 is lower than the resolution of the sparsity constraint calculation performed by the generation unit 12 means that the number of pixels of the imaging element of the imaging unit 11 is relatively small. For example, the low resolution of the imaging unit 11 means that the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of pixels to be able to resolve the spatial frequency of the pattern of the diffuser plate MP. Here, if the regions on the diffusion plate MP having different diffusion rates are referred to as "diffusion regions SC of the diffusion plate MP" one by one, the low resolution of the imaging unit 11 means that the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of diffusion regions of the diffusion plate MP.

In addition, "the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP" means that | g ¬ y included in the expressions (1) and (2)²(i.e., photographed image I)P) is small compared to the size of the matrix of M (i.e., the pattern information RPI of the diffusion plate MP) contained in the formula.

Fig. 10 shows an example of a structure of coded aperture diffraction imaging. In the configuration of this example, the object light OL emitted from the object OB enters the diffusion plate MP. The scattered light SL corresponding to the object light OL is emitted from the scattering plate MP, and the emitted scattered light SL enters the imaging unit 11. That is, in the configuration of this example, the diffusion plate MP is disposed between the object OB and the imaging unit 11.

In the figure, a case is shown where the ratio of the number of pixels PX of the imaging element of the imaging section 11 to the number of scattering regions SC of the scattering plate MP is 1: 4. That is, in the example of the figure, the number of pixels PX of the imaging element of the imaging unit 11 is smaller than the number of scattering regions SC of the scattering plate MP.

Fig. 11 shows an example of a structure of coded illumination diffraction imaging. In the configuration of this example, the irradiation light REW is emitted from the diffusion plate MP, and the emitted irradiation light REW is irradiated to the object OB. When the irradiation light REW is irradiated onto the object OB, the scattered light SL corresponding to the irradiation light REW is emitted from the object OB, and the emitted scattered light SL enters the imaging unit 11. That is, in the configuration of this example, the object OB is disposed between the diffusion plate MP and the imaging unit 11.

In the case of the example of the figure, as in the example shown in fig. 10, the ratio of the number of pixels PX of the imaging element of the imaging unit 11 to the number of scattering regions SC of the scattering plate MP is 1: 4. That is, in the example of the figure, the number of pixels PX of the imaging element of the imaging unit 11 is also smaller than the number of scattering regions SC of the scattering plate MP.

The ratio of the number of pixels PX of the imaging element of the imaging unit 11 to the number of scattering regions SC of the scattering plate MP may be varied by a so-called pixel combination (binning). Here, the pixel combination refers to an imaging operation performed by combining several pixels PX of an imaging element as 1 pixel. As an example, if the pixels PX of the image pickup elements of the image pickup unit 11 are subjected to 2 × 2 pixel combination, the number of pixels after pixel combination becomes 1/4 before pixel combination. For example, when 2 × 2 pixel combination is performed in a case where the number of pixels PX of the imaging element of the imaging unit 11 matches the number of scattering regions SC of the scattering plate MP, the ratio of the number of pixels PX after pixel combination of the imaging element to the number of scattering regions SC of the scattering plate MP is 1: 4. The imaging unit 11 of the present embodiment can also perform pixel combination according to the type and size of the object OB or according to the desired resolution and processing speed.

In the following, the configuration examples shown in fig. 10 and 11 will be described by taking the case of coded illumination diffraction imaging shown in fig. 11 as an example.

As described in the first embodiment, the generation unit 12 generates information indicating the phase and amplitude of the scattered light SL by performing the calculation shown in equations (1) and (2), that is, the sparse constraint calculation based on the pattern information RPI of the scattering plate MP.

In the present embodiment, the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP. In this case, | g ∞ included in formula (1) and formula (2)²The size of the matrix does not correspond to the size of the M matrix contained in the equation. The size of the matrix here is, for example, the number of rows and columns of the matrix. Thus, | g! y included in formula (1) and formula (2)²When the size of the matrix does not match the size of the M matrix included in the equation, the calculation for generating the phase and amplitude of the scattered light SL is performed based on the correspondence relationship between the 2 matrices. When the two matrices do not have the same size, for example, the phase and amplitude of the scattered light SL are calculated by matching the sizes of the two matrices.

As described above, when the sizes of the 2 matrices are not the same, the method of making the sizes of the 2 matrices the same refers to, for example:

the phase and amplitude of the scattered light SL are generated by interpolating (for example, linear interpolation) the captured image IP captured by the imaging unit 11 (method 1: the method of the related art).

(method 2: the method of the present embodiment) the phase and amplitude of the scattered light SL are generated without interpolating the captured image IP captured by the imaging unit 11.

There are two types above. The generation unit 12 of the present embodiment employs (method 2).

Here, in the case of (method 1), by counting | g²The size of the matrix is enlarged by interpolating (for example, linear interpolation) the elements of the matrix (2) so that the sizes of the matrices are made to coincide with each other. In the case of this (method 1), information not included in the captured image IP captured by the imaging unit 11 is generated by interpolation.

On the other hand, in the case of the above (method 2), that is, the generation unit 12 of the present embodiment performs calculation on | g ∞ included in the expressions (1) and (2)²The elements of (2) are calculated without interpolation. Specifically, in the storage unit 16 of the present embodiment, | g ∞²The correspondence relationship between each element of the matrix (i.e., the pixel of the photographing section 11) and each element of the M matrix (i.e., the pixel of the diffusion plate MP) is stored. The generation unit 12 associates the pixel values of the captured image IP with the elements of the M matrix based on the correspondence relationship stored in the storage unit 16, and generates the phase and amplitude of the scattered light SL. The | g |²The correspondence relationship between each element of the matrix and each element of the M matrix is an example of the correspondence relationship between the resolution of the imaging unit 11 and the resolution of the sparse constraint calculation performed by the generation unit 12.

In this case (method 2), that is, since the generation unit 12 of the present embodiment does not interpolate the captured image IP, information not included in the captured image IP captured by the imaging unit 11 is not generated.

[ example of Experimental results ]

An example of the results of an experiment in which the electromagnetic wave phase/amplitude generation device 100-2 of the present embodiment generates an amplitude image and a phase image will be described with reference to fig. 12 to 23. First, an example of an experimental result on an amplitude image will be described with reference to fig. 12 to 17. Next, an example of the experimental results on the phase image will be described with reference to fig. 18 to 23.

[ example of Experimental results on amplitude image ]

Fig. 12 is a diagram showing an example of comparison of the number of pixels of the captured image IP. Fig. 12 a shows an example of a captured image IP (image PIC1) of the imaging unit 11 when the number of pixels of the imaging element of the imaging unit 11 matches the number of scattering areas of the scattering plate MP. Fig. 12 b shows an example of a captured image IP (image PIC2) of the imaging unit 11 when the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP. Here, the object OB is a wire (thin metal wire). In this example, the position of the line as the object OB is periodically moved. Therefore, in this one example, the positions of the lines photographed in the photographed image IP are different from each other according to the photographing time.

As shown in this example, when the number of pixels of the imaging element is smaller than the number of scattering areas of the scattering plate MP (fig. 12 b), the interval between pixels of the captured image IP is larger than when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP (fig. 12 a). That is, when the number of pixels of the imaging element is smaller than the number of scattering areas of the scattering plate MP, the resolution of the captured image IP is low.

Fig. 13 is a diagram showing an example of an amplitude image and a phase image of the scattered light SL when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP. Fig. 13 a is an example of an amplitude image (image PIC3) of the scattered light SL when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP. Fig. 13 b is an example of a phase image (image PIC4) of the scattered light SL when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP. This figure shows an amplitude image and a phase image of the scattered light SL in the case where the number of pixels of the imaging element of the imaging unit 11 is relatively large, that is, in the case where the resolution is high.

Fig. 14 is a diagram showing an example of an amplitude image and a phase image of the scattered light SL generated by the generation unit 12 in the case where the number of pixels of the imaging element is smaller than the number of scattering areas of the scattering plate MP. That is, this figure shows an amplitude image (image PIC5) and a phase image (image PIC6) of the scattered light SL in the case where the number of pixels of the imaging element of the imaging unit 11 is relatively small, that is, in the case where the resolution is low. The generation part 12 passes through the aboveThe amplitude image and the phase image of the scattered light SL are generated in (method 2). More specifically, the generation unit 12 generates information based on | g ∞ included in the expressions (1) and (2)²The correspondence between each element of the matrix (i.e., the pixel of the imaging unit 11) and each element of the M matrix (i.e., the scattering region SC of the scattering plate MP) associates the pixel value of the captured image IP with each element of the M matrix, thereby generating the phase and amplitude of the scattered light SL. That is, the generating unit 12 does not perform | g²Interpolation of the elements of (1). FIG. 14(a) shows no | g shading²An example of an amplitude image of the scattered light SL in the case of interpolation of the elements (a). FIG. 14(b) is a plan view showing no | g shading²An example of a phase image of the scattered light SL in the case of interpolation of the elements (a).

Even when the resolution of the imaging unit 11 is relatively low (the case shown in fig. 14), an amplitude image and a phase image having a resolution equivalent to that in the case where the resolution of the imaging unit 11 is relatively high (the case shown in fig. 13) can be obtained.

Fig. 15 shows an example of the calculation result of the above-described (method 1), i.e., the conventional method, as a comparison target.

Fig. 15 is a diagram showing an example of an amplitude image (image PIC7) and a phase image (image PIC8) of the scattered light SL according to the conventional method. According to the conventional method, both the amplitude image and the phase image have lower resolution than the case of the generation unit 12 of the present embodiment (the case of fig. 14).

In the present embodiment, the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP. That is, in the case of the present embodiment, the resolution of the imaging unit 11 is low relative to the spatial frequency of the diffusion plate MP. Therefore, if the generation unit 12 generates the amplitude image and the phase image by using only 1 frame of the captured image IP captured by the imaging unit 11, the resolution of the generated image becomes low.

However, as described above, the generation unit 12 of the present embodiment repeatedly generates the amplitude image and the phase image using the captured images IP of a plurality of frames. The subject OB is captured in each captured image IP of these plural frames. The photographic subjects OB photographed in these plural photographic images IP are different from each other in each frame. That is, the plurality of captured images IP include mutually different information of the object OB.

The generation unit 12 can acquire more information on the object OB than the information obtained from the captured image IP of 1 frame by repeatedly acquiring information on the object OB included in the captured image IP for each frame. Thus, the generation unit 12 can generate an amplitude image and a phase image having a resolution exceeding the resolution of the imaging unit 11.

Here, the imaging element normally outputs a signal indicating a pixel value of an image to be captured for each pixel. When the output time of the signal indicating the pixel value is constant for each pixel, the output time of the signal from all the pixels of the image sensor is shorter when the number of pixels of the image sensor is small than when the number of pixels is large. That is, when the number of pixels is small, the shooting operation can be made faster than when the number of pixels is large.

Fig. 16 is a diagram showing an example of amplitude images and phase images generated in time series. Fig. 16(a) shows an example of an amplitude image and a phase image generated under the condition shown in fig. 12(a), that is, when the number of pixels of the imaging element of the imaging unit 11 matches the number of scattering areas of the scattering plate MP. Fig. 16(b) and 16(c) each show an example of an amplitude image and a phase image generated under the condition shown in fig. 12(b), that is, when the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP. Here, fig. 16(b) shows an example of an amplitude image and a phase image generated by the above-described (method 1), i.e., the conventional method. Fig. 16(c) shows an example of an amplitude image and a phase image generated by the method adopted by the generation unit 12 of the present embodiment (method 2).

As described above, the resolution of the imaging unit 11 according to the present embodiment is lower than the resolution of the sparsity constraint calculation performed by the generation unit 12. That is, the number of pixels of the imaging unit 11 in the present embodiment is smaller than the number of scattering areas of the scattering plate MP. In other words, the number of pixels of the image pickup device of the image pickup unit 11 is smaller than that in the case where the number of pixels of the image pickup device matches the number of scattering areas of the scattering plate MP. The imaging unit 11 of the present embodiment accelerates the imaging operation (fig. 16 c) compared to the case where the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP (fig. 16 a).

The resolution of the imaging unit 11 according to the present embodiment is such that an amplitude image and a phase image are generated by the above-described (method 2). On the other hand, in the case of the conventional method (method 1), information not included in the captured image IP captured by the imaging unit 11 is generated by interpolation as described above. The information not included in the captured image IP corresponds to a noise (noise) component in the calculation based on the above equations (1) and (2). Therefore, the amplitude image and the phase image (fig. 16(b)) generated by the conventional method (method 1) have lower resolution than the amplitude image and the phase image (fig. 16(c)) generated by the conventional method (method 2). That is, the generation unit 12 of the present embodiment can generate an amplitude image and a phase image having a higher resolution than the conventional method.

Fig. 17 is a diagram showing an example of a result of comparing the resolutions of amplitude images. In the figure, the relationship between the coordinates of the object OB and the amplitude (intensity) shown in the amplitude image is shown. Waveforms of the image pickup device in the case of a higher resolution than the image pickup device of the image pickup unit 11 according to the present embodiment are represented as a waveform W1A and a waveform W1B.

Note that the waveform W1A represents the resolution of the amplitude image in the case where the imaging object OB is stopped in fig. 16 (a). The waveform W1A is a reference example of the resolution of the amplitude image.

The waveform W1B shows the resolution of the amplitude image in the case where the object OB moves in fig. 16 (a). When the image pickup device has a higher resolution than the image pickup device of the image pickup unit 11 of the present embodiment, the speed of the image pickup operation is slow, and therefore, a blur occurs in the image. The waveform W1B shows that the spread of the coordinates is larger than the waveform W1A described above, and the resolution is lower than that of the reference example.

In the case of the resolution of the imaging unit 11 according to the present embodiment, the waveform when the amplitude image is generated by the method (method 2) described above, that is, the method employed by the generating unit 12 according to the present embodiment (that is, the case of fig. 16(c)) is represented as a waveform W1C. The waveform W1C represents the resolution of the amplitude image in the case where the object OB moves in fig. 16 (c). The waveform W1C shows that the spread of the coordinates is smaller than the waveform W1B described above, and the resolution is improved, and even when the object OB moves, the resolution equivalent to that of the reference example can be obtained.

[ example of Experimental results on phase image ]

Fig. 18 is a diagram showing another example of comparison of the number of pixels of the captured image IP. Fig. 18(a) shows an example of a captured image IP (image PIC9) of the imaging unit 11 when the number of pixels of the imaging element of the imaging unit 11 matches the number of scattering areas of the scattering plate MP. Fig. 18 b shows an example of a captured image IP (image PIC10) of the imaging unit 11 when the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP. Here, as an example of an object having a phase different from that of the periphery of the object OB (for example, AIR), a thin glass (for example, a glass cover CG) is used as the object OB. As shown in this example, when the number of pixels of the imaging element is smaller than the number of scattering areas of the scattering plate MP (fig. 18(b)), the angle of view of the captured image IP is narrower than when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP (fig. 18 (a)). That is, when the number of pixels of the imaging element is smaller than the number of scattering areas of the scattering plate MP, the resolution of the captured image IP is low.

Fig. 19 is a diagram showing an example of an amplitude image and a phase image of the scattered light SL when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP. Fig. 19 a is an example of an amplitude image (image PIC11) of the scattered light SL when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP. Fig. 19 b is an example of a phase image (image PIC12) of the scattered light SL when the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP. This figure shows an amplitude image and a phase image of the scattered light SL in the case where the number of pixels of the imaging element of the imaging unit 11 is relatively large, that is, in the case where the resolution is high.

FIG. 20 shows the case where the number of pixels of the imaging element is smaller than the number of scattering areas of the scattering plate MPIn the situation, the generator 12 generates an amplitude image and a phase image of the scattered light SL. That is, this figure shows an amplitude image (image PIC13) and a phase image (image PIC14) of the scattered light SL in the case where the number of pixels of the imaging element of the imaging unit 11 is relatively small, that is, in the case where the resolution is low. The generator 12 generates an amplitude image and a phase image of the scattered light SL by the above-described (method 2). In this case (method 2), the generation unit 12 does not perform | g tintas described above²Interpolation of the elements of (1). FIG. 20(a) shows no | g shading²An example of an amplitude image of the scattered light SL in the case of interpolation of the elements (a) and (b). FIG. 20(b) shows no | g shading²An example of a phase image of the scattered light SL in the case of interpolation of the elements (a).

Even in the case where the resolution of the imaging unit 11 is relatively low (the case shown in fig. 20), an amplitude image and a phase image having a resolution equivalent to that in the case where the resolution of the imaging unit 11 is relatively high (the case shown in fig. 19) can be obtained.

Fig. 21 shows an example of the calculation result of the above-described (method 1), i.e., the conventional method, as a comparison target.

Fig. 21 is a diagram showing an example of an amplitude image (image PIC15) and a phase image (image PIC16) of the scattered light SL according to the conventional method. According to the conventional method, both the amplitude image and the phase image have lower resolution than the case of the generation unit 12 of the present embodiment (the case of fig. 20).

Fig. 22 is a diagram showing another example of amplitude images and phase images generated in time series. Fig. 22(a) shows an example of an amplitude image and a phase image generated under the condition shown in fig. 18(a), that is, when the number of pixels of the imaging element of the imaging unit 11 matches the number of scattering areas of the scattering plate MP. Fig. 22(b) and 22(c) each show an example of an amplitude image and a phase image generated under the condition shown in fig. 18(b), that is, when the number of pixels of the imaging element of the imaging unit 11 is smaller than the number of scattering areas of the scattering plate MP. Here, fig. 22(b) shows an example of an amplitude image and a phase image generated by the above-described (method 1), i.e., the conventional method. Fig. 22(c) shows an example of an amplitude image and a phase image generated by the method adopted by the generation unit 12 of the present embodiment (method 2).

As described above, the resolution of the imaging unit 11 according to the present embodiment is lower than the resolution of the sparsity constraint calculation performed by the generation unit 12. That is, the number of pixels of the imaging unit 11 in the present embodiment is smaller than the number of scattering areas of the scattering plate MP. In other words, the number of pixels of the image pickup device of the image pickup unit 11 is smaller than the number of pixels of the image pickup device matching the number of scattering areas of the scattering plate MP. The imaging unit 11 of the present embodiment accelerates the imaging operation (fig. 22 c) compared to the case where the number of pixels of the imaging element matches the number of scattering areas of the scattering plate MP (fig. 22 a).

The amplitude image and the phase image (fig. 22(b)) generated by the conventional method (method 1) have lower resolution than the amplitude image and the phase image (fig. 22(c)) generated by the conventional method (method 2). That is, the generation unit 12 of the present embodiment can generate an amplitude image and a phase image having a higher resolution than the conventional method.

Fig. 23 is a diagram showing an example of a result of comparison of the resolutions of phase images. In the figure, a relationship between the coordinates and the phase of the object OB shown in the phase image is shown. Waveforms of the image pickup device in the case of a higher resolution than the image pickup device of the image pickup unit 11 according to the present embodiment are represented as a waveform W2A and a waveform W2B.

Note that the waveform W2A represents the resolution of the phase image in the case where the imaging object OB is stopped in fig. 22 (a). The waveform W2A is a reference example of the resolution of the phase image.

The waveform W2B represents the resolution of the phase image in the case where the object OB is moving in fig. 22 (a). When the image pickup device has a higher resolution than the image pickup device of the image pickup unit 11 of the present embodiment, the speed of the image pickup operation is slow, and therefore, a blur occurs in the image. The waveform W2B has an unclear change in phase at the reference coordinates (0 (zero) in this example) as compared with the above-described waveform W2A, and has a lower resolution than the reference example.

In the case of the resolution of the imaging unit 11 according to the present embodiment, the waveform when the phase image is generated by the method (method 2) described above, that is, the method employed by the generating unit 12 according to the present embodiment (that is, the case of fig. 22(c)) is represented as a waveform W2C. The waveform W2C represents the resolution of the phase image in the case where the object OB is moving in fig. 22 (c). The waveform W2C shows that the change in phase is more clear than the waveform W2B described above, that is, the resolution is improved, and even when the object OB moves, the resolution equivalent to that of the reference example can be obtained.

As described above, the electromagnetic wave phase/amplitude generation device 100-2 of the present embodiment can increase the resolution of the amplitude image and the phase image by reducing the number of pixels of the imaging unit 11 to increase the speed of the operation and by generating the amplitude image and the phase image without interpolating the pixel values. That is, the electromagnetic wave phase/amplitude generation device 100-2 of the present embodiment can achieve both a high speed operation and an improvement in the resolution of the generated amplitude image and phase image.

While the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the embodiments, and can be modified as appropriate within a range not departing from the gist of the present invention.

The electromagnetic wave phase/amplitude generating apparatus 100 includes a computer therein. The processes of the respective processes of the apparatus are stored in a computer-readable recording medium in the form of a program, and the processes are performed by reading the program and executing the program by a computer. The computer-readable recording medium is a magnetic disk, an optical magnetic disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. The computer program may be transmitted to a computer via a communication line, and the computer receiving the transmission may execute the program.

The program may be a program for realizing a part of the above functions.

Further, the above-described functions may be realized by a combination with a program already recorded in the computer system, so-called differential file (differential program).

Description of the symbols

The device comprises a terminal device 10, an imaging part 11, a generating part 12, a display part 13, an operation detecting part 14, an image acquiring part 15, a storage part 16, an electromagnetic wave phase amplitude generating device 100, an RL irradiating part, REW irradiating light, SL scattering light and an MP scattering plate.