阅读说明：本技术 一种基于深度学习的傅里叶叠层显微图像重构方法及装置 (Fourier laminated microscopic image reconstruction method and device based on deep learning ) 是由 李秀 刘阳哲 于 2020-05-07 设计创作，主要内容包括：一种基于深度学习的傅里叶叠层显微图像重构方法及装置,其中基于神经网络的迭代优化方式,通过基于物理模型的神经网络PgNN,从低分辨率强度图序列中重构高分辨率幅值和相位分布。本发明使用基于物理模型的神经网络模型,实现从单一样本的数据集中无监督的重构样本高分辨率幅值和相位分布,且能对复杂的光学像差进行矫正。(A Fourier laminated microscopic image reconstruction method and device based on deep learning are disclosed, wherein high-resolution amplitude and phase distribution are reconstructed from a low-resolution intensity map sequence through a neural network PgNN based on a physical model based on an iterative optimization mode of a neural network. The invention uses a neural network model based on a physical model to realize unsupervised reconstruction of high-resolution amplitude and phase distribution of a sample from a data set of a single sample, and can correct complex optical aberration.)

1. A Fourier laminated microscopic image reconstruction method based on deep learning is characterized by comprising the following steps:

s1, collecting an FPM image sequence of a single sample, an incident wave vector sequence and relevant physical parameters of an experimental device;

s2, reconstructing high-resolution amplitude and phase distribution from the low-resolution intensity graph sequence through a neural network PgNN based on a physical model in an iterative optimization mode based on a neural network;

the reconstruction process comprises initialization and iterative optimization; the initialization comprises the following steps: generating an initial high resolution sample distribution O from a pre-acquired FPM image sequence⁰Generating an initial coherent transfer function distribution C from the relevant physical parameters of the experimental device⁰(ii) a The iterative optimization comprises: using a series of PgNNs with identical neural network structure_iN, the initial high resolution estimate O generated by the initialization is estimated⁰And C⁰Input PgNN₁Obtaining an updated high resolution estimate O through a parameter optimization process of the neural network¹And C⁰Repeating the operation, and obtaining a high-resolution estimation O of the sample reconstructed by the network after n times of iteration processesⁿSum coherent transfer function estimation Cⁿ。

2. The fourier stacked microscopy image reconstruction method of claim 1, wherein in the initializing, an initial sample distribution O is generated⁰Directly carrying out ultrasplitting on a low-resolution intensity map acquired under the illumination of the central lamp to obtain high-resolution amplitude distribution of a sample, and setting a zero phase as high-resolution phase distribution of the sample; or averaging all the images and then performing overdivision to obtain sample initial amplitude distribution; generating an initial coherent transfer function distribution C⁰When, C⁰Determined directly by the numerical aperture of the microscopy apparatus and the wavelength of the illumination light source, appears as a standard circular pupil function in the image plane of the camera fourier domain.

3. The fourier stacked microscopy image reconstruction method of claim 1 or 2, wherein the neural network PgNN based on the FPM physical model_iWherein the optical imaging model of the FPM is as follows:

in_lm(k)＝O⁽ⁱ⁾(k+k_m)·C⁽ⁱ⁾(k)

In the formula k_mRecording position information of the illumination light source for the incident wave vector recorded during the data acquisition process, O⁽ⁱ⁾And C⁽ⁱ⁾Are respectively input PgNN_iThe high-resolution complex amplitude distribution and the coherent transfer function of the sample to be reconstructed, and k refers to a Fourier domain coordinate; and introduces an alternate projection principle to realize the updating process of the network self-supervision parameters:

in the formula I_lmIs and k_mLow-resolution intensity maps in a one-to-one correspondence, wherein F represents Fourier transform;

integrating FPM optical imaging model and alternate projection principle, PgNN_iIn the network structure of (1), a high-resolution image O to be reconstructedⁱModeling network hidden layer parameters, fusing the parameters with an incident wave vector k_mCombined to form Oⁱ(k+k_m) Then via the point multiplication module and the coherent transfer function CⁱMultiplying to generate phi_lm，Φ_lmBy alternating projection processing, using I_lmUpdating the spectral information to obtain phi_hm(ii) a Measuring phi by MSE loss function_lmAnd phi_hmThe difference between them, PgNN_iUnsupervised to OⁱAnd CⁱUpdating parameters; extracting PgNN through an iterative optimization process_iThe hidden layer parameters of the middle representative sample distribution and the coherent transfer function are obtained to obtain updated Oⁱ⁺¹And Cⁱ⁺¹。

4. The reconstruction method of the Fourier laminated microscopic image according to claim 3, wherein the alternate projection is derived from a phase recovery algorithm, in the first step, Fourier domain specific frequency spectrum information is projected to a space domain, in the second step, the acquired low-resolution intensity map is used for constraining the space domain amplitude of the frequency spectrum information and keeping the phase unchanged, and finally, the updated space domain complex amplitude distribution is converted back to the Fourier domain for updating the frequency spectrum information; when the deep learning model converges, the frequency spectrum deviation before and after alternate projection tends to 0, so that the network can reconstruct the complex amplitude distribution of the sample without supervision.

5. The method of reconstructing a fourier stacked microscopy image of any one of claims 1 to 4, wherein the PgNN is configured to reconstruct a fourier transform of the image of the object_iIntroducing an aberration correction module to compensate the optical aberration, wherein the aberration correction module is an embedded optical aberration correction module based on an alternate updating process and a physical model; in the optical imaging model of the FPM, C is selected under consideration of optical aberrationⁱ(k) Can represent the complex of all optical disturbance factors in the acquisition process by aligning C in the FPM reconstruction processⁱ(k) Correcting the variable, and compensating optical aberration including defocusing aberration in the acquired image sequence; PgNN, a network-learnable parameter by directly modeling coherent transfer functions_iAn optical aberration variable is learned from a sequence of acquired images.

6. The method of fourier stacked microscopy image reconstruction of claim 5, wherein the fourier stacked microscopy image reconstruction is performedThe alternating update process changes the information flow of the network gradient update so that the network focuses on updating the sample or aberration at the same time; wherein, in PgNN₁In the iterative optimization process, the sample parameter O is firstly activated⁰To obtain updated O¹And a coherent transfer function C⁰Keeping the same; at this point, the updated sample parameter O is considered¹Ratio C⁰Closer to the optimal solution, the network will fix O¹Go to update C⁰To obtain C¹(ii) a And setting alternate updating of the number of rounds to reconstruct both the sample and the coherent transfer function.

7. The method of reconstructing a fourier stacked microscopy image of any one of claims 1 to 6, wherein the PgNN is configured to reconstruct a fourier transform of the image of the object_iIntroducing Zernike polynomial mechanism to compensate optical aberration phase, replacing coherent transfer function C (k) phase part with Zernike polynomial, introducing Zernike polynomial, PgNN_iThe expression of (C), (k) is as follows:

C(k)＝|C(k)|·exp{i∠C(k)}

∠C(k)＝∑c_l·Z_l(k)

in the formula Z_l(k) Zernike polynomials of different orders, c_lAre the corresponding coefficients.

8. The method of reconstructing a fourier stacked microscopy image of any one of claims 1 to 6, wherein the PgNN is configured to reconstruct a fourier transform of the image of the object_iIntroducing a total variation loss function aiming at FPM amplitude and phase distribution to optimize a network structure; introducing full variation terms to the sample amplitude and the phase distribution respectively, wherein the form of the full variation terms and the improved network loss function are as follows:

TV{o}＝∑(|o_x+1，y-o_x，y|²+|o_x，y-1-o_x，y|²)^1/2

where TV items are exemplified by o, tableThe calculation mode is described; the network loss function is based on the original MSE and is used for F after the alternate projection process^-1{Φ_hmThe amplitude and phase components are separately computed to form total variational terms, α₁And α₂Respectively, are corresponding term coefficients.

9. A deep learning-based fourier stack microscopy image reconstruction apparatus comprising a computer readable storage medium and a processor, the computer readable storage medium storing an executable program, wherein the executable program, when executed by the processor, implements the deep learning-based fourier stack microscopy image reconstruction method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the method of deep learning based fourier-stack microscopy image reconstruction according to any one of claims 1 to 8.