阅读说明：本技术 一种图片合成方法、装置、电子设备及存储介质 (Picture synthesis method and device, electronic equipment and storage medium ) 是由 李迪迪 王晓良 范文超 于 2021-10-28 设计创作，主要内容包括：本申请提供了一种图片合成的方法、装置、电子设备及存储介质,包括：获取样品不同区域的局部三维图像,其中,所述局部三维图像为在光片式成像显微镜若干个光片目的束腰位置不同的激发光源下拍摄的；对所述局部三维图像进行相对可靠性筛选,得到拼接参数,其中,所述相对可靠性由所述局部三维图像的亮度和/或分辨率计算得出；基于所述拼接参数对应的样品不同区域,将所述三维图像进行拼接,得到所述样品的完整三维图像。解决了现有技术依靠人工获得图像合成参数导致的数据预处理过程重复性较差的问题,简化了操作流程,提高了图像合成过程的准确性,降低了时间成本和人力成本。(The application provides a method, a device, an electronic device and a storage medium for picture synthesis, which comprise the following steps: acquiring local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are shot under excitation light sources with different beam waist positions of a plurality of polished sections of a polished section type imaging microscope; screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image; and splicing the three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample. The problem of prior art rely on the artifical data preprocessing process repeatability that obtains the image synthesis parameter and lead to relatively poor is solved, operation flow has been simplified, image synthesis process's accuracy has been improved, time cost and human cost have been reduced.)

1. A method for picture synthesis, the method comprising:

acquiring local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are shot under excitation light sources with different target beam waist positions of a plurality of excitation light sheets under a light sheet type imaging microscope;

screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image;

and splicing the local three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

2. The method of claim 1, wherein the step of obtaining localized three-dimensional images of different regions of the sample comprises:

acquiring planar image information of a sample, wherein the planar image information comprises timestamp label data and a planar image;

calculating a time interval between adjacent planar images based on the timestamp label data;

obtaining the physical position of the plane image according to the time interval and the displacement data of the sample displacement table;

obtaining the position of a target beam waist area of the exciting optical sheet according to the time interval and the optical zoom data of the shooting equipment;

and reconstructing the plane image into the local three-dimensional image based on the physical position of the plane image and the beam waist area position of the excitation light sheet.

3. The method of claim 2, wherein the step of obtaining localized three-dimensional images of different regions of the sample further comprises:

calculating time intervals between all adjacent plane images based on the timestamp label data;

clustering displacements of an optical zoom system of the shooting equipment and a sample displacement table during the interval of generation of adjacent pictures by adopting a support vector machine method based on the time interval between all adjacent plane images to obtain all physical positions of the plane images and all beam waist area positions of the excitation polished section;

reconstructing the planar image into a local three-dimensional image based on all physical positions of the planar image and all beam waist region positions of the excitation polished section.

4. The method of claim 2, wherein the step of obtaining localized three-dimensional images of different regions of the sample comprises:

dividing the plane image into at least two small areas according to the direction perpendicular to the light rays emitted by an optical zoom system of the shooting equipment;

evaluating the definition of each small region by adopting a gradient function to obtain an evaluation result of each small region;

obtaining the relative definition change trend of the plane image by comparing the evaluation results;

analyzing the relative definition change trend of the plane image to obtain the change direction and the change period of the target beam waist position of the exciting optical sheet;

obtaining a sample physical position corresponding to the plane image according to the displacement direction of the sample displacement table, the change direction of the target beam waist position of the exciting polished section and the change period;

and three-dimensionally reconstructing the planar image based on the physical position of the sample.

5. The method according to claim 1, wherein the step of screening the local three-dimensional image for relative reliability to obtain a stitching parameter comprises:

calculating relative reliability data of a local three-dimensional image based on the brightness and/or resolution of the three-dimensional image;

selecting the highest value in the reliability data as a boundary of a preset key area, wherein the preset key area is a superposition area between the local three-dimensional image and an adjacent three-dimensional image;

selecting a three-dimensional image with relative reliability difference on two sides of the boundary within a preset range as a superposed image in the superposed region;

and taking the boundary and the overlapped image as the splicing parameter.

6. The method of claim 5, wherein the step of calculating relative reliability data of the local three-dimensional image based on the brightness of the three-dimensional image comprises:

acquiring all effective pixel points of the three-dimensional image to obtain the brightness ratios of all effective pixel points in the beam waist areas of different excitation polished sections under the irradiation of excitation light;

normalizing the brightness ratios of all effective pixel points of the three-dimensional image, and fitting the excitation light energy distribution of different images by adopting normal distribution of negative exponential terms to obtain two-dimensional normal distribution;

calculating the variance of the excitation light energy density of each effective pixel point according to the characteristic that the excitation light energy density obeys two-dimensional normal distribution;

taking the variance as the relative reliability data.

7. The method of claim 5, wherein the step of calculating relative reliability data of the local three-dimensional image based on the resolution of the three-dimensional image comprises:

comparing the image resolution of different three-dimensional images at the same position in the direction vertical to the optical axis of the exciting light by using a gradient function;

taking the resolution as the relative reliability data.

8. The method according to claim 5, wherein the step of stitching the local three-dimensional images based on different regions of the sample corresponding to the stitching parameters comprises:

synthesizing an overlapped three-dimensional image by adopting an interpolation method based on the boundary and the overlapped image;

stitching the three-dimensional images based on the overlapping three-dimensional images.

9. A picture composition apparatus, characterized in that the apparatus comprises:

the reconstruction module is used for acquiring local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are shot under a light sheet type imaging microscope under excitation light sources with different target beam waist positions of a plurality of excitation light sheets;

the screening module is used for screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image;

and the splicing module is used for splicing the local three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

10. An electronic device, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, the processor and the storage medium communicating via the bus when the electronic device is running, the processor executing the machine-readable instructions to perform the steps of the picture composition method according to any one of claims 1 to 8.

11. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, is adapted to carry out the steps of the picture composition method according to any one of claims 1 to 8.

Technical Field

The present application relates to the field of digital image processing technologies, and in particular, to a method and an apparatus for picture synthesis, an electronic device, and a storage medium.

Background

The conventional optical microscope is only suitable for extremely thin samples or can only obtain clear images of the surfaces of the samples, and for samples with large volume and thick thickness, clear optical imaging cannot be carried out on the surfaces and the inside of the samples without damaging the overall structure of the samples, and the three-dimensional structure of the samples can not be obtained. The light sheet type microscope can obtain a micron-sized illumination light source with a limited length and a diameter through a laser and lens system, and can realize micron-sized three-dimensional imaging on the local part inside a biological tissue or an organ on the premise of keeping the integrity of a basic structure by matching with a transparentization and fluorescence labeling technology.

In order to obtain a three-dimensional image with high overall resolution of a large sample, in the current common method, an optical microscope is used for collecting images for multiple times, different local clear images of the sample are obtained each time, clear images are selected and spliced, and finally, a clear image of the whole sample is obtained. However, in the process, parameters required by image splicing depend on manual identification, including parameters such as positions, ranges and numbers of clear images, and the defects of non-uniform identification standards, large errors, long consumed time, complex steps, high labor cost and the like of different experimenters exist.

Disclosure of Invention

In view of the above, an object of the present application is to provide a picture synthesis method, including:

acquiring local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are shot under excitation light sources with different beam waist positions of a plurality of polished sections of a polished section type imaging microscope;

screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image;

and splicing the three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

Optionally, the step of obtaining local three-dimensional images of different regions of the sample includes:

acquiring planar image information of a sample, wherein the planar image information comprises timestamp label data and a planar image;

calculating a time interval between adjacent planar images based on the timestamp label data;

obtaining the physical position of the plane image according to the time interval and the displacement data of the sample displacement table;

obtaining the position of a target beam waist area of the exciting optical sheet according to the time interval and the optical zoom data of the shooting equipment;

and reconstructing the plane image into the local three-dimensional image based on the physical position of the plane image and the beam waist area position of the excitation light sheet.

Optionally, the step of obtaining local three-dimensional images of different regions of the sample further includes:

calculating time intervals between all adjacent plane images based on the timestamp label data;

clustering displacements of an optical zoom system of the shooting equipment and a sample displacement table during the interval of generation of adjacent pictures by adopting a support vector machine method based on the time interval between all adjacent plane images to obtain all physical positions of the plane images and all beam waist area positions of the excitation polished section;

reconstructing the planar image into a local three-dimensional image based on all physical positions of the planar image and all beam waist region positions of the excitation polished section.

Optionally, the step of obtaining local three-dimensional images of different regions of the sample includes:

dividing the plane image into at least two small areas according to the direction perpendicular to the light rays emitted by an optical zoom system of the shooting device;

evaluating the definition of each small region by adopting a gradient function to obtain an evaluation result of each small region;

obtaining the relative definition change trend of the plane image by comparing the evaluation results;

analyzing the relative definition change trend of the plane image to obtain the change direction and the change period of the target beam waist position of the exciting optical sheet;

obtaining a sample physical position corresponding to the plane image according to the displacement direction of the sample displacement table, the change direction of the target beam waist position of the exciting polished section and the change period;

and three-dimensionally reconstructing the planar image based on the physical position of the sample.

Optionally, the step of screening the relative reliability of the local three-dimensional image to obtain the stitching parameter includes:

calculating relative reliability data of a local three-dimensional image based on the brightness and/or resolution of the three-dimensional image;

selecting the highest value in the reliability data as the boundary of the preset key area, wherein the preset key area is the overlapping area between the local three-dimensional image and the adjacent three-dimensional image;

selecting a three-dimensional image with relative reliability difference on two sides of the boundary within a preset range as a superposed image in the superposed region;

and taking the boundary and the overlapped image as the splicing parameter.

Optionally, the step of calculating relative reliability data of the local three-dimensional image based on the brightness of the three-dimensional image includes:

obtaining all effective pixel points of the three-dimensional image to obtain the brightness ratio of all effective pixel points in different target beam waist areas under the irradiation of excitation light;

normalizing the brightness ratios of all effective pixel points of the three-dimensional image, and fitting the excitation light energy distribution of different images by adopting normal distribution of negative exponential terms to obtain two-dimensional normal distribution;

calculating the variance of the excitation light energy density of each effective pixel point according to the characteristic that the excitation light energy density obeys two-dimensional normal distribution;

taking the variance as the relative reliability data.

Optionally, the step of calculating relative reliability data of the local three-dimensional image based on the resolution of the three-dimensional image includes:

comparing the image resolution of different three-dimensional images at the same position in the direction vertical to the optical axis of the exciting light by using a gradient function;

taking the resolution as the relative reliability data.

Optionally, the step of stitching the three-dimensional image based on different regions of the sample corresponding to the stitching parameters includes:

synthesizing an overlapped three-dimensional image by adopting an interpolation method based on the boundary and the overlapped image;

stitching the three-dimensional images based on the overlapping three-dimensional images.

In a second aspect, an embodiment of the present application provides a picture synthesis apparatus for an optical sheet type imaging microscope, the apparatus including:

the reconstruction module is used for acquiring local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are shot under a light sheet type imaging microscope under excitation light sources with different target beam waist positions of a plurality of excitation light sheets;

the screening module is used for screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image;

and the splicing module is used for splicing the three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

In a third aspect, an embodiment of the present application provides an electronic device, including: the picture synthesis method comprises a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the storage medium are communicated through the bus, and the processor executes the machine-readable instructions to execute the steps of the picture synthesis method.

In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the picture synthesis method as described above.

The image synthesis method and the image synthesis device provided by the embodiment of the application replace manual analysis of original image data and calculation of image synthesis parameters, and automation of a preprocessing process is realized. Compared with the image splicing method in the prior art, the image splicing method solves the technical problems that the repeatability of the data preprocessing process is poor and is greatly influenced by the difference of experimenters due to the fact that image synthesis parameters are obtained manually in the prior art, obviously simplifies the operation process and reduces the time cost and the labor cost. The accuracy of picture splicing is improved, human errors are avoided, and the accuracy of obtaining a complete three-dimensional image through the slide imaging microscope is improved.

In order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.

Fig. 1 is a flowchart illustrating a picture synthesis method provided in an embodiment of the present application;

fig. 2 is a schematic structural diagram illustrating a picture synthesis apparatus provided in an embodiment of the present application;

fig. 3 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

fig. 4 shows a schematic structural diagram of a storage medium provided in an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. Every other embodiment that can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present application falls within the protection scope of the present application.

First, an application scenario to which the present application is applicable will be described. The application can be applied to a light sheet type microscope, and can obtain the illumination light source with the diameter of limited length being in a micron order through a laser and a lens system, and can realize the scene of micron-order three-dimensional imaging to the part inside biological tissues or organs on the premise of keeping the integrity of a basic structure by matching with a transparentization and fluorescence marking technology.

Research shows that in order to obtain a three-dimensional image with high overall resolution of a large sample, in the current common practice, an optical-film microscope is used for collecting images for multiple times, different local clear images of the sample are obtained each time, and clear images are selected and spliced to finally obtain a clear image of the whole sample. However, in the process, parameters required by image splicing depend on manual identification, including parameters such as positions, ranges, number and the like of clear images, so that the defects of non-uniform identification standards, large errors, long consumed time, complex steps, high labor cost and the like of different experimenters exist.

Based on this, the embodiment of the application provides a picture synthesis method of a light-sheet type imaging microscope, which adopts a computer to completely replace all manual links in the image synthesis step, improves the consistency and accuracy of the image processing link, improves the automation degree, and reduces the labor cost and the time cost.

Referring to fig. 1, fig. 1 is a flowchart of a picture synthesis method according to an embodiment of the present disclosure. As shown in fig. 1, a picture synthesis method provided in an embodiment of the present application includes:

s101, obtaining local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are shot under excitation light sources with different beam waist positions of a plurality of polished sections of a polished section type imaging microscope;

in a possible embodiment, the step of obtaining local three-dimensional images of different regions of the sample comprises:

acquiring planar image information of a sample, wherein the planar image information comprises timestamp label data and a planar image;

calculating a time interval between adjacent planar images based on the timestamp label data;

obtaining the physical position of the plane image according to the time interval and the displacement data of the sample displacement table;

obtaining the position of a target beam waist area of the exciting optical sheet according to the time interval and the optical zoom data of the shooting equipment;

and reconstructing the plane image into the local three-dimensional image based on the physical position of the plane image and the beam waist area position of the excitation light sheet.

Illustratively, if the tag information of the plane image information is provided with a timestamp, a time interval of the generation time of adjacent pictures is obtained, the actions of the time interval optical zoom system and the sample displacement stage are judged according to the optical zoom of the microscope and the mechanical characteristics of the sample displacement stage, so that the physical position of the picture and the target beam waist area position of the excitation light sheet can be obtained, and the plane image is reconstructed into the local three-dimensional image based on the physical position of the picture and the target beam waist area position of the excitation light sheet.

In a possible embodiment, the step of obtaining the local three-dimensional images of different regions of the sample further comprises:

calculating time intervals between all adjacent plane images based on the timestamp label data;

clustering displacements of an optical zoom system of the shooting equipment and a sample displacement table during the interval of generation of adjacent pictures by adopting a support vector machine method based on the time interval between all adjacent plane images to obtain all physical positions of the plane images and all beam waist area positions of the excitation polished section;

reconstructing the planar image into a local three-dimensional image based on all physical positions of the planar image and all beam waist region positions of the excitation polished section.

Illustratively, applying an SVM (direct vector machine) method to all time intervals, clustering the actions of the optical zoom system and the sample displacement table during the generation interval of adjacent pictures, and obtaining the physical position of the picture and the beam waist area position of the exciting polished section according to the picture shooting sequence. And then, reconstructing the pictures with similar target beam waist area positions of the exciting light sheets into three-dimensional images according to the corresponding physical positions of the pictures to obtain a plurality of three-dimensional images with different target beam waist area positions in the same visual field.

In a possible embodiment, the step of obtaining local three-dimensional images of different regions of the sample comprises:

dividing the plane image into at least two small areas according to the direction perpendicular to the light rays emitted by an optical zoom system of the shooting device;

evaluating the definition of each small region by adopting a gradient function to obtain an evaluation result of each small region;

obtaining the relative definition change trend of the plane image by comparing the evaluation results;

analyzing the relative definition change trend of the plane image to obtain the change direction and the change period of the target beam waist position of the exciting optical sheet;

obtaining a sample physical position corresponding to the plane image according to the displacement direction of the sample displacement table, the change direction of the target beam waist position of the exciting polished section and the change period;

and three-dimensionally reconstructing the planar image based on the physical position of the sample.

For example, if there is no timestamp information in the tag information of the image or the timestamp is not selected to be used as the basis for three-dimensional image reconstruction, the image information of the picture is used as the basis for three-dimensional reconstruction, and in the same field of view, the image closer to the center of the target beam waist region has higher resolution, the sharper the image details are, and the higher the peak brightness is. Therefore, the pictures are divided into small areas according to the direction perpendicular to the light rays, the definition of each part is evaluated by using a Tenengrad gradient function, the relative definition change trend of all or part of the pictures in all the pictures is compared, the change direction and the period of the target beam waist position of the exciting polished section can be obtained, and the physical position of the sample corresponding to each picture can be known by combining the displacement direction of the sample displacement table, so that the three-dimensional reconstruction of the images is realized. When the definition of each part is evaluated, a plurality of definition evaluation methods without reference are available for selection, including but not limited to Tenengrad gradient function, Laplacian gradient function, SMD (grayscale variance) function, Vollant function and entropy function.

S102, screening relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or resolution of the local three-dimensional image;

illustratively, when the exciting light passes through the lens system and irradiates the sample, a more concentrated area exists, the light distribution in the area is more concentrated, the resolution is higher, the farther away from the area, the more dispersed the light is, the lower the resolution is, and the resolution of each position in each three-dimensional image is different.

In a possible embodiment, the step of screening the local three-dimensional image for relative reliability includes:

calculating relative reliability data of a local three-dimensional image based on the brightness and/or resolution of the three-dimensional image;

selecting the highest value in the reliability data as the boundary of the preset key area, wherein the preset key area is the overlapping area between the local three-dimensional image and the adjacent three-dimensional image;

selecting a three-dimensional image with relative reliability difference on two sides of the boundary within a preset range as a superposed image in the superposed region;

and taking the boundary and the overlapped image as the splicing parameter.

Illustratively, the excitation light source is laser light, the excitation light energy density follows a two-dimensional normal distribution, the resolution at the smaller distribution variance is higher, the resolution at the larger distribution variance is lower, and because the excitation light is swept in the ocular focal plane during imaging, the excitation light energy distribution approximately follows a one-dimensional normal distribution on the ocular focal plane, and considering that the energy of the excitation light decreases as the light path increases as the light penetrates through the sample during actual imaging, the remaining energy of the light is approximately in a negative exponential relationship with the light path, and therefore, the excitation light energy distribution in the ocular focal plane can be fitted using a normal distribution with a negative exponential term. This step can also be directly fitted with a negative degree polynomial to simplify the calculation.

In a possible embodiment, the step of calculating relative reliability data of the local three-dimensional image based on the attribute information of the three-dimensional image comprises:

obtaining all effective pixel points of the three-dimensional image to obtain the brightness ratio of all effective pixel points in different target beam waist areas under the irradiation of excitation light;

normalizing the brightness ratios of all effective pixel points of the three-dimensional image, and fitting the excitation light energy distribution of different images by adopting normal distribution of negative exponential terms to obtain two-dimensional normal distribution;

calculating the variance of the excitation light energy density of each effective pixel point according to the characteristic that the excitation light energy density obeys two-dimensional normal distribution;

taking the variance as the relative reliability data.

Illustratively, the brightness of a tiny area in the sample in the picture is approximately proportional to the energy density of the excitation light in the tiny area. Removing background fluorescence, taking all effective pixel points, calculating the brightness ratio of all effective pixel points under the irradiation of excitation light of different target beam waist regions, then carrying out normalization processing on the whole brightness, further fitting the excitation light energy distribution of different images by using the excitation light energy distribution in the ocular focal plane obtained by the analysis, and then calculating the variance of the excitation light energy density at each position according to the characteristic of two-dimensional normal distribution of the excitation light energy density, wherein the smaller the variance is, the higher the relative reliability of the image information at the position is, the larger the variance is, and the lower the relative reliability of the image information at the position is. Thereby obtaining relative reliability data of all three-dimensional images at any position.

In a possible embodiment, the step of calculating relative reliability data of the local three-dimensional image based on the attribute information of the three-dimensional image comprises:

comparing the image definition of different three-dimensional images at the same position in the direction vertical to the optical axis of the exciting light by using a gradient function;

taking the sharpness as the relative reliability data.

Illustratively, the excitation light is scanned in parallel in the focal plane of the eyepiece, and the optical axis of the eyepiece is perpendicular to the scanning plane of the excitation light, so that from the analysis of the illumination thickness of the sample, the illumination thickness of the target beam waist region of the excitation light sheet is thinner, and the illumination thickness of the target beam waist region is larger as the position is farther away, so that the difference of the image resolution of different positions of the sample is more reflected in the direction parallel to the optical axis of the eyepiece, and the resolution is higher as the position is closer to the target beam waist region of the excitation light sheet, and the resolution is lower as the position is farther away from the target beam waist region of the excitation light sheet. The Tenengrad gradient function is used for comparing the image definition of different three-dimensional images at the same position in the direction perpendicular to the optical axis of the exciting light, and the difference of the definition and the difference of the resolution have approximately the same trend, so that the definition of different positions calculated by the Tenengrad gradient function can be used as image information relative reliability data. When the definition of each part is evaluated, a plurality of definition evaluation methods without reference are available for selection, including but not limited to Tenengrad gradient function, Laplacian gradient function, SMD (grayscale variance) function, Vollant function and entropy function.

S103, splicing the three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

In a possible embodiment, the step of stitching the three-dimensional image based on different regions of the sample corresponding to the stitching parameters includes:

synthesizing an overlapped three-dimensional image by adopting an interpolation method based on the boundary and the overlapped image;

stitching the three-dimensional images based on the overlapping three-dimensional images.

Illustratively, an image with the highest relative reliability on the local three-dimensional image is taken for any position area of the sample to obtain a boundary from different image data, partial pixel points are taken to two sides respectively to serve as an overlapping area of a splicing boundary according to the relative reliability of the image of pixel points near the boundary data, the position where the difference of the relative reliability of the taken image reaches 30% to 60% is taken as an end point of the overlapping area, and the adjustment can be carried out according to the actual imaging condition.

Illustratively, the images of the overlapping regions are synthesized by interpolation. The two side regions can be interpolated respectively, the interpolation weights of the data from the two images are equal at the boundary point of the image synthesis, the weight of the data with higher reliability at the edge of the overlapping region is 1, the weight of the data with lower reliability is 0, the weight of the data between the boundary point and the edge is calculated respectively according to the linear interpolation of the weights at the two sides, and other interpolation methods can also be adopted to calculate the weight respectively. Then, the other non-overlapping regions acquire data with the highest relative reliability of the position image, and a synthesized image is obtained.

The computer is adopted for automatic processing, and all steps can be continuously carried out, so that the same image file can be processed only by reading from and writing into the memory once, and the network bandwidth and the processing time can be remarkably saved. The computer is used for calculating the splicing parameters, the splicing parameters obtained by the same image data can be guaranteed to be stable and do not change due to the difference of time, space and experimenters, the error sources are obviously reduced, and the high repeatability of the experimental data is guaranteed. The relative reliability of all local image information of all images is compared while the splicing parameters are calculated, so that the relative reliability of the splicing parameters can be evaluated, and a reference is provided for evaluating the imaging quality of the sample. In the image synthesis step, the overlapped parts of different areas can be adjusted according to the image reliability difference of the areas, so that more original image information can be saved as much as possible on the premise of ensuring the image smoothness, and the three-dimensional structure of the sample can be better reduced.

In one possible implementation, as shown in fig. 2, an embodiment of the present application provides a picture synthesis apparatus 200, including:

the reconstruction module 201 is configured to acquire local three-dimensional images of different areas of a sample, where the local three-dimensional images are captured by an excitation light source with different beam waist positions of a plurality of light sheets of a light sheet type imaging microscope;

the screening module 202 is configured to perform relative reliability screening on the local three-dimensional image to obtain a stitching parameter, where the relative reliability is calculated by brightness and/or resolution of the local three-dimensional image;

and the splicing module 203 is configured to splice the target three-dimensional images based on different sample regions corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

In one possible implementation, as shown in fig. 3, an embodiment of the present application provides an electronic device 300, including: a processor 310, a memory 320 and a bus 330, wherein the memory 320 stores machine-readable instructions executable by the processor 310, when the electronic device is operated, the processor 310 communicates with the memory 320 via the bus 330, and the processor 310 executes the machine-readable instructions to perform acquiring local three-dimensional images of different areas of a sample, wherein the local three-dimensional images are taken under excitation light sources with different beam waist positions of a plurality of light sheets of a light sheet type imaging microscope; screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image; and splicing the three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

In one possible implementation, as shown in fig. 4, the present application provides a computer-readable storage medium 400, where a computer program 411 is stored on the computer-readable storage medium, and when executed by a processor, the computer program 411 performs acquiring local three-dimensional images of different areas of a sample, where the local three-dimensional images are taken under excitation light sources with different beam waist positions of a plurality of light sheets of a light sheet type imaging microscope; screening the relative reliability of the local three-dimensional image to obtain a splicing parameter, wherein the relative reliability is calculated by the brightness and/or the resolution of the local three-dimensional image; and splicing the three-dimensional images based on different areas of the sample corresponding to the splicing parameters to obtain a complete three-dimensional image of the sample.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.