Method and device for positioning focal plane of front lens of light field camera
阅读说明：本技术 光场相机前置镜焦平面定位方法和装置 (Method and device for positioning focal plane of front lens of light field camera ) 是由 袁艳 苏丽娟 朱聪慧 王继超 于 2019-09-26 设计创作，主要内容包括：本发明公开了一种光场相机前置镜焦平面定位方法和装置,其中,方法包括：搭建成像装置,并确定与自准直经纬仪出射光束垂直的参考平面；搭建测量装置,并确定与参考平面垂直的光束,采集不同标称物距下的光场图像；根据不同标称物距下的光场图像得到相邻微透镜质心之间的距离,并根据质心距离指标定位前置镜系统的焦平面位置。该方法可以灵活控制进入光场相机的入射光方向,且适用于其他需要定位焦平面的光学成像系统,并消除了目视解译的人为因素的判别干扰,从而可以准确定位光场相机前置镜系统的焦平面。(The invention discloses a method and a device for positioning a focal plane of a front lens of a light field camera, wherein the method comprises the following steps: constructing an imaging device, and determining a reference plane vertical to an emergent beam of the auto-collimation theodolite; building a measuring device, determining a light beam vertical to a reference plane, and collecting light field images under different nominal object distances; and obtaining the distance between the centroids of the adjacent microlenses according to the light field images under different nominal object distances, and positioning the focal plane position of the front lens system according to the centroid distance index. The method can flexibly control the direction of incident light entering the light field camera, is suitable for other optical imaging systems needing to position the focal plane, eliminates the discrimination interference of human factors of visual interpretation, and can accurately position the focal plane of the front lens system of the light field camera.)
1. A method for positioning a focal plane of a front lens of a light field camera is characterized by comprising the following steps:
constructing an imaging device, and determining a reference plane vertical to an emergent beam of the auto-collimation theodolite;
building a measuring device, determining a light beam vertical to the reference plane, and collecting light field images under different nominal object distances; and
and obtaining the distance between the centroids of the adjacent microlenses according to the light field images under different nominal object distances, and positioning the focal plane position of the front lens system according to the centroid distance index.
2. The method of claim 1, wherein the distance is calculated as the transverse centroid distance:
wherein, Ih,-1And Ih,1In a line of central microlenses IhS is the row number of the pixel, and t is the column number of the pixel.
3. The method of claim 2, wherein the longitudinal centroid distance of said distances is calculated by the formula:
wherein, Iz,-1And Iz,1Is a row of central micro-lenses IzRight and left adjacent columns of microlenses.
4. The method of claim 3, wherein deriving the distance between the centroids of adjacent microlenses from the light field images at the different nominal object distances further comprises:
respectively carrying out statistical analysis on the transverse centroid distance and the longitudinal centroid distance, wherein data of 1 edge microlens at each of two ends of the transverse or longitudinal cross hair and left and right or up and down close to the center of the cross hair are removed
5. The method of claim 1, wherein the centroid distance measure is a centroid distance between two adjacent microlenses of the center microlens
6. A light field camera front mirror focal plane positioning device, comprising:
the first building module is used for building an imaging device and determining a reference plane vertical to an emergent beam of the auto-collimation theodolite;
the second building module is used for building the measuring device, determining a light beam vertical to the reference plane and collecting light field images under different nominal object distances; and
and the positioning module is used for obtaining the distance between the centroids of the adjacent microlenses according to the light field images under different nominal object distances and positioning the position of the focal plane of the front lens system according to the centroid distance index.
7. The apparatus of claim 6, wherein the distance is calculated as the transverse centroid distance:
wherein, Ih,-1And Ih,1In a line of central microlenses IhS is the row number of the pixel, and t is the column number of the pixel.
8. The apparatus of claim 7, wherein the longitudinal centroid distance of said distances is calculated by:
wherein, Iz,-1And Iz,1Is a row of central micro-lenses IzRight and left adjacent columns of microlenses.
9. The apparatus of claim 8, wherein the positioning module is further configured to perform a statistical analysis on the transverse centroid distance and the longitudinal centroid distance, respectively, wherein the data of 1 edge microlens at each end of the transverse or longitudinal cross hair is removed, and the left and right or up and down directions near the center of the cross hair are removed
10. The apparatus of claim 6, wherein the centroid distance measure is a centroid distance between two adjacent microlenses of the center microlens
Technical Field
The invention relates to the technical field of assembling and debugging of light field cameras, in particular to a method and a device for positioning a focal plane of a front lens of a light field camera.
Background
An optical imaging system of a light field camera is mainly composed of three parts: the front mirror system, the micro lens array and the detector surface are shown in the simplified structure of the light field camera shown in fig. 1.
Ng.ng invented the first hand-held light field camera in 2005, and proposed a light field camera 1.0 structure, namely, a microlens array was located at the image plane position of a front mirror system, and a detector plane was located at the focal length of one time of the microlens array. The imaging principle is that a point on a target with a limited distance is focused on a micro-lens array plane after being imaged by a front lens system, and then light rays with different intensities and directions are subjected to secondary imaging by a single micro-lens and reach a detector pixel corresponding to the micro-lens. The light field imaging technology provides more abundant information for people to know the real world, namely the spatial position information and the angle information of a target can be captured through one-time exposure, and the light field imaging technology has important application in the aspects of three-dimensional reconstruction of the target and the like.
Based on the structure of the light field camera 1.0, when imaging an infinite target, the image plane position of the front mirror system is exactly the focal plane position, that is, the micro lens array should be exactly located at the focal plane position of the front mirror system, which is called in focus. In this case it can be ensured that the energy reaching each picture element on the detector will only come from one microlens, with no information aliasing between the microlenses. However, due to the influence of factors such as machining errors and mounting errors of the apparatus, it is not possible to ensure that the microlens array mounted according to the design value is always coincident with the focal plane position of the front mirror system. When the two are not coincident, that is, out-of-focus, energy may overflow into the detector pixel corresponding to the adjacent microlens, causing aliasing of information, as shown in fig. 2. Therefore, the accurate positioning of the focal plane position of the front mirror system has important influence on the imaging effect and the recovery of the three-dimensional information of the target.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, an object of the present invention is to provide a method for positioning a focal plane of a front mirror of a light field camera, which can accurately position the focal plane of the front mirror system of the light field camera.
Another objective of the present invention is to provide a device for positioning the focal plane of the front mirror of the light field camera.
In order to achieve the above object, an embodiment of the present invention provides a method for positioning a focal plane of a front mirror of a light field camera, including the following steps: constructing an imaging device, and determining a reference plane vertical to an emergent beam of the auto-collimation theodolite; building a measuring device, determining a light beam vertical to the reference plane, and collecting light field images under different nominal object distances; obtaining the distance between the centroids of the adjacent microlenses according to the light field images under different nominal object distances, and positioning the focal plane position of the front lens system according to the centroid distance index
The method for positioning the focal plane of the front lens of the light field camera can flexibly control the direction of incident light entering the light field camera, is suitable for other optical imaging systems needing to position the focal plane, and eliminates the discrimination interference of human factors of visual interpretation, thereby accurately positioning the focal plane of the front lens system of the light field camera.
In addition, the light field camera front mirror focal plane positioning method according to the above embodiment of the present invention may further have the following additional technical features:
further, in one embodiment of the present invention, the calculation formula of the transverse centroid distance of the distance is:
wherein, Ih,-1And Ih,1In a line of central microlenses IhS is the row number of the pixel, and t is the column number of the pixel.
Further, in one embodiment of the present invention, the calculation formula of the longitudinal centroid distance of the distance is:
wherein, Iz,-1And Iz,1Is a row of central micro-lenses IzRight and left adjacent columns of microlenses.
Further, in an embodiment of the present invention, the obtaining the distance between the centroids of the adjacent microlenses according to the light field images at the different nominal object distances further includes: respectively carrying out statistical analysis on the transverse centroid distance and the longitudinal centroid distance, wherein data of 1 edge microlens at each of two ends of the transverse or longitudinal cross hair and left and right or up and down close to the center of the cross hair are removed
Further, in one embodiment of the present invention, the centroid distance index is a centroid distance between two adjacent microlenses of the center microlens
In order to achieve the above object, an embodiment of another aspect of the present invention provides a device for positioning a focal plane of a front mirror of a light field camera, including: the first building module is used for building an imaging device and determining a reference plane vertical to an emergent beam of the auto-collimation theodolite; the second building module is used for building the measuring device, determining a light beam vertical to the reference plane and collecting light field images under different nominal object distances; and the positioning module is used for obtaining the distance between the centroids of the adjacent microlenses according to the light field images under different nominal object distances and positioning the position of the focal plane of the front lens system according to the centroid distance index.
The light field camera front mirror focal plane positioning device can flexibly control the direction of incident light entering the light field camera, is suitable for other optical imaging systems needing to position the focal plane, eliminates discrimination interference of human factors of visual interpretation, and can accurately position the focal plane of the light field camera front mirror system.
In addition, the light field camera front mirror focal plane positioning device according to the above embodiment of the present invention may further have the following additional technical features:
further, in one embodiment of the present invention, the calculation formula of the transverse centroid distance of the distance is:
wherein, Ih,-1And Ih,1In a line of central microlenses IhS is the row number of the pixel, and t is the column number of the pixel.
Further, in one embodiment of the present invention, the calculation formula of the longitudinal centroid distance of the distance is:
wherein, Iz,-1And Iz,1Is a row of central micro-lenses IzRight and left adjacent columns of microlenses.
Further, in an embodiment of the present invention, the positioning module is further configured to perform statistical analysis on the transverse centroid distance and the longitudinal centroid distance, respectively, wherein data of 1 edge microlens at each of two ends of the transverse or longitudinal cross hair and left and right or up and down near the center of the cross hair are removedData of each microlens.
Further, in one embodiment of the present invention, the centroid distance index is a centroid distance between two adjacent microlenses of the center microlensAnd judging the position of the focal plane of the positioning front lens system.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a simplified diagram of a related art light field camera configuration;
FIG. 2 is a schematic in-focus and out-of-focus view of a related art microlens array;
FIG. 3 is a flow chart of a light field camera front mirror focal plane positioning method according to an embodiment of the present invention;
FIG. 4 is a flow chart of a light field camera front mirror focal plane positioning method according to one embodiment of the present invention;
FIG. 5 is a schematic diagram of a commissioning imaging apparatus according to an embodiment of the present invention;
FIG. 6 is a schematic view of a measuring device according to an embodiment of the present invention;
FIG. 7 is a schematic view of a light field camera detector measurement according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a centroid distance index according to an embodiment of the invention;
FIG. 9 is a schematic diagram of the mean centroid distances at different nominal object distances for a longitudinal cross hair in accordance with an embodiment of the present invention;
fig. 10 is a schematic structural diagram of a light field camera front mirror focal plane positioning device according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
As is apparent from the background art, the present invention solves the following problems: how to accurately position the focal plane of a light field camera front mirror system. Therefore, the embodiment of the invention provides a variable-focus mounting and debugging device and a method for positioning a front lens focal plane of a light field camera based on a centroid distance index.
The following describes a method and an apparatus for positioning a focal plane of a front mirror of a light field camera according to an embodiment of the present invention with reference to the drawings, and first, a method for positioning a focal plane of a front mirror of a light field camera according to an embodiment of the present invention will be described with reference to the drawings.
Fig. 3 is a flow chart of a light field camera front mirror focal plane positioning method according to an embodiment of the invention.
As shown in fig. 3, the method for positioning the focal plane of the front mirror of the light field camera comprises the following steps:
in step S301, an imaging device is set up and a reference plane perpendicular to the autocollimation theodolite exit beam is determined.
It can be understood that, as shown in fig. 4, the embodiment of the present invention first sets up an imaging device, and debugs to obtain a reference plane perpendicular to the outgoing beam of the autocollimation theodolite.
Specifically, according to the experimental apparatus shown in fig. 5, a horizontal turntable is placed on a horizontal platform, a multidimensional adjustment table is placed on the horizontal turntable, and the surface of the multidimensional adjustment table is adjusted to be level with the ground. And combining and installing a front lens system and a light field detection unit formed by coupling a micro lens array and a detector surface to form a light field camera, and placing the light field camera on a multi-dimensional adjusting table. The front surface of the lens of the light field camera is tightly attached to the flat crystal, and the flat crystal is considered to be completely parallel to the lens plane of the light field camera, so that the flat crystal surface can be used as a reference plane for debugging the light field camera. At the opposite side of horizontal platform, place the auto-collimation theodolite, adjust the height of auto-collimation theodolite, the cross laser beam that makes the auto-collimation theodolite send can be received by the plano in front of the light field camera, and at this moment, adjustment auto-collimation theodolite base knob and geodetic level, digital display y axle on the auto-collimation theodolite is 270. And rotating the horizontal rotating table around the Y axis to perform coarse adjustment, so that the autocollimation theodolite can receive the image of the crossed cross hair reflected by the flat crystal, and performing fine adjustment by using the multidimensional adjusting table to ensure that the reflected image of the crossed cross hair is exactly positioned in the center of the autocollimation theodolite engraving plate. Adjusting the horizontal rotation of the autocollimation theodolite, namely changing the value of a digital display x axis on the autocollimation theodolite, if the vertical direction position of the cross hair on the carving board is observed to be kept unchanged, and only the cross hair moves horizontally, the debugging process is considered to be finished, otherwise, the multidimensional adjusting table is repeatedly finely adjusted until the reflected image of the cross hair only moves horizontally along with the change of the value of the x axis of the autocollimation theodolite.
At this time, the adjustment of the light field camera imaging device is completed, and a reference plane perpendicular to the laser beam emitted from the autocollimation theodolite is obtained.
In step S302, a measuring device is set up, and light beams perpendicular to the reference plane are determined, and light field images at different nominal object distances are collected.
It can be understood that, as shown in fig. 4, a measuring device is set up, a light beam perpendicular to a reference plane is obtained through debugging, a measurement experiment is carried out, and light field images under different nominal object distances are obtained.
Specifically, according to the measuring device shown in fig. 6, the autocollimation theodolite and the flat crystal are removed, and the front leg and the rear leg of the zoom collimator are respectively placed on two lifting tables at a certain distance from the light field camera on the horizontal platform. And installing the cross-shaped cross-hair at the position of the adjustable component of the collimator. The width of the cross hair selected by the experiment meets the requirement
Adjusting the adjustable component of the zoom collimator to change the nominal object distance L when the nominal object distance is LrAnd at the moment, the light field images collected by the light field camera detection unit are observed, the relative positions of the two lifting tables along the Y axis and the X axis are roughly adjusted, so that the light beam emitted by the zooming collimator can completely cover the lens of the light field camera, and the cross hairs are imaged near the central position of the detector. And finely adjusting the two lifting tables to enable the cross hairs to be mainly imaged under a row of M microlenses and a column of M microlenses, wherein the two lifting tables are symmetrical up and down and left and right, each microlens is imaged to cover N multiplied by N pixels on the detector surface, and a light field camera detector measurement schematic diagram is shown in fig. 7. At the moment, the horizontal rotating platform rotates around the Y axis, and the image of the cross hair is observed to move along the horizontal direction all the time, so that the principal ray of the zooming collimator is considered to be vertical to the reference plane, and the whole measuring device is built.
And adjusting the rotation of the horizontal rotary table around the Y-axis direction to return the image of the cross hair to the central position of the detector. Carrying out measurement experiment, adjusting the adjustable component of the zoom collimator, and marking the initial nominal object distance as L1Collecting image I of cross hair1. Repeated multiple changes of nominal object distance LiCollecting image I of cross hairi. In this experiment, K sets of nominal object distances and corresponding acquired light field images, i.e., i ═ 1,2, …, K, were measured and recorded.
In step S303, the distance between the centroids of the adjacent microlenses is obtained according to the light field images at different nominal object distances, and the focal plane position of the front mirror system is positioned according to the centroid distance index.
It can be understood that, as shown in fig. 4, the distance between the centroids of adjacent microlenses is calculated and statistically analyzed, and then the focal plane position of the front mirror system is located according to the proposed centroid distance index.
Further, in an embodiment of the present invention, obtaining the distance between the centroids of the adjacent microlenses from the light field images at different nominal object distances further includes: for transverse centroid distance and longitudinal mass respectivelyPerforming statistical analysis on the center distance, wherein data of 1 edge microlens at each end of the cross-hair in the transverse or longitudinal direction and data of left and right or up and down near the center of the cross-hair are removedData of individual microlenses; the centroid distance index is a centroid distance between two adjacent microlenses of the central microlens
Specifically, step S303 further includes two steps, specifically as follows:
the method comprises the following steps: as shown in fig. 7, the rightmost display gray scale inversion and the segmented binarization are performed, black represents the strongest energy, white represents the weakest energy, and gray represents the energy transition, so that the energy of the collected light field image is more intuitively reflected to be mainly concentrated on a row and a column of microlenses, the microlens with the most concentrated energy is called as a central microlens, and a row of central microlenses I is called as a central microlenshThe upper and lower adjacent rows of microlenses are adjacent microlenses Ih,-1And Ih,1Balance a row of central microlenses IzThe left and right adjacent columns of microlenses are adjacent microlenses Iz,-1And Iz,1。
Image segmentation is realized by using image processing software (such as matlab) to obtain a series of transverse and longitudinal central microlenses and adjacent microlenses, and the values of h and z are
The centroid of an image, which is understood to refer to the gray value of each point in the image as the mass at that point, is the energy center of the image obtained by calculating the energy distribution of the entire image.
Calculating the mass centers of the upper and lower adjacent microlenses of the transverse cross-hair, wherein the column numbers should be consistent theoretically, therefore, only the line numbers are subjected to difference calculation to obtain the distance c between the transverse mass centersh：
Calculating the centroids of the left and right adjacent microlenses of the longitudinal cross-hair, wherein the row numbers of the microlenses should be consistent theoretically, so that only the column numbers are subjected to difference calculation to obtain the longitudinal centroid distance cz。
Respectively carrying out statistical analysis on the transverse and longitudinal mass center distances, removing the data of 1 edge microlens at each of two ends of the transverse (longitudinal) cross hair and the left and right (up and down) near the cross hair center in order to eliminate the influence of energy aliasing of the cross hair center in the transverse and longitudinal directionsData of each microlens.
Statistical analysis of the mean of the transverse centroid distances
Step two: repeating the step one to obtain different nominal object distances LiObtaining a mean of lateral centroid distances for an image
The center-of-mass distance index of the focal plane of the positioning front lens system provided by the embodiment of the invention is that the center-of-mass distance between two adjacent microlenses of the central microlens is
Thus, if i is equal to k, the index
The same is true in the longitudinal direction. Theoretically, the positions of the focal planes positioned transversely and longitudinally should be identical, but the final result may be slightly different due to slight differences in the transverse and longitudinal widths of the crosshairs, and the direction with the smaller standard deviation is taken as the final judgment standard.
FIG. 9 shows the statistical results of the mean centroid distances for longitudinal crosses at different nominal object distances. Wherein, the abscissa is the nominal object distance, and the ordinate is the mean value of the centroid distance.
In summary, the method for positioning the focal plane of the front mirror of the light field camera provided by the embodiment of the invention can flexibly control the direction of incident light entering the light field camera, is suitable for other optical imaging systems needing to position the focal plane, and eliminates discrimination interference of human factors of visual interpretation, thereby accurately positioning the focal plane of the front mirror system of the light field camera.
Next, a light field camera front mirror focal plane positioning apparatus proposed according to an embodiment of the present invention is described with reference to the drawings.
Fig. 10 is a schematic structural diagram of a light field camera front mirror focal plane positioning device according to an embodiment of the present invention.
As shown in fig. 10, the light field camera front mirror focal
The
Further, in one embodiment of the present invention, the calculation formula of the lateral centroid distance of the distance is:
wherein, Ih,-1And Ih,1In a line of central microlenses IhS is the row number of the pixel, and t is the column number of the pixel.
Further, in one embodiment of the present invention, the calculation formula of the longitudinal centroid distance of the distance is:
wherein, Iz,-1And Iz,1Is a row of central micro-lenses IzRight and left adjacent columns of microlenses.
Further, in an embodiment of the present invention, the
Further, in one embodiment of the present invention, the centroid distance index is a centroid distance between two adjacent microlenses of the center microlens
It should be noted that the explanation of the foregoing embodiment of the method for positioning the focal plane of the front mirror of the light field camera is also applicable to the device for positioning the focal plane of the front mirror of the light field camera in this embodiment, and details are not repeated here.
The light field camera front mirror focal plane positioning device provided by the embodiment of the invention can flexibly control the direction of incident light entering the light field camera, is suitable for other optical imaging systems needing to position the focal plane, and eliminates the discrimination interference of human factors of visual interpretation, thereby accurately positioning the focal plane of the light field camera front mirror system.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the present invention, unless otherwise expressly stated or limited, the first feature "on" or "under" the second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.