阅读说明：本技术 摄像机装置、复眼摄像装置和图像处理方法以及程序和记录介质 (Camera device, compound-eye imaging device, image processing method, program, and recording medium ) 是由 栗原康平 丰田善隆 铃木大祐 于 2017-06-26 设计创作，主要内容包括：使视场彼此相同的图像成像于高分辨率摄像区域(15a)和多个低分辨率摄像区域(15b、15c、…)。在摄像区域间设置光学滤波器(13a、13b、…),以得到表示不同种类的信息的图像。能够取得多个彼此不同种类的信息,而且能够在高分辨率摄像区域(15a)中取得优先级高的信息。还能够使用在高分辨率摄像区域中取得的图像的高分辨率成分,使在低分辨率摄像区域(15b、15c、…)中取得的图像高分辨率化(30)。(Images having the same field of view are formed in a high-resolution imaging region (15a) and a plurality of low-resolution imaging regions (15b, 15c, …). Optical filters (13a, 13b, …) are provided between the imaging regions to obtain images representing different types of information. A plurality of different types of information can be acquired, and high-priority information can be acquired in the high-resolution imaging region (15 a). The image acquired in the low-resolution imaging regions (15b, 15c, …) can also be made higher in resolution (30) using the high-resolution component of the image acquired in the high-resolution imaging region.)

1. A camera device, the camera device having:

a plurality of imaging regions;

a plurality of lenses that form images having the same field of view on the plurality of imaging regions; and

a plurality of optical filters for a plurality of optical signals,

the plurality of imaging regions include at least one 1 st type imaging region and a plurality of 2 nd type imaging regions having an area smaller than the 1 st type imaging region and having fewer pixels than the 1 st type imaging region,

the plurality of optical filters are provided so that the image acquired in each of the 2 nd type imaging regions represents information of a different kind from the image acquired in each of the 1 st type imaging regions.

2. The camera device of claim 1,

the plurality of optical filters include optical filters having different optical characteristics provided for one or more of the 1 st image pickup region and the 2 nd image pickup region.

3. The camera device of claim 1 or 2,

the camera device also has an image pick-up element,

the 1 st image pickup region and the 2 nd image pickup region are formed by dividing an image pickup surface of the one image pickup element,

the plurality of lenses include lenses included in a lens array provided for the image pickup surface.

4. The camera device of claim 3,

the camera device further has a partition wall provided so that light from a lens other than the corresponding lens is not incident into each of the plurality of image pickup regions.

5. The camera device of claim 1 or 2,

the camera device also has a 1 st image pickup element and a 2 nd image pickup element,

the at least one type 1 imaging region is constituted by a single imaging region constituted by the entire imaging surface of the type 1 imaging element,

the plurality of 2 nd type imaging regions are formed by dividing an imaging surface of the 2 nd imaging element,

the plurality of lenses include a lens provided for the image pickup surface of the 1 st image pickup element and a lens included in a lens array provided for the image pickup surface of the 2 nd image pickup element.

6. The camera device of claim 5,

the camera device further has a partition wall provided so that light from a lens other than the corresponding lens is not incident into each of the plurality of imaging regions of the 2 nd imaging element.

7. The camera device according to any one of claims 1 to 6,

the image indicating the different kinds of information includes at least one of an image based on light of a specific wavelength band, an image based on light of a specific polarization direction, and an image based on image pickup at a specific exposure amount.

8. The camera device according to any one of claims 1 to 7,

the plurality of optical filters include at least one of a spectral filter, a polarization filter, and an ND filter.

9. The camera device according to any one of claims 1 to 8,

the plurality of optical filters include an optical filter provided in the type 1 imaging region to reduce the amount of transmitted light.

10. The camera device according to any one of claims 1 to 8,

the plurality of optical filters include a G transmission filter, an infrared cut filter, or a complementary color optical filter provided in the type 1 imaging region.

11. The camera device according to any one of claims 1 to 8,

an optical filter for acquiring an image representing the different type of information is not provided in the type 1 imaging region.

12. A compound-eye imaging device includes:

the camera device of claim 1 or 2; and

a processor having at least one high resolution section,

the at least one high resolution unit receives an image acquired in any of the 1 st type imaging regions as a reference image, receives an image acquired in any of the 2 nd type imaging regions as a low resolution image, and generates a high resolution image by performing high resolution processing on the low resolution image using a high resolution component included in the reference image.

13. The compound-eye imaging apparatus according to claim 12,

the high resolution part includes:

a 1 st filter processing unit that extracts a low-frequency component and a high-frequency component of the low-resolution image from the low-resolution image;

a 2 nd filter processing unit that extracts a low frequency component and a high frequency component of the reference image from the reference image;

a low-frequency component synthesis unit that amplifies a low-frequency component of the low-resolution image to a resolution equal to that of the reference image, and synthesizes the amplified low-frequency component and the low-frequency component of the reference image by weighted summation to generate a synthesized low-frequency component;

a high-frequency component synthesis unit that amplifies a high-frequency component of the low-resolution image to the same resolution as the reference image, and synthesizes the amplified high-frequency and low-frequency components with the high-frequency component of the reference image by weighted summation to generate a synthesized high-frequency component; and

and a component synthesizing unit that synthesizes the synthesized low-frequency component and the synthesized high-frequency component to generate the high-resolution image.

14. The compound-eye imaging apparatus according to claim 12,

the high resolution part includes:

a reduction processing unit that reduces the reference image and generates a reduced reference image having the same resolution as the low-resolution image;

a coefficient calculation unit that calculates a linear coefficient that approximates a linear relationship between the reduced reference image and the low-resolution image;

a coefficient map enlarging unit that enlarges a coefficient map composed of the linear coefficients calculated by the coefficient calculating unit to the same resolution as the reference image; and

and a linear conversion unit that generates the high-resolution image having information represented by the low-resolution image, based on a linear coefficient of the enlarged coefficient map and the reference image.

15. The compound-eye imaging apparatus according to claim 12, 13, or 14,

the at least one high resolution section is composed of a plurality of high resolution sections,

the processor further includes a synthesizing unit that synthesizes the plurality of high-resolution images generated by the plurality of high-resolution image generation units to generate one or more synthesized high-resolution images.

16. The compound-eye imaging apparatus according to claim 15,

since the plurality of low-resolution images that have been resolution-enhanced by the plurality of high-resolution enhancing units differ from each other in the type or value of at least one of the parameters indicating the imaging conditions, the plurality of high-resolution images generated by the plurality of high-resolution enhancing units differ from each other in the type or value of at least one of the parameters indicating the imaging conditions,

the synthesizing unit generates a high-resolution image in which the type or value of the at least one parameter is different from the plurality of high-resolution images by interpolation from the plurality of high-resolution images.

17. A compound-eye imaging device includes:

the camera device of claim 1 or 2; and

a processor that processes an image output from the camera device,

the processor has:

a combining unit that combines the plurality of images acquired in the plurality of 2 nd types of imaging regions to generate at least one combined low-resolution image; and

and a resolution increasing unit that receives an image acquired in any of the 1 st type imaging regions as a reference image, increases the resolution of the at least one synthesized low-resolution image using a high-resolution component included in the reference image, and generates at least one high-resolution synthesized image.

18. A program for causing a computer to execute the processing in the compound-eye imaging apparatus according to any one of claims 12 to 17.

19. A computer-readable recording medium having recorded thereon the program according to claim 18.

20. An image processing method having the steps of:

acquiring a plurality of images having different kinds of information and different resolutions, the plurality of images being obtained by imaging in a plurality of imaging regions in which images having the same field of view are imaged;

a resolution increasing step of performing resolution increasing processing on a plurality of images having relatively low resolutions by using a high resolution component included in an image having a relatively high resolution among the plurality of images having different resolutions, and generating high resolution images having different kinds of information; and

and a synthesizing step of synthesizing the plurality of high-resolution images having different types of information to generate one or more synthesized high-resolution images.

Technical Field

The invention relates to a camera device, a compound eye imaging device and an image processing method.

The present invention particularly relates to a camera device that forms images of the same field of view into a plurality of imaging regions and acquires a plurality of images representing different types of information in the plurality of imaging regions. The present invention also relates to a compound eye imaging apparatus having a processor for performing high resolution on a plurality of images acquired by the camera apparatus. The present invention also relates to an image processing method implemented by the compound-eye imaging apparatus.

The present invention also relates to a program for causing a computer to execute processing in the compound-eye imaging apparatus or the image processing method, and a recording medium having the program recorded thereon.

Background

In recent years, there has been a growing demand for imaging devices, and for example, it is desired to acquire not only RGB visible images but also additional information. In particular, near-infrared light is suitable for monitoring, object recognition, and the like because of its feature of high transmittance to atmospheric light and being invisible, and is attracting attention in the fields of monitoring cameras, vehicle-mounted cameras, and the like. Further, an image obtained only from light of a specific polarization direction is useful for removing reflected light on a window glass, a road surface, or the like and recognizing an object that is hard to see, such as a black object or a transparent object, and is attracting attention in the field of an inspection camera used for an in-vehicle camera or an FA (factory automation).

Such heterogeneous information has been generally obtained as a substitute for a normal color image. As the simplest method for simultaneously acquiring a color image and different kinds of information, there is a camera array in which a plurality of cameras are arranged, but there are problems that it is necessary to accurately perform camera positioning, the apparatus is large in size, and the installation cost and maintenance cost are increased.

In recent years, there has been an RGB-X sensor that can simultaneously acquire different types of information by providing a filter that transmits only near infrared light in a color filter array. However, such a sensor requires much cost and time for design and development, and also has many problems in terms of manufacturing.

As a small-sized device for solving these problems, a device has been proposed in which an image pickup device is divided into a plurality of image pickup regions, and images are formed in the plurality of image pickup regions (patent document 1). In this apparatus, different types of information can be acquired simultaneously by providing different optical filters for a plurality of imaging regions. The device of patent document 1 can be manufactured by disposing a lens array on one image pickup element, and has an advantage of being easy to be miniaturized.

Disclosure of Invention

Problems to be solved by the invention

In the apparatus of patent document 1, a plurality of images can be simultaneously acquired from one image pickup device by dividing the image pickup device, but there are problems as follows: the number of pixels per imaging area, i.e., the resolution, decreases as the imaging area increases.

The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a compound-eye camera capable of acquiring a plurality of types of information different from each other and acquiring information with high priority at high resolution.

Means for solving the problems

The camera device of the present invention includes:

a plurality of imaging regions;

a plurality of lenses that form images having the same field of view on the plurality of imaging regions;

and a plurality of optical filters for filtering the light beam,

the plurality of imaging regions include at least one 1 st type imaging region and a plurality of 2 nd type imaging regions having an area smaller than the 1 st type imaging region and having fewer pixels than the 1 st type imaging region,

the plurality of optical filters are provided so that the image acquired in each of the 2 nd type imaging regions represents information of a different kind from the image acquired in each of the 1 st type imaging regions.

The compound-eye imaging device of the present invention includes the above-described camera device and a processor including at least one high-resolution section,

the at least one high resolution unit receives an image acquired in any of the 1 st type imaging regions as a reference image, receives an image acquired in any of the 2 nd type imaging regions as a low resolution image, and generates a high resolution image by performing high resolution processing on the low resolution image using a high resolution component included in the reference image.

Effects of the invention

According to the camera device of the present invention, since the camera device has a plurality of imaging regions, it is possible to acquire images representing a plurality of types of information different from each other. Further, since the plurality of imaging regions include at least one 1 st type imaging region and a plurality of 2 nd type imaging regions having an area smaller than the 1 st type imaging region and having fewer pixels than the 1 st type imaging region, by assigning a relatively large imaging region to an image whose priority is higher among a plurality of types of information different from each other, that is, an image whose high resolution is more strongly desired to be acquired, such an image can be acquired with high resolution.

According to the compound eye imaging device of the present invention, since the resolution of the low-resolution image is increased by the resolution increasing section, a high-resolution image can be obtained even if the imaging area is relatively small.

Drawings

Fig. 1 (a) is an exploded perspective view showing the configuration of a compound eye camera constituting the camera device according to embodiment 1 of the present invention, and fig. 1 (b) is a view showing the size of an image acquired by the compound eye camera.

Fig. 2 (a) to (d) are schematic diagrams showing different examples of methods for dividing the imaging surface of the imaging element of the compound-eye camera shown in fig. 1 (a).

Fig. 3 (e) and (f) are schematic diagrams illustrating different examples of a method of dividing the imaging surface of the image pickup device of the compound-eye camera of fig. 1 (a).

Fig. 4 is a block diagram showing a compound-eye imaging device according to embodiment 2 of the present invention.

Fig. 5 is a block diagram showing an example of a resolution increasing section used in embodiment 2.

Fig. 6 is a block diagram showing another example of the high resolution section used in embodiment 2.

Fig. 7 is a block diagram showing another example of a processor used in embodiment 2.

Fig. 8 is a block diagram showing another example of a processor used in embodiment 2.

Fig. 9 is a block diagram showing an example of a processor used in embodiment 3 of the present invention.

Fig. 10 is a block diagram showing an example of a combining unit used in embodiment 3.

Fig. 11 is a block diagram showing another example of a processor used in embodiment 3.

Fig. 12 is a block diagram showing a processor used in embodiment 4 of the present invention.

Fig. 13 (a) and (b) are diagrams showing an example of interpolation of image information by the processor of embodiment 4.

Fig. 14 is a block diagram showing a processor used in embodiment 5 of the present invention.

Fig. 15 is a block diagram showing a compound-eye imaging device according to embodiment 6 of the present invention

Fig. 16 is an exploded perspective view showing the configuration of a compound-eye camera and a monocular camera according to embodiment 6.

Fig. 17 (a) and (b) are schematic diagrams illustrating an example of a method of dividing the imaging surface of the imaging element of the compound-eye camera of fig. 16.

Fig. 18 is a flowchart showing a processing procedure in the image processing method of embodiment 7 of the present invention.

Detailed Description