阅读说明：本技术 摄像装置及移动摄像装置 (Image pickup apparatus and mobile image pickup apparatus ) 是由 小野修司 于 2018-06-01 设计创作，主要内容包括：本发明提供一种能够对涉及调焦的结构进行小型化及轻量化的摄像装置及移动摄像装置。在具备利用泛焦进行拍摄的中央光学系统(12)、与中央光学系统(12)同心状地配置的环状光学系统(14)及同时拍摄通过中央光学系统(12)成像的像及通过环状光学系统(14)成像的像的图像传感器(20)的摄像装置(1)中,使中央光学系统(12)及图像传感器(20)沿光轴(L)一体地移动而调节环状光学系统(14)的焦点。(The invention provides an image pickup apparatus and a mobile image pickup apparatus capable of reducing the size and weight of a structure related to focusing. In an imaging device (1) provided with a central optical system (12) for imaging by means of a wide focus, an annular optical system (14) arranged concentrically with the central optical system (12), and an image sensor (20) for simultaneously imaging an image formed by the central optical system (12) and an image formed by the annular optical system (14), the central optical system (12) and the image sensor (20) are moved integrally along an optical axis (L) to adjust the focus of the annular optical system (14).)

1. An imaging device includes:

a central optical system configured to perform imaging with a focus spread;

an annular optical system disposed concentrically with the central optical system;

an image sensor that regularly arranges pixels that selectively receive light that has passed through the central optical system and pixels that selectively receive light that has passed through the annular optical system on the same plane, and that simultaneously captures an image that is imaged by the central optical system and an image that is imaged by the annular optical system; and

and an annular optical system focusing mechanism that adjusts a focus of the annular optical system by integrally moving the central optical system and the image sensor along an optical axis with respect to the annular optical system.

2. The image pickup apparatus according to claim 1,

the ring optical system has a longer focal length than the central optical system.

3. The image pickup apparatus according to claim 2,

the central optical system is constituted by a wide-angle optical system,

the ring-shaped optical system is constituted by a telephoto optical system.

4. The image pickup apparatus according to claim 3,

the ring-shaped optical system is constituted by a catadioptric optical system.

5. The image pickup apparatus according to claim 1,

the central optical system is set to shoot a long distance,

the ring-shaped optical system is set to capture a short distance.

6. The image pickup apparatus according to any one of claims 1 to 5,

the imaging device further includes:

an image pickup control unit that causes the image sensor to capture a moving image or continuously capture a still image; and

an annular optical system focus control section that controls the annular optical system focus mechanism,

the annular optical system focus control unit periodically displaces the central optical system and the image sensor during imaging, and periodically changes the distance of an object focused by the annular optical system.

7. The image pickup apparatus according to claim 6,

the annular optical system focus control unit displaces the central optical system and the image sensor in a sinusoidal waveform.

8. The image pickup apparatus according to claim 6,

the annular optical system focus control unit displaces the central optical system and the image sensor in a sawtooth waveform.

9. The image pickup apparatus according to any one of claims 6 to 8,

the imaging device further includes:

and a ring-shaped optical system focused image extracting unit that analyzes a moving image or a still image group captured via the ring-shaped optical system for each period of displacement of the central optical system and the image sensor, and extracts an image of a frame with the highest resolution or a still image with the highest resolution for each period of displacement as a ring-shaped optical system focused image.

10. The image pickup apparatus according to claim 9,

the imaging device further includes:

the central optical system focus image extracting unit extracts, as a central optical system focus image, an image of a frame or a still image captured at the same timing as the annular optical system focus image from among the moving image or the still image group captured via the central optical system.

11. A mobile imaging device is provided with:

the image pickup device according to any one of claims 1 to 10; and

and a moving body on which the imaging device is mounted.

Technical Field

The present invention relates to an imaging device and a mobile imaging device, and more particularly to an imaging device and a mobile imaging device that simultaneously capture two images on a common axis using an imaging lens in which two optical systems are concentrically arranged and an image sensor having directivity.

Background

There is known an imaging apparatus that simultaneously captures two images having different imaging characteristics using imaging lenses arranged concentrically with two optical systems having different imaging characteristics. For example, patent document 1 describes an imaging device that simultaneously captures two images of a wide angle and a telephoto on a common axis using an imaging lens having a wide angle and a telephoto optical system arranged concentrically and an image sensor having directivity. Patent document 2 describes an imaging device in which wide-angle and telephoto optical systems are concentrically arranged, and two image sensors receive light from the respective optical systems, thereby simultaneously capturing two images of a wide angle and a telephoto on the same axis.

However, in an imaging apparatus substantially including two optical systems, it is necessary to perform focusing for each optical system. In patent document 1, a focusing mechanism is provided for each optical system, so that focusing can be performed for each optical system. In patent document 2, the focus of each optical system can be individually adjusted by moving each image sensor relative to each optical system. Patent document 1 proposes a technique of focusing (also referred to as a deep focus) in which a focusing mechanism is provided only for a telephoto optical system and a fixed focus is set for a wide-angle optical system.

To be technical literature

Patent document

Patent document 1: japanese patent laid-open publication No. 2016-012786

Patent document 2: japanese Kokai publication 2011-505022

Disclosure of Invention

Technical problem to be solved by the invention

However, if a focus adjustment mechanism is provided for each optical system, there is a disadvantage that the structure of the imaging apparatus is complicated and large.

On the other hand, a configuration including only the focus adjustment mechanism for the telephoto optical system can simplify the entire configuration compared to a configuration including only the focus adjustment mechanism, but has a problem in terms of downsizing and weight reduction. That is, the telephoto optical system is composed of the outer optical system of the two optical systems arranged concentrically, but the outer optical system is much heavier than the inner optical system. In this way, the focusing mechanism of the very heavy optical system also becomes very heavy. Therefore, there is a disadvantage that the imaging apparatus becomes very heavy. Further, if the operation is performed at a higher speed, a larger actuator is required, and there is a drawback that there is a limit to the increase in speed. Further, the very heavy optical system has a disadvantage that vibration is likely to occur during the focusing operation.

The present invention has been made in view of such circumstances, and an object thereof is to provide an image pickup apparatus and a mobile image pickup apparatus that can reduce the size and weight of a structure related to focusing.

Means for solving the technical problem

Means for solving the above problems are as follows.

(1) The imaging device is provided with: a central optical system configured to perform imaging with a focus spread; an annular optical system disposed concentrically with the central optical system; an image sensor that regularly arranges pixels that selectively receive light that has passed through the central optical system and pixels that selectively receive light that has passed through the annular optical system on the same plane, and that simultaneously captures an image that is imaged by the central optical system and an image that is imaged by the annular optical system; and an annular optical system focusing mechanism that adjusts the focus of the annular optical system by moving the central optical system and the image sensor integrally with the annular optical system along the optical axis.

According to this aspect, the imaging device that simultaneously captures two images on the same axis includes two optical systems arranged concentrically and an image sensor having directivity. The two optical systems are composed of an inner central optical system and an outer annular optical system. The central optical system is set to perform imaging with a wide focus. Therefore, with respect to the central optical system, focusing is not required. On the other hand, the ring optical system is focused by the ring optical system focusing mechanism. The ring optical system focusing mechanism adjusts the focus of the ring optical system by integrally moving the central optical system and the image sensor along the optical axis. Since the inner central optical system is smaller and lighter than the outer annular optical system, the structure required for movement can be made lightweight and compact. Further, the central optical system is small and lightweight, and therefore can be operated at high speed. This makes it possible to speed up the focusing operation of the ring optical system. Further, since the central optical system is small and lightweight, it can be operated without generating vibration. Further, since the optical system moves integrally with the image sensor, the focal point of the central optical system does not change. In addition, the "annular optical system" of the present invention includes an arc-shaped optical system in addition to a complete annular optical system. That is, an optical system disposed concentrically at the outer periphery of the central optical system is a ring-shaped optical system. The "concentric shape" includes a case where the optical axes are completely aligned and a case where the optical axes are substantially aligned. That is, a range regarded as substantially concentric is included.

(2) In the imaging device according to the above (1), the ring optical system has a longer focal length than the central optical system.

According to this aspect, the ring-shaped optical system is constituted by an optical system having a longer focal length than the central optical system. Thus, two images having different focal lengths can be simultaneously captured on the same axis. In particular, by configuring the central optical system with a short-focus optical system, it is possible to easily achieve a focus blur.

(3) In the imaging device according to the above (2), the central optical system is constituted by a wide-angle optical system, and the annular optical system is constituted by a telephoto optical system.

According to this aspect, the central optical system is constituted by a wide-angle optical system, and the annular optical system is constituted by a telephoto optical system. This enables simultaneous imaging of two images of wide and telephoto angles on the same axis. Here, the wide-angle optical system is an optical system that has a wider angle of view (approximately 60 ° or more) than a standard optical system (angle of view of about 50 °), and can capture an image over a wide range. On the other hand, the telephoto optical system is an optical system having a longer focal length than the standard optical system and capable of magnifying and photographing a long-distance object.

(4) In the imaging device according to the above (3), the annular optical system is constituted by a catadioptric optical system.

According to this aspect, the ring-shaped optical system is constituted by a catadioptric optical system. A catadioptric optical system is an optical system in which a lens and a mirror are combined. By configuring the ring-shaped optical system with a catadioptric optical system, the ring-shaped optical system can be made smaller and lighter. Further, this also enables the entire structure to be reduced in size and weight.

(5) In the imaging device according to the above (1), the central optical system is set to take a long distance, and the annular optical system is set to take a short distance.

According to this aspect, the central optical system is configured by an optical system set to take a long distance, and the annular optical system is configured by an optical system set to take a short distance. Thus, two images of a short distance and a long distance can be simultaneously captured on the same axis. As for the depth of field, the distance to the subject becomes longer and deeper. Therefore, a long-distance subject can be imaged by the wide focus. On the other hand, it is difficult to capture a short-distance object with the use of the zoom. Therefore, a long-distance subject is imaged by the pan-focus central optical system, and a short-distance subject is imaged by the ring-shaped optical system having a focus adjustment function. This makes it possible to simultaneously capture images focused at both a short distance and a long distance. Here, "long distance" and "short distance" refer to a relationship defined between the central optical system and the annular optical system. That is, the central optical system is set to photograph a short-distance subject with respect to the annular optical system, and the annular optical system is set to photograph a long-distance subject with respect to the central optical system. However, the central optical system is set to a distance that can be imaged by the focus blur in terms of the relationship of imaging by the focus blur. For example, when the angle is wide to a standard field angle (approximately 50 ° or more), the central optical system is set to shoot 3m to infinity.

(6) Any one of the imaging devices (1) to (5) above further includes: an image pickup control unit for causing the image sensor to pick up a moving image or continuously pick up a still image; and an annular optical system focus control unit that controls the annular optical system focus mechanism, wherein the annular optical system focus control unit periodically displaces the central optical system and the image sensor during imaging, and periodically changes the distance of the subject focused by the annular optical system.

According to this aspect, a moving image is captured while periodically changing the distance of the subject focused by the annular optical system. Alternatively, a still image is continuously captured while periodically changing the distance of the subject focused by the ring-shaped optical system. For example, between a position focused on an Object at the Minimum Object Distance (MOD) and a position focused on an Object at infinity, the central optical system and the image sensor are periodically displaced, and the Distance of the Object focused by the annular optical system is periodically changed. An image focused by the ring-shaped optical system can be easily photographed.

(7) In the imaging device according to the above (6), the annular optical system focus control unit displaces the central optical system and the image sensor in a sinusoidal waveform.

According to this aspect, the central optical system and the image sensor are displaced in a sinusoidal manner to capture a moving image. Alternatively, the central optical system and the image sensor are displaced in a sine wave shape to continuously capture still images.

(8) In the imaging device according to the above (6), the annular optical system focus control unit displaces the central optical system and the image sensor in a sawtooth waveform.

According to this aspect, the central optical system and the image sensor are displaced in a sawtooth waveform to capture a moving image. Alternatively, the central optical system and the image sensor are displaced in a sawtooth waveform to continuously capture still images.

(9) Any one of the imaging devices (6) to (8) above further includes: the annular optical system focused image extracting unit analyzes a moving image or a still image group captured via the annular optical system for each period of displacement of the central optical system and the image sensor, and extracts an image of a frame with the highest resolution or a still image with the highest resolution for each period of displacement as an annular optical system focused image.

According to this aspect, the moving image captured through the ring optical system is analyzed for each period of displacement of the central optical system and the image sensor, and the image of the frame with the highest resolution is extracted as the ring optical system in-focus image for each period of displacement. Alternatively, the group of still images captured via the ring optical system is analyzed for each period of displacement of the central optical system and the image sensor, and the still image with the highest resolution is extracted for each period of displacement as the ring optical system in-focus image.

(10) The imaging device of the above (9) further includes: the central optical system focus image extraction unit extracts, as a central optical system focus image, an image of a frame or a still image captured at the same timing as that of the annular optical system focus image from among a moving image or a still image group captured via the central optical system.

According to this aspect, an image of a frame or a still image captured at the same timing as the ring-shaped optical system focus image is extracted as the central optical system focus image. Here, "the same timing" includes substantially the same timing.

(11) The mobile imaging device is provided with: any one of the imaging devices (1) to (10) above; and a moving body on which the imaging device is mounted.

According to this aspect, any one of the imaging devices (1) to (10) is mounted on a mobile body to constitute a mobile imaging device. The moving body can be configured by, for example, an unmanned aerial vehicle (so-called drone) or an unmanned automobile.

Effects of the invention

According to the present invention, the structure relating to focusing can be made smaller and lighter. Further, focusing can be speeded up. Further, vibration during operation can be prevented.

Drawings

Fig. 1 is a cross-sectional view showing a schematic configuration of an embodiment of an imaging apparatus to which the present invention is applied.

Fig. 2 is a cross-sectional view 2-2 of fig. 1.

Fig. 3 is a cross-sectional view of 3-3 of fig. 1.

Fig. 4 is a diagram showing a lens structure of the central optical system.

Fig. 5 is a diagram showing a lens structure of the ring optical system.

Fig. 6 is a diagram showing a moving state of the central optical system and the image sensor.

Fig. 7 is a diagram showing a schematic configuration of an image sensor.

Fig. 8 is a conceptual diagram of a structure in which each pixel of the image sensor selectively receives light from the corresponding optical system.

Fig. 9 is a block diagram showing an electrical configuration of the image pickup apparatus.

FIG. 10 is a block diagram of the main computer-implemented functions.

Fig. 11 is a flowchart showing the procedure of image pickup processing in the continuous shooting mode a.

Fig. 12 is a flowchart showing the procedure of image pickup processing in the continuous shooting mode B.

Fig. 13 is a diagram showing an example of a method for changing the distance of an object focused by the ring optical system.

Fig. 14 is a diagram showing another example of a method for changing the distance of an object focused by the ring optical system.

Fig. 15 is a diagram showing a schematic configuration of an embodiment of a mobile imaging apparatus.

Fig. 16 is a conceptual diagram of flight control of the unmanned aerial vehicle for the purpose of generating a composite image.

Fig. 17 is a table showing a relationship between the position of the image sensor and the in-focus object distance of the annular optical system in the imaging apparatus shown in fig. 1.

Fig. 18 is a graph showing a relationship between the position of the image sensor and the in-focus object distance of the ring-shaped optical system.

Fig. 19-1 and 19-2 are tables showing the relationship between the position of the image sensor and the focused object distance of the ring-shaped optical system when the image sensor is displaced in a sawtooth waveform.

Fig. 20 is a graph showing a relationship between the position of the image sensor and the focus object distance of the ring-shaped optical system when the image sensor is displaced in a sawtooth waveform.

Fig. 21-1 and 21-2 are tables showing the relationship between the position of the image sensor and the focused object distance of the ring-shaped optical system when the image sensor is displaced in a sine wave shape.

Fig. 22 is a graph showing a relationship between the position of the image sensor and the focus object distance of the ring-shaped optical system when the image sensor is displaced in a sine wave shape.

Fig. 23 is a table showing a relationship between the position of the image sensor in the image pickup apparatus shown in fig. 1 and the in-focus object distance of the annular optical system.

Fig. 24 is a graph showing a relationship between the position of the image sensor and the in-focus object distance of the ring-shaped optical system.

Fig. 25-1 and 25-2 are tables showing the relationship between the position of the image sensor and the focused object distance of the ring-shaped optical system when the image sensor is displaced in a sine wave shape.

Fig. 26 is a graph showing a relationship between the position of the image sensor and the focus object distance of the ring-shaped optical system when the image sensor is displaced in a sine wave shape.

Detailed Description

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

◆◆ image pickup device ◆◆

[ Structure of imaging device ]

Fig. 1 is a cross-sectional view showing a schematic configuration of an embodiment of an imaging apparatus to which the present invention is applied. Fig. 2 is a sectional view taken along line 2-2 of fig. 1, and fig. 3 is a sectional view taken along line 3-3 of fig. 1.

The imaging device 1 of the present embodiment is an imaging device that simultaneously captures two images of a wide angle and a telephoto on the same axis, and includes an imaging lens 10 having two optical systems of a wide angle and a telephoto, and an image sensor 20 having directivity. The coaxial line includes a range regarded as substantially coaxial and a range substantially simultaneous to the coaxial line.

Imaging lens

The imaging lens 10 includes a central optical system 12 constituting a wide-angle optical system, an annular optical system 14 constituting a telephoto optical system, and an annular optical system focusing mechanism 30 for adjusting the focus of the annular optical system 14.

< Central optical System >

Fig. 4 is a diagram showing a lens structure of the central optical system.

The central optical system 12 is an optical system having a fixed focus and is composed of eight lenses 12a to 12 h. The lenses 12a to 12h are arranged along the optical axis L. The light incident on the central optical system 12 passes through the lenses 12a to 12h and enters the image sensor 20.

The central optical system 12 is set to perform imaging with a wide focus. The central optical system 12 includes a diaphragm, not shown, in its optical path. The central optical system 12 achieves a focus expansion by setting an aperture. In other words, the central optical system 12 sets the aperture so as to be in focus.

< Ring optical System >

The ring optical system 14 is disposed concentrically with the central optical system 12. Therefore, the ring optical system 14 is an optical system having the same optical axis L as the central optical system 12, and is an optical system capable of performing imaging coaxially with the central optical system 12.

Fig. 5 is a diagram showing a lens structure of the ring optical system.

The ring optical system 14 is composed of a catadioptric optical system, and is composed of three lenses 14a, 14b, and 14c and two mirrors 14d and 14 e.

The three lenses 14a, 14b, and 14c have a ring shape and are arranged along the optical axis L. The three lenses 14a, 14b, and 14c are set as a 1 st lens 14a, a 2 nd lens 14b, and a 3 rd lens 14c in this order from the object side.

The two mirrors 14d and 14e are composed of a main mirror 14d and a sub-mirror 14 e. The main mirror 14d is provided on the surface of the 3 rd lens 14c on the image surface side. The main mirror 14d is provided on the entire surface of the image surface side of the 3 rd lens 14c by, for example, mirror-finishing the entire surface of the image surface side of the 3 rd lens 14 c. The sub-mirror 14e is provided on the surface of the 2 nd lens 14b on the image surface side. The sub-mirror 14e is formed in an annular shape on the inner portion of the surface on the image plane side of the 2 nd lens 14b by, for example, annularly mirror-finishing the inner portion of the surface on the image plane side of the 2 nd lens 14 b.

The light incident on the ring optical system 14 passes through the 1 st lens 14a, the 2 nd lens 14b, and the 3 rd lens 14c in this order, and is reflected by the main mirror 14 d. The light reflected by the main mirror 14d passes through the 3 rd lens 14c and enters the sub-mirror 14e, and is reflected by the sub-mirror 14e and enters the image sensor 20.

The ring-shaped optical system 14 includes a diaphragm, not shown, in its optical path. The diaphragm adjusts the light quantity of light passing through the ring optical system 14 by scaling its outer diameter.

Annular optical System focusing mechanism

The ring optical system focusing mechanism 30 integrally moves the central optical system 12 and the image sensor 20 along the optical axis L to adjust the focus of the ring optical system 14.

As shown in fig. 1, the ring optical system focusing mechanism 30 includes a fixed cylinder 32, a movable cylinder 34, an image sensor holder 36, a linear guide mechanism, an actuator 50, and a position detection unit 60.

< fixed Cylinder >

The fixed cylinder 32 is fixed to a main body frame, not shown, of the imaging apparatus 1. The fixed cylinder 32 holds the ring optical system 14. The lenses 14a, 14b, and 14c constituting the ring optical system 14 are positioned and attached to the inside of the fixed cylinder 32.

< moving Cylinder >

The movable cylinder 34 is disposed inside the fixed cylinder 32 and is disposed coaxially with the fixed cylinder 32. The moving barrel 34 holds the central optical system 12. The lenses 12a to 12h constituting the central optical system 12 are positioned and attached to the inside of the moving cylinder 34.

< image sensor Carrier >

The image sensor holder 36 holds the image sensor 20. The image sensor holder 36 has a disk shape, and holds the image sensor 20 at the center thereof. The image sensor holder 36 is disposed inside the fixed cylinder 32 and is disposed coaxially with the fixed cylinder 32. The image sensor 20 is held by the image sensor holder 36 and arranged on the optical axis L.

< Linear guide mechanism >

The linear guide mechanism performs linear guide so that the central optical system 12 and the image sensor 20 move integrally along the optical axis L.

The linear guide mechanism includes two linear bearings 38A and 38B and two linear shafts 40A and 40B linearly guided by the two linear bearings 38A and 38B.

Two linear bearings 38A are provided to the fixed cylinder 32. The linear bearings 38A are disposed on the outer peripheral surface of the fixed cylinder 32 and are disposed symmetrically with respect to the optical axis L. The linear bearings 38A are disposed parallel to the optical axis L.

The two linear shafts 40A, 40B are slidably supported by the corresponding linear bearings 38A. Therefore, the linear shafts 40A and 40B are arranged parallel to the optical axis L and slide parallel to the optical axis L.

The distal ends of the linear shafts 40A and 40B are connected to the movable tube support arms 42A and 42B, respectively. As shown in fig. 1 and 2, the moving cylinder support arms 42A and 42B each have a rod shape, and the base end portions thereof are fixed to the outer periphery of the distal end portion of the moving cylinder 34. The fixed cylinder 32 is provided with front slits 44A and 44B through which the movable cylinder support arms 42A and 42B are inserted. The front slits 44A and 44B are provided parallel to the optical axis L.

The rear ends of the linear shafts 40A and 40B are connected to the bracket support arms 46A and 46B, respectively. As shown in fig. 1 and 3, each of the bracket support arms 46A and 46B has a rod shape, and a base end portion thereof is fixed to the outer periphery of the image sensor bracket 36. The fixed tube 32 is provided with rear side slits 48A, 48B through which the bracket support arms 46A, 46B are inserted. The rear slits 48A, 48B are provided parallel to the optical axis L.

In the linear guide mechanism configured as described above, when the linear shafts 40A and 40B are slid, the movable tube 34 and the image sensor holder 36 are integrally moved along the optical axis L. Thereby, the central optical system 12 held by the movable cylinder 34 and the image sensor 20 held by the image sensor holder 36 are integrally moved along the optical axis L.

< actuator >

The actuator 50 is constituted by a linear motor. The linear motor includes a coil 50A constituting a movable element and a magnet 50B constituting a stationary element. The coil 50A is attached to the linear shaft 40A, and the magnet 50B is attached to the fixed cylinder 32. When a current is caused to flow through the coil 50A, a magnetic field is generated, and the coil 50A moves linearly. Thereby, the linear shaft 40A moves back and forth along the optical axis L.

When the actuator 50 is driven to move the linear shaft 40A forward and backward along the optical axis L, the movable tube 34 and the image sensor bracket 36 connected to the linear shaft 40A move forward and backward along the optical axis L integrally. Thereby, the central optical system 12 and the image sensor 20 move back and forth integrally along the optical axis L with respect to the fixed ring-shaped optical system 14.

Fig. 6 is a diagram showing a moving state of the central optical system and the image sensor. Fig. 6(a) shows a state in which the central optical system and the image sensor are moved to the image plane side, and fig. 6(B) shows a state in which the central optical system and the image sensor are moved to the object side.

As shown in fig. 6, the ring optical system focusing mechanism 30 can move the central optical system 12 and the image sensor 20 along the optical axis L with respect to the fixed ring optical system 14. This enables the focus of the ring optical system 14 to be adjusted. That is, since the image sensor 20 can be moved with respect to the fixed ring optical system 14, the focus of the ring optical system 14 can be adjusted. In the relationship with the central optical system 12, the central optical system 12 moves integrally with the image sensor 20, and therefore the state of the focal extension set by the central optical system 12 can be maintained.

The ring optical system 14 focuses on a long-distance subject by moving the central optical system 12 and the image sensor 20 to the subject side, and focuses on a short-distance subject by moving to the image plane side.

< position detection means >

The position detection unit 60 includes a photo interrupter 62 and an MR sensor (MR sensor: Magneto resistive sensor) 64. The photo interrupter 62 detects that the image sensor 20 is located at the origin with respect to the ring optical system 14. The position of the origin is set in advance. The MR sensor 64 detects the amount of displacement of the image sensor 20. The image sensor 20 is detected to be located at the origin by the photo interrupter 62, and the amount of displacement from the origin is detected by the MR sensor 64. This enables detection of the position of the image sensor 20 with respect to the origin.

Image sensor

The image sensor 20 is configured by an image sensor having directivity including a pixel that selectively receives light that has passed through the central optical system 12 and a pixel that selectively receives light that has passed through the annular optical system 14, and simultaneously captures an image imaged by the central optical system 12 and an image imaged by the annular optical system 14.

Fig. 7 is a diagram showing a schematic configuration of an image sensor.

As shown in fig. 7, the image sensor 20 includes a central optical system light-receiving pixel 22a that selectively receives light that has passed through the central optical system 12, and a ring optical system light-receiving pixel 22b that selectively receives light that has passed through the ring optical system 14. The ring-shaped optical system light receiving pixels 22b and the central optical system light receiving pixels 22a are regularly arranged on the same plane. In the example shown in fig. 7, the central optical system light receiving pixels 22a and the annular optical system light receiving pixels 22b are alternately arranged.

Fig. 8 is a conceptual diagram of a structure in which each pixel of the image sensor selectively receives light from the corresponding optical system.

As shown in fig. 8, each pixel includes a photodiode 24, a microlens 26, and a light-shielding mask 28.

The microlens 26 is disposed in front of the photodiode 24. The microlens 26 images pupil images of the ring optical system 14 and the central optical system 12 on the photodiode 24.

The light-shielding mask 28 is disposed between the microlens 26 and the photodiode 24. The light-shielding mask 28 shields a part of the light passing through the microlenses 26. The light-shielding mask 28 of the central optical system light-receiving pixel 22a has a shape that shields the light Lb that has passed through the ring optical system 14, and has a ring shape. The light-shielding mask 28 of the ring-shaped optical system light-receiving pixel 22b has a shape that shields the light La that has passed through the central optical system 12, and has a circular shape.

According to the above configuration, the central optical system light receiving pixel 22a selectively receives the light La having passed through the central optical system 12, and the ring optical system light receiving pixel 22b selectively receives the light Lb having passed through the ring optical system 14. Therefore, by acquiring the image signal of the central optical system light receiving pixel 22a, the image signal of the image ImW acquired via the central optical system 12 can be acquired, and by acquiring the image signal of the annular optical system light receiving pixel 22b, the image signal of the image ImT acquired via the annular optical system 14 can be acquired.

In the imaging lens 10 of the present embodiment, the central optical system 12 is configured by a wide-angle optical system, and the annular optical system 14 is configured by a telephoto optical system. Therefore, the image ImW captured by the central optical system 12 becomes a wide-angle image, and the image ImT captured by the ring optical system 14 becomes a telephoto image. Images of the respective optical systems are taken on the same axis. Therefore, the image ImT of the ring optical system 14 becomes an image in which the central portion of the image ImW of the central optical system 12 is enlarged, and becomes an image without parallax.

When a color image is to be obtained, the annular optical system light receiving pixel 22b and the central optical system light receiving pixel 22a are provided with color filters. The color filters are arranged in a predetermined array. For example, color filters composed of three colors of RED (R: RED), GREEN (G: Green), and BLUE (B: BLUE) are arranged in a Bayer array. Thereby, a color image can be acquired.

[ Electrical Structure of image pickup device ]

Fig. 9 is a block diagram showing an electrical configuration of the image pickup apparatus.

As shown in fig. 9, the imaging apparatus 1 includes an analog signal processing unit 110, a computer 120, a recording unit 130, an operation unit 140, and the like.

Section for analog Signal processing

The analog signal processing unit 110 reads an analog image signal for each pixel output from the image sensor 20 and performs predetermined signal processing (for example, correlated double sampling processing, amplification processing, and the like). The Analog signal processing unit 110 includes an AD Converter (Analog to Digital Converter), and converts an Analog image signal after the predetermined signal processing is completed into a Digital image signal and outputs the Digital image signal.

Computer(s)

The computer 120 includes a CPU (Central Processing Unit), a RAM (random access Memory), and a ROM (Read Only Memory), executes a predetermined program, and executes various processes.

FIG. 10 is a block diagram of the main computer-implemented functions.

The computer 120 functions as a digital signal processing unit 120a, an imaging control unit 120b, an annular optical system focus control unit 120c, a position detection unit 120d, an image output control unit 120e, a recording control unit 120f, and the like by executing predetermined programs.

< digital Signal processing section >

The digital signal processing unit 120a reads the digital image signal output from the analog signal processing unit 110, and performs predetermined signal processing (for example, gradation conversion processing, white balance correction processing, gamma correction processing, synchronization processing, YC conversion processing, and the like) to generate image data. At this time, the digital signal processing unit 120a generates the 1 st image data from the image signal of the central optical system light receiving pixel 22a of the image sensor 20, and generates the 2 nd image data from the image signal of the annular optical system light receiving pixel 22 b. The 1 st image data is image data of an image captured by the central optical system 12, and is wide-angle image data. The 2 nd image data is image data of an image captured by the ring optical system 14, and is tele image data.

Then, the digital signal processing section 120a detects the luminance of the subject necessary for exposure control based on the read image signal.

Then, the digital signal processing unit 120a detects an evaluation value necessary for focusing of the ring optical system 14 from the read image signal. For example, the contrast in a predetermined focal region is detected as an evaluation value of the focal point of the annular optical system 14.

< imaging control Unit >

The imaging control unit 120b controls imaging by the image sensor 20. The image pickup control section 120b controls driving of the image sensor 20 via the image sensor driver 20a, and controls image pickup by the image sensor 20.

In addition, the shooting can be performed for moving images and still images. Which of the shots is set by the operation unit 140.

When capturing a moving image, the image sensor 20 is controlled to be driven so as to capture the moving image at a predetermined frame rate.

When a still image is captured, the driving of the image sensor 20 is controlled in such a manner that the still image is captured under moderate exposure.

< annular optical System Focus control Unit >

The ring optical system focus control unit 120c controls the focus of the ring optical system 14. The annular optical system focus control unit 120c controls the driving of the actuator 50 via the linear motor driver 50a, and controls the focus adjustment of the annular optical system 14. That is, by controlling the driving of the actuator 50, the position of the image sensor 20 with respect to the ring optical system 14 is controlled, and the focus of the ring optical system 14 is controlled.

Further, the focus control of the ring optical system 14 performs processing according to the image pickup mode. This point will be described in detail later.

< position detection section >

The position detector 120d detects the position of the image sensor 20 based on the outputs of the photo interrupter 62 and the MR sensor 64.

< image output control section >

The image output control section 120e controls the output of an image obtained by shooting. The image output control unit 120e converts the image data of the central optical system 12 generated by the digital signal processing unit 120a into an output format for display, and outputs the converted image data from the central optical system image output terminal 150 a. The image data of the ring optical system 14 is converted into an output format for display and output from the ring optical system image output terminal 150 b. A display is connected to the central optical system image output terminal 150a and the annular optical system image output terminal 150b, for example, and an image obtained by shooting is displayed on the display.

< recording control section >

The recording control section 120f controls recording of an image obtained by shooting. The recording control unit 120f converts the image data of the central optical system 12 and the image data of the ring optical system 14 generated by the digital signal processing unit 120a into a data format for recording, and records the converted data in the recording unit 130.

Ministry of record

The recording unit 130 is constituted by a nonvolatile memory such as an EEPROM (Electrically Erasable Programmable Read only memory).

Section for operation

The operation unit 140 includes various operation mechanisms such as a power switch, a release button, and a mode dial, and outputs a signal corresponding to an operation to the computer 120.

[ Effect of the image pickup apparatus ]

The imaging device 1 of the present embodiment can capture a moving image and a still image. Which of the shots is set by the operation unit 140.

In still image shooting, three image capturing modes, i.e., a normal mode, a continuous shooting mode a, and a continuous shooting mode B, are provided, and focus control is performed in accordance with each mode.

In the shooting of a moving image, two image pickup modes, i.e., a normal mode and a focus scan mode, are provided, and focus control is performed in accordance with each mode.

Hereinafter, the contents of the processing for each image capturing mode will be described.

Shooting of still images

< common mode >

In the normal mode, focus control of a so-called contrast system is performed for the ring optical system 14.

When AF (auto focus) is commanded via the operation portion 140, the ring optical system focus control portion 120c drives the actuator 50 and moves the image sensor 20 from one end of the movable range toward the other end. During this movement, information of the evaluation value of the focal point of the ring optical system 14 is acquired, and the position of the image sensor 20 at which the acquired evaluation value becomes the maximum is detected. The detected position of the image sensor 20 is set as the focus position of the ring optical system 14, and the image sensor 20 is moved to the focus position. This enables the ring optical system 14 to be focused.

The AF command is performed by, for example, half-pressing of a release button. When the release button is half pressed and then the full press is performed, image pickup for recording is instructed. The imaging control unit 120b performs imaging processing for recording in accordance with an imaging command for recording. The image obtained by the shooting is recorded in the recording section 130.

In this manner, in the normal mode, the contrast-type focus control is performed on the ring optical system 14.

< continuous shooting mode A >

In the continuous shooting mode a, still images are continuously shot while moving the image sensor 20. The image sensor 20 moves from one end of the movable range toward the other end at a certain pitch. That is, in the continuous shooting mode a, the still image is continuously shot by moving the image sensor 20 within a certain range.

Fig. 11 is a flowchart showing the procedure of image pickup processing in the continuous shooting mode a.

When the image pickup is instructed, the annular optical system focus control unit 120c moves the image sensor 20 to the position (MOD position) of the shortest photographing Distance (MOD) (step S1).

Subsequently, image pickup for recording is performed (step S2). The image obtained by the shooting is recorded in the recording section 130 (step S3).

Next, the annular optical system focus control section 120c determines whether the image sensor 20 has reached a position (INF position) focused on an object at Infinity (INF) (step S4).

If it is determined that the INF position is not reached, the annular optical system focus control unit 120c drives the actuator 50 and moves the image sensor 20 by one step amount (step S5). After the movement, image pickup processing is performed (step S2).

In this manner, in the continuous shooting mode a, the image sensor 20 is moved by one step amount at a time, and shooting is performed every time. The processing is ended at the stage where the image sensor 20 reaches the INF position and the shooting in the INF position is ended.

The still image group obtained by shooting is recorded in the recording section 130 in a state distinguishable from images obtained by other shooting. For example, recording is performed by changing a folder every shooting. Alternatively, a series of images are recorded by assigning the same identification identifier or the like.

If there is a main subject between MOD and INF, at least one in-focus image (annular optical system in-focus image) can be captured by the annular optical system 14 by performing imaging in the continuous shooting mode a. When a plurality of subjects exist between MOD and INF, an arbitrary subject is selected to obtain an in-focus image. The central optical system 12 acquires an image captured at the same timing as the image set as the in-focus image by the ring optical system 14 as the in-focus image of the central optical system 12 (central optical system in-focus image).

< continuous shooting mode B >

In the continuous shooting mode B, a still image is continuously shot during a period in which shooting is commanded. At this time, the image sensor 20 is periodically moved. That is, in the continuous shooting mode B, the still image is continuously shot while the image sensor is periodically moved.

Fig. 12 is a flowchart showing the procedure of image pickup processing in the continuous shooting mode B.

First, whether or not an imaging start command is present is determined (step S11). The start of imaging is commanded by, for example, fully pressing a release button.

When it is determined that the start of image capturing is instructed, the annular optical system focus control unit 120c moves the image sensor 20 to the MOD position (step S12).

After the shift, the recording image is picked up (step S13). The image obtained by the shooting is recorded in the recording section 130 (step S14).

Then, whether or not there is an imaging end command is determined (step S15). The end of imaging is commanded by, for example, fully pressing a release button. That is, the image capturing is started by the 1 st release button full-press command, and the image capturing is ended by the 2 nd release button full-press command. For example, the imaging may be instructed while the release button is fully pressed, and the imaging may be instructed to be ended when the release button is released.

If it is determined that the end of imaging has not been instructed, the ring optical system focus control unit 120c drives the actuator 50 and moves the image sensor 20 by one step (step S16). After the movement, image pickup processing is performed (step S13).

In this way, in the continuous shooting mode B, the still image is continuously shot while periodically changing the distance of the subject in focus by periodically moving the image sensor 20.

Fig. 13 is a diagram showing an example of a method for changing the distance of an object focused by the ring optical system.

As shown in fig. 13, the distance of the object focused between MOD and INF (focused object distance) can be varied in a sine wave. In this case, the image sensor 20 is displaced in a sine wave between the MOD position and the INF position. This enables the distance of the subject in focus between MOD and INF to be changed periodically.

The still image group obtained by imaging is recorded in the recording unit 130 with a single scanning section, which is a moving section (forward path) from MOD to INF and a moving section (backward path) from INF to MOD. As for the ring-shaped optical system 14, as long as there is a main object between MOD and INF, at least one in-focus image can be obtained per scanning section.

The in-focus image of the ring optical system 14 (ring optical system in-focus image) can be acquired by analyzing a still image group of the ring optical system 14 obtained by shooting for each scanning section (for each period of displacement) and extracting a still image with the highest resolution (i.e., the image with the highest focus evaluation value — the image with the highest contrast).

The in-focus image of the central optical system 12 (central optical system in-focus image) can be acquired by extracting an image taken at the same timing as the in-focus image of the ring optical system 14.

Fig. 14 is a diagram showing another example of a method for changing the distance of an object focused by the ring optical system.

As shown in fig. 14, the distance of the object focused between MOD and INF (focused object distance) can be changed in a sawtooth waveform. In this case, the image sensor 20 is displaced in a sawtooth waveform between the MOD position and the INF position. This enables the distance of the subject in focus between MOD and INF to be changed periodically.

When the displacement is in a sawtooth waveform, a moving section (forward path) from MOD to INF is recorded in the recording unit 130 as one scanning section. As for the ring-shaped optical system 14, as long as there is a main object between MOD and INF, at least one in-focus image can be obtained per scanning section.

"imaging of moving image

< common mode >

In the normal mode, the ring optical system 14 is continuously controlled so as to focus on a main subject while continuously performing so-called contrast-type focus control.

< Focus scanning mode >

In the focus scan mode, in shooting, the image sensor 20 is periodically moved, and the distance of the object focused by the ring optical system 14 is periodically changed. For example, periodically varying from MOD to INF.

When the shooting starts, the annular optical system focus control unit 120c drives the actuator 50 and periodically moves the image sensor 20. Thereby, the distance of the object focused by the ring optical system 14 periodically changes. The image sensor 20 is displaced in a sine wave shape (refer to fig. 13) or a sawtooth wave shape (refer to fig. 14).

In this manner, in the focus scan mode, the image sensor 20 is periodically moved, so that the distance of the subject focused by the ring optical system 14 is periodically changed, thereby capturing a moving image.

The dynamic image photographed in the focus scan mode is used to acquire a still image focused by post-processing. That is, in the focus scan mode, the distance of the subject in focus changes periodically, and therefore, as long as the main subject exists within the range of the focusable distance, at least one frame focuses on the main subject in each period. Therefore, with respect to the ring-shaped optical system 14, if the obtained moving image is analyzed for each cycle of the displacement of the image sensor 20, at least one in-focus image can be obtained for each cycle. That is, if an image of a frame with the highest resolution is extracted in each cycle, a still image focused in each cycle can be acquired.

As for the central optical system 12, an image of a frame captured at the same timing as an image of a frame extracted as a focused image by the annular optical system 14 is set as a focused image (central optical system focused image) of the central optical system 12.

[ Effect of the image pickup apparatus ]

As described above, in the imaging apparatus 1 according to the present embodiment, the central optical system 12 is set to be a fixed focus, and only the annular optical system 14 is provided with the focus adjustment function. This can simplify the structure, and can make the entire device light-weight and compact.

Further, since the central optical system 12 and the image sensor 20 are moved integrally when the annular optical system 14 is focused, the central optical system 12 can always be in a state of being in a wide focus to perform imaging.

Further, since the central optical system 12 is smaller and lighter than the ring optical system 14, the mechanism can be made lighter and more compact even when the central optical system moves integrally with the image sensor 20. Further, the actuator 50 can be a small and lightweight actuator. This makes it possible to reduce the size and weight of the entire device.

Further, even when the central optical system 12 and the image sensor 20 are moved integrally, the structure is light and compact, and therefore, the high-speed operation can be performed. Further, the generation of vibration, noise, and the like can also be suppressed.

[ modified example of imaging device ]

Modifications of imaging lens

In the above-described embodiment, the central optical system 12 of the imaging lens 10 is configured by the wide-angle optical system, and the ring-shaped optical system 14 is configured by the telephoto optical system, but the configuration of the imaging lens 10 is not limited to this.

However, since the central optical system 12 is made to be in focus, it is preferable to have a structure that can achieve focus in a relatively easy manner.

Generally, with respect to the pan focus, the shorter the focal length, the easier it is to achieve. Therefore, it is conceivable to configure the central optical system with an optical system having a short focal point, and configure the ring-shaped optical system with an optical system having a longer focal length than the central optical system. Thus, images with different focal lengths can be simultaneously captured on the same axis. Therefore, for example, the central optical system may be configured as a wide-angle optical system, and the ring optical system 14 may be configured as a standard optical system.

In general, the standard optical system is an optical system having a field angle of about 50 °. The wide-angle optical system is an optical system that has a wider field angle (about 60 ° or more) than the standard and can capture an image over a wide range. The telephoto optical system is an optical system having a longer focal length than the standard optical system and capable of magnifying and photographing a long-distance subject.

In general, the more distant imaging is achieved with respect to the focus spread. Therefore, the central optical system can be set to take a long distance image and the annular optical system can be set to take a short distance image. Thus, two images at a long distance and a short distance can be simultaneously captured on the same axis.

In general, the depth of field becomes deeper as the distance to the subject becomes longer. Therefore, a long-distance subject can be imaged by the wide focus. On the other hand, it is difficult to capture a short-distance object with the use of the zoom. Therefore, a long-distance subject is imaged by the center optical system that is in focus, and a short-distance subject is imaged by the ring optical system having a focusing function, whereby an image focused at both a long distance and a short distance can be simultaneously imaged.

Here, "long distance" and "short distance" refer to a relationship defined between the central optical system and the annular optical system. That is, the central optical system is set to photograph a short-distance subject with respect to the annular optical system, and the annular optical system is set to photograph a long-distance subject with respect to the central optical system.

However, the central optical system is set to a distance that can be imaged by the focus blur in terms of the relationship of imaging by the focus blur. For example, when the angle is wide to a standard field angle (approximately 50 ° or more), the central optical system is set to shoot 3m to infinity.

In the case of long-distance and short-distance imaging, the long-distance imaging can be performed by the long-focus optical system, and the short-distance imaging can be performed by the wide-angle optical system.

The lens structure of each optical system in the imaging device shown in fig. 1 is an example, and other lens structures may be employed. In particular, in the above-described embodiment, the ring optical system is configured by a catadioptric optical system, but the ring optical system may be configured not to use reflection. In addition, the ring-shaped optical system can be configured to be compact in overall configuration by being configured by a catadioptric optical system.

In the above embodiment, the annular optical system 14 is configured by one optical system, but may be configured by a plurality of optical systems. For example, the ring-shaped optical system can be divided into two parts and configured by optical systems of different focal lengths. In this case, the image sensor is configured to be able to selectively receive light from each of the ring-shaped optical systems. When the annular optical system is configured by a plurality of optical systems, the annular optical system may be divided in the circumferential direction, or may be concentrically divided into a plurality of portions.

Modification of focusing mechanism for annular optical System

In the above-described embodiment, the central optical system 12 and the image sensor 20 are connected by a linear shaft that moves linearly, and the linear shaft is linearly moved by a linear motor, but the configuration in which the central optical system 12 and the image sensor 20 are integrally moved is not limited to this. For example, the central optical system 12 and the image sensor 20 may be integrally moved by a drive mechanism combining a feed screw and a motor. Further, the central optical system 12 and the image sensor 20 may be integrally moved by a driving mechanism combining a cam cylinder and a motor.

The central optical system 12 and the image sensor 20 may be manually moved. When manually moved, the central optical system 12 and the image sensor 20 are lightweight and can be moved with a light force.

Annular optical system focused image extracting section and central optical system focused image extracting section

The image pickup apparatus may be provided with a function of extracting the ring-shaped optical system in-focus image and the center-optical system in-focus image from the still image group captured in the continuous shooting mode a and the continuous shooting mode B. Similarly, a function of extracting the ring-shaped optical system in-focus image and the central optical system in-focus image from the moving image captured in the focus scanning mode may be provided in the imaging device.

< function of extracting Ring optical System Focus image and Central optical System Focus image from group of still images captured in continuous shooting mode A >

The process of extracting the ring-shaped optical system in-focus image from the group of still images captured in the continuous shooting mode a is performed as follows. That is, a still image group of the ring optical system obtained by photographing is analyzed, and an image with the highest resolution is extracted as a ring optical system in-focus image.

As for the central optical system focus image, a still image of the central optical system, which is captured at the same timing as the still image extracted as the ring-shaped optical system focus image, is set as the central optical system focus image.

This processing is performed by the computer 120, for example. That is, the computer 120 functions as an annular optical system focus image extracting unit and a central optical system focus image extracting unit by executing a predetermined program. The annular optical system focus image extracting unit and the central optical system focus image extracting unit read out the image data recorded in the recording unit 130 and execute extraction processing.

When the present extraction process is performed, only the extracted image may be recorded in the recording unit 130.

< function of extracting Ring optical System Focus image and Central optical System Focus image from group of still images captured in continuous shooting mode B >

The process of extracting the ring-shaped optical system in-focus image from the still image group photographed in the continuous shooting mode B is performed as follows. That is, the group of still images of the ring optical system obtained by photographing is analyzed for each period of displacement of the central optical system and the image sensor, and the image with the highest resolution is extracted for each period of displacement as the ring optical system in-focus image.

As for the central optical system focus image, a still image of the central optical system, which is captured at the same timing as the still image extracted as the ring-shaped optical system focus image, is set as the central optical system focus image.

This processing is performed by the computer 120, for example. That is, the computer 120 functions as an annular optical system focus image extracting unit and a central optical system focus image extracting unit by executing a predetermined program. The annular optical system focus image extracting unit and the central optical system focus image extracting unit read out the image data recorded in the recording unit 130 and execute extraction processing. Alternatively, the extraction process is performed in real time for each cycle.

When the present extraction process is performed, only the extracted image may be recorded in the recording unit 130.

< function of extracting in-focus image of annular optical system and in-focus image of central optical system from moving image captured in focus scan mode >

The process of extracting the ring-shaped optical system in-focus image from the moving image photographed in the focus scan mode is performed as follows. That is, a moving image of the ring optical system obtained by imaging is analyzed for each period of displacement of the central optical system and the image sensor, and an image of a frame with the highest resolution is extracted for each period of displacement as a ring optical system in-focus image.

As for the central optical system focus image, an image of a frame of the central optical system captured at the same timing as an image of a frame extracted as the ring-shaped optical system focus image is set as the central optical system focus image.

This processing is performed by the computer 120, for example. That is, the computer 120 functions as an annular optical system focus image extracting unit and a central optical system focus image extracting unit by executing a predetermined program. The annular optical system focus image extracting unit and the central optical system focus image extracting unit read out the image data recorded in the recording unit 130 and execute extraction processing. Alternatively, the extraction process is performed in real time for each cycle.

When the present extraction process is performed, only the extracted image may be recorded in the recording unit 130.

◆◆ Mobile Camera ◆◆

[ Structure of Mobile imaging device ]

The mobile imaging device is configured such that two images can be simultaneously captured coaxially while moving by mounting the imaging device to which the present invention is applied on a mobile body.

Fig. 15 is a diagram showing a schematic configuration of an embodiment of a mobile imaging apparatus.

The mobile imaging device 200 of the present embodiment includes an imaging device 210, an unmanned aerial vehicle 220 on which the imaging device 210 is mounted, and a controller 230 that remotely operates the imaging device 210 and the unmanned aerial vehicle 220.

Camera device

The configuration of the imaging device 210 is basically the same as that of the imaging device 1 described in the above embodiment. That is, the zoom lens includes a wide-angle central optical system, a telephoto annular optical system, and an image sensor having directivity, and the focus of the annular optical system is adjusted by integrally moving the central optical system and the image sensor.

The imaging device 210 according to the present embodiment is remotely operated by the controller 230, and therefore includes a wireless communication means for performing wireless communication with the controller 230.

Unmanned aircraft

The unmanned aerial vehicle 220 is an example of a mobile body. The unmanned aerial vehicle 220 is a so-called drone, and flies in the atmosphere according to an operation based on the controller 230. The structure of such an unmanned aircraft is well known, and therefore, a detailed description thereof will be omitted.

[ photographing by a mobile imaging device ]

The unmanned aerial vehicle 220 flies in the atmosphere according to the operation of the controller 230 by the mobile camera 200. Also, the image pickup device 210 picks up a still image or a moving image according to the operation of the controller 230.

The imaging apparatus 1 described in the above embodiment is similar to the imaging apparatus described in the above embodiment in that the normal mode, the continuous shooting mode a, and the continuous shooting mode B are provided as the imaging modes of the still image, and the normal mode and the focus scanning mode are provided as the imaging modes of the moving image.

The mobile imaging apparatus 200 of the present embodiment can simultaneously capture telephoto and wide-angle images on the same axis, and is therefore suitable for the following applications. That is, the present invention is suitable for use in which a ground surface is continuously photographed from above, and the obtained image groups are merged to generate one composite image for photographing a wide area. Such a composite image is generally generated by extracting corresponding feature points between adjacent images, and aligning and combining the positions in such a manner that the corresponding feature points overlap each other. Since the mobile imaging device 200 of the present embodiment can simultaneously capture tele and wide images on the same axis, it is possible to perform the extraction processing of the feature points in the wide image and the synthesis processing in the tele image. By performing the extraction processing of the feature points in the wide-angle image, more feature points can be extracted, and the alignment can be performed with higher accuracy. Further, by using the tele image for image synthesis, a higher definition synthesized image can be generated.

As a method of generating a composite image, it is also possible to analyze a series of still image groups obtained by image capturing, estimate the relative position and orientation of the image capturing apparatus when each image is captured, and arrange each image according to the estimation result to generate a composite image. In this case, the group of wide-angle images obtained by imaging is analyzed to estimate the relative position and orientation of the imaging device when each image is imaged, and each of the images in the telephoto range is arranged based on the estimation result to generate a composite image. This enables a high-definition composite image to be generated with high accuracy. As a method of estimating the relative position and orientation of the imaging device when each image is captured, for example, an SfM (Structure from Motion) method can be used.

When the purpose is to generate a composite image, a still image is photographed in the continuous shooting mode B. That is, a still image is continuously captured while periodically changing the distance of the subject in focus. Alternatively, a moving image is photographed in a focus scan mode. That is, a moving image is captured while periodically changing the distance of the subject in focus.

Further, when the purpose is to generate a composite image, it is preferable to control the flight of the unmanned aerial vehicle 220 as follows.

Fig. 16 is a conceptual diagram of flight control of the unmanned aerial vehicle for the purpose of generating a composite image.

Now, consider a case where an image is captured from a certain height toward directly below by the imaging device 210. In this case, the range of the angle of view θ a is imaged by the central optical system, and the range of the angle of view θ B is imaged by the telephoto optical system.

The width of the imaging range of the central optical system in the moving direction of the unmanned aerial vehicle 220 is XA, and the width of the imaging range of the annular optical system is XB.

In this case, the unmanned aerial vehicle 220 is set to a speed at which scanning is performed at least once while moving the distance XB.

Here, the scanning means that the distance of the subject in focus is changed from MOD to INF or from INF to MOD.

By controlling the flight of the unmanned aerial vehicle 220 in this manner, it is possible to capture a focus image of the ring optical system as a composite object without a gap.

◆◆ alternative embodiment ◆◆

In the above embodiments, the functions to be realized by the computer can be realized by various processors. The various processors include a CPU (central processing Unit), which is a general-purpose processor that executes a program and functions as a processing Unit that performs various types of processing, a PLD (Programmable Logic Device), which is a processor whose Circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), and a dedicated electrical Circuit, which is a processor having a Circuit configuration specifically designed to execute a Specific process.

A function may also be implemented by more than two processors of the same kind or of different kinds. For example, a configuration may be realized by a plurality of FPGAs, or a configuration may be realized by a combination of a CPU and an FPGA.

Further, a plurality of functions may be configured by one processor. As an example of a configuration in which a plurality of functions are realized by one processor, the configuration 1 is a configuration in which one processor is configured by a combination of one or more CPUs and software, as typified by a computer such as a client or a server, and the processor is caused to realize a plurality of functions. The 2 nd System uses a processor in which a plurality of functions are realized by one IC Chip (IC) as represented by a System On Chip (SoC). As described above, various functions can be realized as a hardware configuration using one or more of the various processors described above.

More specifically, the hardware configuration of these various processors is an electric circuit in which circuit elements such as semiconductor elements are combined.