阅读说明：本技术 成像装置、电子装置和成像方法 (Imaging device, electronic device, and imaging method ) 是由 渡部刚史 于 2020-01-20 设计创作，主要内容包括：本发明的目的是提供一种成像装置和电子设备,使得可以在被摄体的运动中用户期望的时刻对运动进行成像。CIS(2)是根据本公开的成像装置的示例,以及电子设备(1)具有成像单元(5)、存储器(6)、检测单元(82)和擦除单元(83)。成像单元(5)顺序并且连续地捕获被摄体的图像并且生成图像数据。存储器(6)存储图像数据。检测单元(82)基于按时间序列的图像数据检测被摄体的运动量。擦除单元(83)擦除其中被摄体的运动量小于规定运动量的图像数据。(An object of the present invention is to provide an imaging apparatus and an electronic device that make it possible to image a motion of a subject at a timing desired by a user. The CIS (2) is an example of an imaging device according to the present disclosure, and the electronic apparatus (1) has an imaging unit (5), a memory (6), a detection unit (82), and an erasing unit (83). An imaging unit (5) sequentially and continuously captures images of a subject and generates image data. The memory (6) stores image data. A detection unit (82) detects the amount of movement of the subject based on the time-series image data. An erasing unit (83) erases image data in which the movement amount of the subject is smaller than a prescribed movement amount.)

1. An image forming apparatus comprising:

an imaging unit that successively captures images of a subject to generate image data;

a memory storing the image data;

a detection unit that detects a motion amount of the object based on the image data in time series; and

an erasing unit that erases the image data in which the movement amount of the subject is smaller than a predetermined movement amount among the image data.

2. The imaging apparatus of claim 1, further comprising

A specifying unit that specifies a subject in the image based on the image data,

wherein the detection unit

The movement amount of the object specified by the specifying unit is detected.

3. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

The subject is specified based on image data of a preview image captured by the imaging unit before the start of high-speed imaging performed by the imaging unit.

4. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

Determining a composition of an image by using a Deep Neural Network (DNN) process that uses DNN and then decides a region of interest (ROI) region in which the subject is imaged from the composition, and

the detection unit

Detecting the motion amount of the object in the ROI area.

5. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

The subject in the center of the image is designated as the subject.

6. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

Designating an object selected by a user from the image as the subject.

7. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

Excluding an object selected by a user from the image from targets to be specified as the subject.

8. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

An object close to the focus of the auto-focus is designated as a subject.

9. The imaging apparatus of claim 2, further comprising

A storage unit that stores image data of a subject to be specified as the subject by the specifying unit,

wherein the specifying unit

Specifying a subject based on the image data stored in the storage unit.

10. The image forming apparatus as set forth in claim 2,

wherein the imaging apparatus has a plurality of modes in which types of objects to be specified as subjects by the specifying unit are different from each other, and

the specifying unit

The object is specified as a subject according to the pattern.

11. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

An object determined by artificial intelligence to be likely to move in the image is specified as a subject.

12. The image forming apparatus as set forth in claim 2,

wherein the specifying unit specifies the image when the image includes a plurality of persons

Deciding the person to be designated as a subject based on the age and/or gender of the person.

13. The image forming apparatus as set forth in claim 2,

wherein the specifying unit

A subject continuously existing for a predetermined time or longer in a preview image captured by the imaging unit before the start of high-speed imaging performed by the imaging unit is specified as a subject.

14. An electronic device, comprising:

an imaging unit that successively captures images of a subject to generate image data;

a memory storing the image data;

a detection unit that detects a motion amount of the object based on the image data in time series; and

an erasing unit that erases the image data in which the movement amount of the subject is smaller than a predetermined movement amount among the image data.

15. An imaging method, comprising:

continuously capturing images of a subject to generate image data;

storing the image data in a memory;

detecting a motion amount of the subject based on the image data in time series; and

erasing the image data in which a movement amount of the subject is smaller than a predetermined movement amount among the image data.

Technical Field

The present disclosure relates to an imaging apparatus, an electronic apparatus, and an imaging method.

Background

There is an imaging apparatus having a continuous imaging function capable of capturing a moving subject by continuously imaging the moving subject at a high speed, particularly capturing the motion of the subject at a timing desired by a user in the motion of the subject. For example, patent document 1 describes an imaging apparatus that controls continuous imaging to be performed at a frame rate higher than a current frame rate when it is determined that an operation unit for continuous imaging is pressed by a user, and controls to maintain the current frame rate when it is determined that the operation unit is not pressed.

Reference list

Patent document

Patent document 1: JP 2012 147355A

Disclosure of Invention

Technical problem

However, when high-speed continuous imaging is performed, there is a case where the imaging apparatus cannot image the motion of the subject at a timing desired by the user in the motion of the subject. Accordingly, the present disclosure proposes an imaging apparatus, an electronic apparatus, and an imaging method capable of imaging a motion of a subject at a timing desired by a user in the motion of the subject.

Solution of the problem

An imaging apparatus according to the present disclosure includes an imaging unit, a memory, a detection unit, and an erasing unit. The imaging unit continuously captures images of a subject to generate image data. The memory stores image data. The detection unit detects a motion amount of the object based on the time-series image data. The erasing unit erases image data in which a movement amount of the subject is smaller than a predetermined movement amount.

Drawings

Fig. 1 is a block diagram showing a schematic configuration example of an electronic device according to the present disclosure.

Fig. 2 is an operation diagram of a CIS according to the present disclosure.

Fig. 3 is a diagram illustrating an image captured at high speed according to a comparative example of the present disclosure.

Fig. 4 is a diagram illustrating an image captured at high speed according to the present disclosure.

Fig. 5 is a diagram showing a relationship between the amount of movement of a subject and the number of images stored in a memory according to the present disclosure.

Fig. 6 is a flowchart illustrating an example of a process performed by the CIS according to the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In each embodiment below, the same portions are denoted by the same reference numerals, and a repetitive description thereof will be omitted.

[1. arrangement of electronic device ]

Fig. 1 is a block diagram showing a schematic configuration example of an electronic apparatus according to the present disclosure. The electronic apparatus 1 shown in fig. 1 is an apparatus that captures still images or moving images, and examples thereof include digital cameras and digital video cameras, or tablet terminals having a camera function, smart phones, and the like.

As shown in fig. 1, an electronic device 1 includes a Complementary Metal Oxide Semiconductor (CMOS) image sensor (hereinafter referred to as CIS 2), an application processor (hereinafter referred to as AP 3), and a display operation unit 4.

The display operation unit 4 is a touch panel liquid crystal display having a function of displaying an image and a function of receiving a touch operation by a user. Alternatively, the electronic apparatus 1 may include: a liquid crystal display for displaying an image; and an operation switch for receiving a touch operation by a user, instead of the display operation unit 4.

As an example of an imaging device having a single chip configuration, the CIS 2 captures an image and outputs image data of the captured image to the AP 3. The CIS 2 has a high-speed imaging function (hereinafter referred to as high-speed imaging) of imaging a subject continuously at high speed, in addition to a function of capturing a normal still image or a moving image. The imaging device according to the present disclosure is not limited to the CIS 2, and may be other image sensors such as a Charge Coupled Device (CCD) image sensor.

The AP 3 includes: an AP control unit 31, the AP control unit 31 being an example of a processor (such as a Central Processing Unit (CPU)) that executes various application programs according to an application of the electronic apparatus 1; and a communication interface (I/F)32 that performs information communication with the CIS 2.

The AP 3 performs, for example, correction processing and modification processing on the image data input from the CIS 2, and performs display control processing for displaying the image data on the display operation unit 4. Further, by controlling to display image data of a captured image captured at high speed by the CIS 2 on the display operation unit 4 in time series, the AP 3 can perform ultra-slow playback of the motion of the subject moving at high speed.

Here, when a general electronic device such as a smartphone is equipped with a camera to perform high-speed imaging, the CIS continuously images a subject at high speed and stores image data of captured images in a memory in time series.

Unfortunately, however, electronic devices have an upper limit on the capacity of the memory that stores image data. Therefore, in a general electronic apparatus, for example, when the operation timing of the user to start high-speed imaging is too early, the image data amount of a captured image reaches the upper limit of the capacity of the memory before the time desired by the user comes in the motion of the subject.

In this case, even when image data captured at a high speed is played back at an ultra-slow speed on an electronic apparatus, it may be difficult to play back an image at a time desired by a user in the motion of a subject because the image data of the motion captured at the time desired by the user in the motion of the subject is not stored in the memory. In this way, there is a case where capturing the motion of the subject at the time desired by the user in the motion of the subject is impossible with a general electronic apparatus.

Therefore, the CIS 2 according to the present disclosure has a configuration capable of capturing the motion of the subject at the timing desired by the user even when the operation of the user for starting high-speed imaging is too early. Specifically, the CIS 2 includes an imaging unit 5, a memory 6, a communication I/F7, and an image selection unit 8.

The imaging unit 5 includes a light receiving unit 51 and a readout unit 52. The light receiving unit 51 includes: an optical system equipped with a zoom lens, a focus lens, an aperture, and the like; and a pixel array unit having a configuration in which unit pixels including light receiving elements such as photodiodes are arranged in, for example, a two-dimensional matrix.

The light receiving unit 51 forms an image of light incident from the outside on the light receiving surface of the pixel array unit using an optical system. By performing photoelectric conversion on light incident on the light receiving element, each unit pixel of the pixel array unit readably accumulates electric charges corresponding to the amount of incident light.

The readout unit 52 includes: a readout circuit that reads out the electric charge accumulated in each unit pixel as a pixel signal; and an analog-to-digital converter (ADC) that generates digital image data by converting the analog pixel signals read out by the readout circuit into digital values. The readout unit 52 outputs the generated image data to the memory 6 in units of one frame of an image.

The imaging unit 5 has an Autofocus (AF) function. When focusing (focus setting) is achieved on a subject by the AF function, the imaging unit 5 outputs information indicating the position of the focus in a captured image to the image selection unit 8.

When a signal indicating that the user has performed an imaging operation is input from the display operation unit 4, the imaging unit 5 captures a still image or a moving image. Further, the imaging unit 5 starts high-speed imaging upon receiving a signal (hereinafter, referred to as a high-speed imaging trigger) indicating that the user has performed a high-speed imaging start operation on the display operation unit 4.

The memory 6 is a storage device such as a flash memory, and stores image data representing captured images input from the imaging unit 5 in time series. Upon receiving the image data acquisition request from the AP 3, the communication I/F7 outputs the image data stored in the memory 6 to the AP 3.

The AP control unit 31 outputs image data input from the CIS 2 via the communication I/F32 to the display operation unit 4 to display the image data. For example, when the display operation unit 4 receives an image display operation by a user, the AP control unit 31 outputs an image data acquisition request to the CIS 2 via the communication I/F32, acquires image data from the CIS 2, and displays the image data on the display operation unit 4.

The image selection unit 8 includes, for example, a microcomputer having a Central Processing Unit (CPU), a Read Only Memory (ROM), a Random Access Memory (RAM), and the like, and various circuits.

The image selecting unit 8 includes: a specifying unit 81 and a detecting unit 82; and an erasing unit 83 configured to operate by the CPU executing the program stored in the ROM using the RAM as a work area. Note that the specifying unit 81, the detecting unit 82, and the erasing unit 83 included in the image selecting unit 8 may be partially or entirely formed of a hardware device such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA).

The specifying unit 81, the detecting unit 82, and the erasing unit 83 included in the image selecting unit 8 perform information processing described below, respectively. The internal configuration of the image selecting unit 8 is not limited to the configuration shown in fig. 1, and may be another configuration as long as it is a configuration that performs information processing described below.

The specifying unit 81 reads out image data from the memory 6, and specifies a subject in an image based on the image data. For example, before starting high-speed imaging performed by the imaging unit 5, the specifying unit 81 specifies a subject based on image data of a preview image captured by the imaging unit 5. The preview images are continuously displayed on the display operation unit 4 until the high-speed imaging is started.

The specifying unit 81 specifies a subject by determining a composition of an image by Deep Neural Network (DNN) processing using DNN and then deciding a region of interest (ROI) in which the subject is imaged from the determined composition. Subsequently, the specification unit 81 outputs information indicating the position of the ROI region in the image in which the subject is specified to the detection unit 82.

Incidentally, the specifying unit 81 can specify the object by using various neural networks other than DNN, such as a Recurrent Neural Network (RNN) and a Convolutional Neural Network (CNN). Further, the specifying unit 81 may be configured to specify a subject by using a learning model trained by various other machine learning such as a decision tree or a support vector machine.

When high-speed imaging is started, the detection unit 82 reads out image data of time-series high-speed captured images from the memory 6, and detects the amount of movement of the subject between a plurality of pieces of time-series image data based on information indicating the position of the ROI region input from the specification unit 81.

That is, the detection unit 82 detects the amount of motion of the subject in the ROI region in each time-series high-speed captured image. Subsequently, the detection unit 82 outputs the image data of each high-speed captured image in which the movement amount of the subject has been detected, to the erasing unit 83 together with information indicating the movement amount of the subject.

Among the image data input from the detection unit 82, the erasing unit 83 erases image data in which the movement amount of the subject is smaller than a predetermined movement amount. Further, the erasing unit 83 newly stores, in the image data input from the detecting unit 82, an image in which the movement amount of the subject is a predetermined movement amount or more to the memory 6.

Further, as described above, the specifying unit 81 specifies a subject based on the preview image before the start of high-speed imaging. Therefore, the erasing unit 83 can select image data immediately after the start of high-speed imaging.

In this way, the CIS 2 can extend the time available for high-speed imaging by effectively utilizing the storage region of the memory 6, which becomes a free region by erasing the image data of an image in which the movement amount of the subject is smaller than the predetermined movement amount. This enables the CIS 2 to capture the motion of the subject at a timing desired by the user in the motion of the subject even when the user's operation for starting high-speed imaging is too early.

[2. operation of electronic device ]

Next, an operation of the CIS according to the present disclosure will be described with reference to fig. 2. Fig. 2 is an operation diagram of a CIS according to the present disclosure. Although fig. 2 shows two memories 6 to simplify the operation of the CIS 2, the two memories 6 in fig. 2 are physically identical.

As shown in fig. 2, before starting high-speed imaging, the CIS 2 first specifies a subject from the preview image Pre by the image selecting unit 8 (step S1). After that, the CIS 2 has started the high-speed imaging, and the image data obtained in the high-speed imaging is output from the imaging unit 5 to the memory 6 so that the image data is stored in the memory 6 (step S2).

Subsequently, the image selection unit 8 reads out the image data from the memory 6 (step S3). Subsequently, the image selecting unit 8 erases the image data in which the movement amount of the subject is smaller than the predetermined movement amount (step S4). Thereafter, the image selecting unit 8 stores image data in which the movement amount of the subject is a predetermined movement amount or more to the memory 6 (step S5).

This allows the memory 6 to have the free area 61 in the area where the image data erased in step S4 has been stored. The CIS 2 can continue high-speed imaging by using the free area 61 of the memory 6.

With this configuration, the CIS 2 can extend the duration of high-speed imaging compared to the case where the image data is not deleted (thinned) by the image selecting unit 8. Therefore, even when the operation of the user for starting high-speed imaging is too early, the CIS 2 can capture the motion of the subject at a timing desired by the user in the motion of the subject.

Further, when an image obtained by high-speed imaging is displayed on the display operation unit 4, the CIS 2 outputs the image data from the memory 6 to the communication I/F32 of the AP 3 via the communication I/F7 of the CIS 2 (step S6).

By this operation, by displaying image data captured by high-speed imaging input from the CIS 2 on the display operation unit 4 at a frame rate lower than the imaging rate, the AP 3 can perform ultra-slow playback of an image in which the movement of the subject has been captured at a timing desired by the user in the movement of the subject.

[3. specific example of image captured at high speed ]

Next, a specific example of an image captured at high speed according to the present disclosure will be described. Here, a case where the user attempts to capture an object at the time of contact when hitting a ball with a baseball bat will be described. Further, here, high-speed imaging not performing image data erasure by the image selecting unit 8 as a comparative example will be described first, and then high-speed imaging performing image data erasure by the image selecting unit 8 will be described.

Fig. 3 is a diagram illustrating an image captured at high speed according to a comparative example of the present disclosure. Fig. 4 is a diagram illustrating an image captured at high speed according to the present disclosure. Here, in order to facilitate understanding of the effect of the image selection unit 8, the number of image frames that can be stored in the memory 6 is described as 7 frames, which is much smaller than the actual possible number.

As shown in the upper part of fig. 3, the action of swinging a baseball bat includes a series of actions of retrieving, swinging, and following a swing. For example, when the erasing of the image by the image selecting unit 8 is not performed, the CIS 2 starts the high-speed imaging at time t1 and ends the high-speed imaging at time t2 at the point of capturing the image of the seventh frame.

When the playback of the high-speed captured image captured in this manner is performed at an ultra-slow rate, the playback image is mainly an image of the subject during the retracting movement, as shown in the lower part of fig. 3, and does not include an image desired by the user during the swing and at the time of contact.

In contrast, as shown in the upper part of fig. 4, the procedure in the case where the erasing of the image is performed by the image selecting unit 8 is such that, after the image of the first frame is captured at time t1, the image selecting unit 8 erases the retracted images of the second to fourth frames in which the movement amount of the subject is smaller than the predetermined movement amount.

At this time, the memory 6 stores only one frame of the image, and therefore, can store the remaining six frames of the image. Since the subject enters the swing motion from the fifth frame, the motion amount of the subject is a predetermined motion amount or more. Therefore, the image selecting unit 8 stores the images of the fifth to tenth frames in the memory 6 without erasing them, thereby completing high-speed imaging at time t 3.

When the playback of the high-speed captured image captured in this manner is performed in the ultra-slow mode, as shown in the lower part of fig. 4, the playback image includes images from the start of the swing to the end of the swing, including the timing of the contact desired by the user.

The playback images shown in the lower part of fig. 4 are images obtained by thinning the images of the second to fourth frames, and therefore, strictly speaking, these images are not ultra-slow images that accurately express the actual motion of the subject. Further, the thinned second to fourth frame images are images that include substantially no motion of the subject and are therefore not images in a period in which high-speed imaging is required.

Therefore, the playback image shown in the lower part of fig. 4 is considered as an image that has achieved the intention of high-speed imaging from the viewpoint of the user, and is perceived by the user as an ultra-slow image with an extended imaging time. In this way, according to the CIS 2, even when the user's operation to start high-speed imaging is too early, the motion of the subject can be imaged at a timing desired by the user in the motion of the subject.

[4. relationship between amount of movement of subject and number of images stored in memory ]

Next, referring to fig. 5, the relationship between the amount of motion of the subject and the number of images stored in the memory 6 when the bat swing scene described with reference to fig. 3 and 4 is captured at a frame rate of 1000fps is shown.

Fig. 5 is a diagram showing a relationship between the amount of movement of a subject and the number of images stored in a memory according to the present disclosure. In fig. 5, the horizontal axis represents time, and the vertical axis represents the amount of movement of the subject and the number of stored images. Further, in fig. 5, the magnitude of the movement amount of the object in each time zone is shown by a bar chart.

Further, the solid line diagram shown in fig. 5 shows the number of images to be stored in the memory 6 when the images are erased by the image selecting unit 8. Further, the line graph in the one-dot chain line in fig. 5 shows the number of images to be stored in the memory 6 when the images are not erased by the image selecting unit 8.

When the entire area of the swing motion of the stick is imaged at 1000fps without erasing the image, as shown by the one-dot chain line diagram in fig. 5 (step S11), before the swing starts, the number of captured images reaches the upper limit of the number of images storable in the memory 6, and this ends the imaging (step S12).

In contrast, in the case of erasing an image, as shown in the solid line diagram in fig. 5, an image in which the movement amount of the subject is smaller than the predetermined amount during the retracting is to be erased. Therefore, even when imaging is performed at 1000fps, an image of 100fps will be stored in the memory 6 (step S13).

Thereafter, when the swing is started, the movement amount of the object gradually increases, and therefore, an image of 800fps is stored in the memory 6 (step S14). While in contact with the ball whose movement amount is maximized, an image of 1000fps is stored in the memory 6 (step S15). As a result, according to the CIS 2, all the motions from the start to the end of the swing can be imaged (step S16).

[5. processing performed by the image selecting unit ]

Next, an example of processing performed by the CIS 2 according to the present disclosure will be described with reference to fig. 6. Fig. 6 is a flowchart illustrating an example of processing performed by the CIS 2 according to the present disclosure.

When the electronic device 1 is in the high-speed imaging mode, the CIS 2 continuously performs the processing shown in fig. 6. Specifically, in the CIS 2, when the electronic device enters the high-speed imaging mode, the imaging unit 5 first starts a high-speed imaging preview, as shown in fig. 6 (step S101). Here, the imaging unit 5 captures a preview image at a lower speed than high-speed imaging.

Subsequently, the image selecting unit 8 acquires the preview image (step S102), determines the composition of the image (step S103), and decides the ROI region (step S104). Thereafter, the imaging unit 5 determines whether a high-speed imaging trigger has been received (step S105).

Then, when the imaging unit 5 determines that the high-speed imaging trigger is not received (no at step S105), the imaging unit 5 proceeds to the process at step S102. Further, when the imaging unit 5 determines that the high-speed imaging trigger has been received (yes at step S105), the imaging unit 5 starts high-speed imaging (step S106).

Thereafter, the image selecting unit 8 acquires the image captured at high speed (step S107), and determines whether the acquired image captured at high speed is an image other than the first image (step S108). When the image selection unit 8 determines that the image is the first image (no at step S108), the process proceeds to step S107.

When the image selecting unit 8 determines that the image is an image other than the first image (yes at step S108), the image selecting unit 8 detects the amount of motion of the subject in the ROI region (step S109). Then, the image selecting unit 8 determines whether the movement amount is smaller than a predetermined movement amount (step S110).

When it has been determined that the movement amount is smaller than the predetermined movement amount (yes at step S120), the image selecting unit 8 erases the image (step S111), and proceeds to the process at step S107. In contrast, when it has been determined that the movement amount is not less than the predetermined movement amount, that is, the movement amount is the predetermined movement amount or more (no at step S110), the image selection unit 8 stores the image in the memory 6 (step S112).

Subsequently, the image selecting unit 8 determines whether the number of images stored in the memory 6 has reached the upper limit of the number of images that can be stored in the memory 6 (step S113). When the image selection unit 8 determines that the upper limit of the number of storable images has not been reached (no at step S113), the process proceeds to step S107.

When the image selecting unit 8 determines that the upper limit of the number of storable images has been reached (yes at step S113), the process proceeds to step S102. When the electronic device 1 is in the high-speed imaging mode, the CIS 2 continuously performs the above-described series of processes.

[6. other examples ]

The processing according to each of the embodiments described above may be performed in various different forms (modifications) in addition to each of the embodiments described above.

For example, although the above-described embodiment is an exemplary case where one moving object is present in a captured image, the specifying unit 81 may also specify a moving object that the user may desire to image as a subject in a case where a plurality of moving objects are present in a captured image.

For example, a camera user may fit a subject to be imaged to the center of an image. Therefore, the specifying unit 81 specifies the subject in the center of the image as the subject. By this operation, in the case where a plurality of moving objects exist in the captured image, the specifying unit 81 can specify a moving object that the user may desire to image as a subject.

Further, the specifying unit 81 specifies an object selected by the user from the image as a subject. In this case, the designation unit 81 designates an object selected by, for example, a touch operation by the user as an object in the preview image displayed on the display operation unit 4. By this operation, even when, for example, a moving object that the user desires to image is captured at a position other than the center of the image, the specifying unit 81 can specify the moving object as a subject.

Note that the specifying unit 81 may also exclude an object selected by the user from the image from the targets to be specified as subjects. With this configuration, when the subject automatically specified by the specifying unit 81 is not a subject desired by the user, the user can exclude the subject from the targets to be specified as the subject by the specifying unit 81 by performing an operation of selecting the specified subject displayed on the display operation unit 4.

Further, in general, when the AF function is operating, the user of the camera typically does not release the shutter until a desired subject for imaging is focused. Therefore, when the AF function is operating, the specifying unit 81 specifies the object closest to the autofocus as the subject. By this operation, the specifying unit 81 can specify a moving object that the user may desire to image as a subject.

Further, the CIS 2 may include a storage unit that stores image data of a subject to be specified as a subject by the specifying unit 81. In this case, the CIS 2 allows the user to register and store image data of a face or an object of a person as a candidate of a subject in a storage unit in advance.

Then, the specifying unit 81 specifies a subject based on the image data stored in the storage unit. For example, the specifying unit 81 acquires a feature amount of each divided region obtained by dividing a captured image, calculates a score indicating a degree of similarity between the feature amount acquired from the captured image and a feature amount of image data stored in a memory, and specifies a subject in the divided region having a high score as a subject. By this operation, the specifying unit 81 can specify a subject whose image data may have been registered by the user in advance as a subject.

Further, the CIS 2 has a plurality of modes in which the types of objects to be specified as subjects are different from each other. The specifying unit 81 specifies a subject as a subject according to a pattern. For example, when the user selects the sport mode, the specifying unit 81 specifies only a person as a subject, and when the animal mode is selected, registers only an animal as a subject. This enables the specifying unit 81 to specify an arbitrary object included in a type desired by the user as a subject, instead of a specific object.

Further, the specifying unit 81 may also be provided with Artificial Intelligence (AI) that estimates an object that is likely to move in the captured image. In this case, the specifying unit 81 specifies the object determined by the AI as the most likely moving object as the subject. With this configuration, even when the user does not select a mode or designate an object, the designation unit 81 can designate a possible moving object that the user desires to image as a subject.

Further, when the image contains a plurality of persons, the specifying unit 81 decides a person to be specified as a subject based on at least one of the age or sex of the subject. For example, the specifying unit 81 decides that a person to be specified as a subject has a high priority for children, a medium priority for women, and a low priority for men. The priority setting for age and gender may be arbitrarily changed. With this configuration, the specifying unit 81 can specify a person as a subject according to the preference of the user.

Further, in general, a user who desires to capture an important timing of a subject tends to place the subject within the angle of view even before starting high-speed imaging. In view of this, the specifying unit 81 specifies as a subject that continues to exist for a predetermined time or longer in the preview image. By this operation, the specifying unit 81 can specify a moving object that the user may desire to image at an important timing as a subject.

Although the above-described embodiment is the case where the CIS 2 includes the specifying unit 81, the specifying unit 81 may be provided on the AP 3 side. In this case, the CIS 2 outputs the image stored in the memory 6 to the AP 3, acquires the subject specified by the specifying unit 81 provided in the AP 3 from the AP 3, and selects image data to be erased by a process similar to the above-described process.

However, when the specifying unit 81 is disposed on the AP 3 side, all image data stored in the memory 6 needs to be output to the AP 3, thereby increasing the power consumption of the CIS 2. In view of this, it is desirable to provide the specifying unit 81 on the CIS 2 side.

Unless otherwise specified, the process procedures, control procedures, specific names, and information including various data and parameters shown in the above description or the drawings may be changed in any manner. Further, the specific examples, distributions, numerical values, and the like described in the examples are merely examples, and may be arbitrarily changed.

Further, each component of each apparatus is provided as a description of functions and concepts, and thus need not necessarily be physically configured as shown. That is, the specific form of distribution/integration of each device is not limited to those shown in the drawings, and all or part thereof may be functionally or physically distributed or integrated into any unit according to various loads and use conditions. For example, the specifying unit 81, the detecting unit 82, and the erasing unit 83 shown in fig. 1 may be integrated.

Further, the above-described embodiments and modifications may be appropriately combined within the achievable scope without contradiction of the procedures.

[7. Effect ]

The CIS 2 includes an imaging unit 5, a memory 6, a detection unit 82, and an erasing unit 83. The imaging unit 5 continuously captures images of a subject to generate image data. The memory 6 stores image data. The detection unit 82 detects the movement amount of the object based on the time-series image data. The erasing unit 83 erases image data in which the movement amount of the subject is smaller than a predetermined movement amount. With this configuration, for example, when high-speed imaging is performed, the CIS 2 can capture the motion of the subject at a timing desired by the user in the motion of the subject by erasing an image having a small amount of motion between time-series images and requiring no high-speed imaging to secure a free area in the memory 6.

Further, the CIS 2 includes a specifying unit 81. The specifying unit 81 specifies a subject in an image from image data. The detection unit 82 detects the movement amount of the object specified by the specification unit 81. For example, in the case where the specifying unit 81 is provided outside the CIS 2, the CIS 2 will need to output image data to the external specifying unit 81, which increases power consumption. Fortunately, however, by such a configuration that internally includes the specifying unit 81, power consumption can be reduced.

Further, before the high-speed imaging performed by the imaging unit 5 is started, the specifying unit 81 specifies a subject from the image data of the preview image captured by the imaging unit 5. By this operation, the detection unit 82 can detect the movement amount of the subject immediately after the start of high-speed imaging.

Further, the specifying unit 81 determines the composition of the image by DNN processing using DNN, and decides the ROI region in which the subject is captured based on the composition. The detection unit 82 detects the amount of movement of the subject in the ROI region. With this configuration, the detection unit 82 can detect an accurate movement amount of the object.

Further, the specifying unit 81 specifies a subject in the center of the image as a subject. By this operation, in the case where a plurality of moving objects exist in the captured image, the specifying unit 81 can specify a moving object that the user may desire to image as a subject.

Further, the specifying unit 81 specifies an object selected by the user from the image as a subject. With this configuration, even when a moving object that the user desires to image is captured at a position other than the center of the image, the specifying unit 81 can specify the moving object as a subject.

Further, the specifying unit 81 excludes an object selected by the user from the image from the targets to be specified as the subject. Through this procedure, when the automatically designated subject is not the subject intended by the user, the designation unit 81 can exclude the automatically designated subject from the subject candidates.

Further, the specifying unit 81 specifies an object close to the focus of the auto-focus as an object. Through this procedure, the specifying unit 81 can specify a moving object that the user who starts high-speed imaging may desire as a subject at the time of object focusing.

Further, the CIS 2 includes a storage unit that stores image data of a subject to be specified as a subject by the specifying unit 81. The specifying unit 81 specifies a subject based on the image data stored in the storage unit. By this operation, the specifying unit 81 can specify a subject whose image data may have been registered by the user in advance as a subject.

Further, the CIS 2 has a plurality of modes in which the types of objects to be specified as subjects by the specifying unit 81 are different from each other. The specifying unit 81 specifies a subject as a subject according to a pattern. This enables the specifying unit 81 to specify an arbitrary object included in a type desired by the user as a subject, instead of a specific object.

Further, the specifying unit 81 specifies an object determined by artificial intelligence to be likely to move in the image as a subject. With this configuration, even when the user does not select a mode or designate an object, the designation unit 81 can designate a possible moving object that the user desires to image as a subject.

Further, when the image contains a plurality of persons, the specifying unit 81 decides a person to be specified as a subject based on the age and/or sex of the person. With this configuration, the specifying unit 81 can specify a person as a subject according to the preference of the user.

Further, the specifying unit 81 specifies as a subject that continuously exists for a predetermined time or longer in the preview image captured by the imaging unit 5 before the start of high-speed imaging performed by the imaging unit 5. By this operation, the specifying unit 81 can specify a moving object that the user may desire to image at an important timing as a subject.

Further, the electronic device 1 includes a CIS 2. The CIS 2 includes an imaging unit 5, a memory 6, a detection unit 82, and an erasing unit 83. The imaging unit 5 continuously captures images of a subject to generate image data. The memory 6 stores image data. The detection unit 82 detects the movement amount of the object based on the time-series image data. The erasing unit 83 erases image data in which the movement amount of the subject is smaller than a predetermined movement amount. With this configuration, for example, when high-speed imaging is performed, the electronic apparatus 1 can capture the motion of the subject at a timing desired by the user in the motion of the subject by erasing an image having a small amount of motion between time-series images and requiring no high-speed imaging to secure a free area in the memory 6.

Further, the CIS 2 continuously captures images of a subject to generate image data, stores the image data in the memory 6, detects a movement amount of the subject based on the image data in time series, and erases the image data in which the movement amount of the subject is smaller than a predetermined movement amount. By this operation, the CIS 2 can capture the motion of the subject at a timing desired by the user in the motion of the subject.

The effects described in this specification are merely examples, and thus, there may be other effects not limited to the exemplary effects.

Note that the present technology may also have the following configuration.

(1)

An image forming apparatus comprising:

an imaging unit that successively captures images of a subject to generate image data;

a memory storing image data;

a detection unit that detects a movement amount of an object based on time-series image data; and

an erasing unit that erases image data in which a movement amount of the subject is smaller than a predetermined movement amount.

(2)

The image forming apparatus according to (1), further comprising

A specifying unit that specifies a subject in an image based on the image data,

wherein the detecting unit

The movement amount of the object specified by the specifying unit is detected.

(3)

The image forming apparatus according to (2),

wherein the unit is specified

A subject is specified based on image data of a preview image captured by an imaging unit before high-speed imaging performed by the imaging unit is started.

(4)

The image forming apparatus according to (2) or (3),

wherein the unit is specified

Determining a composition of an image by using a Deep Neural Network (DNN) process using DNN, and then deciding a region of interest (ROI) in which a subject is imaged from the composition, and

detection unit

The movement amount of the object in the ROI area is detected.

(5)

The imaging apparatus according to any one of (2) to (4),

wherein the unit is specified

The subject in the center of the image is designated as the subject.

(6)

The imaging apparatus according to any one of (2) to (5),

wherein the unit is specified

An object selected by a user from an image is designated as a subject.

(7)

The imaging apparatus according to any one of (2) to (6),

wherein the unit is specified

An object selected by a user from an image is excluded from targets to be specified as subjects.

(8)

The imaging apparatus according to any one of (2) to (7),

wherein the unit is specified

An object close to the focus of the auto-focus is designated as a subject.

(9)

The imaging apparatus according to any one of (2) to (8), further comprising

A storage unit that stores image data of a subject to be specified as a subject by the specifying unit,

wherein the unit is specified

A subject is specified based on image data stored in a storage unit.

(10)

The imaging apparatus according to any one of (2) to (9),

wherein the imaging apparatus has a plurality of modes in which types of objects to be specified as subjects by the specifying unit are different from each other, and

specifying unit

The object is specified as a subject according to the mode.

(11)

The imaging apparatus according to any one of (2) to (10),

wherein the unit is specified

An object determined by artificial intelligence to be likely to move in an image is specified as a subject.

(12)

The imaging apparatus according to any one of (2) to (11),

wherein the specifying unit specifies the image when the image contains a plurality of persons

The person to be designated as the subject is decided based on the age and/or sex of the person.

(13)

The imaging apparatus according to any one of (2) to (12),

wherein the unit is specified

A subject continuously existing for a predetermined time or longer in a preview image captured by an imaging unit before the start of high-speed imaging performed by the imaging unit is specified as a subject.

(14)

An electronic device, comprising:

an imaging unit that successively captures images of a subject to generate image data;

a memory storing image data;

a detection unit that detects a movement amount of an object based on time-series image data; and

an erasing unit that erases image data in which a movement amount of the subject is smaller than a predetermined movement amount.

(15)

An imaging method, comprising:

continuously capturing images of a subject to generate image data;

storing the image data in a memory;

detecting a motion amount of a subject based on the image data in time series; and

image data in which the movement amount of the subject is smaller than a predetermined movement amount is erased.

List of reference signs

1 electronic device

2 CIS

3 AP

31 AP control unit

32 communication I/F

4 display operation unit

5 image forming unit

51 light receiving unit

52 read-out unit

6 memory

7 communication I/F

8 image selection unit

81 designation unit

82 detection unit

83 Erase Unit