阅读说明：本技术 图像获取方法、成像装置、电子设备及可读存储介质 (Image acquisition method, imaging device, electronic device, and readable storage medium ) 是由 杨鑫 于 2020-03-11 设计创作，主要内容包括：本申请公开了一种图像获取方法、成像装置、电子设备及非易失性计算机可读存储介质。图像获取方法包括：控制像素阵列曝光,同一子单元中至少一个单颜色感光像素以第一曝光时间曝光,至少一个单颜色感光像素以小于第一曝光时间的第二曝光时间曝光,至少一个全色感光像素以小于第一曝光时间的第三曝光时间曝光；根据第一全色原始图像对第一彩色原始图像及第二彩色原始图像进行插值,并将插值处理后的图像与第一全色原始图像融合以得到分辨率与像素阵列的分辨率相等的目标图像,第一彩色原始图像、第二彩色原始图像、第一全色原始图像分别由以不同曝光时间曝光的感光像素生成的信息得到。(The application discloses an image acquisition method, an imaging device, an electronic device and a non-volatile computer-readable storage medium. The image acquisition method comprises the following steps: controlling the pixel array to expose, wherein at least one single-color photosensitive pixel in the same subunit is exposed with a first exposure time, at least one single-color photosensitive pixel is exposed with a second exposure time which is less than the first exposure time, and at least one full-color photosensitive pixel is exposed with a third exposure time which is less than the first exposure time; the method comprises the steps of interpolating a first color original image and a second color original image according to a first panchromatic original image, fusing the interpolated images with the first panchromatic original image to obtain a target image with the resolution equal to that of a pixel array, and obtaining the first color original image, the second color original image and the first panchromatic original image by information generated by photosensitive pixels exposed at different exposure times.)

1. An image acquisition method for use in an image sensor comprising an array of pixels including a plurality of panchromatic photosensitive pixels and a plurality of color photosensitive pixels having a narrower spectral response than the panchromatic photosensitive pixels, the array of pixels including minimal repeating units, each of the minimal repeating units comprising a plurality of sub-units, each of the sub-units comprising a plurality of single-color photosensitive pixels and a plurality of panchromatic photosensitive pixels; the image acquisition method comprises the following steps:

controlling the pixel array exposure, wherein, for a plurality of photosensitive pixels in the same subunit, at least one of the single-color photosensitive pixels is exposed with a first exposure time, at least one of the single-color photosensitive pixels is exposed with a second exposure time that is less than the first exposure time, and at least one of the panchromatic photosensitive pixels is exposed with a third exposure time that is less than the first exposure time; and

and interpolating a first color original image and a second color original image from a first panchromatic original image, and fusing the interpolated image with the first panchromatic original image to obtain a target image having a resolution identical to that of the pixel array, wherein the first color original image is obtained from first color information generated by the single-color photosensitive pixels exposed at the first exposure time, the second color original image is obtained from second color information generated by the single-color photosensitive pixels exposed at the second exposure time, and the first panchromatic original image is obtained from first panchromatic information generated by the panchromatic photosensitive pixels exposed at the third exposure time.

2. The image acquisition method of claim 1 wherein all of the panchromatic photosensitive pixels are exposed at the third exposure time; the interpolating a first color original image and a second color original image according to a first panchromatic original image and fusing the interpolated image with the first panchromatic original image to obtain a target image having a resolution same as that of the pixel array includes:

performing interpolation processing on the first color original image to obtain a first color intermediate image having a resolution smaller than that of the pixel array, and performing interpolation processing on the second color original image to obtain a second color intermediate image having a resolution smaller than that of the pixel array;

performing interpolation processing on the first panchromatic original image to obtain a first panchromatic intermediate image having a resolution equal to that of the pixel array;

performing brightness alignment processing on the first color intermediate image and the second color intermediate image to obtain a brightness-aligned first color intermediate image;

fusing the first color intermediate image and the second color intermediate image after brightness alignment to obtain a color initial merged image;

interpolating the color initial merged image from the first panchromatic intermediate image to obtain a color intermediate merged image having a resolution equal to a resolution of the pixel array; and

fusing the color intermediate merged image and the first panchromatic intermediate image to obtain the target image.

3. The image acquisition method according to claim 1, wherein a portion of the panchromatic photosensitive pixels in the same subunit are exposed for a fourth exposure time, the remaining panchromatic photosensitive pixels being exposed for the third exposure time, the fourth exposure time being less than or equal to the first exposure time and greater than the third exposure time; the interpolating a first color original image and a second color original image according to a first panchromatic original image and fusing the interpolated image with the first panchromatic original image to obtain a target image having a resolution same as that of the pixel array includes:

performing interpolation processing on the first color original image to obtain a first color intermediate image having a resolution smaller than that of the pixel array, and performing interpolation processing on the second color original image to obtain a second color intermediate image having a resolution smaller than that of the pixel array;

performing interpolation processing on the first panchromatic original image to obtain a first panchromatic intermediate image having a resolution equal to that of the pixel array, and performing interpolation processing on a second panchromatic original image to obtain a second panchromatic intermediate image having a resolution equal to that of the pixel array, wherein the second panchromatic original image is obtained from second panchromatic information generated by the panchromatic photosensitive pixels exposed at the fourth exposure time;

performing brightness alignment processing on the first color intermediate image and the second color intermediate image to obtain a brightness-aligned first color intermediate image;

performing a luminance alignment process on the first panchromatic intermediate image and the second panchromatic intermediate image to obtain a luminance-aligned second panchromatic intermediate image;

fusing the first color intermediate image and the second color intermediate image after brightness alignment to obtain a color initial merged image;

fusing the first panchromatic intermediate image and the brightness-aligned second panchromatic intermediate image to obtain a panchromatic merged image;

interpolating the color initial merged image from the panchromatic merged image to obtain a color intermediate merged image having a resolution equal to a resolution of the pixel array; and

fusing the color intermediate merged image and the panchromatic merged image to obtain the target image.

4. The image acquisition method according to claim 2 or 3, wherein when all of the panchromatic photosensitive pixels are exposed with the third exposure time, the exposure proceeding time of all of the single-color photosensitive pixels exposed with the second exposure time is within the exposure proceeding time of all of the single-color photosensitive pixels exposed with the first exposure time, and the exposure proceeding time of all of the panchromatic photosensitive pixels exposed with the third exposure time is within the exposure proceeding time of all of the single-color photosensitive pixels exposed with the first exposure time;

when a part of the panchromatic photosensitive pixels in the same subunit are exposed in a fourth exposure time and the rest of the panchromatic photosensitive pixels are exposed in the third exposure time, the exposure proceeding time of all the single-color photosensitive pixels exposed in the second exposure time is within the exposure proceeding time of all the single-color photosensitive pixels exposed in the first exposure time, the exposure proceeding time of all the panchromatic photosensitive pixels exposed in the third exposure time is within the exposure proceeding time of all the single-color photosensitive pixels exposed in the first exposure time, and the exposure proceeding time of all the single-color photosensitive pixels exposed in the fourth exposure time is within the exposure proceeding time of all the single-color photosensitive pixels exposed in the first exposure time.

5. The image capturing method according to claim 2 or 3, wherein the performing a brightness alignment process on the first color intermediate image and the second color intermediate image to obtain the brightness-aligned first color intermediate image includes:

identifying overexposed image pixels of which the pixel values are greater than a first preset threshold value in the first color intermediate image;

for each overexposed image pixel, expanding a predetermined area by taking the overexposed image pixel as a center;

searching for intermediate image pixels with pixel values smaller than the first preset threshold value in the preset area; and

correcting the pixel value of the overexposed image pixel by using the intermediate image pixel and the second color intermediate image; and

and updating the first color intermediate image by using the corrected pixel values of the pixels of the overexposed image to obtain the first color intermediate image with aligned brightness.

6. The image acquisition method according to claim 3, wherein said performing a luminance alignment process on the first panchromatic intermediate image and the second panchromatic intermediate image to obtain a luminance-aligned second panchromatic intermediate image comprises:

identifying overexposed image pixels in the second panchromatic intermediate image having pixel values greater than a second preset threshold;

for each overexposed image pixel, expanding a predetermined area by taking the overexposed image pixel as a center;

searching for intermediate image pixels with pixel values smaller than the second preset threshold value in the preset area; and

correcting pixel values of the overexposed image pixels using the intermediate image pixels and the first panchromatic intermediate image; and

updating the second panchromatic intermediate image with the corrected pixel values of the overexposed image pixels to obtain a luminance-aligned second panchromatic intermediate image.

7. The image capturing method according to claim 2 or 3, wherein the fusing the first color intermediate image and the second color intermediate image after the brightness alignment to obtain a color initial merged image comprises:

performing motion detection on the first color intermediate image after brightness alignment;

fusing the first color intermediate image and the second color intermediate image after brightness alignment to obtain the color initial combined image when the first color intermediate image after brightness alignment does not have a motion blur area;

and when the motion blurred region exists in the first color intermediate image after the brightness alignment, fusing the region except the motion blurred region in the first color intermediate image after the brightness alignment and the second color intermediate image to obtain the color initial combined image.

8. The image acquisition method of claim 3, wherein said fusing the first panchromatic intermediate image and the brightness-aligned second panchromatic intermediate image to obtain a panchromatic merged image comprises:

performing motion detection on the second panchromatic intermediate image after brightness alignment;

fusing the first panchromatic intermediate image and the second panchromatic intermediate image after luminance alignment to obtain the panchromatic merged image when there is no motion blur region in the second panchromatic intermediate image after luminance alignment;

when the motion blur area exists in the second panchromatic intermediate image after the brightness alignment, fusing the first panchromatic intermediate image and the area except the motion blur area in the second panchromatic intermediate image after the brightness alignment to obtain the panchromatic combined image.

9. An image forming apparatus, comprising:

an image sensor comprising a pixel array comprising a plurality of panchromatic photosensitive pixels and a plurality of color photosensitive pixels, the color-sensitive pixels having a narrower spectral response than the panchromatic sensitive pixels, the array of pixels including minimal repeating units, each of the minimal repeating units including a plurality of sub-units, each of the sub-units including a plurality of single-color-sensitive pixels and a plurality of panchromatic-sensitive pixels, the array of pixels in the image sensor exposed, wherein for a plurality of light-sensitive pixels in the same subunit, at least one said single-color light-sensitive pixel is exposed for a first exposure time, at least one said single-color light-sensitive pixel is exposed for a second exposure time less than said first exposure time, and at least one said panchromatic light-sensitive pixel is exposed for a third exposure time less than said first exposure time; and

a processor for interpolating a first color raw image and a second color raw image from a first panchromatic raw image, the first color raw image being derived from first color information generated by the single-color photosensitive pixels exposed at the first exposure time, and fusing the interpolated image with the first panchromatic raw image to obtain a target image having a resolution identical to that of the pixel array, the second color raw image being derived from second color information generated by the single-color photosensitive pixels exposed at the second exposure time, the first panchromatic raw image being derived from first panchromatic information generated by the panchromatic photosensitive pixels exposed at the third exposure time.

10. The imaging apparatus of claim 9, wherein all of the panchromatic photosensitive pixels are exposed at the third exposure time; the processor is further configured to:

performing interpolation processing on the first color original image to obtain a first color intermediate image having a resolution smaller than that of the pixel array, and performing interpolation processing on the second color original image to obtain a second color intermediate image having a resolution smaller than that of the pixel array;

performing interpolation processing on the first panchromatic original image to obtain a first panchromatic intermediate image having a resolution equal to that of the pixel array;

performing brightness alignment processing on the first color intermediate image and the second color intermediate image to obtain a brightness-aligned first color intermediate image;

fusing the first color intermediate image and the second color intermediate image after brightness alignment to obtain a color initial merged image;

interpolating the color initial merged image from the first panchromatic intermediate image to obtain a color intermediate merged image having a resolution equal to a resolution of the pixel array; and

fusing the color intermediate merged image and the first panchromatic intermediate image to obtain the target image.

11. The imaging apparatus of claim 9, wherein a portion of the panchromatic photosensitive pixels in the same subunit are exposed to light for a fourth exposure time, the remaining panchromatic photosensitive pixels being exposed to light for the third exposure time, the fourth exposure time being less than or equal to the first exposure time and greater than the third exposure time; the processor is further configured to:

performing interpolation processing on the first color original image to obtain a first color intermediate image having a resolution smaller than that of the pixel array, and performing interpolation processing on the second color original image to obtain a second color intermediate image having a resolution smaller than that of the pixel array;

performing interpolation processing on the first panchromatic original image to obtain a first panchromatic intermediate image having a resolution equal to that of the pixel array, and performing interpolation processing on a second panchromatic original image to obtain a second panchromatic intermediate image having a resolution equal to that of the pixel array, wherein the second panchromatic original image is obtained from second panchromatic information generated by the panchromatic photosensitive pixels exposed at the fourth exposure time;

performing brightness alignment processing on the first color intermediate image and the second color intermediate image to obtain a brightness-aligned first color intermediate image;

performing a luminance alignment process on the first panchromatic intermediate image and the second panchromatic intermediate image to obtain a luminance-aligned second panchromatic intermediate image;

fusing the first color intermediate image and the second color intermediate image after brightness alignment to obtain a color initial merged image;

fusing the first panchromatic intermediate image and the brightness-aligned second panchromatic intermediate image to obtain a panchromatic merged image;

interpolating the color initial merged image from the panchromatic merged image to obtain a color intermediate merged image having a resolution equal to a resolution of the pixel array; and

fusing the color intermediate merged image and the panchromatic merged image to obtain the target image.

12. The image forming apparatus according to claim 10 or 11, wherein when all of the full-color photosensitive pixels are exposed with the third exposure time, an exposure proceeding time of all of the single-color photosensitive pixels exposed with the second exposure time is within an exposure proceeding time of all of the single-color photosensitive pixels exposed with the first exposure time, and an exposure proceeding time of all of the full-color photosensitive pixels exposed with the third exposure time is within an exposure proceeding time of all of the single-color photosensitive pixels exposed with the first exposure time;

when a part of the panchromatic photosensitive pixels in the same subunit are exposed in a fourth exposure time and the rest of the panchromatic photosensitive pixels are exposed in the third exposure time, the exposure proceeding time of all the single-color photosensitive pixels exposed in the second exposure time is within the exposure proceeding time of all the single-color photosensitive pixels exposed in the first exposure time, the exposure proceeding time of all the panchromatic photosensitive pixels exposed in the third exposure time is within the exposure proceeding time of all the single-color photosensitive pixels exposed in the first exposure time, and the exposure proceeding time of all the single-color photosensitive pixels exposed in the fourth exposure time is within the exposure proceeding time of all the single-color photosensitive pixels exposed in the first exposure time.

13. The imaging apparatus of claim 10 or 11, wherein the processor is further configured to:

identifying overexposed image pixels of which the pixel values are greater than a first preset threshold value in the first color intermediate image;

for each overexposed image pixel, expanding a predetermined area by taking the overexposed image pixel as a center;

searching for intermediate image pixels with pixel values smaller than the first preset threshold value in the preset area; and

correcting the pixel value of the overexposed image pixel by using the intermediate image pixel and the second color intermediate image; and

and updating the first color intermediate image by using the corrected pixel values of the pixels of the overexposed image to obtain the first color intermediate image with aligned brightness.

14. The imaging apparatus of claim 11, wherein the processor is further configured to:

identifying overexposed image pixels in the second panchromatic intermediate image having pixel values greater than a second preset threshold;

for each overexposed image pixel, expanding a predetermined area by taking the overexposed image pixel as a center;

searching for intermediate image pixels with pixel values smaller than the second preset threshold value in the preset area; and

correcting pixel values of the overexposed image pixels using the intermediate image pixels and the first panchromatic intermediate image; and

updating the second panchromatic intermediate image with the corrected pixel values of the overexposed image pixels to obtain a luminance-aligned second panchromatic intermediate image.

15. The imaging apparatus of claim 10 or 11, wherein the processor is further configured to:

performing motion detection on the first color intermediate image after brightness alignment;

fusing the first color intermediate image and the second color intermediate image after brightness alignment to obtain the color initial combined image when the first color intermediate image after brightness alignment does not have a motion blur area;

and when the motion blurred region exists in the first color intermediate image after the brightness alignment, fusing the region except the motion blurred region in the first color intermediate image after the brightness alignment and the second color intermediate image to obtain the color initial combined image.

16. The imaging apparatus of claim 11, wherein the processor is further configured to:

performing motion detection on the second panchromatic intermediate image after brightness alignment;

fusing the first panchromatic intermediate image and the second panchromatic intermediate image after luminance alignment to obtain the panchromatic merged image when there is no motion blur region in the second panchromatic intermediate image after luminance alignment;

when the motion blur area exists in the second panchromatic intermediate image after the brightness alignment, fusing the first panchromatic intermediate image and the area except the motion blur area in the second panchromatic intermediate image after the brightness alignment to obtain the panchromatic combined image.

17. The imaging apparatus of claim 9, wherein the pixel array is arranged in a two-dimensional matrix;

for any two adjacent rows of photosensitive pixels, at least one row of photosensitive pixels is provided, the control ends of the exposure control circuits of the single-color photosensitive pixels positioned in the same row are connected with a first exposure control line, the control ends of the exposure control circuits of the panchromatic photosensitive pixels are connected with a second exposure control line, and the control ends of the reset circuits of the single-color photosensitive pixels and the panchromatic photosensitive pixels are connected with a reset line; or

For any two adjacent rows of photosensitive pixels, at least one row of photosensitive pixels is provided, the control ends of the reset circuits of the single-color photosensitive pixels in the same row are connected with a first reset line, the control ends of the reset circuits of the panchromatic photosensitive pixels are connected with a second reset line, and the control ends of the exposure control circuits of the single-color photosensitive pixels and the panchromatic photosensitive pixels are connected with an exposure control line; or

Adjacent to arbitrary two lines photosensitive pixel exists the one line at least photosensitive pixel satisfies a plurality of that are located same line single colour photosensitive pixel's exposure control circuit's control end and a first exposure control line are connected, and are a plurality of panchromatic photosensitive pixel's exposure control circuit's control end and a second exposure control line are connected, and are a plurality of single colour photosensitive pixel's reset circuit's control end and a first line that resets are connected, and are a plurality of panchromatic photosensitive pixel's reset circuit's control end and a second reset line are connected.

18. An electronic device, comprising:

a housing; and

the imaging device of any one of claims 9-17, in combination with the housing.

19. A non-transitory computer-readable storage medium containing a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the image acquisition method of any one of claims 1-8.