阅读说明：本技术 滤光片的异常检测方法、装置、电子装置和存储介质 (Method and device for detecting abnormality of optical filter, electronic device, and storage medium ) 是由 王建淼 康玄烨 朱飞月 孙亮 陈金涛 于 2021-08-26 设计创作，主要内容包括：本申请涉及一种滤光片的异常检测方法、装置、电子装置和存储介质,该方法包括：在摄像机运行在第一工作模式时,确定由摄像机获取的第一环境光亮度值是否落入预设的切换阈值区间,在第一环境光亮度值落入切换阈值区间的情况下,获取第一图像数据和第一摄像参数数据,将摄像机从第一工作模式切换至第二工作模式,并获取第二图像数据和第二摄像参数数据,使用训练后的异常检测模型处理第一图像数据、第一摄像参数数据、第二图像数据和第二摄像参数数据,根据训练后的异常检测模型输出的检测结果确定滤光片是否存在异常。通过本申请,解决了相关技术中滤光片的异常检测可靠性低的问题,实现了提高对滤光片异常检测的可靠性的技术效果。(The present application relates to a method, an apparatus, an electronic apparatus, and a storage medium for detecting an abnormality of an optical filter, the method including: when the camera operates in a first working mode, whether a first environment light brightness value acquired by the camera falls into a preset switching threshold interval or not is determined, under the condition that the first environment light brightness value falls into the switching threshold interval, first image data and first shooting parameter data are acquired, the camera is switched from the first working mode to a second working mode, second image data and second shooting parameter data are acquired, the first image data, the first shooting parameter data, the second image data and the second shooting parameter data are processed by using a trained abnormity detection model, and whether the optical filter is abnormal or not is determined according to a detection result output by the trained abnormity detection model. By the method and the device, the problem of low reliability of abnormity detection of the optical filter in the related technology is solved, and the technical effect of improving the reliability of abnormity detection of the optical filter is achieved.)

1. An abnormality detection method for an optical filter, applied to abnormality detection of an optical filter in a video camera including a dual-filter switcher for switching between a first operation mode and a second operation mode in the video camera, the method comprising:

when the camera operates in a first working mode, determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval;

under the condition that the first ambient light brightness value falls into the switching threshold interval, acquiring first image data and first shooting parameter data acquired when the camera operates in a first working mode;

switching the camera from a first working mode to a second working mode, and acquiring second image data and second shooting parameter data acquired when the camera operates in the second working mode;

and processing the first image data, the first shooting parameter data, the second image data and the second shooting parameter data by using a trained abnormity detection model, and determining whether the optical filter has abnormity according to a detection result output by the trained abnormity detection model.

2. The method according to claim 1, wherein the switching threshold interval includes a first threshold interval and a second threshold interval, wherein an upper limit value of the first threshold interval and a lower limit value of the second threshold interval are preset switching thresholds;

in a case where the first operation mode is a day mode and the second operation mode is a night mode, determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval includes:

determining whether a first ambient light brightness value acquired by the camera falls within the first threshold interval;

in a case where the first operation mode is a night mode and the second operation mode is a day mode, determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval includes:

determining whether a first ambient light brightness value acquired by the camera falls within the second threshold interval.

3. The method according to claim 2, wherein in a case where the first operation mode is a day mode and the second operation mode is a night mode, before determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval, the method further comprises:

after the camera is powered on and started, configuring the camera into a second working mode;

under the condition that the camera operates in a second working mode, switching the working mode of the camera to a first working mode, and determining whether a second ambient light brightness value acquired by the camera falls into a second threshold interval;

and determining that the working mode of the camera is the first working mode under the condition that the second ambient light brightness value falls into the second threshold interval.

4. The method according to claim 2, wherein in a case where the first operation mode is a night mode and the second operation mode is a day mode, before determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval, the method further comprises:

after the camera is powered on and started, configuring the camera into a first working mode;

under the condition that the camera operates in a first working mode, switching the working mode of the camera to a second working mode, and determining whether a second ambient light brightness value acquired by the camera falls within a preset first switching threshold interval;

and switching the working mode of the camera to a first working mode under the condition that the second ambient brightness value falls into the first threshold interval.

5. The method of detecting an abnormality of an optical filter according to claim 1, further comprising:

constructing an initial machine learning model;

obtaining a test sample, wherein the test sample comprises an abnormal sample and a normal sample, and the test sample comprises image data and camera parameter data corresponding to each sample;

and inputting the image data and the camera parameter data corresponding to the abnormal sample and the image data and the camera parameter data corresponding to the normal sample into the initial machine learning model, and updating the parameters of the initial machine learning model to obtain the trained abnormal detection model.

6. The method according to claim 1, wherein the first image data includes a first RGB average value and a first luminance average value; acquiring first image data acquired by the camera while operating in a first mode of operation comprises:

acquiring a first target image acquired by the camera when operating in a first working mode;

dividing the first target image into a plurality of statistical blocks of the same size;

acquiring a first RGB value of each statistical block;

calculating to obtain a first RGB average value of the first target image according to the first RGB value of each statistical block;

and calculating to obtain a first brightness average value of the first target image according to the first RGB average value.

7. The method according to claim 6, wherein the first imaging parameter data includes a first infrared component ratio; the method further comprises the following steps:

calculating to obtain a first white balance parameter value of the statistical block according to the first RGB value of the statistical block;

determining a statistical block corresponding to a first white balance parameter value with a distance from a preset white balance parameter threshold value smaller than a preset first threshold value as a first target statistical block, wherein the white balance parameter threshold value is a white balance parameter value of a preset image shot by the camera in a pure infrared environment;

and determining the ratio of the number of the first target statistical blocks to the number of all statistical blocks in the first target image as a first infrared component ratio acquired when the camera operates in a first working mode.

8. The method according to claim 1, wherein the second image data includes a second RGB average value and a second luminance average value; acquiring second image data acquired by the camera while operating in a second mode of operation comprises:

acquiring a second target image acquired by the camera when operating in a second working mode;

dividing the second target image into a plurality of statistical blocks of the same size;

acquiring a second RGB value of each statistical block;

calculating to obtain a second RGB average value of the second target image according to the second RGB value of each statistical block;

and calculating to obtain a second brightness average value of the second target image according to the second RGB average value.

9. The method according to claim 8, wherein the second imaging parameter data includes a second infrared component ratio; the method further comprises the following steps:

calculating to obtain a second white balance parameter value of the statistical block according to the second RGB value of the statistical block;

determining a statistical block corresponding to a second white balance parameter value with a distance from a preset white balance parameter threshold value smaller than a preset first threshold value as a second target statistical block, wherein the white balance parameter threshold value is a white balance parameter value of a preset image shot by the camera under a pure infrared environment;

and determining the ratio of the number of the second target statistical blocks to the number of all statistical blocks in the second target image as a second infrared component ratio acquired when the camera operates in a second working mode.

10. An apparatus for detecting abnormality of an optical filter, applied to abnormality detection of an optical filter in a video camera including a dual-filter switcher for switching between a first operation mode and a second operation mode in the video camera, the apparatus comprising:

the judging module is used for determining whether a first ambient light brightness value acquired by the camera falls into a preset switching threshold interval or not when the camera operates in a first working mode;

the first acquisition module is used for acquiring first image data and first shooting parameter data which are acquired when the camera operates in a first working mode under the condition that the first ambient light brightness value falls into the switching threshold interval;

the second acquisition module is used for switching the camera from the first working mode to the second working mode and acquiring second image data and second shooting parameter data acquired when the camera operates in the second working mode;

and the output module is used for processing the first image data, the first shooting parameter data, the second image data and the second shooting parameter data by using the trained abnormity detection model and determining whether the optical filter has abnormity according to a detection result output by the trained abnormity detection model.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the method for detecting an abnormality of an optical filter according to any one of claims 1 to 9.

12. A storage medium having a computer program stored therein, wherein the computer program when executed by a processor implements the method for detecting an abnormality of an optical filter according to any one of claims 1 to 9.

Technical Field

The present disclosure relates to the field of camera technologies, and in particular, to a method and an apparatus for detecting an abnormality of an optical filter, an electronic apparatus, and a storage medium.

Background

With the development and progress of science and technology, many scenes now require all-weather monitoring. The light is sufficient daytime, and the colour of image is also guaranteed to be true reduction as accurate as possible when luminance is sufficient, and general image sensor can both respond to near-infrared composition, leads to the color distortion, consequently need filter the infrared composition with the help of infrared cut-off filter, guarantees the authenticity of camera color reduction for colored scene shoots.

When the ambient illumination is insufficient at night, an infrared band-pass filter is needed so that the image sensor can receive light as much as possible to ensure that the brightness of the image is sufficient, and the night vision brightness of the camera at night is increased for shooting of black and white scenes. Therefore, the optical filter is very important for the camera, and if the optical filter of the camera is switched abnormally or fails in the process of leaving a factory or operating, an image is abnormal, so that the monitoring effect and the user experience are affected, so that the normal operation of the optical filter needs to be ensured in the process of leaving a factory or operating the camera.

At present, in the method for detecting an abnormal optical filter in the related art, a scanning black edge existing in the optical filter switching process is calculated and judged, and it is determined whether a scanning black edge not smaller than a preset scanning black edge judgment threshold exists in an image obtained by a camera, so as to judge whether the current optical filter is working normally. However, in such solutions, because the current mainstream optical filter does not generate a black edge in the switching process, such solutions can only be adapted to specific devices, and both the accuracy and the reliability are low.

At present, no effective solution is provided for the problem of low reliability of the anomaly detection of the optical filter in the related technology.

Disclosure of Invention

The embodiment of the application provides an optical filter abnormality detection method, an optical filter abnormality detection device, an electronic device and a storage medium, and aims to at least solve the problem that the reliability of optical filter abnormality detection in the related art is low.

In a first aspect, an embodiment of the present application provides a method for detecting an abnormality of an optical filter, which is applied to detect an abnormality of an optical filter in a camera including a dual-optical-filter switcher, where the dual-optical-filter switcher is used for switching between a first operating mode and a second operating mode in the camera, and the method includes: when the camera operates in a first working mode, determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval; under the condition that the first ambient light brightness value falls into the switching threshold interval, acquiring first image data and first shooting parameter data acquired when the camera operates in a first working mode; switching the camera from a first working mode to a second working mode, and acquiring second image data and second shooting parameter data acquired when the camera operates in the second working mode; and processing the first image data, the first shooting parameter data, the second image data and the second shooting parameter data by using a trained abnormity detection model, and determining whether the optical filter has abnormity according to a detection result output by the trained abnormity detection model.

In some embodiments, the switching threshold interval includes a first threshold interval and a second threshold interval, where an upper limit of the first threshold interval and a lower limit of the second threshold interval are preset switching thresholds; in a case where the first operation mode is a day mode and the second operation mode is a night mode, determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval includes: determining whether a first ambient light brightness value acquired by the camera falls within the first threshold interval; in a case where the first operation mode is a night mode and the second operation mode is a day mode, determining whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval includes: determining whether a first ambient light brightness value acquired by the camera falls within the second threshold interval.

In some embodiments, in a case where the first operation mode is a day mode and the second operation mode is a night mode, before determining whether the first ambient light brightness value acquired by the camera falls within a preset switching threshold interval, the method further includes: after the camera is powered on and started, configuring the camera into a second working mode; under the condition that the camera operates in a second working mode, switching the working mode of the camera to a first working mode, and determining whether a second ambient light brightness value acquired by the camera falls into a second threshold interval; and determining that the working mode of the camera is the first working mode under the condition that the second ambient light brightness value falls into the second threshold interval.

In some embodiments, in a case where the first operation mode is a night mode and the second operation mode is a day mode, before determining whether the first ambient light brightness value acquired by the camera falls within a preset switching threshold interval, the method further includes: after the camera is powered on and started, configuring the camera into a first working mode; under the condition that the camera operates in a first working mode, switching the working mode of the camera to a second working mode, and determining whether a second ambient light brightness value acquired by the camera falls within the first threshold interval; and switching the working mode of the camera to a first working mode under the condition that the second ambient brightness value falls into the first threshold interval.

In some of these embodiments, the method further comprises: constructing an initial machine learning model; obtaining a test sample, wherein the test sample comprises an abnormal sample and a normal sample, and the test sample comprises image data and camera parameter data corresponding to each sample; and inputting the image data and the camera parameter data corresponding to the abnormal sample and the image data and the camera parameter data corresponding to the normal sample into the initial machine learning model, and updating the parameters of the initial machine learning model to obtain the trained abnormal detection model.

In some of these embodiments, the first image data comprises a first RGB average value and a first luminance average value; acquiring first image data acquired by the camera while operating in a first operating mode comprises acquiring a first target image acquired by the camera while operating in the first operating mode; dividing the first target image into a plurality of statistical blocks of the same size; acquiring a first RGB value of each statistical block; calculating to obtain a first RGB average value of the first target image according to the first RGB value of each statistical block; and calculating to obtain a first brightness average value of the first target image according to the first RGB average value.

In some of these embodiments, the first imaging parameter data includes a first infrared component ratio; the method further comprises the following steps: calculating to obtain a first white balance parameter value of the statistical block according to the first RGB value of the statistical block; determining a statistical block corresponding to a first white balance parameter value with a distance from a preset white balance parameter threshold value smaller than a preset first threshold value as a first target statistical block, wherein the white balance parameter threshold value is a white balance parameter value of a preset image shot by the camera in a pure infrared environment; and determining the ratio of the number of the first target statistical blocks to the number of all statistical blocks in the first target image as a first infrared component ratio acquired when the camera operates in a first working mode.

In some of these embodiments, the second image data comprises a second RGB average value and a second luminance average value; acquiring second image data acquired by the camera while operating in a second mode of operation comprises: acquiring a second target image acquired by the camera when operating in a second working mode; dividing the second target image into a plurality of statistical blocks of the same size; acquiring a second RGB value of each statistical block; calculating to obtain a second RGB average value of the second target image according to the second RGB value of each statistical block; and calculating to obtain a second brightness average value of the second target image according to the second RGB average value.

In some of these embodiments, the second imaging parameter data includes a second infrared component ratio; the method further comprises the following steps: calculating to obtain a second white balance parameter value of the statistical block according to the second RGB value of the statistical block; determining a statistical block corresponding to a second white balance parameter value with a distance from a preset white balance parameter threshold value smaller than a preset first threshold value as a second target statistical block, wherein the white balance parameter threshold value is a white balance parameter value of a preset image shot by the camera under a pure infrared environment; and determining the ratio of the number of the second target statistical blocks to the number of all statistical blocks in the second target image as a second infrared component ratio acquired when the camera operates in a second working mode.

In a second aspect, an embodiment of the present application provides an apparatus for detecting an abnormality of an optical filter, which is applied to detect an abnormality of an optical filter in a camera including a dual-optical-filter switcher, wherein the dual-optical-filter switcher is used for switching between a first operation mode and a second operation mode in the camera, the apparatus includes: the judging module is used for determining whether a first ambient light brightness value acquired by the camera falls into a preset switching threshold interval or not when the camera operates in a first working mode; the first acquisition module is used for acquiring first image data and first shooting parameter data which are acquired when the camera operates in a first working mode under the condition that the first ambient light brightness value falls into the switching threshold interval; the second acquisition module is used for switching the camera from the first working mode to the second working mode and acquiring second image data and second shooting parameter data acquired when the camera operates in the second working mode; and the output module is used for processing the first image data, the first shooting parameter data, the second image data and the second shooting parameter data by using the trained abnormity detection model and determining whether the optical filter has abnormity according to a detection result output by the trained abnormity detection model.

In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program, and the processor is configured to run the computer program to execute the method for detecting an abnormality of an optical filter according to the first aspect.

In a fourth aspect, an embodiment of the present application further provides a storage medium, in which a computer program is stored, where the computer program, when executed by a processor, implements the method for detecting an abnormality of an optical filter according to the first aspect.

Compared with the related art, the method, the device, the electronic device, and the storage medium for detecting the abnormality of the optical filter provided in the embodiments of the present application determine whether a first ambient light brightness value obtained by the camera falls within a preset switching threshold interval when the camera operates in the first operating mode, obtain first image data and first imaging parameter data obtained when the camera operates in the first operating mode when the first ambient light brightness value falls within the switching threshold interval, switch the camera from the first operating mode to the second operating mode, obtain second image data and second imaging parameter data obtained when the camera operates in the second operating mode, process the first image data, the first imaging parameter data, the second image data, and the second imaging parameter data using the trained abnormality detection model, determine whether the optical filter has an abnormality according to a detection result output by the trained abnormality detection model, the problem of low reliability of optical filter anomaly detection in the related art is solved, and the technical effect of improving the reliability of optical filter anomaly detection is achieved.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a flowchart of an abnormality detection method of an optical filter according to an embodiment of the present application;

fig. 2 is a flowchart of an abnormality detection method of an optical filter according to a preferred embodiment of the present application;

fig. 3 is a block diagram showing the structure of an abnormality detection device for an optical filter according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.

Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.

Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Reference herein to "a plurality" means greater than or equal to two. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.

The present embodiment provides a method for detecting an abnormality of an optical filter, which is applied to detecting an abnormality of an optical filter in a camera including a dual-optical-filter switcher, wherein the dual-optical-filter switcher is used for switching between a first operating mode and a second operating mode in the camera, and fig. 1 is a flowchart of a method for detecting an abnormality of an optical filter according to an embodiment of the present application, as shown in fig. 1, the method includes:

step S101, when the camera operates in the first working mode, determining whether the first ambient light brightness value acquired by the camera falls into a preset switching threshold interval.

In this embodiment, the camera may include an infrared cut filter and an infrared band pass filter, and when the camera operates in a daytime mode (i.e., when facing a color scene requirement), the dual filter switcher switches the filter of the camera to the infrared cut filter; when the camera operates in a night mode (namely when meeting the requirements of a black-and-white scene), the dual-filter switcher switches the filter of the camera to the infrared band-pass filter, wherein the first working mode can be a day mode or a night mode, and when the first working mode is the day mode, the second working mode is the night mode; when the first operation mode is a night mode, the second operation mode is a day mode.

Step S102, under the condition that the first environment light brightness value falls into the switching threshold value interval, acquiring first image data and first image pickup parameter data acquired when the camera operates in the first working mode.

In this embodiment, the first ambient light brightness value may be obtained by the following formula:

wherein, ev_curIs a first ambient light brightness value, brightness_curThe average image brightness value of the image shot by the camera is indicated, the shutterCur is the current shutter time of the camera, the gainCur is the current gain value of the camera, and the irRation is the current infrared component ratio detected by the camera.

The switching threshold interval includes a first threshold interval and a second threshold interval, wherein an upper limit value of the first threshold interval and a lower limit value of the second threshold interval are preset switching threshold color2black _ thr, and the switching threshold color2black _ thr may be set according to actual needs or set according to experimental data.

In the above-described embodiment, in the case where the first operation mode is the day mode and the second operation mode is the night mode, it may be determined whether the first ambient light luminance value falls within the first threshold interval (— infinity, color2black _ thr), that is, when ev_curLess than color2black _ thr may determine that the camera currently satisfies the condition for switching from the first operation mode (i.e., day mode) to the second operation mode (i.e., night mode).

In a case where the first operation mode is a night mode and the second operation mode is a day mode, it may be determined whether the first ambient light brightness value falls within a rangeThe second threshold interval is (color2black _ thr, + ∞), i.e. when ev_curGreater than color2black _ thr may determine that the camera currently satisfies the condition for switching from the first operating mode (i.e., night mode) to the second operating mode (i.e., day mode).

And step S103, switching the camera from the first working mode to the second working mode, and acquiring second image data and second shooting parameter data acquired when the camera operates in the second working mode.

In this embodiment, the infrared cut filter is used to filter a part of infrared light, generally infrared light over 700nm, so as to ensure the authenticity of color; the infrared band-pass filter is a full-transmission band, so that all light sources can enter an image sensor in the camera, and the image brightness can be guaranteed.

The information that the image sensor of camera obtained includes a plurality of pixel, and every pixel all contains its corresponding information under R, G, B three passageways, and RGB data based on that the image sensor obtained can carry out a plurality of dimensions such as luminance, infrared composition, ultraviolet composition to current environment and carry out statistics and analysis, utilizes mathematical model can analyze luminance information and infrared composition ratio in the current environment.

And step S104, processing the first image data, the first shooting parameter data, the second image data and the second shooting parameter data by using the trained abnormity detection model, and determining whether the optical filter has abnormity according to a detection result output by the trained abnormity detection model.

In the embodiment, the infrared cut-off filter filters out infrared light, so that when the camera operates in a daytime mode, the infrared component ratio of an image shot by the camera is low, and the brightness is low; on the other hand, the infrared band-pass filter does not filter infrared light, and therefore, when the camera operates in the night mode, the infrared component ratio of an image shot by the camera is high, and the brightness is high.

According to the rule, the trained abnormity detection model is used for processing the first image data, the first shooting parameter data, the second image data and the second shooting parameter data, whether the optical filter is abnormal or not is determined according to the detection result output by the trained abnormity detection model, whether the optical filter is abnormal or not is identified in a self-adaptive mode by switching the infrared cut-off optical filter and the infrared band-pass optical filter in the camera, the detection speed of a production line is improved, and the reliability of optical filter abnormity detection is improved.

Through the above steps S101 to S104, when the camera operates in the first working mode, it is determined whether the first ambient light brightness value acquired by the camera falls within the preset switching threshold interval, and when the first ambient light brightness value falls within the switching threshold interval, the first image data and the first imaging parameter data acquired when the camera operates in the first working mode are acquired, the camera is switched from the first working mode to the second working mode, the second image data and the second imaging parameter data acquired when the camera operates in the second working mode are acquired, the first image data, the first imaging parameter data, the second image data and the second imaging parameter data are processed by using the trained abnormality detection model, and whether the optical filter is abnormal is determined according to the detection result output by the trained abnormality detection model. By the method and the device, the problem of low reliability of abnormity detection of the optical filter in the related technology is solved, and the technical effect of improving the reliability of abnormity detection of the optical filter is achieved.

In some embodiments, the training process of the trained anomaly detection pattern includes the following steps:

step 1, constructing an initial machine learning model.

And 2, obtaining a test sample, wherein the test sample comprises an abnormal sample and a normal sample, and the test sample comprises image data and camera parameter data corresponding to each sample.

And 3, inputting the image data and the camera parameter data corresponding to the abnormal sample and the image data and the camera parameter data corresponding to the normal sample into the initial machine learning model, and updating the parameters of the initial machine learning model to obtain the trained abnormal detection model.

In this embodiment, an initial Machine learning model may select a Support Vector Machine (SVM) model, the SVM model is a generalized linear classifier that performs binary classification on data according to a supervised learning manner, a decision boundary of the SVM model is a maximum edge distance hyperplane for solving a learning sample, the SVM model is suitable for an application situation with many sample features such as images and texts, and the SVM model has a good effect in solving problems such as pattern recognition, classification and regression analysis.

In the above embodiment, a Radial Basis Function (RBF) and a Support Vector Classification (C-supported Vector Classification, C _ SVC) may be used as an initial machine learning model, image data and imaging parameter data corresponding to an abnormal sample in a test sample, and image data and imaging parameter data corresponding to a normal sample are used as inputs, parameters such as a maximum threshold value and a minimum threshold value of an obtained value are obtained through training, and a trained anomaly detection model is obtained through training.

In this embodiment, the trained anomaly detection model may process the first image data, the first imaging parameter data, the second image data, and the second imaging parameter data, and output a detection result corresponding to the optical filter, where the detection result includes both anomaly and normal, and perform alarm processing when the detection result is abnormal; and under the condition that the detection result is normal, continuing to monitor the optical filter.

In some embodiments, in the case that the first operation mode is a day mode and the second operation mode is a night mode, before determining whether the first ambient light brightness value acquired by the camera falls within a preset switching threshold interval, the method further performs the following steps:

step 1, after the camera is powered on and started, the camera is configured to be in a second working mode.

And 2, under the condition that the camera operates in the second working mode, switching the working mode of the camera to the first working mode, and determining whether the second ambient light brightness value acquired by the camera falls into a second threshold interval.

And 3, determining the working mode of the camera to be the first working mode under the condition that the second ambient light brightness value falls into the second threshold interval.

In this embodiment, after the camera is powered on and started, the operating mode of the camera needs to be switched back and forth once, that is, the infrared cut-off filter and the infrared band-pass filter are both switched once, so as to prevent the default filter from being out of the correct position due to transportation or other reasons (for example, the infrared cut-off filter is configured when the camera operates in a night mode).

In the above embodiment, after the camera is powered on and started, the optical filter of the camera may be forcibly switched to the infrared band-pass filter and then to the infrared cut-off filter by using a preset software design, and the optical filter of the camera is ensured to be in the correct position by self-checking.

At this time, it is determined whether the current ambient light brightness value (i.e., the second ambient light brightness value) falls within a second threshold interval (color2black _ thr, + ∞), and when the second ambient light brightness value is greater than the switching threshold color2black _ thr, the operation mode of the camera is determined to be the day mode, that is, the operation mode of the camera is maintained to be the first operation mode.

In this embodiment, when the first operation mode is the night mode and the second operation mode is the day mode, the self-test is performed by the following steps:

step 1, after the camera is powered on and started, the camera is configured to be in a first working mode.

And 2, under the condition that the camera operates in the first working mode, switching the working mode of the camera to a second working mode, and determining whether the second ambient light brightness value acquired by the camera falls into a first threshold interval.

And 3, switching the working mode of the camera to the first working mode under the condition that the second ambient light brightness value falls into the first threshold interval.

In the above embodiment, by determining whether the current ambient light brightness value (i.e. the second ambient light brightness value) falls within the first threshold interval (— infinity, color2black _ thr), in the case that the second ambient light brightness value is smaller than the switching threshold color2black _ thr, the operation mode of the camera is determined to be the night mode, that is, the operation mode of the camera is switched from the second operation mode to the first operation mode.

In some of these embodiments, the first image data comprises a first RGB average value and a first luminance average value; acquiring first image data acquired by the camera while operating in the first mode of operation is accomplished by:

step 1, acquiring a first target image acquired by a camera when the camera operates in a first working mode.

And 2, dividing the first target image into a plurality of statistical blocks with the same size.

And 3, acquiring a first RGB value of each statistical block.

And 4, calculating to obtain a first RGB average value of the first target image according to the first RGB value of each statistical block.

And 5, calculating to obtain a first brightness average value of the first target image according to the first RGB average value.

In this embodiment, the first target image may be divided into M × N statistical blocks with the same size, and the first RGB values of each statistical block in R, G, B channels may be recorded respectively, and may be recorded according to the position coordinates (i, j) of each statistical block in the first target image, for example, the first RGB values of the statistical blocks located in (i, j) are (R [ i ] [ j ], G [ i ] [ j ], B [ i ] [ j ]), respectively.

After counting the first RGB values of all statistical patches, a first RGB average value of the first target image is calculated, which is denoted as Ravg0, Gavg0, Bavg 0.

In the above embodiment, the first luminance average value of the first target image may be calculated by the following formula:

Yavg0＝0.2989*Ravg0+0.5866*Gavg0+0.1145*Bavg0；

wherein 0.2989, 0.5866 and 0.1145 can be selected according to actual needs, and the application is not limited herein.

In the present embodiment, the first imaging parameter data includes a first infrared component ratio; the method also implements the steps of:

step 1, calculating to obtain a first white balance parameter value of the statistical block according to the first RGB value of the statistical block.

And 2, determining a statistical block corresponding to a first white balance parameter value with a distance between the statistical block and a preset white balance parameter threshold value smaller than a preset first threshold value as a first target statistical block, wherein the white balance parameter threshold value is the white balance parameter value of a preset image shot by the camera under a pure infrared environment.

And 3, determining the ratio of the number of the first target statistical blocks to the number of all statistical blocks in the first target image as a first infrared component ratio acquired when the camera operates in the first working mode.

In the present embodiment, the first image pickup parameter data further includes a first shutter value shutter0, a first gain value gain0, and a first luminance value brightness0, and the first white balance parameter value includes a red gain value rGain and a blue gain bGain of the statistical block.

Wherein, the first white balance parameter value can be calculated by the following formula:

in this embodiment, the camera may be tested in a pure infrared environment, a preset image shot by the camera in the pure infrared environment is obtained, the white balance parameter values irRGain and irBGain of the preset image are obtained through analysis, and the white balance parameter values of the preset image are used as the preset white balance parameter threshold.

The distance X between the statistical partition and the preset white balance parameter threshold may be calculated by the following formula:

X＝|rGain-irRGain|+|bGain-irBGain|；

the first threshold value irRationThr can be set according to actual needs, the distance X between each statistical block and a preset white balance parameter threshold value is analyzed and counted, if X is smaller than irRationThr, the statistical block is considered to be close to an infrared component, the statistical block is determined to be a first target statistical block, and the total number irTotal of the first target statistical block is counted.

In the above embodiment, the first infrared component ratio acquired when the camera operates in the first operation mode may be determined by the following formula:

wherein, Total is the Total number of statistical blocks in the first target image, and irratio 0 is the first infrared component ratio obtained when the camera operates in the first working mode.

In some of these embodiments, the second image data comprises a second RGB average value and a second luminance average value; acquiring second image data acquired by the camera while operating in the second mode of operation is accomplished by:

step 1, acquiring a second target image acquired by the camera when operating in a night mode.

And 2, dividing the second target image into a plurality of statistical blocks with the same size.

And 3, acquiring a second RGB value of each statistical block.

And 4, calculating to obtain a second RGB average value of the second target image according to the second RGB value of each statistical block.

And 5, calculating to obtain a second brightness average value of the second target image according to the second RGB average value.

In this embodiment, the calculation processes of the second RBG average value Ravg1, Gavg1, Bavg1 corresponding to the second target image and the second luminance average value Yavg1 corresponding to the second target image are similar to those in the foregoing embodiment, and the calculation processes when the camera operates in the first operating mode may refer to the examples described in the foregoing embodiment and the optional embodiments, and this embodiment is not described again here.

In the present embodiment, the second imaging parameter data includes a second infrared component ratio; the method also implements the steps of:

and step 1, calculating to obtain a second white balance parameter value of the statistical block according to the second RGB value of the statistical block.

And 2, determining a statistical block corresponding to a second white balance parameter value with the distance between the statistical block and a preset white balance parameter threshold value smaller than a preset first threshold value as a second target statistical block, wherein the white balance parameter threshold value is the white balance parameter value of a preset image shot by the camera under a pure infrared environment.

And 3, determining the ratio of the number of the second target statistical blocks to the number of all statistical blocks in the second target image as a second infrared component ratio acquired when the camera operates in the second working mode.

In this embodiment, the second image capturing parameter data further includes a second shutter value shutter1, a second gain value gain1, and a second brightness value 1, wherein a calculation process of the second infrared component ratio irratio 1 is similar to that in the above embodiment, and a calculation process when the camera operates in the first operating mode may be referred to the examples described in the above embodiment and the optional embodiments, and details of this embodiment are not repeated herein.

In the above embodiment, when the first image data and the first imaging parameter data acquired when the camera operates in the first operation mode are obtained: ravg0, Gavg0, Bavg0, Yavg0, shutter0, gain0, brightness0 and irRation0, and second image data and second image parameter data Ravg1, Gavg1, Bavg1, Yavg1, shutter1, gain1, brightness1 and irRation1 which are acquired when the camera operates in the second working mode, and 16 characteristic values of the 2 groups of data are input into a trained abnormity detection model, so that whether the optical filter of the camera is abnormal or not can be determined, whether the optical filter is abnormal or not can be identified in a self-adaptive manner by utilizing the switching of an infrared cut-off filter and an infrared band-pass filter in the camera, the detection speed of the optical filter is improved, and the reliability of abnormity detection of the optical filter is improved.

Fig. 2 is a flow chart of a method for detecting an abnormality of an optical filter according to a preferred embodiment of the present application, as shown in fig. 2, which includes, in some embodiments:

in step S201, the camera is powered on and started.

Step S202, the filter of the camera is switched to the infrared band-pass filter.

In step S203, the filter of the camera is switched to the infrared cut filter.

Step S204, determining the working mode of the camera according to the current environment light brightness, and entering step S205 under the condition that the camera is determined to work in the daytime mode; in the case where it is determined that the camera is operated in the night mode, the flow proceeds to step S210.

In step S205, the filter of the camera is maintained as the infrared cut filter.

In step S206, it is determined whether the current ambient light brightness is less than a preset switching threshold.

Step S207, when the current ambient light brightness is smaller than the switching threshold, counting first image data and first image parameter data corresponding to the first target image acquired by the camera.

And step S208, switching the optical filter of the camera to an infrared band-pass optical filter.

In step S209, second image data and second imaging parameter data corresponding to the second target image acquired by the camera are counted.

Step S210, the filter of the camera is switched to the infrared band pass filter.

Step S211, determining whether the current ambient light brightness is greater than a preset switching threshold.

Step S212, when the current ambient light brightness is greater than the switching threshold, counting third image data and third image parameter data corresponding to the third target image acquired by the camera.

In step S213, the filter of the camera is switched to the infrared cut filter.

In step S214, fourth image data and fourth imaging parameter data corresponding to the fourth target image acquired by the camera are counted.

In step S215, 16 feature values of the two sets of data are obtained.

Step S216, input all 16 feature values into the trained abnormality detection model.

Step S217, a detection result output by the trained anomaly detection model is obtained.

Step S218, the detection result is that the optical filter is normal; return to step S204.

Step S219, detecting that the optical filter is abnormal; the process advances to step S218.

And step S220, sending alarm information.

It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.

The present embodiment provides an apparatus for detecting an abnormality of an optical filter, which is applied to detecting an abnormality of an optical filter in a camera including a dual-optical-filter switcher for switching between a first operation mode and a second operation mode in the camera, and fig. 3 is a block diagram of the apparatus for detecting an abnormality of an optical filter according to the embodiment of the present application, as shown in fig. 3, the apparatus includes: the judgment module 30 is configured to determine whether a first ambient light brightness value acquired by the camera falls within a preset switching threshold interval when the camera operates in the first working mode; a first obtaining module 31, configured to obtain, when the first ambient light brightness value falls within the switching threshold interval, first image data and first image capturing parameter data obtained when the camera operates in the first working mode; a second obtaining module 32, configured to switch the camera from the first working mode to a second working mode, and obtain second image data and second shooting parameter data obtained when the camera operates in the second working mode; and an output module 33, configured to process the first image data, the first image capturing parameter data, the second image data, and the second image capturing parameter data by using the trained anomaly detection model, and determine whether the optical filter is anomalous according to a detection result output by the trained anomaly detection model.

In some embodiments, the switching threshold interval includes a first threshold interval and a second threshold interval, where an upper limit value of the first threshold interval and a lower limit value of the second threshold interval are preset switching thresholds; in the case where the first operation mode is a day mode and the second operation mode is a night mode, the judging module 30 is further configured to determine whether a first ambient light brightness value acquired by the camera falls within a first threshold interval; in case the first operation mode is a night mode and the second operation mode is a day mode, the determining module 30 is further configured to determine whether the first ambient light brightness value acquired by the camera falls within a second threshold interval.

In some embodiments, the apparatus further includes a self-test module, where the self-test module is configured to configure the camera to be in a second operating mode after the camera is powered on and started when the first operating mode is a day mode and the second operating mode is a night mode; under the condition that the camera operates in the second working mode, the working mode of the camera is switched to the first working mode, and whether a second ambient light brightness value acquired by the camera falls into a second threshold interval or not is determined; and under the condition that the second ambient light brightness value falls into the second threshold interval, determining the working mode of the camera as the first working mode.

In some embodiments, the self-test module is further configured to configure the camera to be in the first operating mode after the camera is powered on and started, if the first operating mode is a night mode and the second operating mode is a day mode; under the condition that the camera operates in the first working mode, the working mode of the camera is switched to a second working mode, and whether a second ambient light brightness value acquired by the camera falls into a first threshold interval or not is determined; and under the condition that the second ambient light brightness value falls into the first threshold interval, switching the working mode of the camera to the first working mode.

In some embodiments, the apparatus further comprises a training module for constructing an initial machine learning model; obtaining a test sample, wherein the test sample comprises an abnormal sample and a normal sample, and the test sample comprises image data and camera parameter data corresponding to each sample; and inputting the image data and the camera parameter data corresponding to the abnormal sample and the image data and the camera parameter data corresponding to the normal sample into the initial machine learning model, and updating the parameters of the initial machine learning model to obtain the trained abnormal detection model.

In some of these embodiments, the first image data comprises a first RGB average value and a first luminance average value; the first acquisition module 31 is also configured for acquiring a first target image acquired by the camera operating in the first operating mode; dividing a first target image into a plurality of statistical blocks with the same size; acquiring a first RGB value of each statistical block; calculating to obtain a first RGB average value of the first target image according to the first RGB value of each statistical block; and calculating to obtain a first brightness average value of the first target image according to the first RGB average value.

In some of these embodiments, the first imaging parameter data includes a first infrared component ratio; the first obtaining module 31 is further configured to calculate a first white balance parameter value of the statistical block according to the first RGB value of the statistical block; determining a statistical block corresponding to a first white balance parameter value with a distance from a preset white balance parameter threshold value smaller than a preset first threshold value as a first target statistical block, wherein the white balance parameter threshold value is a white balance parameter value of a preset image shot by a camera under a pure infrared environment; and determining the ratio of the number of the first target statistical blocks to the number of all statistical blocks in the first target image as a first infrared component ratio acquired when the camera operates in the first working mode.

In some of these embodiments, the second image data comprises a second RGB average value and a second luminance average value; the second acquisition module 32 is further configured for acquiring a second target image acquired by the camera when operating in the second operating mode; dividing the second target image into a plurality of statistical blocks with the same size; acquiring a second RGB value of each statistical block; calculating to obtain a second RGB average value of a second target image according to the second RGB value of each statistical block; and calculating to obtain a second brightness average value of the second target image according to the second RGB average value.

In some of these embodiments, the second imaging parameter data includes a second infrared component ratio; the second obtaining module 32 is further configured to calculate a second white balance parameter value of the statistical block according to the second RGB value of the statistical block; determining a statistical block corresponding to a second white balance parameter value with a distance from a preset white balance parameter threshold value smaller than a preset first threshold value as a second target statistical block, wherein the white balance parameter threshold value is a white balance parameter value of a preset image shot by a camera under a pure infrared environment; and determining the ratio of the number of the second target statistical blocks to the number of all statistical blocks in the second target image as a second infrared component ratio acquired when the camera operates in the second working mode.

It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.

The present embodiment further provides an electronic device, fig. 4 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application, and as shown in fig. 4, the electronic device includes a memory 404 and a processor 402, the memory 404 stores a computer program, and the processor 402 is configured to execute the computer program to perform the steps in any of the method embodiments.

Specifically, the processor 402 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.

Memory 404 may include, among other things, mass storage for data or instructions. By way of example, and not limitation, memory 404 may include a Hard Disk Drive (Hard Disk Drive, abbreviated to HDD), a floppy Disk Drive, a Solid State Drive (SSD), flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. Memory 404 may include removable or non-removable (or fixed) media, where appropriate. The memory 404 may be internal or external to the abnormality detection device of the optical filter, where appropriate. In a particular embodiment, the memory 404 is a Non-Volatile (Non-Volatile) memory. In particular embodiments, Memory 404 includes Read-Only Memory (ROM) and Random Access Memory (RAM). The ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), Electrically rewritable ROM (EAROM), or FLASH Memory (FLASH), or a combination of two or more of these, where appropriate. The RAM may be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), where the DRAM may be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended data output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), and the like.

Memory 404 may be used to store or cache various data files for processing and/or communication use, as well as possibly computer program instructions for execution by processor 402.

The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement the abnormality detection method for the optical filter in any of the above embodiments.

Optionally, the electronic apparatus may further include a transmission device 406 and an input/output device 408, where the transmission device 406 is connected to the processor 402, and the input/output device 408 is connected to the processor 402.

Optionally, in this embodiment, the processor 402 may be configured to execute the following steps by a computer program:

s1, when the camera operates in the first operating mode, determining whether the first ambient light brightness value obtained by the camera falls within a preset switching threshold interval.

S2, when the first ambient light brightness value falls within the switching threshold interval, acquiring first image data and first imaging parameter data acquired by the camera operating in the first operating mode.

S3, the camera is switched from the first operation mode to the second operation mode, and the second image data and the second imaging parameter data acquired by the camera operating in the second operation mode are acquired.

And S4, processing the first image data, the first imaging parameter data, the second image data and the second imaging parameter data by using the trained abnormity detection model, and determining whether the optical filter has abnormity according to the detection result output by the trained abnormity detection model.

It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.

In addition, in combination with the method for detecting the abnormality of the optical filter in the above embodiments, the embodiments of the present application may be implemented by providing a storage medium. The storage medium having stored thereon a computer program; the computer program, when executed by a processor, implements the method of detecting an abnormality of an optical filter of any of the above embodiments.

It should be understood by those skilled in the art that various features of the above embodiments can be combined arbitrarily, and for the sake of brevity, all possible combinations of the features in the above embodiments are not described, but should be considered as within the scope of the present disclosure as long as there is no contradiction between the combinations of the features.

The above examples are merely illustrative of several embodiments of the present application, and the description is more specific and detailed, but not to be construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.