阅读说明：本技术 雨刮器的控制方法、控制装置和车辆 (control method and control device for windscreen wiper and vehicle ) 是由 李哲 张欣 盛春楠 欧志辉 郭彦东 于 2019-09-19 设计创作，主要内容包括：本申请公开了一种雨刮器的控制方法。控制方法包括：采集多帧车辆上挡风玻璃的图像；根据多帧图像确定雨刮器的控制策略；根据控制策略调节雨刮器的工作档位。本申请实施方式的控制方法中,根据挡风玻璃的图像来判断雨情,从而确定雨刮器的控制策略,无需采用检测雨量的传感器,而采用车辆现有的摄像头来获取挡风玻璃的图像,该摄像头并未专为雨刮器设计使用,同时还能够完成其他车辆的智能功能,集成成本低,可靠性高。本申请还公开了一种控制装置和车辆。(The application discloses a control method of a windscreen wiper. The control method comprises the following steps: collecting a plurality of frames of images of a windshield on a vehicle; determining a control strategy of the windscreen wiper according to the multi-frame image; and adjusting the working gear of the windscreen wiper according to the control strategy. According to the control method, the rain condition is judged according to the image of the windshield, so that the control strategy of the windscreen wiper is determined, a sensor for detecting the rainfall is not needed, the image of the windshield is obtained by the existing camera of the vehicle, the camera is not specially designed and used for the windscreen wiper, meanwhile, the intelligent functions of other vehicles can be completed, the integration cost is low, and the reliability is high. The application also discloses a control device and a vehicle.)

1. A control method of a wiper blade is characterized by comprising the following steps:

acquiring images of a windshield on a multi-frame vehicle, wherein the images comprise a rain image and a no-rain image;

Determining a control strategy of the windscreen wiper according to the multi-frame image;

And adjusting the working gear of the windscreen wiper according to the control strategy.

2. The control method of claim 1, wherein said capturing a plurality of frames of images of a windshield on a vehicle comprises:

acquiring video data of the windshield;

And acquiring a plurality of frames of images of the windshield in one wiper period from the video data at equal time intervals according to the current wiper period.

3. The method of claim 2, wherein said determining a control strategy for said wiper blade from said plurality of frame images comprises:

Calculating the proportion of the rain images in the multi-frame images in the current wiper period;

Determining a control sub-strategy according to the proportion of the rain images;

Putting the control sub-strategies into a decision pool, wherein the decision pool is used for accommodating a preset number of control sub-strategies;

and determining the control strategy according to the state of the decision pool.

4. the control method of claim 3, wherein the determining the control strategy according to the state of the decision pool comprises:

and if the decision pool is full, determining the average result of the preset number of control sub-strategies in the decision pool as the control strategy.

5. the control method of claim 3, wherein the determining the control strategy according to the state of the decision pool comprises:

and if the decision pool is not full, returning to the step of collecting the plurality of frames of images of the windshield on the vehicle.

6. The control method according to claim 3, wherein the calculating of the proportion of the rained image in the plurality of frames of images includes:

inputting the acquired multi-frame image into a preset target rainwater detection model;

Generating a rainwater probability characteristic result by adopting the target rainwater detection model;

Generating rainwater distribution data corresponding to each frame of the image by adopting the rainwater probability characteristic result;

setting a binarization threshold value to carry out binarization processing on the rainwater distribution data;

And determining the rain image and the rain-free image in the multiple frames of images according to the processing result to calculate the proportion of the rain image in the multiple frames of images.

7. The control method according to claim 3, wherein the determining a control sub-strategy according to the proportion of the rained images comprises:

And determining the upshifting or downshifting operation of the windscreen wiper according to the proportion of the rain images.

8. The control method according to claim 7, wherein the determining a control sub-strategy according to the proportion of the rained images comprises:

And determining the number of the gears for upshifting or downshifting the windscreen wiper according to the proportion of the rain images.

9. the control method according to claim 7 or 8, wherein the upshift comprises decreasing a wiper period and/or increasing a wiper speed, and the downshift comprises increasing a wiper period and/or decreasing a wiper speed.

10. the control method of claim 1, wherein said capturing a plurality of frames of images of a windshield on a vehicle comprises:

Acquiring a first image of the windshield at a first preset time point and a second image of the windshield at a second preset time point in one wiper period of the windscreen wiper;

the determining the control strategy of the windscreen wiper according to the multi-frame image comprises the following steps:

respectively judging whether the first image and the second image are rain images;

And determining the upshifting or downshifting operation of the windscreen wiper according to the judgment result.

11. The control method according to claim 10, wherein said determining an upshift or downshift operation of said wiper blade according to the determination result includes:

If the first image is a no-rain image and the second image is a rain image, determining that the windscreen wiper keeps the current working gear;

If the first image is a rainy image and the second image is a rainy image, determining that the windscreen wiper performs an up-shifting operation;

if the first image is a rain-free image and the second image is a rain-free image, determining that the windscreen wiper performs downshifting operation;

and if the first image is a rain image and the second image is a no-rain image, determining that the windscreen wiper performs the gear-up operation.

12. the control method of claim 10, wherein said capturing a plurality of frames of images of a windshield on a vehicle further comprises:

acquiring a third image of the windshield at least one third preset time point in one wiper period of the windscreen wiper;

The determining the control strategy of the windscreen wiper according to the multi-frame image comprises the following steps:

Respectively judging whether the first image, the second image and at least one third image are rain images;

And determining the number of the gears for upshifting or downshifting the windscreen wiper according to the judgment result.

13. A control device for a wiper blade, comprising:

The system comprises an image acquisition module, a storage module and a display module, wherein the image acquisition module is used for acquiring multiple frames of images of windshield on the vehicle, and the images comprise a rain image and a rain-free image;

The determining module is used for determining a control strategy of the windscreen wiper according to the multi-frame image;

and the control module is used for adjusting the working gear of the windscreen wiper according to the control strategy.

14. The vehicle is characterized by comprising a windscreen wiper, a processor and a camera device, wherein the camera device is used for collecting images of a windshield on the vehicle, and the images comprise a rain image and a no-rain image;

The processor is configured to:

Determining a control strategy of the windscreen wiper according to the multi-frame image;

and adjusting the working gear of the windscreen wiper according to the control strategy.

15. the vehicle of claim 14, wherein the processor determines the control strategy and adjusts the operating range of the wiper blade according to the control strategy using different threads.

16. A vehicle, characterized by comprising:

One or more processors, memory; and

One or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs including instructions for executing the control method of the wiper blade according to any one of claims 1 to 12.

17. a non-transitory computer-readable storage medium of computer-executable instructions which, when executed by one or more processors, cause the processors to perform the method of controlling a wiper blade of any one of claims 1 to 12.

Technical Field

the application relates to the technical field of automobiles, in particular to a control method and a control device of a windscreen wiper and a vehicle.

background

along with the development of electronic technology, some traditional spare parts are also gradually electronization, intellectuality in the vehicle, the wiper is as the indispensable configuration of car, can help the driver to obtain better field of vision when the driving, thereby guarantee driving safety, the appearance of automatic wiper compares in traditional manual wiper can be according to the size automatically regulated windscreen wiper speed of rainfall, the driver has greatly made things convenient for, among the correlation technique, the windscreen wiper speed of automatic wiper sets for according to the rainfall size, nevertheless adopt relevant sensor to detect the rainfall, system integration is complicated, the reliability is poor.

Disclosure of Invention

in view of this, embodiments of the present application provide a control method and a control device for a wiper blade, and a vehicle.

the application provides a control method of a windscreen wiper, which comprises the following steps:

The method comprises the steps that collected images of a windshield on a multi-frame vehicle comprise a rain image and a rain-free image;

Determining a control strategy of the windscreen wiper according to the multi-frame image;

and adjusting the working gear of the windscreen wiper according to the control strategy.

in some embodiments, said capturing a plurality of frames of images of a windshield on a vehicle comprises:

Acquiring video data of the windshield;

And acquiring a plurality of frames of images of the windshield in one wiper period from the video data at equal time intervals according to the current wiper period.

In some embodiments, the determining the control strategy of the wiper blade according to the multi-frame image comprises:

Calculating the proportion of the rain images in the multi-frame images in the current wiper period;

Determining a control sub-strategy according to the proportion of the rain images;

putting the control sub-strategies into a decision pool, wherein the decision pool is used for accommodating a preset number of control sub-strategies;

And determining the control strategy according to the state of the decision pool.

In some embodiments, the determining the control strategy according to the state of the decision pool comprises:

And if the decision pool is full, determining the average result of the preset number of control sub-strategies in the decision pool as the control strategy.

In some embodiments, the determining the control strategy according to the state of the decision pool comprises:

And if the decision pool is not full, returning to the step of collecting the plurality of frames of images of the windshield on the vehicle.

In some embodiments, the calculating the proportion of the rain image in the plurality of frames of images includes:

inputting the acquired multi-frame image into a preset target rainwater detection model;

Generating a rainwater probability characteristic result by adopting the target rainwater detection model;

Generating rainwater distribution data corresponding to each frame of the image by adopting the rainwater probability characteristic result;

setting a binarization threshold value to carry out binarization processing on the rainwater distribution data;

And determining the rain image and the rain-free image in the multiple frames of images according to the processing result to calculate the proportion of the rain image in the multiple frames of images.

in some embodiments, the determining a control sub-strategy according to the proportion of the rained images comprises:

And determining the upshifting or downshifting operation of the windscreen wiper according to the proportion of the rain images.

in some embodiments, the determining a control sub-strategy according to the proportion of the rained images comprises:

and determining the number of the gears for upshifting or downshifting the windscreen wiper according to the proportion of the rain images.

In some embodiments, the upshift comprises decreasing a wiper period and/or increasing a wiper speed, and the downshift comprises increasing a wiper period and/or decreasing a wiper speed.

in some embodiments, said capturing a plurality of frames of images of a windshield on a vehicle comprises:

Acquiring a first image of the windshield at a first preset time point and a second image of the windshield at a second preset time point in one wiper period of the windscreen wiper;

the determining the control strategy of the windscreen wiper according to the multi-frame image comprises the following steps:

Respectively judging whether the first image and the second image are rain images;

And determining the upshifting or downshifting operation of the windscreen wiper according to the judgment result.

In some embodiments, the determining an upshift or downshift operation of the wiper blade according to the determination result includes:

if the first image is a no-rain image and the second image is a rain image, determining that the windscreen wiper keeps the current working gear;

If the first image is a rainy image and the second image is a rainy image, determining that the windscreen wiper performs an up-shifting operation;

if the first image is a rain-free image and the second image is a rain-free image, determining that the windscreen wiper performs downshifting operation;

and if the first image is a rain image and the second image is a no-rain image, determining that the windscreen wiper performs the gear-up operation.

In some embodiments, said capturing a plurality of frames of images of a windshield on a vehicle further comprises:

Acquiring a third image of the windshield at least one third preset time point in one wiper period of the windscreen wiper;

The determining the control strategy of the windscreen wiper according to the multi-frame image comprises the following steps:

Respectively judging whether the first image, the second image and at least one third image are rain images;

And determining the number of the gears for upshifting or downshifting the windscreen wiper according to the judgment result.

The application provides a controlling means of wiper, includes:

the system comprises an image acquisition module, a storage module and a display module, wherein the image acquisition module is used for acquiring multiple frames of images of windshield on the vehicle, and the images comprise a rain image and a rain-free image;

The determining module is used for determining a control strategy of the windscreen wiper according to the multi-frame image;

and the control module is used for adjusting the working gear of the windscreen wiper according to the control strategy.

the application provides a vehicle, which comprises a windscreen wiper, a processor and a camera device, wherein the camera device is used for collecting images of a windshield on a multi-frame vehicle, and the images comprise a rain image and a rain-free image;

The processor is configured to:

Determining a control strategy of the windscreen wiper according to the multi-frame image;

And adjusting the working gear of the windscreen wiper according to the control strategy.

In some embodiments, the processor determines the control strategy and adjusts the working position of the windscreen wiper according to the control strategy by using different threads.

a vehicle is provided that includes one or more processors, memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs including instructions for executing the wiper blade control method as described above.

A non-transitory computer-readable storage medium containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the method of controlling a wiper blade is provided.

In the control method, the control device, the vehicle and the computer-readable storage medium of the windscreen wiper, the rain condition is judged according to the image of the windscreen wiper, so that the control strategy of the windscreen wiper is determined, a sensor for detecting the rain amount is not needed, the image of the windscreen wiper is obtained by the existing camera of the vehicle, the camera is not specially designed for the windscreen wiper, meanwhile, the intelligent functions of other vehicles can be completed, the integration cost is low, and the reliability is high.

drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

Fig. 1 is a schematic flow chart of a method for controlling a wiper blade according to some embodiments of the present disclosure.

FIG. 2 is a schematic illustration of a vehicle according to certain embodiments of the present application.

FIG. 3 is a block schematic diagram of a control device according to certain embodiments of the present application.

Fig. 4 is a schematic view of the mounting position of the image pickup apparatus according to some embodiments of the present application.

Fig. 5 to 6 are schematic flow charts of a control method of a wiper blade according to some embodiments of the present application.

FIG. 7 is a schematic illustration of a rain probability characteristic map of certain embodiments of the present application.

fig. 8 is a schematic illustration of a rain water distribution profile of certain embodiments of the present application.

fig. 9 to 10 are schematic flow charts of a wiper blade control method according to some embodiments of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.

The wiper blade is generally used to wipe off rainwater attached to a windshield of a vehicle, thereby providing a better view to a driver and increasing driving safety. In traditional wiper, need the driver to adjust the gear of wiper according to the rain condition manual, nevertheless, can disperse driver's attention in the driving process, have certain potential safety hazard. Along with the development of vehicle-mounted electronic technology, the intelligent windscreen wiper in the related art can detect rainfall by combining a sensor and output a working gear according to the detected rainfall, so that the windscreen wiper can automatically switch the working gear according to the rain condition. However, the integration of multiple rainfall sensors and even humidity sensors is complicated, the reliability is poor, and the cost of the system is high.

Referring to fig. 1, the present application provides a control method for a wiper blade. The control method comprises the following steps:

s10: collecting multiple frames of images of a windshield on a vehicle, wherein the images comprise a rain image and a no-rain image;

s20: determining a control strategy of the windscreen wiper according to the multi-frame image;

s30: and adjusting the working gear of the windscreen wiper according to the control strategy.

referring to fig. 2 and 3, the present embodiment provides a vehicle 100. The vehicle 100 includes a wiper 10 imaging device 11 and a processor 12. The camera device 11 is used for shooting a vehicle windshield, and the processor 12 is used for collecting multiple frames of images of the windshield, determining a control strategy of the windscreen wiper 10 according to the multiple frames of images, and adjusting the working gear of the windscreen wiper 10 according to the control strategy. The processor 12 may be a processor 12 provided independently for adjusting the wiper 10, may be the processor 12 of the vehicle traveling system, or may be a collection of a plurality of sub-controllers and sub-processors, which is not limited herein.

the embodiment of the present application also provides a control device 110, and the control method of the wiper blade according to the embodiment of the present application can be implemented by the control device 110.

Specifically, the control device 110 includes an image acquisition module 111, a determination module 112, and a control module 113. S10 may be implemented by the image acquisition module 111, S20 may be implemented by the determination module 112, and S30 may be implemented by the control module 113. Or, the image acquiring module 111 is configured to acquire a plurality of frames of images of a windshield on the vehicle. The determination module 112 is configured to determine a control strategy of the wiper blade 10 according to the multiple frames of images. The control module 113 is configured to adjust an operating range of the wiper according to a control strategy.

referring to fig. 4, the image of the windshield may be captured by installing an image capturing device in the vehicle 100, or by acquiring an image capturing device 11 (such as a tachograph) installed in the vehicle. The camera 11 of the vehicle 100 is mounted directly in front of the rear view mirror of the vehicle 100, and the light-colored portion may indicate the range of the windshield that the camera 11 can cover.

In the control method of the wiper, the control device 110 and the vehicle 100 according to the embodiment of the present invention, the rain condition is determined according to the image of the windshield, so as to determine the control strategy of the wiper, and the image of the windshield is acquired by using the existing camera of the vehicle without using a sensor for detecting the amount of rain.

Referring to fig. 5, in some embodiments, S10 includes:

s11: acquiring video data of a windshield;

s12: a plurality of frames of images of the windshield for one wiper period are acquired from the video data at equal time intervals according to the current wiper period.

in certain embodiments, S11 and S12 may be implemented by the image acquisition module 111. In other words, the image obtaining module 111 is configured to obtain video data of the windshield, and obtain multiple frames of images of the windshield for one wiper period at equal time intervals from the video data according to the current wiper period.

in some embodiments, the processor 12 is configured to obtain video data of the windshield and to obtain a plurality of frames of images of the windshield for one wiper period at equal time intervals from the video data based on the current wiper period.

specifically, the wiper cycle is a time interval in which the wiper blade 10 passes the same position twice in the same direction, and may be, for example, a time from one homing to the next homing of the wiper blade 10. Take a wiper cycle as an example of a predetermined cycle. In actual operation, the camera device 11 continuously records a video on the windshield, and the processor 12 captures a certain number of frame images from the acquired video data at equal time intervals, that is, the time intervals between two adjacent frame images are equal.

In particular, the wiper period may be defined as the longest wiper period for the wiper blade 10 in the rest state, or the wiper period in the rest state may be the same as the wiper period of the lowest speed gear of the wiper blade 10.

referring to fig. 6, in some embodiments, S20 includes:

S21: calculating the proportion of rain images in the multi-frame images in the current wiper period;

s22: determining a control sub-strategy according to the proportion of the rain images;

s23: putting the control sub-strategies into a decision pool, wherein the decision pool is used for accommodating a preset number of control sub-strategies;

s24: and determining the control strategy according to the state of the decision pool.

In the present embodiment, S24 includes:

s241: if the decision pool is full, determining the average result of the control sub-strategies with the preset number in the decision pool as the control strategy;

s242: if the decision pool is not full, return to S10.

In some embodiments, the determining module 112 includes a calculating unit and a determining unit, S21 may be implemented by the calculating unit, and S22-S24 may be implemented by the determining unit. Or the calculating unit is used for calculating the proportion of the rain image in the multi-frame images in the current wiper period. The determining unit is used for determining a control sub-strategy according to the proportion of the rain images, placing the control sub-strategy into a decision pool and determining the control strategy according to the state of the decision pool, if the decision pool is full, determining the average result of a predetermined number of control sub-strategies in the decision pool as the control strategy, and if the decision pool is not full, repeating the process of determining the control sub-strategy until the decision pool is full.

In some embodiments, the processor 12 calculates a proportion of the plurality of frames of images in the current wiper period that have rain images. The determining unit is used for determining a control sub-strategy according to the proportion of the rain images, placing the control sub-strategy into a decision pool and determining the control strategy according to the state of the decision pool, if the decision pool is full, determining the average result of a predetermined number of control sub-strategies in the decision pool as the control strategy, and if the decision pool is not full, repeating the process of determining the control sub-strategy until the decision pool is full.

Specifically, during rainfall, the wiper blade 10 keeps the working gear constant or the rain force keeps basically constant along with the change of the rain force but the wiper blade 10 works at different gears, the time ratio that the user can see the road through the windshield is different, and during driving in rainy days, in order to ensure the driving safety, the time ratio that the user can see the road in a preset wiper period is ensured to be larger than a preset threshold value. In the present embodiment, the ratio of the rainy image to the no-rain image in the predetermined wiper period is obtained to reflect the ratio of the road that can be seen clearly by the user, and it can be understood that, when the ratio of the rainy image is higher in the predetermined wiper period, the ratio of the road that can be seen clearly by the user is lower, and at this time, the ratio of the rainy image needs to be reduced by adjusting the gear of the wiper 10, so as to increase the ratio of the no-rain image, and the ratio of the time that the road can be seen clearly meets the predetermined threshold. In the embodiment, the control sub-strategies of a plurality of wiper periods are adopted to jointly establish the final control strategy of the wiper.

It can be understood that the wiper blade 10 outputs a control sub-strategy in each wiper cycle, and if the sub-strategy is used as a final control strategy to control the wiper gear change, the control strategy is not accurate enough because the number of image samples used for making the control strategy is small and is easily influenced by noise, and the attention of a driver is more easily influenced because the wiper gear changes too frequently. Therefore, in the present embodiment, a decision pool is set, and the decision pool is used for accommodating a predetermined number of control sub-strategies. That is, the sub-strategies of a plurality of wiper cycles are integrated to jointly determine the control strategy.

In particular, the capacity of the decision pool, i.e. the number of control sub-strategies or wiper cycles, may be set by a user. And if the space of the decision pool is smaller, the control strategy is more sensitive to rainfall change, otherwise, the rainfall change is more gradual, the influence of noise is less, and the driver can focus attention more easily. In practice, the average of the control sub-strategies determined for a predetermined number of wiper cycles is taken as the final control strategy. When the space of the decision pool is full, outputting a primary control strategy, emptying the decision pool after outputting the control strategy, and when the space of the decision pool is not full, repeating the process of determining the control sub-strategy until the decision pool is full, and outputting the final control strategy by taking the current wiper gear.

for example, in one example, the decision pool may accommodate 5 control sub-strategies, that is, each control strategy is determined according to a control sub-strategy of 5 wiper cycles. Wherein, the control strategy determined by each of the 5 wiper periods is up 2, up 3, up 1, up 2, then the control strategy is determined to be up 2 according to the control sub-strategy of the 5 wiper periods, so that in the next wiper period, the working gear of the wiper 10 is up 2 relative to the current working gear.

in the present embodiment, the processor 12 determines a control strategy and performs control of changing the wiper position using different threads.

In particular, the sub-strategy of each wiper cycle is not output immediately, but rather waits for the decision pool to be full. In addition, the communication cycle between the working gear strategy controller and the wiper controller (both are subunits of the processor 12) is short, for example, 30ms, and the processing of one frame of image cannot be completed in this period of time, so that the determination of the control strategy and the output of the control strategy, that is, the adjustment of the wiper gear position control, need to be performed in different threads. For example, a control strategy requires x wiper cycles, one wiper cycle requires processing of y frame images, i.e., a total of xy frame images are required to modify the output of a control strategy. Therefore, between two times of outputting the control strategy, the working position strategy controller periodically (30ms) outputs the previous control strategy to the controller of the wiper blade, which is a thread. And the calculation or determination of the control strategy for the operating position of the wiper blade 10 is performed by another thread.

it is understood that the ratio of the rain image in each wiper period may reflect whether the current operating range of the wiper blade 10 is appropriate. For example, in a wiper period, if the operating range is suitable, the ratio of the rain images in the operating range is about 30%, the driving safety and the user experience can be ensured in the ratio, if it is detected that the ratio of the rain images in a wiper period is higher than the ratio, it can be considered that the current operating range cannot meet the current rain condition, and the range needs to be increased, and if it is detected that the ratio of the rain images in a wiper period is lower than the ratio, it can be considered that the current operating range is too high relative to the current rain condition, unnecessary consumption of the wiper 10 can be increased, and the service life of the wiper 10 can be shortened.

Referring to fig. 7, in some embodiments, S21 includes:

S211: inputting the acquired image of the windshield into a preset target rainwater detection model;

s212: generating a rainwater probability characteristic result by adopting a target rainwater detection model;

S213: generating rainwater distribution data corresponding to each frame of image by adopting a rainwater probability characteristic result;

S214: setting a binarization threshold value to carry out binarization processing on the rainwater distribution data;

S215: and determining the rain image and the rain-free image in the multi-frame images according to the processing result to calculate the proportion of the rain image in the multi-frame images.

in some embodiments, S211-S215 may be implemented by the computing unit, or the computing unit is configured to input the acquired image of the windshield into a preset target rain detection model, generate a rain probability feature result using the target rain detection model, generate rain distribution data corresponding to each frame of the image using the rain probability feature result, set a binarization threshold value to perform binarization processing on the rain distribution data, and determine a rain image and a rain-free image in the multiple frames of images according to the processing result to calculate a ratio of the rain image in the multiple frames of images.

in some embodiments, the processor 12 is configured to input the collected multiple frames of images of the windshield of the vehicle into a preset target rain detection model, generate a rain probability feature result using the target rain detection model, generate rain distribution data corresponding to each frame of the image using the rain probability feature result, set a binarization threshold to perform binarization processing on the rain distribution data, and determine a rain image and a rain-free image in the multiple frames of images according to the processing result to calculate a ratio of the rain image in the multiple frames of images.

In particular, the processor 12 has an image algorithm module therein, which can perform analysis of the distribution of rain on the windshield by collecting image data of a plurality of frames of the windshield.

each frame of the multiple frames of images respectively corresponds to image data of a windshield of the windscreen wiper 10 in a windscreen wiper period, and the image data of each frame of the windshield corresponds to a rainwater probability characteristic result so as to represent a rainwater distribution condition of the windshield in the current state. The rain probability feature result may be in the form of a probability feature function, a probability feature vector, a probability feature map, or the like, and is not particularly limited.

the following describes an example of generating a single-frame rain probability feature map. The preset target rain detection model can be a convolutional neural network model which is generated by pre-training and used for detecting rain, and the target rain detection model can be generated by deep learning convolutional neural network detection algorithm training.

in a specific implementation, the image data is input into a preset target rainwater detection model, and the preset target rainwater detection model is adopted to detect the probability of rainwater existing in the region corresponding to each pixel in the image data.

in this embodiment, the target rain detection model may be generated by:

Acquiring an image sample; the method comprises the steps that an image sample is generated by dividing image data of a windshield according to the size of a preset area, and the image sample comprises a training sample and a verification sample;

training a preset initial rainwater detection model by using a training sample to generate a candidate rainwater detection model;

Verifying the candidate rainwater detection model by using a verification sample, and calculating verification accuracy;

and when the verification accuracy is greater than a preset threshold value, determining the candidate rainwater detection model as a target rainwater detection model.

Wherein the image samples may be samples used for training a target rain detection model. The method comprises the steps of acquiring video data on a large number of windshields through a camera device deployed on a vehicle, obtaining image data by extracting key frames in the video data, and segmenting the image data according to the size of a preset area to obtain an image sample.

The preset region size may be a region of the divided image data set in advance to indicate a region size of the image sample. For example, the preset region size may be a region size corresponding to a square with a side length of m.

Further, in order to further improve the accuracy of the prediction probability of the target rainwater detection model, the image sample may be divided into two parts, one part is used as a training sample for training the target rainwater detection model, and the other part is used as a verification sample for verifying the target rainwater detection model. After the training sample is adopted to train the initial rainwater detection model to generate the candidate rainwater detection model, the verification sample can be adopted to verify the candidate rainwater detection model, and the verification accuracy rate is calculated.

when the verification accuracy is greater than a preset threshold value, the candidate rainwater detection model reaches the expectation, and the candidate rainwater detection model is determined as a target rainwater detection model; when the verification accuracy is smaller than or equal to the preset threshold, the verification result does not reach the expectation, the candidate rainwater detection model can be continuously trained, or the candidate rainwater detection model is discarded, and the substep is executed again: and training the preset initial rainwater detection model by adopting the training sample to generate a candidate rainwater detection model.

the rain probability feature map may be a grayscale map indicating a probability that rain exists in a region corresponding to each pixel in the detected image data, and the magnitude of the probability of the region corresponding to the pixel is expressed by 8 bits in an unsigned manner using a grayscale value. For example, solid black (gray scale value of 0) indicates a probability of 0%, and solid white (gray scale value of 255) indicates a probability of 100%.

referring to fig. 8 and 9, in the present embodiment, the rain probability characteristic map is generated as follows:

generating a probability matrix of each frame of image data by adopting a target rainwater detection model, wherein the probability matrix comprises a probability value;

converting the probability value in the probability matrix into a gray value;

Acquiring the original height, the original width and the original coordinate information of each frame of image data;

Respectively generating corresponding target height, target width and target coordinate information by adopting the original height, original width and original coordinate information of each frame of image data;

And generating a rainwater probability characteristic map by adopting the gray value, the target height, the target width and the target coordinate information.

specifically, each probability value in the probability matrix may be used to indicate a probability that rain exists in a region corresponding to a pixel.

in this embodiment, the probability value in the probability matrix output by the target rainwater detection model may be converted into a gray value.

Specifically, the conversion relationship between the gray-scale value and the probability value can be expressed as follows:

Where p represents the probability value and fc (p) represents the corresponding gray value.

For example, if the probability value is 0.6, the gray-scale value corresponding to the probability value is 255 × 0.6 ═ 153; if the probability value is 0.4, the gray scale value corresponding to the probability value is 255 × 0.4 — 102.

further, after the gray value of the area corresponding to each pixel is determined, the target height and the target width of the rainwater probability feature map and the target coordinate information corresponding to each pixel in the rainwater probability feature map can be further determined.

Specifically, assuming that, when training the target rain detection model, the preset region size for indicating the region size of the image sample is a square with a side length of m, the height-width relationship between the rain probability characteristic diagram and the image data may be expressed as:

(m∈N，s∈N)。

Wherein, H is the original height of the image data, W is the original width of the image data, H 'is the target height of the rainwater probability characteristic diagram, and W' is the target width of the rainwater probability characteristic diagram. And S is the stride step size of the convolutional neural network in the target rainwater detection model, and the numerical value of the stride step size is determined by the overall structure of the convolutional neural network.

When the original height and the original width of the image data are known, the target height and the target width of the rain probability feature map can be determined by the height-width relationship between the rain probability feature map and the image data.

The original coordinate information comprises an original abscissa and an original ordinate, and the target coordinate information comprises a target abscissa and a target ordinate.

The coordinate relationship between the rain probability feature map and the corresponding pixel in the image data can be expressed as:

(m∈N，s∈N)。

wherein x is an original abscissa of a pixel in the image data, y is an original ordinate of a pixel in the image data, x 'is a target abscissa of a corresponding pixel in the rain probability feature map, and y' is a target ordinate of a corresponding pixel in the rain probability feature map. And S is the stride step size of the convolutional neural network in the target rainwater detection model, and the numerical value of the stride step size is determined by the overall structure of the convolutional neural network. The target abscissa and the target ordinate of the corresponding pixel in the rain probability feature map and the image data can be determined by the coordinate relationship between the rain probability feature map and the corresponding pixel in the image data.

After determining the gray value, the target height, the target width and the target coordinate information, the gray value, the target height, the target width and the target coordinate information may be further used to generate a rainwater probability feature map.

After the rain probability feature map is generated, the rain probability feature map may be further employed to generate rain distribution data for the windshield.

in some examples, the rain water distribution data may be generated by:

carrying out binarization processing on the rainwater probability characteristic map to generate a rainwater distribution map;

and generating rainwater distribution data of the windshield by adopting the rainwater distribution map.

the binarization processing is to set the gray value of a pixel point of the rainwater probability characteristic map to be 0 or 255, so as to generate a rainwater distribution map, and the whole image of the rainwater distribution map has an obvious visual effect only including black and white.

Specifically, a binarization threshold may be set, the binarization threshold being used to indicate a critical gray value, the gray value of a pixel greater than the binarization threshold is set as a gray maximum value (i.e., a gray value of 255), and the gray value of a pixel less than the binarization threshold is set as a gray minimum value (i.e., a gray value of 0), thereby achieving binarization.

for example, if the binarization threshold is 120, and if the gray value of a certain pixel in the rain probability feature map is 130, and exceeds the binarization threshold, the gray value of the pixel is adjusted to be the maximum gray value 255, which is white. When the gray value of a certain pixel in the rain probability characteristic map is 40 and does not reach the binarization threshold value, the gray value of the pixel is adjusted to be 0, namely black.

In embodiments of the present invention, the rain profile may be used to generate rain distribution data for the windshield. In the rain distribution graph, more white coverage indicates more rain coverage, more black coverage indicates less rain coverage, and thus, whether a frame of image is a rainy image can be determined according to the area ratio of black or white coverage, for example, when the white proportion in a frame of image is greater than 20%, the current frame of image can be considered as a rainy image.

further, each frame image in a predetermined wiper period may be judged as a rainy image or a no-rain image, thereby calculating a ratio of the rainy image therein. The area ratio for judging whether a frame of image is covered by white or black of a rain image can be set by a user, so that for the same rain condition, different judgment results are generated for judging whether each image in a wiper period is a rain image, the ratio of the rain images is different, different users have different control strategies for the same rain condition, and the wiper 10 is more personalized and intelligent.

In certain embodiments, S22 includes the steps of:

and determining the upshifting or downshifting operation of the windscreen wiper according to the proportion of the rain images.

in some embodiments, the above steps may be implemented by the determination unit. Or the determining unit is used for determining the upshifting or downshifting operation of the windscreen wipers according to the proportion of the rain images.

in some embodiments, processor 12 is configured to determine an up shift or down shift operation for wiper based on the proportion of rain images.

In the present embodiment, S22 includes:

and determining the number of the gears for upshifting or downshifting the windscreen wipers according to the proportion of the rain images.

specifically, the up-shift or down-shift operation of the wiper blade in the present application does not directly select a certain gear, but refers to the number of gears by which the operating gear of the wiper blade is raised or lowered with respect to the current operating gear. For example, the operating range of the wiper blade 10 may include a plurality of ranges of low speed intervals, low speed succession, high speed succession, and the like. The low-speed interval gear further comprises a plurality of sub-gears. For example, the plurality of sub-gear positions are respectively scraped once every x seconds, scraped once every y seconds, scraped once every z seconds, and scraped once every l seconds (x > y > z > l). The low-speed continuous gears have no interval, and the speed of the windscreen wiper is the same as the low-speed interval gears. High speed consecutive gears are also without intervals and the wiper speed is faster than low speed consecutive. If the current gear is a low speed gear with x seconds, and the control strategy is up to 2, the wiper 10 will be operated at the low speed gear with z seconds.

In one example, if the proportion of the rain images is smaller than a first preset proportion, the windscreen wiper is determined to be lowered by a first number of working gears. And if the proportion of the rain images is greater than or equal to a first preset proportion and smaller than a second preset proportion, determining that the wiper reduces a second number of working positions, wherein the second number is smaller than the first number. And if the proportion of the rain images is greater than or equal to a second preset proportion and less than a third preset proportion, determining that the wiper keeps the current working gear. And if the proportion of the rain images is greater than or equal to a third preset proportion and less than a fourth preset proportion, determining that the third number of working gears are added to the windscreen wiper. And if the proportion of the rain images is greater than or equal to a fourth preset proportion and less than a fifth preset proportion, determining that the wiper is increased by a fourth number of working gears, wherein the fourth number is greater than the third number. And if the proportion of the rain images is greater than or equal to a fifth preset proportion and less than a sixth preset proportion, determining that the wiper is increased by a fifth number of working gears, wherein the fifth number is greater than the fourth number. And if the proportion of the rain images is greater than or equal to a sixth preset proportion, determining that the wiper is increased by a sixth number of working gears, wherein the sixth number is greater than the fifth number.

specifically, a control sub-strategy of the corresponding wiper blade 10 may be formulated according to a proportion of the rain image in one wiper cycle, for example, the proportion of the rain image is less than 10%, and the operating range of the wiper blade 10 is determined to be reduced by 2. The rain ratio is more than or equal to 1/10 and less than 20%, and the working gear of the windscreen wiper 10 is determined to be reduced by 1 gear. The ratio of the rain image is more than or equal to 20% and less than 30%, and the working gear of the windscreen wiper 10 is determined to be kept unchanged. The ratio of the rain images is more than or equal to 30% and less than 40%, and the working gear of the windscreen wiper 10 is determined to be shifted up to 1. The ratio of the rain image is more than or equal to 40% and less than 50%, and the working gear of the windscreen wiper 10 is determined to be raised by 2. The ratio of the rain images is more than or equal to 50% and less than 60%, and the working gear of the windscreen wiper 10 is determined to be shifted up to 3. And determining that the working gear of the windscreen wiper 10 is shifted up to 4 when the ratio of the rain images is more than or equal to 60%. It should be noted that the first to sixth predetermined ratios and the first to fifth numbers are only illustrative, and the specific values may be set according to the frame number of the image acquired by the wiper blade and the user's feeling, and the like, and are not limited in particular.

in an embodiment of the present application, the upshift comprises decreasing the wiper period and/or increasing the wiper speed, and the downshift comprises increasing the wiper period and/or decreasing the wiper speed.

Specifically, the operating range of the wiper blade 10 includes dimensions such as a wiper period and a wiper speed, the up-shift refers to reducing the wiper period and/or increasing the wiper speed, and the down-shift refers to increasing the wiper period and/or decreasing the wiper speed. The number of gears indicates the degree of increase or decrease, with larger numbers being deeper.

Therefore, the control strategy of the windscreen wiper 10 is the adjustment of gear up and gear down on the basis of the current gear, the windscreen wiper gear is not directly determined according to the rainfall, the control mode is more accurate, and the calibration process of the windscreen wiper 10 on the rainfall and the windscreen wiper gear when leaving the factory is also reduced.

Referring to fig. 10, in some embodiments, S10 includes:

s13: acquiring a first image of a windshield at a first preset time point and a second image of a windshield at a second preset time point in one wiper period of a windscreen wiper;

S20 includes:

s25: respectively judging whether the first image and the second image are rain images;

S26: and determining the upshifting or downshifting operation of the windscreen wiper according to the judgment result.

in some embodiments, S13 may be implemented by the image acquisition module 111, and S25 and S26 may be implemented by the determination module 112. In other words, the image acquiring module 111 is configured to acquire a first image of a windshield at a first predetermined time point and a second image of a windshield at a second predetermined time point in one wiper cycle of the wiper blade 10. The determining module 112 is configured to determine whether the first image and the second image are rain images, and determine an upshift or downshift operation of the wiper blade according to the determination result.

In some embodiments, the processor 12 is configured to acquire a first image at a first predetermined time point and a second image at a second predetermined time point in one wiper cycle of the wiper blade 10, respectively determine whether the first image and the second image are rain images, and determine an upshift or downshift operation of the wiper blade according to the determination result.

the present embodiment is different from the foregoing embodiment in that the foregoing embodiment makes a control strategy is to make a final control strategy based on sub-decisions in a plurality of wiper cycles together.

In the embodiment, only two frames of images at a specific time point in one wiper period are acquired, and an encoder needs to be additionally arranged on a motor of the wiper 10 in terms of hardware so as to accurately acquire the current wiper period and phase. In this case, an image at a specific time point in each wiper cycle is acquired, and an upshift or downshift strategy of the wiper blade is determined according to the result of the judgment of whether the two frames of images are rain images.

in this embodiment, the manner of determining whether the first image and the second image are rain images is basically the same as that of the foregoing embodiment, and is not described herein again.

in the present embodiment, S26 includes the steps of:

if the first image is a no-rain image and the second image is a rain image, determining that the wiper keeps the current working gear;

If the first image is a rainy image and the second image is a rainy image, determining that the windscreen wiper performs an up-shifting operation;

If the first image is a rain-free image and the second image is a rain-free image, determining that the windscreen wiper performs downshifting operation;

And if the first image is a rain image and the second image is a no-rain image, determining that the windscreen wiper performs the upshifting operation.

In some embodiments, the determining module 112 may be implemented, or the determining module 112 is configured to determine that the wiper blade 10 keeps the current operating position when the first image is a no-rain image and the second image is a rain image, determine that the wiper blade 10 performs an upshift operation when the first image is a rain image and the second image is a rain image, determine that the wiper blade 10 performs a downshift operation when the first image is a no-rain image and the second image is a no-rain image, and determine that the wiper blade 10 performs an upshift operation when the first image is a rain image and the second image is a no-rain image.

In some embodiments, the processor 12 is configured to determine that the wiper blade 10 maintains the current operating position when the first image is a no-rain image and the second image is a rain image, determine that the wiper blade 10 performs an upshift operation when the first image is a rain image and the second image is a rain image, determine that the wiper blade 10 performs a downshift operation when the first image is a no-rain image and the second image is a no-rain image, and determine that the wiper blade 10 performs an upshift operation when the first image is a rain image and the second image is a no-rain image.

specifically, the first image at the first predetermined point in time and the second image at the second predetermined point in time may be the last to last frame image and the last frame image in one wiper cycle. Similar to the foregoing embodiment, one wiper cycle is uniformly divided into a plurality of sub-time points, the intervals between each sub-time point are the same, the last two sub-time points in one wiper cycle are obtained as a first predetermined time point and a second predetermined time point, and the corresponding frame of image is a first image and a second image respectively.

in operation, whether the driver can see the front road at the first preset time point and the second preset time point is reflected by the judgment result of judging whether the first image and the second image are the rain images. If the first preset time point can be seen clearly, namely the first image is a no-rain image, and the second preset time point can not be seen clearly, namely the second image is a rain image, the driver can be considered to see the front road clearly in most of the time in one wiper period, and the wiper gear is kept unchanged.

if the first preset time point is not visible, namely the first image is a rainy image, and the second preset time point is not visible, namely the second image is a rainy image, the current gear can not ensure that the driver can see the road ahead clearly in most of the time in one wiper period, and the gear-up operation is determined to be needed.

If the first image can be seen clearly at the first preset time point, namely the first image is a rain-free image, and the second image can be seen clearly at the second preset time point, namely the second image is a rain-free image, the current windscreen wiper gear is suspected to be too high, and the downshift operation is required.

and if the first preset time point is not visible, namely the first image is a rainy image, and the second preset time point is visible, namely the second image is a no-rain image, the current judgment result is considered to be influenced by noise, and the gear-up operation is carried out in order to ensure the driving safety.

It will be appreciated that the number of image frames acquired by the present embodiment is less and less computationally demanding for the processor than in the previous embodiment.

It should be noted that in the present embodiment, the number of the operating ranges that are increased or decreased is only 1, or the control strategy includes only increasing the first gear, keeping the first gear unchanged or decreasing the first gear. If it is desired to increase or decrease the number of units, more images corresponding to predetermined points in time may be acquired to determine the control strategy.

in such embodiments, S10 further includes:

Acquiring a third image of the windshield at least one third preset time point in one wiper period of the windscreen wiper;

s20 further includes:

Respectively judging whether the first image, the second image and the at least one third image are rain images or not;

And determining the number of the gears for upshifting or downshifting the windscreen wiper according to the judgment result.

In some embodiments, the step of acquiring a third image of the windshield at least one third predetermined time point in one wiper cycle of the wiper blade may be performed by the image acquiring module 111, and the steps of determining whether the first image, the second image and the at least one third image are rain images and determining the number of shift stages for the up-shift or the down-shift of the wiper blade according to the determination result may be performed by the determining module 112. In other words, the image acquiring module 111 is configured to acquire a third image of the windshield at least one third predetermined time point in one wiper period of the wiper. The determining module 112 is configured to determine whether the first image, the second image and the at least one third image are rain images respectively, and determine the number of shift positions for upshifting or downshifting the wiper according to the determination result.

in some embodiments, the processor 12 is configured to acquire a third image of at least one third predetermined time point in one wiper cycle of the wiper blade 10, determine whether the first image, the second image and the at least one third image are rain images respectively, and determine the number of gear positions for upshifting or downshifting the wiper blade according to the determination result.

Specifically, in one example, one wiper cycle may be equally divided into 8 time points, the phase of the first image acquisition may be a time point corresponding to 7/8 cycle, and the phase of the second image acquisition may be a time point corresponding to 8/8 cycle. The third predetermined time point includes a plurality of time points, for example, the third predetermined time point may be a time point corresponding to a period with a phase of 5/8, 6/8, etc. Thus, more frame images can be obtained, and understandably, by obtaining more images, not only can the change of rain or no rain be judged, but also the change of rain vigor can be judged, so that the number of increased or decreased gears can be determined. For example, the third point in time comprises 2, i.e. 4 frames of images to determine the wiper strategy. The control strategy of the gear position of the windscreen wiper needs to be determined according to the judgment result of whether the 4 frames of images have rain or not. For example, if all 4 images are rain images, it is determined that the wiper is increased by 4 steps.

the embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the control method of the wiper blade of any of the above embodiments.

the embodiment of the application also provides a vehicle. The vehicle includes a memory and one or more processors, one or more programs being stored in the memory and configured to be executed by the one or more processors. The program includes instructions for executing the control method of the wiper blade according to any one of the above embodiments.

The processor may be used to provide computational and control capabilities to support the operation of the entire vehicle. The memory of the vehicle provides an environment for the computer readable instructions in the memory to operate.

it will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or the like.

The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.