阅读说明：本技术 一种烤箱烟雾监控方法及装置、存储介质 (Oven smoke monitoring method and device and storage medium ) 是由 刘兵 高洪波 俞国新 刘彦甲 李玉强 于 2018-06-29 设计创作，主要内容包括：本发明公开了一种烤箱烟雾监控方法,该方法包括：在烤箱工作前,获取烤箱内图片,进行图像分割,选取分割出的一个食材区域作为观察对象,并确定观察对象的梯度矩阵和对比度矩阵；烤箱工作时,控制烤箱内摄像头按设定时间间隔抓取烤箱内图片,确定观察对象在当前图片中的梯度矩阵和对比度矩阵；基于两组梯度矩阵和对比度矩阵,确定当前图片中观察对象的清晰度系数；当清晰度系数小于设定阈值时,发送触发打开烤箱排烟雾系统的信号给中央处理器。本发明先通过目标分割的方法将烤箱内的食物进行像素级的分割,然后选取1个食材区域作为观察对象,计算该观察对象的清晰度系数,判断烤箱内是否有过量的烟雾,拍摄的图片是否会影响观测食物烘烤状态。(The invention discloses an oven smoke monitoring method, which comprises the following steps: before the oven works, acquiring an image in the oven, carrying out image segmentation, selecting a segmented food material region as an observation object, and determining a gradient matrix and a contrast matrix of the observation object; when the oven works, a camera in the oven is controlled to capture pictures in the oven according to a set time interval, and a gradient matrix and a contrast matrix of an observed object in the current picture are determined; determining a definition coefficient of an observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix; and when the definition coefficient is smaller than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be opened to the central processing unit. According to the method, firstly, the food in the oven is subjected to pixel-level segmentation by a target segmentation method, then 1 food material area is selected as an observation object, the definition coefficient of the observation object is calculated, and whether excessive smoke exists in the oven or not and whether the baking state of the food is influenced by a shot picture or not is judged.)

1. An oven smoke monitoring method, comprising:

before the oven works, acquiring an image in the oven, carrying out image segmentation, selecting a segmented food material region as an observation object, and determining a gradient matrix and a contrast matrix of the observation object;

when the oven works, a camera in the oven is controlled to capture pictures in the oven according to a set time interval, and a gradient matrix and a contrast matrix of the observed object in the current picture are determined;

determining a definition coefficient of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

and when the definition coefficient is smaller than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be opened to a central processing unit.

2. The method of claim 1, wherein determining the sharpness coefficient for the observed object based on the two sets of gradient matrices and contrast matrices comprises:

calculating a gradient change factor delta and a contrast change factor tau of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

calculating the definition of the observed object in the current picture by combining the gradient change factor delta and the contrast change factor tauDegree coefficient is formed by k₁*δ+k₂τ, wherein k₁+k₂＝1。

3. The method of claim 2, wherein k is k₁＝0.3，k₂＝0.7。

4. The method of claim 1, wherein the threshold is 0.62.

5. The method of claim 1, further comprising:

and when the definition coefficient is larger than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be closed to the central processing unit.

6. The method of claim 1, wherein the obtaining of the picture in the oven, the image segmentation, and the selecting of the segmented food material region as the observation object comprise:

acquiring an image in an oven, and performing image segmentation based on a pre-trained image segmentation model to obtain one or more food material areas;

and selecting one food material area from the one or more food material areas as an observation object.

7. An oven smoke monitoring device, comprising:

the first image analysis module is used for acquiring pictures in the oven before the oven works, carrying out image segmentation, selecting a segmented food material region as an observation object, and determining a gradient matrix and a contrast matrix of the observation object;

the second image analysis module is used for controlling a camera in the oven to capture pictures in the oven according to a set time interval when the oven works, and determining a gradient matrix and a contrast matrix of the observed object in the current picture;

the definition coefficient determining module is used for determining the definition coefficient of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

and the signal sending module is used for sending a signal for triggering the oven smoke discharging system to be opened to the central processing unit when the definition coefficient is smaller than a set threshold value.

8. The apparatus of claim 7, wherein the sharpness factor determination module comprises:

the change factor determination submodule is used for calculating a gradient change factor delta and a contrast change factor tau of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

a definition coefficient calculation submodule for calculating a definition coefficient e ═ k of the observed object in the current picture by combining the gradient change factor delta and the contrast change factor tau₁*δ+k₂τ, wherein k₁+k₂＝1。

9. The apparatus of claim 7, wherein the threshold is 0.62.

10. A storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the oven smoke monitoring method according to any one of claims 1 to 6.

Technical Field

The invention relates to the technical field of image processing, in particular to an oven smoke monitoring method and device and a storage medium.

Background

Along with the popularization of intelligent technology, various electric enterprises have introduced intelligent ovens with built-in cameras. Such intelligent oven can also be interconnected with the cell-phone through the network, and the user can look over the toasting state of food through watching the real-time video in the oven. However, in the baking process of the oven, especially in the machine with the steaming and baking integrated function, the user often cannot see the baking state of the food because the concentration of the smoke in the oven is too high.

CN 102136059 a-a smoke detection method based on video analysis "discloses a smoke detection method based on video analysis, which is implemented with the support of a digital camera as a sensor and a digital signal processing chip, and comprises the following steps: acquiring a digital video by using a digital camera; screening out a foreground part containing a motion part; screening out areas with similar smoke by using a support vector machine detector; analyzing the change of the high-frequency signal by using wavelet transformation, and screening out a digital image of which the background is gradually changed in a fuzzy manner; and finally, screening out digital images with smoke texture characteristics by using an Adaboost cascade classifier pair.

According to the scheme, smoke is detected through combination of methods such as background modeling, SVM and cascade classifier, and the position of a smoke generation point in an image can be found. The disadvantages are too complex and large calculation amount. Cannot be used effectively in this scenario-specific scenario of an oven: it is not necessary to know where the smoke is generated in the oven, and it is only necessary to judge whether the current smoke affects the observation of the food baking state.

Disclosure of Invention

The embodiment of the invention provides an oven smoke monitoring method and device and a storage medium. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

According to a first aspect of embodiments of the present invention, there is provided an oven smoke monitoring method, comprising:

before the oven works, acquiring an image in the oven, carrying out image segmentation, selecting a segmented food material region as an observation object, and determining a gradient matrix and a contrast matrix of the observation object;

when the oven works, a camera in the oven is controlled to capture pictures in the oven according to a set time interval, and a gradient matrix and a contrast matrix of the observed object in the current picture are determined;

determining a definition coefficient of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

and when the definition coefficient is smaller than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be opened to a central processing unit.

In one embodiment, the determining the sharpness coefficient of the observed object based on the two sets of gradient matrices and contrast matrices includes:

calculating a gradient change factor delta and a contrast change factor tau of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

calculating the definition coefficient E ═ k of the observed object in the current picture by combining the gradient change factor delta and the contrast change factor tau₁*δ+k₂τ, wherein k₁+k₂＝1。

In one embodiment, k is₁＝0.3，k₂＝0.7。

In one embodiment, the threshold is 0.62.

In one embodiment, the method further comprises:

and when the definition coefficient is larger than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be closed to the central processing unit.

In one embodiment, the acquiring a picture in an oven, performing image segmentation, and selecting a segmented food material region as an observation object includes:

acquiring an image in an oven, and performing image segmentation based on a pre-trained image segmentation model to obtain one or more food material areas;

and selecting one food material area from the one or more food material areas as an observation object.

According to a second aspect of embodiments of the present invention there is provided an oven smoke monitoring apparatus comprising:

the first image analysis module is used for acquiring pictures in the oven before the oven works, carrying out image segmentation, selecting a segmented food material region as an observation object, and determining a gradient matrix and a contrast matrix of the observation object;

the second image analysis module is used for controlling a camera in the oven to capture pictures in the oven according to a set time interval when the oven works, and determining a gradient matrix and a contrast matrix of the observed object in the current picture;

the definition coefficient determining module is used for determining the definition coefficient of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

and the signal sending module is used for sending a signal for triggering the oven smoke discharging system to be opened to the central processing unit when the definition coefficient is smaller than a set threshold value.

In one embodiment, the sharpness coefficient determination module comprises:

the change factor determination submodule is used for calculating a gradient change factor delta and a contrast change factor tau of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

a definition coefficient calculation submodule for calculating a definition coefficient e ═ k of the observed object in the current picture by combining the gradient change factor delta and the contrast change factor tau₁*δ+k₂τ, wherein k₁+k₂＝1。

In one embodiment, k is₁＝0.3，k₂＝0.7。

In one embodiment, the threshold is 0.62.

In one embodiment, the signal sending module is further configured to:

and when the definition coefficient is larger than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be closed to the central processing unit.

In one embodiment, the first image analysis module is configured to:

acquiring an image in an oven, and performing image segmentation based on a pre-trained image segmentation model to obtain one or more food material areas;

and selecting one food material area from the one or more food material areas as an observation object.

According to a third aspect of embodiments of the present invention, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements an oven smoke monitoring method provided by embodiments of the present invention.

According to a fourth aspect of embodiments of the present invention, there is provided a computer apparatus.

In some optional embodiments, the computer device comprises a memory, a processor and a program stored on the memory and executable by the processor, the processor implementing the oven smoke monitoring method described above when executing the program.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

the invention provides a scheme and a detailed solution for improving the effect of monitoring the state of food in a box body by an oven with a camera. In order to reduce the calculation amount, the food in the oven is firstly subjected to pixel-level segmentation by a target segmentation method, then 1 food material region is selected as an observation object, a gradient matrix and a contrast matrix of the observation object before baking and a gradient matrix and a contrast matrix of the observation object in the baking process are counted, a definition coefficient is calculated based on the two groups of matrixes, and whether excessive smoke exists in the oven at present or not is judged, and whether a shot picture influences the baking state of the food to be observed or not is judged.

According to the scheme, the smoke detection is simplified into the calculation of the change condition of the definition of a certain image observation area, the definition measurement model is constructed, and the problem that the food baking state in the oven is influenced by smoke when a user observes through the camera is effectively solved.

When the smoke concentration is too high to influence the user to watch the food baking state, the smoke exhaust system in the oven can be automatically started, and the smoke concentration in the oven is reduced until the user does not influence the food baking state.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a schematic diagram illustrating the structure of a smart oven according to an exemplary embodiment;

FIG. 2 is a schematic flow diagram illustrating an oven smoke monitoring method according to an exemplary embodiment;

FIG. 3 is a block diagram of an oven smoke monitoring device according to an exemplary embodiment

FIG. 4 is a block diagram illustrating a configuration of a computer device according to an example embodiment.

Detailed Description

The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, process, and other changes. The examples merely typify possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in or substituted for those of others. The scope of embodiments of the invention encompasses the full ambit of the claims, as well as all available equivalents of the claims. Embodiments may be referred to herein, individually or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method or apparatus that comprises the element. The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. As for the methods, products and the like disclosed by the embodiments, the description is simple because the methods correspond to the method parts disclosed by the embodiments, and the related parts can be referred to the method parts for description.

This scheme is applied to the intelligent oven that has built-in camera, and the camera can carry out the image shooting at the oven during operation, the food culinary art state in the control oven. The camera is arranged at the left upper vertex of the oven box body as shown in figure 1, and the shooting range of the camera is widest at the position.

As shown in fig. 2, the present invention provides a method for monitoring oven smoke, comprising:

s201, before the oven works, obtaining an in-oven picture, carrying out image segmentation, selecting a segmented food material region as an observation object, and determining a gradient matrix and a contrast matrix of the observation object;

s202, when the oven works, a camera in the oven is controlled to capture pictures in the oven according to a set time interval, and a gradient matrix and a contrast matrix of the observed object in the current picture are determined;

s203, determining a definition coefficient of the observed object in the current picture based on the two groups of gradient matrixes and the contrast matrix;

and S204, when the definition coefficient is smaller than a set threshold value, sending a signal for triggering the oven smoke discharging fog system to be opened to a central processing unit.

In one embodiment, before the oven works, an in-oven picture is obtained, image segmentation is carried out, a segmented food material area is selected as an observation object, and the method is realized in the following mode:

acquiring an image in an oven, and performing image segmentation based on a pre-trained image segmentation model to obtain one or more food material areas;

and selecting one food material area from the one or more food material areas as an observation object.

In practical application, after the image I in the oven is obtained, the image I in the oven is segmented by using a MASK-RCNN method to obtain accurate regions R of various food materials in the image.

The MASK-RCNN is a detection segmentation algorithm, and can realize pixel-level identification on the target on the basis of detecting the target. For example, a picture is shot in an oven, a chicken wing and a chicken leg are arranged in the picture, and by utilizing the MASK-RCNN algorithm, which pixel points in the picture belong to the chicken wing, which pixel points belong to the chicken leg and which pixel points belong to the background can be accurately determined.

Firstly, pictures of 100 common food materials are collected and labeled to serve as training samples, model training is carried out on the training samples based on the MASK-RCNN algorithm, and an image segmentation model capable of distinguishing the 100 food materials is trained to serve as the image segmentation model. The model can accurately mark 100 food materials.

Then, inputting a picture I, and utilizing the trained image segmentation model to perform image segmentation on each pixel point in the picture IMake a mark, note Pixel_iWherein I is 0,1,2, …, and 100(0 to 99 represent the numbers of 100 common food materials, and correspond to potatoes, chicken wings, corns, …, and 100 correspond to background pixels, that is, pixels not belonging to the 100 food materials), so that a pixel point where each food material is located can be determined, and further, the image I can be divided into one or more food material regions.

Then, one of the segmented food material regions is selected as an observation object, marked as O, and the observation object can be randomly selected.

Further, after the observation object O is determined, the sharpness calculation may be performed for the observation object O.

In practical application, the Canny edge detection operator can be used for calculating the edge binary image I of the observed object O_xeqe。

The method comprises the following specific steps:

(a) obtaining Gray scale image I according to formula Gray of 0.299R +0.587G +0.114B for RGB color image_GWherein, R, G, B are the images of R, G, B three channels of RGB color image respectively.

(b) And (3) carrying out Gaussian filtering on the gray level image, wherein the Gaussian radius is 3, and a two-dimensional Gaussian kernel parameter calculation formula is as follows:

the filtered image is I_s。

(c) Computing an image I_sThe gradient map and the gradient directional diagram of (c), the calculation formula is as follows:

P[i，j]＝(f[i，j+1]-f[i，j]+f[i+1，j+1]-f[i+1，j])/2

Q[i，j]＝(f[j，j]-f[i+1，j]+f[i，j+1]-f[i+1，j+1])/2

θ[i，j]＝arctan(Q[i，j]/p[i，j)]

where f [ i, j ] represents the pixel value at the position of the original image matrix i, j, M is the resulting gradient matrix, and θ is the gradient direction matrix.

At the same time, a contrast matrix C of the image I of the observation object O is calculated. The calculation formula is as follows:

c [ i, j ] represents the contrast value of the image matrix of the observed object O at i, j.

The oven was started. And capturing a picture every 3 seconds by a camera in the oven, and calculating a gradient matrix M 'i, j and a contrast matrix C' i, j of the observed object O aiming at the captured picture.

Based on the gradient matrix and the contrast matrix of the image I and the gradient matrix and the contrast matrix of the image captured after the oven works, calculating a gradient change factor delta and a contrast change factor tau, wherein the calculation formula is as follows:

where M, n are the width and height of the matrix, and M is the total number of pixels of the observed object O

Further, combining the gradient change factor delta and the contrast change factor tau, calculating a definition coefficient epsilon of the observed object in the current picture, wherein the definition coefficient epsilon is k₁*δ+k₂τ, wherein k₁+k₂＝1。

In one embodiment, k is₁＝0.3，k₂＝0.7。

If the current definition coefficient epsilon is smaller than the set threshold value, the judgment result is that smoke exists, the definition of the image is influenced, then a signal is sent to the central processing unit, the central processing unit is triggered to open a smoke discharging system of the oven, the fan is opened to blow air into the oven, and the smoke and the fog in the oven are discharged out of the oven through an air channel in the oven. When the smoke concentration in the oven is reduced to a value which does not affect the observation, namely the definition coefficient epsilon exceeds a set threshold value, a signal is sent to the central processing unit, and the central processing unit is triggered to close a smoke discharging fog system of the oven. In practical applications, the threshold value may preferably be 0.62.

As shown in fig. 3, the present invention also provides an oven smoke monitoring device, comprising:

the first image analysis module 301 is configured to, before the oven works, acquire an image in the oven, perform image segmentation, select a segmented food material region as an observation object, and determine a gradient matrix and a contrast matrix of the observation object;

the second image analysis module 302 is used for controlling a camera in the oven to capture pictures in the oven according to a set time interval when the oven works, and determining a gradient matrix and a contrast matrix of the observed object in the current picture;

a definition coefficient determining module 303, configured to determine a definition coefficient of the observed object in the current picture based on the two sets of gradient matrices and the contrast matrix;

and the signal sending module 304 is used for sending a signal for triggering the oven smoke discharging system to be opened to the central processing unit when the definition coefficient is smaller than a set threshold value.

In one embodiment, the sharpness coefficient determination module 303 includes:

a change factor determination submodule 3031, configured to calculate a gradient change factor δ and a contrast change factor τ of the observation object in the current picture based on the two sets of gradient matrices and the contrast matrix;

a sharpness coefficient calculation submodule 3032, configured to calculate, in combination with the gradient change factor δ and the contrast change factor τ, a sharpness coefficient e ∈ ═ k of the observed object in the current picture₁*δ+k₂τ, wherein k₁+k₂＝1。

In one embodiment, k is₁＝0.3，k₂＝0.7。

In one embodiment, the threshold is 0.62.

In one embodiment, the signal sending module 304 is further configured to:

and when the definition coefficient is larger than a set threshold value, sending a signal for triggering the oven smoke exhaust fog system to be closed to the central processing unit.

In one embodiment, the first image analysis module 301 is configured to:

acquiring an image in an oven, and performing image segmentation based on a pre-trained image segmentation model to obtain one or more food material areas;

and selecting one food material area from the one or more food material areas as an observation object.

According to a third aspect of embodiments of the present invention, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements an oven smoke monitoring method provided by embodiments of the present invention.

As shown in fig. 4, the present invention also provides a computer device.

The computer device comprises a memory 401, a processor 402 and a program stored on the memory 401 and executable by the processor 402, wherein the processor 402 implements the oven smoke monitoring method described above when executing the program

In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor to perform the method described above is also provided. The non-transitory computer readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic tape, an optical storage device, and the like.

Those of skill in the art would appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the embodiments disclosed herein, it should be understood that the disclosed methods, articles of manufacture (including but not limited to devices, apparatuses, etc.) may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form. The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment. In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

It should be understood that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. The present invention is not limited to the procedures and structures that have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.