阅读说明：本技术 基于众包智能的垃圾分类系统 (Garbage classification system based on crowdsourcing intelligence ) 是由 卢向明 钱江波 彭良康 于 2021-09-23 设计创作，主要内容包括：本发明公开了一种基于众包智能的垃圾分类系统,通过设置人机交互界面、众包模块和数据管理模块,众包模块包括众包审核子模块和众包标注子模块,众包审核子模块用于以派发审核订单的形式进行任务分发给众包审核人员,进行垃圾投放分类的人工审查给出识别结果,数据管理模块能够构建训练数据集提供给云服务器,用于对图像识别模块进行训练更新；优点是在图像识别模块不能识别垃圾类别时,能够通过众包审核方式从外界获取垃圾识别结果,以保证垃圾分类投放的顺利进行,并且能够采用众包标注方式构建训练数据集,从而采用训练数据集对图像识别模块进行更新训练,保证图像识别模块识别准确性。(The invention discloses a garbage classification system based on crowdsourcing intelligence, which is characterized in that a man-machine interaction interface, a crowdsourcing module and a data management module are arranged, wherein the crowdsourcing module comprises a crowdsourcing auditing submodule and a crowdsourcing marking submodule, the crowdsourcing auditing submodule is used for carrying out tasks in a mode of dispatching an auditing order and distributing the tasks to crowdsourcing auditors, manual auditing for garbage putting classification is carried out, and an identification result is given out; the method has the advantages that when the image recognition module cannot recognize the garbage category, the garbage recognition result can be obtained from the outside through a crowdsourcing auditing mode so as to ensure that garbage classification is smoothly put in, and a training data set can be constructed in a crowdsourcing labeling mode so as to update and train the image recognition module by adopting the training data set, thereby ensuring the recognition accuracy of the image recognition module.)

1. A garbage classification system based on crowdsourcing intelligence comprises a box body, a control module, a collection transmission module and a cloud server, wherein an image recognition module is arranged on the cloud server, the box body comprises a plurality of sub-boxes marked with garbage categories, each sub-box is respectively provided with a sensor used for recognizing whether garbage is thrown in, a material dialing device used for receiving the recognition position of the thrown garbage and dialing the garbage into the sub-box when the thrown garbage is classified correctly, each sensor and each material dialing device are respectively connected with the control module, the collection transmission module consists of a camera device which is arranged above each sub-box and used for shooting garbage images, each camera device is respectively connected with the control module, and the control module and the cloud server are communicated through a wireless network, the intelligent garbage classification system is characterized by further comprising a human-computer interaction interface, a crowdsourcing module and a data management module, wherein the human-computer interaction interface can be used for a dispenser to select garbage categories, swipe a card for identity verification and display related information of garbage throwing, the human-computer interaction interface is connected with the control module, the image identification module and the crowdsourcing module are used for classifying and identifying the thrown garbage, when the thrown garbage is not accurately thrown and classified by the dispenser or mixed loading of non-heterogeneous garbage exists in the thrown garbage, the control module can remind the dispenser to throw or reclassify again through the human-computer interaction interface, the priority of garbage identification is changed from high to low by the image identification module and the crowdsourcing module, and the crowdsourcing module is used for identifying the garbage unsuccessfully when the image identification module fails, when the dispenser applies for remote assistance through the human-computer interaction interface, tasks are distributed to crowdsourcing auditors in a mode of dispatching audit orders, manual audit of garbage dispensing classification is carried out, and then recognition results are given; the data management module is connected with the image identification module, the crowdsourcing module comprises a crowdsourcing auditing submodule and a crowdsourcing marking submodule, the crowdsourcing auditing submodule is connected with the image identification module, the crowdsourcing marking submodule is connected with the data management module, the crowdsourcing auditing submodule is used for carrying out task distribution to crowdsourcing auditing personnel in a mode of sending an auditing order, carrying out manual review of garbage throwing classification to give an identification result, the data management module is used for collecting thrown garbage pictures applying for remote assistance and issuing the thrown garbage pictures to the crowdsourcing marking submodule, the crowdsourcing marking submodule is used for distributing the thrown garbage pictures as crowdsourcing tasks to crowdsourcing marking personnel with marking qualification, and after the crowdsourcing marking personnel receive the crowdsourcing tasks, corresponding labels are selected from a candidate label set preset in the crowdsourcing marking submodule to mark the thrown garbage pictures, the method comprises the steps of obtaining pictures with labels, namely picture labels, and sending the pictures to a data management module, wherein each label in a candidate label set represents each garbage category, the data management module constructs a training data set based on putting garbage pictures and picture labels to be provided for a cloud server for training and updating an image identification module, crowdsourcing annotating personnel need to register in a crowdsourcing annotating submodule to obtain annotation qualification, the profession and interest of the crowdsourcing annotating personnel need to be written during registration, the picture labels generated after the crowdsourcing annotating personnel complete crowdsourcing tasks are stored as historical picture labels of the crowdsourcing annotating personnel, and the profession, the interest and the historical picture labels of each crowdsourcing annotating personnel form personal information of the crowdsourcing personnel.

2. The garbage classification system based on crowdsourcing intelligence as claimed in claim 1, wherein the specific process of constructing the training data set is as follows:

s1, taking the crowdsourcing annotating personnel which are registered in the crowdsourcing annotating submodule and have historical picture labels as current crowdsourcing annotating personnel, counting the total number of the current crowdsourcing annotating personnel, marking the total number as n, calculating the similarity of every two of the current crowdsourcing annotating personnel, and marking the similarity between the r-th current crowdsourcing annotating personnel and the t-th current crowdsourcing annotating personnel as w_rtWherein r, t is belonged to {1,2 … n }, and r is not equal to t, and w is obtained by calculation by adopting a formula (1)_rt，

Wherein, n (r) is a set of expertise and interest of an r-th current crowdsourcing annotator, n (t) is a set of expertise and interest of a t-th current crowdsourcing annotator, l (r) is a set of tags attached to historical picture tags of the r-th current crowdsourcing annotator, l (t) is a set of tags attached to historical picture tags of the t-th current crowdsourcing annotator, α and β are weight parameters respectively, the value ranges are both [0,1], the symbol |, is an intersection of the two sets, and the symbol | | | represents the number of elements in the calculation set;

s2, sequencing all the similarity degrees obtained in the step S1, determining 5 similarity degrees with the largest value, obtaining pictures corresponding to historical picture labels of crowdsourcing annotators related to the 5 similarity degrees, forming a picture set result to be recommended by adopting the pictures, and recording the total number of the pictures in the result as S;

s3, recording a label set formed by labels attached to historical picture labels of curr-th current crowdsourcing annotator in n current crowdsourcing annotator as L (curr), wherein curr belongs to {1,2 … n }, and recording the similarity between curr-th current crowdsourcing annotator and a picture a in a picture set result to be recommended as sim_curr,aWhere a is e {1,2 … S }, adoptObtaining sim by calculation of formula (2)_curr,a：

Wherein, L ' (a) is all the label sets currently labeled to the a-th picture in the picture set result to be recommended, the symbol | | represents the number of elements in the calculation set, x represents any label in the intersection of the set L ' (a) and the set L (curr), Σ is a summation symbol, num (x) is the number of times of appearance of the label x in the set L ' (a), a picture with similarity higher than a similarity threshold in the picture set result to be recommended is adopted to form a final pushed picture data set final, N is the total number of pictures in the final pushed picture data set final, the range of the similarity threshold is [0.3,0.5], the crowdsourcing labeling submodule pushes each picture in the final pushed picture data set final to K crowdsourcing labels respectively for classification, K is an integer, and the range of the values is [5,10 ];

s4, K crowdsourcing annotators respectively annotate each picture received by the crowdsourcing annotator, and the obtained annotation result of all the pictures is called a crowdsourcing data set D, wherein,x_ipicture i, i ∈ {1,2 … N }, Y, representing the final pushed picture dataset final_iDenotes x_iTag set of (1), by x_iIs formed by K labels obtained after being respectively labeled by K crowdsourcing labels,y_ikrepresenting the kth crowd-sourced tagger pair x_iThe label to be labeled, K is formed by {1,2 … K }, and the label is a candidate label set { y } preset in the crowdsourcing labeling submodule by the kth crowdsourcing labeling person₁,y₂,…,y_cIs selected, c represents the total category of the labels in the candidate label set, y_jRepresenting jth class label in the candidate label set, wherein j belongs to {1,2 … c };

s5, statistics of Y_iNumber of categories of middle label, which is denoted as H_iThen separately counting Y_iThe number of times of occurrence of various labels in_iDividing the occurrence frequency of each type of label by K to obtain Y_iThe frequency of occurrence of each type of tag in (1) will be Y_iMiddle h_iThe frequency of occurrence of class labels is recorded asThe ith picture x_iThe entropy of the labeling result is recorded as E_i，E_iThe calculation is carried out by adopting the formula (3):

wherein, log is a logarithmic sign;

s6, determining the ith picture x of the final pushed picture data set final_iTag set Y of_iThe label with the most occurrence times is taken as the ith picture x of the final pushed picture data set final_iTransition label ofIf the labels with the most occurrence times exist in two or more types, one type of label is randomly selected as a transition labelAt this time, each picture in the final pushed picture data set final corresponds to one transition tag, and the crowd-sourced data set is constructed by adopting the pictures in the final pushed picture data set final and the transition tags of each picture

S7, adopting 3-fold cross validation to crowd-sourced data setProcessing, i.e. crowdsourcing, of data setsDividing the data into 3 parts at random, and respectively calling the three parts as first part data, second part data and third part data; training a classifier constructed by using a Resnet34 network by using the first data and the second data as training data, and predicting each picture in the third data by using the trained classifier to obtain prediction labels of the pictures; training a classifier constructed by using a Resnet34 network by using the second data and the third data as training data, and predicting each picture in the first data by using the trained classifier to obtain prediction labels of the pictures; training a classifier constructed by using a Resnet34 network by using the first data and the third data as training data, and predicting each picture in the second data by using the trained classifier to obtain prediction labels of the pictures; obtaining the prediction labels corresponding to all the pictures in the final pushed picture data set final, and adopting all the pictures in the final pushed picture data set final and the prediction labels corresponding to all the pictures to form a crowdsourcing data set Picture x of the ith representing the final pushed picture data set final_iThe predictive tag of (a);

s8, respectively comparing the transition label and the prediction label of each picture in the final pushed picture data set final, if the transition label and the prediction label of a certain picture are consistent, the entropy of the labeling result of the picture is less than 0.1, selecting the picture as a training picture, otherwise, not selecting the picture as data which does not meet the conditions, recording the total number of the training pictures obtained at the moment as m, and adopting the m training pictures obtained at the moment to form training dataCollectionx'_zRepresents the z picture, y 'of the m training pictures'_zIs x'_zA corresponding transition label;

s9, training classifiers constructed by using a Resnet34 network by adopting a training data set D 'to obtain the trained classifiers, predicting unselected pictures in final pushed picture data sets final by adopting the classifiers to obtain prediction labels of the pictures, forming a group of training data by the unselected pictures and the prediction labels thereof, and merging the training data set D' with the training data formed by the unselected pictures and the prediction labels thereof to obtain a final training data set.

Technical Field

The invention relates to a garbage classification system, in particular to a garbage classification system based on crowdsourcing intelligence.

Background

With the continuous high-speed development of economy and the rapid expansion of urbanization, the garbage production amount is increased sharply. The problems of random stacking of domestic garbage, large amount of garbage occupying land, serious environmental pollution, large resource waste and the like are brought about, and become important factors influencing environmental protection and sustainable development. According to the practical characteristics, different classification and collection and transportation methods are adopted, so that the single mode of mixed clearing and transportation of the modern municipal solid waste is effectively solved, and the method becomes one of the problems to be solved at present.

The Chinese patent application with the publication number of CN109368092A discloses an intelligent classification dustbin, which comprises a dustbin body, an infrared detector, a micro-camera, a control module (realized by adopting a single chip microcomputer) and a cloud server, wherein a rubbish throwing port is arranged on the dustbin body, a plurality of sub-dustbin bodies are arranged in the dustbin body and are flatly laid below the throwing port, a distribution device connected with the control module is arranged in the dustbin body, the infrared detector is arranged at the rubbish throwing port and is used for detecting whether rubbish is thrown in the rubbish throwing port in real time, the control module is communicated with the cloud server through a wireless network, an image recognition module is arranged in the cloud server, when rubbish is thrown in the dustbin body through the rubbish throwing port, the infrared detector sends a signal to the control module, the control module controls the micro-camera to be opened, the micro-camera takes a picture of the rubbish entering the dustbin body and outputs a picture file obtained by shooting to the control module, the control module transmits the picture file to the cloud server, the image recognition module at the cloud server recognizes the picture file, the garbage classification is determined and fed back to the control module, and the control module controls the allocation device to allocate garbage into the corresponding classification sub-box body according to the garbage classification, so that classified garbage putting is realized.

However, the above-mentioned intelligent classification garbage can has the following problems: firstly, the actual accuracy of the image recognition module cannot reach one hundred percent, and because the classification capability and consciousness of the resident garbage are weak at present, the thrown garbage is often mixed with different types of garbage, once the image recognition module cannot recognize the garbage types, the distribution device cannot be driven, and the garbage can be thrown all the time unsuccessfully. Secondly, the image recognition module needs to adopt a large amount of labeled picture data related to garbage to train and then can be used for garbage classification, but the traditional mode for acquiring the labeled picture data is to label the picture manually, and the mode is time-consuming, labor-consuming and high in cost, and the number of manually labeled pictures is limited, so that the continuous updating of the image recognition module is difficult to maintain, the recognition precision of the image recognition module is reduced, and the accuracy of garbage classification is reduced.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a garbage classification system based on crowdsourcing intelligence, which can acquire garbage identification results from the outside through a crowdsourcing auditing mode to ensure the smooth garbage classification and putting, and can establish a training data set through a crowdsourcing marking mode, so that the image identification module is updated and trained through the training data set, and the identification accuracy of the image identification module is ensured.

The technical scheme adopted by the invention for solving the technical problems is as follows: a garbage classification system based on crowdsourcing intelligence comprises a box body, a control module, a collection transmission module and a cloud server, wherein an image recognition module is arranged on the cloud server, the box body comprises a plurality of sub-boxes marked with garbage categories, each sub-box is respectively provided with a sensor used for recognizing whether garbage is thrown in, a material dialing device used for receiving the recognition position of the thrown garbage and dialing the garbage into the sub-box when the thrown garbage is classified correctly, each sensor and each material dialing device are respectively connected with the control module, the collection transmission module consists of a camera device which is arranged above each sub-box and used for shooting garbage images, each camera device is respectively connected with the control module, and the control module and the cloud server are communicated through a wireless network, the intelligent garbage classification system further comprises a human-computer interaction interface, a crowdsourcing module and a data management module, wherein the human-computer interaction interface can be used for a dispenser to select garbage categories, swipe a card for identity verification and display related information of garbage throwing, the human-computer interaction interface is connected with the control module, the image identification module and the crowdsourcing module are both used for classifying and identifying the thrown garbage, when the thrown garbage is not accurately thrown by the dispenser or the thrown garbage has non-heterogeneous garbage mixed loading, the control module can remind the dispenser to throw or reclassify again through the human-computer interaction interface, the priority of garbage identification is changed from high to low when the image identification module and the crowdsourcing module carry out garbage identification, and the crowdsourcing module is used for identifying the garbage unsuccessfully when the image identification module, when the dispenser applies for remote assistance through the human-computer interaction interface, tasks are distributed to crowdsourcing auditors in a mode of dispatching audit orders, manual audit of garbage dispensing classification is carried out, and then recognition results are given; the data management module is connected with the image identification module, the crowdsourcing module comprises a crowdsourcing auditing submodule and a crowdsourcing marking submodule, the crowdsourcing auditing submodule is connected with the image identification module, the crowdsourcing marking submodule is connected with the data management module, the crowdsourcing auditing submodule is used for carrying out task distribution to crowdsourcing auditing personnel in a mode of sending an auditing order, carrying out manual review of garbage throwing classification to give an identification result, the data management module is used for collecting thrown garbage pictures applying for remote assistance and issuing the thrown garbage pictures to the crowdsourcing marking submodule, the crowdsourcing marking submodule is used for distributing the thrown garbage pictures as crowdsourcing tasks to crowdsourcing marking personnel with marking qualification, and after the crowdsourcing marking personnel receive the crowdsourcing tasks, corresponding labels are selected from a candidate label set preset in the crowdsourcing marking submodule to mark the thrown garbage pictures, the method comprises the steps of obtaining pictures with labels, namely picture labels, and sending the pictures to a data management module, wherein each label in a candidate label set represents each garbage category, the data management module constructs a training data set based on putting garbage pictures and picture labels to be provided for a cloud server for training and updating an image identification module, crowdsourcing annotating personnel need to register in a crowdsourcing annotating submodule to obtain annotation qualification, the profession and interest of the crowdsourcing annotating personnel need to be written during registration, the picture labels generated after the crowdsourcing annotating personnel complete crowdsourcing tasks are stored as historical picture labels of the crowdsourcing annotating personnel, and the profession, the interest and the historical picture labels of each crowdsourcing annotating personnel form personal information of the crowdsourcing personnel.

The specific process of constructing the training data set comprises the following steps:

s1, taking the crowdsourcing annotating personnel which are registered in the crowdsourcing annotating submodule and have historical picture labels as current crowdsourcing annotating personnel, counting the total number of the current crowdsourcing annotating personnel, marking the total number as n, calculating the similarity of every two of the current crowdsourcing annotating personnel, and marking the similarity between the r-th current crowdsourcing annotating personnel and the t-th current crowdsourcing annotating personnel as w_rtWherein r, t is belonged to {1,2 … n }, and r is not equal to t, and w is obtained by calculation by adopting a formula (1)_rt，

Wherein, n (r) is a set of expertise and interest of an r-th current crowdsourcing annotator, n (t) is a set of expertise and interest of a t-th current crowdsourcing annotator, l (r) is a set of tags attached to historical picture tags of the r-th current crowdsourcing annotator, l (t) is a set of tags attached to historical picture tags of the t-th current crowdsourcing annotator, α and β are weight parameters respectively, the value ranges are both [0,1], the symbol |, is an intersection of the two sets, and the symbol | | | represents the number of elements in the calculation set;

s2, sequencing all the similarity degrees obtained in the step S1, determining 5 similarity degrees with the largest value, obtaining pictures corresponding to historical picture labels of crowdsourcing annotators related to the 5 similarity degrees, forming a picture set result to be recommended by adopting the pictures, and recording the total number of the pictures in the result as S;

s3, recording a label set formed by labels attached to historical picture labels of curr-th current crowdsourcing annotator in n current crowdsourcing annotator as L (curr), wherein curr belongs to {1,2 … n }, and recording the similarity between curr-th current crowdsourcing annotator and a picture a in a picture set result to be recommended as sim_curr,aWherein a is equal to {1,2 … S }, sim is obtained by calculation by adopting formula (2)_curr,a：

Wherein, L ' (a) is all the label sets currently labeled to the a-th picture in the picture set result to be recommended, the symbol | | represents the number of elements in the calculation set, x represents any label in the intersection of the set L ' (a) and the set L (curr), Σ is a summation symbol, num (x) is the number of times of appearance of the label x in the set L ' (a), a picture with similarity higher than a similarity threshold in the picture set result to be recommended is adopted to form a final pushed picture data set final, N is the total number of pictures in the final pushed picture data set final, the range of the similarity threshold is [0.3,0.5], the crowdsourcing labeling submodule pushes each picture in the final pushed picture data set final to K crowdsourcing labels respectively for classification, K is an integer, and the range of the values is [5,10 ];

s4, K crowdsourcing annotators respectively annotate each picture received by the crowdsourcing annotator, and the obtained annotation result of all the pictures is called a crowdsourcing data set D, wherein,x_ipicture i, i ∈ {1,2 … N }, Y, representing the final pushed picture dataset final_iDenotes x_iTag set of (1), by x_iIs formed by K labels obtained after being respectively labeled by K crowdsourcing labels,y_ikrepresenting the kth crowd-sourced tagger pair x_iThe label to be labeled, K is formed by {1,2 … K }, and the label is a candidate label set { y } preset in the crowdsourcing labeling submodule by the kth crowdsourcing labeling person₁,y₂,…,y_cIs selected, c represents the total category of the labels in the candidate label set, y_jRepresenting jth class label in the candidate label set, wherein j belongs to {1,2 … c };

s5, statistics of Y_iNumber of categories of middle label, which is denoted as H_iThen separately counting Y_iThe number of times that each type of tag appears in the tag,will Y_iDividing the occurrence frequency of each type of label by K to obtain Y_iThe frequency of occurrence of each type of tag in (1) will be Y_iMiddle h_iThe frequency of occurrence of class labels is denoted as f_hi，h_i∈{1,2…H_iH, the ith picture x_iThe entropy of the labeling result is recorded as E_i， E_iThe calculation is carried out by adopting the formula (3):

wherein, log is a logarithmic sign;

s6, determining the ith picture x of the final pushed picture data set final_iTag set Y of_iThe label with the most occurrence times is taken as the ith picture x of the final pushed picture data set final_iTransition label ofIf the labels with the most occurrence times exist in two or more types, one type of label is randomly selected as a transition labelAt this time, each picture in the final pushed picture data set final corresponds to one transition tag, and the crowd-sourced data set is constructed by adopting the pictures in the final pushed picture data set final and the transition tags of each picture

S7, adopting 3-fold cross validation to crowd-sourced data setProcessing, i.e. crowdsourcing, of data setsRandomly dividing the data into 3 parts, and respectively calling the three parts as first part data, second part data and third part data(ii) a Training a classifier constructed by using a Resnet34 network by using the first data and the second data as training data, and predicting each picture in the third data by using the trained classifier to obtain prediction labels of the pictures; training a classifier constructed by using a Resnet34 network by using the second data and the third data as training data, and predicting each picture in the first data by using the trained classifier to obtain prediction labels of the pictures; training a classifier constructed by using a Resnet34 network by using the first data and the third data as training data, and predicting each picture in the second data by using the trained classifier to obtain prediction labels of the pictures; obtaining the prediction labels corresponding to all the pictures in the final pushed picture data set final, and adopting all the pictures in the final pushed picture data set final and the prediction labels corresponding to all the pictures to form a crowdsourcing data set Picture x of the ith representing the final pushed picture data set final_iThe predictive tag of (a);

s8, respectively comparing the transition label and the prediction label of each picture in the final pushed picture data set final, if the transition label and the prediction label of a certain picture are consistent, and the marking result entropy of the picture is less than 0.1, selecting the picture as a training picture, otherwise, not selecting the picture as data which does not meet the conditions, recording the total number of the training pictures obtained at the moment as m, and adopting the m training pictures obtained at the moment to form a training data setx'_zRepresents the z picture, y 'of the m training pictures'_zIs x'_zA corresponding transition label;

s9, training classifiers constructed by using a Resnet34 network by adopting a training data set D 'to obtain the trained classifiers, predicting unselected pictures in final pushed picture data sets final by adopting the classifiers to obtain prediction labels of the pictures, forming a group of training data by the unselected pictures and the prediction labels thereof, and merging the training data set D' with the training data formed by the unselected pictures and the prediction labels thereof to obtain a final training data set.

Compared with the prior art, the invention has the advantages that through the arrangement of the human-computer interaction interface, the crowdsourcing module and the data management module, the human-computer interaction interface can be used for a dispenser to select garbage categories, to punch a card for identity verification and to display related information of garbage dispensing, the human-computer interaction interface is connected with the control module, the image recognition module and the crowdsourcing module are both used for classifying and recognizing the dispensed garbage, when the garbage dispensing classification of the dispenser is not accurate or the dispensed garbage has heterogeneous garbage mixed package, the control module can remind the dispenser to perform re-dispensing or re-classification through the human-computer interaction interface, the priority of the garbage recognition of the image recognition module and the crowdsourcing module is from high to low, the crowdsourcing module is used for distributing tasks to crowdsourcing auditors in the form of distributing audit orders when the image recognition module fails to recognize the garbage, and when the dispenser applies for remote assistance through the human-computer interaction interface, giving out an identification result after manual review of garbage putting classification; the data management module is connected with the image identification module, the crowdsourcing module comprises a crowdsourcing auditing submodule and a crowdsourcing labeling submodule, the crowdsourcing auditing submodule is connected with the image identification module, the crowdsourcing labeling submodule is connected with the data management module, the crowdsourcing auditing submodule is used for carrying out tasks in a mode of sending an auditing order and distributing the tasks to crowdsourcing auditors, manual examination of garbage throwing classification is carried out to give an identification result, the data management module is used for collecting thrown garbage pictures applying remote assistance and distributing the thrown garbage pictures to the crowdsourcing labeling submodule, the crowdsourcing labeling submodule is used for distributing the thrown garbage pictures as crowdsourcing tasks to crowdsourcing annotators with labeling qualification, after the crowdsourcing annotators receive the crowdsourcing tasks, corresponding labels are selected from a candidate label set preset in the crowdsourcing labeling submodule to mark the thrown garbage pictures to obtain pictures with labels, namely picture labels, and send to the data management module, wherein each label in the candidate label set represents each rubbish category respectively, the data management module constructs a training data set based on the thrown-in rubbish picture and the picture label and provides the training data set to the cloud server for training and updating the image identification module, crowdsourcing annotators need to register in the crowdsourcing annotating submodule to obtain the annotation qualification, the specialty and interest need to be written when registering, and the picture labels generated after the crowdsourcing annotators complete crowdsourcing tasks are stored as the historical picture labels, the specialty, interest and historical picture labels of each crowdsourcing annotator form personal information thereof, therefore, when the image identification module can not identify the rubbish categories, the invention can obtain rubbish identification results from the outside through crowdsourcing auditing mode to ensure the smooth progress of rubbish classification throwing, and a training data set can be constructed in a crowdsourcing marking mode, so that the image recognition module is updated and trained by the training data set, and the recognition accuracy of the image recognition module is ensured.

Drawings

Fig. 1 is a schematic block diagram of the garbage classification system based on crowdsourcing intelligence according to the present invention.

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

Example (b): as shown in figure 1, the garbage classification system based on crowdsourcing intelligence comprises a box body, a control module, a collection transmission module and a cloud server, wherein an image recognition module is arranged on the cloud server, the box body comprises a plurality of sub-boxes marked with garbage categories, a sensor used for recognizing whether garbage is thrown in or not, a recognition position used for receiving the thrown garbage and a material poking device used for poking the garbage into the sub-boxes when the thrown garbage is classified correctly are respectively arranged at each sub-box, each sensor and each material poking device are respectively connected with the control module, the collection transmission module consists of a camera device which is arranged above each sub-box and used for shooting garbage images, each camera device is respectively connected with the control module, the control module and the cloud server are communicated through a wireless network, the intelligent garbage classification system further comprises a human-computer interaction interface, a crowdsourcing module and a data management module, the man-machine interaction interface can be used for a dispenser to select garbage categories, a card is swiped for identity verification and relevant information of garbage throwing is displayed, the man-machine interaction interface is connected with the control module, the image recognition module and the crowdsourcing module are used for classifying and recognizing thrown garbage, when the refuse throwing classification of the dispenser is inaccurate or non-homogeneous garbage mixed loading exists in the thrown garbage, the control module can remind the dispenser to carry out throwing again or reclassification through the man-machine interaction interface, the priority of the refuse recognition of the image recognition module and the crowdsourcing module is from high to low, the crowdsourcing module is used for unsuccessfully recognizing garbage in the image recognition module, when the dispenser applies remote assistance through the man-machine interaction interface, tasks are distributed to crowdsourcing auditors in a form of dispatching orders for auditing, and manual auditing of the refuse throwing classification is carried out, and then recognition results are given; the data management module is connected with the image identification module, the crowdsourcing module comprises a crowdsourcing auditing submodule and a crowdsourcing labeling submodule, the crowdsourcing auditing submodule is connected with the image identification module, the crowdsourcing labeling submodule is connected with the data management module, the crowdsourcing auditing submodule is used for carrying out tasks in a mode of sending an auditing order and distributing the tasks to crowdsourcing auditors, manual examination of garbage throwing classification is carried out to give an identification result, the data management module is used for collecting thrown garbage pictures applying remote assistance and distributing the thrown garbage pictures to the crowdsourcing labeling submodule, the crowdsourcing labeling submodule is used for distributing the thrown garbage pictures as crowdsourcing tasks to crowdsourcing annotators with labeling qualification, after the crowdsourcing annotators receive the crowdsourcing tasks, corresponding labels are selected from a candidate label set preset in the crowdsourcing labeling submodule to mark the thrown garbage pictures to obtain pictures with labels, the image tags are sent to the data management module, each tag in the candidate tag set represents each garbage category, the data management module constructs a training data set based on the thrown garbage images and the image tags and provides the training data set to the cloud server for training and updating the image recognition module, crowdsourcing annotators need to register in the crowdsourcing annotating submodule to obtain the annotation qualification, the profession and the interest of the crowdsourcing annotators need to be written during registration, the image tags generated after the crowdsourcing annotators complete crowdsourcing tasks are stored as the historical image tags of the image tags, and the profession, the interest and the historical image tags of each crowdsourcing annotator form personal information of the crowdsourcing annotators.

In this embodiment, the specific process of constructing the training data set is as follows:

s1, taking the crowdsourcing annotating personnel which are registered in the crowdsourcing annotating submodule and have historical picture labels as current crowdsourcing annotating personnel, counting the total number of the current crowdsourcing annotating personnel, marking the total number as n, calculating the similarity of every two of the current crowdsourcing annotating personnel, and marking the similarity between the r-th current crowdsourcing annotating personnel and the t-th current crowdsourcing annotating personnel as w_rtWherein r, t is belonged to {1,2 … n }, and r is not equal to t, and w is obtained by calculation by adopting a formula (1)_rt，

Wherein, n (r) is a set of expertise and interest of an r-th current crowdsourcing annotator, n (t) is a set of expertise and interest of a t-th current crowdsourcing annotator, l (r) is a set of tags attached to historical picture tags of the r-th current crowdsourcing annotator, l (t) is a set of tags attached to historical picture tags of the t-th current crowdsourcing annotator, α and β are weight parameters respectively, the value ranges are both [0,1], the symbol |, is an intersection of the two sets, and the symbol | | | represents the number of elements in the calculation set;

s2, sequencing all the similarity degrees obtained in the step S1, determining 5 similarity degrees with the largest value, obtaining pictures corresponding to historical picture labels of crowdsourcing annotators related to the 5 similarity degrees, forming a picture set result to be recommended by adopting the pictures, and recording the total number of the pictures in the result as S;

s3, recording a label set formed by labels attached to historical picture labels of curr-th current crowdsourcing annotator in n current crowdsourcing annotator as L (curr), wherein curr belongs to {1,2 … n }, and recording the similarity between curr-th current crowdsourcing annotator and a picture a in a picture set result to be recommended as sim_curr,aWherein a is equal to {1,2 … S }, sim is obtained by calculation by adopting formula (2)_curr,a：

Wherein, L ' (a) is all the label sets currently labeled to the a-th picture in the picture set result to be recommended, the symbol | | represents the number of elements in the calculation set, x represents any label in the intersection of the set L ' (a) and the set L (curr), Σ is a summation symbol, num (x) is the number of times of appearance of the label x in the set L ' (a), a picture with similarity higher than a similarity threshold in the picture set result to be recommended is adopted to form a final pushed picture data set final, N is the total number of pictures in the final pushed picture data set final, the range of the similarity threshold is [0.3,0.5], the crowdsourcing labeling submodule pushes each picture in the final pushed picture data set final to K crowdsourcing labels respectively for classification, K is an integer, and the range of the values is [5,10 ];

s4, K crowdsourcing annotators respectively annotate each picture received by the crowdsourcing annotator, and the obtained annotation result of all the pictures is called a crowdsourcing data set D, wherein,x_ipicture i, i ∈ {1,2 … N }, Y, representing the final pushed picture dataset final_iDenotes x_iTag set of (1), by x_iIs formed by K labels obtained after being respectively labeled by K crowdsourcing labels,y_ikrepresenting the kth crowd-sourced tagger pair x_iThe label to be labeled, K is formed by {1,2 … K }, and the label is a candidate label set { y } preset in the crowdsourcing labeling submodule by the kth crowdsourcing labeling person₁,y₂,…,y_cIs selected, c represents the total category of the labels in the candidate label set, y_jRepresenting jth class label in the candidate label set, wherein j belongs to {1,2 … c };

s5, statistics of Y_iNumber of categories of middle label, which is denoted as H_iThen separately counting Y_iThe number of times of occurrence of various labels in_iDividing the occurrence frequency of each type of label by K to obtain Y_iThe frequency of occurrence of each type of tag in (1) will be Y_iMiddle h_iThe frequency of occurrence of class labels is recorded ash_i∈{1,2…H_iH, the ith picture x_iThe entropy of the labeling result is recorded as E_i， E_iThe calculation is carried out by adopting the formula (3):

wherein, log is a logarithmic sign;

s6, determining the ith picture x of the final pushed picture data set final_iTag set Y of_iThe label with the most occurrence times is taken as the ith picture x of the final pushed picture data set final_iTransition label ofIf the labels with the most occurrence times exist in two or more types, one type of label is randomly selected as a transition labelAt this time, each picture in the final pushed picture data set final corresponds to one transition tag, and the crowd-sourced data set is constructed by adopting the pictures in the final pushed picture data set final and the transition tags of each picture

S7, adopting 3-fold cross validation to crowd-sourced data setProcessing, i.e. crowdsourcing, of data setsDividing the data into 3 parts at random, and respectively calling the three parts as first part data, second part data and third part data; training a classifier constructed by using a Resnet34 network by using the first data and the second data as training data, and predicting each picture in the third data by using the trained classifier to obtain prediction labels of the pictures; training a classifier constructed by using a Resnet34 network by using the second data and the third data as training data, and predicting each picture in the first data by using the trained classifier to obtain prediction labels of the pictures; training a classifier constructed by using a Resnet34 network by using the first data and the third data as training data, and predicting each picture in the second data by using the trained classifier to obtain prediction labels of the pictures; obtaining the prediction labels corresponding to all the pictures in the final pushed picture data set final, and adopting all the pictures in the final pushed picture data set final and the prediction labels corresponding to all the pictures to form a crowdsourcing data set Picture x of the ith representing the final pushed picture data set final_iThe predictive tag of (a);

s8, respectively comparing the transition label and the prediction label of each picture in the final pushed picture data set final, if the transition label and the prediction label of a certain picture are consistent, and the marking result entropy of the picture is less than 0.1, selecting the picture as a training picture, otherwise, not selecting the picture as data which does not meet the conditions, recording the total number of the training pictures obtained at the moment as m, and adopting the m training pictures obtained at the moment to form a training data setx'_zRepresents the z picture, y 'of the m training pictures'_zIs x'_zA corresponding transition label;

s9, training classifiers constructed by using a Resnet34 network by adopting a training data set D 'to obtain the trained classifiers, predicting unselected pictures in final pushed picture data sets final by adopting the classifiers to obtain prediction labels of the pictures, forming a group of training data by the unselected pictures and the prediction labels thereof, and merging the training data set D' with the training data formed by the unselected pictures and the prediction labels thereof to obtain a final training data set.