阅读说明：本技术 一种基于Mask-Rcnn识别鱼类生长检测方法 (Fish growth detection method based on Mask-Rcnn recognition ) 是由 牛广宇 朱星臣 钱思文 张继 张振伟 徐淑玲 于 2021-07-28 设计创作，主要内容包括：本发明涉及深度神经网络技术领域,涉及一种基于Mask-Rcnn识别鱼类生长检测方法,包括以下步骤：S1、通过鱼缸上设置摄像头采集鱼视频流数据；S2、对采集到的视频流进行分帧处理；S3、用标注软件1abelme在图像上标注鱼的标签；S4、利用Mask-Rcnn深度神经算法的迁移学习思想,用COC0公开数据集的权重,对Mask-Rcnn初始化,使用训练集进行训练；S5、使用测试集验证Mask-Rcnn模型的检测效果。本发明通过Mask-Rcnn深度神经算法,准确识别鱼类长度,计算出鱼类重量,从而准确计算出投喂饵料量。(The invention relates to the technical field of deep neural networks, and relates to a fish growth detection method based on Mask-Rcnn identification, which comprises the following steps: s1, collecting fish video stream data by arranging a camera on the fish tank; s2, framing the collected video stream; s3, labeling the fish label on the image by using labeling software 1 abelme; s4, initializing the Mask-Rcnn by using the migration learning idea of the Mask-Rcnn deep neural algorithm and the weight of a data set disclosed by COC0, and training by using a training set; s5, verifying the detection effect of the Mask-Rcnn model by using the test set. According to the invention, the length of the fish is accurately identified through a Mask-Rcnn deep neural algorithm, and the weight of the fish is calculated, so that the bait feeding amount is accurately calculated.)

1. A fish growth detection method based on Mask-Rcnn recognition is characterized by comprising the following steps:

s1, collecting fish video stream data by arranging a camera on the fish tank;

s2, performing framing processing on the collected video stream, and sampling at fixed intervals to obtain a fish image;

s3, labeling the labels of the fish on the images by using labeling software 1abelme to obtain json files comprising original images and corresponding labels, and randomly dividing a data set into a training set, a verification set and a test set in a ratio of 7:1: 2;

s4, initializing the Mask-Rcnn by using the migration learning of the Mask-Rcnn deep neural algorithm and the weight of a data set disclosed by COC0, training by using a training set, verifying by using the verification set in each training round in the training process, and entering the step S5 if the verification curve is converged; otherwise, returning to the step S3, adding training samples, labeling the labels of the fishes again, and repeating the training until the verification curve is converged, thereby establishing a Mask-Rcnn model;

wherein, the loss function L of Mask-Rcnn is Llcs + Lbox + Lmak; lcls, Lbox and Lmak respectively represent loss functions of classification, regression and semantic prediction, and the value of L is a loss value;

s5, verifying the detection effect of the Mask-Rcnn model by using the test set.

2. The Mask-Rcnn-based fish growth detection method according to claim 1, wherein the Mask-Rcnn deep neural algorithm of step S4 includes:

s41, constructing a Mask-Rcnn model for detecting the average fish length by a Mask-Rcnn deep neural algorithm, wherein the Mask-Rcnn model comprises the following components: the system comprises an instance segmentation module, a heterogeneous removal module, a skeletonization module, an overlap removal module and a length calculation module;

s42, recognizing semantic masks of the fish through a segmentation module; removing the semantic masks with prediction confidence lower than a threshold value from the semantic masks through a heterogeneous removing module; the skeletonization module extracts the predicted fish outline according to the prediction result of the semantic mask, and extracts a skeleton by using a Zhang-Suen skeleton algorithm, wherein each iteration is to corrode a target pixel meeting a specific condition so that the target becomes thinner and thinner; continuously iterating until no new pixel point is corroded in the current round of operation of the target after the last corrosion, and ending the iteration;

s43, judging whether two or more fishes are overlapped according to the semantic mask and the skeletonized prediction result, if so, keeping the length of a longer fish and cutting off the length of the longer fish;

and S44, calculating the pixel number occupied by the skeleton map according to the skeleton map, wherein the pixel number is the length of the fish.

3. The Mask-Rcnn-based fish growth assay of claim 1, wherein: the Mask-Rcnn deep neural algorithm of step S4 uses the ResNet-50 network and the feature pyramid network as feature extractors for extracting low-level features and high-level features of the image from the original picture, and through this process, it allows the features of each level to be combined with the high-level and low-level features; inputting the characteristics into a regional suggestion network, after generating a suggestion region, aligning and pooling the suggestion region, and identifying a semantic mask; the anchor points are used in the regional suggestion network, and the input with different sizes is adjusted to be the output with the same size, so that the feature graph with any size can be converted into the feature vector with a fixed size.

4. The Mask-Rcnn-based fish growth assay of claim 2, wherein: calculating the fish weight by calculating the length of the fish by using a Logistic model, wherein a relation equation of the length of the fish and the weight of the fish in the Logistic model is as follows:

M＝b×L×a

wherein, a is dimensionless quantity, and the dimension of the parameter b is M/La;

under the condition of recirculating aquaculture, the relation equation of the weight of the fish and the length of the fish is as follows: m is 0.0132 × L × 3.0455.

Technical Field

The invention relates to the technical field of deep neural networks, in particular to a fish growth detection method based on Mask-Rcnn recognition.

Background

In the aquaculture industry, the feed cost is always the largest project in all the investment, and the cost is greatly reduced by optimizing the feeding scheme. Additionally, the improvement of the profit margin of the feed can also reduce the toxin generated by the degradation of the residual bait and reduce the pressure of the environment and the aquaculture water body, so an excellent feeding scheme in the modern aquaculture process is important.

The prior art proposal is to use artificial feeding cultivation, and only depending on the personal experience of the feeding personnel and the observation material platform to feed the feed. In aquaculture, the cultured organisms are in the continuous growth process, so that the feeding scheme needs to be adjusted in real time, and the growth characteristics of fishes and shrimps need to be identified, so that an automatic feeding system is realized.

The disclosed invention patent ('an automatic fish bone phenotype information detection system and method without fish bone damage', 202010585963.8) provides an automatic fish bone phenotype information detection system and method without fish bone damage, and relates to the technical fields of image processing contour extraction technology and deep learning instance segmentation and target contour identification. The system comprises a data acquisition device, a signal transmission system, a terminal computer and a terminal display; the method comprises the steps that a data acquisition device acquires a fish side image and a fish front X-ray image of a fish to be detected, and the fish side image and the fish front X-ray image are transmitted to a terminal computer through a signal transmission system; the terminal computer extracts the profile of the side body of the fish and calculates the thickness information of the fish according to the received image of the side body of the fish by an image processing technology, then controls the data acquisition device to acquire the X-ray image of the front side of the fish, identifies each profile of the fish by an example segmentation technology according to the received X-ray image of the front side of the fish, calculates to obtain the fish bone phenotype information, and displays the detection result by the terminal display module.

In addition, the prior fish culture has the following problems:

1. feed waste: the feed is fed by observing a feeding table according to the experience of a feeder, so that nearly 30 percent of the feed is wasted, and the residual feed is degraded to generate toxin which has influence on the growth of the fishes and the culture water body.

2. The feeding scheme cannot be flexibly adjusted: in the continuous growth process of fishes, the feed feeding mode of the traditional artificial culture mode is fixed, and the personalized customized feeding scheme cannot be carried out.

Disclosure of Invention

The technical problems solved by the invention are as follows: through a Mask-Rcnn deep neural algorithm, the length of the fish is accurately identified, the weight of the fish is calculated, and therefore the bait feeding amount is accurately calculated.

The technical scheme adopted by the invention is as follows: a fish growth detection method based on Mask-Rcnn recognition comprises the following steps:

s1, collecting fish video stream data by arranging a camera on the fish tank;

s2, performing framing processing on the collected video stream, and sampling at fixed intervals to obtain a fish image;

s3, labeling a fish label on the image by using labeling software 1abelme, wherein the fish label is a labeled fish outline, obtaining a json file comprising an original image and a corresponding label, and randomly dividing a data set into a training set, a verification set and a test set in a ratio of 7:1: 2;

s4, initializing the Mask-Rcnn by using the migration learning idea of the Mask-Rcnn deep neural algorithm and disclosing the weight of a data set by using COC0, training by using a training set, wherein in the training process, each training turn is verified by using the verification set, and if the verification curves of the training turns and the loss rate are converged (namely the change of the slope of the curve is less than 5 degrees), the step S5 is carried out; otherwise, returning to the step S3, adding training samples (if the training samples are less, the convergence condition cannot be achieved), re-labeling, and repeating the training until the verification curve converges, thereby establishing a Mask-Rcnn model;

the loss function L of Mask-Rcnn is Llcs + Lbox + Lmak;

wherein, Lcls, Lbox and Lmask respectively represent loss functions of classification, regression and semantic prediction, and the value of L is a loss value;

s41, constructing a Mask-Rcnn neural network model for detecting the average length of the fishes through a Mask-Rcnn deep neural algorithm, wherein the Mask-Rcnn neural network model comprises an example segmentation module, a heterogeneous removal module, a skeletonization module, an overlap removal module and a length calculation module;

s42, wherein the example segmentation module is a semantic Mask which is constructed based on a Mask-Rcnn model and used for identifying the fish;

the heterogeneous removing module is used for removing the semantic mask with the prediction confidence coefficient lower than 80% threshold value in the semantic mask; since obstacles exist in the survival environment of the fish school, the obstacles can be mistakenly identified as the fish school, so that a prediction result with the confidence coefficient less than 80% is eliminated;

the skeletonization module extracts the predicted fish outline according to the prediction result of the semantic mask, and extracts a skeleton by using a Zhang-Suen skeleton algorithm, wherein each iteration of the Zhang-Suen skeleton algorithm is to corrode a target pixel meeting a specific condition, so that the target becomes thinner and thinner; continuously iterating until no new pixel point is corroded in the current round of operation of the target after the last corrosion, and ending the iteration;

(a)2≤B(P1)≤6

the sum of the number of target pixels (1 in binary) around the center pixel P1 is between 2 and 6;

(b)A(P1)＝1

in 8 adjacent pixels, according to the clockwise direction, two adjacent pixels have the frequency of 0- > 1;

(c)P2×P4×P6＝0

(d)P4×P6×P8＝0

s43, the overlap removing module judges whether two or more fishes overlap according to the semantic mask and the skeletonized prediction result, if so, the length of the longer fish is reserved, and the length of the longer fish is cut off;

and S44, a length calculating module is used for calculating the pixel number occupied by the skeleton map according to the skeleton map, wherein the pixel number is the length of the fish.

S5, verifying the detection effect of the Mask-Rcnn model by using the test set, and in order to endow the model with multi-angle universality and reduce the loss value of the model, the data set of the model comprises water surface photos shot at different positions and at the same visual angle.

Further, the Mask-Rcnn model adopts a ResNet-50 network and a feature pyramid network as feature extractors for extracting low-level features and high-level features of an image from an original picture, and through the process, the Mask-Rcnn model allows the features of each level to be combined with the high-level and low-level features; inputting the characteristics into a regional suggestion network, after generating a suggestion region, aligning and pooling the suggestion region, and identifying a semantic mask; the anchor point is used in the area proposal network, and the input with different sizes can be adjusted into the output with the same size, so that the feature map with any size can be converted into the feature vector with fixed size.

Further, the weight of the fish is calculated according to a Logistic model, and a relation equation of the length and the weight of the fish is as follows:

M＝b×L×a

wherein, a is dimensionless quantity, then the dimension of the parameter b is M/(L multiplied by a);

under the condition of recirculating aquaculture, the fish body length L grows linearly, and the body length growth equation is as follows: l is 0.0961 × t +9.1442, and the relationship between body length and age in days (t) is linear; the growth of body mass (m) is exponential, and the growth equation of body mass is: m-0.0651 t 1.6147; the body mass and the age in days are in an exponential relationship, and the relational equation of the body weight and the length is as follows: m is 0.0132 × L × 3.0455;

the amount of the feed to be fed can be calculated according to the weight of the fish, and the gram of the feed is equal to M multiplied by 30 percent.

The invention has the beneficial effects that:

1. the length of the fish is accurately calculated by utilizing image recognition and a Mask-Rcnn algorithm, and the weight is calculated according to the length, so that the feed feeding amount is accurately calculated.

Drawings

FIG. 1 is a flow chart of the fish growth detection method based on Mask-Rcnn identification according to the present invention;

FIG. 2 is a drawing of a fish school framing sample according to the present invention;

FIG. 3 is a graphical representation of a fish school silhouette label according to the present invention;

FIG. 4 is a graph of the relationship between the Mask-Rcnn network training turns and the loss value;

FIG. 5 is a graph of the effect of the present invention after image segmentation;

FIG. 6 is a diagram illustrating the effect of the skeletonization of the image according to the present invention.

Detailed Description

The invention will be further described with reference to the accompanying drawings and examples, which are simplified schematic drawings and illustrate only the basic structure of the invention in a schematic manner, and therefore only show the structures relevant to the invention.

A fish growth detection method based on Mask-Rcnn recognition is disclosed, as shown in figure 1, S1, collecting fish video stream data by arranging a camera on a fish tank;

s2, performing framing processing on the collected video stream, and sampling at fixed intervals to obtain a fish image;

FIG. 2 shows four images of fish taken at 5 minute intervals;

s3, labeling the labels of the fish on the images by using labeling software 1abelme to obtain json files comprising original images and corresponding labels, and randomly dividing a data set into a training set, a verification set and a test set in a ratio of 7:1: 2;

FIG. 3 is a diagram illustrating the labeling effect of the software 1abelme on the image of the fish;

s4, initializing the Mask-Rcnn by using the migration learning idea of the Mask-Rcnn deep neural algorithm and the weight of a data set disclosed by COC0, training by using a training set, verifying by using a verification set in each training round in the training process, and entering step S5 if a verification curve is converged; otherwise, returning to the step S3, expanding the original database, re-labeling, and repeating the training, thereby establishing a Mask-Rcnn model;

as shown in fig. 4, loss rate of 300 fish images after 30 rounds of training is less than 5%;

s5, verifying the detection effect of the Mask-Rcnn model by using the test set.

Further, the Mask-Rcnn deep neural algorithm of step S4 includes: constructing a Mask-Rcnn neural network model for detecting the average length of the fishes through a Mask-Rcnn deep neural algorithm, wherein the Mask-Rcnn neural network model comprises an example segmentation module, a heterogeneous removal module, a skeletonization module, an overlap removal module and a length calculation module;

s41, constructing a Mask-Rcnn neural network model for detecting the average length of the fishes through a Mask-Rcnn deep neural algorithm, wherein the Mask-Rcnn neural network model comprises an example segmentation module, a heterogeneous removal module, a skeletonization module, an overlap removal module and a length calculation module;

s42, wherein the example segmentation module is a semantic Mask which is constructed based on a Mask-Rcnn model and used for identifying the fish;

after the fish is identified and segmented, a minimum circumscribed rectangular frame is drawn at the position of the fish, the coordinate is the center of the rectangular frame, and the number on the frame is the fish identification confidence coefficient.

S43, the overlap removing module judges whether two or more fishes overlap according to the semantic mask and the skeletonized prediction result, if so, the length of the longer fish is reserved, and the length of the longer fish is cut off;

and S44, a length calculating module is used for calculating the pixel number occupied by the skeleton map according to the skeleton map, wherein the pixel number is the length of the fish.

As shown in fig. 6, a Zhang-Suen skeleton algorithm is used to extract a skeleton, and after 30 iterations, a target pixel meeting a specific condition is corroded to make the target become thinner and thinner, and the final fish shape is a slender line, wherein the pixel lengths of the slender lines of fourteen fish in the diagram are respectively: 55.44, 69.52, 39.6, 61.6, 44, 57.2, 69.52, 45.76, 51.05, 46.64, 36.08, 54.56 and 40.48.

The invention has the beneficial effects that: the length of the fish is accurately calculated by utilizing image recognition and a Mask-Rcnn algorithm, and the weight is calculated according to the length, so that the feed feeding amount is accurately calculated.

In light of the foregoing description of the preferred embodiment of the present invention, many modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the invention. The technical scope of the present invention is not limited to the content of the specification, and must be determined according to the scope of the claims.