阅读说明：本技术 基于图像熵k均值聚类的3d-hevc快速cu分割方法 (3D-HEVC (high efficiency video coding) rapid CU (CU) segmentation method based on image entropy K-means clustering ) 是由 张桦 姚王泽 黄鸿飞 沈菲 戴国骏 吴以凡 于 2020-05-18 设计创作，主要内容包括：本发明公开了一种基于图像熵K均值聚类的3D-HEVC快速CU分割方法。快速CU分割预测编码决策方法在CU划分之前,首先将CU平均划分为四块,然后对CU整体以及各小块分别进行二维熵计算,并合成一个五维向量,然后对其进行K均值聚类(K均值聚类模型需要进行离线训练得到各尺寸的聚类中心),若判断为不需要划分,则停止对CU的划分,否则进入递归进一步划分。二维熵具有能突显图像分布特征以及混乱程度的能力,以此为基础进行K均值训练从而能够得到判断是否划分的模型。本发明直接对CU进行判断,减少了递归带来的时间消耗因此具有计算复杂度低、编码时间短和视频重建效果好的特点。(The invention discloses a 3D-HEVC rapid CU segmentation method based on image entropy K-means clustering. The quick CU partition prediction coding decision method comprises the steps of firstly averagely dividing a CU into four blocks before the CU is divided, then respectively carrying out two-dimensional entropy calculation on the whole CU and each small block, synthesizing a five-dimensional vector, then carrying out K-means clustering on the five-dimensional vector (a K-means clustering model needs to be trained offline to obtain clustering centers of all sizes), stopping dividing the CU if the division is judged not to be needed, and otherwise, carrying out recursive further division. The two-dimensional entropy has the capability of highlighting image distribution characteristics and the chaos degree, and K mean value training is carried out on the basis of the capability, so that a model for judging whether to divide or not can be obtained. The invention directly judges the CU, reduces the time consumption brought by recursion, and has the characteristics of low computational complexity, short coding time and good video reconstruction effect.)

1. The 3D-HEVC rapid CU segmentation method based on image entropy K-means clustering is characterized in that the method carries out image two-dimensional entropy calculation on one frame of image in a video and then carries out classification through a K-means model, and specifically comprises the following steps:

step 1: selecting CU matrixes with different sequences, different QP values and different sizes to perform K-means offline training to obtain an offline training model;

dividing an input CU matrix into four sub-blocks, and performing two-dimensional entropy calculation on each sub-block and the whole CU matrix to form a five-dimensional vector; inputting the five-dimensional vector into a training model to obtain an off-line training model, and finally obtaining clustering centers with different sizes and different QPs according to the off-line training model;

step 2: when formal prediction is carried out, a CU matrix is read in, two-dimensional entropy calculation is carried out after the CU matrix is divided into four sub-blocks, a five-dimensional vector is formed, and the five-dimensional vector is input into a K mean value offline training model for clustering;

and step 3: if the cluster belongs to the cluster 0, directly jumping to the step 5;

and 4, step 4: if the current CU matrix belongs to the cluster 1, the read-in CU matrix is subjected to recursion, the CU matrix is divided into 4 subblocks with the size of N/2 × N/2, each subblock is used as a new CU matrix, and the step 2 is skipped; the N is the size of the CU matrix;

and 5: the CU matrix does not need to be partitioned and the partitioning process ends.

2. The method of claim 1, wherein the two-dimensional entropy calculation in step 1 is as follows:

firstly, averagely dividing a CU matrix into four sub-blocks, and respectively calculating each sub-block and the whole CU matrix; for each sub-block and the whole CU matrix, traversing all pixel points of the sub-block and the whole CU matrix and calculating the mean value of the rest 8 pixel points around the pixel points; if the pixel is the four corner points of the CU, calculating the average value of the three surrounding pixel points; if the pixel is the four edge points of the CU, calculating the mean value of the five surrounding pixel points; meanwhile, counting the number of the peripheral pixel mean value j when the pixel value is i, so as to obtain a characteristic binary group f (i, j) which represents the number of the pixel value i and the peripheral pixel mean value j; calculating the proportion value of the characteristic binary f (i, j) through the characteristic binary f (i, j), wherein the formula is as follows:

p_(i,j)＝f(i,j)/N²

wherein N is the size length of the traversed object;

and finally, calculating the two-dimensional entropy of the CU matrix, wherein the formula is as follows:

the two-dimensional entropy of each sub-block and the whole CU matrix is calculated and combined into a set of five-dimensional vectors.

3. The method for 3D-HEVC fast CU segmentation based on image entropy K-means clustering according to claim 1 or 2, wherein the K-means algorithm in step 1 is as follows:

from a test sequence GT _ fly, street, kendo, balloon, share, hall, newsapper, dancer of a 3DHEVC, QPs obtained by using an original algorithm are respectively 25, 30,35 and 40, matrixes with CU sizes of respectively 64,32 and 16 are respectively ten thousand, and all the matrixes are marked whether to be divided or not;

then ten thousand matrixes with the same size as the QP are used as a unit for training; respectively averagely dividing each matrix into four small blocks, calculating the two-dimensional entropy of the small blocks and the whole matrix, and combining the small blocks and the whole matrix into a five-dimensional vector;

preprocessing all the obtained five-dimensional vectors, and removing data with deviation values larger than a preset threshold value;

and finally, performing K-means clustering training on the final data set obtained by the preprocessing screening to finally obtain 12 clustering models with different QPs and different sizes, wherein the cluster 0 of each model is set as the required partition.

4. The image entropy K-means clustering-based 3D-HEVC fast CU segmentation method as claimed in claim 3, wherein the classification of cluster 0 and cluster 1 for each model is implemented as follows:

inputting original training data for classification of a trained model, if the obtained result has more matrix proportion needing to be divided corresponding to cluster 0, then cluster 0 is needed to be divided, and cluster 1 is not needed to be divided, otherwise, if the matrix proportion needing to be divided corresponding to cluster 1 is more, then cluster 1 is needed to be divided, and cluster 0 is not needed to be divided; and finally, setting all the cluster points which represent that the division is needed as cluster 1, and setting all the cluster points which represent that the division is not needed as cluster 0 for unified use.

5. 3D-HEVC fast CU segmentation method based on image entropy K-means clustering as claimed in claim 1, 3 or 4, wherein the step 2 process is as follows:

when formal prediction is carried out, the QP value and the size of the input CU matrix are judged, then the QP value and the size of the input CU matrix are divided into four small blocks, and the two-dimensional entropy and the whole two-dimensional entropy are calculated respectively; and inputting the obtained five-dimensional vector into a K-means clustering model corresponding to the QP value and the size value to obtain a prediction result.

Technical Field

The invention relates to the field of video coding, in particular to a 3D-HEVC rapid CU segmentation method based on image entropy K-means clustering.

Background

In the last few years, with the increasing demand for 3D video services, the presentation, compression and transmission of 3D video has become a new challenging research topic. The multi-view plus depth Map (MVD) video format is one of many promising video representations. The 3D extension to high quality video coding (3D-HEVC), proposed by the joint collaborative group (JCT-3V) developed by 3D video coding extension, is an emerging video coding standard for compressing MVD data.

The conventional CU division is to perform recursive division to the minimum size and then gradually compare upwards to judge whether the CU division is performed, the time complexity is high, the CU matrix can be judged to be divided before the division, if the CU matrix does not need to be divided, the recursive judgment is not needed, otherwise the CU matrix enters the recursive division, and the depth map is smooth and more, so that many CUs do not need to be divided, and the CU division time can be greatly reduced.

The two-dimensional entropy of the image can reflect the characteristic quantity of the spatial characteristic of the gray distribution, can prominently reflect the gray information of the pixel position in the image and the comprehensive characteristic of the gray distribution in the pixel neighborhood on the premise of the information quantity contained in the image, and can also express the disorder degree of the information through the value of the entropy.

The k-means clustering algorithm (k-means clustering algorithm) is a clustering analysis algorithm for iterative solution, and can simply and efficiently classify each object and finally calculate a clustering center as a classification basis.

Disclosure of Invention

The invention aims to provide a 3D-HEVC rapid CU segmentation method based on image entropy K-means clustering in view of the effect of image two-dimensional entropy on image information extraction. The method has the characteristics of low calculation complexity, short coding time and good video reconstruction effect.

In order to achieve the purpose, the technical scheme of the invention comprises the following steps:

step 1: and selecting CU matrixes with different sequences, different QP values and different sizes to perform K-means offline training to obtain an offline training model.

And dividing the input CU matrix into four sub-blocks, and performing two-dimensional entropy calculation on each sub-block and the whole CU matrix to form a five-dimensional vector. And inputting the five-dimensional vector into the training model to obtain an off-line training model, and finally obtaining clustering centers with different sizes and different QPs according to the off-line training model.

Step 2: when formal prediction is carried out, a CU matrix is read in, two-dimensional entropy calculation is carried out after the CU matrix is divided into four sub-blocks, a five-dimensional vector is formed, and the five-dimensional vector is input into a K mean value offline training model for clustering;

and step 3: if the cluster belongs to the cluster 0, directly jumping to the step 5;

and 4, step 4: if the current CU matrix belongs to the cluster 1, the read-in CU matrix is subjected to recursion, the CU matrix is divided into 4 subblocks with the size of N/2 × N/2, each subblock is used as a new CU matrix, and the step 2 is skipped; the N is the size of the CU matrix;

and 5: the CU matrix does not need to be partitioned and the partitioning process ends.

The two-dimensional entropy calculation described in step 1 is as follows:

firstly, the CU matrix is averagely divided into four sub-blocks, and each sub-block and the whole CU matrix are respectively calculated. For each sub-block and the whole CU matrix, traversing all pixel points of the sub-block and the whole CU matrix and calculating the mean value of the rest 8 pixel points around the pixel points; if the pixel is the four corner points of the CU, calculating the average value of the three surrounding pixel points; if the pixel is the four edge points of the CU, calculating the mean value of the five surrounding pixel points; meanwhile, counting the number of the peripheral pixel mean value j when the pixel value is i, so as to obtain a characteristic binary group f (i, j) which represents the number of the pixel value i and the peripheral pixel mean value j; calculating the proportion value of the characteristic binary f (i, j) through the characteristic binary f (i, j), wherein the formula is as follows:

p_(i,j)＝f(i,j)/N²

wherein N is the size length of the traversed object;

and finally, calculating the two-dimensional entropy of the CU matrix, wherein the formula is as follows:

the two-dimensional entropy of each sub-block and the whole CU matrix is calculated and combined into a set of five-dimensional vectors.

The K-means algorithm in step 1 is as follows:

from the test sequences GT _ fly, street, kendo, balloon, share, hall, newsapper, dancer of the 3DHEVC, the QP obtained by the original algorithm (calculating rd cost and comparing the size to determine whether division is needed) is 25, 30,35,40 respectively, and ten thousand matrices (all the matrices are marked whether division is needed) with CU sizes of 64,32,16 respectively are obtained. That is, QP is 25, CU size 64 is ten thousand, QP is 25, CU size 32 is ten thousand, QP is 25, CU size 16 is ten thousand, QP is 30, CU size 64 is ten thousand, and so on.

Then training is performed by taking ten thousand matrices of the same size as the QP as a unit. And respectively averagely dividing each matrix into four small blocks, calculating the two-dimensional entropy of the small blocks and the whole matrix, and combining the small blocks and the whole two-dimensional entropy into a five-dimensional vector.

And preprocessing all the obtained five-dimensional vectors, and removing data with deviation values larger than a preset threshold value.

And finally, performing K-means clustering training on the final data set obtained by the preprocessing screening to finally obtain 12 clustering models with different QPs and different sizes, wherein the cluster 0 of each model is set as the required partition.

The classification of cluster 0 and cluster 1 for each model is implemented as follows:

inputting original training data for classification of a trained model, if the obtained result has more matrix proportion needing to be divided corresponding to cluster 0, then cluster 0 is needed to be divided, and cluster 1 is not needed to be divided, otherwise, if the matrix proportion needing to be divided corresponding to cluster 1 is more, then cluster 1 is needed to be divided, and cluster 0 is not needed to be divided; and finally, setting all the cluster points which represent that the division is needed as cluster 1, and setting all the cluster points which represent that the division is not needed as cluster 0 for unified use.

The process of step 2 is as follows:

when formal prediction is carried out, the QP value and the size of the input CU matrix are judged, then the QP value and the size of the input CU matrix are divided into four small blocks, and the two-dimensional entropy and the whole two-dimensional entropy of the input CU matrix are calculated respectively. And inputting the obtained five-dimensional vector into a K-means clustering model corresponding to the QP value and the size value to obtain a prediction result.

The invention has the following beneficial effects:

the method applies the two-dimensional entropy and the K-means algorithm to the three-dimensional video coding, and can effectively reduce the coding time while reducing the video code rate by utilizing the function of extracting image information of the two-dimensional entropy and the classification capability of the K-means algorithm, avoid a redundant mode of calculation and reduce the calculated amount.

Furthermore, the K-means algorithm training is carried out by calculating five-dimensional vectors synthesized by five two-dimensional entropies of the image and the subimages of the image, so that the judgment accuracy is improved to a greater extent. Meanwhile, when the actual recursion judgment is carried out, the result can be obtained immediately only by inputting the model, so that a large amount of calculation is avoided, and the encoding time is greatly prolonged.

Drawings

FIG. 1 is a flow chart of the K-means algorithm off-line training

FIG. 2 is a flow chart of a CU partition prediction method

FIG. 3 is a complete segmentation flow chart

Detailed Description

The invention is further illustrated by the following figures and examples.

As shown in fig. 1, the present invention specifically includes the following steps:

step 1, performing K-means algorithm off-line training, and taking ten thousand QPs of sequences GT _ fly, street, kendo, balloon, share, hall, newsapper and dancer as 25, 30,35 and 40, wherein the QPs are 64 multiplied by 64,32 multiplied by 32 and 16 multiplied by 16CU matrixes.

And 2, training by taking each ten thousand matrixes as a unit. The QP value for the set of matrices is determined and then the size of the set of matrices is determined.

Step 3, averagely dividing each matrix into four blocks, and carrying out two-dimensional entropy calculation on each block and the whole to form a five-dimensional vector, wherein the two-dimensional entropy calculation process is as follows:

firstly, the CU matrix is averagely divided into four sub-blocks, and each sub-block and the whole CU matrix are respectively calculated. For each sub-block and the whole CU matrix, traversing all pixel points of the sub-block and the whole CU matrix and calculating the mean value of the rest 8 pixel points around the pixel points; if the pixel is the four corner points of the CU, calculating the average value of the three surrounding pixel points; if the pixel is the four edge points of the CU, calculating the mean value of the five surrounding pixel points; meanwhile, counting the number of the peripheral pixel mean value j when the pixel value is i, so as to obtain a characteristic binary group f (i, j) which represents the number of the pixel value i and the peripheral pixel mean value j; calculating the proportion value of the characteristic binary f (i, j) through the characteristic binary f (i, j), wherein the formula is as follows:

p_(i,j)＝f(i,j)/N²

wherein N is the scale length of the CU matrix, if the CU is not divided, N is the scale length of the whole CU matrix, and if the CU is divided, N is the scale of the divided sub-CU matrix;

and finally, calculating the two-dimensional entropy of the CU matrix, wherein the formula is as follows:

the two-dimensional entropy of each sub-block and the whole CU matrix is calculated and combined into a set of five-dimensional vectors.

And 4, preprocessing all the obtained five-dimensional vectors, and removing data with deviation values larger than a preset threshold value. And then inputting the data into a K-means model for training, and setting the cluster 0 as judgment and non-division. And obtaining 12 cluster centers belonging to specific QP values and CU sizes.

As shown in fig. 2, the segmentation prediction method of the present invention specifically includes the following steps:

step (1), obtaining a CU with the size of N × N (where N is 64,32,16, N >8, and 64 for the first time) based on 3D-HEVC, dividing the CU into four matrixes on average, performing two-dimensional entropy calculation on the four matrixes respectively, performing two-dimensional entropy calculation on the whole, and combining the two matrixes into a five-dimensional vector.

Step 2: and carrying out K-means algorithm clustering judgment. If the cluster belongs to the cluster 0, skipping to the step 4, and if the cluster does not belong to the cluster 0, skipping to the step 3

And step 3: the CU is led to enter the recursion, the CU is divided into 4 sub-CUs with the size of N/2, and the step 1 is skipped to for each sub-CU;

and 4, step 4: the CU does not need to be partitioned and the partitioning process ends.

As shown in fig. 3, in the 3D-HEVC fast CU partition prediction method based on image two-dimensional entropy K-means clustering, a complete partition flowchart specifically includes the following steps: reading in a 64 × 64 CU, judging that the CU does not need to be divided through a K mean algorithm based on the two-dimensional entropy of the image, and ending the dividing process of the CU if the CU does not need to be divided; if the division is needed, dividing the image into 4 32 × 32 sub-CUs, then respectively reading the 32 × 32 CUs, judging that the division is not needed through a K-means algorithm based on the two-dimensional entropy of the image, and if the division is not needed, ending the division process of the CU; if the division is needed, dividing the image into 4 sub-CUs of 16 × 16, then respectively reading in the 16 × 16 CUs, judging that the image does not need to be divided through a K-means algorithm based on the two-dimensional entropy of the image, and if the image does not need to be divided, ending the division process of the CU; if the partition is needed, the partition is performed to 4 sub-CUs 8 by 8, the partition is performed to the minimum size, and the partition process is finished.