阅读说明：本技术 图像隐私识别方法、装置、计算机设备和存储介质 (Image privacy identification method and device, computer equipment and storage medium ) 是由 陈晓亮 王新辉 徐延林 于 2019-12-06 设计创作，主要内容包括：本申请涉及一种图像隐私识别方法、装置、计算机设备和存储介质。方法包括：获取待识别图像；提取待识别图像的至少八个不同方向的残差矩阵；根据待识别图像的至少八个不同方向的残差矩阵,确定待识别图像的局部残差模式直方图；获取待识别图像的局部残差模式直方图中的特征向量的Fisher分数,根据Fisher分数对待识别图像的局部残差模式直方图中的特征向量进行筛选,得到待识别图像的目标特征向量；将目标特征向量输入已训练的图像分类模型中,根据图像分类模型的分类结果识别待识别图像是否为含隐私图像。采用本方法可以高效、准确地判断图像中是否隐藏有隐私信息,且判断过程复杂度低,鲁棒性佳。(The application relates to an image privacy identification method and device, a computer device and a storage medium. The method comprises the following steps: acquiring an image to be identified; extracting residual error matrixes of at least eight different directions of the image to be identified; determining a local residual error mode histogram of the image to be identified according to at least eight residual error matrixes in different directions of the image to be identified; obtaining Fisher scores of feature vectors in a local residual error mode histogram of an image to be identified, and screening the feature vectors in the local residual error mode histogram of the image to be identified according to the Fisher scores to obtain a target feature vector of the image to be identified; and inputting the target characteristic vector into the trained image classification model, and identifying whether the image to be identified is an image containing privacy according to the classification result of the image classification model. By adopting the method, whether the private information is hidden in the image can be efficiently and accurately judged, the judgment process is low in complexity, and the robustness is good.)

1. An image privacy recognition method, characterized in that the method comprises:

acquiring an image to be identified;

extracting residual error matrixes of the image to be recognized in at least eight different directions;

determining a local residual error mode histogram of the image to be identified according to at least eight residual error matrixes in different directions of the image to be identified;

obtaining Fisher scores of feature vectors in the local residual error mode histogram of the image to be recognized, and screening the feature vectors in the local residual error mode histogram of the image to be recognized according to the Fisher scores to obtain target feature vectors of the image to be recognized;

inputting the target feature vector into a trained image classification model, and identifying whether the image to be identified is an image containing privacy according to a classification result of the image classification model;

wherein the content of the first and second substances,

the step of extracting residual error matrixes of the image to be recognized in at least eight different directions comprises the following steps:

extracting residual error matrixes of the image to be recognized in eight directions; the eight directions of the image to be recognized comprise two horizontal mutual-reaction directions, two vertical mutual-reaction directions, two diagonal mutual-reaction directions and two anti-diagonal mutual-reaction directions;

the step of determining the local residual error mode histogram of the image to be identified according to the residual error matrixes in at least eight different directions of the image to be identified comprises the following steps:

determining a straight local variance histogram of the image to be identified according to residual matrixes in two horizontal reciprocal directions and two vertical reciprocal directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction;

determining an oblique local variance histogram of the image to be recognized according to residual matrixes of two diagonal reciprocal directions and two opposite diagonal reciprocal directions; the oblique local variance histogram comprises two diagonal reciprocal directions and a variance histogram of two opposite diagonal reciprocal directions;

and fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified.

2. The method of claim 1, wherein the method is performed according to the formula:

extracting residual error matrixes of the image to be recognized in eight directions;

wherein the content of the first and second substances,

3. The method according to claim 2, wherein the step of determining a flat local variance histogram of the image to be recognized from residual matrices of two horizontal reciprocal directions and two vertical reciprocal directions comprises:

according to the formula:

wherein the content of the first and second substances,

determining a straight local variance histogram of the image to be identified;

wherein the content of the first and second substances,representing a flat local variance histogramTo (1) a

4. the method according to claim 3, wherein the step of determining the diagonal local variance histogram of the image to be recognized according to the residual matrix of two diagonal reciprocal directions and two anti-diagonal reciprocal directions comprises:

according to the formula:

determining a diagonal local variance histogram of the image to be identified;

wherein the content of the first and second substances,

。

5. the method according to claim 4, wherein the step of fusing the straight local variance histogram and the diagonal local variance histogram to obtain a local residual mode histogram of the image to be recognized comprises:

according to the formula:

fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified;

wherein the content of the first and second substances,a local residual mode histogram representing the image to be identified.

6. The method according to claim 5, wherein the step of filtering the feature vectors in the local residual error pattern histogram of the image to be recognized according to the Fisher score to obtain the target feature vector of the image to be recognized comprises:

according to the Fisher score, sorting each dimensional feature in the local residual error mode histogram of the image to be recognized from high to low;

and selecting features corresponding to Fisher scores arranged in a preset dimension to form a target feature vector of the image to be recognized.

7. The method according to any one of claims 1 to 6, wherein the image classification model constructing step comprises:

constructing an image training set;

acquiring images to be trained from the image training set;

extracting residual error matrixes of the image to be trained in at least eight different directions;

determining a local residual error mode histogram of the image to be trained according to at least eight residual error matrixes in different directions of the image to be trained;

calculating Fisher scores of the feature vectors in the local residual error mode histogram of the image to be trained to serve as the Fisher scores of the feature vectors in the local residual error mode histogram of the image to be recognized, and screening the feature vectors in the local residual error mode histogram of the image to be trained according to the calculation result to obtain sample feature vectors of the image to be trained;

and inputting the sample feature vector into a support vector machine for training so as to construct the image classification model.

8. The method of claim 7, wherein the step of calculating the Fisher scores of the feature vectors in the local residual pattern histogram of the image to be trained comprises:

according to the formula:

calculating Fisher scores of feature vectors in a local residual error mode histogram of the image to be trained;

wherein the content of the first and second substances,

9. The method of claim 7, further comprising:

extracting residual error matrixes of the image to be trained in eight directions; the eight directions of the image to be trained comprise two horizontal reciprocal directions, two vertical reciprocal directions, two diagonal reciprocal directions and two opposite diagonal reciprocal directions;

determining a straight local variance histogram of the image to be trained according to residual matrixes in two horizontal reciprocal directions and two vertical reciprocal directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction;

determining an oblique local variance histogram of the image to be trained according to residual matrixes of two diagonal reciprocal directions and two anti-diagonal reciprocal directions; the oblique local variance histogram comprises two diagonal reciprocal directions and a variance histogram of two opposite diagonal reciprocal directions;

and fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be trained.

10. The method of claim 7, wherein the support vector machine is a soft-space support vector machine with a gaussian kernel.

11. An image privacy recognition apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring an image to be identified;

the extraction module is used for extracting residual error matrixes in at least eight different directions of the image to be identified; the residual error matrix of the image to be identified in the eight directions is further extracted; the eight directions of the image to be recognized comprise two horizontal mutual-reaction directions, two vertical mutual-reaction directions, two diagonal mutual-reaction directions and two anti-diagonal mutual-reaction directions;

the determining module is used for determining a local residual error mode histogram of the image to be identified according to at least eight residual error matrixes in different directions of the image to be identified; the image recognition device is further used for determining a straight local variance histogram of the image to be recognized according to residual matrixes in two horizontal mutual reverse directions and two vertical mutual reverse directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction; determining an oblique local variance histogram of the image to be recognized according to residual matrixes of two diagonal reciprocal directions and two opposite diagonal reciprocal directions; the oblique local variance histogram comprises two diagonal reciprocal directions and a variance histogram of two opposite diagonal reciprocal directions; fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified;

the screening module is used for obtaining Fisher scores of the feature vectors in the local residual error mode histogram of the image to be identified, and screening the feature vectors in the local residual error mode histogram of the image to be identified according to the Fisher scores to obtain target feature vectors of the image to be identified;

and the classification module is used for inputting the target characteristic vector into a trained image classification model and identifying whether the image to be identified is an image containing privacy or not according to the classification result of the image classification model.

12. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 10.

Technical Field

The present application relates to the field of information security technologies, and in particular, to an image privacy identification method and apparatus, a computer device, and a storage medium.

Background

In recent years, with the development of computer technology and communication technology, the production and the transmission of information are increasingly convenient, the personal privacy is maintained, the personal privacy protection level is improved, and the method is of great significance to the society. There are some lawless persons who can embed the private information of the user in a specific carrier (such as audio, video, image, text, etc.) without being perceived through the privacy hiding technology, so as to achieve the purpose of stealing the privacy of individuals, companies or governments, and seriously jeopardize the information security.

The digital image privacy hiding technology can be understood as a technology for hiding privacy data by taking a digital image as a carrier. The technology can disguise the private data into common digital images, and embeds the private data by modifying the original carrier image as little as possible, so that the modified image and the original image cannot be distinguished. Anti-privacy hiding technology (privacy recognition technology) for digital images is just a technology that is antagonistic to digital image privacy hiding technology.

Binary images, as one of digital images, are often used as carriers for privacy hiding by lawbreakers. At present, most of anti-privacy hiding technologies for binary images extract features related to texture characteristics, morphological characteristics, statistical characteristics and the like from binary images, and then classify the features of the binary images by using a classifier to judge whether information related to privacy is hidden or not; however, the training complexity of the classifier is often high, the robustness is poor, and the accuracy of the classification result is low.

Disclosure of Invention

In view of the above, it is necessary to provide an image privacy recognition method, apparatus, computer device and storage medium for solving the above technical problems.

In one aspect, an embodiment of the present invention provides an image privacy identification method, where the method includes:

acquiring an image to be identified;

extracting residual error matrixes of the image to be recognized in at least eight different directions;

determining a local residual error mode histogram of the image to be identified according to at least eight residual error matrixes in different directions of the image to be identified to obtain Fisher scores of feature vectors in the local residual error mode histogram of the image to be identified, and screening the feature vectors in the local residual error mode histogram of the image to be identified according to the Fisher scores to obtain a target feature vector of the image to be identified;

and inputting the target characteristic vector into a trained image classification model, and identifying whether the image to be identified is an image containing privacy according to a classification result of the image classification model.

In another aspect, an embodiment of the present invention provides an image privacy identification apparatus, where the apparatus includes:

the acquisition module is used for acquiring an image to be identified;

the extraction module is used for extracting residual error matrixes in at least eight different directions of the image to be identified;

the determining module is used for determining a local residual error mode histogram of the image to be identified according to at least eight residual error matrixes in different directions of the image to be identified;

the screening module is used for obtaining Fisher scores of the feature vectors in the local residual error mode histogram of the image to be identified, and screening the feature vectors in the local residual error mode histogram of the image to be identified according to the Fisher scores to obtain target feature vectors of the image to be identified;

and the classification module is used for inputting the target characteristic vector into a trained image classification model and identifying whether the image to be identified is an image containing privacy or not according to the classification result of the image classification model.

In still another aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of an image privacy identification method when executing the computer program.

In still another aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of an image privacy recognition method.

One of the above technical solutions has the following advantages or beneficial effects: extracting residual images in at least eight different directions of the binary image, and enhancing edge characteristics in different directions in the image by using the residual images; counting a local residual error mode histogram, and analyzing the change of the privacy information hiding process on the texture in the residual error; furthermore, by fusing each local residual error mode histogram, the direction invariance of the characteristics is enhanced, and the robustness of the model is enhanced; features are screened according to a Fisher criterion, a target feature vector is constructed, and prediction complexity can be effectively reduced; finally, the target characteristic vector is input into the trained image classification model, so that whether the privacy information is hidden in the image can be judged efficiently and accurately; from the side, the image classification model also has lower training complexity and better robustness, and the classification result has higher accuracy.

Drawings

FIG. 1 is a schematic flow chart diagram of a method for image privacy recognition in one embodiment;

FIG. 2 is a schematic flow chart diagram of a method for image privacy recognition in another embodiment;

FIG. 3 is a graph of the classification effect of an image classification model in one embodiment;

FIG. 4 is a schematic block diagram of an image privacy recognition apparatus in one embodiment;

FIG. 5 is a schematic block diagram of a computer apparatus in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

In one embodiment, as shown in fig. 1, there is provided an image privacy recognition method, including the steps of:

and S202, acquiring an image to be identified.

The image to be recognized can be a binary image or other types of images, and for other types of images, the image to be recognized can be converted into the binary image before being subjected to privacy recognition so as to facilitate operation. The image to be recognized can be acquired from the positions of a memory, a cloud end and the like, and the format, the display content and the like of the image to be recognized are not specifically limited, so that the actual situation is the standard.

And S204, extracting residual error matrixes of the image to be recognized in at least eight different directions.

By extracting the residual error matrixes in at least eight different directions of the image to be recognized, the edge characteristics in different directions in the image can be enhanced, so that the change of the hidden privacy information process on the texture in the residual error can be more accurately acquired and analyzed. In the embodiment of the invention, the specific directions of at least eight different directions are not limited, and can be selected and set according to actual conditions.

And S206, determining a local residual error mode histogram of the image to be recognized according to the residual error matrixes of the image to be recognized in at least eight different directions.

Specifically, a local variance histogram corresponding to each direction of the image to be recognized is determined according to at least eight residual matrixes in different directions, and then the local variance histograms are fused to obtain a local residual mode histogram of the image to be recognized. Of course, the local variance histograms corresponding to at least two directions of the image to be recognized can be determined according to actual needs, and then the local variance histograms are fused.

S208, obtaining Fisher scores of the feature vectors in the local residual error mode histogram of the image to be recognized, and screening the feature vectors in the local residual error mode histogram of the image to be recognized according to the Fisher scores to obtain target feature vectors of the image to be recognized.

The Fisher score of the feature vector in the local residual error mode histogram of the image to be recognized can be calculated in advance and can be directly used in the step; in addition, the Fisher score calculation method and principle can refer to the existing Fisher criterion, and only by aiming at the characteristic vector in the local residual error mode histogram of the image to be recognized, the basic algorithm is combined with the practical application scene of the application, so that the Fisher score of the characteristic vector in the local residual error mode histogram of the image to be recognized can be obtained; features are screened through a Fisher criterion, a target feature vector is constructed, and complexity in prediction can be effectively reduced.

And S210, inputting the target feature vector into the trained image classification model, and identifying whether the image to be identified is an image containing privacy according to the classification result of the image classification model.

In the training process of the image classification model, the Fisher scores of the training set images (including the original images and the corresponding privacy-containing images) also need to be calculated, and the Fisher scores are used for sequencing the features of the training set images in the training process and can also be used for sequencing the features of the images to be recognized.

In the above embodiments of the present invention, the execution subject may be an image processor or other server equipment, and the server equipment may be a single server or a server cluster formed by a plurality of servers. Of course, the selection and the modification can be made according to the actual situation.

In the image privacy identification method of the embodiment, the residual images in at least eight different directions of the binary image are extracted, and the edge characteristics in different directions in the image are enhanced by using the residual images; counting a local residual error mode histogram, and analyzing the change of the privacy information hiding process on the texture in the residual error; furthermore, by fusing each local residual error mode histogram, the direction invariance of the characteristics is enhanced, and the robustness of the model is enhanced; features are screened according to a Fisher criterion, a target feature vector is constructed, and prediction complexity can be effectively reduced; finally, the target characteristic vector is input into the trained image classification model, so that whether the privacy information is hidden in the image can be judged efficiently and accurately; from the side, the image classification model also has lower training complexity and better robustness, and the classification result has higher accuracy.

In some embodiments, the image privacy recognition method further includes: extracting residual error matrixes of the image to be recognized in eight directions; the eight directions of the image to be recognized comprise two horizontal mutual-reaction directions, two vertical mutual-reaction directions, two diagonal mutual-reaction directions and two anti-diagonal mutual-reaction directions; determining a straight local variance histogram of the image to be identified according to residual matrixes in two horizontal reciprocal directions and two vertical reciprocal directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction; determining an oblique local variance histogram of the image to be recognized according to the residual matrixes of the two diagonal reciprocal directions and the two opposite diagonal reciprocal directions; the oblique local variance histogram comprises two variance histograms with diagonal reciprocal directions and two anti-diagonal reciprocal directions; and fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified. According to the calculation method, the corresponding local variance histograms are obtained according to the residual error matrixes in different directions but with the same rule, and then all the local variance histograms are fused to obtain the local residual error mode histogram of the image to be identified, so that the calculation complexity can be effectively reduced, and the accuracy and the stability of the calculation result are ensured.

In some embodiments, S204 may specifically include: according to the formula:

；

；

；

；

extracting residual error matrixes of the image to be recognized in eight directions;

wherein the content of the first and second substances,

representing the first in the image to be recognized

Go to the first

Pixels of a column;

and

second of the residual matrices representing two horizontal reciprocal directions

Go to the firstThe elements of the column are,

，

；and

second of the residual matrices representing two perpendicular reciprocal directionsGo to the first

The elements of the column are,

，

；

andthe first of the residual matrixes representing two diagonal reciprocal directions respectively

Go to the firstThe elements of the column are,

，

；

and

second of the residual matrix representing the two opposite diagonal mutually opposite directions

Go to the firstThe elements of the column are,

，

；

represents the width of the image to be recognized,

indicating a high of the image to be recognized.

In some embodiments, S206 may specifically include: according to the formula:

；

wherein the content of the first and second substances,

；

determining a straight local variance histogram of the image to be identified;

wherein the content of the first and second substances,

expressing a flat local variance histogram

The number of the elements is one,

；

representation matrix

First, theGo to the first

Elements of a column; matrix array

Respectively as follows:

；

；

；

。

in some embodiments, S206 may specifically include: according to the formula:

；

determining a diagonal local variance histogram of the image to be identified;

wherein the content of the first and second substances,

second to express diagonal local variance histogram

The number of the elements is one,

(ii) a Matrix array

Respectively as follows:

in some embodiments, S206 may specifically include: according to the formula:

fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified;

wherein the content of the first and second substances,

a local residual mode histogram representing the image to be identified.

In some embodiments, S208 may specifically further include: according to the Fisher score, sorting each dimensional feature in a local residual error mode histogram of the image to be recognized from high to low; and selecting features corresponding to the Fisher scores and arranged in the front set dimension to form a target feature vector of the image to be recognized.

In some embodiments, the constructing step of the image classification model in S210 includes: constructing an image training set; acquiring images to be trained from the image training set; extracting residual error matrixes of at least eight different directions of the image to be trained; determining a local residual error mode histogram of the image to be trained according to at least eight residual error matrixes in different directions of the image to be trained; calculating Fisher scores of feature vectors in a local residual error mode histogram of the image to be trained, and screening the feature vectors in the local residual error mode histogram of the image to be trained according to the calculation result to obtain a sample feature vector of the image to be trained; and inputting the sample feature vector into a support vector machine for training to construct an image classification model.

In some embodiments, the step of calculating the Fisher score of the feature vector in the local residual mode histogram of the image to be trained comprises:

according to the formula:

wherein the content of the first and second substances,

calculating Fisher scores of feature vectors in a local residual error mode histogram of the image to be trained;

wherein the content of the first and second substances,a second in a local residual pattern histogram representing the image to be trained

Fisher score of dimensional features;

in the image training set representing the image classification model

Local residual error mode histogram of original image;

in the image training set representing the image classification modelLocal residual error mode histogram containing privacy image;

andare respectively

And

to (1) a

An element;

representing the number of original images or privacy-containing images in an image training set of the image classification model;

representing a first intermediate vector;

representing a second intermediate vector;representing a third intermediate vector;

representing a fourth intermediate vector.

In some embodiments, the constructing of the image classification model further comprises: extracting residual error matrixes of the images to be trained in eight directions; the eight directions of the image to be trained comprise two horizontal reciprocal directions, two vertical reciprocal directions, two diagonal reciprocal directions and two opposite diagonal reciprocal directions; determining a straight local variance histogram of the image to be trained according to residual matrixes of two horizontal reciprocal directions and two vertical reciprocal directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction; determining an oblique local variance histogram of the image to be trained according to residual matrixes of two diagonal reciprocal directions and two anti-diagonal reciprocal directions; the oblique local variance histogram comprises two variance histograms with diagonal reciprocal directions and two anti-diagonal reciprocal directions; and fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be trained. In some embodiments, the original model of the image classification model may be a support vector machine, or may be another training model; the support vector machine can be a soft interval support vector machine with a Gaussian kernel, and other types of vector machines can be selected according to actual conditions, which are not limited uniquely.

In order to more clearly explain the privacy recognition method and the training process of the image classification model proposed in the present application, each step will be detailed one by one in conjunction with a specific embodiment.

As shown in fig. 2, the privacy identification method may also be referred to as a binary image anti-privacy hiding method based on a local residual pattern, and as can be seen from the figure, a training process of an image classification model is relatively similar to a recognition process of an image to be identified, and the training process of the image classification model will be described in detail below. Specifically, the training process of the image classification model may include the following steps:

1. and dividing a training set and a testing set from the binary image data set. First, a specific number (assumed to be) is selected from the binary image data set by using a hierarchical sampling method

) Selecting the corresponding image containing the privacy information as a training set; then, a specific number (assumed to be) of the remaining images of the binary image data set is selected by hierarchical sampling

) And selecting the corresponding image containing the privacy information as a test set.

Specifically, for example, the total number of binary image databases may be 24000 (12000 images of the original image and the image containing the private information), and

and

4000 and 200 sheets were taken, respectively.

2. Extracting a residual matrix of the image: for each image in the image dataset, residual matrices for 8 different directions of the image are extracted.

Hypothetical imageRespectively has a width and a height of

And

(in this example, the height and width of the image are, respectively

And），

to representTo middle

Go to the first

The specific processing procedure for the pixels (with values of 0 or 1) in the column is as follows:

2.1 calculating residual error matrix in horizontal direction

And

：

wherein

And

are respectively residual error matrixesAnd

to (1) a

Go to the first

The elements of the column are,

，

。

2.2 computing the residual matrix in the vertical direction

WhereinAnd

are respectively residual error matrixes

To (1) a

Go to the firstThe elements of the column are,

，

。

2.3 computing residual error matrix in diagonal direction

Wherein

And

are respectively residual error matrixes

To (1) a

Go to the firstThe elements of the column are,，

。

2.4 computing the residual error matrix of the anti-diagonal direction

Wherein

Andare respectively residual error matrixes

To (1) a

Go to the first

The elements of the column are,

，。

3. extracting fused local residual error mode histogram features: in this step, for each image in the image data set, the fused local residual mode histogram features of the image are extracted using the 8 residual matrixes in different directions obtained in step 2.

Definition of

The specific processing procedure of the method for extracting the fused local residual error mode histogram feature is as follows:

3.1 computing local variance histogram in horizontal and vertical directions

Wherein the line vector

（

) To (1) a

An element

Comprises the following steps:

wherein

Representation matrix

First, the

Go to the firstElements of a column

3.2 computing local variance histogram of diagonal and anti-diagonal directions

Wherein the line vector

（

) To (1) a

An element

Comprises the following steps:

wherein

Representation matrix

First, the

Go to the first

Elements of a column

3.3 computational fusionLocal residual mode histogram of

4. Constructing sample feature vectors of a training set and a verification set: and (3) performing feature screening on the fused local residual error mode histogram features of the image obtained in the step (3) aiming at each image in the training set and the verification set, and constructing a sample feature vector of the image.

The specific treatment process is as follows:

4.1, calculating Fisher scores corresponding to each dimension feature (each group) in the fused local residual error mode histogram. In the residual images of the binary image database after the training set and the test set are removed, a plurality of original images (assumed to be the original images) are selected by utilizing a hierarchical sampling method

Sheets) and select the corresponding image containing the private information, i.e., 2 in total

An image. Suppose that

The fused local residual error mode histogram of the original image obtained by the steps of S2 and S3 is

（

) Of 1 at

The fused local residual error mode histogram of the private information-containing image calculated in the steps S2 and S3 is

Then it is first

Fisher score of dimensional features

Is composed of

Wherein

And

are respectively

And

to (1) a

And (4) each element. In the present example, the first and second substrates were,and taking 4000.

4.2 for trainingSorting each group (each dimension characteristic) of the fused local residual error mode histogram of each image of the training set and the test set according to the Fisher scores from high to low, and selecting the group with the highest Fisher score in a specific number (assuming that before selection, the groups are selected)

Dimension), a sample feature vector is constructed. In the present example, the first and second substrates were,

and 500 is taken.

5. Training a classifier: and (4) training a support vector machine by using the sample feature vector of the training set image obtained in the step (4). For each image in the training set, if the image is an original image, marking the sample feature vector of the image as + 1; if the image is an image containing private information, its sample feature vector is labeled-1. And the original image and the sample feature vector of the image containing the privacy information form a feature vector set for training a support vector machine. The support vector machine in the experiment is a soft interval support vector machine using a Gaussian kernel, and related parameters are set as follows:

6. and (3) classification prediction: and (5) classifying the sample characteristic vectors of the images of the test set by using the trained support vector machine obtained in the step (5), and predicting whether privacy related information is hidden or not. Wherein, the prediction result is +1, which indicates that the image is judged to be the original image, and the prediction result is-1, which indicates that the image is judged to be the image containing the privacy information. The prediction result in this example is shown in fig. 3, where the first 200 points represent the predicted values for the original image and the last 200 points represent the predicted values for the image containing the private information. As is apparent from fig. 3, most of the images can be predicted accurately, and the classification detection result is ideal.

Compared with the prior art, the technical scheme of the invention has the beneficial effects that: according to the method, residual images in 8 different directions of an image are extracted, edge characteristics in the image in different directions are enhanced by using the residual images, then local residual mode histograms are counted, changes of textures in residual in a privacy information hiding process are analyzed, the local residual mode histograms are further fused, direction invariance of characteristics is enhanced, robustness of a model is enhanced, characteristics are screened according to a Fisher criterion, a target characteristic vector is constructed, complexity in training and prediction is reduced, and finally a classifier is trained, so that whether privacy information is hidden in the image or not can be judged efficiently and accurately.

It should be understood that for the foregoing method embodiments, although the steps in the flowcharts are shown in order indicated by the arrows, the steps are not necessarily performed in order indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the flow charts of the method embodiments may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least a portion of the sub-steps or stages of other steps.

Based on the same idea as the image privacy recognition method in the above embodiment, an image privacy recognition apparatus is also provided herein.

In one embodiment, as shown in fig. 4, there is provided an image privacy recognition apparatus including: an obtaining module 401, an extracting module 402, a determining module 403, a screening module 404, and a classifying module 405, wherein:

an obtaining module 401, configured to obtain an image to be identified;

an extracting module 402, configured to extract residual error matrices in at least eight different directions of an image to be identified;

a determining module 403, configured to determine a local residual error mode histogram of the image to be identified according to at least eight residual error matrices in different directions of the image to be identified;

the screening module 404 is configured to obtain a Fisher score of a feature vector in the local residual error mode histogram of the image to be identified, and screen the feature vector in the local residual error mode histogram of the image to be identified according to the Fisher score to obtain a target feature vector of the image to be identified;

the classification module 405 is configured to input the target feature vector into the trained image classification model, and identify whether the image to be identified is an image with privacy according to a classification result of the image classification model.

In some embodiments, the extracting module 402 is specifically configured to: extracting residual error matrixes of the image to be recognized in eight directions; the eight directions of the image to be recognized comprise two horizontal mutual-reaction directions, two vertical mutual-reaction directions, two diagonal mutual-reaction directions and two anti-diagonal mutual-reaction directions; the determining module 403 is specifically configured to: determining a straight local variance histogram of the image to be identified according to residual matrixes in two horizontal reciprocal directions and two vertical reciprocal directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction; determining an oblique local variance histogram of the image to be recognized according to the residual matrixes of the two diagonal reciprocal directions and the two opposite diagonal reciprocal directions; the oblique local variance histogram comprises two variance histograms with diagonal reciprocal directions and two anti-diagonal reciprocal directions; and fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified.

In some embodiments, the extracting module 402 is specifically configured to: according to the formula:

；

；

；

；

extracting residual error matrixes of the image to be recognized in eight directions;

wherein the content of the first and second substances,

representing the first in the image to be recognizedGo to the first

Pixels of a column;

andsecond of the residual matrices representing two horizontal reciprocal directions

Go to the first

The elements of the column are,，

；

and

second of the residual matrices representing two perpendicular reciprocal directions

Go to the first

The elements of the column are,

，

；

and

the first of the residual matrixes representing two diagonal reciprocal directions respectively

Go to the firstThe elements of the column are,

，

；and

second of the residual matrix representing the two opposite diagonal mutually opposite directions

Go to the first

The elements of the column are,

，

；

represents the width of the image to be recognized,

indicating a high of the image to be recognized.

In some embodiments, the determining module 403 is specifically configured to: according to the formula:

；

wherein the content of the first and second substances,

；

determining a straight local variance histogram of the image to be identified;

wherein the content of the first and second substances,

expressing a flat local variance histogramThe number of the elements is one,

；

representation matrix

First, the

Go to the first

Elements of a column; matrix array

Respectively as follows:

；

；

；

。

in some embodiments, the determining module 403 is specifically configured to: according to the formula:

；

determining a diagonal local variance histogram of the image to be identified;

wherein the content of the first and second substances,

second to express diagonal local variance histogram

The number of the elements is one,

(ii) a Matrix array

Respectively as follows:

in some embodiments, the determining module 403 is specifically configured to: according to the formula:

fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be identified;

wherein the content of the first and second substances,

a local residual mode histogram representing the image to be identified.

In some embodiments, the screening module 404 is further specifically configured to: according to the Fisher score, sorting each dimensional feature in a local residual error mode histogram of the image to be recognized from high to low; and selecting features corresponding to the Fisher scores and arranged in the front set dimension to form a target feature vector of the image to be recognized.

In some embodiments, the image privacy recognition apparatus further includes: an image classification model construction module for: constructing an image training set; acquiring images to be trained from the image training set; extracting residual error matrixes of at least eight different directions of the image to be trained; determining a local residual error mode histogram of the image to be trained according to at least eight residual error matrixes in different directions of the image to be trained; calculating Fisher scores of feature vectors in a local residual error mode histogram of the image to be trained, and screening the feature vectors in the local residual error mode histogram of the image to be trained according to the calculation result to obtain a sample feature vector of the image to be trained; and inputting the sample feature vector into a support vector machine for training to construct an image classification model.

In some embodiments, the image classification model construction module is specifically configured to: according to the formula:

wherein the content of the first and second substances,

calculating Fisher scores of feature vectors in a local residual error mode histogram of the image to be trained;

wherein the content of the first and second substances,

a second in a local residual pattern histogram representing the image to be trained

Fisher score of dimensional features;

in the image training set representing the image classification model

Local residual error mode histogram of original image;

in the image training set representing the image classification model

Local residual error mode histogram containing privacy image;

and

are respectively

And

to (1) a

An element;

representing the number of original images or privacy-containing images in an image training set of the image classification model;

representing a first intermediate vector;representing a second intermediate vector;

representing a third intermediate vector;

representing a fourth intermediate vector.

In some embodiments, the image classification model construction module is specifically configured to: extracting residual error matrixes of the images to be trained in eight directions; the eight directions of the image to be trained comprise two horizontal reciprocal directions, two vertical reciprocal directions, two diagonal reciprocal directions and two opposite diagonal reciprocal directions; determining a straight local variance histogram of the image to be trained according to residual matrixes of two horizontal reciprocal directions and two vertical reciprocal directions; the straight local variance histogram comprises two horizontal reciprocal directions and two variance histograms in a vertical reciprocal direction; determining an oblique local variance histogram of the image to be trained according to residual matrixes of two diagonal reciprocal directions and two anti-diagonal reciprocal directions; the oblique local variance histogram comprises two variance histograms with diagonal reciprocal directions and two anti-diagonal reciprocal directions; and fusing the straight local variance histogram and the oblique local variance histogram to obtain a local residual error mode histogram of the image to be trained.

In some embodiments, the support vector machine is a soft-spaced support vector machine of gaussian kernel.

For specific limitations of the image privacy recognition apparatus, reference may be made to the above limitations of the image privacy recognition method, which are not described herein again. The modules in the image privacy identification device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In addition, in the embodiment of the image privacy recognition apparatus illustrated above, the logical division of each program module is only an example, and in practical applications, the above function distribution may be performed by different program modules according to needs, for example, due to configuration requirements of corresponding hardware or due to convenience of implementation of software, that is, the internal structure of the image privacy recognition apparatus is divided into different program modules to perform all or part of the above described functions.

In one embodiment, as shown in fig. 5, there is provided a computer device comprising a memory and a processor, the memory storing a computer program, the computer program, when executed by the processor, causing the processor to perform the steps of the image privacy recognition method described above. Here, the steps of the image privacy recognition method may be steps in the image privacy recognition methods of the above-described embodiments.

In one embodiment, a computer readable storage medium is provided, storing a computer program that, when executed by a processor, causes the processor to perform the steps of the image privacy recognition method described above. Here, the steps of the image privacy recognition method may be steps in the image privacy recognition methods of the above-described embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The terms "comprises" and "comprising," as well as any variations thereof, of the embodiments herein are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or (module) elements is not limited to only those steps or elements but may alternatively include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

References to "first \ second" herein are merely to distinguish between similar objects and do not denote a particular ordering with respect to the objects, it being understood that "first \ second" may, where permissible, be interchanged with a particular order or sequence. It should be understood that "first \ second" distinct objects may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced in sequences other than those illustrated or described herein.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.