阅读说明：本技术 基于深度学习的社交网络头像比对的分布式检测系统、方法、装置、处理器及其存储介质 (Social network head portrait comparison distributed detection system, method and device based on deep learning, processor and storage medium thereof ) 是由 姚伟 王永剑 王婷 李超 于 2020-12-25 设计创作，主要内容包括：本发明涉及一种基于深度学习的社交网络头像比对的分布式检测系统,其中,该系统包括头像采集存储功能模块,用于采集社交网络中用户的头像图片和基本信息；头像相似度训练功能模块,用于对采集到的图片进行特征值提取并构建分布式头像特征向量索引库；头像实时搜索功能模块,用于针对输入的图片进行特征值提取,计算出最相似的若干张头像图片和社交网络用户ID。本发明还涉及一种基于上述系统的方法、装置、处理器及计算机可读存储介质。采用了本发明相应的系统、方法、装置、处理器及计算机可读存储介质,能够快速实时地对社交网络头像进行比对,且自适应图像的遮挡、旋转、缩放、扭曲等各种变换,也能对同类图像进行准确分类和精准检索。(The invention relates to a social network head portrait comparison distributed detection system based on deep learning, wherein the system comprises a head portrait acquisition and storage function module, a head portrait image acquisition and storage function module and a head portrait image comparison module, wherein the head portrait image acquisition and storage function module is used for acquiring head portrait images and basic information of users in a social network; the head portrait similarity training function module is used for extracting characteristic values of the collected pictures and constructing a distributed head portrait characteristic vector index library; and the head portrait real-time searching functional module is used for extracting characteristic values of the input pictures and calculating a plurality of head portrait pictures and social network user IDs which are most similar. The invention also relates to a method, a device, a processor and a computer readable storage medium based on the system. By adopting the corresponding system, method, device, processor and computer readable storage medium of the invention, the social network head portrait can be compared quickly and in real time, and the system can adapt to various transformations such as shielding, rotation, scaling, distortion and the like of the image, and can also accurately classify and accurately retrieve the similar images.)

1. A social network avatar comparison distributed detection system based on deep learning is characterized in that the system comprises:

the head portrait acquisition and storage function module is used for acquiring head portrait pictures and basic information of a specific user in the social network from the Internet and storing corresponding information;

the head portrait similarity training function module is connected with the head portrait acquisition and storage function module and is used for performing feature vector extraction based on deep learning and conversion from a high-dimensional vector to a low-dimensional vector based on a locality sensitive hashing algorithm on the acquired head portrait picture and constructing a distributed feature vector index library of the head portrait picture; and

and the head portrait real-time searching functional module is connected with the head portrait similarity training functional module and is used for calculating characteristic values of the head portrait input from the head portrait acquisition functional module, calculating the most similar head portrait pictures in the distributed characteristic vector index library by using an approximate neighbor algorithm, and combining all calculation results to obtain the integral similar head portrait pictures and the social network user ID.

2. The social network avatar comparison distributed detection system based on deep learning of claim 1, wherein the avatar collection and storage function module specifically comprises:

the head portrait acquisition unit is used for acquiring the head portrait picture and the basic information of a specific user in the social network from the Internet by adopting a network data crawler;

the head portrait storage unit is used for locally storing the head portrait picture acquired by the head portrait acquisition unit, using the MD5 hash code of the head portrait picture as the unique identification code of the head portrait picture, and storing the acquired basic information of the specific user into a head portrait picture database corresponding to the key value; and

the head portrait picture database is used for storing the social network user ID of the specific user, the local storage position of the head portrait picture and the information of the unique identification code of the head portrait picture.

3. The social network avatar comparison distributed detection system based on deep learning of claim 2, wherein the avatar similarity training function module specifically comprises:

the head portrait preprocessing unit is connected with the head portrait storage unit and is used for carrying out normalization processing on the collected head portrait picture of the specific user;

the deep learning characteristic vector extraction unit is connected with the head portrait preprocessing unit and is used for extracting characteristic vectors of the head portrait pictures input to the head portrait preprocessing unit for processing by using a VGG16 neural network model;

the local sensitive hash processing unit is connected with the deep learning characteristic vector extraction unit and is used for carrying out local sensitive characteristic hash value processing on the characteristic vector of the head portrait picture extracted by the deep learning characteristic vector extraction unit to obtain a binary hash code of the head portrait picture; and

and the distributed head portrait feature vector index library is connected with the local sensitive hash processing unit and used for storing the unique identification code and the binary hash code of the head portrait picture by randomly selecting any node.

4. The system for detecting the social network avatar comparison based on deep learning of claim 3, wherein the avatar real-time search function module specifically comprises:

the head portrait input unit is connected with the head portrait preprocessing unit and is used for sequentially inputting the head portrait picture to be detected to the head portrait preprocessing unit, the deep learning feature vector extraction unit and the local sensitive hash processing unit in the head portrait similarity training function module to perform corresponding feature value calculation processing on the head portrait picture to obtain a binary hash code of the head portrait picture to be detected;

the head portrait feature representation unit is used for carrying out feature value representation on the head portrait picture obtained after the head portrait input unit processes;

the similar head portrait picture calculation unit is respectively connected with the distributed head portrait feature vector index library and the head portrait feature representation unit, and is used for calculating the feature value of each head portrait picture which is most similar in the distributed head portrait feature vector index library by adopting an approximate neighbor algorithm based on the feature value of the head portrait picture acquired by the head portrait feature representation unit; and

and the similar user detection unit is respectively connected with the head portrait picture database and the similar head portrait picture calculation unit and is used for merging and sequencing all results calculated by the similar head portrait picture calculation unit so as to obtain a plurality of head portrait pictures which are most similar integrally and inquiring the corresponding social network user ID from the head portrait picture database according to the unique identification code of each head portrait picture.

5. A distributed detection method for realizing social network avatar comparison based on deep learning by using the system of claim 1 is characterized in that the method specifically comprises the following steps:

(1) the head portrait acquisition function module continuously acquires head portrait pictures of a specific user in a social network in a multi-thread or multi-process concurrent mode, and records the corresponding relation among the ID of the user in the social network, the storage position of the head portrait pictures and the unique identification code of the head portrait picture data;

(2) the head portrait similarity training functional module extracts a feature value of a head portrait picture acquired by the head portrait acquisition functional module by using a VGG16 neural network model trained based on an ImageNet image library to obtain a 512-dimensional feature vector, maps the feature vector by using the locality sensitive hashing algorithm to obtain a binary hashing code, and randomly stores the binary hashing code and the unique identification code of the head portrait picture to any distributed node;

(3) the head portrait similarity training functional module also uses a plurality of nodes to construct a distributed head portrait feature vector index library of user head portrait pictures in a distributed social network, and shares the pressure of head portrait retrieval in a load balancing manner;

(4) the head portrait real-time searching functional module calculates the feature vector of the input head portrait picture by using the VGG16 neural network model in the step (2) for the input head portrait picture, and converts the feature vector into a binary hash code by using the local hash sensitive algorithm; carrying out approximate value query on each head portrait picture on each distributed node by using the approximate neighbor algorithm to obtain a plurality of most similar head portrait pictures;

(5) combining the plurality of most similar head portrait pictures on each node acquired in the step (4), sequencing according to the similarity measurement, and calculating to obtain a plurality of head portraits which are most similar integrally; and searching corresponding relation in the head portrait acquisition functional module through the unique identification code of the head portrait picture to obtain the social network user and the picture file corresponding to the input head portrait picture.

6. The distributed detection method for realizing deep learning-based social network avatar comparison according to claim 5, wherein the step (2) specifically comprises the following steps:

(2.1) the avatar similarity training module adopts a VGG16 network model based on a convolutional neural network to normalize the acquired avatar pictures, performs scaling according to the size of 224 × 224, and inputs the normalized avatar pictures into a VGG16 network model of the convolutional neural network for deep learning, and extracts a 512-dimensional feature vector V, wherein the 512-dimensional feature vector V is specifically expressed by the following formula:

V＝F(I)；

wherein, I is an image of 224 × 224, F is a VGG16 network model, and V is a feature vector of 512 dimensions;

(2.2) performing characteristic value conversion on the 512-dimensional characteristic vector V obtained in the process a through a local hash algorithm to obtain a 64-bit binary characteristic code V of the avatar picture, wherein the 64-bit binary characteristic code V of the avatar picture is specifically represented by the following formula:

v＝LSH(V)；

wherein V is a feature vector with 512 dimensions, LSH is a locality sensitive hash function, and V is a 64-bit binary feature code.

7. The distributed detection method for realizing deep learning-based social network avatar comparison according to claim 6, wherein the step (3) is specifically as follows:

(3.1) randomly selecting any node, and storing the 64-bit binary code feature v and the unique identification code MD5 of the head portrait picture so as to construct the distributed head portrait feature vector index library.

8. The distributed detection method for realizing deep learning-based social network avatar comparison according to claim 7, wherein the step (4) specifically comprises the following steps:

(4.1) for the head portrait picture input with the query, carrying out normalization processing on the head portrait picture according to the size of 224 multiplied by 224, and converting the input head portrait into the 64-bit binary feature code v by sequentially using the VGG network model and the locality sensitive hashing algorithm;

(4.2) using the approximate neighbor algorithm to retrieve m head portraits which are most similar to the 64-bit binary feature code v from each node in the distributed head portraits feature vector index library, and finally obtaining m multiplied by N most similar head portraits; the relation between the similarity measure of the head portrait picture and the unique identification code is specifically expressed by the following formula:

where v is a 64-bit binary feature code, DB_iFor a vector library on the ith node in a distributed head portrait feature vector index library, ANN performs approximate neighbor algorithm calculation, i is the number of selected nodes, k is sequence numbers arranged according to the sequence from large to small of similarity, and the formula ANN (v, DB)_i) Representing that m most similar feature vectors in the feature library are obtained by calculation according to the input 64-bit binary feature code v,for the calculated similarity measure of each avatar picture,the unique identification code of each head portrait picture.

9. The distributed detection method for realizing deep learning-based social network avatar comparison according to claim 8, wherein the step (5) is specifically as follows:

(5.1) summarizing the m × N most similar head portraits and comparing all the head portraits according to the similarity measureSequencing to obtain the most similar m head portraits, and identifying the unique identification code of each similar head portrait pictureAnd inquiring the corresponding social network user ID from the head portrait picture database in the head portrait acquisition module.

10. A distributed detection device for realizing social network avatar comparison based on deep learning is characterized in that the device comprises:

a processor configured to execute computer-executable instructions;

a memory storing one or more computer-executable instructions that, when executed by the processor, perform the steps of any one of claims 5-9 of the method for distributed detection of avatar comparison for a social network based on deep learning.

11. A distributed detection processor for implementing deep learning based social network avatar comparison, the processor being configured to execute computer-executable instructions which, when executed by the processor, implement the steps of the distributed detection method for implementing deep learning based social network avatar comparison as claimed in any one of claims 5 to 9.

12. A computer-readable storage medium, having stored thereon a computer program executable by a processor to perform the steps of any one of claims 5 to 9 of the method for distributed detection of avatar based on deep learning for social networks.

Technical Field

The invention relates to the technical field of deep learning, in particular to the technical field of multimedia image retrieval and image reconnaissance, and specifically relates to a social network head portrait comparison distributed detection system, method, device, processor and computer readable storage medium based on deep learning.

Background

With the rise of social media networks, people are more and more inclined to publish their latest dynamics, express their will, and the like on social network media. People use multimedia such as pictures for information sharing more than before, and the pictures play a more important role in information transmission. The avatar in the social media network is a prominent label that marks the identity and position of the individual. Through head portrait retrieval comparison, the similar crowd interested in a certain topic, even a vest user, can be found, and tracking and tracing of a specific event can be facilitated. How to efficiently search out pictures meeting the requirements of users from massive head portraits has become an important research topic in the fields of information search and computer vision.

The feature learning based on the deep learning convolutional neural network is widely successful in the fields of image classification, target detection and the like, and becomes a new research focus and a new hotspot. The convolutional neural network can automatically learn image characteristics based on a large amount of image data, and due to the depth structure of the convolutional neural network, the characteristics are conducted layer by layer through the network, so that the expression from low-layer simple characteristics to high-layer abstract characteristics of an image can be obtained, and the convolutional neural network has stronger distinguishing and generalization performance compared with the traditional characteristics.

And calculating the similarity between the feature vectors in a high-dimensional vector space based on the head portrait comparison represented by the feature vectors, and returning a retrieval result according to the similarity. Most of traditional similarity image retrieval algorithms are nearest neighbor search methods, and the time complexity of query and the number of samples are in a linear relation. With the increase of the size of social network avatars, the retrieval speed tends to become a bottleneck. In actual engineering practice, a near-neighbor algorithm is proposed that can trade off between query time and accuracy. Therefore, how to design a fast and effective feature vector index and approximate neighbor algorithm also becomes an urgent need in massive image retrieval.

The tree-index-structure-based approximate neighbor query method can reduce the time complexity to a logarithmic level, but as the characteristic dimension is continuously increased, the overhead generated by a tree-structure-based query algorithm is exponentially increased. In order to more effectively process the high-dimensional image query problem, the locality sensitive hashing algorithm projects high-dimensional data to low-dimensional data, so that effective similarity measurement can be performed by using a distance calculation formula with extremely low complexity.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a social network head portrait comparison distributed detection system, method, device, processor and computer readable storage medium thereof based on deep learning, which can accurately classify and accurately retrieve.

In order to achieve the above object, the present invention provides a system, a method, a device, a processor and a computer readable storage medium for detecting social network avatar comparison based on deep learning, wherein the system comprises:

the social network head portrait comparison distributed detection system based on deep learning is mainly characterized by comprising:

the head portrait acquisition and storage function module is used for acquiring head portrait pictures and basic information of a specific user in the social network from the Internet and storing the corresponding information;

the head portrait similarity training function module is connected with the head portrait acquisition and storage function module and used for carrying out feature vector extraction based on deep learning and conversion from a high-dimensional vector to a low-dimensional vector based on a locality sensitive hashing algorithm on the acquired head portrait picture and constructing a distributed feature vector index library of the head portrait picture; and

and the head portrait real-time searching functional module is connected with the head portrait similarity training functional module and is used for calculating characteristic values of the head portraits input from the head portrait acquisition functional module, calculating the most similar head portrait pictures in the distributed characteristic vector index library by using an approximate neighbor algorithm, and combining all calculation results to obtain the integral similar head portrait pictures and the social network user ID.

Preferably, the head portrait acquisition and storage function module specifically includes:

the head portrait acquisition unit is used for acquiring the head portrait picture and basic information of a specific user in the social network from the Internet by adopting a network data crawler;

the head portrait storage unit is used for locally storing the head portrait picture acquired by the head portrait acquisition unit, using the MD5 hash code of the head portrait picture as the unique identification code of the head portrait picture, and storing the acquired basic information of the specific user into a head portrait picture database corresponding to the key value; and

the head portrait picture database is used for storing the social network user ID of the specific user, the local storage position of the head portrait picture, the unique identification code of the head portrait picture and other information.

Preferably, the avatar similarity training function module specifically includes:

the head portrait preprocessing unit is connected with the head portrait storage unit and is used for carrying out normalization processing on the collected head portrait pictures of the specific user;

the deep learning characteristic vector extraction unit is connected with the head portrait preprocessing unit and is used for extracting characteristic vectors of the head portrait pictures input to the head portrait preprocessing unit for processing by using a VGG16 neural network model;

the local sensitive hash processing unit is connected with the deep learning characteristic vector extraction unit and is used for carrying out local sensitive characteristic hash value processing on the characteristic vector of the head portrait picture extracted by the deep learning characteristic vector extraction unit to obtain a binary hash code of the head portrait picture; and

and the distributed head portrait feature vector index library is connected with the binary hash code processing unit and used for storing the unique identification code and the binary hash code of the head portrait picture by randomly selecting any node.

Preferably, the avatar real-time searching function module specifically includes:

the head portrait input unit is connected with the head portrait preprocessing unit and is used for sequentially inputting the head portrait picture to be detected to a head portrait preprocessing model unit, a deep learning characteristic vector extraction unit and a local sensitive hash processing unit in the head portrait similarity training function module to perform corresponding characteristic value calculation processing on the head portrait picture to obtain a binary hash code of the head portrait picture to be detected;

the head portrait feature representation unit is used for carrying out feature value representation on the head portrait picture obtained after the head portrait input unit processes;

the similar head portrait picture calculation unit is respectively connected with the distributed head portrait feature vector index library and the head portrait feature representation unit, and is used for calculating the feature value of each head portrait picture which is most similar in the distributed head portrait feature vector index library by adopting an approximate neighbor algorithm based on the feature value of the head portrait picture acquired by the head portrait feature representation unit; and

and the similar user detection unit is respectively connected with the head portrait picture database and the similar head portrait picture calculation unit and is used for merging and sequencing all results calculated by the similar head portrait picture calculation unit so as to obtain a plurality of head portrait pictures which are most similar integrally, and inquiring the corresponding social network user ID from the head portrait picture database according to the unique identification code of each head portrait picture.

The distributed detection method for realizing the social network head portrait comparison based on the deep learning is mainly characterized by comprising the following steps:

(1) the head portrait acquisition function module continuously acquires head portrait pictures of a specific user in a social network in a multi-thread or multi-process concurrent mode, and records the corresponding relation among the ID of the user in the social network, the storage position of the head portrait pictures and the unique identification code of the head portrait picture data;

(2) the head portrait similarity training functional module extracts a feature value of a head portrait picture acquired by the head portrait acquisition functional module by using a VGG16 neural network model trained based on an ImageNet image library to obtain a 512-dimensional feature vector, and the feature vector is mapped by using the locality sensitive hashing algorithm to obtain a binary hashing code and is randomly stored on any distributed node together with a unique identification code of the head portrait picture;

(3) the head portrait similarity training functional module also uses a plurality of nodes to construct a distributed head portrait feature vector index library of user head portrait pictures in a distributed social network, and shares the pressure of head portrait retrieval in a load balancing manner;

(4) the head portrait real-time searching functional module calculates the feature vector of the input head portrait picture by using the VGG16 neural network model in the step (2) for the input head portrait picture, and converts the feature vector into a binary hash code by using the local hash sensitive algorithm; carrying out approximate value query on each head portrait picture on each distributed node by using the approximate neighbor algorithm to obtain a plurality of most similar head portrait pictures;

(5) combining the plurality of most similar head portrait pictures on each node acquired in the step (4), sequencing according to the similarity measurement, and calculating to obtain a plurality of head portraits which are most similar integrally; and searching corresponding relation in the head portrait acquisition functional module through the unique identification code of the head portrait picture to obtain the social network user and the picture file corresponding to the input head portrait picture.

Preferably, the step (2) specifically comprises the following steps:

(2.1) the avatar similarity training module adopts a VGG16 network model based on a convolutional neural network to normalize the acquired avatar pictures, performs scaling according to the size of 224 × 224, and inputs the normalized avatar pictures into a VGG16 network model of the convolutional neural network for deep learning, so as to extract a 512-dimensional feature vector V, wherein the 512-dimensional feature vector V is specifically expressed by the following formula:

V＝F(I)；

wherein, I is an image of 224 × 224, F is a VGG16 network model, and V is a feature vector of 512 dimensions;

(2.2) performing characteristic value conversion on the 512-dimensional characteristic vector V obtained in the process a through a local hash algorithm to obtain a 64-bit binary characteristic code V of the avatar picture, wherein the 64-bit binary characteristic code V of the avatar picture is specifically represented by the following formula:

v＝LSH(V)；

wherein V is a feature vector with 512 dimensions, LSH is a locality sensitive hash function, and V is a 64-bit binary feature code.

Preferably, the step (3) is specifically:

(3.1) randomly selecting any node, and storing the 64-bit binary code feature v and the unique identification code MD5 of the head portrait picture so as to construct the distributed head portrait feature vector index library.

Preferably, the step (4) specifically includes the following steps:

(4.1) for the head portrait picture input with the query, carrying out normalization processing on the head portrait picture according to the size of 224 multiplied by 224, and converting the input head portrait into the 64-bit binary feature code v by sequentially using the VGG network model and the locality sensitive hashing algorithm;

(4.2) retrieving m head images which are most similar to the 64-bit binary feature code v from each node in the distributed head image feature vector index library by using the approximate neighbor algorithm, and finally obtaining m by N most similar head images; specifically, the following formula is used:

where v is a 64-bit binary feature code, DB_iFor a vector library on the ith node in a distributed head portrait feature vector index library, ANN is used for carrying out approximate neighbor algorithm calculation, i is the number of selected nodes, k is the sequence number arranged from large to small according to the similarity, and the formula ANN (v, DB)_i) Representing that m most similar feature vectors in the feature library are obtained by calculation according to the input 64-bit binary feature code v,for the calculated similarity measure of each avatar picture,the unique identification code of each head portrait picture.

Preferably, the step (5) is specifically:

(5.1) summarizing the m × N most similar head portraits and comparing all the head portraits according to the similarity measureSorting to obtain the most similar m head portraits, and identifying the unique identification code of each similar head portrait pictureAnd inquiring the corresponding social network user ID from the head portrait picture database in the head portrait acquisition module.

The distributed detection device for realizing the social network head portrait comparison based on the deep learning is mainly characterized by comprising:

a processor configured to execute computer-executable instructions;

a memory storing one or more computer-executable instructions that, when executed by the processor, perform the steps of the detection method described above.

The distributed detection processor for realizing the social network head portrait comparison based on deep learning is mainly characterized in that the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the detection method are realized.

The computer-readable storage medium is primarily characterized in that a computer program is stored thereon, which computer program can be executed by a processor to carry out the individual steps of the detection method described above.

By adopting the social network head portrait comparison distributed detection system, the social network head portrait comparison distributed detection method, the social network head portrait comparison distributed detection device, the social network head portrait comparison processor and the computer readable storage medium thereof, matching search can be accurately performed on input head portrait pictures in a constructed distributed head portrait feature vector index library, various transformations such as shielding, rotation, scaling, distortion and the like of images can be also adaptively performed on head portraits which generate various deformations such as scaling, distortion and local correction, the same type of images can be accurately classified and accurately retrieved, and pictures which meet user requirements can be efficiently retrieved from massive head portraits.

Drawings

FIG. 1 is a general flowchart of a distributed detection system for deep learning-based social networking avatar comparison according to the present invention.

FIG. 2 is a schematic diagram of the comparison process of searching the most similar head portrait in the distributed feature vector index library according to the present invention.

Fig. 3 is a schematic diagram of a search result of similarity feature retrieval performed on a head portrait picture based on deformation according to the present invention.

Fig. 4 is a schematic diagram of a search result of similarity feature retrieval based on the same type of head portrait pictures according to the present invention.

Detailed Description

In order to more clearly describe the technical contents of the present invention, the following further description is given in conjunction with specific embodiments.

Before describing in detail embodiments that are in accordance with the present invention, it should be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

Referring to fig. 1, the system for detecting social network avatar comparison based on deep learning includes:

the head portrait acquisition and storage function module is used for acquiring head portrait pictures and basic information of a specific user in the social network from the Internet and storing the corresponding information;

the head portrait similarity training function module is connected with the head portrait acquisition and storage function module and used for carrying out feature vector extraction based on deep learning and conversion from a high-dimensional vector to a low-dimensional vector based on a locality sensitive hashing algorithm on the acquired head portrait picture and constructing a distributed feature vector index library of the head portrait picture; and

and the head portrait real-time searching functional module is connected with the head portrait similarity training functional module and is used for calculating characteristic values of the head portraits input from the head portrait acquisition functional module, calculating the most similar head portrait pictures in the distributed characteristic vector index library by using an approximate neighbor algorithm, and combining all calculation results to obtain the integral similar head portrait pictures and the social network user ID.

As a preferred embodiment of the present invention, the head image acquisition and storage function module specifically includes:

the head portrait acquisition unit is used for acquiring the head portrait picture and basic information of a specific user in the social network from the Internet by adopting a network data crawler;

the head portrait storage unit is used for locally storing the head portrait picture acquired by the head portrait acquisition unit, using the MD5 hash code of the head portrait picture as the unique identification code of the head portrait picture, and storing the acquired basic information of the specific user into a head portrait picture database corresponding to the key value; and

the avatar picture database is used for storing the social network user ID of the specific user, the local storage position of the avatar picture and the information of the unique identification code MD5 of the avatar picture.

As a preferred embodiment of the present invention, the avatar similarity training function module specifically includes:

the head portrait preprocessing unit is connected with the head portrait storage unit and is used for carrying out normalization processing on the collected head portrait pictures of the specific user;

the deep learning characteristic vector extraction unit is connected with the head portrait preprocessing unit and is used for extracting characteristic vectors of the head portrait pictures input to the head portrait preprocessing unit for processing by using a VGG16 neural network model;

the local sensitive hash processing unit is connected with the deep learning characteristic vector extracting unit and is used for carrying out local sensitive characteristic hash value processing on the characteristic vector of the head portrait picture extracted by the deep characteristic vector extracting unit to obtain a binary hash code of the head portrait picture; and

and the distributed avatar feature vector index library is connected with the binary hash code processing unit and used for storing the MD5 hash code and the binary hash code of the avatar picture by randomly selecting any node.

As a preferred embodiment of the present invention, the avatar real-time search function module specifically includes:

the head portrait input unit is connected with the head portrait preprocessing unit and is used for sequentially inputting the head portrait picture to be detected to the head portrait preprocessing unit, the deep learning characteristic vector extraction unit and the local sensitive hash processing unit in the head portrait similarity training functional module to perform corresponding characteristic value calculation processing on the head portrait picture to obtain a binary hash code of the head portrait picture to be detected;

the head portrait feature representation unit is used for carrying out feature value representation on the head portrait picture obtained after the head portrait input unit processes;

the similar head portrait picture calculation unit is respectively connected with the distributed head portrait feature vector index library and the head portrait feature representation unit, and is used for calculating the feature value of each head portrait picture which is most similar in the distributed head portrait feature vector index library by adopting an approximate neighbor algorithm based on the feature value of the head portrait picture acquired by the head portrait feature representation unit; and

and the similar user detection unit is respectively connected with the head portrait picture database and the similar head portrait picture calculation unit and is used for merging and sequencing all results calculated by the similar head portrait picture calculation unit so as to obtain a plurality of head portrait pictures which are most similar integrally and inquiring the corresponding social network user ID from the head portrait picture database according to the unique identification code of each head portrait picture.

The distributed detection method for realizing the social network head portrait comparison based on deep learning specifically comprises the following steps:

(1) the head portrait acquisition function module continuously acquires head portrait pictures of a specific user in a social network in a multi-thread or multi-process concurrent mode, and records the corresponding relation among the ID of the user in the social network, the storage position of the head portrait pictures and the unique identification code of the head portrait picture data;

(2) the head portrait similarity training functional module extracts a feature value of a head portrait picture acquired by the head portrait acquisition functional module by using a VGG16 neural network model trained based on an ImageNet image library to obtain a 512-dimensional feature vector, maps the feature vector by using the locality sensitive hashing algorithm to obtain a binary hashing code, and randomly stores the binary hashing code on any distributed node by combining with a unique identification code of the head portrait picture;

(3) the head portrait similarity training functional module also uses a plurality of nodes to construct a distributed head portrait feature vector index library of user head portrait pictures in a distributed social network, and shares the pressure of head portrait retrieval in a load balancing manner;

(4) the head portrait real-time searching functional module calculates the feature vector of the input head portrait picture by using the VGG16 neural network model in the step (2) for the input head portrait picture, and converts the feature vector into a binary hash code by using the local hash sensitive algorithm; carrying out approximate value query on each head portrait picture on each distributed node by using the approximate neighbor algorithm to obtain a plurality of most similar head portrait pictures;

(5) combining the plurality of most similar head portrait pictures on each node acquired in the step (4), sequencing according to the similarity measurement, and calculating to obtain a plurality of head portraits which are most similar integrally; and searching corresponding relation in the head portrait acquisition functional module through the unique identification code of the head portrait picture to obtain the social network user and the picture file corresponding to the input head portrait picture.

As a preferred embodiment of the present invention, the step (2) specifically comprises the following steps:

(2.1) the avatar similarity training module adopts a VGG16 network model based on a convolutional neural network to normalize the acquired avatar pictures, performs scaling according to the size of 224 × 224, and inputs the normalized avatar pictures into a VGG16 network model of the convolutional neural network for deep learning, so as to extract a 512-dimensional feature vector V, wherein the 512-dimensional feature vector V is specifically expressed by the following formula:

V＝F(I)；

wherein, I is an image of 224 × 224, F is a VGG16 network model, and V is a feature vector of 512 dimensions;

(2.2) performing characteristic value conversion on the 512-dimensional characteristic vector V obtained in the process a through a local hash algorithm to obtain a 64-bit binary characteristic code V of the avatar picture, wherein the 64-bit binary characteristic code V of the avatar picture is specifically represented by the following formula:

v＝LSH(V)；

wherein V is a feature vector with 512 dimensions, LSH is a locality sensitive hash function, and V is a 64-bit binary feature code.

As a preferred embodiment of the present invention, the step (3) specifically comprises:

(3.1) randomly selecting any node, and storing the 64-bit binary code feature v and the unique identification code MD5 of the head portrait picture so as to construct the distributed head portrait feature vector index library.

Referring to fig. 2, as a preferred embodiment of the present invention, the step (4) specifically includes the following steps:

(4.1) for the head portrait picture input with the query, carrying out normalization processing on the head portrait picture according to the size of 224 multiplied by 224, and converting the input head portrait into the 64-bit binary feature code v by sequentially using the VGG network model and the locality sensitive hashing algorithm;

(4.2) retrieving m head images which are most similar to the 64-bit binary feature code v from each node in the distributed head image feature vector index library by using the approximate neighbor algorithm, and finally obtaining m by N most similar head images; the relation between the similarity measure of the head portrait picture and the unique identification code is specifically expressed by the following formula:

where v is a 64-bit binary feature code, DB_iFor a vector library on the ith node in a distributed head portrait feature vector index library, ANN is used for carrying out approximate neighbor algorithm calculation, i is the number of selected nodes, k is the sequence number arranged from large to small according to the similarity, and the formula ANN (v, DB)_i) Representing that m most similar feature vectors in the feature library are obtained by calculation according to the input 64-bit binary feature code v,for the calculated similarity measure of each avatar picture,the unique identification code of each head portrait picture.

As a preferred embodiment of the present invention, the step (5) specifically comprises:

(5.1) summarizing the m × N most similar head portraits and comparing all the head portraits according to the similarity measureSorting to obtain the most similar m head portraits, and identifying the unique identification code of each similar head portrait pictureAnd inquiring the corresponding social network user ID from the head portrait picture database in the head portrait acquisition module.

The distributed detection device for realizing social network avatar comparison based on deep learning comprises:

a processor configured to execute computer-executable instructions;

a memory storing one or more computer-executable instructions that, when executed by the processor, perform the steps of the detection method described above.

The distributed detection processor for realizing the social network head portrait comparison based on deep learning is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the steps of the detection method are realized.

The computer-readable storage medium, in which a computer program is stored which is executable by a processor to implement the steps of the detection method described above.

Referring to fig. 3, in an embodiment of the present invention, fig. 3 is an example of performing similar image retrieval on an avatar picture of a cartoon character based on various deformations such as zooming, distortion, and local correction, and it can be seen that even if various deformations such as zooming, distortion, and local correction are performed on an original image, the detection system of the present technical solution performs overall processing, and then can search out the most similar avatar picture after deformation, so as to effectively implement accurate classification and accurate retrieval on similar images.

Referring to fig. 4, in an embodiment of the present invention, fig. 4 is an example of similar image retrieval based on the same type of avatar pictures, and it can be seen that the technical solution adopted in the present invention can adapt to various transformations such as occlusion, rotation, scaling, and distortion of an image, and can also perform accurate classification and accurate retrieval on the same type of image.

In a specific embodiment of the present invention, in the present technical solution, a distributed avatar feature vector index library is constructed using 10 servers with Intel Xeon E5-2660 processors, a main frequency of 2GHZ, and a memory of 64G, and each node stores approximately 1000 ten thousand vectors of an avatar. Table 1 shows the required run time for the entire inspection system of the present invention, wherein the processing time for each avatar is approximately 26 milliseconds.

TABLE 1 System runtime complexity

The invention provides a distributed head portrait comparison method based on a deep learning model aiming at the retrieval problem of mass social network head portraits, which comprises the following steps: in the training stage, deep migration learning is carried out on the collected head portrait data based on a deep neural network model to obtain a feature vector of the head portrait data, a local sensitive Hash algorithm is further used for projecting the high-dimensional feature vector to a low dimension, then the low-dimensional feature vector is stored in any one of distributed nodes, and a distributed head portrait low-dimensional feature index library is constructed; and extracting image features through deep migration learning during real-time retrieval, then finding the most similar head portrait on the current node from the low-dimensional feature index library by using a nearest neighbor matching algorithm and utilizing the similarity measurement with extremely low complexity on each distributed node, and finally merging and sequencing the results on each node to obtain the most similar head portrait.

It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by suitable instruction execution devices.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware that can be related to instructions of a program, and the program may be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.

In the description herein, references to the description of the terms "an embodiment," "some embodiments," "an example," "a specific example," or "an embodiment," "an implementation," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and not to be construed as limiting the present invention, and that changes, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

By adopting the social network head portrait comparison distributed detection system, the social network head portrait comparison distributed detection method, the social network head portrait comparison distributed detection device, the social network head portrait comparison processor and the computer readable storage medium thereof, matching search can be accurately performed on input head portrait pictures in a constructed distributed head portrait feature vector index library, various transformations such as shielding, rotation, scaling, distortion and the like of images can be also adaptively performed on head portraits which generate various deformations such as scaling, distortion and local correction, the same type of images can be accurately classified and accurately retrieved, and pictures which meet user requirements can be efficiently retrieved from massive head portraits.

In this specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.