阅读说明：本技术 图像分类方法、装置及存储介质、电子设备 (Image classification method and device, storage medium and electronic equipment ) 是由 刘弘一 陈若田 熊军 李若鹏 于 2019-09-06 设计创作，主要内容包括：本说明书的实施例公开了一种图像分类方法、装置及存储介质、电子设备,涉及图像处理领域。首先通过经卷积层对目标图像的图像像素矩阵提取基础特征；然后经胶囊层根据基础特征提取图像特征向量,并对图像特征向量进行传播与路由更新；最后经全连接层根据路由更新后的图像特征向量重构目标图像,从而使得重构后的目标图像保留了目标图像的重要信息,最后将所述目标图像是否为广告图像作为分类目标对重构后的目标图像进行分类。(The embodiment of the specification discloses an image classification method and device, a storage medium and electronic equipment, and relates to the field of image processing. Firstly, extracting basic characteristics of an image pixel matrix of a target image through a convolutional layer; then extracting image characteristic vectors through a capsule layer according to the basic characteristics, and carrying out propagation and routing updating on the image characteristic vectors; and finally, reconstructing the target image according to the image feature vector after route updating through the full connection layer, so that the reconstructed target image retains the important information of the target image, and finally classifying the reconstructed target image by taking whether the target image is an advertisement image as a classification target.)

1. An image classification method is applied to a capsule network, the capsule network is obtained by training historical image samples and classification labels associated with the historical image samples in advance, the classification labels comprise advertisement image labels and non-advertisement image labels, the capsule network comprises a convolution layer, a capsule layer and a full-connection layer which are connected in sequence, and the method comprises the following steps:

extracting basic features of an image pixel matrix of a target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through the capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

2. The image classification method according to claim 1, wherein the capsule layer comprises a main capsule layer and a digital capsule layer, and the extracting of the image feature vectors according to the basic features via the capsule layer and the propagating and routing update of the image feature vectors comprise:

generating an image characteristic vector representing the probability that a target region in the target image contains the target feature and the image characteristic parameter of the target region according to the basic feature through the main capsule layer;

and carrying out propagation and routing update on the image feature vector through the digital capsule layer.

3. The method of claim 2, the image feature vector comprising image feature parameters comprising at least one of position information, flip case, shape, brightness, size.

4. The image classification method according to claim 1, wherein the fully-connected layer includes a plurality of fully-connected units and a softmax layer connected in sequence, the reconstructing a target image from the image feature vector updated by the route via the fully-connected layer, and the classifying whether the target image is an advertisement image or not as a classification target includes:

extracting high-order features of the image feature vector after route updating step by step through a plurality of full-connection units, and reconstructing a target image according to the high-order features;

and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target through the softmax layer.

5. The method of claim 4, wherein the fully connected unit is a network structure that sequentially performs weight calculation, dynamic routing adjustment, and activation function operation.

6. The method of claim 1, prior to said extracting base features via said convolutional layer for an image pixel matrix of a target image, further comprising:

extracting at least one characteristic of the layout, the content and the tone of the historical image sample;

and constructing training samples according to at least one characteristic of the layout, the content and the tone of the historical image samples and the classification labels which are identified to the historical image samples in advance to train the capsule network.

7. The method of claim 1, prior to said inputting an image matrix of target images into a plurality of pre-trained capsule networks, further comprising:

and performing at least one preprocessing operation of zooming, rotating, cutting, edge detecting and resolution enhancing on the target image.

8. The method of claim 1, prior to said extracting base features via said convolutional layer for an image pixel matrix of a target image, further comprising:

loading the capsule network in parallel based on a plurality of processing engines.

9. An image classification device is applied to a capsule network, wherein the capsule network is obtained by training historical image samples and classification labels associated with the historical image samples in advance, the classification labels comprise advertisement image labels and non-advertisement image labels, the capsule network comprises a convolution layer, a capsule layer and a full-connection layer which are connected in sequence, and the device comprises:

a basic feature extraction unit for extracting basic features from an image pixel matrix of a target image through the convolution layer;

the characteristic vector extraction unit is used for extracting image characteristic vectors according to the basic characteristics through the capsule layer and carrying out propagation and routing updating on the image characteristic vectors;

and the image classification unit reconstructs a target image according to the image feature vector after route updating through the full connection layer, and classifies the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements:

extracting basic features of an image pixel matrix of a target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through the capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

11. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement:

extracting basic features of an image pixel matrix of a target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through the capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

Technical Field

The embodiment of the specification provides an image classification method and device, a storage medium and an electronic device.

Background

For some businesses, it may be desirable to outsource advertisements in order to expand promotions. If the advertisement is put in the areas such as websites, newspapers, magazines, elevator rooms, bus stations and the like, the advertisement picture mainly comes from the picture data source in the enterprise. Therefore, before advertisement delivery, advertisement pictures need to be screened from the internal picture data source.

In the current scheme, in the process of processing advertisement pictures, various important information in the pictures is lost, for example, image posture parameters, and then accurate screening of the advertisement pictures cannot be completed, so that a scheme for accurately identifying the advertisement pictures needs to be provided.

Disclosure of Invention

In a first aspect, an embodiment of the present disclosure provides an image classification method applied to a capsule network, where the capsule network is obtained by training historical image samples and classification labels associated with the historical image samples in advance, where the classification labels include advertisement image labels and non-advertisement image labels, the capsule network includes a convolutional layer, a capsule layer, and a full-connection layer, which are connected in sequence, and the method includes:

extracting basic features of an image pixel matrix of a target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through the capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

Optionally, the capsule layer includes a main capsule layer and a digital capsule layer, the extracting, by the capsule layer, the image feature vector according to the basic feature, and the propagating and routing updating the image feature vector includes:

generating an image characteristic vector representing the probability that a target region in the target image contains the target feature and the image characteristic parameter of the target region according to the basic feature through the main capsule layer;

and carrying out propagation and routing update on the image feature vector through the digital capsule layer.

Optionally, the image feature vector includes an image feature parameter, and the image feature parameter includes at least one of position information, flip condition, shape, brightness, and size.

Optionally, the fully-connected layer includes a plurality of fully-connected units and a softmax layer connected in sequence, the reconstructing a target image according to the image feature vector updated by the route through the fully-connected layer, and classifying the reconstructed target image by using whether the target image is an advertisement image as a classification target includes:

extracting high-order features of the image feature vector after route updating step by step through a plurality of full-connection units, and reconstructing a target image according to the high-order features;

and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target through the softmax layer.

Optionally, the full-connection unit is a network structure that sequentially performs weight calculation, dynamic routing adjustment, and activation function operation.

Optionally, before the extracting, via the convolutional layer, a base feature for an image pixel matrix of a target image, the method further includes:

extracting at least one characteristic of the layout, the content and the tone of the historical image sample;

and constructing training samples according to at least one characteristic of the layout, the content and the tone of the historical image samples and the classification labels which are identified to the historical image samples in advance to train the capsule network.

Optionally, before the inputting the image matrix of the target image into the plurality of pre-trained capsule networks, the method further comprises:

and performing at least one preprocessing operation of zooming, rotating, cutting, edge detecting and resolution enhancing on the target image.

Optionally, before the extracting, via the convolutional layer, a base feature for an image pixel matrix of a target image, the method further includes:

loading the capsule network in parallel based on a plurality of processing engines.

In a second aspect, an embodiment of the present specification further provides an image classification device applied to a capsule network, where the capsule network is obtained by training historical image samples and classification labels associated with the historical image samples in advance, where the classification labels include advertisement image labels and non-advertisement image labels, the capsule network includes a convolutional layer, a capsule layer, and a full-link layer, which are connected in sequence, and the device includes:

a basic feature extraction unit for extracting basic features from an image pixel matrix of a target image through the convolution layer;

the characteristic vector extraction unit is used for extracting image characteristic vectors according to the basic characteristics through the capsule layer and carrying out propagation and routing updating on the image characteristic vectors;

and the image classification unit reconstructs a target image according to the image feature vector after route updating through the full connection layer, and classifies the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

In a third aspect, embodiments of the present specification further provide a computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements:

extracting basic features of an image pixel matrix of a target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through the capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

In a fourth aspect, embodiments of the present specification further provide an electronic device, including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement:

extracting basic features of an image pixel matrix of a target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through the capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image or not as a classification target.

The embodiment of the present specification adopts at least one technical scheme that can achieve the following beneficial effects:

firstly, extracting basic characteristics of an image pixel matrix of a target image through a convolutional layer; the capsule layer encapsulates the image characteristic parameters of the image, so that the image characteristic parameters can be ensured not to be lost in the subsequent processing process of the image, then the capsule layer extracts the image characteristic vectors according to the basic characteristics, and the propagation and routing updating are carried out on the image characteristic vectors; and finally, reconstructing the target image according to the image feature vector after route updating through the full connection layer, so that the reconstructed target image also retains the image feature parameters of the target image, and finally classifying the reconstructed target image by taking whether the target image is the advertisement image as a classification target, thereby realizing the accurate screening of the advertisement image.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the specification and are incorporated in and constitute a part of this specification, illustrate embodiments of the specification and together with the description serve to explain the embodiments of the specification and not to limit the embodiments of the specification in any way. In the drawings:

fig. 1 is a schematic diagram of interaction between a user terminal and a server provided in an embodiment of the present specification;

FIG. 2 is a diagram of a hierarchical architecture of a capsule network provided by embodiments of the present description;

FIG. 3 is a flow chart of an image classification method provided by an embodiment of the present description;

FIG. 4 is a flow chart of an image classification method provided by an embodiment of the present description;

FIG. 5 is a flow chart of an image classification method provided by an embodiment of the present description;

FIG. 6 is a flow chart of an image classification method provided by an embodiment of the present description;

fig. 7 is a functional unit block diagram of an image classification apparatus provided in an embodiment of the present specification;

FIG. 8 is a functional unit block diagram of an image classification apparatus according to an embodiment of the present disclosure

FIG. 9 is a functional unit block diagram of an image classification apparatus according to an embodiment of the present disclosure

Fig. 10 is a functional unit block diagram of an image classification apparatus provided in an embodiment of the present specification;

fig. 11 is a circuit connection block diagram of an electronic device provided in an embodiment of the present description.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the embodiments of the present disclosure will be described in detail and completely with reference to the specific embodiments of the present disclosure and the accompanying drawings. It is to be understood that the embodiments described are only some embodiments and not all embodiments of the present disclosure. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort belong to the protection scope of the embodiments in the present specification.

Technical solutions provided by embodiments of the present specification are described in detail below with reference to the accompanying drawings.

Capsule network: the capsule network is composed of capsules, rather than neurons, the capsules being a small set of neurons that are used to learn to detect a particular object (e.g., a rectangle) within a given image region, which outputs a vector (e.g., an 8-dimensional vector) whose length represents an estimated probability of the presence of the detected object, and whose orientation (e.g., in 8-dimensional space) encodes image characteristic parameters (e.g., location information, flip conditions, etc.) of the detected object. If the detected object changes (such as moves, rotates, adjusts size, etc.), the capsule will output the same length vector, but the direction of the vector is different, and thus the capsule is equal.

And (3) rolling layers: the method is characterized by comprising a plurality of convolution units, and the parameters of each convolution unit are optimized through a back propagation algorithm. The convolution operation aims to extract different basic features of an input image, a first layer of convolution layer can only extract some low-level features such as edges, lines, corners and other levels, and more layers of networks can iteratively extract more complex features from the low-level features.

Capsule layer: the capsule layer encapsulates various features that represent particular entities in the image, characterizing the presence of instantiated entities in the image. Instantiated entities are image parameters such as position, size, orientation, deformation, velocity, albedo, hue, texture, etc. The layers of capsules may be organized into different levels, where capsules of the same level predict, learn the shape of the object, and pass it on to capsules of higher levels in the direction of learning. When multiple predictions agree, higher level capsule layers become active, a process that is dynamic routing. In particular, the primary layer of the capsule layer receives as input a region of the image (called the receptive field) to detect the presence and pose of a particular pattern (e.g., a circle). The advanced layers of the capsule layer detect larger and more complex objects, such as the number 8 consisting of two circles, and then use a new squeeze function to ensure that the length of these vectors is between 0-1.

Full connection layer: each node of the fully connected layer is connected with all nodes of the previous layer and is used for integrating the extracted characteristics to reconstruct the image. Specifically, each neuron in the fully-connected layer is fully connected to all neurons in the layer preceding it. The fully connected layer may integrate local information with category distinctiveness in the convolutional layer or the pooling layer. The excitation function of each neuron of the fully-connected layer generally adopts a ReLU function. The output values of the last fully connected layer are passed to the output, which may be sorted using softmax logistic regression, for example.

Referring to fig. 1, an embodiment of the present disclosure provides an image classification method, which is applied to a server 101, where the server 101 is loaded with a capsule network, and the capsule network is obtained by training historical image samples and classification tags associated with the historical image samples in advance, where the classification tags include advertisement image tags and non-advertisement image tags, and the classification tags are marked on the historical image samples by a worker in advance through a user terminal 102. As shown in fig. 2, the capsule network includes a convolutional layer 201, a capsule layer 202, and a full connection layer 203, which are connected in sequence, and the server 101 is communicatively connected to the user terminal 102 for data interaction. As shown in fig. 3, the method includes:

s30: the basis features are extracted for the image pixel matrix of the target image via the convolutional layer 201.

Specifically, the user may input a target image at an application of the user terminal 102, the target image may be an image of an internal picture data source in a business (including an advertisement image and a non-advertisement image), and then click "send", the target image may be sent to the server 101, the server 101 inputs the target image to a pre-trained capsule network, and the pre-trained capsule network starts extracting basic features from the target image, wherein the basic features may be edges, lines, corners, and the like of the image. Specifically, in the capsule network of the embodiments of the specification, convolutional layer 201 may include 256 9x9x1 cores with step size 1, which may be activated by ReLU.

In the embodiment of the present specification, before performing S201, at least one of scaling, rotating, cropping, edge detecting, and resolution enhancing may be performed on the target image in advance, so as to remove the interference factor for classifying the image, so as to lay a foundation for more accurately classifying the target image.

S32: the image feature vectors are extracted through the capsule layer 202 according to the basic features, and the propagation and routing update are performed on the image feature vectors.

The image feature vector contains image feature parameters, and the image feature parameters may include at least one of position information, flipping condition, shape, brightness, and size. The capsule layer 202 includes a plurality of capsules, each of which includes a set of neurons that identify an image characteristic parameter of the target region. For example, a neuron a in a group of neurons recognizes the posture of the target region, a neuron B recognizes the shape of the target region, a neuron C recognizes the inversion of the target region, and the like, and the capsule derives the probability that the target region includes the target feature from the recognition results of a plurality of neurons.

It will be appreciated that the above-described convolutional layer 201, main capsule layer, and digital capsule layer form an encoder for the capsule network.

S34: reconstructing a target image according to the image feature vector after route updating through the full connection layer 203, and classifying the reconstructed target image by taking whether the target image is an advertisement image as a classification target.

For example, target images whose layout, content, and hue are such that they clearly characterize an advertising campaign are classified as advertising images. For example, the layout is the layout of a theme, text content, picture and trademark from top to bottom; the text content comprises a contact telephone, a contact address, and a target image with multiple colors, which is determined as an advertisement image.

In addition, the target image determined as the advertisement image may be fed back to the user terminal 102 and displayed on a display interface of an application program of the user terminal 102 for selection and delivery by the user.

In the image classification method provided by the embodiment of the present specification, first, basic features are extracted from an image pixel matrix of a target image through a convolutional layer 201; because the capsule layer 202 encapsulates the image characteristic parameters of the image, the image characteristic parameters can be ensured not to be lost in the subsequent processing process of the image, then the capsule layer 202 extracts the image characteristic vectors according to the basic characteristics, and the image characteristic vectors are propagated and routed for updating; and finally, reconstructing the target image according to the image feature vector after the route updating through the full connection layer 203, so that the reconstructed target image also keeps the image feature parameters of the target image, and finally classifying the reconstructed target image by taking whether the target image is the advertisement image as a classification target, thereby realizing the accurate screening of the advertisement image.

Alternatively, as shown in fig. 4, S32 includes:

s40: and generating an image feature vector representing the probability that a target region in the target image contains the target feature and the image feature parameter of the target region according to the basic feature through the main capsule layer.

The main capsule layer of the embodiment of the present specification is the bottom layer of a multi-dimensional entity category, and may include 32 channels, each channel is composed of an 8-dimensional convolution structure, and each channel outputs an 8-dimensional vector, which may achieve an 8 × 1 feature encapsulation effect. The main capsule layer contains a set of neurons whose input-output vectors represent instantiation parameters for a particular entity class. A particular entity category of embodiments of the present description is advertising image features, such as phone numbers, contact addresses, brand pictures, and so forth. For example, the convolution kernel of the main capsule layer of the embodiment of the present specification may be 9 × 9 convolution kernel and step 2, where the convolution calculation of the main capsule layer does not use an activation function such as ReLU (Rectified Linear Unit), but is ready to be input into the next digital capsule layer in a vector manner.

S42: and carrying out propagation and routing update on the image feature vectors through the digital capsule layer.

Specifically, in the digital capsule layer, the modulo length of the activation vector of each digital capsule gives whether an instance of each specific entity class exists, for example, the output range of the digital capsule layer is between 0 and 1, 0 indicates that an instance of a specific entity class does not exist, and 1 indicates that an instance of a specific entity class exists. The input of the digital capsule layer is the output vectors of all capsules in the main capsule layer, and the vector dimension of the input vector is [8, 1 ]; the vector dimension of the output vector of the digital capsule layer is [16, 1 ].

Alternatively, as shown in fig. 5, S33 includes:

s50: and extracting high-order features of the image feature vector after route updating step by step through a plurality of full-connection units, and reconstructing a target image according to the high-order features.

Optionally, the full-connection unit is a network structure that sequentially performs weight calculation, dynamic routing adjustment, and activation function operation according to the high-order features. Specifically, in the embodiments of the present specification, 3 fully-connected units connected in sequence may be included, where the first 2 fully-connected units use a ReLU activation function to perform calculation, and the 3 rd fully-connected unit uses a Sigmoid activation function to perform calculation of a reconstruction error, so as to obtain a reconstruction target image. It will be appreciated that the 3 fully connected units described above constitute the decoder of the capsule network.

Specifically, the vectors received from the digital capsule layer of the fully-connected layer 203 are reconstructed to obtain reconstructed vectors, and the reconstructed vectors and the vectors of the input image after matrix stretching are subjected to line variance calculation to obtain reconstruction errors so as to reconstruct the target image.

S52: and classifying the reconstructed target image by taking whether the target image is the advertisement image or not as a classification target through a softmax layer.

Optionally, as shown in fig. 6, before S30, the method further includes:

s60: and extracting at least one characteristic of the layout, the content and the tone of the historical image sample.

S62: and constructing a training sample training capsule network according to at least one characteristic of the layout, content and tone of the historical image samples and the classification labels which are identified to the historical image samples in advance.

S64: the capsule network is loaded in parallel based on multiple processing engines.

When a plurality of processing engines load the capsule network in parallel, the efficiency of classifying the target image can be improved when the above-described processes of S31-S33 are performed using the capsule network.

Referring to fig. 7, an embodiment of the present disclosure further provides an image classification apparatus 700 applied to a capsule network, the capsule network is obtained by training historical image samples and classification labels associated with the historical image samples in advance, the capsule network includes a convolution layer 201, a capsule layer 202, and a full-connection layer 203, which are connected in sequence, the apparatus 700 includes a basic feature extraction unit 701, a feature vector extraction unit 702, and an image classification unit 703, wherein,

the base feature extraction unit 701 extracts a base feature from the image pixel matrix of the target image via the convolution layer 201.

The feature vector extraction unit 702 extracts image feature vectors according to the basic features via the capsule layer 202, and performs propagation and routing update on the image feature vectors.

The image feature vector comprises image feature parameters, and the image feature parameters comprise at least one of position information, turning condition, shape, brightness and size.

The image classification unit 703 reconstructs a target image according to the route-updated image feature vector through the full connection layer 203, and classifies the reconstructed target image by using whether the target image is an advertisement image as a classification target.

The image classification apparatus 700 provided in the embodiment of the present specification can realize the following functions when executing the functions: firstly, extracting basic characteristics from an image pixel matrix of a target image through a convolutional layer 201; because the capsule layer 202 encapsulates the image characteristic parameters of the image, the image characteristic parameters can be ensured not to be lost in the subsequent processing process of the image, then the capsule layer 202 extracts the image characteristic vectors according to the basic characteristics, and the image characteristic vectors are propagated and routed for updating; and finally, reconstructing the target image according to the image feature vector after the route updating through the full connection layer 203, so that the reconstructed target image also keeps the image feature parameters of the target image, and finally classifying the reconstructed target image by taking whether the target image is the advertisement image as a classification target, thereby realizing the accurate screening of the advertisement image.

Optionally, the capsule layer 202 includes a main capsule layer and a digital capsule layer, as shown in fig. 8, the feature vector extraction unit 702 includes:

the feature vector generation unit 801 generates an image feature vector representing the probability that a target region in the target image includes a target feature and an image feature parameter of the target region according to the basic feature through the main capsule layer.

The feature vector processing unit 802 performs propagation and routing update on the image feature vectors through the digital capsule layer.

The full link layer 203 includes a plurality of full link units and a softmax layer connected in sequence, and as shown in fig. 9, the image classification unit 703 includes:

the image reconstructing unit 901 extracts the high-order features of the image feature vector after route updating step by step through a plurality of full-connection units, and reconstructs the target image according to the high-order features.

The classification determination unit 902 classifies the reconstructed target image by using whether the target image is an advertisement image as a classification target via the softmax layer.

Specifically, the full-connection unit is a network structure that sequentially performs weight calculation, dynamic routing adjustment, and activation function operation.

Optionally, as shown in fig. 10, the apparatus 700 may further include:

image feature extraction section 1001 extracts at least one feature of the layout, content, and color tone of a history image sample.

The model training unit 1002 is configured to construct a training sample training capsule network according to at least one feature of the layout, content, and color tone of the historical image sample and the classification label identified in advance for the historical image sample.

And a model loading unit 1003 for loading the capsule network in parallel based on the plurality of processing engines.

And an image preprocessing unit 1004 for performing at least one preprocessing operation of scaling, rotating, cropping, edge detection and resolution enhancement on the target image.

It should be noted that the execution subjects of the steps of the method provided in embodiment 1 may be the same device, or different devices may be used as the execution subjects of the method. For example, the execution subject of steps 21 and 22 may be device 1, and the execution subject of step 23 may be device 2; for another example, the execution subject of step 21 may be device 1, and the execution subjects of steps 22 and 23 may be device 2; and so on.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, where the electronic device may be a server in the foregoing embodiment. Referring to fig. 11, at a hardware level, the electronic device includes a processor, and optionally, an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (peripheral component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 11, but that does not indicate only one bus or one type of bus.

And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.

The processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the image classification device on a logic level. The processor is used for executing the program stored in the memory and is specifically used for executing the following operations:

extracting basic features of an image pixel matrix of the target image through the convolution layer;

extracting image characteristic vectors through a capsule layer according to the basic characteristics, and carrying out propagation and routing updating on the image characteristic vectors;

and reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image as a classification target.

The method performed by the image classification apparatus disclosed in the embodiment of fig. 3 in the present specification can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present specification may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present specification may be embodied directly in the hardware decoding processor, or in a combination of the hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

The electronic device may also execute the method of fig. 3 and implement the functions of the image classification apparatus in the embodiment shown in fig. 3, which are not described herein again in this specification.

Of course, besides the software implementation, the electronic device of the embodiment of the present specification does not exclude other implementations, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.

Embodiments of the present description also propose a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a portable electronic device comprising a plurality of application programs, are capable of causing the portable electronic device to perform the method of the embodiment shown in fig. 3, and in particular for performing the following:

extracting basic features of an image pixel matrix of the target image through the convolution layer;

extracting image characteristic vectors according to the basic characteristics through a capsule layer, and carrying out propagation and routing updating on the image characteristic vectors;

and reconstructing a target image according to the image feature vector after route updating through the full connection layer, and classifying the reconstructed target image by taking whether the target image is an advertisement image as a classification target.

In short, the above description is only a preferred embodiment of the embodiments of the present disclosure, and is not intended to limit the scope of the embodiments of the present disclosure. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the embodiments of the present specification shall be included in the protection scope of the embodiments of the present specification.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.