阅读说明：本技术 电子装置和用于控制该电子装置的方法 (Electronic device and method for controlling the same ) 是由 赵韩秀 金庆勳 朴永焕 金硕镇 金铉中 权东旭 于 2018-06-05 设计创作，主要内容包括：公开了一种电子装置。所述电子装置包括：存储器；以及处理器,针对从根据神经网络的学习产生的多个滤波器选择的每个主样式的滤波器检查用于对神经网络的输入数据进行滤波的操作命令,并且将检查的操作命令存储在存储器中。(An electronic device is disclosed. The electronic device includes: a memory; and a processor checking an operation command for filtering input data of the neural network for each filter of the main pattern selected from the plurality of filters generated according to the learning of the neural network, and storing the checked operation command in the memory.)

1. An electronic device, comprising:

a memory; and

a processor configured to check an operation instruction for filtering input data of the neural network for a filter of each main pattern selected from a plurality of filters generated according to learning by the neural network, and store the checked operation instruction in the memory.

2. The electronic device as claimed in claim 1, wherein the processor performs a filtering operation by the filter of the main pattern by using the stored operation instruction.

3. The electronic apparatus according to claim 1, wherein the filter of the main pattern is a filter selected among the plurality of generated filters based on a statistical weight of an operation amount occupied by each pattern for each pattern in which non-zero elements are arranged.

4. The electronic device of claim 1, wherein the processor selects the primary pattern among the plurality of generated filters based on a frequency of each pattern in which non-zero elements are arranged.

5. The electronic device of claim 1, further comprising: a communication device for communicating with a plurality of communication devices,

wherein the processor transmits the operation instruction to another electronic device through the communicator, and

the other electronic device performs a filtering operation by the filter of the main pattern by using the operation instruction.

6. The electronic device of claim 1, wherein the processor checks optimization data from which zero elements are removed from the selected filter of the main pattern and an operation instruction corresponding to the optimization data, and stores the checked optimization data and operation instruction in the memory.

7. The electronic device of claim 6, wherein the processor performs a filtering operation through the filter of the main pattern by using the stored optimization data and an operation instruction corresponding to the identification information included in the stored optimization data.

8. The electronic device of claim 6, further comprising: a communication device for communicating with a plurality of communication devices,

wherein the processor transmits the optimization data and the operation instruction to another electronic device through the communicator, and

the other electronic device performs a filtering operation through the filter of the main pattern by using the received optimization data and an operation instruction corresponding to the identification information included in the received optimization data.

9. A method for controlling an electronic device, the method comprising:

checking an operation instruction for filtering input data of the neural network for a filter of each main pattern selected from a plurality of filters generated according to learning by the neural network; and is

The checked operation instruction is stored.

10. The method of claim 9, further comprising: performing a filtering operation by the filter of the main pattern by using the stored operation instruction.

11. The method of claim 9, wherein the filter of the main pattern is a filter selected among the plurality of generated filters based on a statistical weight of an operation amount occupied by each pattern for each pattern in which non-zero elements are arranged.

12. The method of claim 9, wherein in the checking of the operation instruction, the main pattern is selected among the generated plurality of filters based on a frequency of each pattern in which non-zero elements are arranged.

13. The method of claim 9, further comprising: transmitting the stored operation instruction to another electronic device,

wherein the other electronic device performs a filtering operation by the filter of the main pattern by using an operation instruction.

14. The method of claim 9, wherein in the checking of the operation instruction, optimization data in which zero elements are removed from the selected filter of the main pattern and an operation instruction corresponding to the optimization data are checked, and

in the step of storing the checked operation instruction, the checked optimization data and the operation instruction are stored.

15. The method of claim 14, further comprising: performing a filtering operation through the filter of the main pattern by using the stored optimization data and an operation instruction corresponding to the identification information included in the stored optimization data.

Technical Field

The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device and a method for controlling the same to reduce the amount of operations required in a process of performing learning through a neural network.

Background

Machine learning in the field of artificial intelligence refers to a technique for studying and constructing a system for predicting the future and improving its performance by collecting and analyzing large-scale big data and an algorithm for the system.

In recent years, as hardware technology has been developed, as computer skills and techniques capable of collecting and storing large data and analyzing the large data have become more complicated and rapid, research into machine learning, which is an algorithm that can recognize an object and understand information like a human, has been actively conducted. In particular, in the field of machine learning technology, research on deep learning of a self-learning method using a neural network is active.

The neural network is an algorithm in which an excitation function is intended to determine a final output by comparing a sum obtained by multiplying a plurality of inputs with weights with a specific boundary value to actively simulate the function of the human brain, and generally includes a plurality of layers. Representative examples of the neural network include a convolutional neural network (hereinafter, CNN) widely used for image recognition, a recurrent neural network (hereinafter, RNN) widely used for voice recognition, and the like.

However, in the conventional CNN, since a filtering operation by convolution is basically performed, a large number of operations are required since a plurality of filters are applied to input data. That is, since the convolution operation is repeated for the generation of the feature map, a large number of operations and large-capacity neural network parameters are required, which in turn causes problems of degradation in operation speed and inefficient use of memory.

Therefore, in machine learning, there is a demand for a method for reducing the burden of such an operation.

Disclosure of Invention

Technical problem

The present disclosure provides an electronic device and a method for controlling the electronic device to improve operation speed and memory usage in an operation process of a neural network.

Technical scheme

According to an embodiment of the present disclosure, an electronic apparatus includes: a memory; and a processor configured to check an operation instruction for filtering input data of the neural network for a filter of each main pattern selected from a plurality of filters generated according to learning by the neural network, and store the checked operation instruction in the memory.

The processor may perform a filtering operation by the filter of the main pattern by using the stored operation instructions.

The filter of the main pattern may be a filter selected among the plurality of generated filters based on a statistical weight of an operation amount occupied by each pattern for each pattern in which non-zero elements are arranged.

The processor may select a main pattern among the generated plurality of filters based on a frequency of each pattern in which non-zero elements are arranged.

The electronic device may further include a communicator, wherein the processor may transmit the operation instruction to another electronic device through the communicator, and the another electronic device may perform a filtering operation through the filter of the main pattern by using the operation instruction.

The processor may check the optimization data from which zero elements are removed from the filter of the selected main pattern and an operation instruction corresponding to the optimization data, and store the checked optimization data and operation instruction in the memory.

The processor may perform a filtering operation through the filter of the main pattern by using the stored optimization data and an operation instruction corresponding to the identification information included in the stored optimization data.

The processor may transmit the optimization data and the operation instruction to the other electronic device through the communicator, and the other electronic device may perform a filtering operation through the filter of the main pattern by using the received optimization data and the operation instruction corresponding to the identification information included in the received optimization data.

According to another embodiment of the present disclosure, a method for controlling an electronic device includes: checking an operation instruction for filtering input data of the neural network for a filter of each main pattern selected from a plurality of filters generated according to learning by the neural network; and stores the checked operation instruction.

The method may further include performing a filtering operation by the filter of the main pattern by using the stored operation instruction.

The filter of the main pattern may be a filter selected among the plurality of generated filters based on a statistical weight of an operation amount occupied by each pattern for each pattern in which non-zero elements are arranged.

In the checking of the operation instruction, a main pattern may be selected among the generated plurality of filters based on a frequency of each pattern in which non-zero elements are arranged.

The method may further include transmitting the stored operation instruction to another electronic device, wherein the another electronic device may perform a filtering operation by the filter of the main pattern using the operation instruction.

In the checking of the operation instruction, the optimization data from which the zero elements are removed from the filter of the selected main pattern and the operation instruction corresponding to the optimization data may be checked, and in the storing of the checked operation instruction, the checked optimization data and the operation instruction may be stored.

The method may further include performing a filtering operation through the filter of the main pattern by using the stored optimization data and the operation instruction included in the stored optimization data.

The method may further include transmitting the stored optimization data and the operation instruction to the other electronic device, wherein the other electronic device may perform a filtering operation through the filter of the main pattern by using the optimization data and the operation instruction corresponding to the identification information included in the optimization data.

Advantageous effects

According to the various embodiments disclosed above, in the operations required for machine learning, the operation speed can be greatly reduced and the operation amount can be reduced, and the memory usage can also be reduced compared to the conventional memory usage.

Drawings

Fig. 1a and 1b are block diagrams schematically illustrating a configuration of an electronic device according to different embodiments of the present disclosure.

Fig. 2 is a diagram illustrating an operation process of a generic CNN performing a classification operation by applying a plurality of filters to input data according to an embodiment of the present disclosure.

Fig. 3a and 3b are diagrams for describing a filter of a main pattern according to an embodiment of the present disclosure.

Fig. 4 is a diagram illustrating a structure of optimization data for a filter having a style a as one of main styles according to an embodiment of the present disclosure.

Fig. 5 is a block diagram schematically illustrating a configuration of another electronic apparatus according to an embodiment of the present disclosure.

Fig. 6 is a diagram illustrating a machine learning system that receives an operation instruction for a main filter from a server and performs a convolution operation using the operation instruction according to an embodiment of the present disclosure.

Fig. 7a and 7b are diagrams for describing a method for learning various types of input images and recognizing the input images using the learned data according to an embodiment of the present disclosure.

Fig. 8 is a flowchart for describing a method of performing learning by a neural network according to an embodiment of the present disclosure.

Detailed Description

Before describing the present disclosure in detail, a method of describing the specification and drawings will be described.

First, general terms have been selected for terms used in the present specification and claims in consideration of the functions of the present disclosure. However, these terms may be changed according to the intention of those skilled in the art, legal or technical explanations, and the emergence of new technologies. In addition, some terms are arbitrarily selected by the applicant. Unless otherwise defined, these terms may be construed as meanings as defined herein and may be construed based on the overall contents of the specification and common technical knowledge in the prior art.

In addition, like reference numerals or symbols in the drawings attached to the present specification denote components or elements performing substantially the same function. For ease of explanation and understanding, the same reference numbers or symbols will be used to describe the different embodiments. That is, although all components having the same reference number are shown in the drawings, the drawings do not imply one embodiment.

In addition, in the present specification and claims, terms including ordinals (such as "first" and "second") may be used to distinguish between components. These ordinals are used to distinguish the same or similar components from each other, and the meaning of the terms should not be construed as being limited by the use of these ordinals. As an example, components combined with these ordinals should not be construed as limiting the order of their use or arrangement. The respective ordinals may be used interchangeably, if necessary.

In this specification, the singular forms include the plural forms unless the context clearly dictates otherwise. It will be understood that the terms "comprises" and "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and components, or groups of features, integers, steps, operations, elements, and components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and components, or groups thereof.

The terms "module", "unit", "means", and the like in the embodiments of the present disclosure are terms used to denote a component that performs at least one function or operation, and such a component may be implemented in hardware or software or a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. may be integrated into at least one module or chip, and may be implemented in at least one processor (not shown), except for the case where the plurality of "modules", "units", "parts", etc. are required to be implemented each in a single specific hardware.

In addition, in the embodiments of the present disclosure, when a component is connected to another component, this includes not only a direct connection but also an indirect connection through another medium. In addition, unless explicitly described otherwise, the meaning of any part "comprising" any component means that any part does not exclude other components and may also include other components.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

Fig. 1a and 1b are block diagrams schematically illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

Referring to fig. 1a, an electronic device 100 for performing learning through a neural network includes a memory 110 and a processor 120.

The memory 110 is configured to store learning data through a neural network. Here, the learning data may include a plurality of filters generated according to learning by the neural network. In this case, the memory 110 may store data on which preprocessing has been performed on the generated plurality of filters, which will be described later.

The memory 110 may be implemented as various formats of memory such as a Hard Disk Drive (HDD), a Solid State Drive (SSD), a DRAM memory, an SRAM memory, a FRAM memory, or a flash memory.

The processor 120 is configured to control the overall operation of the electronic device 100. The processor 120 performs preprocessing on the filter of the main pattern selected from the plurality of filters generated by the learning of the neural network.

Here, the filter is defined as a weight matrix as a mask having weights. The filter is also considered a window or kernel. The weights constituting the matrix in the filter include zero elements of 0 (zero value) or approximately 0 and non-zero elements having a constant value between 0 and 1, and may have various patterns according to the function of the filter.

For example, in the case where the neural network is implemented as a convolutional neural network (hereinafter, CNN) for identifying an image, the processor 120 may apply a filter having a weight to an input image and extract a feature map by determining a sum of values obtained by multiplying the image by the weight of the filter (convolution operation) as a pixel value of an output image. The input image may be extracted a plurality of times through a plurality of filters to extract the robust features, and a plurality of feature maps may be extracted according to the number of filters. Such a convolved image may be repeated through multiple layers. In this case, the filter to be learned changes according to the learning object of CNN, and the style of the selected filter also changes. That is, the filter to be learned and the filter to be selected vary depending on what CNN is learned (such as cats, puppies, pigs, and cows).

As described above, the processor 120 may determine what type of feature the input original data has by combining a plurality of filters capable of extracting different features and applying the combined filters to the CNN.

On the other hand, the filter of the main pattern is a filter in which non-zero elements among a plurality of learned filters are arranged as the main pattern, and refers to a filter including a pattern occupying an operation amount larger than or equal to a predetermined ratio during a filtering operation, or a filter including a pattern occupying a frequency larger than or equal to the predetermined ratio among a plurality of learned filters. This will be described in detail with reference to fig. 3a and 3 b.

On the other hand, the preprocessing performed by the processor 120 includes a process of checking, for each filter of the selected main pattern, an operation instruction for filtering input data of the neural network by using the filter of the selected main pattern. The operation instruction is an instruction to replace a convolution operation for filtering input data of the neural network, and may be compiled and recognized. The checked operation instructions may be stored in the memory 110.

The processor 120 may perform a filtering operation through the filter of the main pattern according to the stored operation instruction. To this end, the processor 120 according to an embodiment may further include an operation module (not shown) for performing a filtering operation.

In addition, the electronic device 100' according to another embodiment may further include a communicator 130 as shown in fig. 1b, and the processor 120 may transmit the operation instruction to another electronic device 200 through the communicator 130. The other electronic device 200 may include an operation module and perform a filtering operation through the filter of the main pattern by using the operation instruction.

The communicator 130 may transmit and receive a wireless signal with the other electronic device 200 in a mobile communication network established according to a global system for mobile communication (GSM), Code Division Multiple Access (CDMA), code division multiple access 2000(CDMA2000), enhanced voice data optimized or enhanced voice data only (EV-DO), wideband CDMA (wcdma), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), long term evolution advanced (LTE-a), or the like.

In addition, communicator 130 may support Wi-Fi, Bluetooth, and Wi-Fi direct, and may include an Ethernet terminal for connecting to a wired network.

In addition, the communicator 130 may transmit the operation instruction to the other electronic device 200 in a wired manner, and in this case, may include a connector for connecting to the other electronic device 200 in a wired manner.

On the other hand, the processor 120 may obtain optimization data in which zero elements are removed from the filter of the selected main pattern, and may check an operation instruction for performing a filtering operation according to the obtained optimization data. The processor 120 may remove zero elements from the filter of the selected main pattern and may generate optimization data for the filter of the main pattern, wherein the optimization data is composed of non-zero elements and identification information for identifying the pattern. In this case, the optimization data has a data structure in which identification information and non-zero elements are arranged in a line.

In addition, the operation instruction is an instruction for causing, in an operation module for filtering input data of the neural network, a specific processing element among a plurality of processing elements included in the operation module to perform a specific operation. The plurality of processing elements may be arranged in a predetermined configuration to construct an operational module.

Such an operation module may be included in the other electronic device 200 performing communication with the electronic device 100 'as described above, and the electronic device 100' may transmit the optimization data and the operation instructions stored in the memory 100 to the other electronic device 200. The other electronic device 200 may perform a filtering operation of the neural network through the filter of the main pattern by using the optimization data and the operation instruction received from the electronic device 100'.

On the other hand, the processor 120 may store optimization data and operation instructions corresponding to the filters of the main pattern in the memory 110 for each type of main pattern.

Fig. 2 illustrates an operation process of a general CNN performing a classification operation by applying a plurality of filters to input data according to an embodiment of the present disclosure.

As shown in fig. 2, a plurality of filters 22-1 to 22-n may be applied to the input image 21 of the CNN. In this case, the filters 22-1 to 22-n differ according to the type of CNN, but may generally include 64 to 512 types of filters. On the other hand, the input image 21 may be divided into a plurality of images before being input to the CNN and input to the CNN.

Here, the input image 21 may have a specific depth for representing a color, and may be composed of three pieces of data represented by, for example, red (R), green (G), and blue (B).

The CNN basically has a structure in which a convolutional layer and a pooling layer are repeated. In the convolutional layer, a plurality of feature maps 23-1 to 23-n for the input image 21 are generated based on the number of filters 22-1 to 22-n applied, and the generated plurality of feature maps are sampled in the pooling layer. As such processes are repeated, multiple feature maps 24-1 through 24-n having reduced dimensions may be produced. Finally, the input image 21 is classified as a specific object through the full connection layer. Fig. 2 shows an example of classifying the input image 21 as "cat".

Fig. 3a and 3b are diagrams for describing a filter of a main pattern according to an embodiment of the present disclosure.

As shown in fig. 3a, in a neural network, a filter having a constant pattern based on a form in which zero values or zero elements (which may be approximated to zero values) and non-zero elements (having a constant value between 0 and 1, non-zero values) are arranged is used.

In this case, as shown in fig. 3b, among the filters used in the neural network, there is a filter having a pattern of use (that is, a main pattern) of a certain high frequency. Filters with a main pattern are widely used for all types of neural networks, and the filters with the main pattern can be used to classify images with a constant high probability. Since the operation amount for the filter having the main pattern statistically occupies a high operation ratio of the total operation amount of the neural network, the operation efficiency can be improved by performing the preprocessing on the main filter.

The processor 120 may select a main pattern among the plurality of filters generated according to learning by the neural network based on a frequency of each pattern in which the non-zero elements are arranged. That is, a predetermined number of patterns may be selected as the main pattern in the order of patterns that occur relatively high frequency among the respective patterns of the generated filter. In addition, a pattern having an appearance frequency of a predetermined number of times or more among the respective patterns of the generated filter may also be selected as the main pattern.

On the other hand, the filter of the main pattern may also be selected among the plurality of filters generated according to learning by the neural network based on the statistical weight of the operation amount occupied by each pattern for each pattern in which the non-zero elements are arranged. That is, filters having a predetermined number of different main patterns may be selected in order of a high ratio of the operation amount to the total operation amount required to process the input data, and in this case, a specific number of filters having a sum of the operation amounts exceeding a specific ratio may be selected.

For example, in the case of VGG16, which is one type of neural network, assuming that the number of recognition objects is 10, the operation amount required to apply 20 types of filters among the 215 types of filters used in VGG16 to input data accounts for 60% of the total operation amount. In this case, the processor 120 may select 20 types of filters, which account for 60% of the operation amount, as the filters that perform the preprocessing. In addition, in the case of ResNet, which is another type of neural network, assuming that the number of recognition targets is 80, the operation amount required to apply 20 types of filters among 277 types of filters used in ResNet to input data accounts for 40% of the total operation amount. In this case, the processor 120 may select 20 types of filters, which account for 40% of the operation amount, as the filters that perform the preprocessing.

That is, although there are several hundreds types of filters used in the neural network, the number of patterns of the operation amount occupying the predetermined threshold ratio may be several tens of types among the several hundreds types of filters. In view of such a tendency, an object of the present disclosure is to reduce the operation amount of the convolution operation of the filter according to the main pattern by storing in advance an operation instruction corresponding to filtering of input data by the applied filter when the filter having the main pattern is applied to the input data.

On the other hand, in the above-described embodiment, although it is assumed that the position of the non-zero element on the two-dimensional plane (x, y) of the filter has a predetermined pattern, the preprocessed object may be extended to a "pattern" unit instead of a "filter" unit. In this case, when a main pattern of a non-element exists in one region of the filter, an operation instruction for the region in which the main pattern exists may be stored in advance. In this case, an embodiment in which an operation is performed by using a pre-stored operation instruction for an area in which a main pattern exists and a filtering operation by conventional convolution is performed for the remaining areas other than the area in which the main pattern exists may also be included in the technical spirit of the present disclosure. In addition, even in the case where a non-element main pattern exists in the depth (z) direction of the filter, an operation instruction for a region where the main pattern exists may be stored in advance.

Fig. 4 shows a structure of optimization data for a filter having a pattern a as one of main patterns.

When input data of a neural network is filtered using a filter having a pattern a as a main pattern (hereinafter, filter a) among the filters 41-1 to 41-n having a plurality of patterns, the filter a applied to the input data may have a data array structure in which non-zero elements and zero elements included in the filter a are arranged in series as shown in (a) of fig. 4.

The optimization data of the filter a proposed in the present disclosure has a structure in which a tag indicating that the applied filter is the pattern a is included in the first field region of the data array and the remaining field region is composed of only non-zero elements of the filter a. Here, the label is used as identification information for identifying the style a. In summary, the optimized data has a structure in which all zero elements are removed from the existing data array structure and only a tag indicating the type of style is added.

Fig. 5 is a block diagram schematically illustrating a configuration of another electronic apparatus according to an embodiment of the present disclosure.

The further electronic device 200 is configured to apply the learning data received from the electronic device 100' to the CNN to produce a result determined or predicted from the input data as output data. Hereinafter, for simplicity, the electronic device 100' will be referred to as a first electronic device, and the other electronic device 200 will be referred to as a second electronic device.

Referring to fig. 5, the second electronic device 200 includes a communicator 210, an operation module 220, a memory 230, and a processor 240.

The communicator 210 is configured to perform communication with the first electronic device 100' performing learning through a neural network. The second electronic device 200 may receive an operation instruction of the filter selected by the first electronic device 100 'from the first electronic device 100' through the communicator 210.

In addition, the second electronic device 200 according to another embodiment may also receive the optimization data of the selected filter and the operation instruction corresponding to the optimization data from the first electronic device 100'. In addition, the communicator 210 may receive an updated operation instruction whenever the operation instruction is updated according to the learning of the first electronic device 100'.

In addition, the communicator 210 of the second electronic device 200 according to another embodiment may receive the updated optimization data and the operation instruction whenever the optimization data of the main filter and the operation instruction corresponding to the optimization data are updated according to the learning of the first electronic device 100'.

The communicator 210 may transmit and receive wireless signals to and from the first electronic device 100' in a mobile communication network established according to a global system for mobile communications (GSM), Code Division Multiple Access (CDMA), code division multiple access 2000(CDMA2000), enhanced voice data optimized or enhanced voice data only (EV-DO), wideband CDMA (wcdma), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), long term evolution-advanced (LTE-a), and the like.

In addition, communicator 210 may support Wi-Fi, Bluetooth, and Wi-Fi direct.

The operation module 220 includes a plurality of Processing Elements (PEs) arranged in a predetermined structure, and is configured to perform operations by sharing data between adjacent processing elements. For example, the PEs may be arranged in the structure of various types of networks, such as mesh topology networks and tree topology networks. The memory 230 is configured to store data on which preprocessing has been performed on the filter of the main pattern. In particular, the memory 230 stores optimization data for the filter of the main pattern and operation instructions corresponding to the optimization data received from the first electronic device 100'.

The memory 230 may be implemented as various formats of memory such as a Hard Disk Drive (HDD), a Solid State Drive (SSD), a DRAM memory, an SRAM memory, a FRAM memory, or a flash memory.

The processor 240 is configured to control the overall operation of the second electronic device 200. Specifically, the processor 240 may control the communicator 230 to receive a filter of a main pattern of the plurality of filters generated through the learning of the neural network from the first electronic device 100' and to receive an operation instruction corresponding to the main pattern (that is, an operation instruction for filtering input data of the neural network through the filter of the main pattern). Processor 240 stores the received operational instructions in memory 230. When the filter of the main pattern is preprocessed using the optimization data in the first electronic device 100 'and an operation instruction corresponding to the optimization data is received from the first electronic device 100', the processor 240 stores the optimization data and the operation instruction corresponding to the optimization data in the memory 230.

On the other hand, when a filter of the main pattern among the plurality of filters is applied to the input data, the processor 240 may check an operation instruction corresponding to the applied main pattern among the operation instructions stored in the memory 230.

The processor 240 derives operation values based on the input data and the pre-processed optimized data according to the checked operation instructions. Specifically, the processor 240 may control the operation module 220 to obtain an operation instruction corresponding to the identification information from the memory based on the identification information included in the optimization data, and perform a filtering operation according to the obtained operation instruction.

In this case, the processor 240 may control the operation module 220 to obtain an operation instruction corresponding to the identification information included in the optimization data from the memory 230 and perform a filtering operation according to the obtained operation instruction. The derived operation values are typically the same as those used to filter the input data based on a convolution operation through a predetermined pattern of filters.

Fig. 6 illustrates a machine learning system that receives operational instructions for a primary filter from a first electronic device and performs convolution operations using the operational instructions, according to an embodiment of the disclosure. Fig. 6 illustrates an embodiment in which the first electronic device 100' performing learning through a neural network is implemented as a server and the second electronic device 200 performing a filtering operation on input data using data learned through the first electronic device is implemented as a user device. In addition, in fig. 6, a filter composed of a main pattern is referred to as a main filter for simplicity.

The server 100' is configured to perform preprocessing on the main filter and generate optimized data and operation instructions corresponding to the main filter.

As shown in fig. 6, the server 100' includes a memory 110, a processor 120, and a communicator 130. The processor 120 learns a plurality of filters applied to the neural network, selects a main filter of the learned plurality of filters, and stores the main filter in the memory 110. The processor 120 may select at least one filter of the predetermined pattern as the main filter in order of the pattern of the high frequencies among the learned filters. For example, as shown in fig. 6, the processor 120 may store first to third data for first to third filter learning in the memory 110, wherein the first to third filters are main filters.

Referring to fig. 6, the processor 120 may generate first to third optimization data for optimizing the first to third filters using a data analysis software tool, respectively. The first to third optimization data each include a label indicating a pattern type of the first to third filters determined according to an arrangement pattern of non-zero elements included in each of the first to third filters.

On the other hand, the first to third optimization data may be data from which zero elements included in the filter are removed. In this case, the first to third optimization data may have a smaller number of occupied bits than the originally learned data. That is, in this case, since the first to third optimization data include only non-zero elements and a tag indicating a type of a pattern of the filter, the number of occupied bits can be saved compared to the original learned data occupying a certain number of bits regardless of the type of the filter. In this case, the number of bits representing the optimized data may be variously determined according to the value of the non-zero element. For example, the non-zero elements at positions "1", "4", and "7" in the first optimization data may be represented by 8 bits, respectively, where 8 bits is an optimal number of bits for representing each non-zero element. On the other hand, the non-zero elements at positions "1" and "9" in the second optimization data may be represented by 16 bits, respectively, where 16 bits is an optimal number of bits for representing each non-zero element.

Meanwhile, the processor 120 may generate the first through third operation instructions as the operation instructions for the first through third filters, respectively, based on the first through third optimization data. In this case, the operation instruction is an instruction for causing a processing element, which is determined among a plurality of processing elements included in the operation module 220 included in the user device 200 according to the main pattern, to perform an operation determined according to the main pattern. The first to third operation commands may be determined according to the pattern types of the first to third filters.

The processor 120 may transmit the first to third optimization data and the first to third manipulation instructions to the user device 200 through the communicator 130.

Referring to fig. 6, the user device 200 may receive first to third optimization data and first to third manipulation instructions through the communicator. The processor 240 may store the received first to third optimization data and first to third operation instructions in the memory 230. In this case, the first to third optimization data and the first to third operation instructions may be stored in different areas (data memory, operation instruction memory) of the memory 230, respectively.

The first to third optimization data stored in the memory 230 may be input to the manipulation module 220, and the manipulation module 220 may perform manipulations according to the first to third manipulation instructions, respectively, when the first to third optimization data are applied to the input image. The processor 240 performs a filtering operation by convolution using a processing element determined among the plurality of processing elements included in the operation block 220 by the first through third operation instructions for each of the first through third optimized data.

On the other hand, the processor 240 may receive the changed first to third optimization data from the server 100 'and perform the update whenever the first to third optimization data stored in the server 100' are changed. In addition, when the main filter selected by the server 100' is changed, the processor 240 may receive optimization data and operation instructions for the changed main filter and perform updating.

Fig. 7a and 7b are diagrams for describing a method for learning various types of input images and recognizing the input images using the learned data according to an embodiment of the present disclosure.

As shown in fig. 7a, the processor 240 of the second electronic device 200 may control the manipulation module 220 to perform a manipulation according to the CNN algorithm for image recognition, and may recognize various animal images such as "cat", "squirrel", "deer", and "dog".

In this case, the optimization data regarding the main pattern learning may be stored in the memory 230 of the second electronic device 200 in advance. The optimization data may include, for example, respective data of a tag and a non-zero element each indicating a type of a main pattern of a filter composed of a pattern a, a pattern B, and a pattern C. In addition, the memory 230 may store operation instructions corresponding to tags representing types of style a, style B, and style C, respectively.

When the image data is input to the operation module 220, various patterns of filters are applied to the image data, and the operation module 220 performs an operation using the learned optimization data and an operation instruction for the filter of the main pattern. The processor 240 selects at least one pattern a filter from the learned plurality of filters for filtering the image data, and when the selected pattern a filter is applied to the image data, the processor 240 uses pre-processing data for the pattern a filter. That is, when the filter of the pattern a is applied to the image data, the processor 240 inputs non-zero elements included in the optimization data corresponding to the filter of the pattern a to the operation module 110. In this case, the processor 240 may extract an operation instruction corresponding to the style a from the memory 230 based on the tag indicating the type of the style a. The processor 240 may input the extracted operation instruction to the operation module 220 to control a filtering operation by the filter including the pattern a to be performed.

In this case, as shown in fig. 7a, the processor 240 may rearrange the plurality of filters (learned data) of the operation module 220 to be sequentially applied to the filter for each main pattern. Specifically, referring to fig. 7b, the processor 240 selects a plurality of filters 71 of pattern a from a plurality of filters generated by learning. Since the result value obtained by performing the filtering operation is not affected by the order of the filters due to the characteristics of the convolution operation, the processor 240 may rearrange the plurality of filters such that the plurality of filters 71 of the selected pattern a are preferentially applied to the operation block 220. That is, the processor 240 may obtain an operation instruction corresponding to the pattern a from the memory 230 only once, and may collectively process a filtering operation through the plurality of filters 71 having the pattern a of the same pattern by using the obtained operation instruction. In this manner, the processor 240 may select and rearrange the plurality of filters 72 of the pattern B and the plurality of filters 73 of the pattern C from the plurality of filters sequentially applied to the filtering of the input data for each of the same patterns, respectively, obtain operation instructions corresponding to the filters 72 of the pattern B and the filters 73 of the pattern C, respectively, only once, and collectively process the filtering operation by the filters 72 of the pattern B and the filters 73 of the pattern C.

Accordingly, the processor 240 may reduce the load and the degradation of the operation speed caused by repeatedly loading the operation instructions from the memory 230.

Fig. 8 is a flowchart for describing a method for controlling a first electronic device according to an embodiment of the present disclosure.

First, an operation instruction for filtering input data of the neural network is checked for a filter of each main pattern selected from a plurality of filters generated according to learning by the neural network (S810). In this case, the filter of the main pattern may be a filter selected among the plurality of generated filters based on a statistical weight of an operation amount occupied by each pattern for each pattern in which the non-zero elements are arranged. Alternatively, the main pattern may be selected among the generated plurality of filters based on a frequency of each pattern in which non-zero elements are arranged. In this case, the plurality of filters generated by learning may differ according to a learning object of the neural network.

Thereafter, an operation instruction for filtering input data of the neural network for the filter of each selected main pattern is checked (S820). In this case, it is possible to check the optimization data from which zero elements are removed from the filter of the selected main pattern and the operation instruction corresponding to the optimization data.

Thereafter, the checked operation instruction is stored.

In an embodiment, the filtering operation by the filter of the main pattern may be performed according to the stored operation instruction.

Additionally, in another embodiment, the stored operational instructions may be sent to the second electronic device. In this case, the second electronic device may filter the input data of the neural network by using the operation instruction.

As described above, by preprocessing the filter of the main pattern occupying the main operation amount in advance, the burden on the memory is reduced and the operation efficiency is improved.

The learning method by the neural network according to the various embodiments described above may be implemented as a program and stored in various recording media. That is, a computer program that is processed by various processors and that can execute the various display methods described above may also be used in a state stored in a recording medium. As an example, a non-transitory computer-readable medium storing a program may be provided, wherein the program performs an operation of checking an operation instruction for filtering input data of a neural network for a filter of each main pattern selected from a plurality of filters learned through the neural network and an operation of storing the checked operation instruction.

A non-transitory computer readable medium is not a medium that stores data for a short time, such as a register, a cache, a memory, and the like, but refers to a machine readable medium that stores data semi-permanently. In particular, the various applications or programs described above may be stored and provided in a non-transitory computer readable medium, such as a Compact Disc (CD), a Digital Versatile Disc (DVD), a hard disk, a blu-ray disc, a Universal Serial Bus (USB), a memory card, a Read Only Memory (ROM), and the like.

Although embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the above-described specific embodiments, but various modifications may be made by those skilled in the art to which the present disclosure pertains without departing from the scope and spirit of the present disclosure as claimed in the claims. Such modifications should also be understood as falling within the technical spirit and scope of the present disclosure.