阅读说明：本技术 电磁传感器和包括其的移动设备 (Electromagnetic sensor and mobile device including the same ) 是由 李昇宰 金秀容 洪善珠 于 2019-05-24 设计创作，主要内容包括：一种电磁(EM)传感器包括：使用从外部源发送的电磁波生成EM信号的前端模块,存储用于识别所述EM信号的多个机器学习模型的一部分的传感器存储器,以及用于通过将从所述EM信号提取的特征值输入到机器学习模型来识别发射电磁波的外部电子设备的微控制器单元。如果存储在所述传感器存储器中的机器学习模型不能识别外部设备,则可以将所述特征值发送到主处理器,并且所述主处理器可以将特征值与另一组机器学习模型进行比较。(An Electromagnetic (EM) sensor comprising: the apparatus includes a front-end module generating an EM signal using an electromagnetic wave transmitted from an external source, a sensor memory storing a portion of a plurality of machine learning models for identifying the EM signal, and a microcontroller unit identifying an external electronic device emitting the electromagnetic wave by inputting a feature value extracted from the EM signal to the machine learning models. If the machine learning model stored in the sensor memory is unable to identify an external device, the feature values may be sent to a host processor, and the host processor may compare the feature values to another set of machine learning models.)

1. an electromagnetic sensor, comprising:

A front-end module configured to generate an electromagnetic signal based on an electromagnetic wave received from an external electronic device;

a sensor memory storing a portion of the plurality of machine learning models as a first machine learning model; and

A microcontroller unit configured to identify an external electronic device that emits an electromagnetic wave by inputting a feature value extracted from the electromagnetic signal to the first machine learning model.

2. The electromagnetic sensor of claim 1, wherein the microcontroller unit is further configured to transmit a feature value extracted from the electromagnetic signal to a master device of the microcontroller unit when the electromagnetic sensor fails to identify an external electronic device.

3. The electromagnetic sensor of claim 1, wherein the microcontroller unit is further configured to send information of an external electronic device that emits electromagnetic waves to a host device of the microcontroller unit when the electromagnetic sensor successfully recognizes an electromagnetic signal.

4. The electromagnetic sensor of claim 1, wherein the microcontroller unit is further configured to input a feature value extracted from the electromagnetic signal to each of the first machine learning models to obtain feature scores, and to compare each of the obtained feature scores with a predetermined reference score to identify the electromagnetic signal.

5. The electromagnetic sensor of claim 4, wherein the microcontroller unit is further configured to identify an external electronic device as a device corresponding to a machine learning model that outputs the feature score when the feature score is higher than the reference score.

6. The electromagnetic sensor of claim 4, wherein the reference scores have different values for at least two of the first machine learning models.

7. the electromagnetic sensor of claim 1, wherein the sensor memory is configured to update the first machine learning model when the cumulative number of uses is greater than a predetermined reference number.

8. The electromagnetic sensor of claim 1, wherein the microcontroller unit is further configured to update the first machine learning model stored in the sensor memory with reference to historical information comprising recognition results of the electromagnetic signals.

9. the electromagnetic sensor of claim 1, wherein the microcontroller unit is further configured to remain in an on state for a predetermined period of time regardless of an operating mode of a master device of the microcontroller unit, and to generate the electromagnetic signal using the electromagnetic wave during the predetermined period of time.

10. The electromagnetic sensor according to claim 1, wherein the microcontroller unit is configured to enter an on state from an off state based on a control command input from an external source, and to generate an electromagnetic signal using an electromagnetic wave in the on state.

11. The electromagnetic sensor of claim 10, wherein the microcontroller unit is configured to enter an on state via a control command input when a master device of the microcontroller unit is operating in a wake mode.

12. The electromagnetic sensor of claim 1, wherein the front end module comprises:

A first front-end module connected to a first antenna configured to receive a first electromagnetic wave, wherein the first front-end module generates a first electromagnetic signal using the first electromagnetic wave; and

A second front-end module connected to a second antenna configured to receive a second electromagnetic wave and separate from the first antenna, wherein the second front-end module is configured to generate a second electromagnetic signal using the second electromagnetic wave.

13. The electromagnetic sensor of claim 12, wherein the microcontroller unit is configured to calculate a difference between the first electromagnetic signal and the second electromagnetic signal to obtain a third electromagnetic signal, and apply a first machine learning model to the third electromagnetic signal to identify an external electronic device.

14. An electromagnetic sensor, comprising:

a first antenna configured to receive a first electromagnetic wave;

A second antenna configured to receive a second electromagnetic wave;

A front-end module configured to generate first and second electromagnetic signals using first and second electromagnetic waves, respectively;

A sensor memory configured to store a plurality of machine learning models; and

A microcontroller unit configured to calculate a difference between the first electromagnetic signal and the second electromagnetic signal to obtain a third electromagnetic signal, and input feature values extracted from the third electromagnetic signal to the plurality of machine learning models to identify an external electronic device.

15. An electromagnetic sensor, comprising:

A first antenna configured to receive a first electromagnetic wave;

a second antenna configured to receive a second electromagnetic wave;

A front-end module configured to generate first and second electromagnetic signals using the first and second electromagnetic waves, respectively; and

A microcontroller unit configured to calculate a difference between the first electromagnetic signal and the second electromagnetic signal to obtain a third electromagnetic signal, and to output a feature value extracted from the third electromagnetic signal.

16. A mobile device, comprising:

An electromagnetic sensor configured to extract a feature value of an electromagnetic signal corresponding to an electromagnetic wave transmitted from an external electronic device, input the feature value to a plurality of first machine learning models stored in a sensor memory included in the electromagnetic sensor, and identify the external electronic device based on the plurality of first machine learning models;

A main memory configured to store a plurality of second machine learning models, wherein at least one of the plurality of second machine learning models is not included in the plurality of first machine learning models; and

a main processor configured to receive a feature value of an electromagnetic signal from the electromagnetic sensor, input the feature value to the plurality of second machine learning models stored in the main memory, and identify an external electronic device when the electromagnetic sensor cannot identify the external electronic device based on the plurality of first machine learning models.

17. The mobile device of claim 16, wherein the main processor is further configured to receive information of an external electronic device from the electromagnetic sensor, and to execute one or more service applications associated with the external electronic device when the electromagnetic sensor successfully identifies the external electronic device.

18. the mobile device of claim 16, wherein the host processor is further configured to receive a plurality of machine learning models from an external server providing cloud services to update a plurality of second machine learning models stored in a host memory.

19. The mobile device of claim 18, wherein the plurality of machine learning models comprises at least one gaussian mixture model.

20. The mobile device of claim 16, wherein the main processor is configured to reference historical information including results of the electromagnetic sensor recognizing the electromagnetic signal, and update the first machine learning model stored in the sensor memory based at least in part on the historical information.

21. The mobile device of claim 16, wherein the electromagnetic sensor comprises a front end module configured to generate electromagnetic signals from electromagnetic waves, and wherein the front end module shares an antenna with other communication modules.

22. The mobile device of claim 16, wherein the main processor is configured to operate in a sleep mode when the electromagnetic sensor recognizes the electromagnetic signal and to enter a wake mode when the recognition procedure of the electromagnetic signal is terminated.

23. The mobile device of claim 16, wherein the electromagnetic sensor comprises a first front end module and a second front end module, and wherein the first front end module and the second front end module are disposed at different locations on a housing comprising the electromagnetic sensor, the host processor, and the host memory.

24. The mobile device of claim 23, wherein the first front end module is configured to generate a first electromagnetic signal corresponding to a first electromagnetic wave, the second front end module is configured to generate a second electromagnetic signal corresponding to a second electromagnetic wave, and

The electromagnetic sensor is configured to generate a third electromagnetic signal corresponding to a difference between the first electromagnetic signal and the second electromagnetic signal, input the third electromagnetic signal to the plurality of first machine learning models, and identify an external electronic device based on the plurality of first machine learning models.

25. The mobile device of claim 23, wherein the first front end module and the second front end module have the same structure.

Technical Field

the inventive concept relates to an EM sensor and a mobile device including the EM sensor.

Background

different electronic devices may emit different Electromagnetic (EM) waves during operation of the device based on various internal circuitry and clock signals. The EM sensor may receive an electromagnetic wave emitted by the electronic device, generate an EM signal, and extract a feature value from the EM signal. Based on these characteristic values, the EM sensors may identify and/or classify external electronic devices based on the EM radiation they emit.

In some cases, the EM sensor may communicate with a separate processor to identify the electronic device based on the extracted features. For example, the EM sensor may transmit the extracted features to a Central Processing Unit (CPU) or an Application Processor (AP) in order to identify the external device. However, this may result in increased power consumption and load on the processor.

Disclosure of Invention

An aspect of the inventive concept is to provide an EM sensor and a mobile device including the same, and the EM sensor performs a process for identifying and/or classifying electronic devices corresponding to EM signals using a machine learning model stored in the EM sensor to reduce power consumption and an operation burden of a main processor.

according to one aspect of the inventive concept, an Electromagnetic (EM) sensor includes: the apparatus includes a front-end module configured to generate an EM signal based on an electromagnetic wave received from an external electronic device, a sensor memory configured to store a portion of a plurality of machine learning models as a first machine learning model, and a microcontroller unit configured to identify the external electronic device that transmits the electromagnetic wave by inputting a feature value extracted from the EM signal to the first machine learning model.

According to one aspect of the inventive concept, an EM sensor includes: the apparatus includes a first antenna configured to receive a first electromagnetic wave, a second antenna configured to receive a second electromagnetic wave, a front-end module configured to generate a first EM signal and a second EM signal using the first electromagnetic wave and the second electromagnetic wave, respectively, a sensor memory configured to store a plurality of machine learning models, and a microcontroller unit configured to calculate a difference between the first EM signal and the second EM signal to obtain a third EM signal, and input a feature value extracted from the third EM signal to the plurality of machine learning models to identify an external electronic device.

According to one aspect of the inventive concept, an EM sensor includes: the apparatus includes a first antenna configured to receive a first electromagnetic wave, a second antenna configured to receive a second electromagnetic wave, a front end module configured to generate a first EM signal and a second EM signal using the first electromagnetic wave and the second electromagnetic wave, respectively, and a microcontroller unit configured to calculate a difference between the first EM signal and the second EM signal to obtain a third EM signal, and output a feature value extracted from the third EM signal.

According to an aspect of the inventive concept, a mobile device includes: an EM sensor configured to extract a feature value of an EM signal corresponding to an electromagnetic wave transmitted from an external electronic device, input the feature value to a plurality of first machine learning models stored in a sensor memory included in the EM sensor, and identify the external electronic device based on the plurality of first machine learning models; a main memory configured to store a plurality of second machine learning models, wherein at least one of the plurality of second machine learning models is not included in the plurality of first machine learning models; and a main processor configured to receive feature values of the EM signal from the EM sensor, input the feature values to the plurality of second machine learning models stored in the main memory, and identify the external electronic device when the EM sensor cannot identify the external electronic device based on the plurality of first machine learning models.

drawings

The above and other aspects, features and other advantages of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic perspective view illustrating a mobile device according to an example embodiment;

FIG. 2 is a diagram illustrating operation of a mobile device according to an example embodiment;

FIG. 3 is a schematic block diagram illustrating a mobile device according to an example embodiment;

FIG. 4 is a schematic block diagram illustrating an EM sensor in accordance with an example embodiment.

FIGS. 5 and 6 are flowcharts illustrating operations of an EM sensor and a mobile device according to example embodiments;

FIG. 7 is a diagram illustrating operation of an EM sensor according to an example embodiment;

FIG. 8 is a diagram illustrating operation of a mobile device according to an example embodiment;

Fig. 9 to 11 are schematic block diagrams illustrating an EM sensor according to an example embodiment;

FIG. 12 is a diagram illustrating operation of an EM sensor and a mobile device according to an example embodiment;

FIG. 13 is a diagram illustrating operation of an EM sensor according to an example embodiment;

Fig. 14 and 15 are diagrams illustrating operations of a mobile device according to example embodiments;

FIG. 16 is a schematic block diagram illustrating a mobile device according to an example embodiment; and

Fig. 17 to 20 are diagrams illustrating a service using a mobile device according to an example embodiment.

Detailed Description

Hereinafter, example embodiments of the inventive concept will be described in detail with reference to the accompanying drawings. Embodiments of the present disclosure relate to an Electromagnetic (EM) sensor including an EM sensor memory. The memory may store one or more machine learning models that are frequently used. The EM sensor may extract eigenvalues of the EM signal and process these eigenvalues using a machine learning model in the EM sensor memory.

When the EM sensor fails to identify the electronic device based on this initial processing, the EM sensor may send the feature values to a host processor that may use a second set of machine learning models (which may be stored in a main memory) to identify the source of the EM signal. Relying on a machine learning model within the EM sensor memory before sending the feature values to the host processor may result in a reduction in power consumption and burden on the host processor.

In some examples, a device may capture EM signals with multiple antennas located at different locations on the device. The signals received at the different antennas may be combined to generate a better signal (e.g., one signal may be subtracted from the other). For example, combining the signals may increase the signal-to-noise ratio, which may improve the ability of the EM sensor to identify the signal source.

Fig. 1 is a schematic perspective view illustrating a mobile device 10 according to an example embodiment.

referring to fig. 1, a mobile device 10 may include a housing 11, a display 12, a camera module 13, an input unit 14, and an EM sensor 15. The mobile device 10 according to an example embodiment is shown as a smartphone in fig. 1, and may conceptually include a variety of mobile devices such as tablets and laptops, wearable devices such as smartwatches, and general electronic devices.

The EM sensor 15 may be installed in the mobile device 10. The EM sensor 15 may include an Analog Front End (AFE) module including circuitry configured to receive electromagnetic waves in a particular frequency band and convert the electromagnetic waves into digital signals. The EM sensor 15 may also include a microcontroller unit configured to identify an external electronic device (i.e., a device that emits electromagnetic waves) using digital signals. The EM sensor 15 may also include an antenna configured to receive electromagnetic waves and convert the electromagnetic waves into analog signals. In some cases, the EM sensor 15 and the wireless communication module (e.g., in a smartphone) may share one or more antennas.

Fig. 2 is a diagram illustrating operation of the mobile device 10 according to an example embodiment.

Referring to fig. 2, a user of the mobile device 10 having the EM sensor mounted thereon may approach an electronic device 20 that emits electromagnetic waves. In some examples, the mobile device 10 may be in contact with the electronic device 20. Many electronic devices 20 emit electromagnetic waves that are generated during operation of certain component electronic circuits. For example, the electronic device 20 may emit a unique electromagnetic wave determined by an electrical signal such as a clock signal used by components of the internal circuitry.

The EM sensor of the mobile device 10 may detect the electromagnetic waves emitted by the electronic device 20 when the two devices are in proximity or contact with each other. When the mobile device 10 receives electromagnetic waves transmitted by the electronic device 20, the EM sensor of the mobile device 10 may extract features of the electromagnetic waves and input the features to a predetermined machine learning model. The predetermined learning model may be configured to identify and/or classify the electronic device 20 based on the emitted radiation. For example, the EM sensor may identify a type or model (model) name, etc. of the electronic device 20.

Fig. 3 is a schematic block diagram illustrating a mobile device 100 according to an example embodiment.

Referring to fig. 3, the mobile device 100 may include a display 110, an EM sensor 120, a memory 130, a main processor 140, and a port 150. In addition, the mobile device 100 may also include wired/wireless communication devices, power supply devices, and the like. Among the components shown in fig. 3, port 150 may be a device provided to allow mobile device 100 to communicate with external devices, such as a video card, sound card, memory card, USB device, computer, and the like. Mobile device 100 may conceptually include a smartphone, a tablet, a wearable device, a laptop computer, and various other electronic devices.

Main processor 140 may perform certain operations, commands, tasks, and the like. The main processor 140 may be provided in the form of a Central Processing Unit (CPU) or a system on a chip (SoC), and may communicate with other components included in the mobile device 100, such as the display 110, the EM sensor 120, the memory 130, and other devices connected to the port 150 via the system bus 160. In an example embodiment, the master processor 140 and the EM sensor 120 may operate in a master-slave manner.

The EM sensor 120 may include an analog front end module configured to receive electromagnetic waves transmitted from an external source and convert the electromagnetic waves into signals. The EM sensor 120 may also include a sensor memory and a microcontroller unit for processing signals. The sensor memory may be installed in the EM sensor 120, and may be separate from the memory 130 of the mobile device 100. The microcontroller unit of the EM sensor 120 may extract characteristic values of signals generated by receiving electromagnetic waves at the analog front end module. EM sensor 120 of mobile device 100 (e.g., using a microcontroller unit) and host processor 140 may both be configured to input feature values into a predetermined machine learning model to identify electronic devices that emit electromagnetic waves.

In one mode of operation, EM sensor 120 may extract features from signals generated by receiving electromagnetic waves, and host processor 140 may apply the features to a machine learning model. That is, in this mode, the main processor 140 can be operated whenever electromagnetic waves are detected from an external source. In one embodiment, the EM sensor 120 may remain on for a predetermined period of time regardless of the mode of operation of the primary processor 140, and generate EM signals using electromagnetic waves during the predetermined period of time.

To efficiently manage the battery powering the mobile device 100, the main processor 140 may alternate between a sleep mode and a wake mode. In some cases, when the EM sensor 120 detects the electromagnetic wave and extracts the characteristic value, the main processor 140 may be converted into the wake mode. However, processing the characteristic values with main processor 140 may increase the overall power consumption of mobile device 100, which may shorten the length of time that mobile device 100 may operate on battery charging.

if the user of the mobile device 100 is able to directly control the on/off state of the EM sensor 120, the user may selectively turn on the EM sensor 120 in order to detect an external device (i.e., by processing electromagnetic waves emitted by the external device). When the EM sensor 120 is turned on and detects an electromagnetic wave, the main processor 140 may be converted into an awake mode, and a machine learning model may be applied to the feature values extracted from the electromagnetic wave.

Enabling direct control of the on/off state of the EM sensor 120 may reduce overall power consumption. However, if the master processor 140 is used to apply a machine learning model to identify devices based on feature values extracted from electromagnetic waves, the master processor 140 will still consume power when the EM sensor 120 receives electromagnetic waves. In addition, applying the machine learning model will also place an operational burden on host processor 140 whenever it is used for this purpose.

Thus, in accordance with embodiments of the present disclosure, the machine learning model may be applied with the EM sensor 120 itself (e.g., using a microprocessor). That is, after receiving the electromagnetic waves and extracting the feature values, the EM sensor may apply a machine learning process to identify the electronic device that transmitted the electromagnetic waves.

The EM sensor 120 may store certain machine learning models in the sensor memory. For example, the EM sensor 120 may store those of a plurality of machine learning models used to identify electronic devices that emit electromagnetic waves that are used relatively frequently. In an example, the usage count may be saved and the model used more than a threshold number of times may be stored in a memory of the EM sensor 120. The machine learning model stored within the EM sensor 120 may be referred to herein as a first machine learning model.

Accordingly, when receiving the electromagnetic waves, the microcontroller unit of the EM sensor 120 may apply the feature values extracted from the electromagnetic waves to the first machine learning model stored in the sensor memory without intervention of the main processor 140. Therefore, power consumption and operation load of the main processor 140 can be reduced. In some cases, when EM sensor 120 identifies an electronic device based on the emitted electromagnetic waves using the first machine learning model, main processor 140 may intervene based on service implementation operations provided with respect to the electronic device.

FIG. 4 is a schematic block diagram illustrating an EM sensor 200 according to an example embodiment.

Referring to fig. 4, the EM sensor 200 may include an Analog Front End (AFE) module 210, a microcontroller unit 220, and a sensor memory 230. The sensor memory 230 may be a memory device packaged inside the EM sensor 200 and may include Static Random Access Memory (SRAM), flash memory, etc.

The analog front end module 210 may be a circuit for converting electromagnetic waves transmitted from an external source into signals, and may include an antenna, a mixer, a filter, a signal amplifier, an analog-to-digital converter, and the like. For example, the analog front end module 210 may be configured to receive electromagnetic waves in a predetermined frequency band (e.g., a frequency band below 3 MHz), and may generate EM signals based on the electromagnetic waves received within the frequency band. The signals generated by the analog front end module 210 may be sent to the microcontroller unit 220.

The microcontroller unit 220 may extract the characteristic values of the EM signal to identify and/or classify the electronic device emitting the electromagnetic wave. The machine learning model used by the microcontroller unit 220 to identify and/or classify electronic devices may be stored in the sensor memory 230. Accordingly, the microcontroller unit 220 of the EM sensor 200 may identify the type, model, etc. of the electronic device that emits the electromagnetic waves without intervention of a main processor (i.e., a main processor that is mounted on the same device as the EM sensor 200 and operates as a main device of the EM sensor 200).

in some cases (i.e., due to limitations of the storage space of sensor memory 230), some machine learning models used to identify and classify electronic devices may not be stored in sensor memory 230. In an example embodiment, a set of first machine learning models determined to have a relatively high cumulative number of uses, availability of uses, or frequency of uses may be stored in sensor memory 230 of EM sensor 200.

In some cases, the first machine learning model may be a subset of the machine learning models stored in the main memory of the mobile device on which the EM sensor 200 is installed. In some cases, the microcontroller unit 220 may select the first machine learning model stored in the sensor memory 230 based on the likelihood that the model will be used (e.g., based on usage counts, or another selection algorithm such as an algorithm that takes into account the efficiency of the machine learning model). Then, the microcontroller unit 220 may apply the first machine learning model to the eigenvalues of the EM signal generated from the electromagnetic wave.

in an example embodiment, the machine learning model may include a Gaussian Mixture Model (GMM). However, any machine learning model suitable for identifying an external device based on feature values may be used. In some examples, the algorithm for identifying the external device may be based on a static model and may not include machine learning.

In some cases, microcontroller unit 220 may not be able to identify or classify the electronic device using the first machine learning model stored in sensor memory 230. Thus, in an example embodiment, when the EM sensor 200 does not identify the electronic device, the characteristic value of the EM signal may be communicated to the host processor (i.e., to the processor operating as the host device for the EM sensor 200).

In other words, when the EM sensor 200 fails to identify the electronic device, the host processor may apply one or more machine learning models from the main memory. Therefore, the operation burden and power consumption of the main processor can be efficiently managed.

Fig. 5 and 6 are flowcharts illustrating operations of an EM sensor and a mobile device according to example embodiments.

First, the operation of the EM sensor will be described with reference to fig. 5. The operation of the EM sensor may be started by receiving the electromagnetic wave and generating the EM signal (S10). The electromagnetic waves received by the EM sensor may be electromagnetic waves emitted by an external electronic device (i.e., electromagnetic waves of a mobile device that does not include the EM sensor itself). Different electronic devices may emit different electromagnetic waves because the circuit components and clock signals may be different. Accordingly, the EM sensor may receive electromagnetic waves emitted by an external electronic device to generate an EM signal indicative of the signal source.

Then, the mobile device having the EM sensor may identify the electronic device that transmitted the electromagnetic wave using the EM signal received in S10 (S11). As described above, the electronic devices may emit different electromagnetic waves according to their types, models, components, etc., and thus the EM sensor may classify or identify the electronic devices using the EM signals. To identify the electronic device, the EM sensor may extract a feature value of the EM signal, and the feature value may be input into one or more predetermined machine learning models. For example, the machine learning model may include a gaussian mixture model.

In some systems, when the EM sensor extracts the eigenvalues of the EM signal, a host processor (i.e., a processor that is installed on the same mobile device as the EM sensor and operates as a master of the EM sensor) may apply one or more machine learning models to the eigenvalues of the EM signal. However, if only the main processor is configured to perform the identification operation, the main processor may be required to enter the wake mode whenever the EM sensor detects an electromagnetic wave from an external electronic device. Therefore, power consumption and operation burden on the main processor may increase.

In an example embodiment, to reduce power consumption and operational burden of the main processor, certain machine learning models having relatively high availability of use or cumulative number of uses may be stored in a sensor memory installed in the EM sensor. Accordingly, the first machine learning model may be stored and updated based on historical information including the recognition result based on the EM signal. The EM sensor may then apply these machine learning models to feature values extracted from electromagnetic signals received from the external device.

In other words, upon receiving the EM signal, the EM sensor may extract feature values from the EM signal and then input the feature values to one or more machine learning models stored in a sensor memory within the EM sensor. Accordingly, the EM sensor may identify the electronic device that emitted the electromagnetic waves corresponding to the EM signal without intervention of the host processor.

When the electronic device recognition is completed, a service may be performed or provided to the user according to the recognition result (S12). The service provided in S12 may be a service provided by a mobile device including an EM sensor. In some cases, S12 may be performed by a main processor of the mobile device. For example, when the external electronic device identified based on the EM signal is an air purifier, an air conditioner, or the like, an image corresponding to the external device (e.g., the concentration of fine or ultra-fine dust, a filter replacement cycle, or the current temperature and humidity) may be displayed on the screen of the mobile device.

in another example, if the identified electronic device is a TV, the mobile device may provide URL information of the video to the TV so that the TV can access the corresponding URL and play the video. In yet another example, a mobile device may initiate a service that provides initial settings of an internet of things (IoT) environment upon identifying an IoT device. Various other services may be provided based on the identified characteristics of the electronic device.

Next, operations of the EM sensor and the mobile device according to an example embodiment will be described with reference to fig. 6. Referring to fig. 6, an operation may be started by detecting an electromagnetic wave and generating an EM signal at an EM sensor (S20). The operation of S20 may be similar to the operation of S10 previously described with reference to fig. 5.

When generating the EM signal, the EM sensor may extract a feature value from the EM signal (S21). For example, the EM sensor may include an analog front end module configured to receive electromagnetic waves and convert the electromagnetic waves into EM signals. The EM sensor may also include a microcontroller unit configured to perform signal processing operations using the EM signal, and a sensor memory. The microcontroller unit may extract a characteristic value from the EM signal, e.g. a sample of a model of the EM signal in the frequency or time domain. The sensor memory may store one or more machine learning models to identify the electronic device using the eigenvalues of the EM signals. The machine learning model stored in the sensor memory may be referred to as a first machine learning model.

the microcontroller unit of the EM sensor may input the feature values of the EM signal to the first machine learning model stored in the sensor memory to calculate one or more feature scores (S22), which may be used to determine whether it is appropriate to send the feature values to the host processor. The first machine learning model used in S22 may be stored in sensor memory within the EM sensor, and thus S22 may be performed without intervention of a main processor.

The one or more feature scores calculated in S22 may represent a probability that the first machine learning model successfully identified the external device. In other words, the feature score may represent the likelihood of correctly identifying feature values extracted from the EM signal from the one or more first machine learning models.

The microcontroller unit of the EM sensor may compare the one or more feature scores with one or more reference scores corresponding to the first machine learning model (S23). The one or more reference scores applied in S23 may correspond to one or more values stored in the sensor memory with the first machine learning model, and may vary depending on which machine learning models are stored in the sensor memory.

when the one or more feature scores are less than the reference score assigned to each of the first machine learning models, it may be determined that the first machine learning model will not identify the external device that transmitted the electromagnetic signal. Accordingly, the EM sensor may transmit the characteristic value of the EM signal to a host device of the EM sensor, for example, a host processor of the mobile device (S24). The main processor may input the feature values of the EM signal to a second set of machine learning models stored in a main memory of the mobile device to identify the electronic device that transmits the electromagnetic wave corresponding to the EM signal (S25). For example, the second machine learning model may include a different machine learning model than the first machine learning model.

Accordingly, the operation of identifying and/or classifying the electronic device may be performed in a main processor of the host device, which may be an EM sensor, when it is determined that the one or more feature scores are lower than the one or more reference scores in S22. For example, the host processor may input the feature values extracted by the EM sensor to the second machine learning model stored in the host memory.

However, if there is a first machine learning model in which the corresponding feature score is higher than the reference score as a result of the comparison of S23, the EM sensor may successfully identify and/or classify the electronic device. Accordingly, operation S25 for identifying the electronic device that emits the electromagnetic wave corresponding to the EM signal may be performed by the EM sensor instead of the main processor.

When the identification and/or classification of the electronic device is completed (by the EM sensor or by the main processor), various services corresponding to the electronic device may be provided by the mobile device (S26).

Further, when the identification and/or classification of the electronic device is complete, one or more machine learning models may be updated (S27). For example, the microcontroller unit of the EM sensor may update the first machine learning model stored in the sensor memory with reference to the number of uses of the first machine learning model that successfully identified the electronic device in S25.

When the first machine learning model of the feature score higher than the reference score is not obtained in S23, the main processor may identify the electronic device using the second machine learning model stored in the main memory. In this case, the EM sensor may compare the number of uses of the second machine learning model for identifying the electronic device with the number of uses of each of the first machine learning models stored in the sensor memory to determine whether the second machine learning model should be stored in the sensor memory. Further, when one or more first machine learning models obtain feature scores higher than the respective reference scores in S23, the order of the first machine learning models may be changed with reference to the updated usage number of each of the first machine learning models.

Fig. 7 is a diagram illustrating an operation of an EM sensor according to an example embodiment.

Referring to fig. 7, an EM sensor 310 according to an example embodiment may be installed on a mobile device 300 and may operate with an application processor 320 of a main processor of the mobile device 300. For example, EM sensor 310 and application processor 320 may exchange data over a system bus of mobile device 300. Further, EM sensor 310 may operate as a slave to application processor 320, and application processor 320 may operate as a master to EM sensor 310.

The EM sensor 310 may include an analog front end module (AFE)311, a microcontroller unit (MCU)312, a sensor memory 313, and the like. The sensor memory 313 may be a memory enclosed inside the EM sensor 310 and may be distinguished from the main memory 330, which may be managed by the application processor 320.

The application processor 320 may be connected to communicate with an external server 350 providing cloud services, and the external server 350 may store a plurality of machine learning models for identifying and/or classifying electronic devices using the feature values of the EM signals. For example, the plurality of machine learning models may include a probabilistic model. The application processor 320 may periodically check the updated machine learning model among the newly stored machine learning models stored in the external server 350 during each predetermined period or in response to an operation of a user. If so, application processor 320 may download the new machine learning model and store it in main memory 330.

in an example embodiment, to reduce the power consumption and operational burden of the application processor 320, certain machine learning models having relatively high availability of use or a relatively large cumulative number of uses may be stored in the sensor memory 313 (i.e., the first machine learning model). The microcontroller unit 312 may input the characteristic value of the EM signal to the first machine learning model stored in the sensor memory 313 to identify the external device. In other words, when the analog front end module 311 receives electromagnetic waves to generate EM signals, the microcontroller unit 312 may extract the characteristic values of the EM signals and then identify and/or classify the electronic device emitting the electromagnetic waves using the first machine learning model stored in the sensor memory 313.

In the example embodiment shown in fig. 7, three first machine learning models are stored in the sensor memory 313, but this should not be considered as a limiting example. In other words, the number of first machine learning models stored in sensor memory 313 within the EM sensor 310 may be changed. The sensor memory 313 may store additional relevant information, such as reference scores, as a result of inputting the eigenvalues of the EM signals into each of the first machine learning models, which are criteria for determining the success of the identification and/or classification of the electronic device. The sensor memory 313 may also store identification times, which may represent the number of uses of each of the first machine learning models, and the types of devices that will be identified using each of the first machine learning models.

For example, the microcontroller unit 312 may extract feature values from the EM signal generated by the analog front end module 311. Further, the microcontroller unit 312 may input the feature value of the EM signal to each of the first machine learning models stored in the sensor memory. The microcontroller unit 312 may apply the first machine learning model in sequence based on which recognition times are higher. In the example embodiment shown in fig. 7, the microcontroller unit 312 may first apply the feature values extracted from the EM signal to the machine learning model a, and may compare the result with a reference score of 90% to determine whether the electronic device emitting the electromagnetic wave is device a. After inputting the feature value extracted from the EM signal to the machine learning model a, when the score is less than 90%, the microcontroller unit 312 may input the feature value to the machine learning model B.

The data stored in the sensor memory 313 may be managed by the microcontroller unit 312. For example, in the example embodiment shown in fig. 7, when the machine learning model D (which may be stored in the main memory 330 instead of the sensor memory 313) is used two or more times, the microcontroller unit 312 may replace the machine learning model C in the first machine learning model (i.e., because the machine learning model C was used only once). Information corresponding to the machine learning model D may also be written to the sensor memory 313. This method of selecting a machine learning model to be stored in the sensor memory 313 can be used to efficiently manage the memory space of the sensor memory 313.

When the first machine learning model (i.e., machine learning models a-C) stored in the sensor memory 313 does not identify the electronic device, the microcontroller unit 312 may transmit the feature value of the EM signal to the application processor 320. The application processor 320 may input the feature values of the EM signals to the second machine learning model stored in the main memory 330 so that the application processor may identify the electronic device that transmits the electromagnetic waves. The second machine learning model used by the application processor 320 may be different from the first machine learning model stored in the sensor memory 313.

Fig. 8 is a diagram illustrating operation of a mobile device according to an example embodiment.

Fig. 8 is a diagram illustrating an environment in which the mobile device 30 is used. The mobile device 30 may include an EM sensor configured to receive an electromagnetic wave to generate an EM signal and detect a characteristic value of the EM signal. In the example embodiment shown in fig. 8, a user may use mobile device 30 to gather information about television 40 and use a predetermined service.

The EM sensor included in the mobile device 30 may include: the system comprises an analog front end module configured to convert the electromagnetic wave into an EM signal as described above, a microcontroller unit configured to extract a feature value of the EM signal, and a sensor memory configured to store a machine learning model required to identify and/or classify an electronic device emitting the electromagnetic wave using the feature value of the EM signal. The analog front end module may be connected to an antenna configured to receive electromagnetic waves. In the example embodiment shown in fig. 8, the mobile device 30 may include a first antenna 31 and a second antenna 32 arranged at different locations.

If the mobile device 30 is a smartphone, the first antenna 31 and the second antenna 32 may be arranged adjacent to different ends of the smartphone. For example, the first antenna 31 may be mounted near the upper end of the display of the smartphone, while the second antenna 32 may be adjacent to the lower end of the display. At least one of the first antenna 31 and the second antenna 32 may be shared by an EM sensor included in the smart phone and the wireless communication device.

the user may allow mobile device 30 to be in close proximity or in contact with television 40, for example, in order to use a predetermined service provided by the EM sensor of mobile device 30. When the first antenna 31 is close to the television 40, the second antenna 32 may be located relatively farther from the television 40 than the first antenna 31. Thus, while the emissions from television 40 may form the majority of the electromagnetic radiation detected by first antenna 31, at second antenna 32, the proportion of the electromagnetic radiation emitted by television 40 may be relatively weak. For example, noise such as radiation emitted from peripheral electronic devices such as the refrigerator 50 may be significant in the electromagnetic radiation detected by the second antenna 32.

In an example embodiment, a first EM signal is generated from the electromagnetic wave detected by the first antenna 31, and a second EM signal is generated from the electromagnetic wave detected by the second antenna 32. The EM sensor calculates a third EM signal corresponding to a difference between the first EM signal and the second EM signal to remove an influence of noise including electromagnetic waves emitted from electronic devices other than the tv 40. Mobile device 30 may then extract the feature values of the third EM signal and input the feature values into a machine learning model to accurately identify and/or classify television 40 (i.e., the target electronic device) to provide relevant services to the user.

fig. 9 to 11 are schematic block diagrams illustrating an EM sensor according to an example embodiment.

First, referring to fig. 9, the EM sensor 400 may include a first antenna 401, a second antenna 402, a first front end module 410, a second front end module 420, a microcontroller unit 430, and the like. The first antenna 401 and the second antenna 402 may be arranged at different positions in the mobile device to be separated from each other. In some cases, the first antenna and the second antenna may be placed as far as possible in the mobile device.

The first front end module 410 may generate a first EM signal based on the electromagnetic waves detected by the first antenna 401, and the second front end module 420 may generate a second EM signal using the electromagnetic waves detected by the second antenna 402. In some examples, the first front end module 410 and the second front end module 420 may have the same structure. Accordingly, noise components generated by the operation of the front end modules 410 and 420 may be similarly generated in the first and second EM signals.

For example, since the amplitude of the electromagnetic radiation received at any given point is determined by the inverse square law, a relatively distant radiation source will result in similar amounts of radiation for front end modules 410 and 420, while the difference in the amounts of radiation received by front end modules 410 and 420 from nearby sources will be much greater.

The microcontroller unit 430 may include a Digital Signal Processor (DSP) or the like. The microcontroller unit 430 may calculate a difference between the first EM signal and the second EM signal to obtain a third EM signal. For example, the microcontroller unit 430 may obtain the third EM signal by an operation in which a signal having a smaller intensity (i.e., between the first EM signal and the second EM signal) is subtracted from a signal having a relatively higher intensity.

A user may use a mobile device with EM sensor 400 to interact with a predetermined service associated with a target electronic device that emits electromagnetic waves. First, a user may bring a mobile device in proximity to or in contact with a target electronic device. When this occurs, one of the first antenna 401 and the second antenna 402 may approach the target electronic device at a closer distance than the other. For example, when the first antenna 401 approaches the target electronic device at a closer distance, the first EM signal may include an electromagnetic wave that is stronger than an electromagnetic wave forming the second EM signal. On the other hand, the second EM signal may include a relatively weak electromagnetic wave emitted by the target electronic device. The two signals may include electromagnetic radiation of approximately equal magnitude emitted by electronic devices other than the target electronic device. Thus, the difference between the first and second EM signals may be calculated, and a third EM signal comprising an even higher signal-to-noise ratio than the first or second EM signal may be obtained.

The microcontroller unit 430 may then input the feature values of the third EM signal to a predetermined machine learning model to identify and/or classify the target electronic device. Since the proportion of the electromagnetic waves emitted by the electronic devices other than the target electronic device in the third EM signal may be relatively small, the performance of the EM sensor 400 in identifying the target electronic device may be improved.

Next, referring to fig. 10, the EM sensor 400 may include a first antenna 401, a second antenna 402, a first front end module 410, a second front end module 420, a microcontroller unit 430, and the like. The first front-end module 410 may include a matching network 411, a mixer 412, a filter 413, an amplifier 414, and an analog-to-digital converter (ADC) 415. The configuration of the first front end module 410 is not limited to that shown in fig. 10 and may vary.

The second front end module 420 may have the same or similar structure as the first front end module 410. The first front end module 410 and the second front end module 420 may generate first and second EM signals using electromagnetic waves received by the first antenna 401 and the second antenna 402, respectively. Noise generated by the operation of the front end modules 410 and 420 may be similarly reflected in each of the first and second EM signals. Accordingly, at least a portion of the noise generated by the operation of the front end modules 410 and 420 may be removed in the process of the microcontroller unit 430 calculating the difference between the first EM signal and the second EM signal to obtain the third EM signal.

After the first antenna 401 receives the electromagnetic wave and converts the electromagnetic wave into an analog signal, the frequency of the analog signal may be converted by the matching network 411 and the mixer 412. As an example, the filter 413 may be a circuit for removing a noise component, and may include a high-pass filter or a band-pass filter. Amplifier 414 may be a variable gain amplifier capable of adjusting gain and may amplify the output of filter 413 to generate the EM signal. The analog-to-digital converter 415 may convert the EM signal to a digital signal. The operation of the second front end module 420 may be similar to that described above.

The microcontroller unit 430 may receive the first and second EM signals and calculate a difference between the first and second EM signals to obtain a third EM signal. In an example embodiment, the microcontroller unit 430 may extract a characteristic value of the third EM signal. When a sensor memory is included within the EM sensor 400 and a machine learning model is stored in the sensor memory, the microcontroller unit 430 may input the feature value of the third EM signal to the machine learning model in the sensor memory to identify the electronic device emitting the electromagnetic wave. Alternatively, the microcontroller unit 430 may extract a characteristic value of the third EM signal to transmit the characteristic value to the master of the EM sensor 400. The host device may be a host processor installed on the same mobile device as the EM sensor 400. The host processor may input the feature values of the third EM signal to one or more machine learning models stored in the host memory to identify the electronic device that transmitted the electromagnetic wave.

the target electronic device identified by the microcontroller unit 430 or the main processor may be a device that emits electromagnetic waves detected by the first antenna 401 or the second antenna 402. For example, the electronic device identified by the microcontroller unit 430 or the host processor may be a device that emits electromagnetic waves corresponding to the first EM signal when the strength of the first EM signal is stronger than the strength of the second EM signal identified by the microcontroller unit 430 or the host processor. On the other hand, when the intensity of the first EM signal is weaker than that of the second EM signal, the electronic device identified by the microcontroller unit 430 or the main processor may be a device that emits an electromagnetic wave corresponding to the second EM signal.

next, referring to fig. 11, the EM sensor 500 may include a first antenna 501, a second antenna 502, a front end module 510, a microcontroller unit 520, and the like. The first antenna 501 and the second antenna 502 may be arranged at different positions in the mobile device to be separated from each other.

In the example embodiment shown in fig. 11, the electromagnetic waves detected by the first antenna 501 or the second antenna 502 may be alternately selected by the selector 503 and transmitted to the front end module 510. The front-end module 510 may convert the electromagnetic wave transmitted in the selector 503 into an analog signal. The analog signal may be signal processed and then converted to a digital signal. The digital signal may then be transmitted to the microcontroller unit 520. The configuration of the front end module 510 may be similar to that described with reference to fig. 10.

The selector 503 may sequentially transmit the electromagnetic wave detected by the first antenna 501 and the electromagnetic wave detected by the second antenna 502 to the front end module 510. Accordingly, the selector 503 can transmit the electromagnetic waves detected by both the first antenna 501 and the second antenna 502 to the front-end module 510 with a short time difference. In an example embodiment, the selector 503 may be provided as a switching circuit or a multiplexer.

fig. 12 is a diagram illustrating operations of an EM sensor and a mobile device according to an example embodiment.

Referring to fig. 12, in the operation of the EM sensor and the mobile device, the EM sensor may generate a first EM signal using a first electromagnetic wave (S30), and may generate a second EM signal using a second electromagnetic wave (S31). Aspects of S30 and S31 may be similar to the corresponding processes described with reference to fig. 9 and 10. The first antenna 401 and the first front-end module 410 may generate a first EM signal using a first electromagnetic wave, and the second antenna 402 and the second front-end module 420 may generate a second EM signal using a second electromagnetic wave.

As in the example embodiment described with reference to fig. 11, the first and second electromagnetic waves detected by each of the first and second antennas 501 and 502 may be transmitted to the front end module 510 by the selector 503 with a predetermined time difference so that the front end module 510 may generate the first and second EM signals.

The microcontroller unit may calculate a difference between the first EM signal and the second EM signal to obtain a third EM signal (S32). As in the example embodiments described with reference to fig. 9 and 10, the mobile device may include the first front end module 410 and the second front end module 420 having the same structure, or the first EM signal and the second EM signal may be generated by a single front end module 510 as in fig. 11. Accordingly, the difference between the first EM signal and the second EM signal is calculated so that noise including noise from the internal circuit such as generated by the front-end module can be removed.

The microcontroller unit may extract a feature value from the third EM signal (S33), and may calculate one or more feature scores using the feature value and one or more machine learning models (S34). In some examples, the operation of calculating the feature score may be performed by a host processor (i.e., not a microcontroller unit) of the mobile device including the EM sensor. However, when the microcontroller unit performs the operation of S34 to calculate the feature score, the operational burden and power consumption of the main processor can be reduced.

The microcontroller unit or the main processor may compare the one or more feature scores calculated in S34 with one or more reference scores (S35). The reference score may be a score assigned to each machine learning model and may be used to calculate a feature score. In some examples, the reference scores assigned to different machine learning models may be different from one another.

If there is a machine learning model having a feature score equal to or greater than the corresponding reference score in S35, the microcontroller unit or the main processor may identify the electronic device and determine its type, model name, etc. (S36). Then, the main processor may execute at least one application to provide a service suitable for the type, model, etc. of the electronic device determined in S36 (S37). Meanwhile, if there is no machine learning model having a feature score greater than the corresponding reference score in S35, it may be determined that the electronic device has not been recognized. In some cases, when the external device cannot be identified, the operation may not be started or may be terminated without providing any other service. Alternatively, the host processor may connect to a cloud service and may search and download one or more new machine learning models.

Fig. 13 is a diagram illustrating an operation of an EM sensor according to an example embodiment.

Referring to fig. 13A, the EM sensor generates a first EM signal EM1 using electromagnetic waves detected by a first antenna, and may generate a second EM signal EM2 using electromagnetic waves detected by a second antenna. Typically, there may be many sources of electromagnetic radiation (e.g., multiple electronic devices) around a mobile device having an EM sensor. Accordingly, electromagnetic waves emitted by the plurality of electronic devices may be mixed and may be received by the first antenna and the second antenna.

When the mobile device is proximate to the target electronic device (i.e., so the user may use the services provided by the EM sensor), one of the first antenna and the second antenna may be closer to the target electronic device. In the example embodiment shown in fig. 13, it is assumed that the first antenna is closer to the target electronic device than the second antenna. Thus, the intensity of first EM signal EM1 may be greater than the intensity of second EM signal EM 2.

Thus, first EM signal EM1 may include both a signal component generated by the electromagnetic waves emitted by the target electronic device and a noise component (e.g., emitted by other electronic devices). Meanwhile, second EM signal EM2 may also include a signal component and a noise component of the target electronic device. The intensity of the signal component of first EM signal EM1 may be relatively stronger than the intensity in second EM signal EM 2.

As shown in fig. 13B, the EM sensor may calculate a difference between the first EM signal EM1 and the second EM signal EM2 to obtain a third EM signal EM 3. Both first EM signal EM1 and second EM signal EM2 may include similar noise components. Thus, in third EM signal EM3, a signal component corresponding to an electromagnetic wave emitted by a target electronic device may be included at a much larger scale than a noise component (e.g., corresponding to an electromagnetic wave emitted by other electronic devices). Furthermore, as described above, the front end modules generating the first EM signal EM1 and the second EM signal EM2 may have the same structure, and thus noise of the circuit itself (i.e., generated by the operation of the front end modules) may also be removed. Accordingly, the third EM signal EM3 may have a relatively high signal-to-noise ratio compared to the first EM signal EM1 and the second EM signal EM 2.

Fig. 14 and 15 are diagrams illustrating operations of a mobile device according to example embodiments.

first, referring to fig. 14, a control window for selectively controlling various sensors and communication modules of the mobile device 600 may be disposed on the display 620. For example, a plurality of control icons 601 to 612 may be displayed on the control window, and a user may touch one or more of the plurality of control icons 601 to 612 to selectively turn on or off various sensors and communication modules. In the example embodiment shown in fig. 14, the Wi-Fi module 601 is turned on and the vibration alert module 602 is turned off. The mobile communication module 609, which is capable of sending and receiving data via the mobile network, may also be turned on, while the EM sensor 608 may be turned off.

Next, referring to fig. 15, when the EM sensor 608 is turned on (e.g., by a user's operation), the mobile device 600 may be in contact with and/or in proximity to another electronic device 700. The EM sensor 608 that has been turned on generates EM signals from electromagnetic waves emitted by the electronic device 700, and may identify and classify the electronic device 700 using one or more machine learning models stored in the sensor memory 608 embedded in the EM sensor.

If EM sensor 608 successfully identifies electronic device 700, the main processor of mobile device 600 may run a service application appropriate for electronic device 700 based on the identification. However, if EM sensor 608 fails to identify electronic device 700, EM sensor 608 may transmit a characteristic value of the EM signal to the host processor. The main processor may input the feature values to a machine learning model stored in the main memory. In other words, the identification operation of the electronic device 700 based on the EM signal is mainly performed in the EM sensor 608, and thus the operation burden and power consumption of the main processor can be reduced.

fig. 16 is a schematic block diagram illustrating a mobile device according to an example embodiment.

Referring to fig. 16, a mobile device 800 may include a sensor unit 810 and a main processor 820. The sensor unit 810 and the main processor 820 may exchange data via a system bus 830.

The sensor unit 810 may include an EM sensor 811, a motion sensor 812, and a sensor hub 813, and may further include various sensors such as an acceleration sensor, an illuminance sensor, a gravity sensor, and the like. The various sensors 811 and 812 included in the sensor unit 810 may exchange data directly through the sensor hub 813. In other words, since the sensors 811 and 812 exchange data through the sensor hub 813 rather than the data bus 830, data may be exchanged between the sensors 811 and 812 without intervention of the main processor 820.

in the example embodiment shown in fig. 16, information regarding the movement of mobile device 800 may be collected by motion sensor 812, which may be used to enable or improve the operation of EM sensor 811. For example, movements that place the mobile device 800 in proximity to other electronic devices may be registered in advance. For example, the EM sensor 811 may receive the sub-signals from the motion sensor 812 through the sensor hub 813. The sub-signals may include information regarding whether the movement of the mobile device 800 detected by the motion sensor 812 is a movement that brings the mobile device 800 into contact and/or proximity with an electronic device that emits electromagnetic waves. Then, the EM sensor 811 extracts a feature value of the EM signal and inputs the feature value to a machine learning model stored in a sensor memory within the EM sensor 811 to identify the electronic device.

for example, EM sensor 811 may generate and extract a characteristic value of an EM signal only when a movement occurs that places mobile device 800 in contact with and/or proximity to an electronic device. In other words, it may be inferred based on the movement of the mobile device 800 whether the user actually has an intent to identify the electronic device. Accordingly, the operational accuracy of the EM sensor 811 is improved, and the power consumption of the mobile device 800 can be effectively managed.

Fig. 17 to 20 are diagrams illustrating a service using a mobile device according to an example embodiment.

First, referring to fig. 17, when the mobile device 900 is in contact with and/or in proximity to the television 1000, the EM sensor of the mobile device 900 may identify the manufacturer, model, or identification number of the television 1000 using electromagnetic waves emitted by the television. The EM sensor of mobile device 900 may be automatically switched to an on mode when mobile device 900 is proximate to television 1000, or may be manually turned on by a user.

If the mobile device 900 successfully identifies the television 1000, various services may be provided (e.g., after a mutual authentication process using account information assigned to each of the television 1000 and the mobile device 900). For example, as shown in fig. 17, the television 1000 receives a URL address of a video played in the mobile device 900 and plays the video. Alternatively, information about the power that the television 1000 consumes during a particular period of time may be displayed on the mobile device 900. Alternatively, the mobile device 900 may receive identification information of the television 1000 and use the identification information to set up an internet of things (IoT) environment.

Next, in the example embodiment shown in fig. 18, mobile device 900 may identify air purifier 1100 using electromagnetic waves emitted by air purifier 1100. Mobile device 900 may then display the value of fine dust and/or ultra-fine dust detected by the dust sensor embedded in air purifier 1100. Alternatively, in a manner similar to the example embodiment shown in fig. 17, the mobile device may receive identification information of air purifier 1100 and use the identification information to set up the IoT environment, or, for example, order filter changes for air purifier 1100.

referring to fig. 19, the mobile device 900 may identify the air conditioner 1200 using electromagnetic waves emitted by the air conditioner 1200. Subsequently, the mobile device 900 may start a replacement cycle or a cleaning cycle of a filter included in the air conditioner 1200, or may collect and display power consumption of the air conditioner 1200.

Referring to fig. 20, an electronic device identified by the mobile device 900 using electromagnetic waves may be a refrigerator 1300. When the EM sensor of the mobile device 900 converts the electromagnetic waves from the refrigerator 1300 into the EM signal, the main processor or the microcontroller of the EM sensor may identify the refrigerator 1300 using the characteristic value of the EM signal. The mobile device 900 may then receive the identification information of the refrigerator 1300 in order to run an application or set an IoT environment.

As described above, according to example embodiments of the inventive concepts, an EM sensor receives an electromagnetic wave to generate an EM signal, and then the EM sensor may perform a process for identifying and/or classifying an electronic device that transmits the electromagnetic wave. The electronic device may be identified using a machine learning model stored in a memory within the EM sensor and the eigenvalues of the EM signals. The EM sensor may transmit the feature values of the EM signals to the host processor only when the EM sensor fails to identify and/or classify the electronic device, and the host processor may input the feature values to the machine learning model in the host memory. Therefore, power consumption and operation load of the main processor can be reduced.

While example embodiments have been shown and described above, it will be apparent to those skilled in the art that modifications and variations can be made without departing from the scope of the disclosure as defined by the appended claims.