阅读说明：本技术 移动设备和通过移动设备配对电子设备的方法 (Mobile device and method for pairing electronic devices through mobile device ) 是由 高在佑 李宰荣 金铉中 于 2015-07-29 设计创作，主要内容包括：一种通过移动设备配对电子设备的方法,所述方法包括：捕获包括至少一个电子设备的图像；经由所述移动设备的显示器从所述移动设备的用户接收输入,所述输入与所述至少一个电子设备和所述移动设备之间的配对相关联；基于经由所述移动设备的显示器的输入,选择所述至少一个电子设备进行配对；以及控制所述至少一个电子设备和所述移动设备相互配对。(A method of pairing electronic devices by a mobile device, the method comprising: capturing an image including at least one electronic device; receiving an input from a user of the mobile device via a display of the mobile device, the input being associated with a pairing between the at least one electronic device and the mobile device; selecting the at least one electronic device for pairing based on input via a display of the mobile device; and controlling the at least one electronic device and the mobile device to pair with each other.)

1. A method of pairing electronic devices by a mobile device, the method comprising:

capturing an image including at least one electronic device;

receiving an input from a user of the mobile device via a display of the mobile device, the input being associated with a pairing between the at least one electronic device and the mobile device;

selecting the at least one electronic device for pairing based on input via a display of the mobile device; and

controlling the at least one electronic device and the mobile device to pair with each other.

2. The method of claim 1, further comprising:

analyzing whether the at least one electronic device is included in the captured image.

3. The method of claim 2, wherein the analyzing comprises: when the input is a gesture input by a user of the mobile device, analyzing whether the at least one electronic device is included in the captured image.

4. The method of claim 1, further comprising:

displaying an image including the at least one electronic device on a display of the mobile device.

5. The method of claim 1, wherein the mobile device comprises glasses.

6. The method of claim 1, wherein the capturing of the image comprises capturing the image based on a direction of an eye of a user of the mobile device.

7. The method of claim 6, wherein controlling the at least one electronic device and the mobile device to pair with each other comprises: controlling the at least one electronic device and the mobile device to pair with each other when the input is a direction of an eye of a user of the mobile device and the input remains for a specified period of time.

8. The method of claim 1, further comprising:

sending a query to a user of the mobile device whether to control at least one electronic device,

wherein the receiving of the input comprises receiving an input made in response to the query.

9. A mobile device, comprising:

a camera that captures an image including at least one electronic device;

a display; and

a processor configured to:

receiving input from a user via the display, the input being associated with a pairing with the at least one electronic device,

determining that the at least one electronic device is for pairing based on input via the display, and

controlling the at least one electronic device to pair with the mobile device.

10. The mobile device of claim 9, wherein the processor analyzes whether the at least one electronic device is included in the captured image.

11. The mobile device of claim 10, wherein the analyzing comprises: analyzing whether the at least one electronic device is included in the captured image when the input is a gesture input of the user.

12. The mobile device of claim 9, wherein the display displays an image comprising the at least one electronic device.

13. The mobile device of claim 9, wherein the mobile device comprises glasses.

14. The mobile device of claim 9, the capturing of the image comprising capturing the image based on a direction of an eye of a user.

15. The mobile device of claim 14, further comprising an eye tracking camera that tracks a direction of an eye of the user.

16. The mobile device of claim 14, wherein controlling the at least one electronic device to pair with the mobile device comprises: controlling the at least one electronic device and the mobile device to pair when the input is a direction of an eye of a user of the mobile device and the input remains for a specified period of time.

17. The mobile device as set forth in claim 9,

wherein the processor sends a query to a user whether to control the at least one electronic device,

wherein the receiving of the input comprises receiving an input made in response to the query.

18. A non-transitory computer-readable storage medium storing instructions thereon that, when executed, cause at least one processor to perform a method comprising:

capturing an image including at least one electronic device;

receiving an input from a user of a mobile device via a display of the mobile device, the input being associated with a pairing between the at least one electronic device and the mobile device;

selecting the at least one electronic device for pairing based on input via a display of the mobile device; and

controlling the at least one electronic device and the mobile device to pair with each other.

Technical Field

One or more embodiments relate to a method and apparatus for pairing with an electronic device.

Background

As the technology is developed, electronic devices are miniaturized and have various shapes, wearable devices to be worn on the body are being developed. For example, smart watches or smart glasses are being developed. A smart watch generally refers to an embedded system wristwatch having more improved functions than a general wristwatch. Smart glasses generally refer to wearable computers that include a Head Mounted Display (HMD). Wearable devices can be classified into stand-alone devices and interlocking devices. The stand-alone device includes: an input/output unit, a processing unit, a storage unit, and a communication unit, and may be used alone. The interlock device refers to a device that can be used after interlocking with a separate device such as a smart phone.

Disclosure of Invention

One or more exemplary embodiments include a method of pairing with an electronic device included in an input image.

Other aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented exemplary embodiments.

According to one or more exemplary embodiments, an eyewear to be paired with an electronic device, the eyewear comprising: a memory that stores identification information and device information about each of a plurality of electronic devices, the device information representing information for pairing the glasses with the plurality of electronic devices; an image acquisition unit that acquires an image of at least one electronic device via a camera included in the glasses; an identification information extraction unit that extracts identification information about an electronic device included in the acquired image; a pairing target selection unit that searches for the stored identification information and selects an electronic device corresponding to the extracted identification information; and a wireless communication unit pairing with the selected electronic device based on the device information of the selected electronic device.

According to one or more exemplary embodiments, a method performed by glasses for pairing with an electronic device, the method comprising: storing identification information and device information about each of a plurality of electronic devices, the device information representing information for pairing the glasses with the plurality of electronic devices; acquiring an image of at least one electronic device via a camera included in the glasses; extracting identification information about an electronic device included in the acquired image; searching for the stored identification information and selecting an electronic device corresponding to the extracted identification information; and pairing with the selected electronic device based on the device information of the selected electronic device.

According to one or more exemplary embodiments, a method of pairing electronic devices by a mobile device, the method comprising: capturing an image including at least one electronic device; receiving an input from a user of the mobile device via a display of the mobile device, the input being associated with a pairing between the at least one electronic device and the mobile device; selecting the at least one electronic device for pairing based on input via a display of the mobile device; and controlling the at least one electronic device and the mobile device to pair with each other.

According to one or more exemplary embodiments, a mobile device includes: a camera that captures an image including at least one electronic device; a display; and a processor configured to: receiving input from a user via the display, the input being associated with a pairing with the at least one electronic device, determining that the at least one electronic device is for pairing based on the input via the display, and controlling the at least one electronic device to pair with the mobile device.

Drawings

These and/or other aspects will become apparent and apparent from the following description of exemplary embodiments, taken in conjunction with the accompanying drawings, in which:

fig. 1 is a diagram for describing a method of pairing a mobile device with an electronic device according to an embodiment;

fig. 2 is a diagram for describing a method of pairing a mobile device with a remote controller and controlling the remote controller;

fig. 3 is a flowchart for describing a method of a mobile device controlling an electronic device according to an embodiment;

fig. 4 is a diagram for describing a method of a mobile device directly controlling a TV according to an embodiment;

fig. 5 is a flowchart for describing a method of a mobile device controlling an electronic device according to an embodiment;

fig. 6 is a diagram showing a configuration of a mobile device according to an embodiment;

fig. 7 is a diagram showing a configuration of a mobile device according to an embodiment;

fig. 8 is a diagram of a method for smart glasses to acquire an image of an electronic device, according to an embodiment;

fig. 9 is a diagram showing a configuration of smart glasses according to an embodiment;

fig. 10 is a diagram for describing a pairing method according to an embodiment;

fig. 11 is a diagram for describing a pairing method according to an embodiment;

fig. 12 is a diagram for describing a pairing method according to an embodiment;

fig. 13 is a diagram for describing a pairing method according to the embodiment;

fig. 14A is a diagram for describing a method in which a mobile device pairs with an electronic device by broadcasting a packet according to an embodiment;

fig. 14B is a diagram for describing a method in which an electronic device pairs with the electronic device by broadcasting a packet according to the embodiment;

fig. 15 is a diagram for describing a method in which a mobile device performs pairing via a broadcast packet according to an embodiment;

fig. 16 is a diagram for describing a method of a mobile device controlling an electronic device by using an input unit according to an embodiment.

Fig. 17 is a diagram for describing a method of a mobile device controlling an electronic device by using an application according to an embodiment;

fig. 18 is a diagram for describing a method in which a mobile device controls an electronic device by providing a screen image displayed on the electronic device according to an embodiment;

fig. 19 is a diagram for describing a method of a mobile device controlling an audio system according to an embodiment;

fig. 20 is a diagram for describing a method of a mobile device controlling an air conditioner according to an embodiment;

fig. 21 is a diagram for describing a method of a mobile device controlling a laptop PC according to an embodiment;

fig. 22 is a diagram for describing a method of a mobile device controlling a refrigerator according to an embodiment;

fig. 23 is a diagram for describing a method of a mobile device controlling a wearable device according to an embodiment;

fig. 24 is a diagram for describing a method of a mobile device controlling the mobile device according to the embodiment;

fig. 25 shows a case where the display unit displays the same screen image as that displayed at another mobile device;

fig. 26A and 26B are diagrams for describing a method in which a mobile device pairs with an electronic device using an image of the electronic device;

FIG. 27 is a diagram for describing a method by which a mobile device is paired with an electronic device via text;

fig. 28 is a diagram for describing an operation of causing a mobile device according to the embodiment to register an electronic device;

fig. 29 is a diagram for describing an operation of causing a mobile device according to the embodiment to register an application;

fig. 30 is a diagram for describing an operation of causing a mobile device according to the embodiment to register an application;

fig. 31 is a diagram for explaining an operation of causing a mobile device according to the embodiment to register an application;

fig. 32 is a diagram for explaining an operation of causing a mobile device according to the embodiment to register a control function;

fig. 33 is a diagram for describing an operation of pairing a mobile device with an electronic device according to the embodiment;

fig. 34 is a diagram for describing an operation of causing a mobile device according to an embodiment to identify an object and provide a related function;

fig. 35 is a diagram for describing an operation of causing a mobile device according to the embodiment to make a phone call to an identified person;

fig. 36 is a diagram for describing an operation of causing a mobile device according to the embodiment to transmit a message to an identified person;

fig. 37 is a diagram for describing an operation of causing a mobile device according to the embodiment to execute an application corresponding to an identified person;

fig. 38 is a diagram for describing a method in which a mobile device performs a function registered with respect to an identified object according to an embodiment;

fig. 39 is a diagram for describing a method of a mobile device controlling a legacy device according to an embodiment;

fig. 40 is a diagram for describing a method of controlling an electronic device by glasses according to an embodiment;

fig. 41 is a flowchart for describing a method of seamlessly playing back content according to an embodiment;

fig. 42 is a flowchart for describing a method of seamlessly playing back content according to an embodiment;

fig. 43 is a diagram for explaining an electronic apparatus according to the embodiment;

fig. 44 is a diagram for describing a seamless playback method according to the embodiment;

FIG. 45 is a diagram for describing an embodiment for utilizing user-customized bookmark information when there are multiple users;

fig. 46 is a diagram for explaining eyeglasses according to the embodiment;

fig. 47 is a flowchart for describing a method of seamlessly playing back content according to an embodiment;

FIG. 48 is a flow chart for describing a method of pairing electronic devices according to an embodiment;

fig. 49 is a diagram for describing a pairing method and a seamless content playback method according to an embodiment;

FIG. 50 is a flow chart describing a method in which a smart phone is paired with a TV and the smart phone delivers content to the TV;

fig. 51A and 51B are diagrams for describing a method of determining that an electronic device plays back content seamlessly;

fig. 52A and 52B are diagrams for describing a method of determining that an electronic device plays back content seamlessly;

FIG. 53 is a flow chart describing a method by which a smart phone is paired with a TV and the smart phone delivers content to the TV;

fig. 54 is a diagram for describing a seamless playback method using bookmarks;

fig. 55 is a diagram for describing a seamless playback method with respect to a plurality of users; and

fig. 56 is a diagram for describing an example of an electronic device that receives content from a nearby electronic device and plays back the content.

Detailed Description

While the principles of the invention are susceptible to various modifications and embodiments, specific embodiments have been described in detail in the written description and are illustrated in the accompanying drawings. However, it is not intended to limit the principle of the present invention to a specific mode of practice, and it is to be understood that all changes, equivalents, and substitutions that do not depart from the spirit and technical scope of the principle of the present invention are included in the principle of the present invention. In describing the principles of the present invention, specific details of the related art are omitted when it is considered that they may unnecessarily obscure the essence of the principles of the present invention.

Although terms such as "first," "second," etc. may be used to describe various components, such components are not necessarily limited to the above terms. The above terms are only used to distinguish one component from another component.

The terminology used in the description is for the purpose of describing particular embodiments only and is not intended to be limiting of the principles of the invention. The use of the singular expression includes the plural unless it is clearly different in context. In this specification, it will be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the specification, and are not intended to preclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof may be present or may be added.

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements. In this regard, the present exemplary embodiment may have different forms and should not be construed as being limited to the description set forth herein. Accordingly, the exemplary embodiments are described below in order to explain aspects of the present specification by referring to the figures. Expressions such as "at least one" modify an entire list of elements in a series of elements when directing the series, and do not modify individual elements in the list.

Fig. 1 is a diagram for describing a method of pairing a mobile device 100 with an electronic device according to an embodiment. Referring to fig. 1, a user may pair a mobile device 1000 with nearby electronic devices 210 to 230. For example, the mobile device 1000 may be a device that may identify nearby electronic devices, such as the smart glasses 110, the smart camera 120, the wearable device 130, and the smart phone 140, and may wirelessly connect to the nearby devices.

The mobile device 1000 identifies an electronic device included in an input image and pairs with the identified electronic device. The identification of the electronic device may be determining whether the input image includes an operation of the electronic device. Further, the identification of the electronic device may be an operation of determining the type of the electronic device included in the input image. Further, the identification of the electronic device may be an operation of determining a name of the electronic device included in the input image.

Electronic devices can include both devices that can communicate directly with the mobile device 1000 and legacy devices that cannot communicate directly with the mobile device 1000. The legacy device may communicate with the mobile device 1000 via a patch (patch). A process of causing the mobile device 1000 to control the conventional device via the patch will be described in detail below with reference to fig. 39.

Pairing refers to an operation for connecting the mobile device 1000 to an electronic device via wire or wirelessly. Further, pairing refers to an operation of establishing settings of the mobile device 1000 to transmit and receive data to and from an electronic device via wire or wirelessly.

The mobile device 1000 stores identification information for identifying a plurality of electronic devices. The identification information may be information unique to the electronic device that distinguishes the corresponding electronic device from other electronic devices. For example, the identification information includes information about the appearance of the electronic device, such as the shape, feature points, and images of the electronic device. Further, the identification information may include text indicating the name of the electronic device, i.e., the type or model name of the electronic device. Further, the identification information may include audio acquired when a person or machine reads the name of the electronic device.

The mobile device 1000 may identify a plurality of electronic devices by scanning a prearranged code. The prearranged code may be data indicative of a technology for storing information, such as a barcode, a QR code, or a color code. The barcode stores information by using a combination of vertical lines having different thicknesses. The QR code is a two-dimensional code in which information is stored as a square lattice-like pattern. The color code is data indicating a technique for storing information by using a combination of fixed colors. For example, the color code may be printed in a matrix-like shape by using colors including red, green, and blue. The code may be printed on the outside of the electronic device and may include information including a name, a type, or a serial code of the electronic device. The mobile device 1000 may scan a barcode, a QR code, or a color code by using a camera and identify an electronic device printed with a code based on information read therefrom. Although a barcode, a QR code, and a color code are exemplified above, any of various codes may be used to identify an electronic device as long as the code can be scanned by the mobile device 1000 to read information therefrom. The mobile device 1000 may identify different features of the respective electronic devices, generate identification information based thereon, and store the generated identification information in a memory. The mobile device 1000 may obtain the average characteristics for each type of electronic device and store the average characteristics in memory. The mobile device 1000 may store the name, features, and shape of the electronic device and an identification number of the electronic device.

The mobile device 1000 may store a portion of the appearance of the electronic device as a feature point. For example, the mobile device 1000 may extract an edge from the appearance of the electronic device and store the extracted edge as identification information about the electronic device. For example, when the mobile device 1000 extracts features of a camera, the mobile device 1000 may extract only edges of the camera and store the extracted edges as identification information about the camera, without storing information about the entire appearance of the camera as the identification information. The camera includes a rectangular body and a circular lens in the body. The mobile device 1000 extracts the edges of the subject and the lens and stores the edges as identification information about the camera.

The mobile device 1000 may store not only edges, but also various other features that may be extracted from the appearance of the electronic device and store the features as identification information about the electronic device. Further, in order for the mobile device 1000 to analyze the image of the electronic device and the extracted features, various techniques including the above-described method may be applied.

The mobile device 1000 recognizes an electronic device included in an input image, extracts identification information about the electronic device, and selects identification information corresponding to the extracted identification information from among the identification information stored in the mobile device 1000. Selecting identification information corresponding to the extracted identification information may refer to searching for identification information that is the same as the extracted identification information to a set percentage or more. The extracted identification information is information on an electronic device acquired from a current input image, and the identification information to be searched is information on an electronic device previously stored in the mobile device 1000.

The mobile device 1000 pairs with the selected electronic device by using device information about the selected electronic device. The device information refers to information required to pair the mobile device 1000 and the electronic device with each other. For example, the device information may be a MAC address or an IP address of the electronic device. The mobile device 1000 pairs with the electronic device by using a MAC address or an IP address. Further, the device information may also be transmitted from a server (not shown).

Because legacy devices cannot be paired directly with the mobile device 1000, patches attached to legacy devices can be paired with the mobile device 1000.

After pairing, the mobile device 1000 may control the electronic device. For example, the term "pairing" may indicate a state in which the mobile device 1000 may transfer data to the electronic device. Controlling the electronic device may mean that the electronic device operates according to data transferred from the mobile device 1000. Fig. 1 shows a case where a mobile device 1000 is paired with an electronic device such as a remote controller 210, an audio system 220, and a smartphone 230 or a conventional device such as a refrigerator 235 attached to a patch 235 a. Here, the refrigerator 235 is a conventional device.

The mobile device 1000 may control the paired electronic devices by using an input device of the mobile device 1000. For example, the mobile device 1000 may adjust the channel and volume of the paired electronic device by using an input device of the mobile device 1000. The input device refers to a device capable of receiving an instruction of a user. For example, the input device may be a button, a touch panel, a camera, or a mouse.

A screen image for controlling the paired electronic device may be displayed at the display unit 1400 of the mobile device 1000. The screen image for controlling the paired electronic device may be a screen image displaying functions included in the electronic device. Alternatively, the screen image for controlling the paired electronic device may be a screen image for transmitting a control signal from the mobile device 1000 to the electronic device.

The mobile device 1000 may display various types of control screen images according to the recognized type of electronic device. The types of electronic devices refer to various groups of electronic devices for performing different functions, such as mobile devices, wearable devices, remote controls, home appliances, and the like. In fig. 1, a screen image for adjusting a channel or volume is displayed at the display unit 1400 of the mobile device 1000. Accordingly, a button for changing a channel and a button for changing a volume may be displayed at the display unit 1400. The mobile device 1000 receives a user input and determines which button the user input selected. The mobile device 1000 transmits a control signal corresponding to the selected button to the electronic device. For example, the mobile device 1000 may receive an input of a user and output a control signal for adjusting a channel or volume to the remote controller 210, the audio system 220, or the smartphone 230.

The audio system 220 or the smartphone 230 may increase the volume of the song currently being played back or change the song to be played back according to a control signal received from the mobile device 1000.

Based on the state set by the user, the mobile device 1000 may or may not recognize the electronic device included in the input image and pair with the recognized electronic device. A User Interface (UI) element for activating or deactivating a corresponding function may be displayed at the display unit 1400 of the mobile device 1000. For example, a button or a check box for activating or deactivating a corresponding function may be displayed at the display unit 1400 of the mobile device 1000. The user may identify and pair with the identified electronic device by touching a button or selecting a check box.

Based on the power remaining in the battery, the mobile device 1000 may or may not identify the electronic device included in the input image and pair with the identified electronic device. The mobile device 1000 may check the power remaining in the battery, and if the power is less than or equal to a preset value, the mobile device 1000 may not perform a corresponding function. For example, if the preset value is 10% of the power remaining in the battery, the mobile device 1000 may not recognize the electronic device included in the input image. Alternatively, the mobile device 1000 may not be paired with an identified electronic device. If more than 10% of the power remains in the battery, the mobile device 1000 may identify and pair with the electronic device included in the input image.

Fig. 2 is a diagram for describing a method in which the mobile device 1000 is paired with the remote controller 210 and controls the remote controller 210. The remote controller 210 controls other electronic devices, such as the TV240, according to control signals received from the mobile device 1000.

The mobile device 1000 acquires an image of the remote controller 210 via the camera. The acquiring of the image may comprise acquiring at least one still image. The mobile device 1000 is paired with the remote control 210. The mobile device 1000 outputs a control signal for adjusting a channel or volume to the remote controller 210. The remote controller 210 receives a control signal from the mobile device 1000 and outputs the control signal to the TV 240. The TV240 operates according to the received control signal.

Fig. 3 is a flowchart for describing a mobile device 1000 controlling an electronic device according to an embodiment. Referring to fig. 3, the mobile device 1000 may control an electronic device after pairing it with the electronic device.

In operation 310, the mobile device 1000 stores identification information and device information about the electronic device. The identification information includes information for identifying an appearance of the electronic device, and the device information includes information for establishing a connection between the mobile device 1000 and the electronic device. The mobile device 1000 may acquire an image of an electronic device via a camera and extract identification information about the electronic device included in the acquired image. The mobile device 1000 stores the extracted identification information. The mobile device 1000 may receive device information from a user. The mobile device 1000 may store device information by receiving information about the electronic device such as a MAC address and capability information. The capability information may be information about functions that may be performed by the electronic device. For example, the capability information is information indicating whether the electronic device is capable of playing back sound or whether the electronic device is capable of displaying an image. Alternatively, the mobile device 1000 may receive device information from a nearby electronic device via wire or wirelessly and store the device information.

In operation 320, the mobile device 1000 acquires an image of the electronic device via the camera. The camera of the mobile device 1000 acquires an image of the electronic device.

In operation 330, the mobile device 1000 extracts identification information about an electronic device included in the acquired image. The mobile device 1000 determines whether an image includes an electronic device by analyzing the image, and extracts identification information about the electronic device.

In operation 340, the mobile device 1000 searches for identification information corresponding to the extracted identification information from among the identification information stored in the mobile device 1000. The term identification information corresponding to the extracted identification information refers to identification information that is the same as the extracted identification information to a set percentage or higher.

In operation 350, the mobile device 1000 selects an electronic device to pair. The mobile device 1000 searches for identification information corresponding to the extracted identification information and selects an electronic device corresponding to the searched identification information.

In operation 360, the mobile device 1000 is paired with the selected electronic device.

In operation 370, the mobile device 1000 receives an input from a user. The mobile device 1000 may receive input from a user via buttons, a touch panel, or a camera.

In operation 380, the mobile device 1000 controls the electronic device based on the user input. The mobile device 1000 transmits a control signal corresponding to a user input to the electronic device.

Fig. 4 is a diagram for describing a method of the mobile device 1000 directly controlling the TV240 according to an embodiment. The mobile device 1000 may control the TV240 via the remote controller 210 or directly control the TV 240.

The mobile device 1000 acquires an image including the remote controller 210 via a camera. The mobile device 1000 recognizes the remote controller 210 included in the image and controls the object controlled by the recognized remote controller 210. The mobile device 1000 may determine the target controlled by the remote controller 210 based on the identification information about the remote controller 210. For example, because the remote controller 210 may control electronic devices such as the TV240 and the audio system 220, the mobile device 1000 determines an electronic device controlled by the remote controller 210 based on identification information about the remote controller 210 and selects a target controlled by the remote controller 210.

The embodiment shown in fig. 4 relates to a case where the target controlled by the remote controller 210 is the TV 240. The mobile device 1000 may select an object controlled by the remote controller 210 by recognizing the shape of the remote controller 210 and buttons included in the remote controller 210. Alternatively, the mobile device 1000 may receive information about an object controlled by the remote controller 210 from the remote controller 210. The mobile device 1000 may receive information about the TV240 from the remote controller 210 and pair with the TV 240.

The mobile device 1000 directly controls the TV 240. If the mobile device 1000 is capable of outputting a signal for controlling the TV240, the mobile device 1000 directly transmits the control signal to the TV 240. For example, the mobile device 1000 may transmit a signal for changing a channel or a signal for changing a volume to the TV 240.

Fig. 5 is a flowchart for describing a method of a mobile device controlling an electronic device according to an embodiment. Referring to fig. 5, a mobile device 1000 may control a B-electronic device by pairing with the B-electronic device.

In operation 510, the mobile device 1000 stores identification information and device information about the electronic device.

In operation 520, the mobile device 1000 acquires an image of the A-electronic device via the camera. For example, a camera of the mobile device 1000 acquires an image of an A-electronic device.

In operation 530, the mobile device 1000 extracts identification information about an A-electronic device included in the image. The mobile device 1000 analyzes the image, determines whether an a-electronic device is included in the image, and extracts identification information about the a-electronic device.

In operation 540, the mobile device 1000 searches for identification information corresponding to the extracted identification information from among the identification information stored in the memory.

In operation 550, the mobile device 1000 selects an electronic device to pair. The mobile device 1000 selects a B-electronic device controlled by the a-electronic device as the electronic device to be paired. If the mobile device 1000 is capable of directly controlling a B-electronic device, the mobile device 1000 selects the B-electronic device as the electronic device to be paired. If the mobile device 1000 cannot directly control the B-electronic device, the mobile device 1000 controls the B-electronic device by using the A-electronic device.

In operation 560, the mobile device 1000 is paired with a B-electronic device.

In operation 570, the mobile device 1000 receives an input from a user. For example, if the B-electronic device is the TV240, the mobile device 1000 may receive an input for changing a channel or changing a volume from a user.

In operation 580, the mobile device 1000 controls the B-electronic device based on an input from a user. The mobile device 1000 transmits a control signal corresponding to an input from a user to the B electronic device.

Fig. 6 is a diagram showing a configuration of a mobile device 1000 according to the embodiment. Referring to fig. 6, the mobile device 1000 includes a processor 1100, a camera 1200, a memory 1300, and a display unit 1400.

The processor 1100 analyzes the image received from the camera 1200 and identifies the electronic device. The processor 1100 receives an image from the camera 1200 and identifies an electronic device included in the image. The processor 1100 may recognize the shapes of the objects included in the image and select an object having the same shape as the shape of the electronic device from among the recognized objects.

The processor 1100 searches for identification information corresponding to the identification information extracted from the image from among the identification information stored in the memory 1300.

The processor 1100 is paired with the identified electronic device.

The processor 1100 displays an interface for controlling the electronic device at the display unit 1400. When the mobile device 1000 is connected to the identified electronic device, the processor 1100 displays an interface for receiving an input from a user at the display unit 1400. The processor 1100 transmits a control signal corresponding to an input of the user to the identified electronic device.

The processor 1100 may change the function of an input unit of the mobile device 1000. For example, if the input unit performs a function for changing the brightness of the display unit 1400, the processor 1100 may change the input unit to perform a function for changing the volume of the electronic device. Accordingly, the processor 1100 can perform different functions even if the same input is received from the user via the input unit.

The camera 1200 acquires an image and outputs the image to the processor 110. The mobile device 1000 includes at least one camera 1200. The camera 1200 may be disposed at a side surface, a front surface, or a rear surface of the mobile device 1000.

The memory 1300 stores identification information and device information about a plurality of electronic devices, wherein the device information represents information for pairing with the respective electronic devices.

The display unit 1400 displays a screen image for controlling the electronic device under the control of the processor 1100. The display unit 1400 may be a touch panel for receiving an input from a user.

Fig. 7 is a diagram showing a configuration of a mobile device 1000 according to the embodiment. Referring to fig. 7, the processor 1100 of the mobile device 1000 includes an image acquisition unit 1110, an identification information extraction unit 1120, a pairing target selection unit 1130, and a wireless communication unit 1140.

The image acquisition unit 1110 acquires an image of at least one electronic device from the camera 1200 of the mobile device 1000. The image acquisition unit 1110 outputs the acquired image to the identification information extraction unit 1120.

The image acquisition unit 1110 may extract edges included in the image acquired by the camera 1200, analyze the shape of the edges, and recognize the electronic device. Since the shapes of the edges of the electrical devices may differ from each other, the electronic devices included in the image may be identified by comparing the extracted shape of the edge with the stored shapes of the edges.

If a fixed image is input via the camera 1200 for a set period of time, the image acquisition unit 1110 may identify an electronic device included in the input image and acquire an image of the electronic device. Further, if an image acquired by the camera 1200 is input, the image acquisition unit 1110 may identify an electronic device included in the image and acquire the image of the electronic device.

The identification information extraction unit 1120 extracts identification information about the electronic device included in the image. The identification information extraction unit 1120 may analyze the image and recognize an object having a shape of the electronic device. For example, the identification information extraction unit 1120 may extract information indicating a boundary of the object as the identification information, or may extract information indicating a polygon and text included in the object as the identification information.

The pairing-target selecting unit 1130 searches for identification information corresponding to the identification information extracted from the image from among the identification information stored in the memory, and selects an electronic apparatus corresponding to the searched identification information as an electronic apparatus to be paired.

The wireless communication unit 1140 pairs with the selected electronic device by using the stored device information about the selected electronic device.

Fig. 8 is a diagram for describing a method of the smart glasses 110 acquiring an image of an electronic device according to an embodiment. Referring to fig. 8, the smart glasses 110 search for an electronic device located in a direction in which a user views. Fig. 8 shows a case where the remote controller 210 is located around the user.

The eye tracking camera 113 tracks the user's eyes and determines the direction in which the user is looking.

The front camera 112 acquires an image in the direction tracked by the eye tracking camera 113. In other words, the front camera 112 acquires an image from the same viewpoint as the user. Fig. 8 shows a case where the front camera 112 acquires an image of the remote controller 210.

Fig. 9 is a diagram illustrating a configuration of the smart glasses 110 according to an embodiment. Referring to fig. 9, the smart glasses 110 include a front camera 112, an eye tracking camera 113, and a processor 114.

The processor 114 registers identification information and device information about the electronic device. The front camera 112 may acquire an image of the electronic device, and the processor 114 may analyze the acquired image and register identification information about the electronic device included in the image. For example, the processor 114 extracts features of the electronic device from the image and registers the extracted features as identification information. The processor 114 may receive device information about the electronic device from the electronic device and register the device information. Alternatively, the processor 114 may store information regarding a method of communicating with a corresponding electronic device based on the type of the electronic device. For example, the processor 114 may register information about a frequency or protocol used by the electronic device as device information about the electronic device.

If the processor 114 determines that the user's eyes have not moved within the specified time period, the processor 114 analyzes the image received from the front camera 112. The processor 114 identifies the electronic devices included in the image. The processor 114 may detect a boundary of an object included in the image and extract a feature of the object. The processor 114 compares the recognized object with the registered electronic devices and determines whether there is a recognized device identical to any one of the registered electronic devices.

The processor 114 receives information from the eye tracking camera 113 regarding the direction of viewing by the user and adjusts the direction of the front camera 112 based on the received information. Thus, the direction of the front camera 112 is synchronized with the direction viewed by the user. When the direction of the front camera 112 is adjusted, the front camera 112 outputs an image obtained in the adjusted direction to the processor 114.

The processor 114 determines whether there is an electronic device identical to the identified electronic device from among the registered electronic devices. If there is an electronic device identical to the identified electronic device among the registered electronic devices, the processor 114 is paired with the identified electronic device and controls the electronic device based on an input from the user.

The smart glasses 110 may be paired with the electronic device via bluetooth or Wi-Fi. When the smart glasses 110 are paired with the electronic device, the smart glasses 110 control the electronic device. The smart glasses 110 may receive control-related inputs from a user and output control signals to the electronic device based on the received inputs.

The smart glasses 110 execute an application. The application is a program that is registered together when identification information or device information about the electronic device is registered. The application is a program for controlling the electronic device or an application that performs the same function as the electronic device. When executing an application, the smart glasses 110 may provide a screen image for controlling the electronic device to the user. Alternatively, the smart glasses 110 may perform a voice recognition program.

The smart glasses 110 receive the user's control. In other words, the smart glasses 110 receive user input. The smart glasses 110 receive a voice of the user or a gesture of the user. The smart glasses 110 output a control signal for controlling the electronic device to the electronic device based on the received voice or the recognized gesture. Speech may be received via the microphone while gestures may be received via the front camera 112.

Fig. 10 is a diagram for describing a pairing method according to an embodiment. Referring to fig. 10, the mobile device 1000 may determine the user's intention and select an electronic device to be paired. For example, if the smart glasses 110 determine that the user is looking in the same direction for a specified period of time, the smart glasses 110 are paired with an electronic device that is located in the direction the user is looking at. Further, if the mobile device 1000 determines that the camera 1200 is operated and the same image is input in a specified time period, the mobile device 1000 is paired with an electronic device included in the input image.

The mobile device 1000 may ask the user whether to control the identified electronic device. Fig. 10 shows a case where the remote controller 210 is included in the input image. The mobile device 1000 may display an image of the remote controller 210 at the display unit 1400 and may receive an input from the user as to whether to pair with the remote controller 210. If an input for pairing with the remote controller is received from the user, the mobile device 1000 is paired with the remote controller 210.

Fig. 11 is a diagram for describing a pairing method according to an embodiment. Referring to fig. 11, the mobile device 1000 may select an electronic device to be paired from among electronic devices included in the acquired image.

If an image is acquired when the mobile device 1000 is ready for pairing, the mobile device 1000 analyzes the acquired image and determines whether an electronic device is included in the acquired image. If the image is acquired in the pairing mode, the mobile device 1000 may analyze whether the electronic device is included in the acquired image. The mobile device 1000 may display an electronic device included in the display unit 1400.

Fig. 11 is a diagram showing a case where the remote controller 210 and the TV240 are included in the obtained image. The mobile device 1000 displays the remote controller 210 and the TV240 at the display unit 1400. Mobile device 1000 may receive input from a user to select an electronic device to be paired from between remote 210 and television 240. For example, if an input for pairing with the TV240 is received from a user via the display unit 1400, the mobile device 1000 is paired with the TV 240.

Fig. 12 is a diagram for describing a pairing method according to an embodiment. Referring to fig. 12, the mobile device 1000 may select electronic devices to be paired based on gestures of a user.

If a user's gesture is detected when the mobile device 1000 is ready for pairing, the mobile device 1000 analyzes the image and determines whether the electronic device is included in the image. When the mobile device 1000 is in the pairing mode, upon detecting a gesture for selecting a user of the electronic device, the mobile device 1000 may search for the electronic device in an area where the gesture is detected. Fig. 12 shows a gesture of a user for selecting the TV 240. Thus, the mobile device is paired with the TV 240.

Fig. 13 is a diagram for describing a pairing method according to an embodiment. Referring to fig. 13, the mobile device 1000 may select an electronic device to be paired from among pre-registered electronic devices.

The mobile device 1000 displays a list of electronic devices registered in advance at the display unit 1400. The mobile device 1000 receives an input from a user and selects an electronic device to be paired from among the displayed electronic devices. Fig. 13 shows a case where the user selects the TV 240. Thus, the mobile device 1000 is paired with the TV 240.

Fig. 14A is a diagram for describing a method in which the mobile device 1000 pairs with an electronic device by broadcasting a packet according to an embodiment. Referring to fig. 14A, a mobile device 1000 may pair with an electronic device by broadcasting a packet 1410 for connection.

The mobile device 1000 identifies the type of electronic device included in the image. The mobile device 1000 broadcasts a packet 1410 requesting pairing with an electronic device of the identified type. The mobile device 1000 receives the response packet and pairs with the electronic device that transmitted the response packet.

In fig. 14A, the mobile device 1000 broadcasts a packet 1410 and is paired with the TV 240. If the electronic device included in the image is determined to be the TV240, the mobile device 1000 broadcasts a packet 1410 for requesting pairing with the TV 240. The packet 1410 may include data for requesting only the TV240 to transmit a response packet. Data for requesting only the TV240 to transmit the response packet may be included in the advertisement data or in a separate item.

The TV240 receives packets 1410 from the mobile device 1000 and transmits response packets to the mobile device 1000. The mobile device 1000 receives the response packet from the TV240 and pairs with the TV240 based on the response packet.

Packet 1410 may include items such as a preamble, an advertisement access address, an advertisement header, an advertiser address, and advertisement data. The packet 1410 may be included in the advertising data of the broadcast and may include data that identifies the mobile device 1000 and indicates that the mobile device 1000 wants to pair with the TV. The mobile device 1000 may transmit the packet 1410 and notify the TV240 of the service to be provided. For example, the mobile device 1000 may transmit a moving image or song to the TV 240.

In addition, the mobile device 1000 can add information regarding the location of the mobile device 1000 to the packet 1410 and broadcast the packet 1410.

The TV240 receives the packet 1410 and transmits a response packet indicating that the TV240 is a TV available for connection to the mobile device 1000. The TV240 may add information on a necessary application for controlling the TV240 to the response packet and transmit the response packet to the mobile device 1000. If the packet 1410 includes data that only requests the TV240 to transmit a response packet, only the TV240 of the electronic devices that received the packet 1410 transmits a response packet to the mobile device 1000.

The TV240 may communicate with the location of the mobile device 1000 included in the packet 1410 via, for example, a beamforming method. In this case, the TV240 may transmit the response packet to the location of the mobile device 1000, and thus power consumption may be reduced.

Fig. 14B is a diagram for describing a method in which an electronic device pairs with the mobile device 1000 by broadcasting a packet according to an embodiment. Referring to fig. 14B, an electronic device may pair with the mobile device 1000 by broadcasting a packet 1410. In other words, if the TV240 broadcasts the packet 1410, the mobile device 1000 may pair with the TV240 by using the device information included in the received packet 1410.

In fig. 14B, the TV240 pairs with the mobile device 1000 through a broadcast packet 1410. The packet 1410 transmitted by the TV240 includes device information about the TV 240. In other words, the packet transmitted by the TV240 includes data indicating that the TV240 is a TV available for pairing and connection information for pairing.

The mobile device 1000 receives a packet 1410 broadcast from the TV240 and pairs with the TV240 by transmitting a response packet to the TV240 that transmitted the packet 1410.

The mobile device 1000 determines that the pairable TV240 is nearby based on the packet 1410 received from the TV240 and stores the device information included in the packet 1410. The mobile device 1000 stores identification information about the TV240 in advance. The mobile device 1000 extracts device information of the electronic device included in the acquired image. The mobile device 1000 compares the extracted identification information with the stored identification information and identifies the electronic device included in the image. If the electronic device included in the image is implemented as the TV240, the mobile device 1000 attempts pairing with the TV240 by transmitting a response packet to the TV240 using device information included in a packet 1410 included in the TV 240.

Fig. 15 is a diagram for describing a method of pairing the mobile device 1000 with the electronic device by broadcasting a packet according to the embodiment. Referring to fig. 15, in operation 1510, the mobile device 1000 stores identification information about an electronic device. The mobile device 1000 may store identification information according to the type of electronic device.

In operation 1520, the mobile device 1000 acquires an image of the electronic device via the camera 1200.

In operation 1530, the mobile device 1000 identifies a type of the electronic device.

In operation 1540, the mobile device 1000 broadcasts a packet for pairing with the identified type of electronic device. The packet includes information about the mobile device 1000 and information about the type of electronic device to be paired.

In operation 1550, the mobile device 1000 receives a response packet from the electronic device. The electronic device that receives the packet from the mobile device 1000 reads information about the type of electronic device indicated in the packet. The electronic device that received the packet from the mobile device 1000 transmits a response packet to the mobile device 1000 only when the electronic device is the electronic device indicated in the read information. The response packet includes information on the type of the electronic device, information indicating whether the electronic device is available for pairing, and device information for pairing.

At operation 1560, the mobile device 1000 is paired with an electronic device. The mobile device 1000 can be paired with the electronic device that transmitted the response packet by using the device information included in the response packet.

Fig. 16 is a diagram for describing a method of the mobile device 1000 controlling an electronic device by using an input unit according to an embodiment. Referring to fig. 16, the smart glasses 110 may control an electronic device by using an input unit.

The first and second input units 115 and 116 of the smart glasses 110 serve as input units for controlling the operation of the smart glasses 110 if the smart glasses 110 do not control the electronic device. However, the smart glasses 110 control nearby electronic devices, and the functions of the first and second input units 115 and 116 are changed to control functions of the nearby electronic devices.

The smart glasses 110 include first and second input units 115 and 116. The functions of the first and second input units 115 and 116 are changed based on the recognized electronic devices. For example, if the smart glasses 110 recognize the remote controller 210 and control the remote controller 210, the first input unit 115 performs a function for changing a track (track), and the second input unit 116 performs a function for changing a volume. If the smart glasses 110 recognize the TV240 and control the TV240, the first input unit 115 performs a function for changing a channel, and the second input unit 116 performs a function for changing a volume.

Fig. 17 is a diagram for describing a method of the mobile device 1000 controlling an electronic device by using an application according to an embodiment. Referring to fig. 17, the smartphone 140 may control the remote controller 210 or the TV240 by using an application.

The smart phone 140 stores an application or program for controlling the electronic device. The smartphone 140 may store an application or program for controlling the electronic device in advance, or acquire information for downloading the application or program, download the application or program, and store the application or program.

The smartphone 140 executes an application or program corresponding to the identified electronic device and controls the identified electronic device. For example, if the smartphone 140 recognizes the remote controller 210 and controls the remote controller 210, the smartphone 140 executes an application or program corresponding to the remote controller 210. Further, if the smart phone 140 recognizes the TV240 and controls the TV240, the smart phone 140 executes an application or program corresponding to the TV 240.

If the smart phone 140 executes an application or program, a screen image for controlling the function of the electronic device is displayed at the smart phone 140. The smartphone 140 may receive a touch input from a user and transmit a control signal for changing a channel or a control signal for changing a volume to the electronic device.

Fig. 18 is a diagram for describing a method of the mobile device 1000 controlling the electronic device by providing a screen image displayed at the electronic device according to an embodiment. Referring to fig. 18, the smartphone 140 may control the wearable device 260 by providing a screen image displayed at the wearable device 260. Although the following description refers to the smartphone 140, any electronic device capable of displaying the same screen image as that displayed at the wearable device 260 and receiving an input from the user may control the wearable device 260 by providing the user with the same screen image as that displayed at the wearable device 260.

When the smartphone 140 is paired with the wearable device 260, the smartphone 140 may provide the same screen image as the screen image displayed at the wearable device 260. The smartphone 140 receives the screen image from the wearable device 260 and displays the received screen image.

The smartphone 140 controls the wearable device 260 based on input from the user. The smartphone 140 transmits a control signal to the wearable device 260 based on the location of the input from the user and the received screen image. For example, the smartphone 140 may determine the location on the touch panel that the user touches and transmit a control signal to the wearable device 260. The wearable device 260 performs an operation based on the control signal, and transmits a screen image showing a result of performing the operation to the smartphone 140. The smartphone 140 displays the received screen image at the display unit.

Fig. 19 is a diagram for describing a method of the mobile device 1000 controlling the audio system 200 according to the embodiment. Referring to fig. 19, the mobile device 1000 is paired with the remote controller 210 and controls the audio system 220 via the remote controller 210.

The mobile device 1000 is paired with the remote control 210. The mobile device 1000 identifies the remote control 210 and pairs with the remote control 210.

The mobile device 1000 controls the remote controller 210. The display unit 1400 displays a screen image for controlling the remote controller 210. Fig. 19 shows a case where the track and the volume level are displayed at the display unit 1400. The mobile device 1000 receives an input from a user and transmits a control signal to the remote controller 210 based on the received input. For example, the mobile device 1000 may receive an input for changing a track or an input for changing a volume from a user.

The remote control 210 controls the audio system 220. The remote control 210 controls the audio system 220 based on control signals received from the mobile device 1000.

The mobile device 1000 may control the audio system 220 by transmitting control signals directly to the audio system 220.

Fig. 20 is a diagram for describing a method of the mobile device 1000 controlling the air conditioner 270 according to an embodiment. Referring to fig. 20, the mobile device 1000 is paired with the remote controller 210 and controls the air conditioner 270 via the controller 210.

The mobile device 1000 is paired with the remote control 210. The mobile device 1000 identifies the remote control 210 and pairs with the remote control 210.

The mobile device 1000 controls the remote controller 210. The mobile device 1000 receives an input from a user and transmits a control signal to the remote controller 210 based on the received input. For example, the mobile device 1000 may receive an input for changing an air volume or an input for changing a temperature from a user.

The mobile device 1000 controls the air conditioner 270. The display unit 1400 displays a screen image for controlling the air conditioner 270. Fig. 20 shows a case where the air volume and the temperature are displayed at the display unit 1400. The remote controller 210 controls the air conditioner 270 based on the control signal received from the mobile device 1000.

The mobile device 1000 may control the air conditioner 270 by directly transmitting a control signal to the air conditioner 270.

Fig. 21 is a diagram for describing a method of the mobile device 1000 controlling the laptop PC 290 according to an embodiment. Referring to fig. 21, the mobile device 1000 is paired with a mouse 280, and controls a laptop PC 290 by using the mouse 280.

The mobile device 1000 is paired with a laptop PC 290. The mobile device 1000 recognizes the mouse 280 and pairs with the mouse 280.

The mobile device 1000 controls the mouse 280. The display unit 1400 displays a screen image for controlling the mouse 280. Fig. 21 shows a case where movement, click, and double click are displayed on the display unit 1400. The mobile device 1000 receives input from a user and transmits a control signal to the mouse 280 based on the received input. For example, the mobile device 1000 may receive a command from a user to move a cursor, click, or double-click.

The mouse 280 controls the laptop PC 290. The mouse 280 controls the laptop PC 290 based on the control signal received from the mobile device 1000.

The mobile device 1000 may control the laptop PC 290 by directly transmitting a control signal to the laptop PC 290.

Fig. 22 is a diagram for describing a method of the mobile device 1000 controlling the refrigerator 295 according to an embodiment. Referring to fig. 22, the mobile device 1000 is paired with a refrigerator 295 and controls the refrigerator 295.

The mobile device 1000 is paired with a refrigerator 295. The mobile device 1000 identifies the refrigerator 295 and pairs with the refrigerator 295.

The mobile device 1000 controls the refrigerator 295. The display unit 1400 displays a screen image for controlling the refrigerator 295. Fig. 22 shows a case where the temperature and the list are displayed at the display unit 1400. Mobile device 1000 receives input from a user and transmits control signals to refrigerator 295 based on the received input. For example, mobile device 1000 may receive a command from a user to adjust the temperature of refrigerator 295.

The mobile device 1000 may receive information about the refrigerator 295 from the refrigerator 295 and display the information at the display unit 1400. For example, the refrigerator 295 can transmit a list of content stored in the refrigerator 295 to the mobile device 1000. The mobile device 1000 displays the list received from the refrigerator 295 at the display unit 1400. The user can adjust the temperature of the refrigerator 295 with reference to the list.

Fig. 23 is a diagram for describing a method of the mobile device 1000 controlling the wearable device 260 according to the embodiment. Referring to fig. 23, a mobile device 1000 is paired with a wearable device 260 and controls the wearable device 260.

Mobile device 1000 is paired with wearable device 260. Mobile device 1000 identifies wearable device 260 and pairs with wearable device 260.

Mobile device 1000 controls wearable device 260. The display unit 1400 displays a screen image for controlling the wearable device 260. Fig. 23 shows a case where the display unit 1400 displays the same screen image as that displayed at the wearable device 260. Mobile device 1000 receives input from a user and transmits control signals to wearable device 260.

Fig. 24 is a diagram for describing a method of the mobile device 1000 controlling the mobile device 260 according to the embodiment. Referring to fig. 24, a mobile device 1000 is paired with another mobile device 260 and controls the other mobile device 260.

The mobile device 1000 is paired with another mobile device 260. The mobile device 1000 identifies another mobile device 260 and pairs with the other mobile device 260.

The mobile device 1000 controls the further mobile device 260. The display unit 1400 displays a screen image for controlling another mobile device 260. Fig. 25 shows a case where the display unit 1400 displays the same screen image as that displayed at another mobile device 260. The mobile device 1000 receives input from a user and transmits control signals to another mobile device 260.

Fig. 25 is a diagram for describing a method of the mobile device 1000 controlling the smartphone 230 according to the embodiment. Referring to fig. 25, the mobile device 1000 is paired with the smartphone 230 and controls the smartphone 230.

Mobile device 1000 is paired with smartphone 230. The mobile device 1000 identifies the smartphone 230 and pairs with the smartphone 230.

The mobile device 1000 controls the smartphone 230. The display unit 1400 displays a screen image for controlling the smartphone 230. Fig. 25 shows a case where the display unit 1400 displays the same screen image as that displayed at the smartphone 230. The mobile device 1000 receives input from a user and transmits control signals to the smartphone 230.

Fig. 26A is a diagram for describing a method of pairing the mobile device 1000 with the electronic device by using an image of the electronic device according to the embodiment. Referring to fig. 26A, the mobile device 1000 may acquire an image of an electronic device and select an electronic device to be paired.

In fig. 26A, images of the smartphone 140 and the wearable device 130 are shown in a book 2600. The mobile device 1000 may recognize the smartphone 230 and the wearable device 260 shown in the book 2600, and display the recognized smartphone 230 and the recognized wearable device 260 at the display unit 1400.

The user selects the electronic device to be paired from before the smartphone or wearable device 260.

The mobile device 1000 is paired with an electronic device selected by a user. The mobile device 1000 determines whether the selected electronic device is nearby and pairs with the selected electronic device by using the stored device information.

The drawings or images shown in the book may be drawings or images drawn or photographed at different angles compared to the angle at which the stored device information (i.e., the stored drawings or images) is photographed. In this case, the mobile device 1000 may not find the drawing or image shown in the book in the stored device information even if the drawing and the device information indicate the same product. To solve the problem, when the mobile device 1000 stores device information, the mobile device 1000 may store images taken at various angles or drawings drawn at different angles. In other words, the stored device information may include multiple images or multiple drawings regarding the same product.

Fig. 26B is a diagram showing an example of device information stored in the mobile device 1000. Referring to fig. 26B, representative images of the smartphone and images of the smartphone taken at various angles are shown.

If the mobile device 1000 recognizes the electronic device, the mobile device 1000 may compare the acquired image of the electronic device with the representative image of the stored device information, and if the acquired image of the electronic device is different from the representative image of the stored device information, may compare the acquired image of the electronic device with images photographed at other angles.

The mobile device 1000 may utilize representative images and images taken at various angles to identify not only the drawings in the book, but also the actual electronic device.

Fig. 27 is a diagram for describing a method of pairing the mobile device 1000 with an electronic device via text. Referring to fig. 27, the mobile device 1000 may recognize the name of the electronic device included in the text 2700 and select an electronic device to be paired.

In fig. 27, a text 2700 includes a name of the electronic device. Mobile device 1000 may identify the name included in text 2700. The mobile device 1000 analyzes text included in the input image and determines whether the text 2700 includes a name of the electronic device.

The mobile device 1000 identifies the name of the electronic device located in the direction of viewing by the user. Alternatively, the mobile device 1000 identifies the name of the electronic device included in the acquired image. The mobile device 1000 displays an electronic device corresponding to the name of the identified electronic device at the display unit 1400. If the names of multiple electronic devices are identified, the electronic devices to be paired may be selected by the user.

The mobile device 1000 is paired with the selected electronic device. The mobile device 1000 determines whether the selected electronic device is nearby and pairs with the selected electronic device by using the device information.

The mobile device 1000 may identify a name of the electronic device received via the microphone and select the electronic device to be paired. If the user speaks the name of the electronic device, the mobile device 1000 identifies the name of the electronic device spoken by the user via the microphone and selects the electronic device to pair.

Fig. 28 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to register an electronic device. Referring to fig. 28, the mobile device 1000 registers the identified electronic device.

The mobile device 1000 registers the electronic device 1200 by using an image acquired via a camera. For example, if the user looks in the same direction for a specified period of time, the mobile device 1000 selects an electronic device located in the direction in which the user looks as an electronic device to be registered. Alternatively, the mobile device 1000 may select an electronic device included in the acquired image as an electronic device to be registered.

The mobile device 1000 may also display text asking whether to register the identified electronic device. Fig. 28 shows an operation for registering the remote controller 210, in which the display unit 1400 displays a text inquiring whether the remote controller 210 is registered.

Fig. 29 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to register an application. Referring to fig. 29, the mobile device 1000 registers an application for controlling the identified electronic device. The application comprises functions for controlling the electronic device.

The mobile device 1000 may also display text asking whether to register the identified electronic device. Fig. 28 illustrates an operation for registering "an application for controlling a remote controller", in which the display unit 1400 displays a text inquiring whether or not an application is registered.

Fig. 30 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to register an application. Referring to fig. 30, a mobile device 1000 registers a plurality of applications for controlling identified electronic devices.

The mobile device 1000 can assign different applications to different time slots for controlling the electronic device. For example, mobile device 1000 may assign "remote control application a" to the morning time slot for controlling remote control 210 in the morning and "remote control application B" to the afternoon time slot for controlling remote control 210 in the afternoon.

The mobile device 1000 may display text asking whether to register the application. Fig. 30 shows an operation for registering "remote controller control application a" and "remote controller control application B", in which the display unit 1400 displays a text inquiring whether or not an application is registered.

Fig. 31 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to register an application. Referring to fig. 31, a mobile device 1000 registers a plurality of applications for controlling identified electronic devices.

The mobile device 1000 can assign different applications to different locations for controlling the electronic device. For example, the mobile device 1000 may assign a "remote control application A" to the A-location for controlling the remote control 210 at the A-location and a "remote control application B" to the B-location for controlling the remote control 210 at the B-location.

The mobile device 1000 may display text asking whether to register the application. Fig. 31 shows an operation for registering "remote controller control application a" and "remote controller control application B", in which the display unit 1400 displays a text inquiring whether or not an application is registered.

Fig. 32 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to register a control function. Referring to fig. 32, the mobile device 1000 registers a control function corresponding to an input unit.

The mobile device 1000 registers a control function for controlling the electronic device. If the electronic device can be controlled by using the input unit, the mobile device 1000 may map the input unit to a control function. Accordingly, when the mobile device 1000 controls the electronic device, the mobile device 1000 changes the default function of the input unit to the control function for controlling the mapping of the electronic device.

For example, a first input unit of the mobile device 1000 may be mapped to a channel changing function of the remote controller 210, and a second input unit of the mobile device 1000 may be mapped to a volume changing function of the remote controller 210.

Fig. 33 is a diagram for describing an operation of pairing the mobile device 1000 according to the embodiment with an electronic device. Referring to fig. 33, a mobile device 1000 may be paired with an electronic device included in an image acquired via a camera 1200.

In operation 3310, the mobile device 1000 stores identification information about a plurality of electronic devices and device information for pairing with the respective electronic devices. The mobile device 1000 stores identification information and device information about controllable electronic devices. The mobile device 1000 may acquire an image of the electronic device and extract features of the electronic device from the acquired image. The mobile device 1000 may store the extracted features of the electronic device as identification information.

In another example, the mobile device 1000 may store features of the same type of electronic device. The mobile device 1000 may extract and store features of the remote control 210, the TV240, or various types of TVs 240. The mobile device 1000 may receive data including features of the electronic device via a network and store the data in memory.

In operation 3320, the mobile device 1000 acquires an image of at least one electronic device via the camera 1200. The mobile device 1000 may acquire an image by using the camera 1200 and acquire an image of the electronic device included in the acquired image. The mobile device 1000 may analyze an image input via the camera 1200 and determine whether the image includes an electronic device.

In operation 3330, the mobile device 1000 extracts identification information about an electronic device included in the acquired image. In operation 3340, the mobile device 1000 searches the stored identification information and selects an electronic device corresponding to the extracted identification information. If identification information corresponding to the extracted identification information exists in the memory, the mobile device 1000 selects an electronic device corresponding to the extracted identification information as an electronic device to be paired. For example, if the extracted identification information corresponds to identification information about the TV240, the mobile device 1000 selects the TV240 as an electronic device to be paired.

In operation 3350, the mobile device 1000 pairs with the selected electronic device by using the stored device information on the selected electronic device.

Fig. 34 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to recognize an object and provide a related function. Referring to fig. 34, the mobile device 1000 may identify a person 3410, a pen 3420, and an eraser 3430, and provide a function related to the identified object.

The mobile device 1000 may analyze an input image and recognize an object included in the image. The image may include a plurality of objects. The mobile device 1000 may extract features of a plurality of objects and identify the type of the object based on the extracted features. For example, the mobile device 1000 may determine whether the object is a person 3410, a pen 3420, or an eraser 3430.

The mobile device 1000 determines whether the same object as the recognized object is registered. The mobile device 1000 compares the features of the identified object with the features of the pre-stored objects and searches for objects having those features that match the identified object.

The mobile device 1000 provides functions set with respect to the recognized object. The mobile device 1000 may preset functions with respect to respective objects. For example, the mobile device 1000 may set a phone call function or a messenger execution function when the mobile device recognizes the person 3410.

Mobile device 1000 can assign functionality to elements based on the type of element. For example, mobile device 1000 may assign a "notebook" function to pen 3420. The mobile device 1000 may assign an "erase" function to the eraser 3430.

When multiple objects are identified, the mobile device 1000 may receive input from a user for selecting a function. The mobile device 1000 may display the identified objects, receive input from a user, and select one of the objects.

The mobile device 1000 provides functionality related to the identified object. For example, if the identified object is a particular person 3410, the mobile device 1000 may place a telephone call to the person 3410. Alternatively, if the identified object is pen 3420, mobile device 1000 may perform a "notebook" function.

Fig. 35 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to make a phone call to an identified person. Referring to fig. 35, the mobile device 1000 identifies a person 3410 and makes a telephone call to the identified person 3410 using a registered telephone number.

The mobile device 1000 identifies a person 3410 included in an image acquired via the camera 1200. The mobile device 1000 determines whether the identified person 3410 is a registered person. If the identified person 3410 is a registered person, the mobile device 1000 performs a function registered with respect to the registered person. In fig. 35, the function registered with respect to the person 3410 is a "phone call" function. The telephone number registered in the mobile device 1000 with respect to the person 3410 is 010-. Thus, the mobile device 1000 makes a telephone call using the telephone numbers 010-.

Fig. 36 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to transmit a message to an identified person. Referring to fig. 36, the mobile device 1000 identifies a person 3410 and transmits a message to the identified person 3410.

The mobile device 1000 recognizes a person 3410 included in an image acquired via the camera 1200. The mobile device 1000 determines whether the identified person 3410 is a registered person. If the identified person 3410 is a registered person, the mobile device 1000 performs a function of registration with respect to the registered person. In fig. 35, the function registered with respect to the person 3410 is a "messenger execution" function. The mobile device 1000 executes a messenger with respect to the person 3410 and displays a screen image for transmitting a message to the person 3410. Alternatively, the mobile device 1000 may display a screen image for sending a message to the person 3410.

Fig. 37 is a diagram for describing an operation of causing the mobile device 1000 according to the embodiment to execute an application corresponding to the identified person. Referring to fig. 37, the mobile device 1000 identifies a person 3410 and executes applications 3710 and 3720 corresponding to the identified person 3410.

The mobile device 1000 identifies a person 3410 included in an image acquired via the camera 1200. The mobile device 1000 determines whether the identified person 3410 is a registered person. If the identified person 3410 is a registered person, the mobile device 1000 performs a function registered with respect to the registered person. In fig. 35, the function registered with respect to the person 3410 is an "application execution" function. The mobile device 1000 executes an application registered with respect to the person 3410. If multiple applications 3710 and 3720 are registered with respect to a single person 3410, the mobile device 1000 displays first and second applications 3710 and 3720, as shown in fig. 37.

If multiple applications 3710 and 3720 are registered, the mobile device 1000 may receive an input from the user for selecting an application to be executed first. The mobile device 1000 may display the registered applications 3710 and 3720, receive input from the user, and select one of the applications 3710 and 3720.

Fig. 38 is a diagram for describing a method of causing the mobile device 1000 according to the embodiment to perform a function of registering with respect to an identified object. Referring to fig. 38, the mobile device 1000 may recognize an object and provide a function of registering with respect to the recognized object.

In operation 3810, the mobile device 1000 may analyze the input image and recognize an object included in the image. The image may include a plurality of objects. The mobile device 1000 may extract features of a plurality of objects and identify the type of the object based on the extracted features.

In operation 3820, the mobile device 1000 determines whether an object identical to the recognized object is registered. The mobile device 1000 compares the characteristics of the identified object with the characteristics of the pre-stored objects and searches for objects having matching characteristics.

In operation 3830, the mobile device 1000 provides the function set with respect to the recognized object. The mobile device 1000 may preset functions with respect to respective objects. For example, mobile device 1000 may set a "phone call" function or a "messenger execution" function with respect to person 3410.

The mobile device 1000 may provide functions related to things. For example, mobile device 1000 may register a "notebook" function with pen 3420. The mobile device 1000 may register an "erase" function with respect to the eraser 3430.

When multiple objects are identified, the mobile device 1000 may receive input from a user for selecting an object to perform a function. The mobile device 1000 may display the identified objects, receive input from a user, and select one of the objects.

The mobile device 1000 provides functionality related to the identified object. For example, if the identified object is a particular person 3410, the mobile device 1000 may place a telephone call to the person 3410. Alternatively, if the identified object is pen 3420, mobile device 1000 may perform a "notebook" function.

Fig. 39 is a diagram for describing a method in which the mobile device 1000 controls the legacy device 3901 according to the embodiment. Referring to fig. 39, a mobile device 1000 may be paired with a patch 3903 and control the legacy device 3901 via the patch 3903. The legacy device 3901 is a legacy electronic device that does not have communication capabilities and cannot be directly connected to the mobile device 1000. For example, the conventional apparatus 3901 may be an air conditioner having an Infrared (IR) communication function.

The patch 3903 is an electronic device attached to the legacy device 3901, and can function as a communication relay device. The patch 3903 can communicate with the mobile device 1000 via a first type of communication method and with the legacy device 3901 via a second type of communication method. For example, the patch 3903 may communicate with the mobile device 1000 via Wi-Fi and with the legacy device 3901 via IR communication methods.

If the patch 3903 is paired with the mobile device 1000, the display unit 1400 of the mobile device 1000 displays a screen image for controlling the legacy device 3901.

Control data for controlling the legacy device 3901 may be stored in the patch 3903. Control data may be received from the mobile device 1000. The operation of causing the mobile device 1000 to acquire control data for controlling the legacy device 3901 is the same as the operation of causing the mobile device 1000 to recognize an electronic device.

If the identified electronic device is determined to be a legacy device 3901, the mobile device 1000 can transmit control data for controlling the legacy device 3901 to a patch 3903 attached to the legacy device 3901. The mobile device 1000 may display a screen image for controlling the legacy device 3901.

Fig. 39 is a diagram showing a case where the display unit 1400 displays the air volume and the temperature of the conventional apparatus 3901. The user can control the legacy device 3901 by using virtual buttons displayed at a display unit of the mobile device 1000.

For example, mobile device 1000 controls patch 3903. The mobile device 1000 can receive user input for changing air volume or user input for changing temperature, and transmit control data to the patch 3903 based on the received input. The patch 3903 controls an air conditioner.

Fig. 40 is a diagram for describing a method in which the glasses 5000 controls an electronic device according to an embodiment. Referring to fig. 40, the glasses 5000 may control the TV6000 and the smart phone 7000 to be paired with each other. Further, the smart phone 7000 can seamlessly play the same content that the user is watching via the TV 6000.

The glasses 5000 controls the TV6000 and the smart phone 7000 to pair with each other. The glasses 5000 may track the eyes of the user and determine the electronic devices to be paired with each other. For example, if the glasses 5000 detects that the user is watching the TV6000, and then at the smart phone 7000, the glasses 5000 may control the TV6000 and the smart phone 7000 to pair with each other. The expression that the TV6000 and the smart phone 7000 are paired with each other means that the TV6000 and the smart phone 7000 are connected to each other via wire or wirelessly, and transmit and receive data to and from each other.

The glasses 5000 may track the eyes of the user. The glasses 5000 may track the user's eyes or detect a change in the pupils. Accordingly, the glasses 5000 may detect that the user is gazing in a specific direction, and determine the electronic devices located in the specific direction as the electronic devices to be paired with each other.

The glasses 5000 may recognize an iris of the user. The glasses 5000 may extract feature points of the iris of the user and determine the identity of the user wearing the glasses 5000. Further, the glasses 5000 may transmit information about the identified iris to the TV6000 and the smart phone 7000 for user authentication. Therefore, only users registered to the glasses 5000, the TV6000, and the smart phone 7000 can be authorized to control the glasses 5000, the TV6000, and the smart phone 7000.

If the iris information received from the glasses 5000 is the same as the iris information stored in the TV6000, the TV6000 may transmit device information on the TV6000 to the glasses 5000. The glasses 5000 may pair with the TV6000 using the received device information.

If the iris information received from the glasses 5000 is the same as the iris information stored in the smart phone, the smart phone 7000 may transmit the device information on the smart phone 7000 to the glasses 5000. The glasses 5000 may pair with the smart phone 7000 using the received device information.

If the user is watching the TV6000 while watching the smart phone 7000, the glasses 5000 may control the smart phone 7000 to seamlessly play back the content being played back at the TV 6000. The glasses 5000 may transmit information about the content being played back at the TV6000 to the smart phone 7000. The smart phone 7000 can receive the content from the TV6000 and play back the content, or if the smart phone 7000 stores the same content as the content being played back at the glasses 5000, the smart phone 7000 can play back the stored content. The smart phone 7000 can receive bookmark information from the TV 6000. The bookmark information is information indicating a playback point of content played back on the TV 6000. When the smart phone 7000 receives bookmark information from the TV6000, the smart phone 7000 can determine a playback point of the content based on the bookmark information. Therefore, the smart phone 7000 can seamlessly play back the content being played back at the TV 6000.

Fig. 41 is a flowchart for describing a method of seamlessly playing back content according to an embodiment. Referring to fig. 41, the smart phone 7000 can seamlessly play back the content being played back at the TV 6000.

In operation 4101, the glasses 5000 tracks the eyes of the user and determines the electronic devices to be paired. The glasses 5000 may track the direction in which the user looks and detect changes in the pupil size of the user. The glasses 5000 may determine a direction in which the user views based on a change in the user's eyes or pupils. The glasses 5000 may determine an electronic device located in a direction of viewing by the user as an electronic device to be paired. Fig. 41 illustrates a case where the glasses 5000 determines the TV6000 as an electronic device to be paired.

In operation 4102, the glasses 5000 recognize an iris of the user. The glasses 5000 extract feature points of the iris of the user.

In operation 4103, the glasses 5000 performs user authentication via the iris of the user. The user authentication is for determining whether the user wearing the glasses 5000 is properly authorized to control the operation of the TV 6000. The glasses 5000 transmit the extracted feature points of the iris to the TV6000, and the TV6000 performs user authentication by comparing the stored feature points of the iris with the received feature points of the iris. When the user authentication is completed, the glasses 5000 and the TV6000 may transmit and receive data to and from each other, or the glasses 5000 may control the TV 6000.

In operation 4104, the TV6000 stores usage information about the user. The TV6000 can store information on contents played back by the user. The information on the content includes a name of the content, a playback point of the content, and the like. The TV6000 can store usage information on each user. Accordingly, if the user of the glasses 5000 is changed, the TV6000 may store usage information about the changed user in a separate memory.

In operation 4105, the glasses 5000 tracks the eyes of the user and determines the electronic devices to be paired. The glasses 5000 may track the direction in which the user looks and detect changes in the pupil size of the user. Fig. 41 shows a case where the electronic device which the glasses 5000 face is changed from the TV6000 to the smart phone 7000. Since the user of the glasses 5000 looks at the smart phone 7000, the glasses 5000 determines the smart phone 7000 as the electronic device to be paired.

In operation 4106, the glasses 5000 recognize an iris of the user. The glasses 5000 extract feature points of the iris of the user.

In operation 4107, the glasses 5000 performs user authentication via the iris of the user. The user authentication is for determining whether the user wearing the glasses 5000 is properly authorized to control the operation of the smart phone 7000. The glasses 5000 transmit the extracted feature points to the smart phone 7000, and the smart phone 7000 performs user authentication by comparing the stored feature points of the iris with the received feature points of the iris. When the user authentication is completed, the glasses 5000 and the smart phone 7000 may transmit and receive data to and from each other, or the glasses 5000 may control the smart phone 7000.

In operation 4108, the glasses 5000 transmit information about the previous electronic device to the smart phone 7000. The previous electronic device may be the electronic device that was most recently paired with the glasses 5000. The information about the previous electronic device may include information identifying what the previous electronic device is, information for pairing with the previous electronic device, or information about content played back by the previous electronic device. In fig. 41, the previous electronic device is a TV 6000. Accordingly, the glasses 5000 transmit information about the TV6000 to the smart phone 7000.

In operation 4109, the smart phone 7000 is paired with the TV 6000. The smart phone 7000 may pair with the TV6000 based on the information about the TV6000 received from the glasses 5000. The smart phone 7000 requests the TV6000 to transmit the content.

In operation 4110, the TV6000 may transmit the content being played back or the content recently played back to the smart phone 7000. Further, the TV6000 can transmit bookmark information indicating information of a playback point of the content as well as the content to the smart phone 7000.

In operation 4111, the smart phone 7000 plays back the received content. The smart phone 7000 can determine the playback point of the content based on the bookmark information. The smart phone 7000 plays back the content based on the playback point. Therefore, the smart phone 7000 can seamlessly play back the content being played back at the TV 6000.

Fig. 42 is a flowchart for describing a method of seamlessly playing back content according to an embodiment. Fig. 42 is a diagram for describing a method of playback by the smart phone 7000 if the smart phone 7000 stores the contents. Since the smart phone 7000 stores the content, it is not necessary for the smart phone 7000 to receive the content from the TV6000, and the smart phone 7000 can play back the content based on the information about the content.

The above description regarding operations 4101 to 4107 of fig. 41 may also be applied to operations 4201 to 4207.

In operation 4208, the glasses 5000 transmits usage information about the user and information about the content to the smart phone 7000. Since the content played back at the TV6000 is also stored in the smart phone 7000, it is not necessary to cause the glasses 5000 to transmit the content being played back at the TV6000 to the smart phone 7000. Therefore, the glasses 5000 transmit content information including what content is being played back and a playback point of the content to the smart phone 7000.

In operation 4209, the smart phone 7000 plays back the stored content. The smart phone 7000 plays back the content based on the playback point of the content. Therefore, the smart phone 7000 can seamlessly play back the content being played back at the TV 6000.

Fig. 43 is a diagram for describing an electronic apparatus according to the embodiment. Referring to fig. 43, an electronic device 8000 includes a communication unit 8100, a memory 8200, a display unit 8300, and a processor 8400.

The communication unit 8100 receives iris information about a user wearing the glasses 5000 from the glasses 5000, and receives device information about previous electronic devices used by the user from the glasses 5000. For example, the iris information may be information on feature points extracted from the iris of the user.

The communication unit 8100 receives content and bookmark information on the content from a previous electronic device. The content may be media files such as music files, video files, text files, image files, and the like.

The memory 8200 stores iris information about at least one user.

The processor 8400 identifies a user wearing the glasses 5000 by comparing the received iris information with the stored iris information, and pairs with a previous electronic device by using device information on the previous electronic device. The received iris information refers to information about the iris of the user wearing the glasses 5000, and the stored iris information refers to the iris information of the user stored in the memory 8200 of the electronic device 8000. The processor 8400 determines similarity between the received iris information and the stored iris information, and may determine that the received iris information is identical to the stored iris information if the similarity is equal to or higher than a set critical value. The processor 8400 may pair with the glasses 5000 or another electronic device only when the received iris information is the same as the stored iris information.

The processor 8400 determines a playback point of the content based on the bookmark information and controls the display unit 8300 to play back the content from the playback point. The bookmark information indicates a playback point of the content. The processor 8400 distinguishes bookmark information associated with different users and determines a playback point of the content by using the bookmark information about the authenticated user.

The processor 8400 determines a playback point of the content stored in the memory 8200 based on the bookmark information and controls the display unit 8300 to play back the content.

The display unit 8300 plays back the content being played back at the previous electronic device according to the control of the processor 8400.

Fig. 44 is a diagram for describing a seamless playback method according to an embodiment. Referring to fig. 44, the TV6000 and the smart phone 7000 can seamlessly play back the contents.

The glasses 5000 may track the user's eyes and determine the electronic device the user is gazing at, and based on the user's eye movement, the glasses 5000 again determines the electronic device the user is gazing at. For example, if a user looks at a second electronic device while the user is viewing content via a first electronic device, the second electronic device seamlessly plays back the content being played back at the first electronic device.

Fig. 44 is a diagram for describing a method in which the smart phone 7000 seamlessly plays back a video being played back at the TV6000 when the user a watches the smart phone 7000 while the user a watches the video via the TV 6000. The TV6000 plays back the video and stores bookmark a for user a. Bookmark a is updated while the video is being played back. If the user a gazes at the smart phone 7000, the glasses 5000 track the eyes of the user a and determine the electronic device at which the user a gazes. The glasses 5000 controls the smart phone 7000 to seamlessly play back the video being played back at the TV 6000. The smart phone 7000 receives information about the video from the glasses 5000 and the TV 6000. The smart phone 7000 plays back the video from the point corresponding to bookmark a based on the received information. Bookmark a is received from TV6000 or glasses 5000. Accordingly, the smart phone 7000 can play back the video from the point corresponding to bookmark a, and therefore, the user a can seamlessly view the video being played back on the TV6000 via the smart phone 7000.

Fig. 45 is a diagram for describing an embodiment of utilizing user-customized bookmark information if there are a plurality of users. Referring to fig. 45, if a user B wears glasses 5000, a TV6000 stores and updates bookmarks B with respect to the user B. If the smart phone 7000 plays back the video seamlessly, the smart phone 7000 requests the TV6000 about the user B bookmark and video. The TV6000 transmits the bookmark B to the smart phone 7000. The smart phone 7000 plays back the video based on the received bookmark B. Thus, the smart phone 7000 can seamlessly play back the video that user B is watching.

Fig. 46 is a diagram for describing the eyeglasses 5000 according to the embodiment. Referring to fig. 46, the glasses 5000 includes an eye tracking camera 5100, a memory 5200, and a processor 5300.

The eye-tracking camera 5100 acquires images of the eyes of the user wearing the glasses 5000. The eye tracking camera 5100 transmits the acquired image to the processor 5300.

The memory 5200 stores device information about at least one device.

The processor 5300 tracks the user's eye via the acquired image, and extracts feature points of the iris. The processor 5300 may analyze the acquired image and detect a change in the pupil of the user. Further, the processor 5300 may analyze the acquired image and extract feature points of the iris.

The processor 5300 determines, based on the eyes of the user, a first electronic device and a second electronic device to be paired. The first electronic device is an electronic device that is playing back content under the control of the glasses 5000, and the second electronic device is an electronic device for seamlessly playing back content that is being played back at the first electronic device. In other words, the first electronic device and the second electronic device are electronic devices that the user wearing the glasses 5000 sequentially gazes at.

The processor 5300 performs user authentication with the first and second electronic devices by using the feature points of the iris of the user wearing the glasses 5000. Processor 5300 performs iris authentication to determine that the user wearing glasses 5000 is authorized to control the first and second electronic devices. The processor 5300 may perform iris authentication by transmitting the feature points of the iris to the first and second electronic devices.

The processor 5300 controls the first and second electronic devices to pair with each other. The processor 5300 controls the first and second electronic devices such that the second electronic device seamlessly plays back the content being played back by the first electronic device. The processor 5300 may transmit device information about the second electronic device to the first electronic device. Further, the processor 5300 may transmit device information about the first electronic device to the second electronic device.

Fig. 47 is a flowchart for describing a method of seamlessly playing back content according to an embodiment.

In operation 4710, the electronic device 8000 receives iris information about the user wearing the glasses 5000 and identifies the user. The electronic device 8000 determines whether the user is authorized to control the glasses 5000, and transmits a result of the iris authentication to the glasses 5000.

In operation 4720, the electronic device 8000 receives device information about a previous electronic device used by the user from the glasses 5000. If the user authentication is successful, the electronic device 8000 may request the glasses 5000 for information about previous electronic devices previously controlled by the user controller.

In operation 4730, the electronic device 8000 is paired with the previous electronic device by using device information on the previous electronic device. The electronic device 8000 attempts connection to a previous electronic device based on device information about the previous electronic device.

In operation 4740, the electronic device 8000 plays back content that was being played back at the previous electronic device. The electronic device 8000 may receive content from a previous electronic device and play back the received content. Alternatively, if the same content as that being played back at the previous electronic device is stored in the electronic device 8000, the electronic device 8000 may play back the stored content.

Fig. 48 is a flowchart for describing a method of pairing electronic devices according to an embodiment.

In operation 4810, the glasses 5000 tracks eyes of a user wearing the glasses 5000, and determines a first electronic device to pair.

In operation 4820, the glasses 5000 performs user authentication with the first electronic device by using the feature points of the iris of the user. If the user authentication is successful, the glasses 5000 controls the first electronic device. For example, the glasses 5000 may control the first electronic device to play back a moving image or music.

In operation 4830, the glasses 5000 tracks a change in the eyes of the user and determines a second electronic device to pair. The second electronic device is an electronic device different from the first electronic device, and may be connected to the same network as the first electronic device.

In operation 4840, the glasses 5000 performs user authentication with the second electronic device by using the feature points of the iris of the user. If the user authentication is successful, the glasses 5000 controls the second electronic device.

In operation 4850, the glasses 5000 controls the first electronic device and the second electronic device such that the first electronic device and the second electronic device are paired with each other. The glasses 5000 may transmit device information about the first electronic device to the second electronic device. The second electronic device may attempt to pair with the first electronic device by using the device information about the first electronic device.

Fig. 49 is a diagram for describing a pairing method and a seamless content playback method according to an embodiment. Referring to fig. 49, the smart phone 7000 may pair with the TV6000 by recognizing the iris of the user and tracking the eyes of the user, wherein the smart phone 7000 may transmit the content to the TV 6000. Further, the TV6000 can pair with the smart phone 7000 by recognizing the iris of the user and tracking the eyes of the user, wherein the TV6000 can seamlessly play back the content being played back at the smart phone 7000. The smart phone 7000 may identify the user by identifying the iris of the user. The smart phone 7000 may include an iris recognition unit such as a camera, and recognize the iris of the user via the camera. The smart phone 7000 may extract an image of the iris of the user by analyzing the image acquired through the camera. The smart phone 7000 may identify the user by comparing the extracted iris image with the stored iris image. The smart phone 7000 may store images of the irises of a plurality of users.

The smart phone 7000 may track the eyes of the user. The smartphone 7000 may track the user's eyes based on changes in the image acquired by the camera. The smart phone 7000 can determine the direction in which the user looks by tracking the eyes of the user and determine the electronic device located in the direction in which the user looks as the electronic device to be paired. For example, if the user gazes at TV6000, the smart phone 7000 may determine the TV6000 as the electronic device to be paired.

The TV6000 can recognize the user by recognizing the iris of the user. The TV6000 may include an iris recognition unit such as a camera, and recognize the iris of the user via the camera. The TV6000 may extract an image of the iris of the user by analyzing the image acquired by the camera. The TV6000 may recognize the user by comparing the extracted iris image with the stored iris image. The TV6000 may store images of irises of a plurality of users.

The TV6000 can track the eyes of the user. The TV6000 can track the eyes of the user based on the images acquired by the camera. The TV6000 can determine whether the user is gazing at the TV6000 by tracking the eyes of the user, and if the user is gazing at the TV6000, the TV6000 can perform iris authentication.

In addition, the TV6000 can determine the electronic devices that the user has previously used. The TV6000 transmits information on the identified user to nearby electronic devices, and may receive data for pairing from an electronic device, among the nearby electronic devices, that authenticates the identified user.

The TV6000 can pair with nearby electronic devices by using data received from the nearby electronic devices. For example, the TV6000 may receive a MAC address and a device ID from a nearby electronic device and pair with the nearby electronic device. For example, if the user is watching TV6000, the TV6000 may broadcast user information or connection information to nearby electronic devices. The TV6000 can receive the device information from the smart phone 7000 and pair with the smart phone 7000 based on the received device information.

Fig. 50 is a flowchart for describing a method in which the smart phone 7000 is paired with the TV6000 and the smart phone 7000 transfers content to the TV 6000. Referring to fig. 50, the smart phone 7000 may track the eyes of the user and transmit the content to the TV 6000.

In operation 5010, the smart phone 7000 performs identification of the iris of the user and user authentication with respect to the user. The smart phone 7000 may recognize the iris of the user via the camera and perform user authentication by using the recognized iris.

In operation 5020, the smart phone 7000 plays back the content. The smart phone 7000 may play back and forth movie, drama or music based on input from the user.

In operation 5030, the smartphone 7000 tracks the user's eyes and determines the electronic device to be paired. The smartphone 7000 may track the user's eyes via the camera. Based on the result of tracking the user's eyes, the smart phone 7000 can determine the electronic device at which the user gazes as the electronic device to be paired. For example, the smart phone 7000 may determine the TV6000 as the electronic device to be paired.

In operation 5040, the smart phone 7000 is paired with the TV 6000. The smart phone 7000 can broadcast the connection information and the iris information to the TV 6000. The TV6000 may also recognize the iris of the user and determine whether the corresponding user is the same as the user authenticated by the smart phone 7000.

In operation 5050, the smart phone 7000 transfers the content to the TV 6000. If the TV6000 stores the same content, the smart phone 7000 can transmit information on the content to the TV 6000. The smart phone 7000 can transmit information on the content being played back to the TV6000 and receive information indicating whether the corresponding content is stored in the TV6000 from the TV 6000. The smart phone 7000 can additionally transmit bookmark information to the TV 6000.

In operation 5060, the TV6000 plays back the content received from the smart phone 7000. The TV6000 can determine a playback point of the content based on the received bookmark information.

Accordingly, the user can seamlessly view the content via the smart phone 7000 and the TV 6000.

Fig. 51A and 51B are diagrams for describing a method of determining that an electronic device plays back content seamlessly. Referring to fig. 51A and 51B, the smart phone 7000 can track the eyes of the user and transmit content to the TV6000 to play back the content seamlessly.

At step 1, the user watches TV6000 while the user watches the content via smart phone 7000.

In step 2, the smart phone 7000 displays a content streaming (streaming) button. Since the user is watching TV6000, the smart phone 7000 displays an icon corresponding to the TV6000 at the content streaming button 7100. If there are a plurality of electronic devices in the direction in which the user views, the smart phone 7000 displays content streaming buttons indicating the plurality of electronic devices, respectively. The smart phone 7000 may determine the electronic device to be displayed at the streaming button 7100 by directly tracking the user's eyes, or display the electronic device that detected the user's eyes at the streaming button 7100 according to a request from the electronic device that detected the user's eyes.

At step 3, the smart phone 7000 receives input from the user. The user may select an electronic device to which to stream the content by touching a button displayed at the smart phone 7000.

At step 4, the smart phone 7000 streams the content to the selected electronic device. For example, the smart phone 7000 can stream the content to the TV 6000.

Fig. 52A and 52B are diagrams for describing a method of determining an electronic device to play back content seamlessly.

At step 1, the smart phone 7000 detects a long press by the user on the button 7200. The user may long press a button 7200 displayed on the touch screen of the smart phone 7000 to share the content.

At step 2, the smart phone 7000 tracks the user's eyes while the user presses the button 7200. While the user presses the button 7200, the user gazes at the electronic device to which the content is to be streamed. For example, while the user presses the button 7200 of the smart phone 7000, the user watches the TV 6000. Alternatively, if the user keeps pressing the button 7200, the smart phone 7000 determines whether the nearby electronic device detects the user's eyes. In other words, if a nearby electronic device detects the user's eyes, the nearby electronic device may transmit the result of the detection to the smart phone 7000.

In step 3, if the TV6000 is determined as the electronic apparatus to which the content is to be streamed, the smart phone 7000 displays the TV6000 on the touch screen and generates a vibration, thereby notifying the user of the electronic apparatus to which the content is determined to be streamed. The smart phone 7000 may provide feedback to the user not only via vibrations but also via sound or images.

In step 4, when the user finishes touching the button 7200, the smart phone 7000 streams the content to the TV 6000. The TV6000 plays back the streamed content.

Fig. 53 is a flowchart for describing a method in which the smart phone 7000 is paired with the TV6000 and the smart phone 7000 transfers content to the TV 6000. Referring to fig. 53, the TV6000 may detect the eyes of the user and receive the content from the smart phone 7000.

In operation 5310, the smart phone 7000 performs identification of the iris of the user and user authentication with respect to the user. The smart phone 7000 may recognize the iris of the user via the camera and perform user authentication by using the recognized iris.

In operation 5320, the smart phone 7000 plays back the content. The smart phone 7000 may play back and forth movie, drama or music based on input from the user.

In operation 5330, the TV6000 performs recognition of the iris of the user and user authentication with respect to the user. The TV6000 may recognize an iris of the user via the camera and perform user authentication by using the recognized iris.

In operation 5340, the TV6000 may broadcast information about the identified irises to nearby electronic devices. The TV6000 searches for electronic devices previously used by the user.

In operation 5350, the smart phone 7000 compares the iris information received from the TV6000 with the iris information acquired via the camera. In other words, the smart phone 7000 determines whether the iris information acquired by the smart phone 7000 is the same as the iris information acquired by the TV 6000.

In operation 5360, the smart phone 7000 transmits the result of the comparison to the TV 6000. If the irises of the same user are recognized by the smart phone 7000 and the TV6000, the smart phone 7000 can transmit device information for pairing to the TV 6000.

In operation 5370, the TV6000 determines electronic devices to be paired. The TV6000 determines an electronic device used by the user as an electronic device to be paired. Since the user views the content via the smart phone 7000, the TV6000 determines the smart phone 7000 as the electronic device to be paired.

In operation 5380, the TV6000 and the smart phone 7000 are paired with each other.

In operation 5390, the smart phone 7000 streams the content to the TV 6000. If the smart phone 7000 does not have to stream the content, the smart phone 7000 can transmit information only about the content to the TV 6000.

In operation 5395, the TV6000 plays back the streamed content. Alternatively, the TV6000 can play back the stored content, or can play back dramas or music.

Fig. 54 is a diagram for describing a seamless playback method using bookmarks.

Referring to fig. 54, the TV6000 and the smart phone 7000 can play back content seamlessly by using bookmark information about the user. Even if the user does not wear glasses, the TV6000 and the smart phone 7000 can recognize the iris or the eyes of the user and play back the content seamlessly.

The TV6000 recognizes the iris of the user and recognizes the user watching the TV 6000. For example, user a may be watching TV 6000. The TV6000 generates and stores a bookmark a with respect to the user a. Bookmark a indicates a playback point of the content with respect to user a.

The TV6000 tracks the eyes of the user a and determines whether the user a continuously watches the content. If it is determined that the user A is no longer watching the content, the TV6000 stores bookmark A for the user A and stops the playback of the content. The TV6000 generates a bookmark a indicating that a playback point at which the user a has suspended viewing of the content is determined. For example, if the eyes of user a are not detected, the TV6000 may determine that user a is no longer watching the content.

If it is determined that the user A watches the TV6000 again, the TV6000 plays back the content based on the bookmark A. Bookmark a includes information about the content that user a is viewing and information indicating a playback point at which user a discontinues viewing the content.

The smart phone 7000 can recognize the iris and eyes of the user a and seamlessly play back the content being played back at the TV 6000. The smart phone 7000 tracks the eyes of the user a and determines whether the user a wishes to watch the smart phone 7000. The smart phone 7000 identifies the iris of the user a and identifies the user a.

The smart phone 7000 is paired with the TV 6000. The smart phone 7000 can receive content from the TV6000 and play back the content, play back content stored in the smart phone 7000, or receive content from a server and play back the content.

The smart phone 7000 additionally receives bookmark information from the TV 6000. The smart phone 7000 receives the bookmark a about the user a from the TV6000 and determines the playback point of the content. The smart phone 7000 can play back the content based on the determined playback point.

Fig. 55 is a diagram for describing a seamless playback method with respect to a plurality of users.

The TV6000 can track eyes of a plurality of users and generate corresponding bookmark information about the plurality of users. The TV6000 continuously tracks the eyes of the user a and the user B watching the TV 6000. If so, determining; user a no longer views the content, the TV6000TV generates bookmark a, which includes a playback point that determines that user a no longer views the content. The TV6000 stores bookmark a for user a and plays back the content continuously because user B continuously watches the content. If it is determined that the user B is no longer watching the content, the TV6000 generates a bookmark B including a playback point at which it is determined that the user B is no longer watching the content. If it is determined that neither user A nor user B is no longer watching the content, the TV6000 may stop the playback of the content.

When the eyes of the user a are detected again, the TV6000 plays back the content based on the bookmark a. Alternatively, when the eyes of the user B are detected again, the TV6000 plays back the content based on the bookmark B.

Even if the user a or the user B views the content via an electronic device other than the TV6000, the electronic device can receive the bookmark a or the bookmark B from the TV6000 and play back the content seamlessly.

Fig. 56 is a diagram for describing an example of an electronic device 9000 which receives content from a nearby electronic device and plays back the content. Referring to fig. 56, an electronic device 9000 includes a communication unit 9100, a memory 9200, a display unit 9300, a camera 9400, and a processor 9500.

The communication unit 9100 may receive iris information or device information about a user from a nearby electronic device.

The communication unit 9100 receives content and bookmark information about the content from nearby electronic devices. The content may be media files such as music files, video files, text files, image files, and the like.

The memory 9200 stores iris information about at least one user. In addition, the memory 9200 can store content.

The display unit 9300 displays content under the control of the processor 9500. The display unit 9300 can display electronic devices to be paired or content streaming buttons.

The processor 9500 obtains an image of the user's eyes. The camera 9400 transmits the acquired images to the processor 9500.

The processor 9500 tracks the eyes of the user, and extracts feature points of the iris via an image acquired by the camera 9400. The camera 9400 performs user authentication based on iris information about the user.

The processor 9500 determines the electronic devices to be paired. The processor 9500 can determine the electronic devices to pair by tracking the eyes of the user. Further, when the eyes of the user are detected, the processor 9500 may broadcast the result of iris authentication to nearby electronic devices and determine an electronic device to be paired.

The processor 9500 can transmit content being played back to a nearby electronic device via the communication unit 9100. The processor 9500 transmits content to the electronic device at which the user is gazing.

The processor 9500 can play back content received from nearby electronic devices. The processor 9500 can play back content based on the received information on content and the received bookmark information.

The electronic device 9000 according to an embodiment can seamlessly play back content by tracking the eyes of the user.

The electronic device 9000 according to an embodiment may determine whether to play back content seamlessly via iris authentication.

As described above, according to one or more of the above-described exemplary embodiments, the mobile device may recognize the electronic device included in the input image and pair with the electronic device.

A mobile device according to embodiments may be paired with another electronic device controlled by the identified electronic device.

A mobile device according to embodiments may track the eyes of a user and select an electronic device to pair.

The mobile device according to the embodiment may select an electronic device to be paired from among a plurality of electronic devices included in the acquired image based on a selection of a user.

A mobile device according to embodiments may select electronic devices to pair based on gestures of a user.

The devices described herein may include a processor, memory for storing and executing program data, a persistent storage unit such as a disk drive, a communication port for handling communications with external devices, and user interface devices including touch panels, keys, buttons, and the like. When referring to software modules or algorithms, these software modules may be stored on a computer readable medium as program instructions or computer readable code executable on a processor. Examples of the computer readable recording medium include magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) and optical recording media (e.g., CD-ROMs, or DVDs). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. The medium may be read by a computer, stored in a memory, and executed by a processor.

The present principles may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of hardware and/or software components configured to perform the specified functions. For example, the present principles may employ various Integrated Circuit (IC) components, e.g., memory elements, processing elements, logic elements, look-up tables, or the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, when software programming or software elements are used to implement the elements of the present principles, the present principles may be implemented using various algorithms in any programming or scripting language, such as C, C + +, Java, and assembly, among others, with the various algorithms being implemented by data structures, objects, procedures, routines, or other programming elements. The functional aspects may be implemented in algorithms that are executed on one or more processors. Moreover, the present principles may be employed in any number of conventional techniques for electronic configuration, signal processing and/or control, data processing, etc. The words "mechanism," "element," "device," and "configuration" are used broadly and are not limited to mechanical or physical implementations, but may include software routines in conjunction with a processor, and the like.

The specific embodiments shown and described herein are illustrative of the principles of the present invention and are not intended to otherwise limit the scope of the principles of the present invention in any way. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems may not be described in detail. Furthermore, the connecting lines or connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements. It should be noted that many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device.

The use of the terms "a" and "an" and "the" and similar referents in the context of describing the principles of the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural. Furthermore, recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. Moreover, all method steps described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The present principles are not limited to the described sequence of steps. The use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate the inventive principles and does not pose a limitation on the scope of the inventive principles unless otherwise claimed. Many modifications and adaptations may be apparent to those of ordinary skill in the art without departing from the spirit and scope of the principles of the present invention.