阅读说明：本技术 一种指向遥控方法及系统 (Pointing remote control method and system ) 是由 胡玉冰 刘康仲 景涛 李亮 于 2020-04-30 设计创作，主要内容包括：本申请公开了一种指向遥控系统,包括：遥控设备和显示设备；遥控设备向所述显示设备发送图形化红外信号。显示设备通过多个红外接收器接收图形化红外信号。显示设备获取所述图形化红外信号在显示屏上的投影图形。显示设备根据所述投影图形,确定出遥控设备在显示屏上的指向位置。显示设备在显示屏的该指向位置上显示提示标记,该提示标记用于提示用户遥控设备在显示屏上的指向位置。这样,显示设备可以通过遥控设备发送的图形化红外信号识别遥控设备在显示设备的显示屏上的指向位置,简化了操作步骤。(The application discloses point to remote control system includes: a remote control device and a display device; the remote control device sends a graphical infrared signal to the display device. The display device receives the graphical infrared signals through a plurality of infrared receivers. And the display equipment acquires a projection graph of the graphical infrared signal on a display screen. And the display equipment determines the pointing position of the remote control equipment on the display screen according to the projection graph. The display device displays a prompt mark on the pointing position of the display screen, and the prompt mark is used for prompting the user of the pointing position of the remote control device on the display screen. Therefore, the display equipment can identify the pointing position of the remote control equipment on the display screen of the display equipment through the graphical infrared signal sent by the remote control equipment, and the operation steps are simplified.)

1. A pointing remote control system, comprising: a remote control device and a display device; wherein the content of the first and second substances,

the remote control equipment is used for sending graphical infrared signals to the display equipment;

the display device is used for receiving the graphical infrared signals through a plurality of infrared receivers;

the display equipment is also used for acquiring a projection graph of the graphical infrared signal on a display screen;

the display equipment is further used for determining the pointing position of the remote control equipment on the display screen according to the projection graph;

the display device is further configured to display a prompt mark at the pointing position of the display screen, where the prompt mark is used to prompt a user of the pointing position of the remote control device on the display screen.

2. The system of claim 1, wherein the remote control device is further configured to:

receiving a first input acted on a first key by a user;

responding to the first input, and sending control information corresponding to the first key to the display equipment;

the display device is further configured to:

determining a first control operation according to the control information corresponding to the first key and the pointing position;

the first control operation is executed.

3. The system of claim 2, wherein the remote control device is specifically configured to:

and responding to the first input, and sending control information corresponding to the first key to the display equipment through the graphical infrared signal.

4. The system of claim 2, wherein the remote control device is specifically configured to:

and responding to the first input, and sending control information corresponding to the first key to the display equipment through Bluetooth.

5. The system of claim 1, wherein the plurality of infrared receivers are arranged in an infrared receiver array, the infrared receiver array being coupled in the display screen.

6. The system of claim 1, wherein the plurality of infrared receivers are arranged around the display screen to form an infrared receiver bezel.

7. The system of claim 5, wherein the patterned infrared signal comprises a circular infrared signal; the projection graph comprises a circular projection graph or an elliptical projection graph;

the display device is specifically configured to:

and determining the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received graphical infrared signal on the display screen.

8. The system of claim 5, wherein the patterned infrared signal comprises a donut-shaped infrared signal; the projection graph comprises a circular ring projection graph or an elliptical ring projection graph, wherein the energy in different rings in the circular ring infrared signal is different;

the display device is specifically configured to:

acquiring positions of M receiving points covered by the annular projection graph or the elliptical projection graph on the infrared receiving array, wherein M is a positive integer;

taking the position of the receiving point as the circle center, and determining a circular ring corresponding to the receiving point by using the outer ring radius and the inner ring radius corresponding to the signal intensity of the circular ring-shaped infrared signal received by the receiving point;

and determining the overlapping areas of the M rings according to the rings corresponding to the M receiving points respectively, and determining the central position of the overlapping areas as the pointing position.

9. The system of claim 6, wherein the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern;

the display device is specifically configured to:

acquiring positions of four receiving points of the cross-shaped infrared signal received by the infrared receiving frame;

determining the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points;

and calculating the position of the intersection point in the cross-shaped graph, and determining the position of the intersection point as the pointing position.

10. The system of claim 2, wherein the first control operation comprises any one of: click operation, long press operation, frame selection operation and drag operation.

11. The system of claim 1, wherein the display device comprises any one of the following device types: displays, televisions, and tablets; the remote control device comprises any one of the following device types: remote controller, mouse, smart mobile phone.

12. A display device, comprising: the system comprises a processor, a display screen and a plurality of infrared receivers; wherein the content of the first and second substances,

the infrared receivers are used for receiving graphical infrared signals sent by the remote control equipment;

the processor is used for acquiring a projection graph of the graphical infrared signal on the display screen;

the processor is further configured to determine a pointing position of the remote control device on the display screen according to the projection graph;

the display screen is further used for displaying a prompt mark at the pointing position of the display screen, and the prompt mark is used for prompting a user of the pointing position of the remote control device on the display screen.

13. The display device according to claim 12, characterized in that the display device further comprises: a wireless communication module;

the wireless communication module is used for receiving control information corresponding to the first key sent by the remote control equipment;

the processor is further configured to determine a first control operation according to the control information corresponding to the first key and the pointing position;

the processor is further configured to perform the first control operation.

14. The display device according to claim 13, wherein the wireless communication module is specifically configured to:

and receiving control information corresponding to the first key sent by the remote control equipment through the graphical infrared signal.

15. The display device according to claim 13, wherein the wireless communication module is specifically configured to:

and receiving control information corresponding to the first key sent by the remote control equipment through Bluetooth.

16. The display device of claim 12, wherein the plurality of infrared receivers are arranged in an infrared receiver array, the infrared receiver array being coupled in the display screen.

17. The display device of claim 12, wherein the plurality of infrared receivers are arranged around the display screen to form an infrared receiver bezel.

18. The display device of claim 16, wherein the graphical infrared signal comprises a circular infrared signal; the projection graph comprises a graph projection or an oval projection graph;

the processor is specifically configured to:

and determining the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received circular infrared signal on the display screen.

19. The display device of claim 16, wherein the graphical infrared signal comprises a donut-shaped infrared signal; the projection graph comprises a circular ring projection graph or an elliptical ring projection graph, wherein the energy in different rings in the circular ring infrared signal is different;

the processor is specifically configured to:

acquiring positions of M receiving points covered by the circular ring-shaped graph or the elliptical ring projection graph on the infrared receiving array, wherein M is a positive integer;

taking the position of the receiving point as a circle, and determining a circular ring corresponding to the receiving point by using the outer ring radius and the inner ring radius corresponding to the signal intensity of the circular ring-shaped infrared signal received by the receiving point;

and determining the overlapping areas of the M rings according to the rings corresponding to the M receiving points respectively, and determining the central position of the overlapping areas as the pointing position.

20. The display device of claim 17, wherein the patterned infrared signals comprise cross-shaped infrared signals; the projected pattern comprises a cross pattern;

the processor is specifically configured to:

acquiring positions of four receiving points of the cross-shaped infrared signal received by the infrared receiving frame;

determining the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points;

and calculating the position of the intersection point in the cross-shaped graph, and determining the position of the intersection point as the pointing position.

21. The display device according to claim 13, wherein the first control operation includes any one of: click operation, long press operation, frame selection operation and drag operation.

22. The display device according to claim 12, characterized in that the display device comprises any of the following device types: displays, televisions, and tablets; the remote control device comprises any one of the following device types: remote controller, mouse, smart mobile phone.

23. A pointing remote control method, comprising:

the display equipment receives graphical infrared signals sent by the remote control equipment through a plurality of infrared receivers;

the display equipment acquires a projection graph of the graphical infrared signal on a display screen;

the display equipment determines the pointing position of the remote control equipment on the display screen according to the projection graph;

and the display equipment displays a prompt mark at the pointing position of the display screen, wherein the prompt mark is used for prompting a user of the pointing position of the remote control equipment on the display screen.

24. The method of claim 23, further comprising:

the display equipment receives control information corresponding to a first key sent by the remote control equipment;

the display equipment determines a first control operation according to the control information corresponding to the first key and the pointing position;

the display device performs the first control operation.

25. The method according to claim 24, wherein the receiving, by the display device, the control information corresponding to the first key sent by the remote control device specifically includes:

and the display equipment receives control information corresponding to the first key sent by the remote control equipment through the graphical infrared signal.

26. The method according to claim 24, wherein the receiving, by the display device, the control information corresponding to the first key sent by the remote control device specifically includes:

and the display equipment receives the control information corresponding to the first key sent by the remote control equipment through Bluetooth.

27. The method of claim 23, wherein the plurality of infrared receivers are arranged in an infrared receiver array, the infrared receiver array being coupled in the display screen.

28. The method of claim 23, wherein the plurality of infrared receivers are arranged around the display screen to form an infrared receiver bezel.

29. The method of claim 27, wherein the patterned infrared signal comprises a circular infrared signal; the projection graph comprises a graph projection or an oval projection graph;

the display device determines the pointing position of the remote control device on the display screen according to the projection graph, and the method specifically includes:

and the display equipment determines the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received circular infrared signal on the display screen.

30. The display device of claim 27, wherein the graphical infrared signal comprises a donut-shaped infrared signal; the projection graph comprises a circular ring projection graph or an elliptical ring projection graph, wherein the energy in different rings in the circular ring infrared signal is different;

the display device determines the pointing position of the remote control device on the display screen according to the projection graph, and the method specifically includes:

the display equipment acquires the positions of M receiving points covered by the annular projection graph or the elliptical projection graph on the infrared receiving array, wherein M is a positive integer;

the display device takes the position of the receiving point as the center of a circle, the receiving point receives the outer ring radius and the inner ring radius corresponding to the signal intensity of the annular infrared signal, and a circular ring corresponding to the receiving point is determined;

and the display equipment determines the overlapping areas of the M circular rings according to the circular rings corresponding to the M receiving points respectively, and determines the central position of the overlapping areas as the pointing position.

31. The method of claim 28, wherein the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern;

the display equipment obtains the projection graph of the graphical infrared signal on the display screen, and the method specifically comprises the following steps:

the display equipment acquires the positions of four receiving points of the cross-shaped infrared signal received on the infrared receiving frame;

the display equipment determines the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points;

the display device determines the pointing position of the remote control device on the display screen according to the projection graph, and the method specifically includes:

and calculating the position of the intersection point in the cross-shaped graph, and determining the position of the intersection point as the pointing position.

32. The method of claim 24, wherein the first control operation comprises any one of: click operation, long press operation, frame selection operation and drag operation.

33. The method of claim 23, wherein the display device comprises any one of the following device types: displays, televisions, and tablets; the remote control device comprises any one of the following device types: remote controller, mouse, smart mobile phone.

34. A chip system applied to a display device, the chip system comprising: a processor; the display device comprises a plurality of infrared receivers and a display screen; wherein the processor is configured to:

acquiring projection graphs of the graphical infrared signals sent by the remote control equipment and received by the plurality of infrared receivers on the display screen;

determining the pointing position of the remote control equipment on the display screen according to the projection graph;

and indicating the display screen to display a prompt mark at the pointing position, wherein the prompt mark is used for prompting a user of the pointing position of the remote control equipment on the display screen.

35. The chip system according to claim 34, further comprising a wireless communication module; the wireless communication module is used for receiving control information corresponding to the first key sent by the remote control equipment;

the processor is further configured to determine a first control operation according to the control information corresponding to the first key and the pointing position;

the processor is further configured to perform the first control operation.

36. The chip system according to claim 35, wherein the wireless communication module is specifically configured to: and receiving control information corresponding to the first key sent by the remote control equipment through the graphical infrared signal.

37. The chip system according to claim 35, wherein the wireless communication module is specifically configured to:

and receiving control information corresponding to the first key sent by the remote control equipment through Bluetooth.

38. The on-chip system of claim 34, wherein the patterned infrared signal comprises a circular infrared signal; the projection graph comprises a graph projection or an oval projection graph; the plurality of infrared receivers are arranged in an infrared receiver array, and the infrared receiver array is coupled in the display screen;

the processor is specifically configured to:

and determining the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received circular infrared signal on the display screen.

39. The on-chip system of claim 34, wherein the patterned infrared signal comprises a donut-shaped infrared signal; the projection graph comprises a circular ring projection graph or an elliptical ring projection graph, wherein the energy in different rings in the circular ring infrared signal is different; the plurality of infrared receivers are arranged in an infrared receiver array, and the infrared receiver array is coupled in the display screen;

the processor is specifically configured to:

acquiring positions of M receiving points covered by the annular projection graph or the elliptical projection graph on the infrared receiving array, wherein M is a positive integer;

taking the position of the receiving point as the circle center, and determining a circular ring corresponding to the receiving point by using the outer ring radius and the inner ring radius corresponding to the signal intensity of the circular ring-shaped infrared signal received by the receiving point;

and determining the overlapping areas of the M rings according to the rings corresponding to the M receiving points respectively, and determining the central position of the overlapping areas as the pointing position.

40. The on-chip system of claim 34, wherein the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern; the plurality of infrared receivers are arranged around the display screen to form an infrared receiver frame;

the processor is specifically configured to:

acquiring positions of four receiving points of the cross-shaped infrared signal received by the infrared receiving frame;

determining the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points;

and calculating the position of the intersection point in the cross-shaped graph, and determining the position of the intersection point as the pointing position.

41. The chip system according to claim 34, wherein the first control operation comprises any one of: click operation, long press operation, frame selection operation and drag operation.

42. A computer-readable storage medium comprising instructions that, when executed on a display device, cause the display device to perform the method of any of claims 23 to 24.

Technical Field

The present application relates to the field of communications technologies, and in particular, to a pointing remote control method and system.

Background

As display devices such as televisions enter the intelligent era, the simplified design of remote controllers has become a consensus in the industry. The remote controller is only provided with keys of direction control, confirmation, return, volume +, volume-, homepage and the like, and can realize all operations by matching with a user interface of an operating system on the display equipment.

Currently, a display device such as a television may display a user interface with a selection pane and display a selection box that may be used to represent the selection pane selected on the user interface. The user can move the option box on the option grid which the user wants to select through the direction control key by means of the remote controller. For more highly-tabbed user interfaces, such as full-keyboard input interfaces. The user needs to use the direction key on the remote controller for a long time and frequently to select the desired option.

Therefore, when the display device such as a television is operated by the remote controller, the operation steps are complicated.

Disclosure of Invention

The application provides a pointing remote control method and system, which realize that a display device can identify the pointing position of the remote control device on a display screen of the display device through a graphical infrared signal sent by the remote control device, and simplify operation steps.

In a first aspect, the present application provides a pointing remote control system comprising: a remote control device and a display device; wherein the remote control device is configured to send a graphical infrared signal to the display device. The display device is used for receiving the graphical infrared signals through a plurality of infrared receivers; the display device is also used for acquiring a projection graph of the graphical infrared signal on the display screen. The display device is further used for determining the pointing position of the remote control device on the display screen according to the projection graph. The display device is also used for displaying a prompt mark at the pointing position of the display screen, and the prompt mark is used for prompting a user of the pointing position of the remote control device on the display screen.

Wherein, this display device includes above-mentioned a plurality of infrared receivers and above-mentioned display screen.

In one possible implementation, the remote control device is further configured to: receiving a first input acted on a first key by a user; and responding to the first input, and sending control information corresponding to the first key to the display equipment. The display device is further configured to: determining a first control operation according to the control information corresponding to the first key and the pointing position; the first control operation is executed.

In a possible implementation manner, the remote control device is specifically configured to: and responding to the first input, and sending control information corresponding to the first key to the display equipment through the graphical infrared signal.

In a possible implementation manner, the remote control device is specifically configured to: and responding to the first input, and sending control information corresponding to the first key to the display equipment through Bluetooth.

In one possible implementation, the plurality of infrared receivers are arranged in an infrared receiver array, the infrared receiver array being coupled in the display screen.

In one possible implementation, the plurality of infrared receivers are arranged around the display screen to form an infrared receiver frame.

In one possible implementation, the patterned infrared signal comprises a circular infrared signal; the projected pattern includes a circular projected pattern or an elliptical projected pattern. The display device is specifically configured to: and determining the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received graphical infrared signal on the display screen.

In one possible implementation, the patterned infrared signal comprises a donut-shaped infrared signal; the projected pattern comprises a circular projected pattern or an elliptical projected pattern, wherein the energy within different rings in the circular infrared signal is different. The display device is specifically configured to: acquiring positions of M receiving points covered by the annular projection graph or the elliptical projection graph on the infrared receiving array, wherein M is a positive integer; taking the position of the receiving point as the circle center, and determining a circular ring corresponding to the receiving point by using the outer ring radius and the inner ring radius corresponding to the signal intensity of the circular ring-shaped infrared signal received by the receiving point; and determining the overlapping areas of the M rings according to the rings corresponding to the M receiving points respectively, and determining the central position of the overlapping areas as the pointing position.

In one possible implementation, the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern. The display device is specifically configured to: acquiring positions of four receiving points of the cross-shaped infrared signal received by the infrared receiving frame; determining the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points; the position of the intersection point in the cross-shaped pattern is calculated and determined as the pointing position.

In one possible implementation, the first control operation includes any one of: click operation, long press operation, frame selection operation and drag operation.

In one possible implementation, the display device includes any of the following device types: displays, televisions, and tablets; the remote control device includes any one of the following device types: remote controller, mouse, smart mobile phone.

The embodiment of the application provides a pointing remote control system, and display equipment can receive graphical infrared signals sent by remote control equipment through an infrared receiver array or an infrared receiver frame. The display device may determine the pointing position of the remote control device on the display device based on the projection of the graphical infrared signal on the infrared receiver array or the infrared receiver bezel. Therefore, the display equipment can identify the pointing position of the remote control equipment on the display screen of the display equipment through the graphical infrared signal sent by the remote control equipment, and the operation steps are simplified. In addition, the user can perform touch-type interactive operation (for example, clicking, frame selection, long-time pressing, dragging and the like) on the display device from a long distance through the remote control device.

In a second aspect, the present application provides a display device comprising: the device comprises a processor, a display screen and a plurality of infrared receivers. The infrared receivers are used for receiving graphical infrared signals sent by the remote control equipment. The processor is used for acquiring a projection graph of the graphical infrared signal on the display screen. The processor is further configured to determine a pointing position of the remote control device on the display screen based on the projected pattern. The display screen is also used for displaying a prompt mark at the pointing position of the display screen, and the prompt mark is used for prompting a user of the pointing position of the remote control equipment on the display screen.

In one possible implementation, the display device further includes: and a wireless communication module. The wireless communication module is used for receiving control information corresponding to the first key sent by the remote control equipment. The processor is further configured to determine a first control operation according to the control information corresponding to the first key and the pointing position. The processor is also configured to perform the first control operation.

In a possible implementation manner, the wireless communication module is specifically configured to: and receiving control information corresponding to the first key sent by the remote control equipment through the graphical infrared signal.

In a possible implementation manner, the wireless communication module is specifically configured to: and receiving control information corresponding to the first key sent by the remote control equipment through Bluetooth.

In one possible implementation, the plurality of infrared receivers are arranged in an infrared receiver array, the infrared receiver array being coupled in the display screen.

In one possible implementation, the plurality of infrared receivers are arranged around the display screen to form an infrared receiver frame.

In one possible implementation, the patterned infrared signal comprises a circular infrared signal; the projected pattern includes a pattern projection or an elliptical projected pattern. The processor is specifically configured to: and determining the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received circular infrared signal on the display screen.

In one possible implementation, the patterned infrared signal comprises a donut-shaped infrared signal; the projected pattern comprises a circular projected pattern or an elliptical projected pattern, wherein the energy within different rings in the circular infrared signal is different. The processor is specifically configured to: acquiring positions of M receiving points covered by the circular ring-shaped graph or the elliptical ring projection graph on the infrared receiving array, wherein M is a positive integer; taking the position of the receiving point as a circle, and determining a circular ring corresponding to the receiving point by using the outer ring radius and the inner ring radius corresponding to the signal intensity of the circular ring-shaped infrared signal received by the receiving point; and determining the overlapping areas of the M rings according to the rings corresponding to the M receiving points respectively, and determining the central position of the overlapping areas as the pointing position.

In one possible implementation, the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern. The processor is specifically configured to: acquiring positions of four receiving points of the cross-shaped infrared signal received by the infrared receiving frame; determining the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points; the position of the intersection point in the cross-shaped pattern is calculated and determined as the pointing position.

In one possible implementation, the first control operation includes any one of: click operation, long press operation, frame selection operation and drag operation.

In one possible implementation, the display device includes any of the following device types: displays, televisions, and tablets; the remote control device includes any one of the following device types: remote controller, mouse, smart mobile phone.

The embodiment of the application provides a display device, and the display device can receive graphical infrared signals sent by a remote control device through an infrared receiver array or an infrared receiver frame. The display device may determine the pointing position of the remote control device on the display device based on the projection of the graphical infrared signal on the infrared receiver array or the infrared receiver bezel. Therefore, the display equipment can identify the pointing position of the remote control equipment on the display screen of the display equipment through the graphical infrared signal sent by the remote control equipment, and the operation steps are simplified. In addition, the user can conveniently perform touch interactive operation (such as clicking, frame selection, long-time pressing, dragging and the like) on the display device from a long distance through the remote control device.

In a third aspect, the present application provides a pointing remote control method, including: first, the display device receives a graphical infrared signal transmitted by the remote control device through a plurality of infrared receivers. Then, the display device obtains a projection graph of the graphical infrared signal on the display screen, and then the display device determines the pointing position of the remote control device on the display screen according to the projection graph. Then, the display device displays a prompt mark at the pointing position of the display screen, wherein the prompt mark is used for prompting a user of the pointing position of the remote control device on the display screen.

In one possible implementation, the method further includes: the display equipment receives control information corresponding to the first key sent by the remote control equipment. And the display equipment determines a first control operation according to the control information corresponding to the first key and the pointing position. The display device performs the first control operation.

In a possible implementation manner, the receiving, by the display device, control information corresponding to the first key sent by the remote control device specifically includes: the display equipment receives control information corresponding to the first key sent by the remote control equipment through the graphical infrared signal.

In a possible implementation manner, the receiving, by the display device, control information corresponding to the first key sent by the remote control device specifically includes: the display equipment receives control information corresponding to the first key sent by the remote control equipment through Bluetooth.

In one possible implementation, the plurality of infrared receivers are arranged in an infrared receiver array, the infrared receiver array being coupled in the display screen.

In one possible implementation, the plurality of infrared receivers are arranged around the display screen to form an infrared receiver frame.

In one possible implementation, the patterned infrared signal comprises a circular infrared signal; the projected pattern includes a pattern projection or an elliptical projected pattern. The display device determines the pointing position of the remote control device on the display screen according to the projection graph, and the method specifically includes: the display equipment determines the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received circular infrared signal on the display screen.

In one possible implementation, the patterned infrared signal comprises a donut-shaped infrared signal; the projected pattern comprises a circular projected pattern or an elliptical projected pattern, wherein the energy within different rings in the circular infrared signal is different. The display device determines the pointing position of the remote control device on the display screen according to the projection graph, and the method specifically includes: the display equipment determines the pointing position according to the annular projection graph or the elliptical projection graph and the infrared signal receiving intensity information in the projection graph.

In one possible implementation, the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern. This display device obtains the projection figure of this graphical infrared signal on the display screen, specifically includes: the display equipment acquires the positions of four receiving points of the cross-shaped infrared signal received on the infrared receiving frame; and the display equipment determines the cross-shaped graph of the graphical infrared signal on the display screen through the positions of the four receiving points. The display device determines the pointing position of the remote control device on the display screen according to the projection graph, and the method specifically includes: the position of the intersection point in the cross-shaped pattern is calculated and determined as the pointing position.

In one possible implementation, the first control operation includes any one of: click operation, long press operation, frame selection operation and drag operation.

In one possible implementation, the display device includes any of the following device types: displays, televisions, and tablets; the remote control device includes any one of the following device types: remote controller, mouse, smart mobile phone.

The embodiment of the application provides a pointing remote control method, and a display device can receive graphical infrared signals sent by a remote control device through an infrared receiver array or an infrared receiver frame. The display device may determine the pointing position of the remote control device on the display device based on the projection of the graphical infrared signal on the infrared receiver array or the infrared receiver bezel. Therefore, the display equipment can identify the pointing position of the remote control equipment on the display screen of the display equipment through the graphical infrared signal sent by the remote control equipment, and the operation steps are simplified. In addition, the user can conveniently perform touch interactive operation (such as clicking, frame selection, long-time pressing, dragging and the like) on the display device from a long distance through the remote control device.

In a fourth aspect, the present application provides a chip system applied to a display device, where the chip system includes: a processor; the display device includes a plurality of infrared receivers and a display screen. Wherein the processor is configured to: acquiring projection graphs of graphical infrared signals sent by a plurality of infrared receivers on a display screen, wherein the graphical infrared signals are received by the infrared receivers; determining the pointing position of the remote control equipment on the display screen according to the projection graph; and instructing the display screen to display a prompt mark at the pointing position, wherein the prompt mark is used for prompting the user of the pointing position of the remote control equipment on the display screen.

In a possible implementation manner, the system further comprises a wireless communication module; the wireless communication module is used for receiving control information corresponding to the first key sent by the remote control equipment. The processor is further configured to determine a first control operation according to the control information corresponding to the first key and the pointing position. The processor is also configured to perform the first control operation.

In a possible implementation manner, the wireless communication module is specifically configured to: and receiving control information corresponding to the first key sent by the remote control equipment through the graphical infrared signal.

In a possible implementation manner, the wireless communication module is specifically configured to: and receiving control information corresponding to the first key sent by the remote control equipment through Bluetooth.

In one possible implementation, the patterned infrared signal comprises a circular infrared signal; the projection pattern comprises a pattern projection or an elliptical projection pattern; the plurality of infrared receivers are arranged in an infrared receiver array, and the infrared receiver array is coupled in the display screen. The processor is specifically configured to: and determining the central position of the circular projection graph or the elliptical projection graph as the pointing position according to the circular projection graph or the elliptical projection graph of the received circular infrared signal on the display screen.

In one possible implementation, the patterned infrared signal comprises a donut-shaped infrared signal; the projection graph comprises a circular ring projection graph or an elliptical ring projection graph, wherein the energy in different rings in the circular ring infrared signal is different; the plurality of infrared receivers are arranged in an infrared receiver array, and the infrared receiver array is coupled in the display screen. The processor is specifically configured to: acquiring positions of M receiving points covered by a circular projection graph or an elliptical projection graph on an infrared receiving array, wherein M is a positive integer; taking the position of the receiving point as the circle center, and determining a circular ring corresponding to the receiving point by using the outer ring radius and the inner ring radius corresponding to the signal intensity of the circular ring-shaped infrared signal received by the receiving point; and determining the overlapping areas of the M rings according to the rings corresponding to the M receiving points respectively, and determining the central position of the overlapping areas as the pointing position.

In one possible implementation, the patterned infrared signal comprises a cross-shaped infrared signal; the projected pattern comprises a cross pattern; the plurality of infrared receivers are arranged around the display screen to form an infrared receiver frame. The processor is specifically configured to: acquiring positions of four receiving points of the cross-shaped infrared signal received by the infrared receiving frame; determining the cross-shaped graph of the graphical infrared signal on the display screen according to the positions of the four receiving points; the position of the intersection point in the cross-shaped pattern is calculated and determined as the pointing position.

In one possible implementation, the first control operation includes any one of: click operation, long press operation, frame selection operation and drag operation.

In a fifth aspect, an embodiment of the present application provides a display device, including: one or more functional units configured to perform the pointing remote control method in any of the possible implementations of any of the above aspects.

In a sixth aspect, an embodiment of the present application provides a computer storage medium, which includes computer instructions, and when the computer instructions are executed on a display device, the display device executes a pointing remote control method in any one of the possible implementations of any one of the foregoing aspects.

In a seventh aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, causes the computer to execute the pointing remote control method in any one of the possible implementations of any one of the above aspects.

Drawings

Fig. 1A is a schematic diagram of a hardware structure of a display device according to an embodiment of the present disclosure;

fig. 1B is a schematic structural diagram of an infrared receiver array according to an embodiment of the present disclosure;

fig. 1C is a schematic structural diagram of an infrared receiver frame according to an embodiment of the present disclosure;

fig. 1D is a schematic structural diagram of a direct type backlight liquid crystal display panel according to an embodiment of the present disclosure;

fig. 1E is a schematic structural diagram of a side-entry type backlight liquid crystal display panel according to an embodiment of the present application;

fig. 1F is a schematic structural diagram of an oled display panel according to an embodiment of the present disclosure;

fig. 1G is a schematic position diagram of a projection curtain and an infrared receiver array in projection display according to an embodiment of the present disclosure;

fig. 2 is a schematic hardware structure diagram of a remote control device according to an embodiment of the present application;

fig. 3A is a schematic diagram illustrating a principle of generating a circular infrared signal according to an embodiment of the present disclosure;

fig. 3B is a schematic diagram illustrating a generation principle of a circular ring-shaped infrared signal according to an embodiment of the present disclosure;

fig. 3C is a schematic diagram illustrating a principle of generating a cross-shaped infrared signal according to an embodiment of the present disclosure;

FIG. 3D is a schematic diagram illustrating another exemplary cross-shaped infrared signal generated according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a pointing remote control system according to an embodiment of the present application;

fig. 5A is a schematic view of an application scenario of an infrared receiving array according to an embodiment of the present disclosure;

fig. 5B is a schematic view of an application scenario of an infrared receiver frame according to an embodiment of the present disclosure;

fig. 6 is a schematic diagram of a projection principle provided in an embodiment of the present application;

7A-7C are schematic diagrams illustrating a set of computing pointing positions based on a circular projected pattern according to an embodiment of the present application;

FIGS. 8A-8C are schematic diagrams illustrating a set of computing pointing positions from an elliptical projected pattern according to an embodiment of the present application;

FIGS. 9A-9C are schematic diagrams illustrating a set of computing pointing positions from a torus projection pattern according to an embodiment of the present application;

FIGS. 10A-10D are schematic diagrams illustrating a set of computing pointing positions according to an elliptical ring projection pattern according to an embodiment of the present application;

11A-11B are schematic diagrams illustrating a set of computing pointing positions from a cross-shaped projection pattern according to an embodiment of the present application;

12A-12B are schematic diagrams illustrating another set of computing pointing positions from a cross-shaped projection pattern according to embodiments of the present application;

fig. 13 is a schematic flowchart of a pointing remote control method according to an embodiment of the present application;

fig. 14A-14F are schematic diagrams illustrating a set of human-machine interactions provided by an embodiment of the present application.

Detailed Description

The technical solution in the embodiments of the present application will be described in detail and removed with reference to the accompanying drawings. In the description of the embodiments herein, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.

In the following, the terms "first", "second" are used for descriptive purposes only and are not to be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of embodiments of the application, unless stated otherwise, "plurality" means two or more.

At present, an air mouse is available, which can move in the air. The air mouse can control a cursor on a screen of a display device by using data output by a sensor such as a gyroscope. The basic principle is as follows: the air mouse can map the angular velocity of the X axis and the angular velocity of the Z axis of the air mouse to the moving velocity of the air mouse, and the moving position of the cursor on the screen of the display device is calculated through the change of the relative position. Specifically, a digital processing chip of the air mouse converts a motion signal acquired by a sensor into a digital signal, and transmits a control signal to display equipment by using radio frequency wireless communication. The display device then determines and displays the position of the cursor based on the control signal. Because the air mouse carries out the positioning of the cursor through the motion data sensed by the sensor, the air mouse needs to display the initial position of the cursor on the screen of the display device firstly, and then the final position of the cursor can be determined according to the relative movement position of the translation operation after the translation operation of the slow movement of the user. Therefore, the accuracy of cursor positioning is not sufficient, and the user may be required to move the cursor to a specified position by moving the cursor several times.

The embodiment of the application provides a pointing remote control method and system, and a display device can receive graphical infrared signals sent by a remote control device through an infrared receiver array or an infrared receiver frame. The display device may determine the pointing position of the remote control device on the display device based on the projection of the graphical infrared signal on the infrared receiver array or the infrared receiver bezel. Therefore, the display equipment can identify the pointing position of the remote control equipment on the display screen of the display equipment through the graphical infrared signal sent by the remote control equipment, and the operation steps are simplified. In addition, the user can perform touch-type interactive operation (for example, clicking, frame selection, long-time pressing, dragging and the like) on the display device from a long distance through the remote control device.

A display device 100 provided in an embodiment of the present application is described below.

Fig. 1A shows a schematic structural diagram of the display device 100.

As shown in fig. 1A, the display device 100 may include a processor 111, a memory 112, a wireless communication processing module 113, a power switch 114, a display screen 115, an audio module 116, a speaker 117, and an infrared receiver 118.

Among other things, processor 111 may include one or more processing units, such as: the processor 111 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

Among the controllers may be the neural center and the command center of the display device 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 111 for storing instructions and data. In some embodiments, the memory in the processor 111 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 111. If the processor 111 needs to use the instruction or data again, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 111, thereby increasing the efficiency of the system.

The memory 112 is coupled to the processor 111 for storing various software programs and/or sets of instructions. The memory 112 may be used to store computer-executable program code, which includes instructions. The processor 111 executes various functional applications of the display device 100 and data processing by executing instructions stored in the memory 112. The memory 112 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like. The storage data area may store data (such as audio data, image data to be displayed, etc.) created during use of the display apparatus 100, and the like. Further, the memory 112 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like.

The wireless communication module 113 may provide solutions for wireless communication applied to the display device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like.

In some embodiments, the wireless communication processing module 113 may include a Bluetooth (BT) communication processing module 113A, WLAN communication processing module 113B, an infrared communication processing module 113C. One or more of the Bluetooth (BT) communication processing modules 113A, WLAN and the communication processing modules 113B may listen to signals transmitted by other devices, such as probe requests, scan signals, etc., and may transmit response signals, such as probe responses, scan responses, etc., so that the other devices may discover the display device 100 and establish wireless communication connections with the other devices, communicating with the other devices via one or more wireless communication technologies in bluetooth or WLAN. Among other things, the Bluetooth (BT) communication processing module 113A may provide solutions including one or more of classic Bluetooth (BR/EDR) or Bluetooth Low Energy (BLE) Bluetooth communication. The WLAN communication processing module 113B may include solutions for one or more of Wi-Fi direct, Wi-Fi LAN, or Wi-Fi softAP WLAN communications. The infrared communication processing module 113C may process the position where the infrared receiver 118 receives the infrared signal and receives the infrared signal.

A power switch 114 operable to control the power supplied by the power supply to the display device 100. In some embodiments, the power switch 114 may be used to control the power supplied by an external power source to the display device 100.

In some embodiments, the display device 100 may further include a battery, a charging management module, and a power management module. The battery may be used to provide power to the display device 100. The charging management module is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module may receive charging input from a wired charger via a USB interface. In some wireless charging embodiments, the charging management module may receive a wireless charging input through a wireless charging coil of the display device 100. The charging management module can charge the battery and supply power to the electronic equipment through the power management module. The power management module is used for connecting a battery, the charging management module and the processor 111. The power management module receives input from the battery and/or the charging management module to power the processor 111, the memory 112, the display 115, the wireless communication module 113, and the like. The power management module may also be used to monitor parameters such as battery capacity, battery cycle number, battery state of health (leakage, impedance), etc. In other embodiments, the power management module may also be disposed in the processor 111. In other embodiments, the power management module and the charging management module may be disposed in the same device.

And a display screen 115 for displaying images, video, etc. The display screen 115 includes a display panel. The display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), and the like.

And an infrared receiver 118 operable to receive the infrared signal. Illustratively, the infrared receiver 118 may receive a patterned infrared signal transmitted by the remote control device, such as a circular infrared signal, a cross-shaped infrared signal, and so forth. It should be noted that the figure may refer to a space shape which can be defined by a contour in a two-dimensional space.

In the embodiment of the present application, the display device 100 includes a plurality of infrared receivers 118. As shown in fig. 1B, the plurality of infrared receivers 118 may be arranged in a matrix arrangement as an infrared receiver array.

In other embodiments, as shown in FIG. 1C, the plurality of infrared receivers 118 may be arranged in a bezel around the display screen 115. In an embodiment of the present application, the plurality of infrared receivers 118 arranged on the bezel around the display screen 115 may be referred to as an infrared receiver bezel. In one possible implementation, when display device 100 is a projector, the plurality of infrared receivers 118 are arranged around the projection screen as an infrared receiver bezel of the projection screen.

In the present embodiment, when the display device 100 includes an infrared receiver array, the infrared receiver array may be coupled in the display screen 115.

Illustratively, as shown in fig. 1D, the display 115 may be a direct-lit backlight liquid crystal display. The direct-type liquid crystal display may include a back plate 121, a backlight plate 122, an optical film 124, and an LCD panel 125. The infrared receiver array 122 may be between the backlight panel 122 and the optical film 124. The back plate 121 may be used to support the entire direct-type lcd panel. The backlight panel 122 may be used to emit light to provide light to the LCD panel 125. The optical film material 124 can be used to excite the light of a single color emitted from the backlight panel 122 to generate a plurality of light sources of different color lights, and mix the different color lights to form a white light as the backlight source of the LCD panel. The LCD panel 125 can change the display color and brightness of each pixel on the display screen 115 under the control of the electrical signals.

As shown in fig. 1E, the display 115 may be a side-entry back-lit liquid crystal display. The side-entry type liquid crystal display panel may include a back plate 131, a Light Guide Plate (LGP)133, an optical film 134, and an LCD panel 135. The infrared receiver array 132 may be between the back plate 131 and the light guide plate 133. The back plate 131 may be used to support the entire side entry type lcd panel. The light guide plate 133 can be used to adjust the light emitted from the LED light bars distributed at the side of the display screen 115 to be incident into the LCD panel 135 from a specific direction. The optical film 134 can be used to excite the light of a single color emitted from the backlight panel 132 to generate a plurality of light sources of different color lights, and mix the different color lights to form a white light as the backlight source of the LCD panel. The LCD panel 135 can change the display color and brightness of each pixel on the display screen 115 under the control of electrical signals.

As shown in FIG. 1F, the display screen 115 may be a self-emissive display screen, such as an OLED display screen. The OLED display panel includes a back plate 141 and an OLED panel 143. The infrared receiver array 142 may be between the backplane 141 and the OLED panel 143. The back plate 141 may be used to support the entire OLED display. The OLED panel 143 may include a plurality of self-luminous sub-pixels, each of which can be independently illuminated. Due to the difference in the molecules of the organic material in the sub-pixels, the sub-pixels may have three types generating three primary colors of red, green, and blue (RGB), or four types generating four colors of red, green, blue, and white (RGBW). Each pixel on the OLED display screen may include sub-pixels of three primary colors, red, green, and blue (RGB), such that each pixel may display a different color. Alternatively, each pixel on the OLED display screen may include four color sub-pixels of red, green, blue, and white (RGBW), so that each pixel may display a different color.

In some embodiments, display device 100 may be a projection device such as a projector. As shown in fig. 1G, the display screen 115 of the display device 100 may be referred to as a projection screen 152. The infrared receiver array 151 may be behind a projection curtain 152.

The audio module 116 may be used to convert digital audio signals into analog audio signals for output, and may also be used to convert analog audio signals into digital audio signals for input. The audio module 116 may also be used to encode and decode audio signals. In some embodiments, the audio module 116 may be disposed in the processor 111, or some functional modules of the audio module 116 may be disposed in the processor 111. The audio module 116 can transmit the audio signal to the wireless communication module 113 through a bus interface (e.g., a UART interface, etc.), so as to implement the function of playing the audio signal through the bluetooth speaker.

The speaker 117 may be used to convert the audio signal transmitted by the audio module 116 into a sound signal.

In some embodiments, the display device 100 may also include a microphone, also referred to as a "microphone," for converting sound signals into electrical signals. When a voice control command occurs, a user can make a sound through the mouth of the user and input a sound signal into the microphone.

In some embodiments, the display device 100 may further include a wired Local Area Network (LAN) communication processing module, a High Definition Multimedia Interface (HDMI) communication processing module, and a communication serial bus (USB) communication processing module. The wired LAN communication processing module is operable to communicate with other devices in one LAN through a wired LAN, and is also operable to connect to a WAN through a wired LAN, and to communicate with devices in the WAN. The HDMI communication processing module can be used for communicating with other equipment through an HDMI interface. For example, the HDMI communication processing module may receive HDR video data transmitted by the set-top box through the HDMI interface, and so on. The USB communication processing module can be used for communicating with other equipment through a USB interface.

In the embodiment of the present application, the display device 100 may be any one of the following device types: displays, televisions, personal computers, tablet computers, projectors, and the like. When the display device 100 is a projector, the display screen of the display device 100 may be referred to as a projection screen.

A remote control device provided in an embodiment of the present application is described below.

Fig. 2 shows a schematic configuration of a remote control device 200.

As shown in fig. 2, the remote control device 200 may include: a processor 201, a memory 202, a wireless communication processing module 203, an infrared transmitter 204, a power module 205, and so forth. Wherein:

the processor 201 is operable to read and execute computer readable instructions. In particular implementations, the processor 201 may mainly include a controller, an operator, and a register. The controller is mainly responsible for instruction decoding. And sends out control signals for the operation corresponding to the instruction. The arithmetic unit is mainly responsible for storing register operands, intermediate operation results and the like temporarily stored in the instruction execution process. In a specific implementation, the hardware architecture of the processor 201 may be an Application Specific Integrated Circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

In some embodiments, the processor 201 may be configured to interpret signals received by the wireless communication processing module 203, such as bluetooth signals transmitted by the display device 100, and so on.

In some examples, the processor 201 may also be configured to generate signals sent out by the wireless communication processing module 203, such as bluetooth broadcast signals, infrared signals with control information, and so on.

A memory 202 is coupled to the processor 201 for storing various software programs and/or sets of instructions. In particular implementations, memory 202 may include high speed random access memory and may also include non-volatile memory. The memory 202 may store a communication program that may be used to communicate with the display device 100.

The wireless communication processing module 203 may include an infrared communication processing module 203C. In some embodiments, the wireless communication processing module 203 may also include one or more of a bluetooth communication processing module 203A, WLAN communication processing module 203B.

Among them, one or more of the bluetooth communication processing modules 203A, WLAN and the communication processing module 203B may listen to signals transmitted by other devices (e.g., the display device 100), such as probe requests, scanning signals, etc., and may send response signals, such as probe responses, scanning responses, etc., so that the other devices (e.g., the display device 100) may discover the remote control device 200 and establish a wireless communication connection. The bluetooth communication processing module 203 may provide a solution including one or more of classic bluetooth or Bluetooth Low Energy (BLE) bluetooth communication. The WLAN communication processing module 203B may include solutions for one or more of Wi-Fi direct, Wi-Fi LAN, or Wi-Fi softAP WLAN communications. The infrared communication processing module 203C may modulate the infrared signal with the control information and transmit the infrared signal with the control information through the infrared transmitter 204.

Wherein, the infrared emitter 204 may be an infrared light emitting diode for radiating an infrared signal outwards. In some embodiments, remote control device 200 may include a plurality of infrared emitters 204.

The power management module 205 may be used to connect a battery (not shown), a charge management module (not shown) and the processor 201. The power management module 205 receives input from a battery (not shown) and/or a charge management module (not shown) to power the processor 201, memory 202, wireless communication processing module 203, infrared transmitter 204, etc. The power management module 205 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc.

Remote control device 200 may also include one or more keys (e.g., an ok key, a home key, a volume + key, a volume-key, a channel + key, a channel-key, etc.), where the one or more keys may be physical keys. In some embodiments, the remote control device 200 may further include a touch screen, and the one or more keys may be virtual keys displayed on the touch screen.

In the embodiment of the present application, the remote control device 200 further includes one or more micro-lens structures (not shown). The one or more microlens structures may be placed in the direction of light emission from the infrared emitter 204. When the infrared emitter 204 can cooperate with the microlens structure to output an infrared signal having a specified pattern of a particular energy distribution.

Illustratively, as shown in FIG. 3A, the infrared emitter 204 may act as an infrared light source to emit an infrared signal. The infrared signal may form a circular infrared signal after passing through the microlens structure. Wherein, in the circular figure, the energy distribution of infrared signal is even.

As shown in FIG. 3B, infrared emitter 204 may act as an infrared light source, emitting an infrared signal. The infrared signal passing through the micro-lens structure can form a ring-shaped pattern infrared signal with a specified energy distribution. For example, the innermost ring of the ring pattern infrared signal has the highest energy, and the further the ring is, the lower the energy is.

As shown in fig. 3C, infrared emitter 204 may act as an infrared light source, emitting an infrared signal. The infrared signal may form a cross-shaped pattern of infrared signals after passing through the microlens structure. In the cross-shaped pattern, the energy distribution of the infrared signals is uniform.

In one possible implementation, remote control device 200 may output an infrared signal having a specified pattern of a specified energy distribution via a plurality of infrared emitters 204 and a plurality of microlens structures.

Illustratively, as shown in FIG. 3D, the remote control device 200 may include two IR emitters 204, and two microlens structures (microlens structure 1 and microlens structure 2). Wherein, the two infrared emitters 204 can be respectively used as two infrared light sources (infrared light source 1 and infrared light source 2) to emit infrared signals. After an infrared signal emitted by the infrared light source 1 passes through the micro-lens structure, an infrared signal of a negative 45-degree bar graph can be formed. After the infrared signal sent by the infrared light source 2 passes through the micro-lens structure, an infrared signal of a positive 45-degree bar-shaped graph can be formed. The infrared signals of the negative 45 degree bar pattern and the infrared signals of the positive 45 degree bar pattern may intersect to form a cross-shaped pattern of infrared signals. The above example shown in fig. 3D is merely used to explain the present application, and should not be construed as a limitation, and in a specific implementation, the intersection angle between two bars of the cross-shaped pattern may be other angles, which is not limited herein.

In the embodiment of the present application, when a plurality of infrared receivers 118 on the display device 100 are arranged in a matrix form as an infrared receiver array, coupled within the display screen 115, the remote control device 200 may employ a microlens structure and an infrared transmitter that output an infrared signal of any of the following patterns: infrared signals of a circular pattern, infrared signals of a circular ring pattern, infrared signals of a cross pattern, infrared signals of a square ring pattern, infrared signals of an H-shaped pattern, and the like. When the plurality of infrared receivers 118 on the display device 100 are arranged as an infrared receiving frame around the display screen 115, the remote control device 200 may employ a microlens structure and an infrared transmitter that output infrared signals in a pattern of a cross or the like.

In the present embodiment, the remote control device 200 may be any one of the following device types: remote controls (e.g., television remote controls), mice, smart phones, tablets, and the like. It should be noted that, when the remote control device 200 is a device with a touch screen, such as a smart phone or a tablet computer, one or more keys on the remote control device 200 may be virtual keys displayed on the touch screen.

A pointing remote control system provided in an embodiment of the present application will be described below.

Fig. 4 shows a schematic diagram of a framework of a pointing remote control system 400 according to an embodiment of the present application.

As shown in fig. 4, the pointing remote control system 400 may include a display device 100 and a remote control device 200. The display device 100 may include a processor 411, an infrared receiver array (or infrared receiver bezel) 412, a display screen 413, and a wireless communication module 414. The processor 411 may be a Micro Controller Unit (MCU). The wireless communication module 414 may include a 2.4G radio frequency transmission module (e.g., a bluetooth transmission module, an infrared transmission module, etc.). In one possible implementation, the wireless communication module 414 may include a Wi-Fi transmission module. For a specific description of the display device 100, reference may be made to the embodiment described in fig. 1A, which is not repeated herein.

The remote control device 200 may include a processor 421, an infrared transmitter 422, keys 423, and a wireless communication module 424. The processor 421 may be a Micro Controller Unit (MCU). Infrared emitter 422 in combination with the microlens structure can emit an infrared signal in a specified pattern. There may be one or more of the keys 423. The key 423 may be a physical key or a virtual key. The wireless communication module 424 may include a 2.4G radio frequency transmission module (e.g., a bluetooth transmission module, an infrared transmission module, etc.). The wireless communication module 424 may send control information corresponding to the key 423 to the display device 100 through a control signal. In one possible implementation, the wireless communication module 414 may include a Wi-Fi transmission module. For a specific description of the remote control device 200, reference may be made to the embodiment described in fig. 2, which is not described herein again.

The following describes the operation of the hardware on the display device 100 and the hardware on the remote control device 200 in detail with reference to the pointing remote control system 400.

The infrared transmitter 422 may send a patterned infrared signal having a specified energy distribution through the microlens structure 425 to the infrared receiver array (or infrared receiver bezel) 412. The patterned infrared signal may project a pattern of different shapes, such as a circle, a cross, etc., on the opposite projection plane due to the different microlens structures 425. When the patterned infrared signal is projected laterally onto the projection plane, the projection pattern on the projection plane may be an elliptical pattern, an elliptical ring pattern, a cross pattern, or the like.

The infrared receiver array (or infrared receiver frame) 412 may report the receiving information of the patterned infrared signal to the processor 411 after receiving the patterned infrared signal, where the receiving information may include the position information of the patterned infrared signal. In some embodiments, the received information may include location information of the received patterned infrared signal and received signal strength information of the patterned infrared signal.

The processor 411 may determine the pointing position of the remote control device 200 based on the received information of the infrared signal, and give the pointing position to the display 413. Display 413 may display a cursor at the pointing position. The cursor may be used to prompt the user that the remote control device 200 is pointing to the location on the display 413 where the cursor is located. For the process of determining the pointing position of the remote control device 200, reference may be made to the following embodiments, which are not described herein again.

The processor 421 may detect an operation (e.g., pressing, long-pressing, single-clicking, double-clicking, etc.) of the user through the key 423, and in response to the operation, the processor 421 may match a corresponding control instruction of the key 423. The processor 421 can instruct the wireless communication module 424 to send a control signal to the wireless communication module 414 according to the control instruction. The wireless communication module 414 may parse the control information from the control signal and send the parsed control information to the processor 411. The processor 411 may perform operations corresponding to the control information.

For example, as shown in FIG. 5A, the infrared receiver array on display device 100 may include a plurality of infrared receivers arranged in a rectangular array. The array of infrared receivers may be coupled within the display screen 413. The infrared transmitter 422 may send a patterned infrared signal to the infrared receiver array through the microlens structure 425. When the patterned infrared signal is projected onto the infrared receiver array, the patterned infrared signal can present a designated pattern on the plane of the display screen. For example, the patterned infrared signal may show a circular ring pattern on the plane of the display screen, and the signal strength of the circular ring pattern decreases (or increases) from the inner ring to the outer ring. The processor 411 may calculate the center position of the circular reception area from the reception information of the infrared signal, and use the center position of the reception area as the pointing position of the remote control device 200 on the display screen 413.

As shown in fig. 5B, a plurality of infrared receivers on the display device 100 may be arranged around the display screen to form an infrared receiver bezel. Infrared transmitter 422 may send a patterned infrared signal to the infrared receiver bezel via the microlens structure. When the patterned infrared signal is projected onto the display screen 413, the infrared signal may exhibit a specified pattern on the plane of the display screen. For example, the patterned infrared signal may appear as a cross-shaped pattern in the plane of the display screen. The processor 411 may calculate a cross point of the cross pattern based on the reception information of the infrared signal and use the cross point of the cross pattern as the pointing position of the remote control device 200 on the display screen 413.

In one possible implementation, when the wireless communication module 414 includes an infrared transmission module, the wireless communication module 414 may modulate an infrared signal with control information and transmit the modulated infrared signal to the infrared receiver array (or infrared receiver bezel) 412 via the plurality of infrared transmitters 412. The wireless communication module 414 may parse control information from the infrared signals received by the infrared receiver array (or infrared receiver bezel) 412 and provide the control information to the processor 411. The processor 411 may perform operations corresponding to the control information.

In the following, how the display device 100 determines the pointing position of the remote control device 200 on the display screen according to the received graphical infrared signal in the embodiment of the present application will be specifically described with reference to different graphical infrared signals.

In some application scenarios, an array of infrared receivers is coupled within a display screen of display device 100. An infrared transmitter on the remote control device 200 may emit a circular infrared signal through the microlens structure. When the display device 100 receives the infrared signal through the infrared receiver array, the center position of the infrared signal may be calculated according to the receiving area where the infrared signal is received, and the center position of the infrared signal may be used as the pointing position of the remote control device 200 on the display screen of the display device 100. In this way, the remote control device 200 can be accurately pointed at a position on the display screen of the display device 100 where the user wants to operate.

As shown in fig. 6, when the remote control device 200 is emitting a circular infrared signal to the display screen of the display device 100 as a point of emission, the projection of the infrared signal on the display screen of the display device 100 is a circular projection. When the remote control device 200 is a point of emission that laterally emits a circular infrared signal to the display screen of the display device 100, the projection of the infrared signal on the display screen of the display device 100 is an elliptical projection.

The display device 100 may determine, via the infrared receiving array, where one or more infrared receivers that detected the infrared signal are located. Then, the display device 100 may determine the receiving area of the infrared signal according to the position of the one or more infrared receivers that detect the infrared signal. The display apparatus 100 may calculate a center point of a reception area of the infrared signal from the reception area. Then, the display device 100 may use the center position of the receiving area as the center point of the infrared signal projected on the display screen, that is, the pointing position of the remote control device 200 on the display screen.

Illustratively, as shown in fig. 7A, the display device 100 may receive a circular infrared signal emitted by the remote control device 200 toward the display screen through the infrared receiving array. The projected pattern of the infrared signal on the infrared receiver array may be circular. Wherein the circular projection of the infrared signal may be overlaid with one or more infrared receivers on an infrared receiving array, e.g., four infrared receivers on the infrared receiving array detect the infrared signal.

Fig. 7B shows a portion of an infrared receiver array. As shown in fig. 7B, the circular projection of the infrared signal may be overlaid with four infrared receivers, e.g., infrared receiver 1, infrared receiver 2, infrared receiver 3, infrared receiver 4, on the infrared receiving array.

As shown in fig. 7C, the positions of the four infrared receivers, i.e., the infrared receiver 1, the infrared receiver 2, the infrared receiver 3, and the infrared receiver 4, may form a receiving area (e.g., a rectangular area). The display device 100 can calculate the center position of the receiving area according to the positions of the four receivers, which is the center point of the infrared signal projected on the display screen, that is, the pointing position of the remote control device 200 on the display screen.

As another example, as shown in fig. 8A, the display device 100 may receive an infrared signal emitted from the remote control device 200 to the display screen through an infrared receiving array. The projection pattern of the infrared signal on the infrared receiver array may be an ellipse. Wherein the circular projection of the infrared signal may be overlaid with one or more infrared receivers on an infrared receiving array, e.g., four infrared receivers that detect the infrared signal.

Fig. 8B shows a portion of an infrared receiver array. As shown in fig. 8B, the elliptical projection of the infrared signal may be overlaid with four infrared receivers on the infrared receiving array, e.g., four infrared receivers on the infrared receiving array detect the infrared signal.

As shown in fig. 8C, the positions of the four infrared receivers 1, 2, 3, 4 may form a receiving area (e.g., a rectangular area). The display device 100 can calculate the center position of the receiving area according to the positions of the four receivers, which is the center point of the elliptical projection of the infrared signal on the display screen, that is, the pointing position of the remote control device 200 on the display screen.

In some application scenarios, an array of infrared receivers is coupled within a display screen of display device 100. The remote control device 200 may emit a ring-shaped infrared signal through an infrared emitter and a microlens structure. Wherein, the signal intensity of the ring infrared signal decreases (or increases) from the inner ring to the outer ring. When the display device 100 receives the ring-shaped infrared signal through the infrared receiver array, the pointing position of the remote control device 200 on the display screen of the display device 100 can be determined according to the receiving area where the ring-shaped infrared signal is received and the signal intensity distribution in the receiving area. In this way, the remote control device 200 can be accurately pointed at a position on the display screen of the display device 100 where the user wants to operate.

The display device 100 may determine, through the infrared receiving array, positions of M receiving points in the infrared receiving array, where the infrared signals are detected, and signal strength values of the infrared signals received by the M receiving points. Then, the display device 100 may determine the pointing position of the remote control device 200 on the display screen according to the positions of the M receiving points where the infrared signal is detected and the signal strength values of the infrared signals received by the M receiving points, where M is a positive integer.

Illustratively, as shown in fig. 9A, the display device 100 may receive a circular ring-shaped infrared signal emitted by the remote control device 200 toward the display screen through the infrared receiving array. The projection pattern of the ring-shaped infrared signal on the infrared receiver array can be a ring shape. Wherein the circular projection of the infrared signal may be overlaid with M receiver points on an infrared receiver array, e.g., four receiver points on the infrared receiver array detect the infrared signal.

Fig. 9B shows a portion of an infrared receiver array. As shown in fig. 9B, the circular projection of the circular infrared signal may be overlaid with four infrared receivers on the infrared receiving array, which are the receiving points of the circular projection. For example, an infrared receiver 1 (i.e., reception point 1), an infrared receiver 2 (i.e., reception point 2), an infrared receiver 3 (i.e., reception point 3), and an infrared receiver 4 (reception point 4). Wherein, because the energy distribution of each ring of the ring-shaped infrared signal is different, the signal intensity of the infrared signal received by the four infrared receivers can be different.

For example, the annular projection may have three rings, an inner ring having the highest signal strength, a middle ring having the medium signal strength, and an outer ring having the lowest signal strength. The infrared receiver 1 is located at the outer ring of the circular projection, and the signal strength of the infrared signal received by the infrared receiver 1 can be-60 dBm. The infrared receiver 2 is located in the middle ring of the circular projection, and the signal strength of the infrared signal received by the infrared receiver 2 can be-40 dBm. The infrared receiver 3 is located in the middle ring of the circular projection, and the signal strength of the infrared signal received by the infrared receiver 3 can be-40 dBm. The infrared receiver 4 is positioned in the inner ring of the circular projection, and the signal strength of the infrared signal received by the infrared receiver 3 can be-20 dBm.

Wherein, the correspondence between the signal intensity and the inner and outer radii of the ring can be as shown in the following table 1:

TABLE 1

Signal strength	Inner radius of ring	Outer radius of ring
			[-20dBm，0)	0	12mm
[-40dBm，-20dBm)	12mm	16mm
			(-∞，-40dBm)	16mm	20mm

As can be seen from table 1, when the signal strength of the infrared receiver receiving the infrared signal is greater than or equal to-20 dBm, the inner radius of the ring corresponding to the infrared receiver is 0, and the outer radius is 12 mm. When the signal intensity of the infrared receiver receiving the infrared signal is larger than or equal to-40 dBm and smaller than-20 dBm, the inner radius of the corresponding circular ring of the infrared receiver is 12mm, and the outer radius of the corresponding circular ring of the infrared receiver is 16 mm. When the signal intensity of the infrared receiver receiving the infrared signal is less than-40 dBm, the inner radius of the corresponding ring of the infrared receiver is 16mm, and the outer radius is 20 mm. The above table 1 example is merely to illustrate the present application and should not be construed as limiting.

The display device 100 may use the position of the infrared receiver receiving the ring-shaped infrared signal as the center of a circle, and the outer radius and the inner radius of the circle corresponding to the signal receiving strength of the infrared receiver receiving the infrared signal are made into a circle. When only one infrared receiver receives the infrared signal, the display device 100 may set the position of the infrared receiver that has received the infrared signal uniquely as the pointing position of the remote control device 200 on the display screen. When a plurality of infrared receivers receive the ring-shaped infrared signal, the display device 100 may determine the overlapping area of the plurality of rings according to the respective rings corresponding to the plurality of infrared receivers that receive the ring-shaped infrared signal. The display device 100 can determine the overlapping area of the plurality of circles as the pointing position of the remote control device 200 on the display screen. In one possible implementation, the display device 100 may determine the geometric center point of the coinciding areas of the plurality of circles as the pointing position of the remote control device 200 on the display screen.

Illustratively, as shown in fig. 9C, for example, the received signal strength of the infrared receiver 1 may be-60 dBm, and the outer radius of the infrared receiver 1 corresponding to the ring 1 may be 20mm and the inner radius may be 16 mm. The signal strength received by the infrared receiver 2 can be-40 dBm, and the outer radius of the infrared receiver 2 corresponding to the circular ring 2 can be 16mm, and the inner radius is 12 mm. The signal strength received by the infrared receiver 3 can be-40 dBm, and the outer radius of the infrared receiver 3 corresponding to the circular ring 3 can be 16mm, and the inner radius is 12 mm. The signal strength received by the infrared receiver 4 can be-20 dBm, the outer radius of the infrared receiver 4 corresponding to the circular ring 4 can be 12mm, and the inner radius is 0. Wherein, when the inner radius of the circular ring 4 is 0, the circular ring 4 is circular.

The display device 100 may determine the overlapping area of the ring 1, the ring 2, the ring 3, and the ring 4 according to the ring 1, the ring 2, the ring 3, and the ring 4. The display device 100 may use the overlap area as the pointing position of the remote control device 200 on the display screen. In one possible implementation, the display device 100 may use the geometric center position of the coincident region as the pointing position of the remote control device 200 on the display screen.

In some embodiments, the display device 100 may determine that the rings corresponding to all the infrared receivers that detect the ring-shaped infrared signals do not have overlapping areas. The display apparatus 100 may determine a class a infrared receiver (e.g., the infrared receiver 2, the infrared receiver 4) having a received signal strength greater than a first signal strength (e.g., -30dBm) from among all the infrared receivers (e.g., the infrared receiver 1, the infrared receiver 2, the infrared receiver 3, the infrared receiver 4, the infrared receiver 5, and the infrared receiver 6) that receive the doughnut-shaped infrared signal. The display device 100 may calculate the coincidence area of the ring corresponding to the class a infrared receiver (e.g., infrared receiver 1, infrared receiver 2) and regard the coincidence area of the ring corresponding to the class a infrared receiver as the pointing position of the remote control device 200 on the display screen. In one possible implementation, the display device 100 may use the geometric center position of the coincident region as the pointing position of the remote control device 200 on the display screen.

Illustratively, as shown in fig. 10A, the display device 100 may receive a circular ring-shaped infrared signal emitted from the remote control device 200 toward the display screen through the infrared receiving array. The projection pattern of the annular infrared signal on the infrared receiver array can be an elliptical ring. The elliptical ring projection of the ring-shaped infrared signal may cover M receiving points on the infrared receiving array, for example, six receiving points on the infrared receiver array detect the infrared signal.

Fig. 10B shows a portion of an infrared receiver array. As shown in fig. 10B, the circular projection of the infrared signal may be overlaid with six receiving points on the infrared receiving array, for example, an infrared receiver 1 (i.e., receiving point 1), an infrared receiver 2 (i.e., receiving point 2), an infrared receiver 3 (i.e., receiving point 3), an infrared receiver 4 (receiving point 4), an infrared receiver 5 (i.e., receiving point 5), and an infrared receiver 6 (i.e., receiving point 6). Wherein, because the energy distribution of each ring of the ring-shaped infrared signal is different, the signal intensity of the infrared signal received by the four infrared receivers can be different.

For example, the elliptical ring projection may have three rings, with the inner ring having the highest signal strength, the middle ring having the medium signal strength, and the outer ring having the lowest signal strength. The infrared receiver 1, the infrared receiver 3, the infrared receiver 5 and the infrared receiver 6 are located at an outer ring of the circular projection, the signal strength of the infrared signal received by the infrared receiver 1 may be-60 dBm, the signal strength of the infrared signal received by the infrared receiver 3 may be-62 dBm, the signal strength of the infrared signal received by the infrared receiver 5 may be-64 dBm, and the signal strength of the infrared signal received by the infrared receiver 6 may be-65 dBm. The infrared receiver 4 is located in the middle ring of the circular projection, and the signal strength received by the infrared receiver 4 can be-40 dBm. The infrared receiver 2 is positioned in the inner ring of the circular projection, and the signal strength received by the infrared receiver 2 can be-20 dBm. In conjunction with table 1 above, the outer radius of ring 1, ring 3, ring 5, and ring 6 may be 20mm, and the inner radius may be 16 mm. The outer radius of the ring 4 may be 16mm and the inner radius may be 12 mm. The outer radius of the ring 2 may be 12mm and the inner radius may be 0. Wherein, when the inner radius of the ring 2 is 0, the ring 4 is circular.

As shown in fig. 10C, the display device 100 may determine that all infrared receivers (e.g., the infrared receiver 1, the infrared receiver 2, the infrared receiver 3, the infrared receiver 4, the infrared receiver 5, and the infrared receiver 6) that detect the infrared signal of the ring do not have a coincidence region with the corresponding ring (e.g., the ring 1, the ring 2, the ring 3, the ring 4, the ring 5, and the ring 6).

As shown in fig. 10D, the display device 100 may determine a class a reception point (e.g., the infrared receiver 2, the infrared receiver 4) having a received signal strength greater than a first signal strength (e.g., -30dBm) from among all reception points (e.g., the infrared receiver 1, the infrared receiver 2, the infrared receiver 3, the infrared receiver 4, the infrared receiver 5, and the infrared receiver 6) that receive the ring infrared signal. The display device 100 may calculate the coincidence area of the type a reception points (e.g., infrared receiver 2, infrared receiver 4) corresponding to the circle, and regard the coincidence area of the type a reception points corresponding to the circle as the pointing position of the remote control device 200 on the display screen. In one possible implementation, the display device 100 may use the geometric center position of the coincident region as the pointing position of the remote control device 200 on the display screen.

In some application scenarios, the plurality of infrared receivers on the display device 100 may be arranged on a bezel in display screen four. The remote control device 200 may emit an infrared signal in a cross pattern through an infrared emitter and a microlens structure. When the display device 100 receives the cross-shaped infrared signal through the infrared receiver frame, the intersection point of the cross-shaped pattern can be determined, and the intersection point can be used as the pointing position of the remote control device 200 on the display screen of the display device 100. In this way, the infrared signal emitted by the remote control device 200 may not pass through the display screen, so that the signal strength of the infrared signal received by the display device 100 is higher, and the user may accurately point to the position on the display screen of the display device 100 where the user wants to operate.

For example, as shown in fig. 11A, the display device 100 may receive a cross-shaped infrared signal transmitted by the remote control device 200 through an infrared receiving frame around the display screen. The projection of the cross-shaped infrared signal onto the infrared receiver array may be a cross. The cross projection of the infrared signal can be covered with four receiving points on the infrared receiving frame.

As shown in fig. 11B, four non-adjacent infrared receivers on the infrared receiving frame receive the infrared signals, for example, the infrared receiver 1, the infrared receiver 2, the infrared receiver 3, and the infrared receiver 4. The display device 100 may use the positions of the four complementary adjacent infrared receivers as four receiving points of the infrared signal. Display device 100 may determine a line intersection point located within the bezel of the infrared receiver with the four receive points connected two by two (e.g., receive point 1 and receive point 3, receive point 2 and receive point 4), and use the line intersection point as the pointing location of remote control device 200 on the display screen of display device 100.

In some embodiments, as shown in fig. 12A, the display device 100 may receive a cross-shaped infrared signal emitted from the remote control device 200 through an infrared receiving frame around the display screen. The infrared receiving frame has four non-adjacent receiving areas (e.g., receiving area 1, receiving area 2, receiving area 3, and receiving area 4) for receiving the infrared signals. Wherein, each receiving area comprises one or more infrared receivers for receiving the received infrared signals.

As shown in fig. 12B, four non-adjacent receiving areas on the infrared receiving frame receive the infrared signals, for example, receiving area 1, receiving area 2, receiving area 3, and receiving area 4. The receiving area 1 comprises an infrared receiver 1 and an infrared receiver 2, the receiving area 2 comprises an infrared receiver 3 and an infrared receiver 4, the receiving area 3 comprises an infrared receiver 5 and an infrared receiver 6, and the receiving area 4 comprises an infrared receiver 7 and an infrared receiver 8. The display device 100 may use the central point of each receiving area as a receiving point, and may connect two receiving points by two to determine a connection intersection point located within the infrared receiver frame. The display device 100 may use the intersection of the lines within the infrared receiver bezel as the pointing location of the remote control device 200 on the display screen of the display device 100.

A pointing remote control method provided in the embodiment of the present application is described below.

Fig. 13 is a flowchart illustrating a pointing remote control method according to an embodiment of the present application. Wherein, the pointing remote control method can be applied to a remote control system. The remote control system may include a display device 100 and a remote control device 200. The display device 100 may include an array of infrared receivers coupled within the display screen or an infrared receiving bezel disposed around the display screen. As shown in fig. 13, the method may include:

and S1301, transmitting the graphical infrared signal by the remote control device 200.

Illustratively, the patterned infrared signal may be a circular pattern of infrared signals, a cross pattern of infrared signals, and so forth. Specifically, reference may be made to the embodiments shown in fig. 3A to fig. 3D, which are not described herein again. In the embodiment of the present application, the patterned infrared signal may also be an infrared signal with other patterns, for example, a square ring pattern, an H-shaped pattern, an X-shaped pattern (not intersecting at 90 °), and so on. In some embodiments, the energy distribution in each ring of the circular or square ring infrared signals may be different, for example, the circular or square ring infrared signal energy decreases from inside to outside, or the circular or square ring infrared signal energy increases from inside to outside, and so on.

S1302, the display device 100 determines the pointing position of the remote control device 200 on the display screen according to the projection graph of the received graphical infrared signal on the display screen.

Specifically, the display device 100 may determine the pointing position of the remote control device 200 on the display screen according to the position information of the one or more infrared receivers that detected the projected pattern. For specific content, reference may be made to the embodiments shown in fig. 7A to 7C, or the embodiments shown in fig. 8A to 8C, or the embodiments shown in fig. 9A to 9C, or the embodiments shown in fig. 10A to 10D, or the embodiments shown in fig. 11A to 11B, or the embodiments shown in fig. 12A to 12B, which will not be described again here.

S1303, the display device 100 displays a prompt mark on the display screen at the pointing position of the remote control device 200.

Wherein the prompt indicia may be used to prompt the user of the pointing position of the remote control device 200 on the display screen of the display device 100. The cue marker may be an arrow, aperture, or the like type of marker.

Illustratively, as shown in FIG. 14A, the display device 100 may be displayed with a home interface 1410. The home interface 1410 includes a plurality of application icons (e.g., slide application icons, video application icons, music application icons, application store application icons, gallery application icons 1411, etc.). The main interface 1410 may also include a task bar 1412. The taskbar 1412 may include shortcuts for one or more applications (e.g., shortcuts for voice assistant applications, document applications, spreadsheet applications, etc.), status bars (including volume adjustment icons, Wi-Fi signal icons, time), and the like, among others. After the display device 100 determines the pointing position of the remote control device 200 on the display screen, the display device 100 may display an indication mark 1413 on the display screen. The index mark 1413 is used to indicate the pointing position of the remote control device 200 on the display screen.

In an embodiment of the present application, the steps S1301 to S1303 may be performed periodically. For example, the execution period of the above steps S1301 to S1303 may be 10 ms. In a possible implementation manner, the execution periods of the above steps S1301, S1302, and S1303 may be different. For example, the execution period of the step S1301 may be 26.3us, and the execution periods of the step S1302 and the step S1303 may be 10 ms.

S1304, the remote control device 200 receives an input applied to the first key by the user.

S1305, the remote control device 200 sends control information corresponding to the first key to the display device.

The remote control device 200 may include one or more keys, which may be physical keys.

As shown in fig. 14A, the remote control device 200 may include a power key 1401, a confirmation key 1402, a return key 1403, a home key 1404, a menu key 1405, a volume up key 1406, and a volume down key 1407. Wherein, the keys are physical keys.

In one possible implementation, the one or more keys may also be virtual keys. For example, the remote control device 200 may be a smart terminal with a touch screen, such as a mobile phone. Remote control device 200 may display one or more virtual keys via a touch screen.

The first key may be any one of the one or more keys described above. After receiving an input (e.g., a single click) from a user on a first key, the remote control device 200 may match control information corresponding to the first key. The remote control device 200 may send the control information corresponding to the first key to the display device 100 through a wireless communication technology such as a graphical infrared signal, bluetooth, WLAN direct connection, and the like.

When the remote control device 200 sends the control information corresponding to the first key to the display device 100 by using the graphical infrared signal, the control information corresponding to the first key may be an infrared remote control code. The remote control device 200 may first determine the infrared remote control code corresponding to the first key from the infrared remote control code library. Then, the remote control device 200 may modulate an infrared signal according to the infrared remote control code corresponding to the first key, and transmit the patterned infrared signal to the display device 100 through an infrared transmitter in combination with the microlens structure.

For example, the infrared remote control code library may be as shown in table 2 below:

TABLE 2

Push-button	Infrared remote control code
		Power key	0x01
Confirmation key	0x02
		Return key	0x03
Main page key	0x04
		Menu key	0x05
Volume plus key	0x06
		Volume reduction key	0x07
……	……

As can be seen from table 2, the infrared remote control code corresponding to the power key is "0 x 01". The infrared remote control code corresponding to the confirmation key is "0 x 02". The infrared remote control code corresponding to the return key is "0 x 03". The infrared remote control code corresponding to the home key is "0 x 04". The infrared remote control code corresponding to the menu key is '0 x 05'. The infrared remote control code corresponding to the volume up key is "0 x 06". The infrared remote control code corresponding to the volume down key is "0 x 07". The examples shown in table 2 above are merely for explaining the present application and should not be construed as limiting.

S1306, the display device 100 determines a first control operation according to the control information corresponding to the first key and the pointing position of the remote control device 200.

S1307, the display apparatus 100 performs the first control operation.

For example, the first control operation may be a click operation. As shown in fig. 14B, the display apparatus 100 may display an indication mark 1413 at the position of the gallery application icon 1411 when it is determined that the remote control apparatus 200 is pointed at the gallery application icon 1411. Then, after the display apparatus 100 receives the control information corresponding to the confirmation key 1402 sent by the remote control apparatus 200, the display apparatus 100 may determine that the first control operation is a click operation for the gallery application icon 1411.

As shown in fig. 14C, in response to the click operation for the gallery application icon 1411, the display device 100 may display a gallery application interface 1420. The gallery application interface 1420 may include thumbnails of one or more pictures (e.g., thumbnail 1421, thumbnail 1422, thumbnail 1423, thumbnail 1424, thumbnail 1425, thumbnail 1426), and one or more albums (e.g., family album 1431, landscape album 1432, favorite album 1433). Wherein, the number of pictures included in each photo album is displayed below each photo album. For example, the family album 1431 includes 2 pictures, the landscape album 1432 includes 4 pictures, and the favorite album includes 0 picture.

As another example, the first control operation may be a frame selection operation. As shown in fig. 14D, the user can press the enter key 1402 for a long time to adjust the pointing position of the remote control device 200 on the display device 100 from the initial position 1441 to the lower right corner position of the thumbnail 1424. When the display device 100 receives the control information corresponding to the confirmation key 1402 sent by the remote control device 200 for a long time (for example, more than 1s), and the position (for example, the position 1441) where the indication mark 1413 is located does not select an object, the display device 100 may determine that the first control operation is a frame selection operation. The display device 100 may determine the rectangular frame 1442 by using the position 1411 as an initial position, and a position where the indicator 1413 is detected when the user releases the enter key 1402 as an end position, and using the initial position and the end position as diagonal positions of the rectangle. The display apparatus 100 may combine one or more selection objects (e.g., the picture thumbnail 1423 and the picture thumbnail 1424) included in the rectangular frame 1442 into one combined object.

As another example, the first control operation may be a drag operation. As shown in fig. 14E, when the display device 100 receives the control information corresponding to the confirmation key 1402 sent by the remote control device 200 for a long time (for example, more than 1s), and the combined object frame 1443 exists at the position (for example, the position 1441) where the indicator 1413 is located, the display device 100 may determine that the first control operation is the drag operation. When the user controls the pointing mark 1413 to move on the display screen of the display device 100 through the remote control device 200, the display device 100 may control one or more selection objects (e.g., the thumbnail images 1423 and 1424) included in the combined object frame 1433 to move on the display screen following the pointing mark 1413.

As shown in fig. 14F, when the display device 100 detects that the drag operation is ended and the position of the indicator 1413 is located on the favorite album 1433, the display device 100 may store the picture corresponding to the thumbnail 1423 and the picture corresponding to the thumbnail 1424 in the combined object frame 1443 in the storage path corresponding to the favorite album 1433. When the user clicks on the album 1433 by operating the remote control device 200, the display device 100 may display the picture 1423 and the picture 1424.

The above example is only used for explaining the present application, and should not be construed as limiting, and in a specific implementation, the first control operation is not limited to the above operations of pointing, frame selection, and dragging, but may be other operations.

Through the pointing remote control method provided by the embodiment of the application, the display device 100 may receive the graphical infrared signal sent by the remote control device 200 through the infrared receiver array or the infrared receiver frame. The display device 100 may determine the pointing position of the remote control device 200 on the display device 100 based on the projection of the graphical infrared signals onto the infrared receiver array or infrared receiver bezel. In this way, the remote control device 200 can be accurately pointed at a position on the display screen of the display device 100 where the user wants to operate. In addition, the user can conveniently perform touch-type interactive operation (such as clicking, frame selection, long-time pressing, dragging and the like) on the display device 100 from a long distance through the pointing remote control device 200.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.