阅读说明：本技术 投影设备控制方法、校正方法、遥控装置以及投影设备 (Projection equipment control method, correction method, remote control device and projection equipment ) 是由 孙世攀 张聪 胡震宇 于 2021-08-17 设计创作，主要内容包括：本公开涉及一种投影设备控制方法、存储介质、遥控装置以及投影设备,涉及投影设备技术领域,通过获取遥控装置的位姿信息以及空间位置信息,并将该位姿信息以及空间位置信息发送至投影设备,可以控制投影设备根据该位姿信息以及空间位置信息调整投影设备的投影平面的中心点,使得投影设备的投影平面的中心点与遥控装置的指向中心点重合,从而实现投影设备的投影平面投影至遥控装置指向的方位,实现通过遥控装置控制投影设备,以提高用户在使用投影设备时的用户体验。(The utility model relates to a projection equipment control method, storage medium, remote control unit and projection equipment, relate to projection equipment technical field, through obtaining the position appearance information and the spatial position information of remote control unit, and send this position appearance information and spatial position information to projection equipment, can control projection equipment according to this position appearance information and spatial position information adjustment projection equipment's the central point of plane, make projection equipment's the central point of plane and the directional central point of remote control unit coincide, thereby realize projection equipment's projection plane projection to the directional position of remote control unit, realize controlling projection equipment through remote control unit, in order to improve the user experience of user when using projection equipment.)

1. A control method of a projection device is applied to a remote control device, and comprises the following steps:

determining pose information and spatial position information of the remote control device;

and sending the pose information and the spatial position information to a projection device so that the projection device adjusts the center point of the projection plane of the projection device according to the pose information and the spatial position information, and the center point of the projection plane is coincided with the pointing center point of the remote control device.

2. The method of claim 1, wherein the determining pose information for the remote control device comprises:

calculating to obtain a current pitch angle of the remote control device based on the first gravimeter and/or the first gyroscope;

and obtaining the current yaw angle of the remote control device based on the first yaw angle of the remote control device calculated by the first gyroscope and the second yaw angle of the remote control device calculated by the flight time sensor.

3. The method for controlling a projection apparatus according to claim 2, wherein the obtaining a current yaw angle of the remote control device based on a first yaw angle of the remote control device calculated by the first gyroscope and a second yaw angle of the remote control device calculated by a time-of-flight sensor comprises:

determining a current angular acceleration of the remote control device in a reference direction of yaw based on the first gyroscope;

determining a second yaw angle of the remote control device based on the time-of-flight sensor when the current hook angular acceleration is less than a preset threshold;

obtaining a first yaw angle of the remote control device calculated based on the first gyroscope, wherein the first yaw angle is the yaw angle of the remote control device at the last moment;

and calculating to obtain the current yaw angle of the remote control device based on the first yaw angle and the second yaw angle by combining a Kalman filtering algorithm.

4. The projection device control method according to claim 3, wherein the preset threshold is obtained by:

and obtaining the preset threshold value according to the ratio of the preset angle error to the refresh rate of the flight time sensor.

5. A projection device control method is applied to a projection device, and the method comprises the following steps:

receiving pose information and spatial position information of a remote control device sent by the remote control device;

determining coordinate information of a pointing center point of the remote control device according to the pose information and the spatial position information of the remote control device;

and adjusting the projection plane of the projection equipment according to the coordinate information so as to enable the center point of the projection plane of the projection equipment to be coincided with the pointing center point of the remote control device.

6. The projection apparatus control method according to claim 5, wherein the pose information of the remote control device includes a current pitch angle of the remote control device calculated based on a first gravimeter and/or a first gyroscope and a current yaw angle calculated based on a first yaw angle of the remote control device calculated based on the first gyroscope and a second yaw angle of the remote control device calculated based on a time-of-flight sensor;

the determining the coordinate information of the pointing center point of the remote control device according to the pose information and the spatial position information of the remote control device comprises:

and determining coordinate information of a pointing center point of the remote control device according to the current pitch angle, the current yaw angle and the spatial position information of the remote control device.

7. A method for calibrating a projection device, comprising:

calculating to obtain a current pitch angle and a current roll angle of the projection equipment based on the second gravity meter and/or the second gyroscope;

obtaining a current yaw angle of the projection equipment based on a first yaw angle of the projection equipment calculated by the second gyroscope and a second yaw angle of the projection equipment calculated by the flight time sensor;

and correcting the projection image of the projection equipment according to the current pitch angle, the current roll angle and the current yaw angle.

8. The projection correction method according to claim 7, wherein the obtaining of the current yaw angle of the projection device based on the first yaw angle of the projection device calculated by the second gyroscope and the second yaw angle of the projection device calculated by the time-of-flight sensor comprises:

determining a current angular acceleration of the projection device in a reference direction of yaw based on the second gyroscope;

determining a second yaw angle of the projection device based on the time-of-flight sensor when the current hook angular acceleration is less than a preset threshold;

obtaining a first yaw angle of the projection equipment through calculation of the second gyroscope, wherein the first yaw angle is the yaw angle of the projection equipment at the last moment;

and calculating to obtain the current yaw angle of the projection equipment based on the first yaw angle and the second yaw angle by combining a Kalman filtering algorithm.

9. The projection correction method according to claim 8, wherein the preset threshold is obtained by:

and obtaining the preset threshold value according to the ratio of the preset angle error to the refresh rate of the flight time sensor.

10. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that the program, when being executed by a processor, is adapted to carry out the steps of the projection device control method of any one of claims 1-6 and/or the steps of the projection device correction method of any one of claims 7-9.

11. A remote control device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the projection device control method of any of claims 1-4.

12. A projection device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the projection device control method of any of claims 5-6 and/or to implement the steps of the projection device correction method of any of claims 7-9.

Technical Field

The disclosure relates to the technical field of projection equipment, in particular to a projection equipment control method, a correction method, a remote control device and projection equipment.

Background

The projection device is a device capable of projecting images or videos onto a curtain, and is widely applied to scenes such as offices, schools, meeting rooms and the like. During use, the projection angle of the projection device may need to be adjusted. In the related art, a user is generally required to manually adjust the position of the projection device to adjust the projection angle, so as to change the orientation of the projected image. However, this method not only needs manual work by the user, but also has a high requirement on the placement position of the projection device, which results in an extremely poor user experience.

Disclosure of Invention

An object of the present disclosure is to provide a projection apparatus control method, a correction method, a remote control device, and a projection apparatus, to partially or wholly solve the above technical problems.

According to a first aspect of the embodiments of the present disclosure, there is provided a method for controlling a projection apparatus, which is applied to a remote control device, the method including:

determining pose information and spatial position information of the remote control device;

and sending the pose information and the spatial position information to a projection device so that the projection device adjusts the center point of the projection plane of the projection device according to the pose information and the spatial position information, and the center point of the projection plane is coincided with the pointing center point of the remote control device.

Optionally, the determining pose information of the remote control device includes:

calculating to obtain a current pitch angle of the remote control device based on the first gravimeter and/or the first gyroscope;

and obtaining the current yaw angle of the remote control device based on the first yaw angle of the remote control device calculated by the first gyroscope and the second yaw angle of the remote control device calculated by the flight time sensor.

Optionally, the obtaining a current yaw angle of the remote control device based on the first yaw angle of the remote control device calculated by the first gyroscope and the second yaw angle of the remote control device calculated by the time-of-flight sensor includes:

determining a current angular acceleration of the remote control device in a reference direction of yaw based on the first gyroscope;

determining a second yaw angle of the remote control device based on the time-of-flight sensor when the current hook angular acceleration is less than a preset threshold;

obtaining a first yaw angle of the remote control device calculated based on the first gyroscope, wherein the first yaw angle is the yaw angle of the remote control device at the last moment;

and calculating to obtain the current yaw angle of the remote control device based on the first yaw angle and the second yaw angle by combining a Kalman filtering algorithm.

Optionally, the preset threshold is obtained by:

and obtaining the preset threshold value according to the ratio of the preset angle error to the refresh rate of the flight time sensor.

According to a second aspect of the embodiments of the present disclosure, there is provided a projection apparatus control method, applied to a projection apparatus, the method including:

receiving pose information and spatial position information of a remote control device sent by the remote control device;

determining coordinate information of a pointing center point of the remote control device according to the pose information and the spatial position information of the remote control device;

and adjusting the projection plane of the projection equipment according to the coordinate information so as to enable the center point of the projection plane of the projection equipment to be coincided with the pointing center point of the remote control device.

Optionally, the pose information of the remote control device includes a current pitch angle and a current yaw angle of the remote control device, wherein the current pitch angle is calculated by the remote control device based on a first gravitometer and/or a first gyroscope, and the current yaw angle is calculated by the remote control device based on a first yaw angle of the remote control device calculated by the first gyroscope and a second yaw angle of the remote control device calculated by a time-of-flight sensor;

the determining the coordinate information of the pointing center point of the remote control device according to the pose information and the spatial position information of the remote control device comprises:

and determining coordinate information of a pointing center point of the remote control device according to the current pitch angle, the current yaw angle and the spatial position information of the remote control device.

According to a third aspect of the embodiments of the present disclosure, there is provided a projection apparatus correction method, including:

calculating to obtain a current pitch angle and a current roll angle of the projection equipment based on the second gravity meter and/or the second gyroscope;

obtaining a current yaw angle of the projection equipment based on a first yaw angle of the projection equipment calculated by the second gyroscope and a second yaw angle of the projection equipment calculated by the flight time sensor;

and correcting the projection image of the projection equipment according to the current pitch angle, the current roll angle and the current yaw angle.

Optionally, the obtaining a current yaw angle of the projection device based on the first yaw angle of the projection device calculated by the second gyroscope and the second yaw angle of the projection device calculated by the time-of-flight sensor includes:

determining a current angular acceleration of the projection device in a reference direction of yaw based on the second gyroscope;

determining a second yaw angle of the projection device based on the time-of-flight sensor when the current hook angular acceleration is less than a preset threshold;

obtaining a first yaw angle of the projection equipment through calculation of the second gyroscope, wherein the first yaw angle is the yaw angle of the projection equipment at the last moment;

and calculating to obtain the current yaw angle of the projection equipment based on the first yaw angle and the second yaw angle by combining a Kalman filtering algorithm.

Optionally, the preset threshold is obtained by:

and obtaining the preset threshold value according to the ratio of the preset angle error to the refresh rate of the flight time sensor.

According to a fourth aspect of embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the projection apparatus control method of any one of the first and second aspects, and/or implements the steps of the projection apparatus correction method of the third aspect.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a remote control device including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the projection apparatus control method according to the first aspect.

According to a sixth aspect of embodiments of the present disclosure, there is provided a projection apparatus, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the projection apparatus control method of the second aspect and/or to implement the steps of the projection apparatus correction method of the third aspect.

Based on the technical scheme, the position and pose information and the space position information of the remote control device are obtained and sent to the projection equipment, the projection equipment can be controlled to adjust the center point of the projection plane of the projection equipment according to the position and the space position information, so that the center point of the projection plane of the projection equipment is overlapped with the pointing center point of the remote control device, the projection plane of the projection equipment is projected to the pointing direction of the remote control device, the projection equipment is controlled through the remote control device, and the user experience of a user when the projection equipment is used is improved.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:

FIG. 1 is a schematic diagram of a scene in which a control method for a projection apparatus is proposed in an exemplary embodiment;

FIG. 2 is a flowchart illustrating a method for controlling a projection device according to an exemplary embodiment;

FIG. 3 is a schematic illustration of a determination of a pointing center point of a remote control device in accordance with an exemplary embodiment;

FIG. 4 is a schematic diagram of adjusting a center point of a projection plane as set forth in an exemplary embodiment;

FIG. 5 is a schematic flow chart diagram for determining pose information for a robot, according to an exemplary embodiment;

FIG. 6 is a schematic illustration of a current yaw angle and a current pitch angle of a remote control device according to an exemplary embodiment;

FIG. 7 is a schematic flow chart diagram for determining a current yaw angle of a remote control device in accordance with an exemplary embodiment;

fig. 8 is a flowchart illustrating a method for controlling a projection apparatus according to another exemplary embodiment;

FIG. 9 is a flowchart of a method for calibrating a projection device in accordance with an exemplary embodiment;

fig. 10 is a block diagram illustrating a projection device according to an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 1 is a schematic view of a scene of a control method of a projection apparatus according to an exemplary embodiment. As shown in fig. 1, the remote control device 100 and the projection apparatus 700 are included, wherein the remote control device 100 is used for performing remote control on the projection apparatus 700 to adjust a projection angle of the projection apparatus 700, or adjust a projection plane of the projection apparatus 700, and the like. The remote control device 100 and the projection apparatus 700 may be connected in a communication manner, such as WiFi or bluetooth, so as to send the pose information and the spatial position information to the projection apparatus 700. Of course, the remote control device 100 and the projection apparatus 700 may also be in the same internet of things network, and the remote control device 200 sends data to the internet of things control center, and the internet of things control center forwards the data to the projection apparatus 700.

It should be noted that the remote control device 100 may be a remote control separately provided to cooperate with the projection apparatus 700, or may be a functional module installed on other electronic devices, for example, the remote control device 100 is installed on a mobile terminal, and a user may control the projection apparatus 700 through the remote control device 100 on the mobile terminal.

Fig. 2 is a flowchart illustrating a control method of a projection apparatus according to an exemplary embodiment. The projection apparatus control method may be applied to a remote control device, as shown in fig. 2, and may include the following steps.

In step 110, pose information and spatial position information of the robot are determined.

Here, the attitude information of the remote control device refers to the attitude of the remote control device, such as the yaw angle and pitch angle of the remote control device. The spatial position information of the remote control device may refer to a geometric position of the remote control device in space, and the spatial position information may include height information of the remote control device, a distance from a projection surface (a wall or a curtain). For example, when the projection apparatus is used indoors, the spatial position information of the remote control device may refer to a spatial position of the remote control device in the room, and the spatial position may refer to height information of the remote control device and distance information between the remote control device and a wall surface. For example, information about the location of the remote in the living room and the height of the user holding the remote may be included.

It should be noted that the remote control device may be a conventional infrared remote control device, or may be a remote control device based on gesture control.

In some embodiments, the spatial location information of the remote control device may be obtained based on a communication module of the remote control device.

The communication module based on the remote control device can determine the spatial position information of the remote control device by combining the indoor spatial positioning technology. For example, the spatial position information of the remote control device is calculated by UWB (Ultra wide band) spatial positioning technology or WiFi 6.0 (a wireless network standard) spatial positioning technology.

In step 120, the pose information and the spatial position information are sent to a projection device, so that the projection device adjusts a center point of a projection plane of the projection device according to the pose information and the spatial position information, so that the center point of the projection plane coincides with a pointing center point of the remote control device.

Here, the remote control device transmits the pose information and the spatial position information to the projection apparatus after determining the pose information and the spatial position information to itself. And after receiving the pose information and the spatial position information, the projection equipment calculates a pointing central point of the remote control device according to the pose information and the spatial position information, and adjusts a projection plane of the projection equipment so that the central point of the projection plane of the projection equipment coincides with the pointing central point.

The pointing center point of the remote control device is a point on a projection area (wall or curtain) mapped by a ray in which the pointing of the remote control device is located, and the pointing center point reflects the projection area selected by the user. For example, when the remote control device is an infrared remote control device, the pointing center point refers to a point at which infrared rays are mapped on the projection area. The central point of the projection plane of the projection device refers to the central point of the projection image of the projection device projected on the wall or the curtain.

Fig. 3 is a schematic diagram of the determination of the pointing center point of a remote control device according to an exemplary embodiment. As shown in fig. 3, in some examples, the projection apparatus may calculate a pointing direction of the remote control device 100 according to pose information of the remote control device 100, and determine coordinate information of a pointing center point 101 in space, on which a ray in which the pointing direction is projected on a wall surface 102, according to the pointing direction and spatial position information of the remote control device 100. Wherein the coordinate information reflects the position of the pointing center point 101 of the remote control device 100 on the wall surface 102. It should be understood that the spatial location information may include three-dimensional coordinates of the remote control device 100 in space, which may be the height 104 of the remote control device 100 from the ground 103 and the distance 105 of the remote control device 100 from the wall 102.

FIG. 4 is a schematic diagram of adjusting a center point of a projection plane in accordance with an exemplary embodiment. As shown in fig. 4, when the pointing center point 101 of the remote control is located at the lower right of the wall surface 102, the original projection plane 401 of the projection apparatus is moved from the upper left of the wall surface 102 to the lower right of the wall surface 102 so that the center point of the adjusted projection plane 402 coincides with the pointing center point 101 of the remote control.

Therefore, by acquiring the pose information and the spatial position information of the remote control device and sending the pose information and the spatial position information to the projection equipment, the projection equipment can be controlled to adjust the central point of the projection plane of the projection equipment according to the pose information and the spatial position information, so that the central point of the projection plane of the projection equipment is coincided with the pointing central point of the remote control device, and the projection plane of the projection equipment is projected to the pointing direction of the remote control device.

Fig. 5 is a schematic flow chart for determining pose information of a remote control device according to an exemplary embodiment. In some possible implementations, as shown in fig. 5, the pose information of the robot can be obtained by the following steps.

In step 111, a current pitch angle of the remote control device is calculated based on the first gravimeter and/or the first gyroscope.

Here, the current pitch angle of the remote control device may be calculated by the first gravitometer, may be calculated by the first gyroscope, or may be calculated by the first gravitometer and the first gyroscope.

And calculating the current pitch angle of the remote control device based on the first gravimeter according to the included angle between the current gravity direction calculated by the first gravimeter and the coordinate axis of the chip. The current pitch angle of the remote control device is calculated through the first gyroscope, angular accelerations of the remote control device in the reference direction of the pitch angle and the reference direction of the roll angle are respectively obtained through the first gyroscope, and then the current pitch angle of the remote control device can be obtained through integration of the angular accelerations based on time. Specifically, when the corresponding angular acceleration calculated by the first gyroscope is smaller than a specific threshold value, the current pitch angle of the remote control device is calculated by combining a pitch angle calculated by the first gravimeter and a pitch angle at the last moment calculated by the first gyroscope with a kalman filter algorithm. It should be understood that the error of the first gyroscope increases gradually during the integration process, and the drift correction of the first gyroscope is performed through the pitch angle calculated by the first gravimeter, so that the error of the first gyroscope can be corrected.

In step 112, a current yaw angle of the remote control device is obtained based on the first yaw angle of the remote control device calculated by the first gyroscope and the second yaw angle of the remote control device calculated by the time-of-flight sensor.

Here, the first yaw angle of the remote control device calculated by the first gyroscope is a yaw angle calculated by time integration from an angular acceleration of the remote control device in a reference direction of the yaw angle measured by the first gyroscope. The second yaw angle is a yaw angle of the remote control device calculated by the time-of-flight sensor, specifically, the time-of-flight sensor projects a plurality of light spots to a projection surface (a wall or a curtain) to obtain depth information of the plurality of light spots, for each light spot, according to the depth information of the light spot, a three-dimensional coordinate of the light spot on the projection surface is determined, then according to the three-dimensional coordinates of the plurality of light spots, a normal vector of the projection surface relative to the remote control device is calculated, and finally, according to the normal vector, the yaw angle of the remote control device is calculated.

It will be appreciated that the first gravimeter, the first gyroscope and the time-of-flight sensor may be mounted on a remote control. The first gravimeter, the first gyroscope, and the time-of-flight sensor may be disposed on a mobile terminal when the remote control device is disposed on the mobile terminal.

Fig. 6 is a schematic diagram of the current yaw and pitch angles of the proposed remote control device in an exemplary embodiment. As shown in fig. 6, point H is the position of the robot, and the coordinates of point H can be determined by spatial location techniques. HN is perpendicular to the plane MONP and M is the current pointing centre of the remote. The angle PHN is the current yaw angle of the remote control device and can be determined by a first gyroscope and a time-of-flight sensor. The angle OHN is the current pitch angle of the remote control device and can be determined by the first gravimeter and/or the first gyroscope.

FIG. 7 is a schematic flow chart for determining a current yaw angle of a remote control device in accordance with an exemplary embodiment. As shown in fig. 7, in some embodiments, the current yaw angle of the remote control device may be obtained by the following steps.

In step 1121, the current angular acceleration of the remote control device in the reference direction of yaw angle is determined based on the first gyroscope.

Here, the first gyroscope measures in real time a current angular acceleration of the remote control device in a reference direction of yaw angle, which is a coordinate axis in which the yaw angle of the remote control device is located.

In step 1122, a second yaw angle of the remote control device is determined based on the time-of-flight sensor when the current angular acceleration is less than a preset threshold.

Here, when the current angular acceleration in the reference direction of the yaw angle is smaller than the preset threshold, the second yaw angle of the remote control device is calculated by the time-of-flight sensor, and the specific process of calculating the yaw angle by the time-of-flight sensor has been described in detail in the above embodiments, and is not described again here. When a current angular acceleration in a reference direction of the yaw angle is equal to or greater than a preset threshold value, a current yaw angle of the remote control device is obtained based on an integral over time of the current angular acceleration. It should be noted that the specific process of obtaining the current yaw angle of the remote control device based on the time integral of the current angular acceleration may be to obtain a third yaw angle of the remote control device through the time integral of the current angular acceleration, and then obtain the current yaw angle based on the fourth yaw angle obtained by the time integral of the third yaw angle and the angular acceleration of the remote control device in the reference direction of the yaw angle at the previous time, and by combining with the kalman filter algorithm.

It will be appreciated that when the current acceleration of the remote control device in the reference direction of yaw angle is less than a preset threshold, this indicates that the error of the first gyroscope has accumulated to a certain extent, and therefore calculating the current yaw angle directly from the current acceleration would result in an unacceptable error.

In some embodiments, the preset threshold may be obtained according to a ratio between a preset angle error and a refresh rate of the time-of-flight sensor. The preset threshold may be calculated by the following equation M ═ k × (J/T), where M is the preset threshold, k is a constant, J is the angle error, and T is the refresh rate of the time-of-flight sensor. It should be noted that the preset angle error may be determined according to the angle error that the human eye can tolerate.

In step 1123, a first yaw angle of the remote control device calculated based on the first gyroscope is obtained, where the first yaw angle is a yaw angle of the remote control device at a previous time.

Here, the first yaw angle is a yaw angle of the remote control device at the last time point calculated based on the first gyroscope. The specific process of calculating the yaw angle based on the first gyroscope has been described in detail in the above embodiments, and is not described herein again.

In step 1124, a current yaw angle of the remote control device is calculated based on the first yaw angle and the second yaw angle in combination with a kalman filter algorithm.

Here, the kalman filter algorithm is a recursive feedback algorithm, which includes two parts, a time update equation and a measurement state update equation. And the first yaw angle and the second yaw angle are used as the input of a Kalman filtering algorithm to obtain the current yaw angle of the remote control device.

The detailed calculation process of the current yaw angle of the remote control device comprises the following steps: when the current yaw angle of the remote control device is calculated by using the first gyroscope, the current yaw angle can be obtained by integral calculation of the angular velocity calculated by using the first gyroscope in time, and due to integral errors, the error of the yaw angle calculated by using the first gyroscope is larger and larger, and in the process, a Kalman filtering algorithm can be introduced to carry out error estimation and compensation on the yaw angle calculated by using the first gyroscope. Specifically, the current yaw angle calculated by the first gyroscope and the yaw angle of the remote control device at the last moment are used as the input of a Kalman filtering algorithm, and the output result of the Kalman filtering algorithm is the current angular acceleration of the remote control device. However, when the angular acceleration of the remote control device in the reference direction of the yaw angle is smaller than the preset threshold, the generated error is not acceptable by the user, and therefore, the yaw angle calculated by the first gyroscope needs to be corrected by combining the yaw angle calculated by the time-of-flight sensor, specifically, the current yaw angle calculated by the time-of-flight sensor is used as the input of the kalman filter algorithm instead of the current yaw angle calculated by the first gyroscope, so as to obtain a more accurate current yaw angle of the remote control device.

Fig. 8 is a flowchart illustrating a method for controlling a projection apparatus according to another exemplary embodiment. As the projection apparatus control method may be applied to a projection apparatus, as shown in fig. 8, the projection apparatus control method may include the following steps.

In step 210, pose information and spatial position information of a remote control device sent by the remote control device are received.

Here, the projection apparatus may receive the pose information and the spatial position information sent by the remote control device through the communication module, where the pose information and the spatial position information about the remote control device have been described in detail in the above embodiments, and are not described herein again.

In step 220, coordinate information of the pointing center point of the remote control device is determined according to the pose information and the spatial position information of the remote control device.

Here, after receiving the pose information and the spatial position information, the projection apparatus calculates coordinate information of the pointing center point of the remote control device based on the pose information and the spatial position information. In some examples, the projection apparatus may calculate a pointing direction of a pointing center point of the remote control device according to the pose information of the remote control device, and determine coordinate information of the pointing center point in space according to the pointing direction and spatial position information of the remote control device. Wherein the coordinate information reflects the position of the pointing center point of the remote control device on the wall surface.

In step 230, according to the coordinate information, the projection plane of the projection apparatus is adjusted so that the center point of the projection plane of the projection apparatus coincides with the pointing center point of the remote control device.

Here, the projection apparatus adjusts the projection plane of the projection apparatus according to the coordinate information so that the center point of the projection plane of the projection apparatus coincides with the pointing center point of the remote control device.

It should be noted that, in some embodiments, the projection plane adjustment of the projection device may be performed by a mechanical structure, for example, a multi-degree-of-freedom cradle head is installed on the projection device, and the projection angle of the projection device is adjusted by the projection device by adjusting the angle of the cradle head, so that the center point of the projection plane coincides with the pointing center point of the remote control device. In other embodiments, the projection device may change the size of the projected image, or the angle of the projected light, such that the center point of the projection plane coincides with the pointing center point of the remote control.

Therefore, by acquiring the pose information and the spatial position information of the remote control device, the projection equipment controls the center point of the projection plane of the projection equipment to coincide with the pointing center point of the remote control device according to the pose information and the spatial position information, so that the projection plane of the projection equipment is projected to the pointing direction of the remote control device.

In some implementations, the pose information of the remote control device includes a current pitch angle and a current yaw angle of the remote control device, where the current pitch angle is calculated by the remote control device based on a first gravimeter and/or a first gyroscope, and the current yaw angle is calculated by the remote control device based on a first yaw angle of the remote control device calculated by the first gyroscope and a second yaw angle of the remote control device calculated by a time-of-flight sensor.

It should be understood that the detailed calculation processes of the current pitch angle, the current roll angle and the current yaw angle have been described in detail in the above embodiments, and will not be described in detail herein.

In some embodiments, coordinate information of a pointing center point of the remote control device may be determined based on the current pitch angle, the current yaw angle, and the spatial position information of the remote control device.

It should be understood that the specific process of determining the coordinate information of the pointing center point of the remote control device according to the current pitch angle, the current yaw angle and the spatial position information of the remote control device has been described in detail in the above embodiments, and will not be described herein again.

Fig. 9 is a flowchart of a calibration method for a projection apparatus according to an exemplary embodiment. The embodiment of the disclosure also provides a projection device correction method, which can be applied to projection devices. As shown in fig. 9, the projection apparatus calibration method may include the following steps.

In step 310, a current pitch angle and a current roll angle of the projection device are calculated based on the second gravity meter and/or the second gyroscope.

Here, the second gravity meter and/or the second gyroscope may be provided on the projection device, which may be an IMU (Inertial Measurement Unit) on the projection device. In some embodiments, the current pitch angle and the current roll angle of the projection device may be calculated by the second gravimeter, the current pitch angle and the current roll angle of the projection device may be calculated by the second gyroscope, and the current pitch angle and the current roll angle of the projector may be calculated by the second gravimeter and the second gyroscope.

And calculating the current pitch angle and the current roll angle of the projection equipment based on the second gravity meter according to the included angle between the current gravity direction calculated by the second gravity meter and the coordinate axis of the chip. The current pitch angle and the current roll angle of the remote controller are calculated through the second gyroscope, angular accelerations of the projection equipment in the reference direction of the pitch angle and the reference direction of the roll angle are respectively obtained through the second gyroscope, and then the current pitch angle and the current roll angle of the projection equipment can be obtained based on the integral of the angular accelerations of time. Specifically, when the corresponding angular acceleration calculated by the second gyroscope is smaller than a specific threshold value, the pitch angle and the roll angle calculated by the second gravimeter and the pitch angle and the roll angle at the last moment calculated by the second gyroscope are combined with a kalman filter algorithm to calculate the current pitch angle and the current roll angle of the projection device. It should be understood that the error of the second gyroscope is gradually increased during the integration process, and the drift correction of the second gyroscope is performed through the pitch angle and the roll angle calculated by the second gravitometer, so that the error of the second gyroscope can be corrected.

In step 320, a current yaw angle of the projection device is obtained based on the first yaw angle of the projection device calculated by the second gyroscope and the second yaw angle of the projection device calculated by the time-of-flight sensor.

Here, the first yaw angle of the projection apparatus calculated by the second gyroscope means a yaw angle calculated by time integration from an angular acceleration of the projection apparatus in a reference direction of the yaw angle measured by the second gyroscope. The second yaw angle is a yaw angle of the projection device calculated by the time-of-flight sensor, specifically, the time-of-flight sensor projects a plurality of light spots to a projection surface (a wall or a curtain) to obtain depth information of the plurality of light spots, for each light spot, according to the depth information of the light spot, a three-dimensional coordinate of the light spot on the projection surface is determined, then according to the three-dimensional coordinates of the plurality of light spots, a normal vector of the projection surface relative to the projection device is calculated, and finally, according to the normal vector, the yaw angle of the projection device is calculated.

In some embodiments, in step 320, obtaining a current yaw angle of the projection device based on the first yaw angle of the projection device calculated by the second gyroscope and the second yaw angle of the projection device calculated by the time-of-flight sensor may include:

determining a current angular acceleration of the projection device in a reference direction of yaw based on the second gyroscope;

determining a second yaw angle of the projection device based on the time-of-flight sensor when the current hook angular acceleration is less than a preset threshold;

obtaining a first yaw angle of the projection equipment through calculation of the second gyroscope, wherein the first yaw angle is the yaw angle of the projection equipment at the last moment;

and calculating to obtain the current yaw angle of the projection equipment based on the first yaw angle and the second yaw angle by combining a Kalman filtering algorithm.

Here, the projection apparatus measures, in real time, a current angular acceleration of the projection apparatus in a reference direction of a yaw angle, which is a coordinate axis in which the projection apparatus is located, by the second gyroscope.

When the current angular acceleration of the projection device in the reference direction of the yaw angle is smaller than the preset threshold, the second yaw angle of the remote controller is calculated through the time-of-flight sensor, and the specific process of calculating the yaw angle by the time-of-flight sensor has been described in detail in the above embodiments, and is not described herein again. When the current angular acceleration of the projection device in the reference direction of the yaw angle is larger than or equal to a preset threshold value, the current yaw angle of the projection device is obtained based on the integral of the current angular acceleration in time. It should be noted that a specific process of obtaining the current yaw angle of the projection device based on the time integral of the current angular acceleration may be to obtain a third yaw angle of the projection device through the time integral of the current angular acceleration, and then obtain the current yaw angle through calculation based on the fourth yaw angle obtained by the third yaw angle and the time integral of the angular acceleration of the projection device in the reference direction of the yaw angle at the previous time, and by combining with a kalman filter algorithm.

It should be understood that when the current acceleration of the projection device in the reference direction of the yaw angle is smaller than the preset threshold, it indicates that the error of the second gyroscope has accumulated to a certain extent, and therefore, calculating the current yaw angle directly from the current acceleration may result in an unacceptable error.

Therefore, the current yaw angle of the projection device needs to be calculated based on the first yaw angle and the second yaw angle in combination with a kalman filter algorithm.

And the first yaw angle is the yaw angle of the projection equipment at the last moment calculated based on the second gyroscope. The specific process of calculating the yaw angle based on the second gyroscope has been described in detail in the above embodiments, and is not described herein again.

The Kalman filtering algorithm is a recursive feedback algorithm and comprises two parts, namely a time updating equation and a measurement state updating equation. And taking the first yaw angle and the second yaw angle as the input of a Kalman filtering algorithm to obtain the current yaw angle of the projection equipment.

The detailed calculation process of the current yaw angle of the projection equipment comprises the following steps: when the current yaw angle of the projection device is calculated by using the second gyroscope, the current yaw angle can be obtained by integral calculation of the angular velocity calculated by the second gyroscope in time, and due to integral errors, the error of the yaw angle calculated by the second gyroscope is larger and larger, and in the process, a Kalman filtering algorithm can be introduced to carry out error estimation and compensation on the yaw angle calculated by the second gyroscope. Specifically, the current yaw angle calculated by the second gyroscope and the yaw angle of the projection device at the previous moment are used as the input of a Kalman filtering algorithm, and the output result of the Kalman filtering algorithm is the current angular acceleration of the remote controller. However, when the angular acceleration of the projection device in the reference direction of the yaw angle is smaller than the preset threshold, the generated error is not acceptable by the user, and therefore, the yaw angle calculated by the second gyroscope needs to be corrected by combining the yaw angle calculated by the time-of-flight sensor, specifically, the current yaw angle calculated by the second gyroscope is replaced by the current yaw angle calculated by the time-of-flight sensor and is used as the input of the kalman filter algorithm, so that the more accurate current yaw angle of the projection device is obtained.

In some embodiments, the preset threshold may be obtained according to a ratio between a preset angle error and a refresh rate of the time-of-flight sensor. The preset threshold may be calculated by the following equation M ═ k × (J/T), where M is the preset threshold, k is a constant, J is the angle error, and T is the refresh rate of the time-of-flight sensor. It should be noted that the preset angle error may be determined according to the angle error that the human eye can tolerate.

In step 330, the projection image of the projection device is corrected according to the current pitch angle, the current roll angle and the current yaw angle.

Here, calibrating the projection image of the projection apparatus based on the current pitch angle, the current roll angle, and the current yaw angle into a trapezoidal vertex correction technique of the projection apparatus will not be described in detail herein. For example, after the current pitch angle, the current roll angle, and the current yaw angle of the projection device are obtained through calculation, a normal vector of a projection image, projected onto a projection plane (wall/curtain), of an original image of the projection device relative to the projection device may be obtained through calculation according to the current pitch angle, the current roll angle, and the current yaw angle of the projection device, and then position information of the plane where the projection image is located may be obtained through calculation based on the normal vector. The position information is the position of the plane where the projection image is located in space, which can be understood as the plane where the projection plane (wall/curtain) is located. And then, based on ray vectors established by the four vertexes of the projection image projected by the projection equipment, combining the position information of the plane where the projection image is located, and obtaining the three-dimensional coordinates of the four vertexes of the projection image projected on the projection plane (wall/curtain). And then carrying out vector decomposition on the three-dimensional coordinates of the four vertexes of the projected image to obtain two-dimensional vertex coordinates, and then adjusting the scale of the original image on the basis of the homography matrix relationship between the two-dimensional vertex coordinates and the vertex coordinates of the original image to obtain the adjusted original image. And finally, controlling the projection equipment to project the adjusted original image, so that the image seen by the user can always maintain a rectangular shape.

Therefore, the pose of the projection equipment can be calculated more accurately and quickly through the second gyroscope, the second gravity meter and the flight time sensor, and the effect is more obvious when the pose is particularly applied to portable projection equipment. The projection equipment correction method can also display the offset angle of the projection equipment in real time so as to assist a user in correctly placing the projection equipment.

It should be noted that the projection apparatus calibration method may be implemented alone, or may be implemented in combination with the projection apparatus control method. For example, after the projection device performs steps 210 to 230, steps 310 to 330 may be further performed to correct the projection image of the projection device.

In some embodiments, after step 230, the method for controlling a projection device may further include the steps of:

calculating to obtain a current pitch angle and a current roll angle of the projection equipment based on the second gravity meter and/or the second gyroscope;

obtaining a current yaw angle of the projection equipment based on a first yaw angle of the projection equipment calculated by the second gyroscope and a second yaw angle of the projection equipment calculated by the flight time sensor;

and correcting the projection image of the projection equipment according to the current pitch angle, the current roll angle and the current yaw angle.

Here, after the projection apparatus coincides the center point of the projection plane of the projection apparatus with the pointing center point of the remote control device based on the projection apparatus control method described above, the projection apparatus may further calculate a current pitch angle and a current roll angle of the projection apparatus based on a second gravity meter and/or a second gyroscope provided on the projection apparatus, and calculate a current yaw angle of the projection apparatus based on the second gyroscope and the time-of-flight sensor, and then correct the projection image of the projection apparatus based on the current pitch angle, the current roll angle, and the current yaw angle so that the image seen by the user can always be maintained in a rectangular shape.

It should be understood that the principle of the projection device calibration has been described in detail in the above embodiments of the calibration method for the projection device, and will not be described herein again.

Thus, after the center point of the projection plane of the projection apparatus is adjusted based on the remote control device, the projection image of the projection apparatus can be corrected so that the projection image displayed by the adjusted projection plane viewed by the user is rectangular in shape.

According to an embodiment of the present disclosure, there is provided a remote control device including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the projection apparatus control method according to the above embodiment.

Fig. 10 is a block diagram illustrating a projection device according to an example embodiment. As shown in fig. 10, the projection device 700 may include: a processor 701 and a memory 702. The projection device 700 may also include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.

The processor 701 is configured to control the overall operation of the projection apparatus 700, so as to complete all or part of the steps in the projection apparatus control method. Memory 702 is used to store various types of data to support operation at the projection device 700, such as instructions for any application or method operating on the projection device 700, as well as application-related data, such as spatial layout information in a room, and so forth. The Memory 702 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 703 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 702 or transmitted through the communication component 705. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the projection device 700 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, which is not limited herein. The corresponding communication component 705 may thus include: Wi-Fi module, Bluetooth module, NFC module, etc.

In an exemplary embodiment, the projection Device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the projection Device control method described above.

In another exemplary embodiment, there is also provided a computer readable storage medium including program instructions, which when executed by a processor, implement the steps of the projection apparatus control method described above. For example, the computer readable storage medium may be the above-described memory 702 comprising program instructions executable by the processor 701 of the projection device 700 to perform the above-described projection device control method.

The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.

In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.