阅读说明：本技术 输入方法、装置、显示设备及存储介质 (input method, input device, display equipment and storage medium ) 是由 伍思远 戴景文 贺杰 于 2018-05-31 设计创作，主要内容包括：本申请提供了一种输入方法、装置、显示设备及存储介质,涉及显示技术领域。该输入方法应用于显示设备,所述显示设备与第一控制器以及第二控制器连接,所述方法包括：基于所述第一控制器的第一姿势信息,于虚拟空间中显示所述第一控制器对应的第一基准指示线；基于所述第二控制器的第二姿势信息,于虚拟空间中显示所述第二控制器对应的第二基准指示线；当检测到所述第一基准指示线与所述第二基准指示线不平行时,基于所述第一基准指示线以及所述第二基准指示线生成输入至所述显示设备的输入点。本方法能实现于虚拟世界中的空间输入点的输入,便于用户进行虚拟世界中的绘画、模型雕刻及位置获取。(The application provides an input method, an input device, display equipment and a storage medium, and relates to the technical field of display. The input method is applied to display equipment, the display equipment is connected with a first controller and a second controller, and the method comprises the following steps: displaying a first reference indicator line corresponding to the first controller in a virtual space based on first posture information of the first controller; displaying a second reference indicating line corresponding to the second controller in a virtual space based on second posture information of the second controller; when it is detected that the first and second fiducial indicator lines are not parallel, generating an input point input to the display device based on the first and second fiducial indicator lines. The method can realize the input of the space input point in the virtual world, and is convenient for a user to carry out drawing, model carving and position acquisition in the virtual world.)

1. An input method is applied to a display device, wherein the display device is connected with a first controller and a second controller, and the method comprises the following steps:

displaying a first reference indicator line corresponding to the first controller in a virtual space based on first posture information of the first controller;

Displaying a second reference indicating line corresponding to the second controller in a virtual space based on second posture information of the second controller;

When it is detected that the first and second fiducial indicator lines are not parallel, generating an input point input to the display device based on the first and second fiducial indicator lines.

2. The method of claim 1, wherein after displaying a second baseline indicator line corresponding to the second controller in the virtual space, the method further comprises:

When the first baseline indicator line and the second baseline indicator line are detected to be parallel, prompting a user to move the first controller and/or the second controller so that the first baseline indicator line and the second baseline indicator line are not parallel.

3. The method of claim 1, wherein generating input points for input to the display device based on the first baseline indicator line and the second baseline indicator line when it is detected that the first baseline indicator line and the second baseline indicator line are not parallel comprises:

And if the first reference indicating line is intersected with the second reference indicating line, acquiring an intersection point of the first reference indicating line and the second reference indicating line, and taking the intersection point as an input point input to the display equipment.

4. The method of claim 2, wherein generating input points for input to the display device based on the first baseline indicator line and the second baseline indicator line when it is detected that the first baseline indicator line and the second baseline indicator line are not parallel comprises:

And if the first reference indicating line and the second reference indicating line are different, acquiring an intersection point of projection straight lines of the first reference indicating line and the second reference indicating line on the same plane, and taking the intersection point of the projection straight lines as an input point input to the display equipment.

5. The method of claim 2, wherein generating input points for input to the display device based on the first baseline indicator line and the second baseline indicator line when it is detected that the first baseline indicator line and the second baseline indicator line are not parallel comprises:

If the first reference indicating line and the second reference indicating line are different, judging whether the distance between the first reference indicating line and the second reference indicating line is smaller than a preset distance;

And if the distance is smaller than the preset distance, acquiring an intersection point of projection straight lines of the first reference indicating line and the second reference indicating line on the same plane as an input point input to the display equipment.

6. The method according to any one of claims 1-5, wherein displaying a first baseline indicator line corresponding to the first controller based on the first pose information of the first controller comprises:

Obtaining first pose information of the first controller;

Generating a straight line corresponding to the first posture information to obtain a first reference indicating line corresponding to the first controller;

And displaying the first benchmark indicating line.

7. The method according to any one of claims 1-5, wherein when it is detected that the first baseline indicator line is not parallel to the second baseline indicator line, after generating an input point to the display device based on the first baseline indicator line and the second baseline indicator line, the method further comprises:

Locking the input point according to the key operation of the first controller/the second controller by a user;

Moving the input point based on a change in a pose of the first controller and/or the second controller, generating a line segment input to the virtual space.

8. The method of claim 7, wherein after moving the input point to form a line segment based on the change in the pose of the first controller and/or the second controller, the method further comprises:

locking the line segment according to the key operation of a user on the first controller/the second controller;

moving the line segment based on a change in the pose of the first controller and/or the second controller, generating a surface input to the virtual space.

9. An input device applied to a display device connected to a first controller and a second controller, the device comprising: a first indication line display module, a second indication line display module and an input execution module, wherein,

the first indicator line display module is used for displaying a first reference indicator line corresponding to the first controller in a virtual space based on first posture information of the first controller;

The second indicator line display module is used for displaying a second reference indicator line corresponding to the second controller in a virtual space based on second posture information of the second controller;

the input execution module is used for generating an input point input to the display device based on the first datum indicating line and the second datum indicating line if a preset geometric relationship is met.

10. A display device comprising a memory coupled to a processor and a processor, the memory storing instructions that when executed by the processor perform the method of any of claims 1-8.

11. A computer-readable storage medium having program code executable by a processor, the program code causing the processor to perform the method of any one of claims 1-8.

Technical Field

The present application relates to the field of display technologies, and in particular, to an input method, an input device, a display device, and a storage medium.

Background

In recent years, with the progress of science and technology, technologies such as Augmented Reality (AR) and Virtual Reality (VR) have become hot spots of research at home and abroad. Taking augmented reality as an example, augmented reality is a technique for increasing the user's perception of the real world through information provided by a computer system, which overlays computer-generated virtual objects, scenes, or system cues into a real scene to enhance or modify the perception of the real world environment or data representing the real world environment.

Currently in the AR/VR industry, a handle is typically utilized as an auxiliary input means. In an application requiring spatial positioning, a handle with three degrees of freedom (DOF) can only realize use conditions such as a stylus and a pointer mode, and cannot meet requirements for determining spatial positions of points in a space such as drawing, model carving and position acquisition in a virtual space.

Disclosure of Invention

in view of this, the embodiments of the present application provide an input method, an input apparatus, a display device, and a storage medium, which utilize the principle that reference indicator lines corresponding to two handles may determine a point under a non-parallel condition, so as to input a point in a space, so as to meet requirements of drawing, model carving, position acquisition, and the like in a virtual space.

In order to achieve the purpose, the technical scheme adopted by the application is as follows:

In a first aspect, an embodiment of the present application provides an input method, which is applied to a display device, where the display device is connected to a first controller and a second controller, and the method includes: displaying a first reference indicator line corresponding to the first controller in a virtual space based on first posture information of the first controller; displaying a second reference indicating line corresponding to the second controller in a virtual space based on second posture information of the second controller; when it is detected that the first and second fiducial indicator lines are not parallel, generating an input point input to the display device based on the first and second fiducial indicator lines.

In a second aspect, an embodiment of the present application provides an input apparatus, which is applied to a display device, where the display device is connected to a first controller and a second controller, and the apparatus includes: the display device comprises a first indicator line display module, a second indicator line display module and an input execution module, wherein the first indicator line display module is used for displaying a first reference indicator line corresponding to a first controller in a virtual space based on first posture information of the first controller; the second indicator line display module is used for displaying a second reference indicator line corresponding to the second controller in a virtual space based on second posture information of the second controller; the input execution module is used for generating an input point input to the display device based on the first datum indicating line and the second datum indicating line if a preset geometric relationship is met.

In a third aspect, an embodiment of the present application provides a display device, including a memory and a processor, where the memory is coupled to the processor, and the memory stores instructions, and when the instructions are executed by the processor, the processor performs the input method provided in the first aspect.

In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a program code executable by a processor, where the program code causes the processor to execute the input method provided in the first aspect.

According to the input method, the input device, the display device and the storage medium, the first reference indicating line of the first controller is displayed in the virtual space based on the first posture information of the first controller, the second reference indicating line of the second controller is displayed in the virtual space based on the second posture information of the second controller, and when the first reference indicating line is not parallel to the second reference indicating line, the input point input to the display device is generated based on the first reference indicating line and the second reference indicating line. The method realizes the input of the input point in the space according to the principle that the reference indicating lines corresponding to the two controllers can determine one point under the condition of non-parallel, and solves the problems that the three-degree-of-freedom controller in the prior art cannot realize the input of the spatial midpoint and the six-degree-of-freedom controller has high cost and low precision in realizing the input of the spatial midpoint.

in order to make the aforementioned objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Fig. 1 is a schematic structural diagram illustrating a display system provided in an embodiment of the present application;

FIG. 2 is a schematic diagram illustrating an application scenario of a display system provided by an embodiment of the present application;

Fig. 3 is a block diagram illustrating a display device according to an embodiment of the present disclosure;

FIG. 4 is a flow chart illustrating an input method provided by an embodiment of the present application;

Fig. 5 is a flowchart illustrating step S110 in an input method provided in an embodiment of the present application;

FIG. 6 is a schematic diagram illustrating a display of fiducial indicator lines provided by an embodiment of the present application;

Fig. 7 is a flowchart illustrating step S120 in an input method provided in an embodiment of the present application;

FIG. 8 is a schematic diagram illustrating a first fiducial pointer intersecting a second fiducial pointer provided by an embodiment of the present application;

FIG. 9 illustrates another schematic diagram of a first fiducial pointer intersecting a second fiducial pointer provided by an embodiment of the present application;

FIG. 10 is a schematic diagram illustrating a first fiducial pointer provided by an embodiment of the present application when the first fiducial pointer is different from the second fiducial pointer;

FIG. 11 is a functional block diagram of an input device provided in an embodiment of the present application;

FIG. 12 is a block diagram illustrating another embodiment of an input device;

Fig. 13 is a functional block diagram illustrating a first indicator line display module in an input device according to an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.

In the existing test scheme of the mobile terminal, the test equipment is required to be in communication connection with the mobile terminal to be tested, and the test equipment generally scans the two-dimensional code carrying the internet protocol address and displayed on the mobile terminal to be tested, so that the internet protocol address is obtained, and the communication connection with the mobile terminal to be tested is realized. However, in an actual test, the mobile terminal to be tested is off the screen or enveloped, and under such conditions, the two-dimensional code of the mobile terminal to be tested cannot be scanned, so that the communication connection with the mobile terminal to be tested cannot be realized.

In the current AR/VR/MR industry, interactive design is an important technology research and development direction, and a user can interact with a virtual world through an interactive technology to feed back real-world operations to the virtual world. The current interaction mode is as follows: gesture interaction, voice interaction, controller interaction. Among them, controller interaction (i.e., interaction with a handle) mainly has two kinds: a 3DOF (degree of freedom) handle and a 6DOF (degree of freedom) handle.

When interacting with a handle in an application requiring spatial positioning, for example drawing a hand-held device as a brush in a virtual world, the input point needs to be determined in space. However, the 3DOF handle (three-degree-of-freedom information of the handle is acquired by an Inertial Measurement Unit (IMU)) cannot realize the above operation because the forward extension and the backward movement cannot be sensed. The cost of a 6DOF handle (acquiring six-degree-of-freedom information of the handle by tracking means such as spot tracking and an IMU) is high, for example, a camera is required for spot tracking; in addition, there are problems that the mark is blocked when tracking is performed, and there is a problem that the light is too strong in the spot tracking mode, so that there is a problem of accuracy.

Therefore, the inventor proposes an input method, an input device, a display device and a storage medium in the embodiments of the present application. Embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Referring to fig. 1, a display system 10 provided by an embodiment of the present application is shown. The display system 10 includes: a first controller 100, a second controller 200, and a display device 300. The first controller 100 and the second controller 200 are in communication connection with the display device 300. As shown in fig. 2, which is an application scenario diagram of the display system 10, a user may wear the display device 300 and perform input control using the first controller 100 and the second controller 200. When the user performs input control using the first controller 100 and the second controller 200, the user may control the first controller 100 with one hand and the second controller 200 with the other hand, thereby implementing key input, position movement, and the like of the user to the first controller 100 and the second controller 200.

In the embodiment of the present application, the first controller 100 and the second controller 200, as interaction devices, may both realize inputting control instructions, data, and the like to the display device, so as to realize interaction with the display device 300. The first controller 100 and the second controller 200 may be 3DOF handles or 6DOF handles, and the specific types of the first controller 100 and the second controller 200 are not limited in the embodiments of the present application.

As an embodiment, referring to fig. 3, the display device 300 may include: a processor 310, a memory 320, a display device 330, and an image acquisition device 340. The display device 330, the memory 320, and the image capture device 340 are all coupled to the processor 310.

The image capturing device 340 is used for capturing an image of an object to be photographed and sending the image to the processor 310. The image capturing device 340 may be an infrared camera, a color camera, or the like, and the specific type of the image capturing device 340 is not limited in the embodiments of the present application.

the processor 310 is configured to output the display content to the display device, and further configured to perform operations of identifying and tracking the first handle and the second handle.

The processor 310 may comprise any suitable type of general or special purpose microprocessor, digital signal processor, or microcontroller. The processor 310 may be configured to receive data and/or signals from various components of the system via, for example, a network. The processor 310 may also process the data and/or signals to determine one or more operating conditions in the system. For example, the processor 310 generates image data of a virtual world from image data stored in advance, and transmits the image data to the display device for display; the image data sent by the intelligent terminal or the computer can be received through a wired or wireless network, and the image of the virtual world is generated and displayed according to the received image data; and the corresponding display content in the virtual world can be determined by carrying out identification tracking operation according to the image acquired by the image acquisition device and then sent to the display device for display. It is understood that the processor 310 is not limited to being disposed within the display device 300.

The memory 320 may be used to store software programs and modules, and the processor 310 executes various functional applications and data processing by operating the software programs and modules stored in the memory 320. The memory 320 may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

in the embodiment of the present application, the display device 300 may further include a communication module, and the communication module is connected to the processor 310. The communication module is used for communication between the display device 300 and other devices, for example, the first handle 100 and the second handle 200 of the display system in the embodiment of the present application.

As another embodiment, the display apparatus 300 may include: the display device and the image acquisition device of the display equipment are connected with a terminal device with the storage function of a memory and the processing function of a processor. It is to be understood that the processing performed by the processor in the above-described embodiments is performed by the processor of the terminal device, and the data stored by the memory in the above-described embodiments is stored by the terminal device. The terminal device may be a mobile terminal or a PC, that is, in this embodiment, the head-mounted display device may be a plug-in mobile phone or an external computer.

In the embodiment of the present application, the display device 300 may be a head-mounted display device, and the head-mounted display device may implement different effects such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR).

of course, the display device provided in the embodiments of the present application is not limited to the components in the above embodiments, and may include more or less components than the above components.

aiming at the display system, the embodiment of the application provides a method for realizing the input of an input point in space by the principle that the reference indicating lines corresponding to the two handles can determine one point under the condition of non-parallel. The specific input method is described in detail in the following examples.

Referring to fig. 4, fig. 4 is a schematic flowchart illustrating an input method according to an embodiment of the present disclosure. The method is applied to a display device of a display system shown in fig. 1, the display system comprising: the device comprises a first controller, a second controller and a display device. As shown in fig. 4, the input method includes:

Step S110: and displaying a first reference indicating line corresponding to the first controller in a virtual space based on the first posture information of the first controller.

In this embodiment, the display device may display a first reference indication line corresponding to the first controller in a virtual space displayed by the display device according to the first posture information of the first controller. It is understood that the first reference indicating line should correspond to the first posture information of the first controller, and the first reference indicating line is used for indicating the user to adjust the posture of the first controller, so that the posture of the first reference indicating line in the virtual space changes along with the change of the posture of the first controller, and the purpose of subsequently determining the input point is achieved.

In an embodiment of the present application, please refer to fig. 5, the displaying a first reference indicating line corresponding to a first controller in a virtual space based on first posture information of the first controller may include:

Step S111: first pose information of the first controller is obtained.

When the first reference indication line corresponding to the first controller is displayed, first posture information of the first controller may be acquired. The first controller may acquire first pose information of the first controller using its IMU and then transmit the first pose information to the display apparatus, so that the display apparatus obtains the first pose information of the first controller, which may include an orientation, a rotation angle, and the like of the first controller.

The IMU is a device for measuring the three-axis attitude angle (or angular velocity) and acceleration of the controller. Generally, an IMU includes three single-axis accelerometers and three single-axis gyroscopes, the accelerometers detect acceleration signals of the object in three independent axes of the carrier coordinate system, and the gyroscopes detect angular velocity signals of the carrier relative to the navigation coordinate system, and measure angular velocity and acceleration of the object in three-dimensional space, and thus, the attitude of the controller is calculated.

of course, the specific implementation manner of the display device obtaining the first posture information of the first controller is not limited in the embodiment of the present application, and may also be another existing implementation manner, for example, a mark pattern may be provided on the first controller, the mark pattern has a plurality of feature points, and the display device may utilize the image acquisition device to identify the feature points of the mark pattern, so as to obtain the first posture information of the first controller.

Step S112: generating a straight line corresponding to the first posture information to obtain a first reference indicating line corresponding to the first controller.

after the first posture information of the first controller is obtained, a straight line corresponding to the first posture information may be generated in the virtual space, and the generated straight line may be used as a first reference indicating line corresponding to the first controller.

It is understood that the posture of the first reference indicating line in the virtual space corresponds to the first posture information of the first controller, for example, the orientation of the first reference indicating line in the virtual space is the same as the orientation of the first controller, and the rotation angle of the first reference indicating line in the virtual space is the same as the rotation angle of the first controller.

Step S113: and displaying the first benchmark indicating line.

Finally, after the first reference indication line of the first controller is obtained, the obtained first reference indication display is displayed to indicate the purpose that the user moves the posture of the first reference indication line by changing the posture of controlling the first controller.

In the embodiment of the present application, the first reference indicating line may be further generated in combination with the first posture information and the position of the first controller. The first controller may be spatially positioned to obtain a position of the first controller. The spatial positioning can be positioning and measuring by adopting an optical or ultrasonic mode, and the spatial position of the first controller is deduced by establishing a model.

For example, a plurality of infrared point light sources are arranged on the first controller, and the display device uses the infrared camera thereof to identify the infrared point light sources, so as to obtain the spatial position of the first controller. The display device stores position information of light spots on a corresponding picture when the first controller is located at various spatial positions, the infrared point light source which is set on the first controller and flashes in a first flashing mode is set on the first controller, the display device records the position relation of the lighted infrared point light source on the image according to the light spots on the image, the first controller sequentially lights the infrared point light sources according to a preset flashing sequence, the display device records position data of the corresponding flashing points, and finally the display device obtains the spatial coordinates of the first controller according to the measured position data.

After the first posture information and the position of the first controller are obtained, a straight line passing through the position of the first controller and corresponding to the posture of the first controller may be generated based on the first posture information and the position to obtain a first reference indicating line.

As one of the embodiments, the first reference indicating line may be a straight line which passes through an axis of the first controller and extends in a forward direction of the first controller. As shown in fig. 6, a first fiducial indicating line 102 extends along the axis 101 of the first controller 100. The indicator line is made visible to the user and the position and attitude of the first reference indicator line is changed by controlling the attitude change and the position movement of the first controller.

step S120: and displaying a second reference indicating line corresponding to the second controller in the virtual space based on the position of the second controller.

In the embodiment of the present application, since it is necessary to determine the input point in the space according to the principle that one point can be determined under the condition that two straight lines are not parallel. Therefore, it is also necessary to display the indication line corresponding to the second controller, so that the user can change the postures of the first reference indication line and the second reference indication line in the space by moving the first controller and the second controller, so as to reach the condition that the first reference indication line and the second reference indication line are not parallel, and thus the position of the input point can be determined.

In an embodiment of the present application, please refer to fig. 7, the displaying a second reference indicating line corresponding to the second controller based on the position of the second controller may include:

step S121: second pose information of the second controller is obtained.

When the second reference indication line corresponding to the second controller is displayed, first, the second posture information of the second controller may be acquired. In the embodiment of the present application,

for a specific manner of obtaining the second posture of the second controller by the display device, reference may be made to the manner of obtaining the position of the first controller in step S110, which is not described herein again.

Step S122: generating a straight line corresponding to the second posture information to obtain a second reference indicating line corresponding to the second controller.

After the second posture information of the second controller is obtained, a straight line corresponding to the second posture information in the displayed virtual space is generated according to the position of the second controller, and a second reference indicating line corresponding to the second controller is obtained.

Similarly, the manner of generating the second reference indicating line of the second controller may refer to the manner of generating the first reference indicating line of the first controller in step S110, which is not described herein again.

Step S123: and displaying the second benchmark indicating line.

And finally, displaying the obtained second reference indication corresponding to the second controller.

In the embodiment of the present application, the second reference indicating line may also be generated from the second posture information and the position of the second controller as well. As one way, the second reference indicating line may be a straight line extending along the axis of the second controller in the forward direction of the second controller. As shown in fig. 6, a second fiducial indicating line 202 extends along the axis 201 of the second controller 200. The user can observe the indicating line and the position and the posture of the second reference indicating line are changed by controlling the movement and the posture change of the second controller.

Step S130: when it is detected that the first and second fiducial indicator lines are not parallel, generating an input point input to the display device based on the first and second fiducial indicator lines.

In the embodiment of the application, after a first reference indicating line corresponding to a first controller and a second reference indicating line corresponding to a second controller are displayed in a displayed space, a geometric relationship between the first reference indicating line and the second reference indicating line can be detected in real time, when the first reference indicating line is detected to be not parallel to the second reference indicating line, a point can be determined according to the first reference indicating line and the second reference indicating line, and the point is used as an input point input into the display space of the display device.

Where parallel means that two lines are said to be parallel when they do not have any point in common in the plane. If the first reference indicating line and the second reference indicating line are judged not to satisfy the condition that the two straight lines on the plane do not have any common point, the two lines are not parallel. When the first datum indication line and the second datum indication line are not parallel, the first datum indication line and the second datum indication line are out of plane and intersect. The non-coplanar refers to that the first datum indicating line and the second datum indicating line are not on the same plane, and the non-coplanar straight line is a straight line which is not intersected or parallel. The intersection means that the first reference indicating line and the second reference indicating line are in the same plane, and if the first reference indicating line and the second reference indicating line have only one common point, the first reference indicating line and the second reference indicating line are called to be intersected.

In an embodiment of the present application, the input method may further include:

When the first baseline indicator line and the second baseline indicator line are detected to be parallel, prompting the user to move the first controller and/or the second controller so that the first baseline indicator line and the second baseline indicator line are not parallel.

it can be understood that, according to the spatial position parameter when the first reference indicating line and the second reference indicating line are generated, whether the displayed first reference indicating line and the second reference indicating line are parallel or not is judged, and a result that the first reference indicating line and the second reference indicating line are parallel or a result that the first reference indicating line and the second reference indicating line are not parallel is obtained.

When the first reference indicating line and the second reference indicating line are judged to be parallel, the input point input into the virtual space cannot be determined according to the first reference indicating line and the second reference indicating line. Accordingly, the user may be prompted to move the first control and/or the second control to cause a change in the position of the first control and/or the second control. Therefore, when the position of the first controller and/or the second controller is changed, the display device also changes the position of the real reference indicating line according to the position of the first controller and the position of the second controller, and finally, the user can move the first reference indicating line and the second reference indicating line to be not parallel according to the prompt.

When the first reference indicating line and the second reference indicating line are not parallel, an input point input into the displayed virtual space can be determined according to the first reference indicating line and the second reference indicating line.

In one embodiment, if the first fiducial indicating line and the second fiducial indicating line are not parallel, generating an input point to the display device based on the first fiducial indicating line and the second fiducial indicating line may include:

and if the first reference indicating line is intersected with the second reference indicating line, acquiring an intersection point of the first reference indicating line and the second reference indicating line, and taking the intersection point as an input point input to the display equipment.

It will be appreciated that under the condition that the first and second lines of reference are not parallel, the first and second lines of reference may intersect, be out of plane or coincide, and when the first and second lines of reference coincide, the only one point, i.e. the input point to the display device, cannot be determined. And the first fiducial-indicating line intersects the second fiducial-indicating line, the point of intersection (the only point) can be determined.

therefore, when it is determined that the first reference indicating line intersects the second reference indicating line, an intersection point of the first reference indicating line and the second reference indicating line may be acquired as an input point into a virtual space displayed by the display device. Referring to fig. 8 and 9, when the first reference indicating line intersects the second reference indicating line, an intersection 103 of the first reference indicating line and the second reference indicating line is used as an input point input into a virtual space displayed by the display device.

in one embodiment, if the first fiducial indicating line and the second fiducial indicating line are not parallel, generating an input point to the display device based on the first fiducial indicating line and the second fiducial indicating line may include:

and if the first reference indicating line and the second reference indicating line are different from each other, acquiring an intersection point of projection straight lines of the first reference indicating line and the second reference indicating line on the same plane as an input point input to the display equipment.

It will be appreciated that under the condition that the first and second lines of reference are not parallel, the first and second lines of reference may intersect, be out of plane or coincide, and when the first and second lines of reference coincide, the only one point, i.e. the input point to the display device, cannot be determined. When the first reference indicating line and the second reference indicating line are different from each other, a unique point, that is, an intersection point of projection straight lines of the first reference indicating line and the second reference indicating line on the same plane, can be determined according to the first reference indicating line and the second reference indicating line.

therefore, when the first reference indicating line and the second reference indicating line are determined to be different from each other, the intersection point of the projection straight lines of the first reference indicating line and the second reference indicating line in the same plane can be obtained and used as the input point input into the virtual space displayed by the display device.

Specifically, the same plane may be any plane close to the first reference indicating line or the second reference indicating line, and may be, for example, a horizontal plane on which a lowest point in the vertical direction on the first reference indicating line/the second reference indicating line is located, or a horizontal plane on which a highest point in the vertical direction on the first reference indicating line/the second reference indicating line is located. The specific first reference indicating line and the plane to which the second reference indicating line needs to be projected are not limited in the embodiments of the present application.

In one embodiment, generating an input point to the display device based on the first baseline indicator line and the second baseline indicator line if the first baseline indicator line and the second baseline indicator line are not parallel comprises:

If the first reference indicating line and the second reference indicating line are different from each other, judging whether the distance between the first reference indicating line and the second reference indicating line is smaller than a preset distance or not, wherein the distance between the first reference indicating line and the second reference indicating line refers to the shortest distance between the first reference indicating line and the second reference indicating line and can be represented by a common perpendicular line between the first reference indicating line and the second reference indicating line, and the common perpendicular line refers to a line segment which is perpendicular to the first reference indicating line and the second reference indicating line simultaneously; and if the distance is smaller than the preset distance, acquiring the intersection point of the projection straight lines of the first reference indicating line and the second reference indicating line on the same plane as an input point input to the display equipment.

in the embodiment of the present application, an implementation manner is further provided, where in a case that the first reference indicating line and the second reference indicating line are not coplanar, when a distance between the first reference indicating line and the second reference indicating line is within a certain range, an input point input to a virtual space displayed by the display device may be determined according to the first reference indicating line and the second reference indicating line. As shown in fig. 10, the first reference indicating line is a straight line or a ray where the AC 'line segment is located in the drawing, the second reference indicating line is a straight line or a ray where the CB' line segment is located in the drawing, the AC 'and the CB' are opposite surfaces, a common perpendicular line between the AC 'and the CB' is determined to be a straight line or a ray where the IH line segment is located, and then whether the length of the common perpendicular line IH is smaller than a preset distance is determined.

thereby, a situation is avoided where the first fiducial pointer is further from the second fiducial pointer, and the user does not want to determine the input point at this time. In addition, user's option is also provided so that the user can set the condition determined by the input point according to the preference, the condition may be a preset distance between the first reference indicating line and the second indicating line, the thickness of the first reference indicating line and the second indicating line, etc. It can be understood that the larger the preset distance is, the easier it is to determine whether the distance between the first reference indicating line and the second reference indicating line is smaller than the preset distance, and thus the easier it is to determine the input point according to the first reference indicating line and the second indicating line; the thicker the first reference indicating line and the second indicating line are, the easier it is to determine whether the distance between the first reference indicating line and the second reference indicating line is less than the preset distance, and thus the easier it is to determine the input point according to the first reference indicating line and the second indicating line.

Since the positions of the first reference indicating line and the second reference indicating line in the virtual space are known, the determined coordinates of the input point can also be obtained, so that the input of the input point in the virtual space displayed by the display device through the first controller and the second controller is realized.

in the embodiment of the application, when the input point is generated, information for promoting whether the user needs to input the point is displayed, and when the user determines that the obtained input point needs to be input, the input point is input, so that the situation that the obtained input point does not meet the requirements of the user due to misoperation of the user on the controller is avoided.

In an embodiment of the present application, the input method may further include: locking the input point according to the key operation of the first controller/the second controller by a user; moving the input point based on a change in a pose of the first controller and/or the second controller, generating a line segment input to the virtual space.

It is understood that after an input point is input into the virtual space, the point may be locked by a key of the first controller/the second controller, and dragged to form a line segment by controlling a change in the posture of the first controller and/or the second controller.

For example, when a user needs to draw a line in a virtual space, the user may lock an input point, drag the input point by using the first controller and/or the second controller, and form a line segment on a moving track of the input point, so as to meet a requirement of the user for drawing the line segment.

In addition, the method is also applicable to other scenes needing line segment input, for example, when the model is carved and needs to be cut, after the input point is locked, the first controller and/or the second controller is used for dragging the input point to form a cutting line for cutting the model, so as to cut the model needing to be carved, and the like.

Of course, the specific application scenario of the method for forming the line segment input to the virtual space is not limited in the embodiment of the present application, and may also be other scenarios requiring the input of the line segment to the virtual space.

In an embodiment of the present application, the input method may further include: locking the line segment according to the key operation of a user on the first controller/the second controller; moving the line segment based on a change in the pose of the first controller and/or the second controller, generating a surface input to the virtual space.

it can be understood that, after the line segment is input into the virtual space, the user may also lock the input line segment through the keys of the first controller and/or the second controller, and then drag the line segment by changing the posture of the first controller and/or the second controller, so as to form a surface input into the virtual space.

For example, when a user needs to draw in a virtual space, the user can use the first controller and/or the second controller to drag the input line segment after locking the input line segment, and the input line segment and the moving track thereof form a surface, so as to meet the requirement of the user on drawing the surface.

Of course, the specific application scenario of the method for forming the surface input to the virtual space is not limited in the embodiment of the present application, and may also be another scenario that requires the input surface to the virtual space, for example, a scenario such as model carving.

In the embodiment of the present application, the surface input to the virtual space may be locked according to a key operation of the first controller and/or the second controller by the user, and the surface may be moved based on a change in the posture of the first controller and/or the second controller to form a three-dimensional body input to the virtual space.

Likewise, the method for forming a three-dimensional body input to a virtual space is also applicable to a scene drawn in the virtual space, and of course, the application scenario of the method for forming a three-dimensional body input to a virtual space is not limited in the embodiment of the present application.

in the embodiment of the application, shortcut operations such as locking a shaft and Snap specific distance can be realized by matching with the key. For example, in the case where the first controller does not need to move, key a may be held down to hold the gesture, so that the real left hand may be relaxed. When a case is pressed, the minimum change of the indicating line is a specified distance, for example, when the snap distance of the X axis is 1cm, when the B key is pressed and moved on the X axis, the indicating line changes only when the distance exceeds 1 cm.

in the embodiment of the present application, one of the first controller and the second controller may be replaced by a display device, when the input method is implemented, a preset reference line (a reference line corresponding to the display device) can be displayed by the display device, and the reference line is generated according to the posture information of the first controller or the second controller, then, the reference line corresponding to the display device and the reference line corresponding to the controller satisfy the condition for determining the input point in the above embodiment by the change of the posture of the first controller or the second controller by the user, and finally, when the reference line corresponding to the display device and the reference line corresponding to the controller satisfy the condition for determining the input point in the above embodiment, and acquiring an input point according to the reference line corresponding to the display equipment and the reference line corresponding to the controller, and inputting the input point into the virtual space displayed by the display equipment. Therefore, the input of the space points in the virtual space can be completed based on a single controller, the operation is convenient and fast, and the user experience is high.

according to the input method provided by the embodiment of the application, the determination and the input of the space point in the virtual space can be realized only by two controllers, and the problem that the 3DOF controller cannot determine the space point and input the space point in the space is solved. In addition, since the method only needs two controllers to be performed, the 6DOF controller has a problem of high cost in realizing the spatial point input. In addition, the user can input the space point only by changing the position of the controller, so that the operation of the user is facilitated, and the experience of the user is improved.

Referring to fig. 11, in an embodiment, the present application provides an input device applied to a display apparatus of a display system, where the display system includes: the device comprises a first controller, a second controller and a display device. Referring to fig. 11, the input device 400 includes: a first indication line display module 410, a second indication line display module 420 and an input execution module 430. The first indicator line display module 410 is configured to display a first reference indicator line corresponding to the first controller in the virtual space based on the first posture information of the first controller; the second indicator line display module 420 is configured to display a second reference indicator line corresponding to the second controller in the virtual space based on the second posture information of the second controller; the input executing module 430 is configured to generate an input point to the display device based on the first reference indicating line and the second reference indicating line if the preset geometric relationship is satisfied.

In the second embodiment of the present application, please refer to fig. 12, the input device 400 may further include a prompt module 440. The prompting module 440 is configured to prompt a user to move the first controller and/or the second controller when the first baseline indicator is detected to be parallel to the second baseline indicator, so that the first baseline indicator is not parallel to the second baseline indicator.

further, the input execution module 430 may be specifically configured to: and if the first reference indicating line is intersected with the second reference indicating line, acquiring an intersection point of the first reference indicating line and the second reference indicating line, and taking the intersection point as an input point input to the display equipment.

In this embodiment, the input execution module 430 may specifically be configured to: and if the first reference indicating line and the second reference indicating line are different from each other, acquiring an intersection point of projection straight lines of the first reference indicating line and the second reference indicating line on the same plane as an input point input to the display equipment.

In this embodiment, the input execution module 430 may specifically be configured to: if the first reference indicating line and the second reference indicating line are different, judging whether the distance between the first reference indicating line and the second reference indicating line is smaller than a preset distance; and if the distance is smaller than the preset distance, acquiring the intersection point of the projection straight lines of the first reference indicating line and the second reference indicating line on the same plane as an input point input to the display equipment.

In the embodiment of the present application, please refer to fig. 13, the first indicator line display module 410 includes: a first position obtaining unit 411, a first indication line generating unit 412 and a first display executing unit 413. Wherein the first position obtaining unit 411 is configured to obtain first posture information of the first controller; the first indicator line generating unit 412 is configured to generate a straight line corresponding to the first posture information, and obtain a first reference indicator line corresponding to the first controller; the first display execution unit 413 is configured to display the first reference indicating line.

In the embodiment of the present application, the input device 400 may further include: an input point locking module and a line segment generating module. The input point locking module is used for locking the input point according to the key operation of a user on the first controller/the second controller; the line segment generation module is used for moving the input point and generating a line segment input to the virtual space based on the posture change of the first controller and/or the second controller.

In the embodiment of the present application, the input device 400 may further include: the device comprises a line segment locking module and a face generating module. The line segment locking module is used for locking the line segment according to the key operation of a user on the first controller/the second controller; the face generation module is used for moving the line segment based on the posture change of the first controller and/or the second controller and generating a face input to the virtual space.

According to the input method, the input device, the display device and the storage medium, the first reference indicating line of the first controller is displayed in the virtual space based on the position of the first controller, the second reference indicating line of the second controller is displayed in the virtual space based on the position of the second controller, and when the first reference indicating line and the second reference indicating line are not parallel, the input point input to the display device is generated based on the first reference indicating line and the second reference indicating line. The method realizes the input of the input point in the space according to the principle that the reference indicating lines corresponding to the two controllers can determine one point under the condition of non-parallel, and solves the problems that the three-degree-of-freedom controller in the prior art cannot realize the input of the spatial midpoint and the six-degree-of-freedom controller has high cost and low precision in realizing the input of the spatial midpoint.

It should be noted that the embodiments in the present specification are mainly described as different from other embodiments, and the same and similar parts in the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.