阅读说明：本技术 一种画面显示的方法及装置 (Picture display method and device ) 是由 曲磊 杨雪洁 刘帅帅 孙萁浩 唐至威 张振铎 杨小娜 吴风炎 于 2020-10-22 设计创作，主要内容包括：本发明公开了一种画面显示的方法及装置,包括：构建交互应用的相机坐标系和世界坐标系,其中,交互应用用于多个用户在虚拟场景内进行互动,相机坐标系是以交互应用中第一用户的视角确定的三维坐标系,然后确定交互应用的待显示画面中的至少一个对象在世界坐标系下的三维坐标,再通过所述相机坐标系将至少一个对象在世界坐标系下的三维坐标转换为像素坐标系下的像素坐标,像素坐标用于对至少一个对象进行显示。通过二维的像素坐标系下的待显示画面具有的三维立体视觉效果,提升用户在游戏虚拟场景中的真实临场感,增加了用户与用户之间交流的虚拟现实效果,提高了用户的游戏体验。(The invention discloses a method and a device for displaying pictures, which comprises the following steps: the method comprises the steps of constructing a camera coordinate system and a world coordinate system of interactive application, wherein the interactive application is used for a plurality of users to interact in a virtual scene, the camera coordinate system is a three-dimensional coordinate system determined by the visual angle of a first user in the interactive application, then determining the three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, converting the three-dimensional coordinates of the at least one object under the world coordinate system into pixel coordinates under the pixel coordinate system through the camera coordinate system, and the pixel coordinates are used for displaying the at least one object. The three-dimensional visual effect of the picture to be displayed under the two-dimensional pixel coordinate system improves the real presence of the user in the game virtual scene, increases the virtual reality effect of communication between the user and the user, and improves the game experience of the user.)

1. A method for displaying a screen, comprising:

constructing a camera coordinate system and a world coordinate system of the interactive application; the interactive application is used for a plurality of users to interact in a virtual scene; the camera coordinate system is a three-dimensional coordinate system determined with a perspective of a first user in the interactive application;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system;

converting three-dimensional coordinates of the at least one object in the world coordinate system into pixel coordinates in a pixel coordinate system through the camera coordinate system, wherein the pixel coordinates are used for displaying the at least one object.

2. The method of claim 1, wherein converting three-dimensional coordinates of the at least one object in the world coordinate system to pixel coordinates in a pixel coordinate system by the camera coordinate system comprises:

determining a conversion relationship between the camera coordinate system and the world coordinate system; the transformation relation is used for determining three-dimensional coordinates of the at least one object in the camera coordinate system;

determining imaging parameters under the camera coordinate system; the imaging parameters are used for converting three-dimensional coordinates of the at least one object in the camera coordinate system into two-dimensional pixel coordinates in a pixel coordinate system;

and determining the pixel coordinates of the at least one object according to the conversion relation and the imaging parameters.

3. The method of claim 2, wherein the at least one object is a scene object within the virtual scene;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

and according to the geometric figure of the scene object, constructing a geometric function of the scene object in the world coordinate system, thereby determining the three-dimensional coordinates of the scene object in the world coordinate system.

4. The method of claim 2, wherein the at least one object is a user window within the virtual scene;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

determining a user window in the virtual scene;

and determining the three-dimensional coordinates of the user window in the world coordinate system, so as to obtain the three-dimensional coordinates of the user window in the world coordinate system.

5. The method of claim 2, wherein the at least one object is a second user within the virtual scene; the second user is any one of a plurality of users except the first user;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

acquiring an interactive picture of the second user from the video picture of the second user;

projecting the interactive picture of the second user into a user window of the second user;

and determining the three-dimensional coordinates of the user window of the second user in the world coordinate system, so as to obtain the three-dimensional coordinates of the second user in the world coordinate system.

6. The method of claim 1, wherein the three-dimensional origin of coordinates of the world coordinate system is projected in a direction perpendicular to the ground through the three-dimensional origin of coordinates of the camera coordinate system.

7. The method of any of claims 1 to 6, wherein constructing a camera coordinate system for an interactive application comprises:

acquiring a camera coordinate system corresponding to the view angle switching instruction according to the view angle switching instruction of the first user; a camera coordinate system with a plurality of visual angles is preset in the interactive application; or

Creating the camera coordinate system according to the visual angle creating instruction of the first user; or

And creating the camera coordinate system according to a preset camera coordinate system in the interactive application.

8. An apparatus for displaying a screen, comprising:

the building module is used for building a camera coordinate system and a world coordinate system of the interactive application; the interactive application is used for a plurality of users to interact in a virtual scene; the camera coordinate system is a three-dimensional coordinate system determined with a perspective of a first user in the interactive application;

the processing module is used for determining the three-dimensional coordinates of at least one object in the picture to be displayed of the interactive application under the world coordinate system;

converting three-dimensional coordinates of the at least one object in the world coordinate system into pixel coordinates in a pixel coordinate system through the camera coordinate system; the pixel coordinates are used to display the at least one object.

9. A computing device, comprising:

a memory for storing program instructions;

a processor for calling program instructions stored in said memory to perform the method of any of claims 1 to 7 in accordance with the obtained program.

10. A computer-readable storage medium having stored thereon computer-executable instructions for causing a computer to perform the method of any one of claims 1 to 7.

Technical Field

The present invention relates to the field of image display, and in particular, to a method and an apparatus for displaying a picture.

Background

At present, online games (such as chess and card games, such as a fighting landlord, a three kingdoms killer and the like) generally adopt cartoon character head portraits to replace head portraits of real users, so that the whole game interface is not real enough, and the users lack the sense of pleasure of face-to-face game playing. In the prior art, when a user uses a scene online game of video conversation, the real-time picture of the user replaces an animation character, and double pleasure of playing and video chatting is realized. However, the method simply replaces the head portrait of the cartoon character with the video picture, and lacks the sense of real presence. Particularly, the spatial position relationship between the game scene and the video picture of the user in the game is disordered, so that the display is not real, and the game experience of the user is influenced.

Therefore, there is a need for a picture display method for improving the realistic presence, which provides users with realistic game experience and improves the experience of users.

Disclosure of Invention

The embodiment of the invention provides a picture display method and device, which are used for improving the real presence of a user in a game virtual scene, increasing the virtual reality effect of communication between the user and improving the game experience of the user.

In a first aspect, an embodiment of the present invention provides a method for displaying a picture, including: constructing a camera coordinate system and a world coordinate system of the interactive application; the interactive application is used for a plurality of users to interact in a virtual scene; the camera coordinate system is a three-dimensional coordinate system determined with a perspective of a first user in the interactive application;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system;

converting three-dimensional coordinates of the at least one object in the world coordinate system into pixel coordinates in a pixel coordinate system through the camera coordinate system; the pixel coordinates are used to display the at least one object.

According to the technical scheme, a camera coordinate system is determined according to the visual angle of a first user, then the three-dimensional coordinates of a picture to be displayed in the world coordinate system are converted into pixel coordinates in the pixel coordinate system through the camera coordinate system, the display picture of at least one object is determined according to the pixel coordinates of the at least one object, and the display picture is sent to a display screen to be displayed, so that the display picture in the two-dimensional pixel coordinate system has a three-dimensional stereoscopic vision effect, namely a virtual reality effect. Therefore, the real presence of the user in the game virtual scene is improved, the virtual reality effect of communication between the user and the user is increased, and the game experience of the user is improved.

Optionally, converting the three-dimensional coordinates of the at least one object in the world coordinate system to pixel coordinates in a pixel coordinate system by the camera coordinate system, including:

determining a conversion relationship between the camera coordinate system and the world coordinate system; the transformation relation is used for determining three-dimensional coordinates of the at least one object in the camera coordinate system;

determining imaging parameters under the camera coordinate system; the imaging parameters are used for converting three-dimensional coordinates of the at least one object in the camera coordinate system into two-dimensional pixel coordinates in a pixel coordinate system;

and determining the pixel coordinates of the at least one object according to the conversion relation and the imaging parameters.

According to the technical scheme, the relation between the world coordinate systems and the pixel coordinate systems is determined according to the camera coordinate systems, so that the coordinates of the coordinates in the world coordinate systems under the pixel coordinate systems can be obtained, the three-dimensional coordinates are converted into the two-dimensional coordinates, the two-dimensional coordinates have the three-dimensional stereoscopic vision effect, the real presence of a user in a game virtual scene is improved, the virtual reality effect of communication between the user and the user is improved, and the game experience of the user is improved.

Optionally, the at least one object is a scene object within the virtual scene;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

and according to the geometric figure of the scene object, constructing a geometric function of the scene object in the world coordinate system, thereby determining the three-dimensional coordinates of the scene object in the world coordinate system.

According to the technical scheme, after the three-dimensional coordinates of the scene object in the world coordinate system are determined, the coordinates of the scene object in the virtual scene in the pixel coordinate system are obtained according to the scheme, so that the real presence of the user in the game virtual scene is improved.

Optionally, the at least one object is a user window within the virtual scene;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

determining a user window in the virtual scene;

and determining the three-dimensional coordinates of the user window in the world coordinate system, so as to obtain the three-dimensional coordinates of the user window in the world coordinate system.

Optionally, the at least one object is a second user within the virtual scene; the second user is any one of a plurality of users except the first user;

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

acquiring an interactive picture of the second user from the video picture of the second user;

projecting the interactive picture of the second user into a user window of the second user;

and determining the three-dimensional coordinates of the user window of the second user in the world coordinate system, so as to obtain the three-dimensional coordinates of the second user in the world coordinate system.

According to the technical scheme, after the three-dimensional coordinates of the interactive picture and the user window in the world coordinate system are determined, the coordinates of the interactive picture and the user window in the pixel coordinate system are obtained according to the scheme, so that the virtual reality effect of communication between the user and the user is improved, and the game experience of the user is improved.

Optionally, the three-dimensional origin of coordinates of the world coordinate system is obtained by projecting the three-dimensional origin of coordinates of the camera coordinate system in a direction perpendicular to the ground.

The three-dimensional coordinate origin of the camera coordinate system is projected in the direction vertical to the ground and preset as the three-dimensional coordinate origin of the world coordinate system, so that the complexity of the conversion relation between the camera coordinate system and the world coordinate system is reduced, and the calculation amount is reduced.

Optionally, constructing a camera coordinate system of the interactive application includes:

acquiring a camera coordinate system corresponding to the view angle switching instruction according to the view angle switching instruction of the first user; a camera coordinate system with a plurality of visual angles is preset in the interactive application; or creating the camera coordinate system according to the visual angle creating instruction of the first user; or creating the camera coordinate system according to a preset camera coordinate system in the interactive application.

According to the technical scheme, different display picture effects are provided for the user to select, different preferences of the user are met, and game experience of the user is improved.

In a second aspect, an embodiment of the present invention provides an apparatus for displaying a screen, including:

the building module is used for building a camera coordinate system and a world coordinate system of the interactive application; the interactive application is used for a plurality of users to interact in a virtual scene; the camera coordinate system is a three-dimensional coordinate system determined with a perspective of a first user in the interactive application;

the processing module is used for determining the three-dimensional coordinates of at least one object in the picture to be displayed of the interactive application under the world coordinate system;

converting three-dimensional coordinates of the at least one object in the world coordinate system into pixel coordinates in a pixel coordinate system through the camera coordinate system; the pixel coordinates are used to display the at least one object.

Optionally, the processing module is specifically configured to:

determining a conversion relationship between the camera coordinate system and the world coordinate system; the transformation relation is used for determining three-dimensional coordinates of the at least one object in the camera coordinate system;

determining imaging parameters under the camera coordinate system; the imaging parameters are used for converting three-dimensional coordinates of the at least one object in the camera coordinate system into two-dimensional pixel coordinates in a pixel coordinate system;

and determining the pixel coordinates of the at least one object according to the conversion relation and the imaging parameters.

Optionally, the at least one object is a scene object within the virtual scene;

the processing module is specifically configured to:

and according to the geometric figure of the scene object, constructing a geometric function of the scene object in the world coordinate system, thereby determining the three-dimensional coordinates of the scene object in the world coordinate system.

Optionally, the at least one object is a user window within the virtual scene;

the processing module is specifically configured to:

determining a user window in the virtual scene;

and determining the three-dimensional coordinates of the user window in the world coordinate system, so as to obtain the three-dimensional coordinates of the user window in the world coordinate system.

Optionally, the at least one object is a second user within the virtual scene; the second user is any one of a plurality of users except the first user;

the processing module is specifically configured to:

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

acquiring an interactive picture of the second user from the video picture of the second user;

projecting the interactive picture of the second user into a user window of the second user;

and determining the three-dimensional coordinates of the user window of the second user in the world coordinate system, so as to obtain the three-dimensional coordinates of the second user in the world coordinate system.

Optionally, the three-dimensional origin of coordinates of the world coordinate system is obtained by projecting the three-dimensional origin of coordinates of the camera coordinate system in a direction perpendicular to the ground.

Optionally, the processing module is specifically configured to:

acquiring a camera coordinate system corresponding to the view angle switching instruction according to the view angle switching instruction of the first user; a camera coordinate system with a plurality of visual angles is preset in the interactive application; or creating the camera coordinate system according to the visual angle creating instruction of the first user; or creating the camera coordinate system according to a preset camera coordinate system in the interactive application.

In a third aspect, an embodiment of the present invention further provides a computing device, including:

a memory for storing program instructions;

and the processor is used for calling the program instruction stored in the memory and executing the picture display method according to the obtained program.

In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where computer-executable instructions are stored, and the computer-executable instructions are configured to enable a computer to execute the above-mentioned method for displaying a screen.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic view of a chess game according to an embodiment of the present invention;

FIG. 2 is a system architecture according to an embodiment of the present invention;

FIG. 3 is a flowchart of a method for displaying a frame according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a camera coordinate system according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a world coordinate system provided by an embodiment of the present invention;

fig. 6 is a schematic diagram of a camera coordinate system according to an embodiment of the present invention;

fig. 7 is a schematic diagram of a scene object in a virtual scene according to an embodiment of the present invention;

fig. 8 is a schematic diagram of a scene object in a virtual scene according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating a user window according to an embodiment of the present invention;

FIG. 10 is a diagram illustrating a user window according to an embodiment of the present invention;

FIG. 11 is a schematic diagram of a display screen according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of an image display apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the prior art, chess and card games lack real presence in a video chat scene while playing, and especially, the spatial position relationship between a table and a user in a virtual scene is disordered to cause unrealistic display, and fig. 1 exemplarily shows a scene schematic diagram of a chess and card game. As shown in fig. 1, the coordinate systems of the character and the desktop in the image are not consistent, that is, the coordinate systems of the character and the desktop at the display viewing angle are not consistent, and the coordinate system at the display viewing angle is not consistent with the coordinate system at the real viewing angle, which results in unrealistic display. Specifically, the desktop in the virtual scene lacks a spatial mapping relationship with the senses of the eyes in the real space, that is, lacks a mapping relationship between the geometric shape of the desktop in the real space and the geometric shape on the displayed two-dimensional picture, which results in the virtual table being unreal, and the character lacks a spatial mapping relationship with the senses of the eyes in the real space, that is, lacks a mapping relationship between the spatial position of the character in the space and the spatial position on the displayed two-dimensional picture, which results in the character being unreal.

Therefore, the embodiment of the invention discloses a picture display method and a picture display device, so as to improve the picture display method of real telepresence, give the user a realistic game experience, and improve the experience of the user.

Fig. 2 exemplarily shows a system architecture to which the embodiment of the present invention is applicable, and the system architecture includes a server 201, an interactive terminal 1, an interactive terminal 2, and a display screen 202.

The interactive terminal 1 and the interactive terminal 2 are used for acquiring video pictures, such as video information and the like, of an interactive user through a camera. And then uploading the video frame to the server 201, and optionally, there may be a plurality of interactive terminals, which is not limited herein. The interactive terminal can be an intelligent device, such as a mobile phone, a tablet computer, a notebook computer and the like, and the camera can be an external camera of the terminal, an internal camera of the terminal or a television camera and the like.

The server 201 is configured to construct a camera coordinate system and a world coordinate system, determine a conversion relationship between the camera coordinate system and the world coordinate system and an imaging parameter in the camera coordinate system, determine, according to coordinates of an interactive picture in the video picture and a user window in the virtual scene in the world coordinate system, pixel coordinates of the interactive picture in the video picture and the user window in the virtual scene in the pixel coordinate system through the conversion relationship and the imaging parameter, determine a display picture (including the video picture of the interactive user and the virtual game scene) according to the pixel coordinates of the interactive picture in the video picture and the user window in the virtual scene in the pixel coordinate system, and send the display picture to the display screen 202.

The display screen 202 may be a liquid crystal display, an organic light emitting display, a projection device, or the like, and displays a display screen.

It should be noted that the structure shown in fig. 2 is only an example, and the embodiment of the present invention is not limited thereto.

Based on the above description, fig. 3 exemplarily illustrates a flow of a method for screen display provided by an embodiment of the present invention, and the flow can be executed by a device for screen display.

As shown in fig. 3, the process specifically includes:

step 301, a camera coordinate system and a world coordinate system of the interactive application are constructed.

In the embodiment of the invention, the interactive application is used for a plurality of users to interact in a virtual scene, and the camera coordinate system is a three-dimensional coordinate system determined by the visual angle of the first user in the interactive application. As shown in fig. 4, fig. 4 exemplarily shows a schematic diagram of a camera coordinate system, where a in fig. 4 is a top view of the camera coordinate system, b in fig. 4 is a side view of the camera coordinate system, and according to the point of tangency of the first user with the real game table is an origin, an x-axis is parallel to the table, a z-axis is toward the table (i.e., the direction of sight), and a y-axis is perpendicular to the plane of the xOz-axis. It should be noted that the camera coordinate system and the world coordinate system may be preset to reduce the number of calculation links.

FIG. 5 is a schematic diagram illustrating an exemplary world coordinate system, as shown in FIG. 5, wherein a in FIG. 5 is a top view of the world coordinate system, and b in FIG. 5 is a side view of the world coordinate system, with an origin being a tangent point of the first user with the real game table, x_wAxis parallel to table top, z_wAxial normal to the desktop (i.e. line of sight), y_wAxis perpendicular to x_wOz_wThe axial plane is vertical.

Further, a camera coordinate system corresponding to the view angle switching instruction is obtained according to the view angle switching instruction of the first user, wherein the camera coordinate system with multiple view angles is preset in the interactive application, or the camera coordinate system is created according to the view angle creating instruction of the first user, or the camera coordinate system is created according to the camera coordinate system preset in the interactive application. Fig. 6 is a schematic diagram exemplarily showing a camera coordinate system, as shown in fig. 6, a in fig. 6 is a top view of the camera coordinate system, b in fig. 6 is a side view of the camera coordinate system, according to the center of a real game table as an origin, an x-axis is parallel to the table, a z-axis is toward the table (i.e., a line of sight direction), and a y-axis is perpendicular to an xOz-axis plane. The first user may create a camera coordinate system according to information about the set viewing angle height, the tilt angle, the position, and the like defined by the creation instruction, and further switch the screen viewing angle.

Step 302, determining three-dimensional coordinates of at least one object in the picture to be displayed of the interactive application under the world coordinate system.

In the embodiment of the present invention, the coordinates of the object in the picture to be displayed in the world coordinate system are determined according to the constructed world coordinate system, for example, the radius of the real game table is set to 0.5 m, and as can be seen from fig. 5, the world coordinate of the center of the real game table is (0, 0, 0.5). It should be noted that one object may include a scene object within the virtual scene and/or a user window within the virtual scene and a plurality of interactive users (e.g., a second user and a third user) within the virtual scene.

Step 303, converting the three-dimensional coordinates of the at least one object in the world coordinate system into pixel coordinates in a pixel coordinate system through the camera coordinate system.

In the embodiment of the invention, the three-dimensional coordinates of at least one object in the world coordinate system are converted into the pixel coordinates in the pixel coordinate system according to the relationship between the world coordinate system and the pixel coordinate system.

Further, a conversion relation between a camera coordinate system and a world coordinate system is determined, wherein the conversion relation is used for determining a three-dimensional coordinate of at least one object in the camera coordinate system, and then an imaging parameter in the camera coordinate system is determined, wherein the imaging parameter is used for converting the three-dimensional coordinate of the at least one object in the camera coordinate system into a two-dimensional pixel coordinate in the pixel coordinate system, and then the pixel coordinate of the at least one object is determined according to the conversion relation and the imaging parameter.

In the embodiment of the invention, the world coordinate system can be obtained by rotating and translating the camera coordinate system, namely, the world coordinate system and the camera coordinate system have a conversion relation, and the camera coordinate system can be obtained according to the imaging parameters, so that the coordinates of the pixel coordinate system can be expressed by the coordinates of the world coordinate system according to the imaging parameters and the conversion relation.

For example, the three-dimensional origin of coordinates of the world coordinate system may be obtained by projecting the three-dimensional origin of coordinates of the camera coordinate system in a direction perpendicular to the ground, and x of the world coordinate system_wThe axes can also be obtained by projecting the x-axis of the camera coordinate system in the direction perpendicular to the ground, so as to reduce the complexity of the relation between the world coordinate system and the pixel coordinate system. The relation of the world coordinate system to the pixel coordinate system is set forth in the example below with reference to fig. 4 and 5 above.

Example 1

As shown in fig. 4 and 5, the world coordinate system in fig. 5 can be obtained by rotating the pitch angle of the first user in the camera coordinate system in fig. 4 to be parallel to the real game table and then translating the camera coordinate system downward along the y-axis, so that the relationship between the three-dimensional coordinate points in the camera coordinate system and the three-dimensional coordinate points in the world coordinate system can be expressed as the following formula (1).

Wherein, R is a rotation matrix, and T is an offset vector.

Since the three-dimensional origin of the world coordinate system is perpendicular to the ground through the three-dimensional origin of the camera coordinate system according to the preset position of the camera coordinate system and the world coordinate system (i.e., the distance between the pitch angle of the first user and the y-axis in fig. 4)Projected in the direction, x of the world coordinate system_wThe axes are projected from the x-axis of the camera coordinate system in the direction perpendicular to the ground, so the R, T matrix in equation (1) can be derived:

where h is the distance between the camera coordinate system and the world coordinate system on the y-axis, and θ is the pitch angle of the first user.

In the embodiment of the present invention, a relationship between a camera coordinate system and a pixel coordinate system can be obtained according to an imaging parameter, and specifically, a three-dimensional coordinate point in the camera coordinate system is perspective projected to the pixel coordinate system according to the imaging parameter, so that:

wherein f is an imaging parameter.

Therefore, the two-dimensional pixel coordinates of the three-dimensional coordinate points in the world coordinate system projected in the pixel coordinate system according to perspective can be obtained as follows:

the two-dimensional pixel coordinates of the three-dimensional coordinate points under the world coordinate system under the pixel coordinate system according to perspective projection can be obtained by the formula (2):

wherein u is_oAnd v_oIs the pixel coordinate system of the optical centerAnd (4) marking.

For example, when the pitch angle θ of the first user is equal to 0, sin θ is equal to 0, and cos θ is equal to 1, so equation (2) can be simplified to equation (3) below:

illustratively, when at least one object is a scene object in the virtual scene, a geometric function of the scene object in the world coordinate system is constructed according to the geometric figure of the scene object, so as to determine the three-dimensional coordinates of the scene object in the world coordinate system.

In the embodiment of the invention, after the three-dimensional coordinates of the scene objects in the virtual scene in the world coordinate system are determined, the three-dimensional coordinates of the scene objects in the virtual scene in the world coordinate system are converted into the pixel coordinates in the pixel coordinate system according to the formula. In particular, the following description will be made in conjunction with the above-described real game table of fig. 5 in a specific example.

Example 2

With reference to fig. 4 to 6, the scene object in the virtual scene is a real game table, and it can be obtained that the three-dimensional coordinates of the real game table in the world coordinate system are:

where r is the radius of the real game table. According to the formula (2), the two-dimensional pixel coordinates of the real game table under the pixel coordinate system are obtained as follows:

and satisfies the following conditions:

FIG. 7 is a schematic diagram illustrating an exemplary scene object within a virtual scene, as shown in FIG. 7, when the first user has a pitch angle θ>When 0, the two-dimensional pixel coordinate of the real game table under the pixel coordinate system is one line, and u is equal to u_oIs to v₀Is weighed as the shaftBeing a parabola of the vertex.

Based on the above fig. 7, when the pitch angle θ of the first user is equal to 0, the two-dimensional pixel coordinates of the real game table in the pixel coordinate system can be obtained according to the three-dimensional coordinates of the real game table in the world coordinate system and the above equation (3):

and satisfies the following conditions:

fig. 8 is a schematic diagram illustrating an exemplary scene object in a virtual scene, and as shown in fig. 8, when the pitch angle θ of the first user is 0, the two-dimensional pixel coordinate of the real game table in the pixel coordinate system is one line with u-u_oAs an axis of symmetry, toBeing a parabola of the vertex.

For example, when at least one object is a user window in a virtual scene, the user window in the virtual scene is determined, and three-dimensional coordinates of the user window in a world coordinate system are determined, so that three-dimensional coordinates of the user window in the world coordinate system are obtained.

In the embodiment of the invention, after the three-dimensional coordinates of the vertex of the user window in the world coordinate system are determined, the three-dimensional coordinates of the vertex in the world coordinate system are projected into the pixel coordinate system in a perspective mode, and the two-dimensional pixel coordinates of the vertex in the pixel coordinate system are obtained. Specifically, the following will be explained in specific examples.

Example 3

Fig. 9 is a schematic diagram of an exemplary user window, and as shown in a in fig. 9, three-dimensional coordinates of a vertex ABCD of the user window of the third user in the world coordinate system are determined according to the radius r and the included angle σ of the real game table:

according to the above formula (2), the two-dimensional pixel coordinates of the vertex ABCD of the user window in the pixel coordinate system are:

as can be seen from the two-dimensional pixel coordinates of the vertices, the u coordinate of the two-dimensional pixel coordinate a 'at the point a is smaller than the u coordinate of the two-dimensional pixel coordinate C' at the point C, the u coordinate of the two-dimensional pixel coordinate B 'at the point B is larger than the u coordinate of the two-dimensional pixel coordinate D' at the point D, the v coordinate of the two-dimensional pixel coordinate a 'at the point a is larger than the v coordinate of the two-dimensional pixel coordinate B' at the point B, and the v coordinate of the two-dimensional pixel coordinate C 'at the point C is smaller than the v coordinate of the two-dimensional pixel coordinate D' at the point D, so that the two-dimensional pixel coordinate a 'B' C 'D' of the three-dimensional coordinate of the rectangular window ABCD in the world coordinate system is shown in fig. 9B. Similarly, for the window LMNS of the second user, the u coordinate of the two-dimensional pixel coordinate L 'of the point L is greater than the u coordinate of the two-dimensional pixel coordinate N' of the point N, the u coordinate of the two-dimensional pixel coordinate M 'of the point M is less than the u coordinate of the two-dimensional pixel coordinate S' of the point S, the v coordinate of the two-dimensional pixel coordinate L 'of the point L is greater than the v coordinate of the two-dimensional pixel coordinate M' of the point M, and the v coordinate of the two-dimensional pixel coordinate N 'of the point N is less than the v coordinate of the two-dimensional pixel coordinate S' of the point S.

Based on the above description of fig. 9, when the pitch angle θ of the first user is equal to 0, according to the radius r and the included angle σ of the real game table, it is determined that the two-dimensional pixel coordinate of the vertex ABCD of the user window of the third user in the pixel coordinate system is:

since the u coordinate of the two-dimensional pixel coordinate a 'at the point a is the same as the u coordinate of the two-dimensional pixel coordinate C' at the point C, and the u coordinate of the two-dimensional pixel coordinate B 'at the point B is the same as the u coordinate of the two-dimensional pixel coordinate D' at the point D, a 'C' is parallel to B 'D', and thus the lengths of a 'C' and B 'D' and a 'B' and C 'D' are respectively:

namely: a 'C'// B 'D' and A 'C' > B 'D', A 'B' C 'D', FIG. 10 illustrates an exemplary schematic diagram of a user window, and as can be seen in FIG. 10, the vertex ABCD of the user window of the third user in the world coordinate system is located at A 'B' C 'D' in the pixel coordinate system. Similarly, for the position of the vertex LMNS of the user window of the second user in the world coordinate system in the pixel coordinate system, the position of L 'M' N 'S' is specifically: m 'S'// L 'N', M 'S' L 'N', M 'L' S 'N'.

Illustratively, the at least one object is a second user in the virtual scene, the second user is any one of the users except the first user, an interactive picture of the second user is obtained from a video picture of the second user, the interactive picture of the second user is projected into a user window of the second user, and three-dimensional coordinates of the user window of the second user in the world coordinate system are determined, so that the three-dimensional coordinates of the second user in the world coordinate system are obtained.

Step 304, determining a display picture of the at least one object according to the pixel coordinates of the at least one object.

In the embodiment of the present invention, the pixel coordinates are used for displaying at least one object, and specifically, a display screen is determined according to the pixel coordinates of the at least one object and the resolution of the display screen, and then the determined display screen is sent to the display screen, so that the display screen displays the determined display screen. Fig. 11 exemplarily shows a display screen schematic diagram, as shown in fig. 11, a display image of a virtual desktop on an image coordinate system is obtained according to information such as a viewing angle selected by a first user, and then a camera of a second user is used to collect a user video, where the camera may be a television camera, a mobile phone camera, or the like, a built-in camera, or an external camera. And then displaying the collected video information of the second user in a user window of the corresponding second user in the world coordinate system.

For example, in order to increase the display effect of the display screen, the second user in the user window in the world coordinate system is in proportion to a real person or in proportion to a scene object (e.g., a virtual desktop) in a virtual scene, and then the user window in the world coordinate system is projected onto the scene object (e.g., a virtual game desktop) in the virtual scene in the pixel coordinate system for display according to information such as a size of the scene object (e.g., a size of the virtual desktop), a yaw angle, a height of a viewing angle, and a pitch angle of the first user in the virtual scene.

Based on the same technical concept, fig. 12 exemplarily shows a schematic structural diagram of a screen display apparatus according to an embodiment of the present invention, which can execute a flow of a screen display method.

As shown in fig. 12, the apparatus specifically includes:

a building module 1201, configured to build a camera coordinate system and a world coordinate system of an interactive application; the interactive application is used for a plurality of users to interact in a virtual scene; the camera coordinate system is a three-dimensional coordinate system determined with a perspective of a first user in the interactive application;

a processing module 1201, configured to determine three-dimensional coordinates of at least one object in a to-be-displayed screen of the interactive application in the world coordinate system;

converting three-dimensional coordinates of the at least one object in the world coordinate system into pixel coordinates in a pixel coordinate system through the camera coordinate system; the pixel coordinates are used to display the at least one object.

Optionally, the processing module 1202 is specifically configured to:

determining a conversion relationship between the camera coordinate system and the world coordinate system; the transformation relation is used for determining three-dimensional coordinates of the at least one object in the camera coordinate system;

determining imaging parameters under the camera coordinate system; the imaging parameters are used for converting three-dimensional coordinates of the at least one object in the camera coordinate system into two-dimensional pixel coordinates in a pixel coordinate system;

and determining the pixel coordinates of the at least one object according to the conversion relation and the imaging parameters.

Optionally, the at least one object is a scene object within the virtual scene;

the processing module 1202 is specifically configured to:

and according to the geometric figure of the scene object, constructing a geometric function of the scene object in the world coordinate system, thereby determining the three-dimensional coordinates of the scene object in the world coordinate system.

Optionally, the at least one object is a user window within the virtual scene;

the processing module 1202 is specifically configured to:

determining a user window in the virtual scene;

and determining the three-dimensional coordinates of the user window in the world coordinate system, so as to obtain the three-dimensional coordinates of the user window in the world coordinate system.

Optionally, the at least one object is a second user within the virtual scene; the second user is any one of a plurality of users except the first user;

the processing module 1202 is specifically configured to:

determining three-dimensional coordinates of at least one object in a picture to be displayed of the interactive application under the world coordinate system, wherein the three-dimensional coordinates comprise:

acquiring an interactive picture of the second user from the video picture of the second user;

projecting the interactive picture of the second user into a user window of the second user;

and determining the three-dimensional coordinates of the user window of the second user in the world coordinate system, so as to obtain the three-dimensional coordinates of the second user in the world coordinate system.

Optionally, the three-dimensional origin of coordinates of the world coordinate system is obtained by projecting the three-dimensional origin of coordinates of the camera coordinate system in a direction perpendicular to the ground.

Optionally, the processing module 1202 is specifically configured to:

acquiring a camera coordinate system corresponding to the view angle switching instruction according to the view angle switching instruction of the first user; a camera coordinate system with a plurality of visual angles is preset in the interactive application; or creating the camera coordinate system according to the visual angle creating instruction of the first user; or creating the camera coordinate system according to a preset camera coordinate system in the interactive application.

Based on the same technical concept, an embodiment of the present invention further provides a computing device, including:

a memory for storing program instructions;

and the processor is used for calling the program instruction stored in the memory and executing the picture display method according to the obtained program.

Based on the same technical concept, the embodiment of the invention also provides a computer-readable storage medium, which stores computer-executable instructions for causing a computer to execute the above-mentioned method for displaying pictures.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.