阅读说明：本技术 一种云游戏的图像渲染方法 (Image rendering method of cloud game ) 是由 黄颖 于 2021-01-14 设计创作，主要内容包括：本发明涉及一种云游戏的图像渲染方法,该方法在云游戏过程中,使客户端和服务器建立三个连接,分别用于传输操作指令流、渲染指令流和图像流,服务器在操作指令流中获取客户端负载信息,根据负载信息调节渲染指令流和图像流的比例关系。该方法有利于提高云游戏中的图像渲染效率,提高资源利用率。(The invention relates to an image rendering method of a cloud game, which is characterized in that in the process of the cloud game, a client and a server are connected and respectively used for transmitting an operation instruction stream, a rendering instruction stream and an image stream, the server obtains client load information in the operation instruction stream, and the proportional relation between the rendering instruction stream and the image stream is adjusted according to the load information. The method is beneficial to improving the image rendering efficiency in the cloud game and improving the resource utilization rate.)

1. An image rendering method of a cloud game, comprising the steps of:

step 100: three connections are established between a client and a server, wherein the first connection is used for transmitting an operation instruction stream between the client and the server, the second connection is used for sending a rendering instruction stream to the client by the server, and the third connection is used for sending an image stream to the client by the server;

the operation instruction stream comprises a game operation instruction sent by the client to the server and a game message and/or a command sent by the server to the client; the rendering instruction stream comprises rendering instructions of the game picture image by the server; the image stream comprises game picture images which are rendered by the server;

step 200: the client regularly sends the load information of the client to the server through a first connection;

step 300: the server determines the proportional relation between the rendering instruction stream and the image stream in a subsequent period of time according to the load information; the proportional relation refers to the proportion of the game image frames transmitted by adopting the rendering instruction stream and the game image frames transmitted by adopting the image stream;

the step 300 specifically includes:

step 310: the server reads the latest uploaded load information of the client and determines the load proportion of the client;

step 320: if the load proportion is less than a first threshold value T₁All game image frames are sent over the second connection in the form of rendering instructions;

step 330: if the load ratio is greater than a second threshold T₂If the game image frames are not sent to the server, the server does not send rendering instructions any more, and all the game image frames are directly rendered by the server and sent on the third connection;

step 340: if the load ratio is at T₁And T₂Then, the ratio P of the image stream is calculated as: p ═ L-T₁)/(T₂-T₁)]^wThe ratio of the rendering instruction stream to the image stream is (1-P) to P; wherein, L is a load proportion, w is a load weighting coefficient, and the value of w is predetermined according to a specific client type.

2. The method of claim 1, wherein the load information comprises a load proportion of the client GPU, and wherein the load proportion is expressed in percentage.

3. The method of claim 2, wherein the load information comprises a load proportion of a client CPU.

4. The method according to any one of claims 1 to 3, wherein the client actively uploads load information when its load fluctuation is large.

5. The method according to any one of claims 1 to 4, wherein the server determines whether to generate a rendering instruction or directly render the next frame of game image according to the proportional relationship, and then transmits the generated frame data to the client through the corresponding second connection or third connection.

6. The method of claim 5, wherein the frame data comprises a corresponding frame number.

7. The method of claim 6, wherein the frame number is a sequential number based on a generation time order.

8. The method of claim 6, wherein the frame number is a time stamp of a frame data generation time.

9. The method according to any one of claims 6 to 8, wherein the client determines the sequence of frames according to the frame numbers according to the received rendering instruction stream and image stream, executes the rendering instruction to generate the game image if the frame is the frame of the rendering instruction stream, and directly displays the frame image if the frame is the frame of the image stream.

10. The method of any of claims 1-9, wherein the subsequent period of time is a period of time from a current time to a next time the server receives load information.

[ technical field ] A method for producing a semiconductor device

The invention belongs to the field of computers, and particularly relates to an image rendering method of a cloud game.

[ background of the invention ]

The cloud game is a game mode based on cloud computing, all games are operated at a server side in an operation mode of the cloud game, a client side serving as game equipment does not need high-end performance equipment, only needs to receive game pictures of the server side, and does not need downloading and installing processes of the games. Particularly after 5G is developed, the cloud game has a wide development space based on the characteristics of high bandwidth and low time delay of 5G.

There are two types of typical cloud games: in the first type, the server performs all calculations of the game screen, generates a game screen image for each frame, and thus forms an image stream composed of successive image frames, and the server transmits the image stream to the client, which displays each frame of game screen image in sequence. In the first type, the client only needs to actually send the game instructions of the user to the server and then receive the image stream from the server for playing, and the performance requirements on the client are low, but the performance requirements on the server are high.

In the second type, the server does not generate a final game screen image, but generates a rendering instruction (for example, a rendering instruction of openGL) of the graphics library through calculation of the game logic, the server forms a rendering instruction stream by generating a rendering instruction of each frame of game image, sends the rendering instruction stream to the client, and the client parses and executes the rendering instruction to generate a corresponding game screen image. In the second type, the server performs logic calculation and generates rendering instructions, so that most of calculation work is borne, the client executes the rendering instructions and executes part of work, the capacity of the client can be fully utilized, the pressure of the server is reduced, and meanwhile, the rendering instruction stream has low cost on network bandwidth relative to the image stream. However, compared with the first type, the second type has a certain requirement on the performance of the client, but the current situation of the client is complex, and particularly when a smartphone is used for performing a cloud game, the performance of the smartphone is high or low, the variation fluctuation of the load of the smartphone is large, and the performance and the variation of the load of the smartphone easily affect the execution efficiency of rendering instructions.

[ summary of the invention ]

In order to solve the above problems in the prior art, the present invention provides an image rendering method for a cloud game.

The technical scheme adopted by the invention is as follows:

an image rendering method of a cloud game, comprising the steps of:

step 100: three connections are established between a client and a server, wherein the first connection is used for transmitting an operation instruction stream between the client and the server, the second connection is used for sending a rendering instruction stream to the client by the server, and the third connection is used for sending an image stream to the client by the server;

the operation instruction stream comprises a game operation instruction sent by the client to the server and a game message and/or a command sent by the server to the client; the rendering instruction stream comprises rendering instructions of the game picture image by the server; the image stream comprises game picture images which are rendered by the server;

step 200: the client regularly sends the load information of the client to the server through a first connection;

step 300: the server determines the proportional relation between the rendering instruction stream and the image stream in a subsequent period of time according to the load information; the proportional relation refers to the proportion of the game image frames transmitted by adopting the rendering instruction stream and the game image frames transmitted by adopting the image stream;

the step 300 specifically includes:

step 310: the server reads the latest uploaded load information of the client and determines the load proportion of the client;

step 320: if the load proportion is less than a first threshold value T₁All game image frames are sent over the second connection in the form of rendering instructions;

step 330: if the load ratio is greater than a second threshold T₂If the game image frames are not sent to the server, the server does not send rendering instructions any more, and all the game image frames are directly rendered by the server and sent on the third connection;

step 340: if the load ratio is at T₁And T₂Then, the ratio P of the image stream is calculated as: p ═ L-T₁)/(T₂-T₁)]^wThe ratio of the rendering instruction stream to the image stream is (1-P) to P; wherein, L is a load proportion, w is a load weighting coefficient, and the value of w is predetermined according to a specific client type.

Further, the load information includes a load proportion of the client GPU, the load proportion being expressed in percentage.

Further, the load information includes a load proportion of the client CPU.

Further, when the load fluctuation of the client is large, the client actively uploads the load information.

Further, the server determines whether to generate a rendering instruction or directly render the next frame of game image according to the proportional relation, and then transmits the generated frame data to the client through the corresponding second connection or third connection.

Further, the frame data includes a corresponding frame number.

Further, the frame number is a sequential number based on a generation time order.

Further, the frame number is a time stamp of the frame data generation time.

Further, the client determines the sequence of the frames according to the received rendering instruction stream and the received image stream and the frame number, if the frames are the frames of the rendering instruction stream, the rendering instruction is executed to generate the game image, and if the frames are the frames of the image stream, the frame image is directly displayed.

Further, the subsequent period of time is a period of time from the current time to the next time the server receives the load information.

The invention has the beneficial effects that: according to the load condition of the client, different modes of game image rendering are realized through three connections, the capabilities of the client and the server are balanced, the image rendering efficiency in the cloud game is improved, and the resource utilization rate is improved.

[ description of the drawings ]

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, and are not to be considered limiting of the invention, in which:

fig. 1 is a diagram of a system for executing a cloud game according to the present invention.

[ detailed description ] embodiments

The present invention will now be described in detail with reference to the drawings and specific embodiments, wherein the exemplary embodiments and descriptions are provided only for the purpose of illustrating the present invention and are not to be construed as limiting the present invention.

Referring to fig. 1, a system structure diagram for executing a cloud game according to the present invention is shown, where the system includes a server and a client, the server is configured to receive a game instruction transmitted by the client, perform calculation on a game image, and return a calculation result to the client; the client is a specific game device used by a user, and is used for receiving a game instruction of the user, sending the game instruction to the server, receiving a calculation result of a game image from the server, and displaying a corresponding game image on a display of the client according to the calculation result.

It should be noted that although only one client is shown in fig. 1, in practical use, there may be multiple clients, and multiple clients may be simultaneously connected to the server for performing a cloud game, and typical clients may be a smart phone, a PC, a notebook, and so on. The server shown in fig. 1 is also only an example, and in practical applications, the server may also be a server cluster, which is only externally represented as a server.

Data is transmitted between the server and the client through a network, and three network connections are established between the server and the client to transmit data (the specific connections and data are described later) in fig. 1.

Based on the above system structure, the method flow of the present invention is described in detail below.

Step 100: three connections are established between the client and the server, wherein the first connection is used for transmitting an operation instruction stream between the client and the server, the second connection is used for sending a rendering instruction stream to the client by the server, and the third connection is used for sending an image stream to the client by the server.

Specifically, when the client needs to send a game operation instruction of the user to the server, the client sends the game operation instruction to the server over the first connection, and similarly, if the server needs to send a message or a command related to the game (for example, a response message to the game operation instruction) to the client, the client also sends the game operation instruction over the first connection. Therefore, the first connection is used for bidirectional transmission of operation instructions between the server and the client, and a bidirectional operation instruction stream is formed.

The second connection and the third connection are used for the server to send game pictures to the client side. The rendering instruction is sent on the second connection to form a rendering instruction stream; the game images are transmitted over the third connection, forming an image stream.

Different data are sent on different three connections, which is beneficial to improving the transmission and processing efficiency of the data.

Step 200: and the client side sends the load information of the client side to the server through the first connection at regular time.

The load information mainly includes a load proportion of the GPU at the client, which is expressed by a percentage, and of course, a person skilled in the art may also add load information of other components (for example, a CPU) at the client according to specific situations, which is not limited in the present invention.

Although the client is limited to upload the load information at regular time, the client may actively upload the load information at any time in a special case, for example, when the load fluctuation is large.

Step 300: and the server determines the proportional relation between the rendering instruction stream and the image stream in a subsequent period of time according to the load information.

Specifically, the proportional relationship refers to a proportion of the game image frames transmitted by the rendering instruction stream and the game image frames transmitted by the image stream. For example, if the server determines that the ratio of the rendering instruction stream to the image stream is 2:1, the server generates two corresponding frames of rendering instructions for the game images of the 1 st to 2 nd frames, and transmits the two frames of rendering instructions to the client side on the second connection; for the game image of the 3 rd frame, the server directly renders and generates the game image and transmits the game image to the client on the third connection; for the game images of the 4 th frame to the 5 th frame, the server generates corresponding two-frame rendering instructions and transmits the two-frame rendering instructions to the client side on the second connection; for the game image of the 6 th frame, the server directly renders and generates the game image and transmits the game image to the client on the third connection; and so on. Namely, 2-frame rendering commands are generated, and 1-frame game images are generated, so that a 2:1 proportional relation is formed.

In summary, for each frame of game picture image, the server determines whether the frame of image generates rendering instructions or directly renders the frame of image to generate a game image according to the proportional relationship, selects one of the rendering instructions and the game image, and then transmits the game image to the client through the corresponding second connection or third connection.

In order to facilitate the client to identify the sequence of the frames, the data of each frame needs to carry a corresponding frame number, and preferably, the frame number may be a sequential number based on the generation time sequence, or the frame number may be a timestamp of the generation time. The client determines the sequence of the frames according to the received rendering instruction stream and the received image stream and the frame number, if the frames are the frames of the rendering instruction stream, the rendering instruction is executed to generate game images, and if the frames are the frames of the image stream, the frame images can be directly displayed.

The subsequent period of time is generally a period of time from the current time to the next time the server receives the load information, that is, each time the server receives the load information of the client, the server needs to re-determine the proportional relationship between the rendering instruction stream and the image stream according to the new load information.

Further, the invention also provides a preferable scheme for determining the proportional relation, which specifically comprises the following steps:

step 310: and the server reads the latest uploaded load information of the client and determines the load proportion of the client.

As mentioned above, the load information transmitted by the client to the server is expressed in percentage, and preferably, the load proportion is the load percentage of the GPU at the client, for example, the load proportion is 100% when the GPU is in full-load operation.

Step 320: if the load proportion is less than a first threshold value T₁All game image frames are sent in the form of rendering instructions. I.e., the ratio of the rendering instruction stream to the image stream is 1: 0.

That is, when the load ratio of the client is low, the client can finish rendering by itself, and at this time, the server hands all rendering work to the client to finish, and does not transmit the game image directly, and all frames are sent in the rendering instruction stream of the second connection.

Step 330: if the load ratio is greater than a second threshold T₂And the server does not send the rendering instruction any more, all the game image frames are directly rendered by the server, and the game images are directly sent. I.e., the ratio of the rendering instruction stream to the image stream is 0: 1.

That is, when the client load ratio is high, the game image is not rendered by the client any more, but is directly rendered and transmitted by the server, and all frames are transmitted in the image stream of the third connection.

Apparently 0 < T₁＜T₂<1, in particular T₁And T₂The value of (a) can be selected according to the specific strategy situation.

Step 340: if the load ratio is at T₁And T₂Then, the ratio P of the image stream is calculated as: p ═ L-T₁)/(T₂-T₁)]^wThe ratio of rendering instruction stream to image stream is (1-P): P.

Wherein, L is the load proportion, w is the load weighting coefficient, and the value of w is determined according to the specific client type. Because different client conditions are different and the influence of load increase on the client is also different, the load weighting coefficient is introduced to represent the performance of the client. In general, the lower the client performance, the smaller the value of w, and vice versa.

Based on the method, the invention realizes different modes of game image rendering through three connections according to the load condition of the client, balances the capabilities of the client and the server, and is beneficial to improving the image rendering efficiency in the cloud game and improving the resource utilization rate.

The above description is only a preferred embodiment of the present invention, and all equivalent changes or modifications of the structure, characteristics and principles described in the present invention are included in the scope of the present invention.