阅读说明：本技术 数据传输方法、装置及服务器 (Data transmission method and device and server ) 是由 李超 王知明 范志刚 于 2019-07-16 设计创作，主要内容包括：本公开实施例提供的数据传输方法、装置及服务器,涉及云计算技术领域,能够解决目前云桌面技术中服务器无法支撑大量3D图像渲染工作任务以及升级服务器难度很高的问题。具体技术方案为：数据处理装置接收客户端的请求后转发给服务器；当该请求为图像渲染请求时,服务器获取图像渲染对应的指令；其中,指令包括：显卡配置指令、特效指令及绘图指令；服务器为客户端在数据处理装置中确定对应的图像处理单元,将指令传输给数据处理装置后对应的图像处理单元完成图像渲染后将图像数据传输给客户端；其中,在服务器内不需要根据指令进行图像渲染。(The data transmission method, the data transmission device and the server provided by the embodiment of the disclosure relate to the technical field of cloud computing, and can solve the problems that the server cannot support a large number of 3D image rendering work tasks and the difficulty in updating the server is high in the existing cloud desktop technology. The specific technical scheme is as follows: the data processing device receives the request of the client and forwards the request to the server; when the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out; the server determines a corresponding image processing unit in the data processing device for the client, transmits the instruction to the data processing device, finishes image rendering by the corresponding image processing unit and transmits image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server.)

1. A method of data transmission, comprising:

the server receives a request of the client forwarded by the data processing device;

When the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

the server determines an image processing unit corresponding to the client in the data processing device;

the server transmits the instruction to the data processing device so that the image processing unit transmits image data to the client after finishing image rendering according to the instruction; wherein image rendering according to the instructions is not required within the server.

2. The method of claim 1, further comprising, prior to the method:

The server divides an image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete the image rendering task.

3. The method according to claim 2, wherein the server determining an image processing unit in the data processing apparatus corresponding to the image rendering request of the client, comprises:

the server acquires the identification of the client from the image rendering request;

The server judges whether an image processing unit corresponding to the client exists or not according to the identification of the client;

And if the client does not exist, the server allocates a corresponding image processing unit for the client and binds the client and the corresponding image processing unit.

4. the method of claim 3, wherein the server allocates a corresponding image processing unit to the client, comprising:

The server judges whether an idle image processing unit exists at present;

if so, the server selects an idle image processing unit to be distributed to the client;

And if the image processing unit does not exist, the server selects an image processing unit to be distributed to the client according to a preset rule.

5. The method of claim 4, wherein the server selects an image processing unit to be allocated to the client according to a preset rule, comprising:

The server finds out the image processing unit with the least bound client number and distributes the image processing unit to the client; alternatively, the first and second electrodes may be,

and the server finds out the image processing unit with the lowest current processing load to be distributed to the client.

6. The method of claim 1, wherein the server obtains instructions corresponding to the rendering of the image, and comprises:

The server intercepts the drawing instruction sent by a D3D RT program in an operating system of the server to a user mode display driver;

The server intercepts the graphics kernel subsystem in the operating system of the server and transmits the graphics card configuration instruction and the special effect instruction to a display micro-port driver.

7. the method of claim 1, further comprising:

and when the request is a request except the image rendering request, the server transmits the processing result to the data processing device after processing according to the request so that the data processing device forwards the processing result to the client.

8. A method of data transmission, comprising:

the data processing device receives a request of a client;

the data processing device sends a request of the client to a server;

When the request is an image rendering request, the data processing device receives an instruction corresponding to the image rendering sent by the server; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

The data processing device completes image rendering through an image processing module of the data processing device according to the instruction;

and the data processing device sends the image data obtained after the image rendering is finished to the client.

9. The method of claim 8, wherein the data processing apparatus performs image rendering by an image processing module of the data processing apparatus according to the instruction, comprising:

an image processing unit corresponding to the client in the image processing module finishes image rendering according to the instruction; wherein the image processing unit is the image rendering request which is distributed by the server in the image processing module for the client to correspondingly process the client.

10. The method of claim 8, further comprising:

when the request is a request except the image rendering request, the data processing device receives a processing result obtained after the server processes according to the request;

And the data processing device sends the processing result to the client.

11. a server, comprising:

the receiving module is used for receiving the request of the client forwarded by the data processing device;

the acquisition module is used for acquiring an instruction corresponding to image rendering when the request is an image rendering request; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

The determining module is used for determining an image processing unit corresponding to the client in the data processing device;

the transmission module is used for transmitting the instruction to the data processing device so that the image processing unit can complete image rendering according to the instruction and then transmit image data to the client; wherein image rendering according to the instructions is not required within the server.

12. the server of claim 1, further comprising:

A dividing module for dividing an image processing module of the data processing apparatus into N image processing units; wherein each image processing unit can independently complete the image rendering task.

13. The server according to claim 12, wherein the determining module is specifically configured to:

Acquiring the identification of the client from the image rendering request;

Judging whether an image processing unit corresponding to the client exists according to the identification of the client;

and if the client does not exist, distributing a corresponding image processing unit for the client, and binding the client and the corresponding image processing unit.

14. the server according to claim 13, wherein the determining module is specifically configured to:

Judging whether an idle image processing unit exists at present;

If so, selecting an idle image processing unit to be distributed to the client;

If not, selecting one image processing unit to be distributed to the client according to a preset rule.

15. the server according to claim 14, wherein the determining module is specifically configured to:

finding out the image processing unit with the least bound client number to be distributed to the clients; alternatively, the first and second electrodes may be,

and finding out the image processing unit with the lowest current processing load to be distributed to the client.

16. the server according to claim 1, wherein the obtaining module is specifically configured to:

intercepting the drawing instruction sent by a D3D RT program in an operating system of the server to a user mode display driver;

and intercepting the graphics kernel subsystem in the operating system of the server to send the graphics card configuration instruction and the special effect instruction to a display micro-port driver.

17. the server of claim 1, further comprising:

and the processing module is used for calling the transmission module to transmit the processing result to the data processing device after the server processes the request according to the request so as to enable the data processing device to forward the processing result to the client when the request is other than the image rendering request.

18. A data processing apparatus, comprising: the device comprises a communication module, a processor, an image processing module and a memory; the data processing device is communicated with the client and the server through the communication module respectively; the communication module includes: the system comprises a request receiving module, a request forwarding module, an instruction receiving module, an image acquisition module and an image transmission module;

the request receiving module is used for receiving the request of the client;

the request forwarding module is used for sending the request of the client to the server;

the instruction receiving module is used for receiving an instruction corresponding to image rendering sent by the server by the data processing device when the request is an image rendering request; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

the memory is used for caching the instruction corresponding to the image rendering received from the server and the image obtained after the image rendering is finished;

the processor is used for indicating the image processing module to finish image rendering according to the instruction;

The image acquisition module is used for acquiring the image obtained after the image rendering is finished;

and the image transmission module is used for sending the image obtained after the image rendering is finished to the client.

19. the data processing apparatus of claim 18, wherein the processor is specifically configured to:

instructing an image processing unit corresponding to the client in the image processing module to finish image rendering according to the instruction; wherein the image processing unit is the image rendering request which is distributed by the server in the image processing module for the client to correspondingly process the client.

20. The data processing apparatus of claim 18, wherein the image acquisition module is further configured to: when the request is a request except the image rendering request, acquiring an image which is sent by the server and processed according to the request;

correspondingly, the image transmission module is further configured to: and sending the image which is sent by the server and processed according to the request to the client.

Technical Field

The disclosure relates to the technical field of cloud computing, and in particular to a data transmission method, a data transmission device and a server.

Background

The cloud desktop technology refers to a technology that a user remotely accesses a server through a client, and the server provides a desktop image of a current server host to the remote client in a virtual mode. The server transmits the images to the user through screen copy, and the user obtains the desktop image sequence. After the image is compressed, the image is transmitted to a cloud terminal through a network, and the terminal decodes the image to obtain desktop image content.

in many existing scenarios, for example, network games, three-dimensional (3D) videos, or 3D remote desktop transmission, etc., 3D images need to be transmitted and displayed on a client. In this case, the server often needs to support a large number of clients for 3D image rendering at the same time, which requires very high Graphics Processing Unit (GPU) resources on the server, but the existing server generally does not have such powerful GPU resources, and even does not have 3D rendering capability. If the configuration upgrading is directly carried out on the server, the server is required to be interrupted, a certain time cost is consumed by the server within a period of interruption, and the configuration upgrading process is required to be carried out on the inside of a server rack, so that the process is very complicated and difficult.

Disclosure of Invention

The embodiment of the disclosure provides a data transmission method, a data transmission device and a server, and can solve the problems that the server cannot support a large number of 3D image rendering tasks and the server is difficult to upgrade in the existing cloud desktop technology. The technical scheme is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a data transmission method, including:

the server receives a request of the client forwarded by the data processing device;

When the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

The server determines an image processing unit corresponding to the client in the data processing device;

The server transmits the instruction to the data processing device so that the image processing unit completes image rendering according to the instruction and then transmits the image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server.

The image rendering task of the client is completed by the data processing device, which is a device disposed between the server and the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

In one embodiment, the method further comprises, before:

the server divides an image processing module of the data processing device into N image processing units; wherein each image processing unit can independently complete the image rendering task.

the server divides an image processing module of the data processing device into a plurality of image processing units which can independently complete image tasks and distributes the image processing units to each client, so that the image rendering can be completed by utilizing the image processing units more efficiently, and the phenomenon that a plurality of clients compete for resources and wait for longer processing time is avoided.

In one embodiment, the server determines an image processing unit of the data processing apparatus that corresponds to processing an image rendering request of the client, including:

the server acquires the identification of the client from the image rendering request;

the server judges whether an image processing unit corresponding to the client exists or not according to the identification of the client;

and if the image processing unit does not exist, the server allocates the corresponding image processing unit for the client and binds the client and the corresponding image processing unit.

By binding the image processing units with the client, the corresponding image processing units can be conveniently selected for direct processing after the client sends the image rendering request, and the image rendering requests of the same client can be handed to the same image processing unit, so that the processing efficiency is improved.

in one embodiment, the server allocates a corresponding image processing unit to the client, including:

the server judges whether an idle image processing unit exists at present;

If yes, the server selects an idle image processing unit to be distributed to the client;

if not, the server selects one image processing unit to be distributed to the client according to a preset rule.

in one embodiment, the server selects one image processing unit to distribute to the client according to a preset rule, and the method comprises the following steps:

The server finds out the image processing unit with the least bound client number and distributes the image processing unit to the client; alternatively, the first and second electrodes may be,

the server finds out the image processing unit with the lowest current processing load to be distributed to the client.

in one embodiment, the server obtains instructions for rendering an image, comprising:

The server intercepts and captures a drawing instruction sent by a D3D RT program in an operating system of the server to a user mode display driver;

The server intercepts and captures a graphics kernel subsystem in an operating system of the server and a display card configuration instruction and a special effect instruction which are sent to a display micro-port driver.

the server operating system generates an instruction according to the request of the 3D image rendering of the client, and can execute the 3D image rendering without a server by intercepting the instruction, so that the change is small, and no transformation burden is caused on the server.

in one embodiment, the method further comprises:

and when the request is a request except for the image rendering request, the server transmits a processing result to the data processing device after processing according to the request so that the data processing device forwards the processing result to the client.

for a general cloud desktop client request, a server can directly process the request and directly process the request in the server without being processed by a data processing device, and only a processing result, such as computer desktop image data, needs to be forwarded to the client through the data processing device.

According to a second aspect of the embodiments of the present disclosure, there is provided a data transmission method, including:

The data processing device receives a request of a client;

the data processing device sends a request of a client to a server;

When the request is an image rendering request, the data processing device receives an instruction corresponding to image rendering sent by the server; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

The data processing device finishes image rendering through an image processing module of the data processing device according to the instruction;

And the data processing device sends the image data obtained after the image rendering is finished to the client.

the image rendering task of the client is completed by the data processing device, which is a device disposed between the server and the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

in one embodiment, the data processing apparatus performs image rendering by an image processing module of the data processing apparatus according to the instruction, including:

the image processing unit corresponding to the client in the image processing module finishes image rendering according to the instruction; the image processing unit is an image rendering request which is distributed by the server in the image processing module for the client and is used for correspondingly processing the client.

the server divides an image processing module of the data processing device into a plurality of image processing units which can independently complete image tasks and distributes the image processing units to each client, so that the image rendering can be completed by utilizing the image processing units more efficiently, and the phenomenon that a plurality of clients compete for resources and wait for longer processing time is avoided.

in one embodiment, the method further comprises:

when the request is a request except for the image rendering request, the data processing device receives a processing result obtained after the server carries out processing according to the request;

and the data processing device sends the processing result to the client.

For a general cloud desktop client request, a server can directly process the request and directly process the request in the server without being processed by a data processing device, and only a processing result, such as computer desktop image data, needs to be forwarded to the client through the data processing device.

According to a third aspect of the embodiments of the present disclosure, there is provided a server, including:

the receiving module is used for receiving the request of the client forwarded by the data processing device;

The acquisition module is used for acquiring an instruction corresponding to image rendering when the request is an image rendering request; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

The determining module is used for determining an image processing unit of a corresponding client in the data processing device;

The transmission module is used for transmitting the instruction to the data processing device so that the image processing unit can complete image rendering according to the instruction and then transmit the image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server.

The image rendering task of the client is completed by the data processing device, which is a device disposed between the server and the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

in one embodiment, the server further comprises:

a dividing module for dividing an image processing module of the data processing apparatus into N image processing units; wherein each image processing unit can independently complete the image rendering task.

the server divides an image processing module of the data processing device into a plurality of image processing units which can independently complete image tasks and distributes the image processing units to each client, so that the image rendering can be completed by utilizing the image processing units more efficiently, and the phenomenon that a plurality of clients compete for resources and wait for longer processing time is avoided.

in one embodiment, the determining module is specifically configured to:

Acquiring an identifier of a client from an image rendering request;

judging whether an image processing unit corresponding to the client exists according to the identification of the client;

and if the image processing unit does not exist, distributing the corresponding image processing unit for the client, and binding the client and the corresponding image processing unit.

by binding the image processing units with the client, the corresponding image processing units can be conveniently selected for direct processing after the client sends the image rendering request, and the image rendering requests of the same client can be handed to the same image processing unit, so that the processing efficiency is improved.

in one embodiment, the determining module is specifically configured to:

judging whether an idle image processing unit exists at present;

if yes, selecting an idle image processing unit to be distributed to the client;

If not, selecting one image processing unit to distribute to the client according to a preset rule.

in one embodiment, the determining module is specifically configured to:

finding out the image processing unit with the least bound client number to be distributed to the clients; alternatively, the first and second electrodes may be,

And finding out the image processing unit with the lowest current processing load to be distributed to the client.

In one embodiment, the obtaining module is specifically configured to:

capturing a drawing instruction sent by a D3D RT program in an operating system of a server to a user mode display driver;

And intercepting and capturing a display card configuration instruction and a special effect instruction which are sent to a display micro-port driver by a graphics kernel subsystem in an operating system of the server.

The server operating system generates an instruction according to the request of the 3D image rendering of the client, and can execute the 3D image rendering without a server by intercepting the instruction, so that the change is small, and no transformation burden is caused on the server.

In one embodiment, the server further comprises:

And the processing module is used for calling the transmission module to transmit the processing result to the data processing device after the server processes the request according to the request so as to enable the data processing device to forward the processing result to the client when the request is other than the image rendering request.

for a general cloud desktop client request, a server can directly process the request and directly process the request in the server without being processed by a data processing device, and only a processing result, such as computer desktop image data, needs to be forwarded to the client through the data processing device.

according to a fourth aspect of the embodiments of the present disclosure, a data processing apparatus includes: the device comprises a communication module, a processor, an image processing module and a memory; the data processing device is communicated with the client and the server through the communication module respectively; the communication module includes: the system comprises a request receiving module, a request forwarding module, an instruction receiving module, an image acquisition module and an image transmission module;

the request receiving module is used for receiving a request of a client;

The request forwarding module is used for sending a request of the client to the server;

The instruction receiving module is used for receiving an instruction corresponding to image rendering sent by the server by the data processing device when the request is an image rendering request; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

the memory is used for caching an instruction corresponding to image rendering received from the server and an image obtained after the image rendering is finished;

The processor is used for indicating the image processing module to finish image rendering according to the instruction;

the image acquisition module is used for acquiring an image obtained after image rendering is finished;

and the image transmission module is used for sending the image obtained after the image rendering is finished to the client.

the image rendering task of the client is completed by the data processing device, which is a device disposed between the server and the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

In one embodiment, the processor is specifically configured to:

instructing an image processing unit of a corresponding client in the image processing module to finish image rendering according to the instruction; the image processing unit is an image rendering request which is distributed by the server in the image processing module for the client and is used for correspondingly processing the client.

The server divides an image processing module of the data processing device into a plurality of image processing units which can independently complete image tasks and distributes the image processing units to each client, so that the image rendering can be completed by utilizing the image processing units more efficiently, and the phenomenon that a plurality of clients compete for resources and wait for longer processing time is avoided.

In one embodiment, the image acquisition module is further configured to: when the request is a request except for an image rendering request, acquiring an image which is sent by a server and processed according to the request;

Correspondingly, the image transmission module is further configured to: and sending the image which is sent by the server and processed according to the request to the client.

for a general cloud desktop client request, a server can directly process the request and directly process the request in the server without being processed by a data processing device, and only a processing result, such as computer desktop image data, needs to be forwarded to the client through the data processing device.

According to the data transmission method, the data transmission device and the server, the data processing device receives the request of the client and forwards the request to the server; when the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out; the server determines a corresponding image processing unit in the data processing device for the client, transmits the instruction to the data processing device, finishes image rendering by the corresponding image processing unit and transmits image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server. Therefore, the workload of the server is greatly reduced, and even a large number of 3D image rendering tasks of the client can be supported without a graphics processor, and the server does not need to be upgraded.

it is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a system architecture diagram of an embodiment of the present disclosure;

FIG. 2 is a prior art WDDM driven model architecture diagram;

fig. 3 is a schematic flow chart of a data transmission method provided by an embodiment of the present disclosure;

FIG. 4 is a WDDM driven model architecture diagram of a server operating system in an embodiment of the disclosure;

fig. 5 is a schematic flow chart of another data transmission method provided by the embodiment of the present disclosure;

Fig. 6 is a schematic structural diagram of a server provided in the embodiment of the present disclosure;

Fig. 7 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure;

Fig. 8 is a schematic diagram illustrating an implementation effect provided by an embodiment of the present disclosure.

Detailed Description

reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

a schematic diagram of a system architecture of the embodiment of the disclosure is shown in fig. 1, where a server communicates with M (M is a positive integer) clients through a data processing device, requests of the M clients and responses of the server are forwarded through the data processing device, and the data processing device and the server both need to configure an operating system supporting a Windows Display Driver Model (WDDM).

WDDM is a display driver architecture that is initially supported in the Windows Vista operating system developed by microsoft corporation. WDDM is an innovation of the structure of the display driver in the Windows XP operating system, and can provide users with desktop experience with better performance and higher reliability while supporting new schemes, graphics and applications.

fig. 2 is a schematic diagram of a WDDM-driven model architecture, when an application program has an image rendering requirement, an operating system calls a related interface function of a D3D RT (Direct 3D runtime) program, and sends parameters related to image rendering (configuration parameters for a graphics card), special effect parameters (parameters related to special effects), drawing parameters, and the like to a D3D RT program. On the one hand, the D3D RT program interacts with a User Mode Display Driver (UMDD), to obtain the status of the display card and transmit the drawing instruction (the drawing instruction is generated according to the drawing parameter) to the user mode display driver, the user mode display driver translates the received drawing instruction and then puts the translated instruction into the memory, so as to facilitate the Display miniport driver (Display miniport driver) to read the translated command, meanwhile, the user mode display driver informs the D3D RT program of finishing the translation of the instruction, at this time, the D3D RT program informs the display miniport driver to obtain the translated instruction from the memory through a Graphics kernel subsystem (Direct Graphics kernel subsystem or Dxgkrnl. sys), and the data is transmitted to the video Memory of the video card in a Direct Memory Access (DMA) mode and then is executed by the video card; on the other hand, the D3D RT program transmits a graphics card configuration instruction (generated according to the graphics card configuration parameters) and a special effect instruction (generated according to the special effect parameters) to the graphics kernel subsystem. Among other things, the graphics hardware vendor (graphics card manufacturer) must provide the user-mode display driver and the display miniport driver. The user mode display driver is a Dynamic Link Library (DLL) loaded at runtime by Microsoft Direct3D (a set of 3D graphics programming interfaces developed by Microsoft corporation on Microsoft Windows operating system, which is part of DirectX and is currently widely supported by various display card manufacturers). The display miniport driver communicates with the DirectX (multimedia programming interface created by microsoft corporation) graphics kernel subsystem. The graphics kernel subsystem is a system file for the operating system to complete image display management, and participates in all image generation management of the system.

an embodiment of the present disclosure provides a data transmission method, as shown in fig. 3, the method includes the following steps:

101. The server receives the request of the client forwarded by the data processing device.

Specifically, the server and the client communicate with each other through the data processing device, the request from the client and the response from the server are both forwarded through the data processing device, and the data processing device is provided with an operating system similar to the operating system on the server, for example, an operating system supporting WDDM is installed.

102. when the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the display card comprises a configuration instruction, a special effect instruction and a drawing instruction.

In an embodiment, the step 102 of acquiring, by the server, an instruction corresponding to image rendering may specifically include:

102a, the server intercepts and captures a drawing instruction sent by a D3D RT program in an operating system of the server to a user mode display driver;

102b, the server intercepts and captures a graphics kernel subsystem in an operating system of the server and a display card configuration instruction and a special effect instruction sent by a display miniport driver.

specifically, fig. 4 shows a WDDM driver model architecture for a server provided in an embodiment of the present disclosure, and compared to fig. 2, a user-mode filter driver (or a virtual driver) is added between a D3D RT program in an operating system of the server and a user-mode display driver, and a kernel-mode filter driver (or a virtual driver) is added between a graphics kernel subsystem and a display miniport driver. The function of these two added filter drivers is described below:

1) User mode filter driver

the user mode filter driver is used for intercepting drawing instructions transmitted by the D3D RT program to the user mode display driver. The user mode filter driver is equivalent to a bridge between the D3D RT program and the user mode display driver, and listens for commands between them, but does not change the content of the commands themselves.

specifically, as shown in fig. 4, when the D3D RT program in the operating system of the server receives an image rendering request sent by the application layer, the D3D RT program firstly notifies a user mode display driver (for example, draw primative), and the user mode display driver converts the received drawing instruction to obtain an instruction that can be recognized by the display card, and transmits the instruction to the video memory of the display card in a DMA manner for instruction execution by the display card. In the embodiment of the disclosure, the user mode filter driver intercepts the drawing instruction before the user mode display driver executes the instruction conversion processing action.

2) kernel mode filter driver

The kernel mode filter driver is used for intercepting a graphics card configuration instruction and a special effect instruction which are sent to the display miniport driver by the graphics kernel subsystem, wherein the instructions are received by the D3D RT program from an application layer and are transmitted to the graphics kernel subsystem.

it should be noted that, for a virtualized server, since there is no physical graphics card, for such a scenario, when applying the technical solution of the embodiment of the present disclosure, a virtual user mode filter driver is set in a WDDM driver model architecture in an operating system of the virtualized server to serve as a role of a user mode display driver so as to intercept a drawing instruction, and a virtual display miniport driver is set to serve as a role of a display miniport driver so as to facilitate a graphics card configuration instruction and a special effect instruction.

103. The server determines an image processing unit of the data processing apparatus corresponding to the client.

In the embodiment of the present disclosure, the data Processing apparatus has an image rendering capability, that is, is configured with a Graphics Processing Unit (GPU), and the plurality of clients are connected to the server through the data Processing apparatus, in this case, 3D image rendering requirements from the plurality of clients are also very large, and the data Processing apparatus needs to configure a plurality of display cards as image Processing modules of the data Processing apparatus according to a task amount of the rendering requirements.

in one embodiment, the server divides an image processing module of the data processing apparatus into N image processing units; wherein each image processing unit can independently complete the image rendering task.

Specifically, the server logically divides an image processing module of the data processing device to obtain N (N is a positive integer) image processing units capable of independently completing an image rendering task after dividing the image processing module, specifically, the image processing units may be divided according to the number of the display cards, or the image processing units may be obtained by dividing the video memory resources of a plurality of display cards. It is worth mentioning that one image processing unit may provide 3D image rendering services to one or more clients.

The image processing module of the data processing device is divided into a plurality of image processing units which can independently complete image tasks and are distributed to the clients, so that the image rendering can be completed by the image processing units more efficiently, and the phenomenon that a plurality of clients contend for resources and wait for longer processing time is avoided.

in one embodiment, the server determines an image processing unit of the data processing apparatus that corresponds to processing an image rendering request of the client, including:

The server acquires the identification of the client from the image rendering request;

The server judges whether an image processing unit corresponding to the client exists or not according to the identification of the client;

and if the image processing unit does not exist, the server allocates the corresponding image processing unit for the client and binds the client and the corresponding image processing unit.

Specifically, the identifier of the client may be an IP Address, a Media Access Control Address (MAC Address), and the like. The image processing unit has two states, idle and occupied. The server sets a corresponding number for each image processing unit, records a corresponding state in real time at the server, and records the number of occupied clients and corresponding identifiers at the same time in an occupied state. In the embodiment of the disclosure, the server allocates a corresponding image processing unit to the client and binds the client and the image processing unit through the identifier of the client and the number of the image processing unit, so that the image processing resources of the data processing device can be efficiently managed, and image rendering services are provided for a plurality of clients.

It is worth mentioning that after a client is bound with an image processing unit, if the client does not send a 3D image rendering request for a long time, such as more than 30 minutes or 40 minutes, the binding relationship between the client and its corresponding image processing unit is released; after the unbinding, if the client sends the 3D image rendering request again, it is sufficient to newly allocate an image processing unit thereto.

By binding the image processing units with the client, the corresponding image processing units can be conveniently selected for direct processing after the client sends the image rendering request, and the image rendering requests of the same client can be handed to the same image processing unit, so that the processing efficiency is improved.

In one embodiment, the server allocates a corresponding image processing unit to the client, including:

the server judges whether an idle image processing unit exists at present;

If yes, the server selects an idle image processing unit to be distributed to the client;

Specifically, selecting an idle image processing unit to allocate to the client includes:

Randomly selecting one image processing unit from all idle image processing units; or selecting the image processing unit with the minimum number from all idle image processing units; alternatively, the image processing unit with the largest number is selected from all the idle image processing units.

if not, the server selects one image processing unit to be distributed to the client according to a preset rule.

In one embodiment, the server selects one image processing unit to distribute to the client according to a preset rule, and the method comprises the following steps:

The server finds out the image processing unit with the least bound client number and distributes the image processing unit to the client; alternatively, the first and second electrodes may be,

The server finds out the image processing unit with the lowest current processing load to be distributed to the client.

Specifically, the image processing unit with the lowest processing load may refer to: and the image processing unit which receives the least number of 3D image rendering requests in unit time. The unit time refers to a period of time from a time point before the current time to the current time, and the time length can be set and adjusted according to needs.

If a plurality of image processing units with the minimum number of bound clients and a plurality of image processing units with the lowest current processing load are bound, one image processing unit with the minimum/maximum selection number can be randomly selected and distributed to the clients.

it should be noted that the execution order of step 102 and step 103 may be exchanged, specifically, when the request is an image rendering request, the server may first perform the step of determining the image processing unit of the corresponding client in the data processing apparatus, and then perform the step of obtaining the instruction corresponding to image rendering.

104. The server transmits the instruction to the data processing device so that the image processing unit completes image rendering according to the instruction and then transmits the image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server.

specifically, the user mode filter driver sends the intercepted drawing instruction to the data processing device, the kernel mode filter driver sends the intercepted graphics card configuration instruction and the intercepted special effect instruction to the data processing device, and the data processing device operating system uses the graphics card of the data processing device to complete image rendering according to the instruction. Specifically, the data processing apparatus operating system completes the image rendering process through the graphics card according to the instruction related to the image rendering, and refers to the technical data of the related graphics card rendering image, which is not described in detail herein.

In one embodiment, the method further comprises:

105. And when the request is a request except for the image rendering request, the server transmits a processing result to the data processing device after processing according to the request so that the data processing device forwards the processing result to the client.

for a general cloud desktop client request, a server can directly process the request and directly process the request in the server without being processed by a data processing device, and only a processing result, such as computer desktop image data, needs to be forwarded to the client through the data processing device.

according to the data transmission method provided by the embodiment of the disclosure, a data processing device receives a request of a client and forwards the request to a server; when the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out; the server determines an image processing unit corresponding to the client in the data processing device; the server transmits the instruction to the data processing device so that the image processing unit completes image rendering according to the instruction and then transmits the image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server. Therefore, the workload of the server is greatly reduced, and even a large number of 3D image rendering tasks of the client can be supported without a graphics processor, and the server does not need to be upgraded.

an embodiment of the present disclosure provides another data transmission method, as shown in fig. 5, the method includes the following steps:

201. the data processing device receives a request of a client.

202. The data processing apparatus transmits a request of the client to the server.

203. when the request is an image rendering request, the data processing device receives an instruction corresponding to image rendering sent by the server; wherein the instructions include: the display card comprises a configuration instruction, a special effect instruction and a drawing instruction.

204. And the data processing device finishes image rendering through an image processing module of the data processing device according to the instruction.

In the embodiment of the present disclosure, the data Processing apparatus has an image rendering capability, that is, is configured with a Graphics Processing Unit (GPU), and in practical applications, a plurality of clients are connected to a server through the data Processing apparatus, in this case, 3D image rendering requirements from the clients are very large, and the data Processing apparatus needs to configure a plurality of display cards as image Processing modules of the data Processing apparatus according to a task amount of the rendering requirements.

205. And the data processing device sends the image data obtained after the image rendering is finished to the client.

in one embodiment, step 204 specifically includes:

The image processing unit corresponding to the client in the image processing module finishes image rendering according to the instruction; the image processing unit is an image rendering request which is distributed by the server in the image processing module for the client and is used for correspondingly processing the client. The image processing resources of the data processing apparatus can be efficiently utilized by dividing the image processing units.

A user mode display driving program in an operating system of the data processing device converts and processes the drawing instruction into an instruction which can be recognized by the display card and is executed by the display card corresponding to the image processing unit; a display micro port driver in an operating system of the data processing device acquires a display card configuration instruction and a special effect instruction and is executed by a display card corresponding to the image processing unit.

Specifically, after the data processing device receives an instruction sent by the server, a microsoft windows operating system installed in the data processing device supports a WDDM driver model architecture, a user mode display driver in the data processing device operating system converts the received drawing instruction to obtain an instruction that can be recognized by the video card and transmits the instruction to a video memory of the video card in a Direct Memory Access (DMA) manner, and the video card performs instruction execution to complete image rendering by combining the received video card configuration instruction and the received special-effect instruction.

illustratively, the image rendering process performed by the data processing apparatus is implemented based on the WDDM model framework shown in fig. 2, and the data processing apparatus receives, via the network, instructions sent by the server (including configuration instructions, special effects instructions, and drawing instructions for the graphics card), where the drawing instructions are composed of one or more functions containing specific parameters, and these functions can be reproduced in the operating system of the data processing apparatus, so as to complete the final 3D image rendering operation. For example, the received drawing command is: func1 (parameter 1, parameter 2, parameter 3); func2 (parameter 1, parameter 2); func3 (parameter 1, parameter 2, parameter 3, parameter 4); where func1, func2, and func3 are three exemplary function names given to distinguish different functions, according to the above example, the data processing apparatus may perform three functions in the drawing instruction to perform 3D image rendering (i.e., function reproduction process).

in one embodiment, the method may further comprise:

206. when the request is a request except for the image rendering request, the data processing device receives a processing result obtained after the server carries out processing according to the request;

207. and the data processing device sends the processing result to the client.

for a general cloud desktop client request, a server can directly process the request and directly process the request in the server, and the server only needs to forward a processing result, such as computer desktop image data, to a client through a data processing device.

according to the data transmission method provided by the embodiment of the disclosure, a data processing device receives a request of a client and forwards the request to a server; when the request is an image rendering request, the data processing device receives an instruction corresponding to image rendering sent by a server, wherein the instruction comprises: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out; the data processing device finishes image rendering through an image processing module of the data processing device according to the instruction; and the data processing device sends the image data obtained after the image rendering is finished to the client. Therefore, the workload of the server is greatly reduced, and even a large number of 3D image rendering tasks of the client can be supported without a graphics processor, and the server does not need to be upgraded.

Based on the data transmission method described in the corresponding embodiment, the following is an embodiment of the apparatus of the present disclosure, which can be used to execute the embodiment of the method of the present disclosure.

An embodiment of the present disclosure provides a server, as shown in fig. 6, where the server 60 includes:

A receiving module 601, configured to receive a request of a client forwarded by a data processing apparatus;

An obtaining module 602, configured to obtain, when the request is an image rendering request, an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

A determining module 603, configured to determine an image processing unit of a corresponding client in the data processing apparatus;

A transmission module 604, configured to transmit the instruction to the data processing apparatus, so that the image processing unit completes image rendering according to the instruction and transmits the image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server.

The image rendering task of the client is completed by the data processing device, which is a device disposed between the server and the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

in one embodiment, as shown in fig. 6, the server 60 may further include:

A dividing module 600 for dividing an image processing module of the data processing apparatus into N image processing units; wherein each image processing unit can independently complete the image rendering task.

the server divides an image processing module of the data processing device into a plurality of image processing units which can independently complete image tasks and distributes the image processing units to each client, so that the image rendering can be completed by utilizing the image processing units more efficiently, and the phenomenon that a plurality of clients compete for resources and wait for longer processing time is avoided.

in an embodiment, the determining module 603 is specifically configured to:

acquiring an identifier of a client from an image rendering request;

judging whether an image processing unit corresponding to the client exists according to the identification of the client;

And if the image processing unit does not exist, distributing the corresponding image processing unit for the client, and binding the client and the corresponding image processing unit.

by binding the image processing units with the client, the corresponding image processing units can be conveniently selected for direct processing after the client sends the image rendering request, and the image rendering requests of the same client can be handed to the same image processing unit, so that the processing efficiency is improved.

in an embodiment, the determining module 603 is specifically configured to:

Judging whether an idle image processing unit exists at present;

If yes, selecting an idle image processing unit to be distributed to the client;

If the image processing unit does not exist, selecting one image processing unit to be distributed to the client according to a preset rule, wherein the method comprises the following steps:

Finding out the image processing unit with the least bound client number to be distributed to the clients; alternatively, the first and second electrodes may be,

And finding out the image processing unit with the lowest current processing load to be distributed to the client.

in one embodiment, as shown in fig. 6, the server 60 may further include:

And the processing module 605 is configured to, when the request is a request other than the image rendering request, call the transmission module 603 to transmit the processing result to the data processing apparatus after processing according to the request, so that the data processing apparatus forwards the processing result to the client.

For a general cloud desktop client request, a server can directly process the request and directly process the request in the server without being processed by a data processing device, and only a processing result, such as computer desktop image data, needs to be forwarded to the client through the data processing device.

in an embodiment, the obtaining module 602 is specifically configured to:

Capturing a drawing instruction sent by a D3D RT program in an operating system of a server to a user mode display driver;

And intercepting and capturing a display card configuration instruction and a special effect instruction which are sent to a display micro-port driver by a graphics kernel subsystem in an operating system of the server.

In the server provided by the embodiment of the disclosure, the data processing device receives the request of the client and forwards the request to the server; when the request is an image rendering request, the server acquires an instruction corresponding to image rendering; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out; the server determines an image processing unit corresponding to the client in the data processing device; the server transmits the instruction to the data processing device so that the image processing unit completes image rendering according to the instruction and then transmits the image data to the client; wherein, the image rendering is not required to be carried out according to the instruction in the server. Therefore, the workload of the server is greatly reduced, and even a large number of 3D image rendering tasks of the client can be supported without a graphics processor, and the server does not need to be upgraded.

the embodiment of the present disclosure provides a data processing apparatus, as shown in fig. 7, the data processing apparatus 70 includes:

a communication module 701, a processor 702, an image processing module 703, and a memory 704; the data processing device 70 communicates with the client and the server through a communication module 701 respectively; the communication module 701 includes: a request receiving module 7011, a request forwarding module 7012, an instruction receiving module 7013, an image acquisition module 7014, and an image transmission module 7015;

A request receiving module 7011, configured to receive a request of a client;

a request forwarding module 7012, configured to send a request of a client to a server;

An instruction receiving module 7013, configured to, when the request is an image rendering request, receive, by the data processing apparatus, an instruction corresponding to image rendering sent by the server; wherein the instructions include: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out;

the memory 704 is used for caching an instruction corresponding to image rendering received from the server and caching an image obtained after the image rendering is completed;

a processor 702, configured to instruct the image processing module 703 to complete image rendering according to the instruction;

The image acquisition module 7014 is configured to acquire an image obtained after image rendering is completed;

And the image transmission module 7015 is configured to send the image obtained after the image rendering is completed to the client.

the image rendering task of the client is completed by the data processing device, which is a device disposed between the server and the client, so that the server can support a large number of image rendering requests from the client without troublesome upgrades.

in one embodiment, the processor 702 is specifically configured to:

Instructing an image processing unit of a corresponding client in the image processing module to finish image rendering according to the instruction; the image processing unit is an image rendering request which is distributed by the server in the image processing module for the client and is used for correspondingly processing the client. A user mode display driving program in an operating system of the data processing device converts and processes the drawing instruction into an instruction which can be recognized by the display card and is executed by the display card corresponding to the image processing unit; a display micro port driver in an operating system of the data processing device acquires a display card configuration instruction and a special effect instruction and is executed by a display card corresponding to the image processing unit.

Illustratively, the data processing device is configured with an operating system supporting WDDM, and after receiving an instruction corresponding to image rendering sent by the server, completes image rendering through a graphics card configured by itself according to a flow of executing the instruction in the operating system.

the server divides an image processing module of the data processing device into a plurality of image processing units which can independently complete image tasks and distributes the image processing units to each client, so that the image rendering can be completed by utilizing the image processing units more efficiently, and the phenomenon that a plurality of clients compete for resources and wait for longer processing time is avoided.

In one embodiment, the image acquisition module 7014 is further configured to: when the request is a request except for an image rendering request, acquiring an image which is sent by a server and processed according to the request;

Accordingly, the image transmission module 7015 is further configured to: and sending the image which is sent by the server and processed according to the request to the client.

For a general cloud desktop client request, a server can directly process the request and directly process the request in the server, and the server only needs to forward a processing result, such as computer desktop image data, to a client through a data processing device.

Based on the data processing apparatus shown in fig. 7, fig. 8 is a schematic diagram illustrating a detailed implementation effect provided by the embodiment of the disclosure.

according to the data processing device provided by the embodiment of the disclosure, the data processing device receives the request of the client and forwards the request to the server; when the request is an image rendering request, the data processing device receives an instruction corresponding to image rendering sent by a server, wherein the instruction comprises: the method comprises the following steps that (1) a display card configuration instruction, a special effect instruction and a drawing instruction are carried out; the data processing device finishes image rendering through an image processing module of the data processing device according to the instruction; and the data processing device sends the image data obtained after the image rendering is finished to the client. Therefore, the workload of the server is greatly reduced, and even a large number of 3D image rendering tasks of the client can be supported without a graphics processor, and the server does not need to be upgraded.

Based on the data transmission method described in the embodiment corresponding to fig. 3/5, an embodiment of the present disclosure further provides a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a Read Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The storage medium stores computer instructions for executing the data transmission method described in the embodiment corresponding to fig. 3/5, and details are not repeated here.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.