阅读说明：本技术 一种基于虚拟化技术的Linux视频编码方法 (Linux video coding method based on virtualization technology ) 是由 刘波 史杨 张琳 姜兆艺 于 2021-08-30 设计创作，主要内容包括：本发明提供一种基于虚拟化技术的Linux视频编码方法,通过Linux虚拟机中的capturemode(桌面抓取模块)、agentmode(代理模块)、servermode(服务模块)和clientmode(客户端模块)实现,该基于虚拟化技术的Linux视频编码方法设计合理,通过给linux虚拟机配置gpu显卡,终端连接linux虚拟机,利用gpu显卡抓取桌面,终端显示和运行,能够充分利用gpu的计算能力和硬编功能,运行大型的3D游戏和3D软件。(The invention provides a Linux video coding method based on a virtualization technology, which is realized by capturemode (a desktop grabbing module), an agentmaode (an agent module), a servermode (a service module) and a clientmode (a client module) in a Linux virtual machine.)

1. A Linux video coding method based on virtualization technology is characterized by being realized through a capture mode (desktop capture module), an agent mode (proxy module), a server mode (service module) and a client mode (client module) in a Linux virtual machine, and the specific coding method is as follows:

the method comprises the following steps: the Linux virtual machine is started to automatically start the agent process in a service form to serve as a network service end, and the agent process processes a monitoring (list) state after the network service initialization is finished and waits for the network connection (capture mode) of the client; when a user inputs a user name and a password at a remote client (namely a client mode) to connect with a linux virtual machine, if the user passes verification, the agent process generates a capture subprocess; the capture subprocess is used as a client side and an agent process to perform local socket inter-process communication in the linux virtual machine, desktop data are transmitted from the capture subprocess to the agent process, and data stream coding parameters are confirmed before real desktop data are transmitted;

step two: the message interaction is carried out among all modules, and the specific flow is as follows:

VGPU _ SEND _ ADAPTER _ MONITOR: the capture mode sends the relevant information of the vgpu display card;

VGPU _ SET _ PARAM _ START _ REQUEST: the agent mode sends coding parameter information of the data stream;

VGPU _ GET _ DATA _ REQUEST: after the agent mode sends the information, the capture subprocess starts to transmit the coded desktop data to the agent service process;

VGPU _ STOP _ REQUEST: the capture subprocess suspends the sending of data;

VGPU _ destination _ REQUEST: ending the capture subprocess and exiting;

step three: the operation principle of each module is as follows:

capture mode: after the capture process is generated, the capture process is used as a client to be connected with a socket of an agent server process, after the connection, a main process can generate three sub-threads to work (including a main thread therein), main thread, capture thread and send thread, and the specific division is as follows: the main thread mainly receives and processes information sent by an agent server, and nvidia gpu hard coding or cpu soft coding is carried out on ARGB desktop data captured by the capture thread by using an ffmpeg library, so that h264 and h265 coding are supported at present; the capture thread is mainly used for capturing desktop data of a linux virtual machine, and is realized by calling xcb library functions xcb _ shm _ get _ image and xcb _ shm _ get _ image _ reply, and acquiring an original data shmget by using a linux shared memory mode; the function of the send thread is to realize that the desktop data of the main thread after being coded is sent to the agent process through tcp connection;

agent mode: the method comprises the steps that an agent process can be automatically started in a service mode when a Linux virtual machine is started, a Linux module system is used, when a user is connected with the Linux virtual machine at a remote client, the agent can obtain corresponding DATA stream coding parameters from the remote client, generate a capture subprocess, transmit the capture subprocess to the capture subprocess in a message VGPU _ SET _ PARAM _ START _ REQUEST mode, then send a message VGPU _ GET _ DATA _ REQUEST to the capture subprocess, continuously obtain desktop DATA of the encoded Linux virtual machine from the capture subprocess, and forward the coded desktop DATA of the Linux virtual machine to a server through a virtio character device (/ dev/virtio-ports/com.dhat.0) created when the Linux virtual machine is started through qemu after the DATA is obtained, and when the coded DATA is large, the agent process needs to packetize the DATA, the number of the packetize packets is included, the size of each packet, and the information of the packets are required in the server model;

③ server mode: the linux virtual machine is created by qemu in a process mode, generally runs in a centros operating system, reads data and determines whether to package the data or not by calling a spice server library when coded data of an agent mode is transmitted from a character device, and transmits the data to a client mode through tcp connection after all complete packet data are read;

client mode: when a user is connected to a linux virtual machine through a remote client, the coded data transmitted by the spice server can be continuously read, and two processing threads, decode thread and render thread, can be generated; performing ffmpeg decoding on the decode thread, specifically including a CPU soft solution and a GPU hard solution, converting the decoded data into rgb format data, rendering and drawing the render thread by using a caliro, displaying the rendered thread by using a display, and circulating the data between the two threads through a queue;

step four: through mutual assistance of a capture mode (desktop capture module), an agent mode (agent module), a server mode (service module) and a client mode (client module), when a user connects a linux virtual machine, the virtual machine can fully utilize the computing power and the hard-editing function of a GPU (graphics processing unit) to run a large 3D game and 3D software.

2. The Linux video coding method based on virtualization technology of claim 1, wherein: before the first step, Linux vgpu environment configuration is required, which comprises installation and management of a server gpu display card device driver, installation of a Linux virtual machine virtual display card driver and license configuration.

3. The Linux video coding method based on virtualization technology of claim 1, wherein: the data stream coding parameter confirmation in the first step includes but is not limited to a coding mode (h264, h265), a gpu hard coding or a cpu soft coding, and a width, a height and a frame rate of the data stream.

4. The Linux video coding method based on virtualization technology of claim 1, wherein: in the third step, when the coded data is larger than 16kb, the agent process needs to packetize the data.

5. The Linux video coding method based on virtualization technology of claim 1, wherein: in the fifth step, when the user disconnects the linux virtual machine, a message VGPU _ DESTROY _ REQUEST is sent to the capture subprocess to finish the operation, and when the user connects again, a new capture subprocess is generated.

Technical Field

The invention belongs to the technical field of video coding, and particularly relates to a Linux video coding method based on a virtualization technology.

Background

On a physical computer, one or more virtual computers simulated by software are called virtual machines. The virtual machine uses hardware resources of a host machine, has most functions of a real computer, is an operating system widely applied to a server, has no gpu display module, cannot fully utilize the hardware functions of gpu, cannot run a large-scale 3D game, and is provided with modeling software, and the powerful computing power and the hard-coding function of the gpu cannot be fully utilized.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a Linux video coding method based on a virtualization technology, which is reasonable in design, and can be used for fully utilizing the computing capacity and the hard-coding function of gpu and running a large-scale 3D game and 3D software by configuring a gpu display card for a Linux virtual machine, connecting a terminal with the Linux virtual machine, grabbing a desktop by using the gpu display card and displaying and running the terminal.

In order to achieve the purpose, the invention is realized by the following technical scheme: a Linux video coding method based on virtualization technology is realized by a capture mode (desktop capture module), an agent mode (proxy module), a server mode (service module) and a client mode (client module) in a Linux virtual machine, and the specific coding method is as follows:

the method comprises the following steps: the Linux virtual machine is started to automatically start the agent process in a service form to serve as a network service end, and the agent process processes a monitoring (list) state after the network service initialization is finished and waits for the network connection (capture mode) of the client; when a user inputs a user name and a password at a remote client (namely a client mode) to connect with a linux virtual machine, if the user passes verification, the agent process generates a capture subprocess; the capture subprocess is used as a client side and an agent process to perform local socket inter-process communication in the linux virtual machine, desktop data are transmitted from the capture subprocess to the agent process, and data stream coding parameters are confirmed before real desktop data are transmitted;

step two: the message interaction is carried out among all modules, and the specific flow is as follows:

VGPU _ SEND _ ADAPTER _ MONITOR: the capture mode sends the relevant information of the vgpu display card;

VGPU _ SET _ PARAM _ START _ REQUEST: the agent mode sends coding parameter information of the data stream;

VGPU _ GET _ DATA _ REQUEST: after the agent mode sends the information, the capture subprocess starts to transmit the coded desktop data to the agent service process;

VGPU _ STOP _ REQUEST: the capture subprocess suspends the sending of data;

VGPU _ destination _ REQUEST: ending the capture subprocess and exiting;

step three: the operation principle of each module is as follows:

capture mode: after the capture process is generated, the capture process is used as a client to be connected with a socket of an agent server process, after the connection, a main process can generate three sub-threads to work (including a main thread therein), main thread, capture thread and send thread, and the specific division is as follows: the main thread mainly receives and processes information sent by an agent server, and nvidia gpu hard coding or cpu soft coding is carried out on ARGB desktop data captured by the capture thread by using an ffmpeg library, so that h264 and h265 coding are supported at present; the capture thread is mainly used for capturing desktop data of a linux virtual machine, and is realized by calling xcb library functions xcb _ shm _ get _ image and xcb _ shm _ get _ image _ reply, and acquiring an original data shmget by using a linux shared memory mode; the function of the send thread is to realize that the desktop data of the main thread after being coded is sent to the agent process through tcp connection;

agent mode: the method comprises the steps that an agent process can be automatically started in a service mode when a Linux virtual machine is started, a Linux module system is used, when a user is connected with the Linux virtual machine at a remote client, the agent can obtain corresponding DATA stream coding parameters from the remote client, generate a capture subprocess, transmit the capture subprocess to the capture subprocess in a message VGPU _ SET _ PARAM _ START _ REQUEST mode, then send a message VGPU _ GET _ DATA _ REQUEST to the capture subprocess, continuously obtain desktop DATA of the encoded Linux virtual machine from the capture subprocess, and forward the coded desktop DATA of the Linux virtual machine to a server through a virtio character device (/ dev/virtio-ports/com.dhat.0) created when the Linux virtual machine is started through qemu after the DATA is obtained, and when the coded DATA is large, the agent process needs to packetize the DATA, the number of the packetize packets is included, the size of each packet, and the information of the packets are required in the server model;

③ server mode: the linux virtual machine is created by qemu in a process mode, generally runs in a centros operating system, reads data and determines whether to package the data or not by calling a spice server library when coded data of an agent mode is transmitted from a character device, and transmits the data to a client mode through tcp connection after all complete packet data are read;

client mode: when a user is connected to a linux virtual machine through a remote client, the coded data transmitted by the spice server can be continuously read, and two processing threads, decode thread and render thread, can be generated; performing ffmpeg decoding on the decode thread, specifically including a CPU soft solution and a GPU hard solution, converting the decoded data into rgb format data, rendering and drawing the render thread by using a caliro, displaying the rendered thread by using a display, and circulating the data between the two threads through a queue;

step four: through mutual assistance of a capture mode (desktop capture module), an agent mode (agent module), a server mode (service module) and a client mode (client module), when a user connects a linux virtual machine, the virtual machine can fully utilize the computing power and the hard-editing function of a GPU (graphics processing unit) to run a large 3D game and 3D software.

As a preferred embodiment of the present invention, the first step needs to perform Linux vgpu environment configuration, which includes installation and management of server gpu graphics card device driver, installation of Linux virtual machine virtual graphics card driver, and configuration of license.

As a preferred embodiment of the present invention, the data stream coding parameter confirmation in the first step includes, but is not limited to, a coding mode (h264, h265), a gpu hard coding or a cpu soft coding, and a width, a height, and a frame rate of the data stream.

In a preferred embodiment of the present invention, in step three, the agent process needs to packetize the data when the encoded data is larger than 16 kb.

In the fifth step, when the user disconnects the linux virtual machine, a message VGPU _ hierarchy _ REQUEST is sent to the capture subprocess to end the operation, and when the user connects again, a new capture subprocess is generated.

The invention has the beneficial effects that: according to the Linux video coding method based on the virtualization technology, the Linux virtual machine is provided with the gpu display card, the terminal is connected with the Linux virtual machine, the desktop is grabbed by using the gpu display card, and the terminal displays and runs, so that the computing power and the hard-coding function of the gpu can be fully utilized, and a large-scale 3D game and 3D software can be run.

Drawings

FIG. 1 is a block communication diagram of a Linux video coding method based on virtualization technology;

FIG. 2 is a real-time data flow diagram of a Linux video coding method based on virtualization technology;

fig. 3 is a message interaction flow chart of a Linux video coding method based on a virtualization technology.

Detailed Description

In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with the specific embodiments.

Referring to fig. 1 to 3, the present invention provides a technical solution: a Linux video coding method based on virtualization technology is realized by a capture mode (desktop capture module), an agent mode (proxy module), a server mode (service module) and a client mode (client module) in a Linux virtual machine, and the specific coding method is as follows:

the method comprises the following steps: the Linux virtual machine is started to automatically start the agent process in a service form to serve as a network service end, and the agent process processes a monitoring (list) state after the network service initialization is finished and waits for the network connection (capture mode) of the client; when a user inputs a user name and a password at a remote client (namely a client mode) to connect with a linux virtual machine, if the user passes verification, the agent process generates a capture subprocess; the capture subprocess is used as a client side and an agent process to perform local socket inter-process communication in the linux virtual machine, desktop data are transmitted from the capture subprocess to the agent process, and data stream coding parameters are confirmed before real desktop data are transmitted;

step two: the message interaction is carried out among all modules, and the specific flow is as follows:

VGPU _ SEND _ ADAPTER _ MONITOR: the capture mode sends the relevant information of the vgpu display card;

VGPU _ SET _ PARAM _ START _ REQUEST: the agent mode sends coding parameter information of the data stream;

VGPU _ GET _ DATA _ REQUEST: after the agent mode sends the information, the capture subprocess starts to transmit the coded desktop data to the agent service process;

VGPU _ STOP _ REQUEST: the capture subprocess suspends the sending of data;

VGPU _ destination _ REQUEST: ending the capture subprocess and exiting;

step three: the operation principle of each module is as follows:

capture mode: after the capture process is generated, the capture process is used as a client to be connected with a socket of an agent server process, after the connection, a main process can generate three sub-threads to work (including a main thread therein), main thread, capture thread and send thread, and the specific division is as follows: the main thread mainly receives and processes information sent by an agent server, and nvidia gpu hard coding or cpu soft coding is carried out on ARGB desktop data captured by the capture thread by using an ffmpeg library, so that h264 and h265 coding are supported at present; the capture thread is mainly used for capturing desktop data of a linux virtual machine, and is realized by calling xcb library functions xcb _ shm _ get _ image and xcb _ shm _ get _ image _ reply, and acquiring an original data shmget by using a linux shared memory mode; the function of the send thread is to realize that the desktop data of the main thread after being coded is sent to the agent process through tcp connection;

agent mode: the method comprises the steps that an agent process can be automatically started in a service mode when a Linux virtual machine is started, a Linux module system is used, when a user is connected with the Linux virtual machine at a remote client, the agent can obtain corresponding DATA stream coding parameters from the remote client, generate a capture subprocess, transmit the capture subprocess to the capture subprocess in a message VGPU _ SET _ PARAM _ START _ REQUEST mode, then send a message VGPU _ GET _ DATA _ REQUEST to the capture subprocess, continuously obtain desktop DATA of the encoded Linux virtual machine from the capture subprocess, and forward the coded desktop DATA of the Linux virtual machine to a server through a virtio character device (/ dev/virtio-ports/com.dhat.0) created when the Linux virtual machine is started through qemu after the DATA is obtained, and when the coded DATA is large, the agent process needs to packetize the DATA, the number of the packetize packets is included, the size of each packet, and the information of the packets are required in the server model;

③ server mode: the linux virtual machine is created by qemu in a process mode, generally runs in a centros operating system, reads data and determines whether to package the data or not by calling a spice server library when coded data of an agent mode is transmitted from a character device, and transmits the data to a client mode through tcp connection after all complete packet data are read;

client mode: when a user is connected to a linux virtual machine through a remote client, the coded data transmitted by the spice server can be continuously read, and two processing threads, decode thread and render thread, can be generated; performing ffmpeg decoding on the decode thread, specifically including a CPU soft solution and a GPU hard solution, converting the decoded data into rgb format data, rendering and drawing the render thread by using a caliro, displaying the rendered thread by using a display, and circulating the data between the two threads through a queue;

step four: through mutual assistance of a capture mode (desktop capture module), an agent mode (agent module), a server mode (service module) and a client mode (client module), when a user connects a linux virtual machine, the virtual machine can fully utilize the computing power and the hard-editing function of a GPU (graphics processing unit) to run a large 3D game and 3D software.

As a preferred embodiment of the present invention, the first step needs to perform Linux vgpu environment configuration, which includes installation and management of server gpu graphics card device driver, installation of Linux virtual machine virtual graphics card driver, and configuration of license.

As a preferred embodiment of the present invention, the data stream coding parameter confirmation in the first step includes, but is not limited to, a coding mode (h264, h265), a gpu hard coding or a cpu soft coding, and a width, a height, and a frame rate of the data stream.

In a preferred embodiment of the present invention, in step three, the agent process needs to packetize the data when the encoded data is larger than 16 kb.

In the fifth step, when the user disconnects the linux virtual machine, a message VGPU _ hierarchy _ REQUEST is sent to the capture subprocess to end the operation, and when the user connects again, a new capture subprocess is generated.

As a preferred embodiment of the invention, the gpu display card is configured for the linux virtual machine, the terminal is connected with the linux virtual machine, the gpu display card is used for grabbing the desktop, and the terminal displays and runs, so that the computing power and the hard-coding function of the gpu can be fully utilized, and a large-scale 3D game and 3D software can be run.

While there have been shown and described what are at present considered the fundamental principles and essential features of the invention and its advantages, it will be apparent to those skilled in the art that the invention is not limited to the details of the foregoing exemplary embodiments, but is capable of other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.

Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.