阅读说明：本技术 一种远程智能联动投屏方法及系统 (Remote intelligent linkage screen projection method and system ) 是由 唐嵩 唐淼 赵建青 邓伟 吴呈祥 于 2020-06-18 设计创作，主要内容包括：本发明公开了一种远程智能联动投屏方法及系统,其方法包括：设置多屏联动模块的客户端总数N,并依次编号为1～N；投屏模块在接收到投屏请求后,向网络发送投屏请求指令,并接收所有客户端的投屏请求应答指令且分别计算权重；以权重最大的客户端为主客户端,其余为从客户端,且主从客户端设置不同的数据接收端口；投屏模块获取屏幕的镜像数据并发送到网络；主客户端从网络中接收镜像数据,并转发给从客户端；所有客户端均将接收到的镜像数据按宽度等分为N份数据并按联屏顺序依次编号；每个客户端均展示编号与自身分配的投屏分区序列编号对等的的1份屏幕数据,实现远程智能联动投屏。本发明能完成联动投屏应用,降低成本,提高设备利用率。(The invention discloses a remote intelligent linkage screen projection method and a system, wherein the method comprises the following steps: setting the total number N of the clients of the multi-screen linkage module, and sequentially numbering the clients as 1-N; after receiving the screen projection request, the screen projection module sends a screen projection request instruction to the network, receives screen projection request response instructions of all the clients and respectively calculates the weight; the client with the largest weight is taken as a master client, the rest are taken as slave clients, and the master client and the slave clients are provided with different data receiving ports; the screen projection module acquires mirror image data of a screen and sends the mirror image data to a network; the method comprises the following steps that a master client receives mirror image data from a network and forwards the mirror image data to a slave client; all the clients equally divide the received mirror image data into N parts of data according to the width and sequentially number the data according to the screen connection sequence; and each client displays 1 part of screen data with the number equal to the screen projection partition sequence number distributed by the client, so that remote intelligent linked screen projection is realized. The invention can complete the linkage screen projection application, reduce the cost and improve the utilization rate of equipment.)

1. A remote intelligent linkage screen projection method is characterized by comprising the following steps:

step 10, setting the total number N of the client sides of the multi-screen linkage module, and setting screen projection partition sequence numbers of 1-N for all the client sides according to the screen linkage sequence;

step 20, after receiving the screen projection request, the screen projection module sends a screen projection request instruction to the network, receives screen projection request response instructions of all the clients, and obtains information of each client from the screen projection request response instructions to calculate the weight of each client;

step 30, the client with the largest weight is taken as a master client, the other clients are taken as slave clients, and different data receiving ports are arranged on the master client and the slave clients;

step 40, the screen projection module acquires mirror image data of a screen and sends the mirror image data to a network;

step 50, the master client receives the mirror image data from the network through the data receiving port of the master client and sends the mirror image data to the network through the port which is the same as the data receiving port of the slave client; receiving mirror image data from a client through a data receiving port of the client;

step 60, equally dividing the received mirror image data into N parts of data according to the width by all the client sides, and sequentially numbering the N parts of data into 1-N according to the screen connection sequence; and each client displays 1 part of data with the serial number equal to that of the screen-casting partition sequence of the client, so that remote intelligent linked screen casting is realized.

2. The method of claim 1, wherein the screen-casting module sets overtime detection when receiving screen-casting request response instructions, and exits the screen-casting process if screen-casting request response instructions of all clients are not received within a preset time period; and if not, continuously receiving screen-casting request response instructions within the preset time length until screen-casting request instructions of all the clients are received within the preset time length.

3. The method according to claim 1, wherein the screen-casting request response instruction includes a screen-casting partition sequence number allocated to the client, a terminal ID, a memory usage rate, a CPU usage rate, and a disk usage rate, and the screen-casting module calculates the weight of the client according to the memory usage rate, the CPU usage rate, and the disk usage rate.

4. The method of claim 3, wherein the weights are calculated by: the weight is memory usage 0.6+ CPU usage 0.3+ disk usage 0.1.

5. The method of claim 1, further comprising: step 70, when receiving a screen projection stopping request, a screen projection stopping module sends a screen projection stopping instruction to a network and stops acquiring mirror image data of a screen; and when all the clients receive the screen projection stopping instruction, the data receiving ports stop mirror image data receiving.

6. The method of claim 1, wherein the screen-shot request instruction comprises a type of screen shot;

if the type of the screen projection is the same local area network multi-screen linkage screen projection, the screen projection module and all the client terminals receive and send instructions and data through the local area network in a UDP mode;

if the screen projection type is cross-network multi-screen linkage screen projection, the screen projection module and all the client sides communicate through TCP long connection established with the message transfer module.

7. The method of claim 1, wherein the screen-shot request instruction comprises a manner of screen shot;

if the screen projection mode is real-time screen projection, the screen projection module acquires mirror image data of a screen, namely a terminal screen image of the screen projection module is intercepted, and when the screen projection module intercepts each frame of terminal screen image, the acquired screen image is compressed into a WEBP (web-based service provider) picture format and then is sent to a network in real time; when the client receives the terminal screen image data, if the terminal screen image data of the current frame is not completely received and a new frame of terminal screen image data is received, the terminal screen image data of the current frame is abandoned for reception;

if the screen projection mode is delayed screen projection, the screen projection module acquires the mirror image data of the screen, namely, the terminal screen video of the screen projection module is recorded according to preset delay time, and when the screen projection module records a section of the terminal screen video, the screen video is compressed into a WEBP (web-based service provider) motion picture format and then is sent to the network in real time.

8. The method of claim 1, wherein the screen projection module uses different ports A1, B1 and C1 to perform command reception, command transmission except for mirror image data and transmission of mirror image data respectively;

the host client uses different ports A2, B2, C2 and D2 to respectively transmit commands except mirror image data, receive mirror image data and transmit mirror image data;

the slave client uses different ports a3, B3, and D3 to transmit commands other than mirror data, receive commands other than mirror data, and receive mirror data, respectively.

9. The method of claim 1, wherein the screen projection module sends the mirror data to the network with an interval of 100us between every two data packets.

10. A remote intelligent linkage screen projection system is characterized by comprising a screen projection module and a multi-screen linkage module, wherein the multi-screen linkage module comprises N clients; the screen projection module and all the clients comprise a processor and a memory, the memory is used for storing computer instructions of the screen projection module or the clients, and the processor is used for executing the computer instructions stored in the memory of the screen projection module or the clients and realizing the method of any one of claims 1 to 9.

Technical Field

The invention relates to screen projection of mobile equipment, in particular to a remote intelligent linkage screen projection method and system.

Background

There are three technical modes for transmitting screen signals of a mobile device to a remote display device (such as a television, a display screen, etc.) via wireless transmission: DLNA, AIRPLAY, Miracast. The screen projection technology of mobile equipment is becoming an important tool in scenes such as teaching and meetings and is widely applied.

Dnla (digital Living Network alliance) is a set of protocol for interconnection and intercommunication among PCs, mobile devices and consumer appliances, which is initiated by sony, intel, microsoft and the like. AirPlay is a wireless technology developed by apple, pictures, audio and video on iOS equipment such as iPhone and iPad can be wirelessly transmitted to AirPlay supporting equipment through WiFi, and the AirPlay has a mirror image function which is not possessed by DLNA. Miracast is a wireless display standard based on WiFi direct, which was established by WiFi alliance in 2012, and devices supporting this standard can share video pictures in a wireless manner, for example, a mobile phone can play movies or photos directly on a television or other devices through Miracast.

However, at present, three screen projection modes are one-to-one, screen projection is performed on a multi-terminal device client, and the terminal device client is used as a scene of integral linkage screen display or partial area screen projection display, and currently, no special and direct solution is available. Aiming at the current situation, in the market, a scheme of realizing a synchronous playing mode of playing each part of the whole picture by a plurality of terminals through pure software is not common, most of the schemes are realized through hardware or HDMI separators and the like, special hardware is required for supporting, the wiring distance is limited, and the flexibility is low. The deployment and maintenance costs are also increased, and large-scale application expansion is inconvenient.

Disclosure of Invention

The invention aims to solve the technical problem of providing a remote intelligent linkage screen projection method and system, which can complete linkage screen projection application, reduce cost and improve equipment utilization rate.

In order to achieve the technical purpose, the invention adopts the following technical scheme:

a remote intelligent linkage screen projection method comprises the following steps:

step 10, setting the total number N of the client sides of the multi-screen linkage module, and setting terminal IDs of 1-N for all the client sides according to the linkage sequence;

step 20, after receiving the screen projection request, the screen projection module sends a screen projection request instruction to the network, receives screen projection request response instructions of all the clients, and obtains information of each client from the screen projection request response instructions to calculate the weight of each client;

step 30, the client with the largest weight is taken as a master client, the other clients are taken as slave clients, and different data receiving ports are arranged on the master client and the slave clients;

step 40, the screen projection module acquires mirror image data of a screen and sends the mirror image data to a network;

step 50, the master client receives the mirror image data from the network through the data receiving port of the master client and sends the mirror image data to the network through the port which is the same as the data receiving port of the slave client; receiving mirror image data from a client through a data receiving port of the client;

step 60, equally dividing the received mirror image data into N parts of data according to the width by all the client sides, and sequentially numbering the N parts of data into 1-N according to the screen connection sequence; and each client displays 1 part of data with the serial number equal to the serial number of the screen projection partition allocated to the client, so that remote intelligent linked screen projection is realized.

In a more preferred technical scheme, when receiving screen-casting request response instructions, the screen-casting module sets overtime detection, and if screen-casting request response instructions of all clients are not received within a preset time, the screen-casting process is exited; and if not, continuously receiving screen-casting request response instructions within the preset time length until screen-casting request instructions of all the clients are received within the preset time length.

In a more preferred technical scheme, the screen-casting request response instruction includes a screen-casting partition sequence number, a terminal ID, a memory utilization rate, a CPU utilization rate, and a disk utilization rate of the client, and the screen-casting module calculates the weight of the client according to the memory utilization rate, the CPU utilization rate, and the disk utilization rate of the client.

In a more preferred technical solution, the weight calculation method comprises: the weight is memory usage 0.6+ CPU usage 0.3+ disk usage 0.1. According to the technical scheme, the main client is dynamically assigned according to the calculation weight of the real-time system resource idle state of the client, the main client is a communication and data processing hub center of the linkage screen projection module, the real-time requirement is high, the client with the most sufficient local system resources is assigned as the main client, and the real-time processing performance of the main client can be guaranteed.

In a more preferred embodiment, the method further comprises: step 70, when receiving a screen projection stopping request, a screen projection stopping module sends a screen projection stopping instruction to a network and stops acquiring mirror image data of a screen; and when all the clients receive the screen projection stopping instruction, the data receiving ports stop mirror image data receiving.

In a more preferred technical scheme, the screen-casting request instruction comprises a screen-casting type;

if the type of the screen projection is the same local area network multi-screen linkage screen projection, the screen projection module and all the client terminals receive and send instructions and data through the local area network in a UDP mode;

if the screen projection type is cross-network multi-screen linkage screen projection, the screen projection module and all the client sides communicate through TCP long connection established with the message transfer module.

In a more preferred technical scheme, the screen-casting request instruction comprises a screen-casting mode;

if the screen projection mode is real-time screen projection, the screen projection module acquires mirror image data of a screen, namely a terminal screen image of the screen projection module is intercepted, and when the screen projection module intercepts each frame of terminal screen image, the acquired screen image is compressed into a WEBP (web-based service provider) picture format and then is sent to a network in real time; when the client receives the terminal screen image data, if the terminal screen image data of the current frame is not completely received and a new frame of terminal screen image data is received, the terminal screen image data of the current frame is abandoned for reception;

if the screen projection mode is delayed screen projection, the screen projection module acquires the mirror image data of the screen, namely, the terminal screen video of the screen projection module is recorded according to preset delay time, and when the screen projection module records a section of the terminal screen video, the screen video is compressed into a WEBP (web-based service provider) motion picture format and then is sent to the network in real time.

In a more preferable technical scheme, the screen projection module uses different ports A1, B1 and C1 to respectively receive commands, send commands except mirror image data and send mirror image data;

the host client uses different ports A2, B2, C2 and D2 to respectively transmit commands except mirror image data, receive mirror image data and transmit mirror image data;

the slave client uses different ports a3, B3, and D3 to transmit commands other than mirror data, receive commands other than mirror data, and receive mirror data, respectively.

In a more preferred technical scheme, when the screen projection module sends mirror image data to the network, the interval between every two data packets is 100 us. The scheme can ensure the ordered transmission and the integrity of the transmitted data and provide more time for the client to receive and process the current data packet completely.

The invention also provides a remote intelligent linkage screen projection system, which comprises a screen projection module and a multi-screen linkage module, wherein the multi-screen linkage module comprises N client sides; the screen projection module and all the clients comprise a processor and a memory, the memory is used for storing computer instructions of the screen projection module or the clients, and the processor is used for executing the computer instructions stored in the memory of the screen projection module or the clients and is used for realizing any one of the methods.

Advantageous effects

The invention has the following beneficial effects: 1) the intelligent terminal with any screen projection application can be used as a client, so that the application is simple and wide; 2) the plurality of clients display the screen images of the screen projection module in a linkage manner, so that the visual effect of the screen images is improved; 3) meanwhile, remote screen projection under network conditions of a local area network, a cross-network segment, a public network and the like is supported; 4) the multi-screen linkage screen projection can be completed only by the main client side in the multi-screen linkage module downloading the screen image from the network, the main client side sends image data to the slave client side based on the UDP protocol and carries out linkage screen projection control, and all the client sides display the images of the corresponding parts, so that the flow is saved, and the bandwidth is saved.

Drawings

Fig. 1 is a sequence format of an instruction packet between a screen projection module and a client in an embodiment of the present invention;

FIG. 2 is a diagram illustrating a screen-casting request response command format according to an embodiment of the present invention;

FIG. 3 is a system configuration diagram according to a first embodiment of the present invention;

FIG. 4 is a block diagram of a multi-screen linkage module according to the first and second embodiments of the present invention;

FIG. 5 is a flow chart of a real-time screen projection method according to the first embodiment and the second embodiment of the present invention;

fig. 6 is a system configuration diagram according to a second embodiment of the present invention.

Detailed Description

The following describes embodiments of the present invention in detail, which are developed based on the technical solutions of the present invention, and give detailed implementation manners and specific operation procedures to further explain the technical solutions of the present invention.

The invention provides a remote intelligent linkage screen projection method, which is applied between a screen projection module and a multi-screen linkage module, wherein when the screen projection module receives a screen projection request input by a user through a screen projection application, after the screen projection application of N client sides of the multi-screen linkage module agrees, screen mirror image data of a terminal where the screen projection module is located is obtained, and the mirror image data is remotely displayed in a linkage manner on the N client sides as an integral screen through a network. The screen projection module and each client can be any intelligent terminal provided with screen projection application. For example, a user initiates a screen projection request through a smart phone or a tablet, and the plurality of self-service retail terminals are used as a plurality of clients to perform linkage screen projection on screen images of the mobile phone.

The communication between the screen projection module and the client mainly comprises the following instruction sequences (the format is shown in fig. 1):

instruction 1, namely a system mirror image screen instruction sequence (screen data does not exceed the longest message length of 1020 bytes);

instruction 2, i.e. the screen projection request instruction sequence (sent by the screen projection module to the client): the message includes a screen-casting type (1 byte) and a group ID (optional), and the values are 1: same LAN many screens linkage is thrown the screen, 2: cross-network segment multi-screen linkage screen projection;

instruction 3, screen projection request response instruction sequence (the client responds to the screen projection module): the message comprises a screen projection partition sequence number, a total number N of equipment clients, a memory utilization rate R, CPU utilization rate C, a disk utilization rate D and a terminal ID. Wherein R, C, D is in the range of [0,100 ]. As shown in fig. 2.

Instruction 4, the device client master control selects an instruction sequence (the screen projection module selects the host client): the message comprises screen projection partition sequence numbers and a terminal ID;

and 5, stopping the screen projection instruction sequence.