阅读说明：本技术 多媒体数据的传输方法、装置、自动驾驶车辆及云服务器 (Multimedia data transmission method and device, automatic driving vehicle and cloud server ) 是由 孙庆瑞 冯靖超 夏黎明 陈卓 于 2020-12-18 设计创作，主要内容包括：本申请公开了多媒体数据的传输方法、装置、自动驾驶车辆及云服务器,涉及计算机技术领域,具体涉及智能交通、自动驾驶等人工智能技术领域。具体实现方案：获取配置在车辆上的至少一组传感器采集的多媒体数据；分别对各组多媒体数据进行数据压缩处理；采用实时音视频服务RTC协议对压缩处理后的各组多媒体数据进行协议转换,得到与各组多媒体数据对应的目标多媒体数据；经由RTC协议将目标多媒体数据传输至云服务器中的实时音视频服务RTC模块,能够实现将多媒体数据实时地上传至云服务器中的实时音视频服务RTC模块,从而提升整体的多媒体数据传输的时效性,降低多媒体数据传输时延。(The application discloses a multimedia data transmission method and device, an automatic driving vehicle and a cloud server, relates to the technical field of computers, and particularly relates to the technical field of artificial intelligence such as intelligent transportation and automatic driving. The specific implementation scheme is as follows: acquiring multimedia data acquired by at least one group of sensors configured on a vehicle; respectively compressing each group of multimedia data; performing protocol conversion on each set of compressed multimedia data by adopting a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each set of multimedia data; the target multimedia data are transmitted to the real-time audio and video service RTC module in the cloud server through the RTC protocol, and the real-time uploading of the multimedia data to the real-time audio and video service RTC module in the cloud server can be realized, so that the timeliness of the whole multimedia data transmission is improved, and the multimedia data transmission delay is reduced.)

1. A method of transmitting multimedia data, the method comprising:

acquiring multimedia data acquired by at least one group of sensors configured on a vehicle;

respectively carrying out data compression processing on each group of multimedia data;

performing protocol conversion on each set of compressed multimedia data by adopting a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each set of multimedia data;

and transmitting the target multimedia data to a real-time audio and video service (RTC) module in a cloud server through the RTC protocol.

2. The method of claim 1, wherein the transmitting the target multimedia data via the RTC protocol to a real-time audio video service, RTC, module in a cloud server comprises:

determining a group identifier corresponding to each group of the sensors;

determining a data transmission link corresponding to the group identifier;

transmitting the target multimedia data of the corresponding set of sensors to the RTC module in real time via the RTC protocol and over the data transmission link.

3. The method of claim 2, prior to said transmitting the target multimedia data via the RTC protocol to a real-time audio video service, RTC, module in a cloud server, further comprising:

creating a plurality of data transmission links between the electronic device and the RTC module by adopting the RTC protocol;

and associating the data transmission link with the group identifier of the corresponding group of sensors, and associating and storing the group identifier and the link information of the data transmission link.

4. The method of claim 3, further comprising:

monitoring the connection state of the data transmission link in the process of transmitting the target multimedia data;

when the connection state indicates that the data transmission link is disconnected, reading prestored link information corresponding to the data transmission link;

and triggering the data transmission link to reconnect according to the link information.

5. A transmission method of multimedia data is used in a cloud server, and the cloud server comprises the following steps: a real-time audio and video service RTC module, the RTC module supporting a real-time audio and video service RTC protocol, the method comprising:

receiving at least one group of target multimedia data, wherein the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding group of sensors and performing protocol conversion on the compressed multimedia data by adopting the RTC protocol;

transmitting the at least one set of target multimedia data to a remote control device via the RTC protocol.

6. The method of claim 5, wherein the receiving at least one set of target multimedia data comprises:

receiving a plurality of groups of the target multimedia data;

performing confluence processing on a plurality of groups of target multimedia data to process the plurality of groups of target multimedia data into real-time video streams;

wherein the transmitting the at least one set of target multimedia data to a remote control device via the RTC protocol comprises:

transmitting the real-time video stream to the remote control device via the RTC protocol.

7. An apparatus for transmitting multimedia data, the apparatus comprising:

the acquisition module is used for acquiring multimedia data acquired by at least one group of sensors configured on the vehicle;

the compression module is used for respectively carrying out data compression processing on each group of multimedia data and carrying out protocol conversion on each group of compressed multimedia data by adopting a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each group of multimedia data;

and the first transmission module is used for transmitting the target multimedia data to a real-time audio and video service (RTC) module in a cloud server through the RTC protocol.

8. The apparatus of claim 7, wherein the first transmission module is specifically configured to:

determining a group identifier corresponding to each group of the sensors;

determining a data transmission link corresponding to the group identifier;

transmitting the target multimedia data of the corresponding set of sensors to the RTC module in real time via the RTC protocol and over the data transmission link.

9. The apparatus of claim 8, further comprising:

the establishing module is used for establishing a plurality of data transmission links between the electronic equipment and the RTC module by adopting the RTC protocol, associating the data transmission links with the group identifiers of the sensors of the corresponding group, and associating and storing the group identifiers and the link information of the data transmission links.

10. The apparatus of claim 9, further comprising:

the monitoring module is used for monitoring the connection state of the data transmission link in the process of transmitting the target multimedia data;

the establishing module is further configured to, when the connection status indicates that the data transmission link is disconnected, read pre-stored link information corresponding to the data transmission link, and trigger the data transmission link to reconnect according to the link information.

11. An apparatus for transmitting multimedia data, the apparatus comprising:

the receiving module is used for receiving at least one group of target multimedia data, and the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding group of sensors and performing protocol conversion on the compressed multimedia data by adopting a real-time audio/video service (RTC) protocol;

and the second transmission module is used for transmitting the at least one group of target multimedia data to the remote control equipment through the real-time audio and video service (RTC) protocol.

12. The apparatus of claim 11, wherein the receiving module is specifically configured to:

receiving a plurality of groups of the target multimedia data;

the device further comprises:

the processing module is used for performing confluence processing on the multiple groups of target multimedia data so as to process the multiple groups of target multimedia data into real-time video streams;

the second transmission module is specifically configured to transmit the real-time video stream to the remote control device via the RTC protocol.

13. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4 or to perform the method of any one of claims 5-6.

14. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-4 or perform the method of any one of claims 5-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-4, or implements the method of any one of claims 5-6.

16. An autonomous vehicle, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

17. A cloud server, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of claim 5 or 6.

Technical Field

The application relates to the technical field of computers, in particular to the technical field of artificial intelligence such as intelligent transportation and automatic driving, and particularly relates to a multimedia data transmission method and device, an automatic driving vehicle and a cloud server.

Background

Artificial intelligence is the subject of research that makes computers simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.

In the automatic driving remote control system, a cloud driver remotely controls a vehicle in a cockpit, so that the cloud driver needs to know traffic and road information around a vehicle body in time to make an accurate driving behavior.

Disclosure of Invention

Provided are a multimedia data transmission method, apparatus, electronic device, storage medium, computer program product, autonomous vehicle, and cloud server.

According to a first aspect, there is provided a method for transmitting multimedia data, comprising: acquiring multimedia data acquired by at least one group of sensors configured on a vehicle; respectively carrying out data compression processing on each group of multimedia data; performing protocol conversion on each set of compressed multimedia data by adopting a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each set of multimedia data; and transmitting the target multimedia data to a real-time audio and video service (RTC) module in a cloud server through the RTC protocol.

According to a second aspect, there is provided a multimedia data transmission method used in a cloud server, the cloud server including: a real-time audio and video service RTC module, the RTC module supporting a real-time audio and video service RTC protocol, the method comprising: receiving at least one group of target multimedia data, wherein the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding group of sensors and performing protocol conversion on the compressed multimedia data by adopting the RTC protocol; transmitting the at least one set of target multimedia data to a remote control device via the RTC protocol.

According to a third aspect, there is provided a transmission apparatus of multimedia data, comprising: the acquisition module is used for acquiring multimedia data acquired by at least one group of sensors configured on the vehicle; the compression module is used for respectively carrying out data compression processing on each group of multimedia data and carrying out protocol conversion on each group of compressed multimedia data by adopting a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each group of multimedia data; and the first transmission module is used for transmitting the target multimedia data to a real-time audio and video service (RTC) module in a cloud server through the RTC protocol.

According to a fourth aspect, there is provided a transmission apparatus of multimedia data, comprising: the receiving module is used for receiving at least one group of target multimedia data, and the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding group of sensors and performing protocol conversion on the compressed multimedia data by adopting a real-time audio/video service (RTC) protocol; and the second transmission module is used for transmitting the at least one group of target multimedia data to the remote control equipment through the real-time audio and video service (RTC) protocol.

According to a fifth aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for transmitting multimedia data according to the embodiment of the application.

According to a sixth aspect, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a transmission method of multimedia data disclosed in an embodiment of the present application is provided.

According to a seventh aspect, a computer program product is proposed, comprising a computer program which, when executed by a processor, implements the transmission method of multimedia data disclosed in embodiments of the present application.

According to an eighth aspect, there is provided an autonomous vehicle comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for transmitting multimedia data provided by the first aspect of the present application.

According to a ninth aspect, there is provided a cloud server comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the method for transmitting multimedia data provided in the second aspect of the present application.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

FIG. 1 is a schematic diagram according to a first embodiment of the present application;

FIG. 2 is a schematic diagram according to a second embodiment of the present application;

FIG. 3 is a schematic illustration according to a third embodiment of the present application;

FIG. 4 is a schematic diagram of an application scenario in an embodiment of the present application;

FIG. 5 is a schematic illustration according to a fourth embodiment of the present application;

FIG. 6 is a schematic illustration according to a fifth embodiment of the present application;

FIG. 7 is a schematic illustration according to a sixth embodiment of the present application;

FIG. 8 is a schematic illustration according to a seventh embodiment of the present application;

fig. 9 is a block diagram of an electronic device for implementing a transmission method of multimedia data according to an embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic diagram according to a first embodiment of the present application.

It should be noted that an execution main body of the transmission method of multimedia data of this embodiment is a transmission apparatus of multimedia data, the apparatus may be implemented by software and/or hardware, the apparatus may be configured in an electronic device, and the electronic device may include, but is not limited to, a terminal, a server, and the like.

The embodiment of the application relates to the technical field of artificial intelligence such as intelligent transportation and automatic driving.

Wherein, Artificial Intelligence (Artificial Intelligence), english is abbreviated as AI. The method is a new technical science for researching and developing theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

The intelligent transportation is to use the scientific technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operation research, artificial intelligence, etc.) in transportation, service control and vehicle manufacture.

Automatic driving relies on the cooperation of artificial intelligence, visual computation, radar, monitoring devices and global positioning systems, so that a computer can automatically and safely operate a motor vehicle without any human active operation.

The multimedia data transmission device in this embodiment may be disposed in a vehicle, such as in a vehicle-mounted electronic device, or may be disposed in other electronic devices, which is not limited to this.

The embodiment of the application can be exemplified by arranging the transmission device of multimedia data in a vehicle, and in an automatic driving scene, several groups of sensors are arranged for the vehicle, for example, different groups of sensors are arranged in the front, rear, left, right and the like directions of the vehicle, each group of sensors may include one or more sensors, and the sensors, for example, infrared sensors, image sensors, audio sensors and the like, collect environmental information around the vehicle in real time through the sensors, so as to assist automatic driving.

As shown in fig. 1, the method for transmitting multimedia data includes:

s101: multimedia data collected by at least one set of sensors disposed on a vehicle is acquired.

Based on the description of the application scenario, the sensors are used to collect environmental data around the vehicle in real time, where the environmental data includes visual data, auditory data, and the like, that is, may be referred to as multimedia data.

After the multimedia data around the vehicle are collected in real time through the sets of sensors, the sets of sensors can transmit the corresponding multimedia data to the vehicle-mounted electronic equipment, so that the multimedia data collected by at least one set of sensors configured on the vehicle are acquired by the vehicle-mounted electronic equipment.

S102: and respectively carrying out data compression processing on each group of multimedia data.

The above-mentioned multimedia data that is gathering at least a set of sensor that obtains configuration on the vehicle can carry out data compression processing to each set of multimedia data respectively, for example, can adopt H264 video compression coding standard to carry out data compression processing to each set of multimedia data respectively, can assist the transmission efficiency who promotes follow-up transmission multimedia data.

S103: and performing protocol conversion on each set of compressed multimedia data by adopting a Real-Time Communication (RTC) protocol to obtain target multimedia data corresponding to each set of multimedia data.

In the embodiment of the application, a data acquisition module can be configured in the vehicle-mounted electronic equipment, so that multimedia data can be acquired based on the data acquisition module, and compression processing logic, protocol conversion logic and link protection logic which are described below can be integrated in the data acquisition module, so that the time cost of transmitting the multimedia data between different processing modules is reduced.

In this embodiment of the present application, after the multimedia data is acquired, data compression processing and protocol conversion may be respectively triggered in real time on each set of multimedia data, and in addition, multiple parallel execution threads may be started, so that compression processing and protocol conversion are performed on multiple sets of multimedia data in a parallel processing manner, and the multimedia data after compression processing and protocol conversion is used as target multimedia data.

It can be understood that, in the embodiment of the present application, first, data compression processing is performed on multimedia data, and then, protocol conversion is performed on the compressed multimedia data, because the data amount of the multimedia data is reduced after the compression processing, the data amount required to be processed by the protocol conversion is reduced, and thus, the processing efficiency of the protocol conversion is ensured.

S104: and transmitting the target multimedia data to a real-time audio and video service RTC module in the cloud server through an RTC protocol.

After the multimedia data is subjected to compression processing and protocol conversion, and the multimedia data subjected to the compression processing and the protocol conversion is used as target multimedia data, the target multimedia data is transmitted to a real-time audio and video service (RTC) module in the cloud server through an RTC protocol.

For example, a communication connection between the electronic device and the cloud server may be established in advance, so that the target multimedia data is transmitted to a real-time audio/video service RTC module in the cloud server via an RTC protocol based on a data transmission link on the communication connection.

The real-time audio/video service RTC module can rely on powerful streaming media processing and transmission capability of the cloud server, so that target multimedia data are transmitted to the real-time audio/video service RTC module in the cloud server through an RTC protocol, and the RTC module is triggered to perform subsequent corresponding processing steps (see the subsequent embodiments specifically), thereby greatly improving the timeliness of the whole multimedia data transmission.

In this embodiment, multimedia data acquired by at least one group of sensors configured on a vehicle are acquired, data compression processing is performed on each group of multimedia data, protocol conversion is performed on each group of compressed multimedia data by using a real-time audio/video service (RTC) protocol, target multimedia data corresponding to each group of multimedia data is obtained, the target multimedia data is transmitted to a real-time audio/video service (RTC) module in a cloud server through the RTC protocol, and the real-time uploading of the multimedia data to the real-time audio/video service (RTC) module in the cloud server is realized, so that the timeliness of the whole multimedia data transmission is improved, and the transmission delay of the multimedia data is reduced.

Fig. 2 is a schematic diagram according to a second embodiment of the present application.

As shown in fig. 2, the method for transmitting multimedia data includes:

s201: multimedia data collected by at least one set of sensors disposed on a vehicle is acquired.

S202: and respectively carrying out data compression processing on each group of multimedia data.

S203: and performing protocol conversion on each set of compressed multimedia data by adopting a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each set of multimedia data.

The descriptions of S201 to S203 may refer to the above embodiments, and are not described herein again.

S204: and determining a group identifier corresponding to each group of sensors.

The group identifier can be used to uniquely identify the group to which the sensor belongs, and the group identifier is not limited to, for example, a group number, a group name, and the like.

S205: a data transmission link corresponding to the group identification is determined.

That is, when it is determined that target multimedia data corresponding to a group of sensors is to be transmitted, it is possible to dynamically determine a group identification of the group of sensors and determine a data transmission link corresponding to the group identification in real time.

In the embodiment of the present application, one data transmission link is dynamically configured for each group of sensors, so that the data transmission link is used to transmit the target multimedia data corresponding to the corresponding group of sensors, transmission congestion of the target multimedia data of different groups of sensors can be effectively avoided, and efficient and parallel transmission of each path of target multimedia data is realized.

The data transmission links may be pre-configured, for example, before the target multimedia data is transmitted to the real-time audio/video service RTC module in the cloud server via the RTC protocol, multiple data transmission links between the electronic device and the RTC module may be further created by using the RTC protocol; the data transmission link is associated with the group identifier of the sensor corresponding to the group, and the group identifier and the link information of the data transmission link are stored in an associated manner, so that the pre-established data transmission link can be directly called to perform data transmission when the data transmission link corresponding to the group identifier is determined, and the determination efficiency of the data transmission link is effectively improved.

S206: the target multimedia data of the corresponding set of sensors is transmitted in real time to the RTC module via the RTC protocol and over the data transmission link.

After the multimedia data is subjected to compression processing and protocol conversion, and the multimedia data subjected to the compression processing and the protocol conversion is used as target multimedia data, the target multimedia data is transmitted to a real-time audio and video service (RTC) module in the cloud server through an RTC protocol.

For example, a communication connection between the electronic device and the cloud server may be established in advance, so that the target multimedia data is transmitted to a real-time audio/video service RTC module in the cloud server via an RTC protocol based on a data transmission link on the communication connection.

The real-time audio/video service RTC module can rely on powerful streaming media processing and transmission capability of the cloud server, so that target multimedia data are transmitted to the real-time audio/video service RTC module in the cloud server through an RTC protocol, and the RTC module is triggered to perform subsequent corresponding processing steps (see the subsequent embodiments specifically), thereby greatly improving the timeliness of the whole multimedia data transmission.

S207: and monitoring the connection state of the data transmission link in the process of transmitting the target multimedia data.

The embodiment of the present application further provides a link protection mechanism, for example, the link protection processing logic may be integrated in the data acquisition module, so that in the process of transmitting the target multimedia data, the link protection processing logic may be invoked in real time to monitor the connection state of the data transmission links, for example, the connection state of any one data transmission link may be monitored, or the connection states of all data transmission links may also be monitored, which is not limited thereto.

S208: and when the connection state indicates that the data transmission link is disconnected, reading prestored link information corresponding to the data transmission link.

Among them, link information such as a port number.

S209: and triggering the data transmission link to reconnect according to the link information.

When the link protection processing logic is called in real time to monitor the connection state of the data transmission link, and when the connection state indicates that the data transmission link is disconnected, the link information corresponding to the data transmission link is read in advance.

For example, the link information of the data transmission link may be read from the group identifier and the link information of the data transmission link stored in advance in an associated manner, and then the data transmission link is triggered to reconnect according to the link information, and corresponding multimedia data may be retransmitted after the reconnection is successful.

The method comprises the steps of monitoring the connection state of a data transmission link in the process of transmitting target multimedia data, reading prestored link information corresponding to the data transmission link when the connection state indicates that the data transmission link is disconnected, and triggering the data transmission link to reconnect according to the link information, so that each data transmission link is monitored in real time in the process of transmitting the multimedia data by the data transmission link, and if a certain data transmission link is lost, the reconnection operation is triggered immediately, so that the stability of the data transmission link is effectively ensured.

In the specific implementation process of the embodiment of the present application, a fifth generation mobile communication technology (5th generation mobile networks, 5G) may also be used as a data transmission network, so as to avoid that the network bandwidth affects the transmission delay.

In this embodiment, multimedia data acquired by at least one group of sensors configured on a vehicle are acquired, data compression processing is performed on each group of multimedia data, protocol conversion is performed on each group of compressed multimedia data by using a real-time audio/video service (RTC) protocol, target multimedia data corresponding to each group of multimedia data is obtained, the target multimedia data is transmitted to a real-time audio/video service (RTC) module in a cloud server through the RTC protocol, and the real-time uploading of the multimedia data to the real-time audio/video service (RTC) module in the cloud server is realized, so that the timeliness of the whole multimedia data transmission is improved, and the transmission delay of the multimedia data is reduced. The method comprises the steps of monitoring the connection state of a data transmission link in the process of transmitting target multimedia data, reading prestored link information corresponding to the data transmission link when the connection state indicates that the data transmission link is disconnected, and triggering the data transmission link to reconnect according to the link information, so that each data transmission link is monitored in real time in the process of transmitting the multimedia data by the data transmission link, and if a certain data transmission link is lost, the reconnection operation is triggered immediately, so that the stability of the data transmission link is effectively ensured.

Fig. 3 is a schematic diagram according to a third embodiment of the present application.

As shown in fig. 3, the method for transmitting multimedia data includes:

the transmission device of multimedia data in this embodiment may be disposed in a cloud server, and the cloud server includes: the real-time audio and video service RTC module supports a real-time audio and video service RTC protocol.

S301: and receiving at least one group of target multimedia data, wherein the target multimedia data is obtained by performing data compression processing on the multimedia data acquired by the corresponding group of sensors and performing protocol conversion on the compressed multimedia data by adopting an RTC (real time clock) protocol.

The real-time audio and video service RTC module can rely on powerful streaming media processing and transmission capability of the cloud server, so that the RTC module RTC protocol can be controlled to receive at least one group of target multimedia data, and the timeliness of the whole multimedia data transmission can be greatly improved.

The method for obtaining the target multimedia data can be referred to the above embodiments, and is not described herein again.

S302: at least one set of target multimedia data is transmitted to the remote control device via the RTC protocol.

After the RTC module RTC protocol is controlled to receive the at least one set of target multimedia data, the at least one set of target multimedia data can be transmitted to the remote control device via the RTC protocol.

Referring to fig. 4, fig. 4 is a schematic view of an application scenario in the embodiment of the present application. Assuming that the plurality of groups of sensors are a left sensor group, a right sensor group, a front sensor group and a rear sensor group, the multimedia data acquired by each group of sensors can be compressed based on the H264 video compression coding standard, and then a data acquisition module in the vehicle-mounted device transmits target multimedia data to a real-time audio and video service (RTC) module in the cloud server through an RTC protocol, so that the real-time audio and video service (RTC) module transmits at least one group of target multimedia data to the remote control device through the RTC protocol.

When the at least one set of target multimedia data is transmitted to the remote control device via the RTC protocol, the at least one set of target multimedia data may be transmitted to the remote control device in parallel in a parallel processing manner.

In the embodiment of the present application, a plurality of sets of target multimedia data are received, and a streaming process is performed on the plurality of sets of target multimedia data to process the plurality of sets of target multimedia data into a real-time video stream, and at least one set of target multimedia data is transmitted to the remote control device via the RTC protocol, or the real-time video stream is transmitted to the remote control device via the RTC protocol.

That is to say, this application embodiment supports to carry out the confluence processing to multiunit multimedia data, for example can decompress and decode the processing to every group target multimedia data, thereby restore initial multimedia data, then, can also carry out the confluence with each group multimedia data, thereby merge into a complete video stream, so that this video stream can present the environment scene information around the vehicle completely, then, still transmit this real-time video stream to remote control equipment in real time, thereby can be convenient for remote control equipment carries out remote monitoring to the vehicle, promote automatic driving's intelligent driving effect, make automatic driving based on remote control equipment can simulate the effect of real driving scene better, thereby guarantee driving safety.

In this embodiment, at least one group of target multimedia data is received, the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding group of sensors, performing protocol conversion on the compressed multimedia data by using an RTC protocol, and transmitting the at least one group of target multimedia data to a remote control device via the RTC protocol, so that the at least one group of target multimedia data is pushed to the remote control device in real time, thereby improving the timeliness of overall multimedia data transmission and reducing the transmission delay of the multimedia data.

Fig. 5 is a schematic diagram according to a fourth embodiment of the present application.

As shown in fig. 5, the multimedia data transmission apparatus 50 includes:

an obtaining module 501, configured to obtain multimedia data collected by at least one set of sensors configured on a vehicle;

the compression module 502 is configured to perform data compression processing on each set of multimedia data, and perform protocol conversion on each set of compressed multimedia data by using a real-time audio/video service (RTC) protocol to obtain target multimedia data corresponding to each set of multimedia data;

the first transmission module 503 is configured to transmit the target multimedia data to a real-time audio/video service RTC module in the cloud server via an RTC protocol.

In some embodiments of the present application, the first transmission module 503 is specifically configured to:

determining a group identifier corresponding to each group of sensors;

determining a data transmission link corresponding to the group identifier;

the target multimedia data of the corresponding set of sensors is transmitted in real time to the RTC module via the RTC protocol and over the data transmission link.

In some embodiments of the present application, as shown in fig. 6, fig. 6 is a schematic diagram of a fifth embodiment of the present application, and the multimedia data transmission apparatus 60 includes: the obtaining module 601, the compressing module 602, and the first transmitting module 603 further include:

the establishing module 604 is configured to establish multiple data transmission links between the electronic device and the RTC module by using an RTC protocol, associate the data transmission links with group identifiers of corresponding groups of sensors, and store the group identifiers and link information of the data transmission links in an associated manner.

In some embodiments of the present application, as shown in fig. 6, further comprising:

a monitoring module 605, configured to monitor a connection state of a data transmission link in a process of transmitting target multimedia data;

the establishing module 604 is further configured to, when the connection status indicates that the data transmission link is disconnected, read pre-stored link information corresponding to the data transmission link, and trigger the data transmission link to reconnect according to the link information.

It is understood that the transmission device 60 of multimedia data in fig. 6 of the present embodiment and the transmission device 50 of multimedia data in the above-mentioned embodiment, the obtaining module 601 and the obtaining module 501 in the above-mentioned embodiment, the compressing module 602 and the compressing module 502 in the above-mentioned embodiment, and the first transmission module 603 and the first transmission module 503 in the above-mentioned embodiment may have the same functions and structures.

It should be noted that the foregoing explanation of the method for transmitting multimedia data is also applicable to the apparatus for transmitting multimedia data of the present embodiment, and is not repeated herein.

In this embodiment, multimedia data acquired by at least one group of sensors configured on a vehicle are acquired, data compression processing is performed on each group of multimedia data, protocol conversion is performed on each group of compressed multimedia data by using a real-time audio/video service (RTC) protocol, target multimedia data corresponding to each group of multimedia data is obtained, the target multimedia data is transmitted to a real-time audio/video service (RTC) module in a cloud server through the RTC protocol, and the real-time uploading of the multimedia data to the real-time audio/video service (RTC) module in the cloud server is realized, so that the timeliness of the whole multimedia data transmission is improved, and the transmission delay of the multimedia data is reduced.

Fig. 7 is a schematic diagram according to a sixth embodiment of the present application.

As shown in fig. 7, the multimedia data transmission apparatus 70 includes:

the receiving module 701 is configured to receive at least one set of target multimedia data, where the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding set of sensors and performing protocol conversion on the compressed multimedia data by using a real-time audio/video service (RTC) protocol;

a second transmission module 702, configured to transmit at least one set of target multimedia data to the remote control device via a real-time audio video service, RTC, protocol.

In some embodiments of the present application, the receiving module 701 is specifically configured to receive multiple sets of target multimedia data.

As shown in fig. 8, fig. 8 is a schematic diagram of a multimedia data transmission apparatus 80 according to a seventh embodiment of the present application, including: the receiving module 801 and the second transmitting module 802 further include:

a processing module 803, configured to perform merging processing on multiple sets of target multimedia data, so as to process the multiple sets of target multimedia data into a real-time video stream;

the second transmission module 802 is specifically configured to transmit the real-time video stream to the remote control device via an RTC protocol.

It is understood that the transmission device 60 of multimedia data in fig. 8 of the present embodiment may have the same functions and structures as the transmission device 70 of multimedia data in the above-mentioned embodiment, the receiving module 801 and the receiving module 701 in the above-mentioned embodiment, and the second transmission module 802 and the second transmission module 702 in the above-mentioned embodiment.

It should be noted that the foregoing explanation of the method for transmitting multimedia data is also applicable to the apparatus for transmitting multimedia data of the present embodiment, and is not repeated herein.

In this embodiment, at least one group of target multimedia data is received, the target multimedia data is obtained by performing data compression processing on multimedia data acquired by a corresponding group of sensors, performing protocol conversion on the compressed multimedia data by using an RTC protocol, and transmitting the at least one group of target multimedia data to a remote control device via the RTC protocol, so that the at least one group of target multimedia data is pushed to the remote control device in real time, thereby improving the timeliness of overall multimedia data transmission and reducing the transmission delay of the multimedia data.

There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product.

Fig. 9 is a block diagram of an electronic device for implementing a transmission method of multimedia data according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 9, the apparatus 900 includes a computing unit 901, which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The calculation unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 901 performs the respective methods and processes described above, for example, a transmission method of multimedia data.

For example, in some embodiments, the method of transmission of multimedia data may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 900 via ROM 902 and/or communications unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the transmission method of multimedia data described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the transmission method of the multimedia data by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

The program code for implementing the transmission method of multimedia data of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

The embodiment of the present application further provides an automatic driving vehicle, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the aforementioned transmission method of multimedia data.

An embodiment of the present application further provides a cloud server, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to perform the aforementioned transmission method of multimedia data.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.