阅读说明：本技术 一种视频通话方法、装置及设备 (Video call method, device and equipment ) 是由 陈维强 李蕾 高雪松 于 2020-04-29 设计创作，主要内容包括：本申请公开了一种视频通话方法、装置及设备,用于解决现有重新将被停止进行视频通话的第一RTSP数据流接入视频通话中时耗时较多,降低用户体验的问题。本申请中第一RTSP数据流被第二RTSP数据流替换后,电子设备在接收到停止显示该第二RTSP数据流的第一停止指令时,因为第一RTSP数据流的缓存空间依然在缓存该第一RTSP数据流,因此可以直接从该缓存空间中获取该第一RTSP数据流对应的第一视频帧,并拼接显示,从而可以重新快速将第一RTSP数据流接入视频通话中,提高了用户体验。(The application discloses a video call method, a video call device and video call equipment, which are used for solving the problems that the time consumption is more and the user experience is reduced when the first RTSP data stream which is stopped to carry out the video call is accessed into the video call again in the prior art. After the first RTSP data stream is replaced by the second RTSP data stream in the application, when the electronic device receives a first stop instruction for stopping displaying the second RTSP data stream, because the buffer space of the first RTSP data stream still buffers the first RTSP data stream, the electronic device can directly obtain the first video frame corresponding to the first RTSP data stream from the buffer space, and perform splicing display, so that the first RTSP data stream can be rapidly accessed into a video call again, and user experience is improved.)

1. A method for video telephony, the method comprising:

receiving a first stop instruction; according to the information of the second image acquisition equipment which stops the video call carried in the first stop instruction, stopping receiving the second RTSP data stream acquired by the second image acquisition equipment;

and acquiring a corresponding first video frame from a cache space for storing a first RTSP data stream according to the first stop instruction, wherein the first RTSP data stream is a data stream replaced by a second RTSP data stream, and splicing the first video frame to a display position corresponding to the second RTSP data stream of a video call display window for display.

2. The method of claim 1, wherein prior to receiving the first stop instruction, the method further comprises:

receiving a video call instruction, wherein the video call instruction carries information of a second image acquisition device to be accessed into a video call, and receiving a second RTSP data stream sent by the second image acquisition device;

and determining a first RTSP data stream replaced by the second RTSP data stream according to the video call instruction, caching the first RTSP data stream and the second RTSP data stream in a corresponding cache space, acquiring a corresponding second video frame from the cache space corresponding to the second RTSP data stream, splicing the second video frame to a display position corresponding to the first RTSP data stream of a video call display window, and displaying the second video frame.

3. The method of claim 2, wherein said determining, according to the video call instruction, the first RTSP data stream replaced by the second RTSP data stream comprises:

and if the video call instruction is an access instruction, determining that the second RTSP data stream replaces a preset first RTSP data stream when the number of the RTSP data streams of the currently accessed video call reaches a preset number threshold.

4. The method of claim 2, wherein said determining, according to the video call instruction, the first RTSP data stream replaced by the second RTSP data stream comprises:

and if the video call instruction is a replacement instruction, the replacement instruction carries information for replacing the first image acquisition device with the second image acquisition device, and the RTSP data stream acquired by the first image acquisition device is used as the replaced first RTSP data stream.

5. The method of any of claims 2-4, wherein after determining the first RTSP data stream replaced by the second RTSP data stream according to the video call instruction, the method further comprises:

replacing a display plug-in a Gstreamer pipeline receiving the first RTSP data stream for displaying a video frame of the first RTSP data stream with a temporary consumption plug-in.

6. The method of claim 5, wherein prior to replacing the display plug-in with the temporary drain plug-in, the method further comprises:

setting the state of the Gstreamer pipeline to be a pause state;

after replacing the display plug-in with the temporary drain plug-in, the method further comprises:

and setting the state of the Gstreamer pipeline as an operation state.

7. The method of claim 1, wherein before said retrieving the corresponding first video frame from the buffer space for storing the first RTSP data stream, the method further comprises:

replacing the temporary consumption plug-in the Gstreamer pipeline receiving the first RTSP data stream with a display plug-in.

8. The method of claim 7, wherein prior to replacing the temporary consumption plug-in with the display plug-in, the method further comprises:

setting the state of the Gstreamer pipeline to be a pause state;

after replacing the temporary consumption plug-in with the display plug-in, the method further comprises:

and setting the state of the Gstreamer pipeline as an operation state.

9. A video call apparatus, the apparatus comprising:

a receiving unit, configured to receive a first stop instruction; according to the information of the second image acquisition equipment which stops the video call carried in the first stop instruction, stopping receiving the second RTSP data stream acquired by the second image acquisition equipment;

and the processing unit is used for acquiring a corresponding first video frame from a cache space for storing a first RTSP data stream according to the first stop instruction, wherein the first RTSP data stream is a data stream replaced by a second RTSP data stream, and the first video frame is spliced to a display position corresponding to the second RTSP data stream in a video call display window and displayed.

10. An electronic device, characterized in that the electronic device comprises at least a processor and a memory, the processor being adapted to implement the steps of the video call method according to any of claims 1-8 when executing a computer program stored in the memory.

Technical Field

The present application relates to the field of video call technologies, and in particular, to a video call method, apparatus, system, device, and medium.

Background

With the development of video call technology, in the existing home, the intelligent housekeeper server can already realize real-time video call through the display interface of the intelligent television.

For example: the father makes a video call with the grander's milk through the image acquisition equipment in the living room. Mom is busy in the kitchen and cannot take out to get to the living room. If a Real Time Streaming Protocol (RTSP) data stream of the kitchen image capture device is to be accessed to the video call to obtain the video picture of mom in the kitchen, dad can input a video call command "access the kitchen image capture device" to the intelligent housekeeper server through voice. The intelligent housekeeper server sends a notification of pulling the RTSP data stream to the image acquisition equipment in the kitchen, and the image acquisition equipment in the kitchen can prompt a user whether to access the video call after receiving the notification sent by the intelligent housekeeper server. And if the information that the video call is accessed is received, the RTSP data stream is sent to the intelligent housekeeper server by the image acquisition equipment of the kitchen, the RTSP data stream of the image acquisition equipment of the kitchen is pulled by the intelligent housekeeper server, the video frames in the RTSP data stream are spliced to the video call display window, and the video frames are displayed in the display interface of the intelligent television.

Or, the father and the mother make video calls with the grander's milch family and the uncle's family through the image acquisition equipment of the living room. Friends and relatives want to watch the daughter's piano performance. The piano is not in the living room or the best viewing position of the image acquisition device in the living room, the piano is in other household scenes such as a piano room or a bedroom of a daughter, and the piano is in the piano room as an example, at this time, if the RTSP data stream of the image acquisition device of the piano room is accessed to the video call, dad can input a video call instruction 'accessing the image acquisition device of the piano room' to the intelligent housekeeper server through voice. The intelligent housekeeper server sends a notice of pulling the RTSP data stream to the image acquisition equipment of the musical instrument room, and the image acquisition equipment of the musical instrument room can prompt a user whether to access a video call after receiving the notice sent by the intelligent housekeeper server. And if the information that the video call is accessed is received, the image acquisition equipment of the organ room sends the RTSP data stream to the intelligent housekeeper server, and the intelligent housekeeper server starts to pull the RTSP data stream of the image acquisition equipment of the organ room, splices the video frames in the RTSP data stream to a video call display window and displays the video frames in a display interface of the intelligent television.

Fig. 1 is a schematic diagram of a system architecture for implementing a video call provided in the prior art, as shown in fig. 1: the image acquisition equipment sends the RTSP data stream to the intelligent housekeeper server, meanwhile, the cloud server sends the RTSP data stream to the intelligent housekeeper server after receiving the RTSP data stream of other users, the intelligent housekeeper server processes the video frame in the obtained RTSP data stream, and sends the audio frame in the RTSP data stream and the processed video frame to the intelligent television to display the content of the video call.

When a multi-person video call is performed, taking six-person video call as an example, fig. 2 is a schematic flow diagram of an RTSP data stream for implementing a video call provided in The prior art, where an intelligent housekeeper server acquires one RTSP data stream from a local image acquisition device, and other five RTSP data streams need to be acquired from a cloud server, video frames in The six RTSP data streams are all decoded in The intelligent housekeeper server, and a video frame in a color coding (YUV) format is obtained after decoding, The intelligent housekeeper server implements scaling and splicing of The video frames in The six YUV format, splices The video frames in The YUV format of a six-grid YUV picture, sends The spliced video frames in The format to a High-speed communication module, sends The video frames in The format to an internet television (Over-The-Top TV, OTT) through The High Definition Multimedia box Interface, HDMI) line is sent to the smart television, and the display is carried out in the display interface of the smart television.

However, the amount of RTSP data stream accessing to the video call is limited due to the limitation of cloud load and bandwidth. At present, after the number of RTSP data streams accessed to a video call reaches a preset number threshold, if the RTSP data streams of other image acquisition devices in a user's home need to be accessed to the video call, an intelligent housekeeper server needs to first stop receiving a first RTSP data stream of a first image acquisition device currently performing the video call, and then pull a second RTSP data stream of a second image acquisition device to be accessed to the video call, so that the second RTSP data stream is accessed to the video call. Therefore, after the second RTSP data stream stops performing the video call, if the first RTSP data stream still needs to be accessed to the video call, the intelligent housekeeper server still needs to pull the first RTSP data stream again, so that the first RTSP data stream is accessed to the video call, and the process of pulling the first RTSP data stream again consumes a certain time, thereby reducing the user experience.

Disclosure of Invention

The application provides a video call method, a device, a system, equipment and a medium, which are used for solving the problems that the time consumption is more and the user experience is reduced when the first RTSP data stream which is stopped to carry out the video call is accessed into the video call again in the prior art.

In a first aspect, the present application provides a video call method, including:

receiving a first stop instruction; according to the information of the second image acquisition equipment which stops the video call carried in the first stop instruction, stopping receiving the second RTSP data stream acquired by the second image acquisition equipment;

and acquiring a corresponding first video frame from a cache space for storing a first RTSP data stream according to the first stop instruction, wherein the first RTSP data stream is a data stream replaced by a second RTSP data stream, and splicing the first video frame to a display position corresponding to the second RTSP data stream of a video call display window for display.

In a second aspect, the present application further provides a video call device, including:

a receiving unit, configured to receive a first stop instruction; according to the information of the second image acquisition equipment which stops the video call carried in the first stop instruction, stopping receiving the second RTSP data stream acquired by the second image acquisition equipment;

and the processing unit is used for acquiring a corresponding first video frame from a cache space for storing a first RTSP data stream according to the first stop instruction, wherein the first RTSP data stream is a data stream replaced by a second RTSP data stream, and the first video frame is spliced to a display position corresponding to the second RTSP data stream in a video call display window and displayed.

In a third aspect, the present application further provides a video call system, where the system includes any one of the above-mentioned video call devices applied to the electronic device, and a smart television for displaying a video frame obtained by splicing the electronic device to a video call display window.

In a fourth aspect, the present application further provides an electronic device, where the electronic device at least includes a processor and a memory, and the processor is configured to implement the steps of any of the video call methods when executing a computer program stored in the memory.

In a fifth aspect, the present application further provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any of the video call methods described above.

In the application, after the first RTSP data stream is replaced by the second RTSP data stream, when the electronic device receives a first stop instruction for stopping displaying the second RTSP data stream, because the buffer space of the first RTSP data stream still buffers the first RTSP data stream, the electronic device can directly obtain the first video frame corresponding to the first RTSP data stream from the buffer space, and perform splicing display, so that the first RTSP data stream can be quickly accessed into a video call again, and user experience is improved.

Drawings

Fig. 1 is a schematic diagram of a system architecture for implementing a video call according to the prior art;

fig. 2 is a schematic flow diagram of an RTSP data stream for implementing a video call according to the prior art;

fig. 3 is a schematic diagram of a video call process according to some embodiments of the present application;

fig. 4 is a schematic diagram illustrating a process of pulling an RTSP data stream by an electronic device according to some embodiments of the present application;

fig. 5 is a schematic diagram illustrating a Gstreamer pipeline parsing flow of two video call real-time RTSP data streams according to some embodiments of the present application;

fig. 6 is a schematic diagram illustrating a detailed implementation process of a first video call according to some embodiments of the present application;

FIG. 7 is a schematic diagram of a relationship between a family account and a sub-account as provided by some embodiments of the present application;

fig. 8 is a schematic diagram illustrating a detailed implementation process of a second video call according to some embodiments of the present application;

fig. 9 is a schematic diagram illustrating a detailed implementation process of a third video call according to some embodiments of the present application;

fig. 10 is a schematic diagram of adding a second RTSP data stream to a first RTSP data stream for video call according to some embodiments of the present application;

fig. 11 is a schematic diagram illustrating a detailed implementation process of a fourth video call according to some embodiments of the present application;

fig. 12 is a schematic diagram of a first RTSP data stream replaced by a second RTSP data stream according to some embodiments of the present application;

fig. 13 is a schematic structural diagram of a video call device according to some embodiments of the present application;

fig. 14 is a schematic structural diagram of an electronic device according to some embodiments of the present application;

fig. 15 is a schematic structural diagram of a video call system according to some embodiments of the present application.

Detailed Description

After the first RTSP data stream is replaced by the second RTSP data stream, in order to rapidly access the first RTSP data stream into the video call again, the application provides a video call method, a device, a system, equipment and a medium.

In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the practical application process, the electronic device can respectively pull the RTSP data streams of one or more local image acquisition devices, receive one or more RTSP data streams sent by the cloud server, and splice video frames in the RTSP data streams to a video call display window of the smart television for video call. In the application, after the first RTSP data stream is replaced by the second RTSP data stream, the electronic device may obtain a corresponding first video frame from a cache space storing the first RTSP data stream when receiving a first stop instruction for stopping displaying the second RTSP data stream, and splice and display the first video frame to a display position corresponding to the second RTSP data stream in a video call display window of the smart television, so as to quickly access the first RTSP data stream to the video call again.

Example 1:

fig. 3 is a schematic diagram of a video call process according to some embodiments of the present application, where the process includes the following steps:

s301: receiving a first stop instruction; and stopping receiving a second RTSP data stream acquired by a second image acquisition device according to information of the second image acquisition device for stopping video call, which is carried in the first stop instruction.

In the application, the video call method is applied to the electronic device, and the electronic device can be an intelligent housekeeper server or other intelligent devices. When the electronic device is an intelligent housekeeping server, the intelligent housekeeping server is a server located locally. The electronic equipment is based on a Gstreamer framework, wherein the Gstreamer framework is a framework for creating a streaming media application program, and the Gstreamer framework is based on plug-ins which can be conveniently accessed into the Gstreamer pipeline to realize the multiplexing of codes.

The electronic device may receive a first stop instruction, where the first stop instruction may be an instruction generated after the electronic device receives a trigger operation for the electronic device, and may also be sent by another received intelligent device. And the first stop instruction carries information of the second image acquisition device which stops the video call, so that the electronic device can know which image acquisition device RTSP data stream is stopped to receive. Wherein the information of the second image capturing device may be: the name of the room where the second image capturing device is located, such as a kitchen, a bedroom, etc., or may be one or more of information such as an IP address of the second image capturing device.

The first stop instruction may be a voice instruction, and after the electronic device receives the first stop instruction, the electronic device may perform semantic recognition on the first stop instruction by using a voice recognition technology, so as to recognize information of the second image capturing device that stops performing the video call, which is carried in the first stop instruction.

In order to enable the electronic device to determine the image capturing device and the RTSP data stream thereof according to the information of the image capturing device, the electronic device may pre-store a corresponding relationship between the information of the image capturing device and an identifier of the RTSP data stream of the image capturing device. Because the information of the image acquisition device can be the room name and/or the IP address of the image acquisition device, the corresponding relationship between the room name of the image acquisition device, the IP address of the image acquisition device and the RTSP data stream identification address of the image acquisition device can be stored, wherein the IP addresses of different image acquisition devices are different, and because the RTSP data stream identification address of the image acquisition device includes the IP address of the image acquisition device, the RTSP data stream identification addresses corresponding to different image acquisition devices are also different, and different RTSP data streams can be distinguished according to the IP address of the image acquisition device included in the RTSP data stream identification address of the image acquisition device.

Therefore, after the electronic device receives the first stop instruction, the identification address of the RTSP data stream of the second image acquisition device is determined according to the information of the second image acquisition device stopping the video call carried in the first stop instruction, and the stored corresponding relationship between the name of the room where the image acquisition device is located, the IP address of the image acquisition device, and the RTSP data stream identification address of the image acquisition device. And stopping receiving the second RTSP data stream acquired by the second image acquisition device according to the determined second RTSP data stream identification address.

S302: and acquiring a corresponding first video frame from a cache space for storing a first RTSP data stream according to the first stop instruction, wherein the first RTSP data stream is a data stream replaced by a second RTSP data stream, and splicing the first video frame to a display position corresponding to the second RTSP data stream of a video call display window for display.

In this application, the electronic device respectively pulls the RTSP data streams of the local one or more image capturing devices, and receives one or more RTSP data streams sent by the cloud server to perform video call, which is not repeated herein.

The image acquisition equipment can be equipment such as an intelligent sensor, a camera and a video camera, and after the image acquisition equipment acquires the RTSP data stream, the RTSP data stream can be sent to the electronic equipment in a wired transmission mode, a wireless transmission mode and the like.

Fig. 4 is a schematic diagram of a process of pulling an RTSP data stream by the electronic device, as shown in fig. 4, each local image capture device may access to the home lan through a network (router) to be connected to the electronic device, the electronic device may pull an RTSP data stream of the local image capture device through the network, may transmit the local RTSP data stream to the cloud server, and may also obtain the RTSP data stream from the cloud server.

In order to cache the received RTSP data streams of each image capturing device, a one-to-one cache space is respectively provided in the electronic device for each received RTSP data stream, because the IP addresses of each image capturing device are different, the electronic device can distinguish the cache spaces corresponding to the RTSP data streams of different image capturing devices according to the IP addresses of the image capturing devices.

An RTSP data stream received by the electronic device may be an RTSP data stream sent by the local image capturing device or an RTSP data stream sent by the cloud server, fig. 5 is a block diagram illustrating a Gstreamer pipeline parsing flow of a two-way video call Real-time RTSP data stream provided in some embodiments of the present application, as shown in fig. 5, in the RTSP data stream received by the electronic device, one RTSP data stream is an RTSP data stream obtained from the local image capturing device (e.g., an intelligent sensor), the other RTSP data stream is an RTSP data stream obtained from the cloud server, the two RTSP data streams respectively go through the steps of source (rtsrc), Real-time Transport Protocol (RTP) packet parsing (RTP 264depay), coding format parser (h264parse), decoder (omxh264dec) and display plug-in (appsnink plug-in) to generate two decoded YUV format video frames, the two paths of RTSP data streams are cached in the corresponding cache spaces, the electronic equipment acquires video frames from the corresponding cache spaces, the video frames in the YUV format of the two paths of RTSP data streams are spliced into video frames in the YUV format of two palace grid pictures, and the video frames in the YUV format of the two palace grid pictures are sent to the top box end of the OTT machine through the high-speed passing module and displayed in the display interface of the intelligent television.

In this application, after the first RTSP data stream is replaced by the second RTSP data stream, the electronic device does not stop pulling the first RTSP data stream, but always receives the first RTSP data stream and buffers the first RTSP data stream in a buffer space corresponding to the first RTSP data stream. After receiving the first stop instruction, the video frame corresponding to the buffer space for storing the first RTSP data stream may be obtained from the buffer space for storing the first RTSP data stream, and the video frame of the first RTSP data stream may be spliced to a position of the video call display window corresponding to the second RTSP data stream and displayed, so that the first RTSP data stream may be quickly re-accessed to the video call.

Specifically, the video frame of the first RTSP data stream may be spliced to the display position corresponding to the second RTSP data stream in the video call display window by using the prior art, and displayed, for example, the decoded video frame in the YUV format may be obtained from the buffer space corresponding to the first RTSP data stream by using an opencv interface, and then the video frame is spliced to the display position corresponding to the second RTSP data stream in the video call display window and displayed, which is not described herein again.

In addition, the Gstreamer pipeline of the electronic device is used for pulling, decoding, scaling, splicing and data transmission of audio frames and video frames during video call, and the pipeline can also be called a video call playing pipeline. The RTSP data stream of the image capturing device may include video frames and/or audio frames, and for convenience of understanding, the video frames in the RTSP data stream are represented by the video frames in this application, for example, in this application, the video frames in the RTSP data stream are spliced and displayed in the video call display window, and it is indicated that the video frames in the RTSP data stream are displayed in the video call display window, and/or the audio frames are played in the video call.

In the application, after the first RTSP data stream is replaced by the second RTSP data stream, when the electronic device receives a first stop instruction for stopping displaying the second RTSP data stream, because the buffer space of the first RTSP data stream still buffers the first RTSP data stream, the electronic device can directly obtain the first video frame corresponding to the first RTSP data stream from the buffer space, and perform splicing display, so that the first RTSP data stream can be quickly accessed into a video call again, and user experience is improved.

In order to reduce the time consumption of the first RTSP data stream to re-access the video call, on the basis of the foregoing embodiments, in this application, before the acquiring the corresponding first video frame from the buffer space for storing the first RTSP data stream, the method further includes:

replacing the temporary consumption plug-in the Gstreamer pipeline receiving the first RTSP data stream with a display plug-in.

In this application, the electronic device is based on a gsstreamer framework, and when the electronic device pulls a plurality of RTSP data streams into a video call, a corresponding gsstreamer pipeline may be created for each RTSP data stream respectively to receive the RTSP data streams.

After the first RTSP data stream is replaced by the second RTSP data stream, because the first RTSP data stream may be consumed but not displayed, the first RTSP data stream may be consumed by using the temporary consumption plug-in, so as to realize that the first RTSP data stream may be consumed in real time, the first RTSP data stream may be reconnected to the video call without a delay, and the first RTSP data stream may not be displayed. When the second RTSP data stream stops performing the video call, that is, when the electronic device receives the first stop instruction, in order to reconnect the replaced first RTSP data stream to the video call, the temporary consuming plug-in the Gstreamer pipeline of the first RTSP data stream may be replaced by the display plug-in, so that the first RTSP data stream is decoded into the video frame in the YUV format by the display plug-in, and the decoded first video frame in the YUV format may be acquired from the buffer space corresponding to the first RTSP data stream, so that the first video frame of the first RTSP data stream may be displayed on the video call display window.

In the present application, the display plug-in may be an applink plug-in. The appsnik plug-in is used as a display plug-in to decode the first RTSP data stream into a video frame in YUV format, which is the prior art and is not described herein again.

In the application, after the first RTSP data stream is replaced by the second RTSP data stream, the buffer space of the first RTSP data stream still buffers the first RTSP data stream, and when the electronic device receives a first stop instruction for stopping displaying the second RTSP data stream, the electronic device may replace the temporary consumption plug-in the Gstreamer pipeline of the first RTSP data stream with the display plug-in, acquire the first RTSP data stream from the buffer space of the first RTSP data stream, decode the first RTSP data stream into the first video frame in the YUV format, and perform splicing display, so that the first RTSP data stream may be quickly accessed into a video call again, thereby improving user experience.

In the prior art, when the number of RTSP data streams accessing a video call reaches a preset number threshold, if a second RTSP data stream is to be accessed to the video call, the receiving of the first RTSP data stream needs to be stopped first, and then the second RTSP data stream is pulled, which may take about 200 ms. When the second RTSP stream is pulled, it takes about 14ms for the plug creation, decoding, and the like in the Gstreamer pipe of the second RTSP stream, and it takes about 214ms until this time. When the second RTSP data stream stops performing the video call and the first RTSP data stream is pulled again, it also probably takes about 200ms, and after the second RTSP data stream and the second RTSP data stream stop performing the video call, the total consumed time for pulling the first RTSP data stream is about 414 ms. When the first RTSP stream is pulled, it takes about 14ms for the plug creation, decoding, and the like in the Gstreamer pipe of the first RTSP stream, and it takes about 428ms until this time. That is, in the prior art, the second RTSP data stream replaces the first RTSP data stream to access the video call, and after the second RTSP data stream stops performing the video call, the first RTSP data stream is pulled again, and in the whole process of re-accessing the first RTSP data stream to the video call, it takes about 428 ms.

In the application, even if the first RTSP data stream is not displayed in the display window after being replaced, the first RTSP data stream is still cached, and the first RTSP data stream does not need to be pulled again after the second RTSP data stream stops the video call, so that the time of about 200ms probably consumed in the process of pulling the first RTSP data stream again can be at least saved. Specifically, in the present application, only in the process of pulling the second RTSP data stream, it takes about 200ms, and when the second RTSP data stream is pulled, it takes about 14ms for the plug creation, decoding, and the like in the Gstreamer pipe of the second RTSP data stream, and it takes about 214ms until this time. When the first RTSP data stream is reconnected to the video call after the second RTSP data stream stops the video call, the time of creating and decoding the plug in the Gstreamer pipeline of the first RTSP data stream is about 14ms, and the time of about 228ms is consumed by the time. That is, in the present application, it takes about 228ms in the whole process of switching the second RTSP data stream into the video call instead of the first RTSP data stream, and switching the first RTSP data stream into the video call again after the second RTSP data stream stops performing the video call. Therefore, compared with the prior art, the method and the device for receiving the RTSP data stream have the advantages that time consumption in the process of re-accessing the first RTSP data stream into the video call can be greatly reduced, and user experience is improved.

In order to more smoothly replace the temporary consumed plugin in the Gstreamer pipeline of the first RTSP data stream with the display plugin, on the basis of the foregoing embodiments, in this application, before replacing the temporary consumed plugin with the display plugin, the method further includes:

setting the state of the Gstreamer pipeline to be a pause state;

after replacing the temporary consumption plug-in with the display plug-in, the method further comprises:

and setting the state of the Gstreamer pipeline as an operation state.

Plug-in replacement can only be implemented when the pipe is in a PAUSED STATE (GST _ STATE _ used), so the STATE of the Gstreamer pipe can be set to a PAUSED STATE before the temporary consumed plug-in the Gstreamer pipe of the first RTSP data stream is replaced with a display plug-in, and the temporary consumed plug-in the pipe can be replaced with a display plug-in when the Gstreamer pipe is in a PAUSED STATE. After the temporary consumption plug-in is replaced by the display plug-in, in order to facilitate splicing and displaying of a video frame of a subsequent first RTSP data stream, the gsstreamer pipeline of the first RTSP data stream is set to be in an operating STATE (GST _ STATE _ PLAYING), so that the first video frame in the YUV format after decoding is acquired from a buffer space corresponding to the first RTSP data stream, and the first video frame of the first RTSP data stream is subsequently spliced and displayed.

Specifically, switching of the Gstreamer pipe between different states may be accomplished by the application calling the gst _ element _ set _ state interface function, e.g., the state of the Gstreamer pipe may be set to a paused state by the application calling the gst _ element _ set _ state interface function before replacing the temporary consumption plug-in with the display plug-in. After the temporary consumption plug-in is replaced with the display plug-in, the state of the Gstreamer pipe can be switched back to the running state, again by the gst _ element _ set _ state interface function.

Fig. 6 is a schematic diagram of a detailed implementation process of a first video call according to some embodiments of the present application, where the process includes:

s601: and receiving a first stop instruction carrying information of a second image acquisition device for stopping video call, and stopping receiving the second RTSP data stream.

S602: and setting the state of the Gstreamer pipeline of the first RTSP data stream to be a pause state according to the first stop instruction.

S603: and replacing the fakesink plug-in the Gstreamer pipeline of the first RTSP data stream with the appsink plug-in.

S604: and setting the state of the Gstreamer pipeline of the first RTSP data stream as a running state.

S605: and acquiring the decoded YUV format first video frame from a buffer space corresponding to the first RTSP data stream.

S606: and splicing the acquired first video frame in the YUV format after decoding to a display position corresponding to a second RTSP data stream of the video call display window and displaying the video frame.

In order to implement access of the second RTSP data stream, on the basis of the foregoing embodiments, before receiving the first stop instruction, in this application, the method further includes:

receiving a video call instruction, wherein the video call instruction carries information of a second image acquisition device to be accessed into a video call, and receiving a second RTSP data stream sent by the second image acquisition device;

and determining a first RTSP data stream replaced by the second RTSP data stream according to the video call instruction, caching the first RTSP data stream and the second RTSP data stream in a corresponding cache space, acquiring a corresponding video frame from the cache space corresponding to the second RTSP data stream, splicing the video frame to a display position corresponding to the first RTSP data stream of a video call display window, and displaying the video frame.

When a new RTSP data stream needs to be accessed to a video call, a user may send a video call instruction to the electronic device, where the video call instruction may be an instruction generated after the electronic device receives a trigger operation on the electronic device, or may be sent by another received intelligent device.

In order to facilitate the electronic device to know which RTSP data stream is accessed to the video call, the video call instruction carries information of the second image capturing device to be accessed to the video call. The information of the image capturing device may be the same as the information of the image capturing device in the above embodiments, and the process of determining the RTSP data stream identification address of the second image capturing device according to the information of the second image capturing device is also the same as the above embodiments, and is not described herein again.

For convenience of processing, in the present application, the information of the image capturing device may further include information of a sub-account corresponding to the image capturing device. In this application, if a plurality of image capturing devices are installed locally, a sub-account may be set for each image capturing device, and the following steps of setting a sub-account for each image capturing device will be described in detail by taking an electronic device as an example of an intelligent housekeeping server:

first, a home account is created.

The family account is in one-to-one correspondence with the intelligent housekeeper server, generally speaking, a natural family only has one intelligent housekeeper server, and therefore only one family account is configured for the family.

In order to facilitate management of the image capture devices located in the natural family, for each image capture device present in the family, a corresponding sub-account is created for each image capture device under a family account. Each sub-account corresponds to different image acquisition devices, and it can be understood that the corresponding relationship between the sub-account of the image acquisition device and the RTSP data stream identification address of the image acquisition device may be pre-stored in the intelligent housekeeper server. The sub account may carry a name of a room where the image capturing device is located, and may include, for example, a name of a room where the image capturing device is located, or an identifier of a name of a room where the image capturing device is located. Because the intelligent housekeeper server pre-stores the corresponding relationship between the sub-account of the image acquisition device and the RTSP data stream identification address of the image acquisition device, when the information of the image acquisition device further includes the information of the sub-account corresponding to the image acquisition device, the RTSP data stream identification address of the image acquisition device corresponding to the sub-account can be determined according to the sub-account.

Fig. 7 is a schematic diagram of a relationship between a family account and a sub-account according to some embodiments of the present application, where a family account may be created first, and then a corresponding sub-account is created for each image capture device under the family account, and in order to identify the sub-account and the family account to which the sub-account belongs, the sub-account may simultaneously include information of the family account and the sub-account. For example, if the family account is family a, the sub-account corresponding to the image capturing device in the living room may be: family A, living room; the sub-accounts of the kitchen's image capture devices may be: family A, kitchen; the sub-account of the image acquisition device of the bedroom may be: family A, bedroom; the sub-account of the image capturing device of the piano room may be: family A, music room, etc.

Generally, in the present application, a main account may be configured in advance in a sub-account according to actual requirements, or a sub-account corresponding to an image capturing device in a certain room may be used as the main account by default. The main account can preferentially carry out video call, and if the video call instruction does not contain the information of the replaced first image acquisition device, the image acquisition device corresponding to the main account is defaulted to be the first image acquisition device. In addition, in order to be able to quickly reconnect the first RTSP data stream of the first image capturing device to the video call when the second RTSP data stream stops performing the video call, the first image capturing device is generally an image capturing device corresponding to the primary account, so the video call instruction may only include information of the second image capturing device, and at this time, the second image capturing device is defaulted to replace the first image capturing device.

In general, a sub account corresponding to an image capturing device in a living room may be set as a main account, that is, the family a and the living room are main accounts.

The video call instruction can be a voice instruction, and after the electronic equipment receives the video call instruction, semantic recognition can be carried out on the video call instruction by adopting a voice recognition technology, so that information of second image acquisition equipment to be accessed into the video call, which is carried in the video call instruction, is recognized.

After determining the information of the second image acquisition device to be accessed to the video call, the electronic device may send a notification to the second image acquisition device to pull the RTSP data stream of the second image acquisition device. And after receiving the RTSP data stream pulling notification sent by the electronic equipment, the second image acquisition equipment can prompt the user whether to access the video call. Specifically, the second image capturing device may prompt the user whether to access the video call in a voice manner. After receiving the selection of the user, the second image acquisition device can send the selection of the user to the electronic device, and the electronic device can perform corresponding operation according to the selection of the user.

If the user selects that the user agrees to access the second RTSP data stream of the second image acquisition device to the video call, the second image acquisition device sends the second RTSP data stream to the electronic device, the electronic device starts to receive the second RTSP data stream sent by the second image acquisition device, and the electronic device splices video frames in the second RTSP data stream to a video call display window and displays the video frames in a display interface of the intelligent television.

If the user selects that the second RTSP data stream of the second image acquisition device is not approved to be accessed into the video call, the second image acquisition device does not send the second RTSP data stream to the electronic device, the electronic device does not perform the subsequent step of receiving the second RTSP data stream, and meanwhile, the electronic device does not perform the subsequent step of splicing the video frame of the second RTSP data stream to the video call display window.

When a video call instruction is received, the number of the RTSP data streams currently accessed to the video call may reach a preset number threshold, and according to information of the second image acquisition device to be accessed to the video call, carried in the video call instruction, it is required to determine which RTSP data stream is replaced by the second RTSP data stream.

In this application, the first RTSP data stream replaced by the second RTSP data stream is determined before the second RTSP data stream is displayed. And if the video call instruction only contains the information of the second image acquisition device, the RTSP data stream corresponding to the preset main account is used as the replaced first RTSP data stream by default. If the video call instruction also contains the information of the first image acquisition device, the RTSP data stream of the first image acquisition device contained in the video call instruction may be used as the replaced first RTSP data stream.

And in order to ensure that the first RTSP data stream replaced subsequently can be quickly re-accessed into the video call, the received first RTSP data stream is continuously buffered to the corresponding buffer space.

In addition, after the second RTSP data stream is received, in order to facilitate the subsequent splicing of the second video frame of the second RTSP data stream to the video call display window, the second RTSP data stream is decoded and then stored in the cache space corresponding to the second RTSP data stream, and during the splicing, the video frame in the YUV format decoded from the second RTSP data stream is obtained from the cache space corresponding to the second RTSP data stream, replaces the first RTSP data stream, and is spliced at the display position corresponding to the first RTSP data stream of the video call display window and displayed.

To further improve the user experience, on the basis of the foregoing embodiment, in this application, the determining, according to the video call instruction, the first RTSP data stream replaced by the second RTSP data stream includes:

and if the video call instruction is a replacement instruction, the replacement instruction carries information for replacing the first image acquisition device with the second image acquisition device, and the RTSP data stream acquired by the first image acquisition device is used as the replaced first RTSP data stream.

In this application, in order to determine the replaced first RTSP data stream, the video call instruction may be a replacement instruction, where the replacement instruction carries information for replacing the first image capturing device with the second image capturing device. For example, if the replacement instruction is "replace the image capturing device in the living room with the image capturing device in the kitchen", the electronic device may determine that the room name corresponding to the replaced first RTSP data stream is the living room, and accordingly, the RTSP data stream of the image capturing device in the living room is used as the replaced first RTSP data stream according to the pre-stored correspondence between the information of the image capturing device and the RTSP data stream identification address of the image capturing device. Meanwhile, according to the replacing instruction, the room name of the second image acquisition device to be accessed to the video call is determined to be a kitchen, so that the second RTSP data stream to be accessed to the video call is determined.

In order to further improve the user experience, on the basis of the foregoing embodiments, in this application, the determining, according to the video call instruction, the first RTSP data stream replaced by the second RTSP data stream includes:

and if the video call instruction is an access instruction, determining that the second RTSP data stream replaces a preset first RTSP data stream when the number of the RTSP data streams of the currently accessed video call reaches a preset number threshold.

Due to the limitation of cloud load, bandwidth and the like, the number of RTSP data streams accessed to the video call is limited to a certain extent, so that the threshold value of the number of RTSP data streams which can be simultaneously accessed to the video call can be preset. In the application, when the number of the RTSP data streams accessed to the video call reaches the number threshold, the second RTSP data stream may be substituted for the preset first RTSP data stream to access to the video call. The value is that when the number of the RTSP data streams in the video call does not reach the number threshold, the second RTSP data stream may be used to replace the preset first RTSP data stream to access the video call, or the second RTSP data stream and the access video call may be directly connected, and at this time, the second RTSP data stream and the first RTSP data stream are both connected to the video call, which may be flexibly selected according to the user's requirements.

In this application, when the video call instruction is the access instruction, the electronic device may first determine whether the number of RTSP data streams of the currently accessed video call reaches a preset number threshold, and if so, determine that the second RTSP data stream replaces the first RTSP data stream corresponding to the preset primary account.

It can be understood that, if the electronic device determines that the number of RTSP data streams of the currently accessed video call does not reach the preset number threshold, the second RTSP data stream may be accessed to the video call based on the prior art, which is not described herein again.

Fig. 8 is a schematic diagram of a detailed implementation process of a second video call according to some embodiments of the present application, where the process includes:

s801: and receiving an access instruction, wherein the access instruction carries information of a second image acquisition device to be subjected to video call, and receiving a second RTSP data stream sent by the second image acquisition device.

S802: and judging whether the number of RTSP data streams of the currently accessed video call reaches a preset number threshold, if so, performing S803, and if not, performing S804.

S803: and determining that the second RTSP data stream replaces the preset first RTSP data stream, splicing the second video frame of the received second RTSP data stream to the display position corresponding to the first RTSP data stream of the video call display window, and displaying the second video frame.

S804: and determining an idle display area in the video call display window, splicing the received video frame of the second RTSP data stream to the idle display area of the video call display window, and displaying the video frame.

When the video frames of the second RTSP data stream are spliced to the idle display area of the video call display window and displayed, the first RTSP data stream is not influenced by the access of the second RTSP data stream to the video call, and the first video frame of the first RTSP data stream and the second video frame of the second RTSP data stream are spliced to the video call display window.

Fig. 9 is a schematic diagram of a detailed implementation process of a third video call according to some embodiments of the present application, where the process includes:

s901: and receiving an access instruction, wherein the access instruction carries information of a second image acquisition device to be subjected to video call, and receiving a second RTSP data stream sent by the second image acquisition device.

S902: and judging whether the number of RTSP data streams of the currently accessed video call reaches a preset number threshold, if so, performing S903, and if not, performing S907.

S903: and determining that the second RTSP data stream replaces the preset first RTSP data stream, caching the first RTSP data stream and the second RTSP data stream in the corresponding cache space, acquiring a corresponding video frame from the cache space corresponding to the second RTSP data stream, splicing the video frame of the second RTSP data stream to a display position corresponding to the first RTSP data stream of the video call display window, and displaying the video frame.

S904: and receiving a first stopping instruction, wherein the first stopping instruction carries information of second image acquisition equipment stopping video call.

S905: the reception of the second RTSP data stream is stopped.

S906: and according to the first stop instruction, acquiring a first video frame from a cache space corresponding to the first RTSP data stream, splicing the first video frame of the first RTSP data stream to a display position corresponding to the second RTSP data stream of the video call display window, and displaying the first video frame.

S907: and determining an idle display area in the video call display window, and splicing the video frame of the second RTSP data stream to the idle display area of the video call display window and displaying the video frame.

Fig. 10 is a schematic diagram of adding a second RTSP data stream to a first RTSP data stream for video call, according to some embodiments of the present disclosure, where the left side (left and right in the drawing) is a schematic diagram of video call by the first RTSP data stream, and the right side (left and right in the drawing) is a schematic diagram of video call by adding the second RTSP data stream to the first RTSP data stream for video call.

For better buffering the first RTSP data stream, on the basis of the foregoing embodiments, in this application, after determining the first RTSP data stream replaced by the second RTSP data stream according to the video call instruction, the method further includes:

replacing a display plug-in a Gstreamer pipeline receiving the first RTSP data stream for displaying a video frame of the first RTSP data stream with a temporary consumption plug-in.

In the present application, because the electronic device is based on the gsstreamer framework, the video frames of the first RTSP data stream in the gsstreamer pipeline of the first RTSP data stream are received by employing the display plug-in before the first RTSP data stream is replaced by the second RTSP data stream so that the video frames of the first RTSP data stream can be displayed in the video call display window. And after the first RTSP data stream is replaced by the second RTSP data stream, the first RTSP data stream is only cached but not displayed, at this time, the temporary consumption plug-in can be adopted to replace the display plug-in to consume the first RTSP data stream, so that the first RTSP data stream can be consumed in real time, the first RTSP data stream can not be delayed when being reconnected to the video call, and the first RTSP data stream can not be displayed. After the temporary consumption plug-in replaces the display plug-in, the first RTSP data stream is not decoded into video frames in YUV format during the process of consuming the first RTSP data stream.

In the present application, the temporary consumption plug-in is a fakesink plug-in. Because the fakesink plug-in can receive any data stream on the pipeline in the Gstreamer frame, and the fakesink plug-in only receives the video frame in the data stream on the pipeline and does not display the video frame, when the fakesink plug-in is called to replace the display plug-in to consume the data stream, the receiving of the first RTSP data stream cannot be interrupted, and the received first RTSP data stream can be completely consumed, so that the first RTSP data stream can be reconnected to the video call without time delay.

In order to replace the display plug-in with the temporary consumption plug-in, on the basis of the foregoing embodiments, in this application, before replacing the display plug-in with the temporary consumption plug-in, the method further includes:

setting the state of the Gstreamer pipeline to be a pause state;

after replacing the display plug-in with the temporary drain plug-in, the method further comprises:

and setting the state of the Gstreamer pipeline as an operation state.

Therefore, the plug-in replacement can be implemented only when the pipe is in a pause state, so that before replacing the display plug-in the Gstreamer pipe of the first RTSP data stream with the temporary consumption plug-in, the state of the Gstreamer pipe may be set to the pause state, when the Gstreamer pipe is in the pause state, the display plug-in the pipe for displaying the video frame of the first RTSP data stream is replaced with the temporary consumption plug-in, and after the display plug-in is replaced with the temporary consumption plug-in, the Gstreamer pipe of the first RTSP data stream is set to the running state for the convenience of buffering and consumption of the subsequent first RTSP data stream.

Specifically, the electronic device may call a gst _ element _ set _ state interface function through an application program to realize switching between different states of the gstream pipe, for example, before replacing a display plug-in with a temporary consumption plug-in, the application program may call the gst _ element _ set _ state interface function to set the state of the gstream pipe to a pause state, and after replacing the display plug-in with the temporary consumption plug-in, the state of the gstream pipe may also be set to a running state through the gst _ element _ set _ state interface function.

Fig. 11 is a schematic diagram of a detailed implementation process of a fourth video call according to some embodiments of the present application, where the process includes:

s1101: and receiving a video call instruction, wherein the video call instruction carries information of a second image acquisition device to be accessed into the video call, and receiving a second RTSP data stream sent by the second image acquisition device.

And creating a corresponding Gstreamer pipeline for the second RTSP data stream to receive the second RTSP data stream.

S1102: and caching the second RTSP data stream in the corresponding cache space according to the video call instruction.

S1103: and determining a first RTSP data stream replaced by a second RTSP data stream according to the received video call instruction, and setting the state of the Gstreamer pipeline of the first RTSP data stream to be a pause state.

S1104: and replacing the appsink plug-in the Gstreamer pipeline of the first RTSP data stream with the fakesink plug-in.

S1105: and setting the state of the Gstreamer pipeline of the first RTSP data stream as a running state, and continuously caching the first RTSP data stream in the corresponding cache space.

S1106: and acquiring a decoded YUV format second video frame from a cache space corresponding to the second RTSP data stream, splicing the second video frame to a display position corresponding to the first RTSP data stream of the video call display window, and displaying the second video frame.

It should be noted that the steps included in the video call process are only an example, and the present application does not limit the sequence between S1101-S102 and S1103-S1105, that is, after receiving the video call instruction, the second RTSP data stream is first buffered, or the sequence between the first RTSP data stream replaced by the second RTSP data stream is first determined.

Fig. 12 is a schematic diagram of the first RTSP data stream replaced by the second RTSP data stream according to some embodiments of the present disclosure, where the left side (left and right in the figure) of the diagram indicates that the first RTSP data stream performs a video call, and the right side (left and right in the figure) of the diagram indicates that the second RTSP data stream performs a video call after the first RTSP data stream is replaced by the second RTSP data stream.

In addition, the states of the Gstreamer pipe also include a NULL (NULL) state and a READY (READY) state in the prior art, which are not described herein because they are not relevant to the present application.

Based on the same technical concept, the application also provides a video call device, and the object pushing device can realize the process executed by the electronic equipment in the embodiment.

Fig. 13 is a schematic structural diagram of a video call device according to some embodiments of the present application, and as shown in fig. 13, based on the above embodiments, the video call device according to the present application includes:

a receiving unit 1301, configured to receive a first stop instruction; according to the information of the second image acquisition equipment which stops the video call carried in the first stop instruction, stopping receiving the second RTSP data stream acquired by the second image acquisition equipment;

a processing unit 1302, configured to obtain, according to the first stop instruction, a corresponding first video frame from a cache space in which a first RTSP data stream is stored, where the first RTSP data stream is replaced by a second RTSP data stream, and splice and display the first video frame to a display position of a video call display window corresponding to the second RTSP data stream.

In some embodiments, the receiving unit 1301 is further configured to receive a video call instruction before receiving the first stop instruction, where the video call instruction carries information of a second image capturing device to be connected to a video call, and receive a second RTSP data stream sent by the second image capturing device;

the processing unit 1302 is further configured to determine a first RTSP data stream replaced by the second RTSP data stream, cache the first RTSP data stream and the second RTSP data stream in a corresponding cache space, obtain a corresponding second video frame from the cache space corresponding to the second RTSP data stream, and splice and display the second video frame to a display position corresponding to the first RTSP data stream in a video call display window.

In some embodiments, the processing unit 1302 is specifically configured to determine that the second RTSP data stream replaces a preset first RTSP data stream when the video call instruction is an access instruction and the number of RTSP data streams of a currently accessed video call reaches a preset number threshold.

In some embodiments, the processing unit 1302 is specifically configured to, if the video call instruction is a replacement instruction, the replacement instruction carries information for replacing the first image capturing device with the second image capturing device, and use an RTSP data stream acquired by the first image capturing device as a replaced first RTSP data stream.

In some embodiments, the processing unit 1302 is further configured to, after determining the first RTSP data stream replaced by the second RTSP data stream according to the video call instruction, replace a display plug-in used for displaying a video frame of the first RTSP data stream in a Gstreamer pipeline receiving the first RTSP data stream with a temporary consumption plug-in.

In some embodiments, the processing unit 1302 is further configured to set the state of the Gstreamer pipe to a suspended state before replacing the display plug-in with the temporary consumed plug-in;

and after replacing the display plug-in with the temporary consumption plug-in, setting the state of the Gstreamer pipeline as an operation state.

In some embodiments, the processing unit 1302 is further configured to replace a temporary consumed plug-in the Gstreamer pipe receiving the first RTSP data stream with a display plug-in before retrieving the corresponding first video frame from the buffer space storing the first RTSP data stream.

In some embodiments, the processing unit 1302 is further configured to set the state of the Gstreamer pipe to a suspended state before replacing the temporary consumed plug-in with the display plug-in;

and after replacing the temporary consumption plug-in with the display plug-in, setting the state of the Gstreamer pipeline as an operating state.

For the concepts, explanations, detailed descriptions and other steps related to the video call device in the present application, please refer to the descriptions of the foregoing methods or other embodiments, which are not repeated herein.

Fig. 14 is a schematic structural diagram of an electronic device according to some embodiments of the present application, and on the basis of the foregoing embodiments, the present application further provides an electronic device, as shown in fig. 14, including: the processor 1401, the communication interface 1402, the memory 1403 and the communication bus 1404, wherein the processor 1401, the communication interface 1402 and the memory 1403 are communicated with each other through the communication bus 1404;

the memory 1403 has stored therein a computer program which, when executed by the processor 1401, causes the processor 1401 to perform the steps of the above method in which the electronic device performs the respective functions.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface 1402 is used for communication between the above-described electronic apparatus and other apparatuses.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

On the basis of the foregoing embodiments, the present application provides a computer-readable storage medium, in which a computer program executable by an electronic device is stored, and computer-executable instructions are used for causing a computer to execute the procedures performed by the foregoing method parts.

The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memory such as floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc., optical memory such as CDs, DVDs, BDs, HVDs, etc., and semiconductor memory such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND FLASH), Solid State Disks (SSDs), etc.

Fig. 15 is a schematic structural diagram of a video call system according to some embodiments of the present application, and based on the foregoing embodiments, the present application provides a video call system, including: the video call device applied to the electronic device 1500 and the smart television 1600 for splicing the electronic device 1500 to the video frame of the video call display window to be displayed are described in any of the above embodiments.

The electronic device 1500 is configured to receive a first stop instruction, and stop receiving a second RTSP data stream acquired by a second image acquisition device according to information of the second image acquisition device that stops performing a video call, where the information is carried in the first stop instruction; and acquiring a corresponding first video frame from a cache space for storing a first RTSP data stream according to the first stop instruction, wherein the first RTSP data stream is a data stream replaced by a second RTSP data stream, and splicing and displaying the first video frame to a display position corresponding to the second RTSP data stream in a video call display window of the smart television 1600.

For specific functions of the electronic device, reference is made to the above description, and details are not repeated here.

The smart television 1600 is a smart television that can splice the electronic device 1500 to a video frame of a video call display window for display in the prior art, and details are not repeated here.

In the application, after the first RTSP data stream is replaced by the second RTSP data stream, when the electronic device receives a first stop instruction for stopping displaying the second RTSP data stream, because the buffer space of the first RTSP data stream still buffers the first RTSP data stream, the electronic device can directly obtain the first video frame corresponding to the first RTSP data stream from the buffer space, and perform splicing display, so that the first RTSP data stream can be quickly accessed into a video call again, and user experience is improved.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.