阅读说明：本技术 三维直播影像的横向纵向控制装置 (Transverse and longitudinal control device for three-dimensional live broadcast image ) 是由 施清德 于 2020-09-16 设计创作，主要内容包括：本发明公开一种三维直播影像的横向纵向控制装置,其包括一方向检测器与一控制接口,方向检测器用于接收一方向传感器的信号,并判断是否自动执行三维直播影像的显示转换,控制接口用于提供一用户手动切换三维直播影像的显示转换的接口。三维直播影像的显示转换为屏幕显示的长宽比的调整以及软件按键与功能显示位置的调整。(The invention discloses a transverse and longitudinal control device of a three-dimensional live video, which comprises a direction detector and a control interface, wherein the direction detector is used for receiving a signal of a direction sensor and judging whether to automatically execute display conversion of the three-dimensional live video, and the control interface is used for providing an interface for a user to manually switch the display conversion of the three-dimensional live video. And converting the display of the three-dimensional live video into the adjustment of the length-width ratio of the screen display and the adjustment of the software keys and the functional display position.)

1. A horizontal and vertical control device for three-dimensional live video comprises:

the direction detector is used for receiving a signal of a direction sensor and judging whether to automatically execute display conversion of the three-dimensional live video; and

the control interface is used for providing an interface for manually switching the display conversion of the three-dimensional live video;

and converting the display of the three-dimensional live video into the adjustment of the length-width ratio of screen display and the adjustment of software keys and functional display positions.

2. The control device of claim 1, wherein the orientation sensor is a gyroscope internal to a mobile device.

3. The control device of claim 1, wherein the control interface is a button disposed on a mobile device.

4. The control device of claim 1, wherein the control interface is a software-enabled interface installed on a mobile device.

5. The control device as claimed in claim 1, further comprising a live video adjuster connected to the direction detector, wherein when the direction detector detects a display change of the three-dimensional live video of a mobile device, the live video adjuster determines whether the mobile device supports steered stereoscopic live video display.

6. The control device of claim 5, wherein the live video adjuster changes the mobile device to a two-dimensional live display if the mobile device does not branch to a steered stereoscopic live display.

Technical Field

The invention relates to a transverse and longitudinal control device for three-dimensional live video, in particular to a transverse and longitudinal control device for three-dimensional live video.

Background

Since the invention of real-time communication software, through many years of development and technological progress, from the beginning of pure text two-party communication and chat, the communication and chat mode of video and characters has been entered now, and gradually becomes a part of modern people's life. After the expression modes of expressions, self-timer and filters of the real-time communication are evolved into the social trend of 'all pictures and videos', a new field of self-expression of live broadcast is emerging.

The shopping mode of the network live broadcast is characterized in that live broadcast can broadcast product introduction contents to the network through a short time gap of a background, so that a user terminal or a viewer can see the product introduction contents in real time and can interact with a main broadcast in real time. As shown in fig. 1, it shows a schematic diagram of directly broadcasting and selling goods to a user terminal 11 through a software (APP) of a mobile device by an anchor 12 on the internet, and this transaction and communication mode is different from the traditional tv shopping and internet shop shopping, and tv shopping adopts a mode of prerecording and broadcasting again, so that shopping is not real-time; the traditional online store adopts a hanging network for placing product introduction pictures on the network and then completes product transaction by matching with real-time service of an online store operator.

The direct broadcast visibility and real-time interaction mode is gradually pursued by modern people, so that when the mode is applied to the online shopping mode, the time consumed by people for purchasing articles outside can be greatly saved, the entertainment interest is full in the shopping process, and the real-time property of shopping is also improved. However, in the real-time shopping mode through the network, although a host or a professional introduces a product on the live broadcasting site, a shopper watches a 2D product image on a display screen in a remote indoor environment or in a mobile situation to determine whether to shop, because the shopper does not observe the product on the live broadcasting site, the 2D product introduction image seen by a mobile phone or an intelligent television of a user terminal generates an error with an actual article, and when the consumer receives the physical product, the shopper finds a difference with psychological expectation.

The live broadcast chat, entertainment and shopping modes are popular and popular with people, besides the visibility of real-time and vivid pictures and the driving of the atmosphere of the anchor, an observer can interact with the anchor or leave a message, and the character interaction results of the observer and the anchor can be directly displayed on a display screen. Therefore, the range of the platform applicable to live broadcast is wider and more direct compared with the prior various real-time communication software, television shopping and plane type network shopping, and the boundary between the prior platforms is broken.

However, the current live platform 10, as shown in fig. 1, still relies on conventional two-dimensional (2D) video and image display technology, and therefore, the viewer or shopping user's understanding of the anchor 12 and the live shopping still differs greatly from the real world. For example, when a live broadcasting party or a selling party is performing an explanation scene of cosmetics, as shown in fig. 2, since the image 20 broadcasted on the conventional 2D video does not have deep stereoscopic information, the selling party has painted the color and shape of the cosmetics on the live broadcasting site, and a remote viewer has difficulty in feeling bright stereoscopic color and luster on the live broadcasting site due to the limitation of image formation, which greatly reduces the display effect of the product. This is because such products do not show the complete features of the products when displayed without the three-dimensional stereoscopic light and three-dimensional image having depth information. Similar to such application examples, many limitations have been imposed on the current two-dimensional image display.

Therefore, how to solve the flat display limitation of the live broadcast platform and improve the communication efficiency through the improvement of the design has become one of the important issues to be solved by the business.

Disclosure of Invention

The technical problem to be solved by the present invention is to provide a horizontal and vertical control device for three-dimensional live broadcast images, which allows a handheld device to manually or automatically adapt to a horizontal or vertical three-dimensional live broadcast display mode through the horizontal and vertical control device.

In order to solve the above technical problems, one of the technical solutions adopted by the present invention provides a horizontal and vertical control device for three-dimensional live video, which includes a direction detector and a control interface, wherein the direction detector is configured to receive a signal from a direction sensor and determine whether to automatically perform display conversion of the three-dimensional live video, and the control interface is configured to provide an interface for a user to manually switch live display conversion of the three-dimensional live video. The live broadcast display of the three-dimensional live broadcast image is converted into the adjustment of the length-width ratio of the screen display and the adjustment of the software keys and the functional display position.

The transverse and longitudinal control device for the three-dimensional live video has the advantages that in order to reduce real-time flow of data, a method of mixing and editing two-dimensional images and three-dimensional images is adopted, so that a viewer can place a focus on a product or a person needing attention to view the best and most needed three-dimensional image, and the mode of mixing the two-dimensional images and the three-dimensional images can select and set two-dimensional image viewing or three-dimensional image viewing on a user interface, namely, the user can change the position of the three-dimensional image needing to be viewed at any time, and can be set in a regional mode or a full-image three-dimensional display mode. The two-dimensional image can be a left image or a right image in the three-dimensional left-right type image as a display image of the two-dimensional image. In addition, besides the stereoscopic display mode of the user terminal device, the control end of the anchor end or the cloud background can also be provided with stereoscopic display equipment so as to provide stereoscopic display preview of the anchor end and interactive stereoscopic display with the user terminal. In addition, the user terminal or the main broadcasting terminal can select the horizontal or vertical mode through the horizontal and vertical control device of the three-dimensional live video to carry out the three-dimensional live broadcasting display.

For a better understanding of the features and technical content of the present invention, reference should be made to the following detailed description of the invention and accompanying drawings, which are provided for purposes of illustration and description only and are not intended to limit the invention.

Drawings

Fig. 1 is a schematic diagram of a conventional live broadcast platform.

Fig. 2 is a schematic diagram of an existing live software implementation.

Fig. 3 is a flowchart illustrating steps of a live broadcast method with a real-time three-dimensional video display function according to the present invention.

Fig. 4A is a schematic diagram of image adjustment of a three-dimensional image according to the present invention.

FIG. 4B is a diagram illustrating an artificial intelligence learning module learning usage habits of a user terminal according to the present invention.

Fig. 5 is a system block diagram of a live broadcast system with real-time three-dimensional video display function according to the present invention.

Fig. 6 is a schematic diagram of image processing of the three-dimensional live broadcast operation module according to the present invention.

Fig. 7A is a schematic diagram of displaying a three-dimensional image by using the live broadcasting system of the present invention.

Fig. 7B is a schematic diagram of interaction between a main broadcast and a user terminal by applying the live broadcast system of the present invention.

Fig. 8 is a block diagram of a three-dimensional live video control device according to the present invention.

Fig. 9 is a flowchart of a determination step of two-dimensional live video and three-dimensional live video conversion of the three-dimensional live video control device according to the present invention.

[ notation ] to show

10: live broadcast platform

11: user terminal

12: anchor (R)

APP: software

20: image forming method

S301-S305: step (ii) of

40: three-dimensional image

50: live broadcast system

51: image acquisition module

52: three-dimensional live broadcast calculation module

521: image correction unit

522: image separating unit

523: image synthesizing unit

524: image correction unit

525: image allocation unit

526: image management unit

527: video coding unit

53: live broadcast system server

54: artificial intelligence learning module

55: three-dimensional decoder

56: three-dimensional display

71: anchor (R)

72: mobile device

73: gift

74: display screen

75: user terminal

80: transverse and longitudinal control device for three-dimensional live broadcast image

81: direction detector

82: control interface

83: live video adjuster

S901-S905: step (ii) of

Detailed Description

The following is a description of the embodiments of the present disclosure relating to a "horizontal and vertical control device for three-dimensional live video" with specific embodiments, and those skilled in the art can understand the advantages and effects of the present disclosure from the disclosure of the present disclosure. The invention is capable of other and different embodiments and its several details are capable of modifications and various changes in detail, all without departing from the spirit and scope of the present invention. The drawings of the present invention are for illustrative purposes only and are not intended to be drawn to scale. The following embodiments will further explain the related art of the present invention in detail, but the disclosure is not intended to limit the scope of the present invention. In addition, the term "or" as used herein should be taken to include any one or combination of more of the associated listed items as the case may be.

[ example of the inventive direct broadcast method ]

Fig. 3 is a flowchart showing steps of a live broadcast method for real-time three-dimensional image display according to the present invention, fig. 4A is a schematic diagram showing image adjustment of a three-dimensional image according to the present invention, fig. 4B is a schematic diagram showing an artificial intelligence learning module learning usage habit of a user terminal according to the present invention, and fig. 5 is a system block diagram showing a live broadcast system with a real-time three-dimensional image display function according to the present invention.

As shown in fig. 3 and with reference to the component labels of fig. 5, a live broadcast method for real-time three-dimensional video display according to an embodiment of the present invention includes the following steps. In step S301, an entity is captured, a three-dimensional image of the entity is obtained, and an image obtaining module 51 is used to photograph the anchor, where the image obtaining module 51 may be a three-dimensional camera or a two-dimensional camera, and any electronic device capable of photographing may be the image obtaining module 51 of the present invention.

In the preferred embodiment of the present invention, the image capturing module 51 is a three-dimensional camera, and to solve the problem of the display information of the three-dimensional image, in the preferred embodiment of the present invention, the capturing and the compiling of the three-dimensional image are utilized, and the three-dimensional image capturing is to capture the three-dimensional image by using a real-time embedded three-dimensional dual-lens camera. Alternatively, in different embodiments, a two-dimensional camera may be applied to obtain a two-dimensional image of a main broadcast, and then convert the two-dimensional image into a three-dimensional image by using a two-dimensional to three-dimensional image conversion technique, and the technique for how to obtain a three-dimensional image or how to convert a two-dimensional image into a three-dimensional image is well known to those skilled in the art and will not be described herein again. In order to reduce the real-time flow of data, a method of coexisting two-dimensional images and three-dimensional images and performing mixed editing is adopted, so that a user terminal can place a focus on a product or a person needing attention when watching so as to watch the optimal and most needed three-dimensional images, and the mode of coexisting the two-dimensional images and the three-dimensional images can be used for selectively setting two-dimensional image watching or three-dimensional image watching on a user interface, namely, the user terminal can change the position of the three-dimensional images needing to be watched at any time and can be set in a regional mode or a full-image three-dimensional display mode. In addition, it should be noted that, the data amount required to be transmitted by the three-dimensional video is larger than the data amount required to be transmitted by the two-dimensional video, and the invention can transmit a smaller data amount during video playback than during full-view three-dimensional display by including the three-dimensional image and the two-dimensional image in the video at the same time, so that the smoothness of playback during live broadcast can be maintained and the occurrence of data transmission delay (lag) can be reduced.

In step S302, a three-dimensional live broadcast operation module 52 performs image processing on the three-dimensional video. After the image obtaining module 51 of the present invention receives the three-dimensional image 40, image processing is performed, because the mobile devices used by the user terminals are different, and the locations or environments of the user terminals are different, different mobile devices have different data transmission speeds, and the locations are different, and the data transmission speeds thereof are also different, so that in the step of optimizing the three-dimensional image, the three-dimensional image is cut and adjusted, and the three-dimensional image is optimized through the artificial intelligence learning module 54 according to the video allocation combination.

The live broadcast method with the real-time three-dimensional image display function can adjust images according to different data transmission speeds. In the existing stereoscopic vision technology, an entire image is processed, so that the data volume at least twice that of a two-dimensional image is often generated, such data transmission volume creates a considerable obstacle for a live broadcast system of real-time transmission, and the most frequently encountered situation is frame dropping (frame) or serious delay of the image, so that the use interest and patience are reduced. Therefore, in the present invention, the image is automatically detected, and the foreground, background and image segmentation are adapted, as shown in fig. 4A, the three-dimensional image 40 is further subjected to image fault tolerance and correction, and a plurality of objects in the three-dimensional image 40 are detected and separated. If the object is a person, image processing such as image softening and filtering, background blurring, filter lens effect or skin and muscle beautifying is carried out on the object; if the object is an object, performing image processing such as background stripping, edge enhancement, contrast enhancement or image amplification on the object; if the object is the same as the human object, the color, beauty and figure edge optimization calculation is carried out on the image. Background stripping is to separate the background in the three-dimensional image from the image, so that different backgrounds can be replaced in the subsequent steps according to the use habits of the user terminal. Finally, the optimized three-dimensional image 40 after image processing is output.

In addition, the live broadcasting method of the invention further comprises the step of taking images of at least one user terminal to obtain a three-dimensional image of the user terminal, so as to execute a three-dimensional display mode of multi-person interaction. The live broadcasting method of the invention can not only display three-dimensionally, but also use a one-to-many display mode of the anchor broadcast, besides a general one-to-one mode, a three-dimensional display mode of synchronous interaction of a plurality of people can be used, therefore, the live broadcasting method of the invention not only can provide intuition, but also can improve entertainment fun, and can improve pleasure of human life besides avoiding wrong shopping areas.

Through the image processing of the three-dimensional image, the user terminal can see the target display area at a glance, and the data loading capacity of the network bandwidth can be effectively reduced. In the present invention, the three-dimensional image can be processed by a three-dimensional live-broadcasting calculation module 52. The reason why the data load of the network bandwidth can be effectively reduced by the image processing of the three-dimensional image is that the local three-dimensional image data is much smaller than the traditional full-image three-dimensional data, and the data compression technology only has a plurality of bytes (bytes) to a plurality of kilobytes of data than the data of a general full-image two-dimensional image.

In step S303, a video allocation combination is generated by an artificial intelligence learning module 54 according to at least one video selection condition of a user terminal. The at least one video selection condition of the user terminal is the use habit of the user terminal to a live broadcast system, the use habit of each user terminal is different, and the live broadcast method displays the favorite play mode of the user terminal according to the use habit of each user terminal when the user terminal uses the live broadcast software of the invention through an artificial intelligent learning module 54. The habit of the user terminal is recorded by the learning mode of the artificial intelligence learning module 54, and the area of the habit of the user terminal can be automatically displayed or the full-image stereoscopic picture can be displayed when the same or similar picture is opened by the user terminal next time.

Further, each user terminal has a personal usage habit, so that when the user terminal watches the live broadcast, the artificial intelligence learning module 54 of the present invention records the usage habit of each user terminal when watching the live broadcast. For example, as shown in fig. 4B, when the user terminal is watching a live broadcast, the artificial intelligence learning module 54 records the usage pattern of the user terminal, for example, which gift the user terminal likes to send, and the user terminal switches between two-dimensional image and three-dimensional image for that particular person, object or person and object. The live broadcasting method of the invention records the use habit of each user terminal when watching live broadcasting, and the artificial intelligent learning module 54 displays a specific live broadcasting picture to the user terminal according to the recorded use habit when the user terminal watches the live broadcasting next time, so that the user terminal has better live broadcasting watching use experience.

In addition, in step S304, the three-dimensional video is optimized according to the video deployment combination and the usage environment of the live system. The optimization of the three-dimensional video may be based on the usage environment of the live broadcast system, such as the number of viewers and the network speed. In addition, in the live broadcasting method, when people or products are shot, the people or products can be shot in real time, and the people or products can also be added in real time through the pre-recorded background, so that the use scene of virtual reality is increased, the user terminal can conveniently change the use scene, and the use pleasure of the environment is improved. In the live broadcasting method, different video allocation combinations are provided for each user terminal to generate an optimized three-dimensional image, and different background patterns can be selected according to different user terminals. In addition, in order to avoid the wrong or inappropriate video or video stream from being played to the terminal, no matter whether the user terminal chats or communicates with the main broadcast, or enjoys the program performance of the main broadcast, or watches the products sold by the main broadcast, after the video is taken into the human body or the object in real time by the camera, except for real-time three-dimensional editing and compression of the video, the video is automatically checked through artificial intelligence or manual check, after the check is finished, the video is correctly corresponding to the background system according to the setting or the requirement of the user terminal, and then the corresponding code stream (Data Streaming) is sent to the mobile phone or the terminal interaction device of the user terminal, and the user can see the display result corresponding to the setting and the requirement of the terminal.

In step S305, the optimized three-dimensional image is displayed in the user terminal in accordance with at least one video selection condition of the user terminal. The live broadcasting method of the invention not only provides the cutting and display setting of the foreground and the background required by the user terminal at the background end according to the setting of the user terminal, but also sends the corresponding three-dimensional image to the device of the user terminal according to the setting and the requirement of the user terminal. For example, when the user terminal purchases a gift, the live broadcast system sends a stereoscopic display image or video of the gift to the device of the anchor according to the corresponding gift purchased by the user terminal, so that the anchor can view the video or image of the corresponding stereoscopic gift on the display screen of the device of the anchor.

According to the live broadcast method with the real-time three-dimensional image display function, the output quality of the image can be adjusted according to different data transmission speeds, different experience feelings can be provided according to different user terminals, and each user terminal can obtain better watching experience or shopping experience.

[ embodiment of live broadcast System of the present invention ]

Fig. 5 is a system block diagram of a live broadcast system for real-time three-dimensional video display according to the present invention. As shown in fig. 5, the live broadcasting system 50 for real-time three-dimensional image display according to the present invention includes an image acquisition module 51, a three-dimensional live broadcasting calculation module 52, a live broadcasting system server 53, an artificial intelligence learning module 54, a three-dimensional decoder 55 and a three-dimensional display 56.

The image capturing module 51 may be a three-dimensional camera, a two-dimensional camera, or the like, and any electronic device capable of photographing may be the image capturing module 51 of the present invention. In the preferred embodiment of the present invention, the image capturing module 51 is a three-dimensional camera, which is to solve the problem of insufficient display information of three-dimensional images, in the preferred embodiment of the present invention, the three-dimensional image is captured and compiled, and the three-dimensional image is captured by a real-time embedded three-dimensional dual-lens camera, such a camera, embedded in a device used by a host or in a device of a user terminal, such as a smart phone, a tablet computer, etc., is embedded, therefore, the cameras are built in the electronic device, and the two three-dimensional cameras are connected with the electronic device through interfaces, such as Mobile Industry Processor Interface (MIPI), serial Interface, and the like, and is directly connected with a handheld device end, and then is connected with a live broadcast system server 53 through a three-dimensional live broadcast calculation module 52 of the invention, so as to send three-dimensional images and videos to a user terminal in real time. For a personal computer end or a handheld device, if an embedded three-dimensional camera is not installed, an external monocular camera or a binocular stereo camera can be connected with a host through a Universal Serial Bus (USB) interface or a Wireless (WIFI) mode, and after the direct broadcasting system is connected, data processing is carried out according to a processing mode or flow of converting two-dimensional to three-dimensional or stereo cameras.

Alternatively, in different embodiments, a two-dimensional camera may be applied to obtain a two-dimensional image of a main broadcast, and then convert the two-dimensional image into a three-dimensional image by using a two-dimensional to three-dimensional image conversion technique, and the technique for how to obtain a three-dimensional image or how to convert a two-dimensional image into a three-dimensional image is well known to those skilled in the art and will not be described herein again. The image obtaining module 51 may be installed on the devices of the anchor and the user terminal, and may obtain the three-dimensional images of the anchor and the user terminal at the same time, so as to facilitate the subsequent one-to-one mode or one-to-many stereoscopic display mode of the anchor and the at least one user terminal.

The three-dimensional live broadcast operation module 52 is connected to the image acquisition module 51, and is configured to receive the three-dimensional image and optimize the three-dimensional image, where the three-dimensional live broadcast operation module 52 may be set in live broadcast software of a main broadcast or a user terminal, or the three-dimensional live broadcast operation module 52 may also be installed in the live broadcast system server 53, which is not limited herein. Further, as shown in fig. 6 and referring to fig. 5, the three-dimensional live broadcasting operation module 52 includes an image correction unit 521, an image separation unit 522, an image synthesis unit 523, an image modification unit 524, an image allocation unit 525, an image management unit 526 and a video coding unit 527. The image correction unit 521 is connected to the image acquisition module 51, receives the three-dimensional image, and is configured to correct the received three-dimensional image, so that a small number of image errors or video and audio asynchronism may occur during the process of two-dimensional to three-dimensional conversion of the acquired three-dimensional image or the image, and errors generated during the acquisition or conversion of the three-dimensional image are corrected by the image correction unit 521. The image separating unit 522 is connected to the image correcting unit 521, and divides the foreground, background, person or object in the corrected three-dimensional image, so as to perform specific image processing on any person or object in the three-dimensional image, for example, the background in the three-dimensional image is cut, so that a specific background pattern can be added to the background in the subsequent image processing, or the specific object or image is three-dimensionally displayed in a specific live broadcast operation.

The image synthesizing unit 523 is connected to the image separating unit 522, the separated three-dimensional image may be subjected to image processing on a specific pattern, a person or an object, respectively, the processed specific pattern, person or object may be combined by the image synthesizing unit 523, the image synthesizing unit 523 may combine an individual three-dimensional pattern, person or object with another two-dimensional image, instead of reducing the original three-dimensional image, or may add a specific background pattern to the background by the image synthesizing unit 523. The image correction unit 524 is connected to the image synthesis unit 525, and the image correction unit 524 may select and adjust the viewing angle of the image according to the image capture of different patterns, or the image correction unit 524 may perform an image special effect rendering effect on the gift image in the three-dimensional image, and the image correction unit 524 also includes functions of general image edge enhancement, image turning, image background blurring, filter effect, or image area enlargement.

The image allocating unit 525 is connected to the image modifying unit 524, and the image allocating unit 525 adjusts the image output of the three-dimensional image according to different network speeds or the performance of the mobile device. For example, when the network speed is fast, the three-dimensional image can be completely output, when the network speed is slow, the three-dimensional image can be partially displayed, and the rest of the images are displayed in two dimensions. The video management unit 526 is connected to the video allocation unit 525, integrates and outputs the three-dimensional video after image processing, and the video encoding unit 527 is connected to the video management unit 526, converts the three-dimensional video into a data signal through the video encoding unit 527, and further can transmit the data signal with the three-dimensional video to the live broadcast system server 53 in a wired or wireless manner.

The live broadcast system server 53 is wirelessly connected to a three-dimensional live broadcast calculation module 52, which may also be referred to as a live broadcast system cloud, and the live broadcast system server 53 is configured to receive a data signal with a three-dimensional image, in other words, live broadcast content of each anchor broadcast is transmitted to the live broadcast system server 53 and then is pushed to a device of the user terminal through the live broadcast system server 53. The artificial intelligence learning module 54 is connected to the live broadcast system server 53, in a preferred embodiment of the present invention, the artificial intelligence learning module 54 may be disposed on the live broadcast system server 53, or in a different embodiment, the artificial intelligence learning module 54 may be disposed on a different server or a different computer host, and then connected to the live broadcast system server 53 in a wireless or wired connection manner, which is not limited herein. The screen selection condition or usage habit of each user terminal is recorded by the learning mode of the artificial intelligence learning module 54, and a video deployment combination is output. Next time the user terminal turns on the same or similar picture, the artificial intelligence learning module 54 can automatically output the video allocation combination according to the video selection condition, in other words, display the display area to which the user terminal is accustomed or display the stereoscopic picture of the full picture.

Therefore, no matter the user terminal chats or communicates with the anchor, or enjoys the program performance of the anchor, or watches the products sold by the anchor, after the images are taken into human body or object by the camera in real time, except for real-time three-dimensional editing and compression of the images, the images are automatically checked by the artificial intelligent learning module 54 in the background of the live broadcasting system server 53, and after the check is completed, the corresponding code stream (Data) is sent to the mobile phone or the terminal interaction device of the user terminal according to the setting or the requirement of the user terminal and after the correct correspondence of the background system, the user terminal can see the display result set by the terminal and corresponding to the requirement.

The three-dimensional decoder 55 is connected to the live broadcast system server 53, or the three-dimensional decoder 55 may be installed on the live broadcast system server 53 and connected to the artificial intelligence learning module 54, and the three-dimensional video can be encoded and decoded by the three-dimensional decoder 55. The live broadcasting system 50 displays the stereoscopic images or videos of the gifts 73 on the mobile device 72 of the anchor 71 according to the corresponding gifts purchased by the user terminals, as shown in fig. 7A, so that the anchor 71 can view the videos or images of the corresponding stereoscopic gifts 73 on the display screen 74 of the mobile device 72. The three-dimensional display 56 is provided on the devices of the anchor and the user terminal, and the anchor and the user terminal receive the three-dimensional video pushed from the live system server 53 in a wireless transmission manner and display the three-dimensional video on the devices of the anchor and the user terminal through the three-dimensional display 56. As shown in fig. 7B, on the mobile device 72, in addition to that the anchor 71 can send stereoscopic videos and images to the user terminal 75 to receive and watch the stereoscopic videos and images, the user terminal 75 can also send stereoscopic images or videos to the anchor 71 through the bidirectional stereoscopic display technology, so that the anchor 71 can also receive stereoscopic image information of the user terminal 75 in real time, and the requirement of the user terminal 75 can be quickly judged, thereby achieving the purpose of real-time interaction.

In addition, referring to fig. 6, the user terminal also includes a function of displaying a three-dimensional video at the anchor terminal, and the device of the user terminal also includes a function of the live broadcasting system of the present invention, because the user terminal can also live a video of itself to the anchor terminal for viewing when viewing the live broadcast, or the user terminal can also automatically or manually switch between two-dimensional and three-dimensional when receiving the video at the anchor terminal. Therefore, when the user terminal receives the two-dimensional or three-dimensional image, if the received two-dimensional image can be converted into a three-dimensional image, the three-dimensional image of the user terminal is also subjected to image processing by the components such as the image correction unit 521, the image separation unit 522, the image synthesis unit 523, the image modification unit 524, the image allocation unit 525, the image management unit 526 and the like, so that the three-dimensional image of the user terminal can be displayed as a panoramic three-dimensional image, or can be displayed as a three-dimensional image for a specific person, object or person and object, and the display of the two-dimensional or three-dimensional image on the client terminal can be actively or manually switched. How to process the image through the three-dimensional live broadcast operation module 52 is described in the previous paragraphs, and therefore, details of the image processing will not be described herein.

For example, when the data transmission speed of the user terminal is not fast, the three-dimensional live broadcast calculation module 52 may automatically display only a portion of the three-dimensional image on the three-dimensional display 56 of the user terminal, but not display the panoramic three-dimensional image, or, when the network speed is too low, the three-dimensional live broadcast calculation module 52 may automatically switch the three-dimensional image to the two-dimensional image. In addition, the three-dimensional images of the user terminal can be switched automatically or manually, and the user terminal can manually designate a specific person, object or person and object to display the three-dimensional images. In other words, the three-dimensional image of the present invention is not limited to the full-image three-dimensional image, and the three-dimensional image of the present invention may be a partial three-dimensional image in which a specific object (person, object, or person and object) is displayed in a three-dimensional manner and the remaining images are displayed in a two-dimensional manner. By the image presentation mode, the transmission quantity of data during live broadcasting can be reduced, and the probability of generation of live broadcasting display delay can be reduced.

[ horizontal or vertical display embodiment of three-dimensional live video of the invention ]

In addition, in the preferred embodiment of the present invention, a horizontal and vertical control device 80 for three-dimensional live video can be further provided on the devices of the anchor and the user terminal. Because the user terminal may have a habit of viewing by using vertical display or horizontal live broadcast display, the invention also provides a horizontal and vertical control device 80 for three-dimensional live broadcast images, and for the setting of the display direction of the mobile device, the setting of the live broadcast display direction can be automatically detected or manually set on APP software of the user terminal, and the device is also suitable for monitoring and displaying of a display terminal and a cloud terminal of a main broadcast. Therefore, the apparatus 80 for controlling the horizontal and vertical directions of a three-dimensional live video of the present invention comprises a direction detector 81 and a control interface 82.

The direction detector 81 is capable of sensing a signal of a gyroscope or a direction sensor inside the mobile device or the handheld device in the direction of automatically detecting the live display, and the direction of the live display is adjusted by the device of the anchor or the user terminal according to the direction signal fed back by the terminal device. The adjustment of the direction can be matched with the steering of the three-dimensional live video or video to perform display conversion of the three-dimensional live video, and the display conversion comprises adjustment of length, width ratio or interlaced ratio (Interlace) of screen display, adjustment of software keys and functional display positions, and the like of steering setting related to 3D live video. The aforementioned settings also include live display function settings for the three-dimensional display 56. The control interface 82 is provided on the mobile device or handheld device of the anchor and the user terminal, and further, the control interface 82 may be a controller such as a button, etc., or the control interface 82 may also be a software interface including functions of the controller. The anchor or the user terminal can automatically turn over the screen for live display through the direction detector 81, or can automatically turn over the screen for live display through the control interface 82, so that the vertical live display viewing or the horizontal live display viewing is realized.

The horizontal and vertical control device 80 for three-dimensional live video of the present invention has a function of displaying three-dimensional live video in both horizontal and vertical directions, and has automatic or manual steering detection and display adjustment. For a stereoscopic display terminal only supporting one direction, the invention further comprises a live video adjuster 83 in the horizontal and vertical control device 80 of the three-dimensional live video, the live video adjuster 83 is connected with the direction detector 81, the step of the live video adjuster 83 judging whether to adjust the three-dimensional live video can be as shown in fig. 9, in step S901, the direction detector 81 detects the change of the setting direction of the mobile device, or, in step S902, the user rotates the live video through the control interface 82, in step S903, the live video adjuster 83 judges whether the mobile device supports the three-dimensional live video horizontally or vertically, if not, in step S904, the live video adjuster 83 can remind the user terminal to inform that the steered stereoscopic live video is not supported, and the stereoscopic live video is displayed on the display of the user terminal in a two-dimensional plane display mode through the live video adjuster 83, and the user terminal can see the stereoscopic live broadcast display until the user terminal turns to display the stereoscopic live broadcast display again. On the contrary, in step S905, the live video adjuster 83 adjusts the optimized three-dimensional live video after the turning.

[ advantageous effects of the embodiments ]

The transverse and longitudinal control device for the three-dimensional live video has the advantages that in order to reduce real-time flow of data, a method of mixing and editing two-dimensional images and three-dimensional images is adopted, so that a viewer can place a focus on a product or a person needing attention to view the best and most needed three-dimensional image, and the mode of mixing the two-dimensional images and the three-dimensional images can select and set two-dimensional image viewing or three-dimensional image viewing on a user interface, namely, the user can change the position of the three-dimensional image needing to be viewed at any time, and can be set in a regional mode or a full-image three-dimensional display mode. The two-dimensional image can be a left image or a right image in the three-dimensional left-right type image as a display image of the two-dimensional image. In addition, besides the stereoscopic display mode of the user terminal device, the control end of the anchor end or the cloud background can also be provided with stereoscopic display equipment so as to provide stereoscopic display preview of the anchor end and interactive stereoscopic display with the user terminal. In addition, the user terminal or the main broadcasting terminal can easily select the horizontal or vertical mode to carry out the three-dimensional live broadcasting display through the horizontal and vertical control device of the three-dimensional live broadcasting image.

The disclosure is only a preferred embodiment of the invention and should not be taken as limiting the scope of the invention, which is defined by the appended claims.