阅读说明：本技术 一种全景原始画面视频分块传输方法 (Method for transmitting panoramic original picture video in blocks ) 是由 胡志发 于 2020-11-23 设计创作，主要内容包括：本发明公开了一种全景原始画面视频分块传输方法,包括：获取使用者在全景场景中视角位置和视角范围,用户视角信息计算转换原始视角区域传递给全景摄像机,截取对应区域图像进行传输,继续第一步实现使用者实时观看交互。本发明通过在全景画面中按照实际的观看需求对全景画面进行分块区域传输,在使用者进行切换视角时,通过快速计算区域范围进行传输,实现用户无感的对全景任意区域进行观看,并且可以达到在相同的带宽条件下更高清全景画面效果的实时传输,用户体验更好。(The invention discloses a method for transmitting a panoramic original picture video block, which comprises the following steps: and acquiring the visual angle position and the visual angle range of a user in the panoramic scene, calculating and converting the visual angle information of the user into an original visual angle area, transmitting the original visual angle area to the panoramic camera, intercepting the image of the corresponding area for transmission, and continuing the first step to realize real-time watching interaction of the user. According to the invention, the block area transmission is carried out on the panoramic picture according to the actual watching requirement in the panoramic picture, and when the user switches the visual angle, the transmission is carried out through quickly calculating the area range, so that the user can realize the noninductive watching of any panoramic area, the real-time transmission of higher panoramic picture effect under the same bandwidth condition can be realized, and the user experience is better.)

1. A method for transmitting panoramic original picture video blocks is characterized by comprising the following steps:

s1, acquiring the view angle position and the view angle range of the user in the panoramic scene;

s2, calculating and converting the original view angle area by the user view angle information and transmitting the original view angle area to the panoramic camera;

s3, intercepting and transmitting the corresponding area image;

and S4, continuing the step S1, and realizing real-time viewing interaction of the user.

2. The panoramic original picture video blocking transmission method according to claim 1, characterized in that: step S1 acquires that the viewing angle position and the viewing angle range of the user in the panoramic scene are:

the original spherical picture of the panoramic scene is projected onto a screen to be displayed in a plane, when a user watches one part of the panoramic picture, the size of a window in the display process is controlled and determined by the user through a mouse or a touch screen, and the picture display content can be switched by dragging the picture;

and after the visual angle range is determined, the visual angle position is determined by the longitude and latitude corresponding to the center of the picture.

3. The method of claim 2, wherein the user-controlled determined range of viewing angles is determined by the aspect ratio of the display screen, the range of viewing angles being:

ω_H＝ω

wherein, W and H are the display resolution size of the screen effective window, and the unit is as follows: a pixel; omega is the calculation range of the view angle range of the current viewing picture, omega_H、ω_WIs the actual viewing angle area range.

4. The panorama original picture video block transmission method of claim 3, wherein: step S2, the user view angle information is calculated and converted into the original view angle area and transmitted to the panoramic camera, after the user controls and determines the view angle position and the view angle range, the picture area and the position watched by the screen are obtained at the display end, the area range watched by the current user is determined through the latitude and the longitude, and the determined latitude and longitude range is as follows:

obtaining an original visual angle area through the projection change relation between the panoramic spherical picture and a plane original picture projected onto a screen; in the process of calculating the region, the region blocks are used for dividing the region in the final original image, a modular layout is used for representing the calculation result, the modular layout divides the final original image into w × h grids, and each grid represents a small region of the original image; for the final region, determining whether the corresponding image region is a region needing transmission or not by marking each square value in the template domain, wherein 1 represents that transmission is needed, and 0 represents that transmission is not needed;

in the coordinate calculation process, for four vertices of longitude and latitude coordinates: (lon)_L,lat_U)，(lon_L,lat_D)，(lon_R,lat_U)，(lon_R,lat_D) And performing projection calculation to obtain an original image coordinate, performing equivalent calculation on the coordinate to obtain an approximate domain area. Firstly, the coordinates of the circle centers of the original double fisheye pictures are calculated as (x)_L,y_L),(x_R,y_R) The radius of the image circle is r, and the equivalent calculation mode of the template map coordinate is as follows:

when 0 ° < lon <180 °

x＝x_L-r*cos(lon)；

y＝y_L-r*sin(lat)；

When-180 ° < lon <0 ° ]

x＝x_R+r*cos(lon)；

y＝y_R-r*sin(lat)。

5. The panoramic original picture video block transmission method according to claim 4, characterized in that: in step S2, the template map is generated by calculating the original coordinates of the two fish eyes after obtaining the coordinates (lon, lat) of the corresponding fixed points, and by calculating (x'_L,y’_U)，(x’_R,y’_D) Then the corresponding template region is calculated as:

x_L＝min(x’_L,x’_R),x_R＝max(x’_L,x’_R),y_U＝min(y’_U,y’_D),y_D＝max(y’_U,y’_D)；

the seed template regions are: (x)_L,y_U),(x_R,y_U),(x_L,y_D),(x_R,y_D) A rectangular area is formed by the enclosing.

6. The panoramic original picture video block transmission method according to claim 4, characterized in that: in step S2, the template map is generated by calculating the original coordinates of the two fish eyes after obtaining the coordinates (lon, lat) of the corresponding fixed points, and by calculating (x'₁,y’₁)，(x’₂,y’₂)，(x’₃,y’₃)，(x’₄,y’₄) The corresponding template region is calculated as:

x_L＝min(x’₁,x’₂，x’₃,x’₄),x_R＝max(x’₁,x’₂，x’₃,x’₄)，y_u＝min(y’₁,y’₂，y’₃,y’₄)，y_D＝max(y’₁,y’₂，y’₃,y’₄)；

the seed template regions are: (x)_L,y_U),(x_R,y_U),(x_L,y_D),(x_R,y_D) A rectangular area is formed by the enclosing.

7. The panoramic original picture video block transmission method according to any one of claims 4 to 6, characterized in that: in step S3, after the expansion processing of the template layout, the region buffering is performed, which includes the following steps: when the visual angle of the user changes, the region change trend direction is adopted for caching, and the size of the region is changed according to the visual angle change speed, wherein the calculation mode is as follows:

for the current view angle range, for the range center coordinate P_v(lon, lat), calculating a continuous area of the view angle range according to the view angle movement change direction and speed;

lon_d＝lon+v*arc_x；

lat_d＝lat+v*arc_y；

P_d(lon_d，lat_d) And P_vThe continuous region between (lon, lat) is the region range needing transmission currently, and the view width is kept to be omega_H、ω_W。

Technical Field

The invention belongs to the technical field of real-time data transmission, particularly belongs to a technology applied to ultrahigh-definition panoramic video data transmission, and particularly relates to a panoramic original picture video blocking transmission method which is used for saving video data transmission data amount when a panoramic real-time video picture is watched in real time.

Background

Panoramic video is a new data form, and particularly, the panoramic data volume of each industry is also increased sharply along with the development of monitoring security and protection and VR industries in a panoramic 720 degree. Compared with the conventional common plane image video data, the panoramic image data has great difficulty in the data processing and data transmission part, and particularly, in order to obtain better user experience and obtain a panoramic image video effect with high image quality and low time delay, the realization of the panoramic image video effect in the conventional streaming media technology is a great challenge.

With conventional streaming media techniques, panoramic image video data can be encoded into a standard data stream format for normal data transmission. Transmitting such high resolution and high image quality data often requires high bandwidth network requirements and correspondingly causes data transmission network delay. For the streaming media server, the requirement pressure of bandwidth is increased by multiple times, and large-resolution high-definition image data is transmitted to multiple users according to the viewing requirements of different users. However, in the actual panoramic viewing process, the viewing angle of the user viewed in most scenes is relatively stable, and in order to meet the requirement of switching the panoramic viewing angle at a relatively low frequency, the real-time bandwidth is increased to a relatively large level, which causes the waste of the data bandwidth. There is therefore a need for a better transmission method based on the existing bandwidth.

Disclosure of Invention

The invention discloses a method for transmitting panoramic original picture video blocks according to the defects of the prior art. The invention aims to transmit partial panoramic contents in real time according to the actual requirements of panoramic real-time transmission and adjust the transmission contents on line aiming at the original panoramic video data obtained by the panoramic double-fisheye acquisition equipment, thereby reducing the transmission bandwidth requirement of images, improving the transmission effect and obtaining better user experience.

The invention is realized by the following technical scheme.

A method for transmitting panoramic original picture video blocks comprises the following steps:

1. acquiring a view angle position and a view angle range of a user in a panoramic scene;

2. calculating and converting the user visual angle information into an original visual angle area and transmitting the original visual angle area to the panoramic camera;

3. intercepting and transmitting the corresponding area image;

4. and step 1 is continued, so that the real-time watching interaction of the user is realized.

Generally, because the resolution of the panoramic picture is too high compared with that of the ordinary video, the transmission bandwidth requirement is also high. The invention provides a method for transmitting a panoramic original picture video in a blocking mode, and particularly aims to obtain an original panoramic picture by a panoramic double-fisheye acquisition device. When the terminal equipment is played, the position of the area where the user watches the panoramic view angle is obtained, the original data range of the corresponding area is obtained through coordinate calculation, and the surrounding area to be selected is further expanded, so that the real-time image preview updating speed when the user switches the view angle is improved.

The invention has the advantages that the panoramic picture is transmitted in the blocked area according to the actual watching requirement in the panoramic picture, and the transmission is carried out by quickly calculating the area range when the user switches the visual angle, so that the user can realize the noninductive watching of any area of the panoramic picture, the real-time transmission of the effect of a higher clear panoramic picture under the same bandwidth condition can be realized, and the user experience is better.

Drawings

FIG. 1 is a block flow diagram of the method of the present invention;

FIG. 2 is a schematic view of a user viewing area viewing angle range calculation;

FIG. 3 is a schematic view of a panoramic field of view area projected onto an original panoramic picture area;

FIG. 4 is a schematic diagram of a panoramic original picture area dilation calculation;

FIG. 5 is a schematic diagram of a panoramic original picture transfer area calculation;

FIG. 6 is a schematic diagram of a panoramic original picture calculation considering a transmission area of a picture buffer;

FIG. 7 is a schematic diagram of a panoramic original picture calculation considering a transmission area of a picture buffer;

fig. 8 is a schematic diagram of calculation of a buffer area in consideration of a panoramic view angle range.

Detailed Description

The present invention is further described below in conjunction with the following detailed description, which is intended to further illustrate the principles of the invention and is not intended to limit the invention in any way, but is equivalent or analogous to the present invention without departing from its scope.

With reference to the attached drawings.

For the transmission of panoramic picture data, particularly high-definition panoramic pictures, in order to solve the problem of smooth transmission of the panoramic pictures, the invention provides a real-time blocking transmission scheme so as to achieve the purposes of saving bandwidth and improving the browsing experience of the high-definition panoramic pictures.

The invention relates to a panoramic original picture video block transmission method, which comprises the following steps:

and S1, acquiring the position and range of the visual angle in the panoramic scene.

When a user views a part of a panoramic picture, the panoramic ball model is usually used to project an original spherical picture onto a screen for planar display, the size of a window in the display process is often controlled by the user through a mouse or a touch screen, and the display content of the picture can be switched by dragging the picture. From the display range and the angle of view of the display screen, the picture area and position viewed in the panoramic picture can be determined. The view angle position is determined by the longitude and latitude corresponding to the center of the picture, and the view angle range is calculated as shown in fig. 2;

ω＝2θ+fov

wherein, E is a viewpoint position, α R is a size of OE, and is changed by a control input when a user watches, R is a size of a panoramic spherical picture AO and BO, a general normalized value is 1, fov is a viewpoint viewing angle, a general value is a fixed set value, and ω is a range calculated from a current viewing angle range of the picture.

In the actual screen display process, a complete viewpoint viewing angle can be ensured in one direction, so that the display range on the screen which is finally determined to be watched by the user is determined by the length-width ratio of the display screen, and if the viewpoint viewing angle in the longitudinal direction of the screen is ensured to be fixed, the final viewing angle range is as follows:

ω_H＝ω

wherein W and H areThe screen effective window displays the resolution size, unit: a pixel; omega_H、ω_WIs the actual viewing angle area range.

And S2, calculating and converting the original view angle area by the user view angle information, and transmitting the original view angle area to the panoramic camera.

After the display end acquires the picture area and the position viewed by the screen, the area range viewed by the current user is confirmed through the longitude and latitude, as shown in fig. 3. The determined latitude and longitude ranges are as follows:

and solving an original visual angle area through the projection change relation of the panorama and the original picture. In the process of calculating the area, the area in the final original picture is divided by a certain area block, as shown in fig. 4. The calculation result is represented by using a modular graph, the modular graph is divided into w × h grids from the final original image, and each grid represents a small area of the original image. And for the final area, determining whether the corresponding image area is an area needing transmission or not by marking each square value in the template domain, wherein 1 represents that transmission is needed, and 0 represents that transmission is not needed. And in the coordinate calculation process, for four vertexes of longitude and latitude coordinates: (lon)_L,lat_U)， (lon_L,lat_D)，(lon_R,lat_U)，(lon_R,lat_D) And performing projection calculation to obtain an original image coordinate, performing equivalent calculation on the coordinate to obtain an approximate domain area.

When 0 ° < lon <180 °

x＝x_L-r*cos(lon)；

y＝y_L-r*sin(lat)；

When-180 ° < lon <0 ° ]

x＝x_R+r*cos(lon)；

y＝y_R-r*sin(lat)；

And S3, intercepting and transmitting the corresponding area image.

The image corresponding to the original picture is intercepted from the template picture, the template picture is firstly subjected to effective region expansion, as shown in figure 5, namely, the region around the region needing to be intercepted is expanded, and the operation is executed to realize the advanced selection of the region to be selected of the target region, so that the advanced cache of the data is realized. In addition, for the final region selection, the region selection can be expanded by analyzing the operation records of the user and predicting in advance, namely, the region which the user wants to observe can be loaded in advance, and the loading delay of the real-time data watching of the user is reduced.

In step S2, the generation of the template map may adopt one of the following methods:

first, S21, calculating the original coordinates of two fish eyes after obtaining the coordinates (lon, lat) of the corresponding fixed point, and (x 'after obtaining the corresponding point by calculation'_L,y’_U)，(x’_R,y’_D) Then the corresponding template region is calculated as:

x_L＝min(x’_L,x’_R)，x_R＝max(x’_L,x’_R)，y_U＝min(y’_U,y’_D)，y_D＝max(y’_U,y’_D)

the seed template regions are: (x)_L,y_U),(x_R,y_U),(x_L,y_D),(x_R,y_D) A rectangular area is formed by the enclosing.

Second, S22, after obtaining the corresponding vertex coordinates (lon, lat), calculating the original two-fish-eye coordinates, and after calculating to obtain the corresponding point (x'₁,y’₁)，(x’₂,y’₂)，(x’₃,y’₃)，(x’₄,y’₄) The corresponding template region is calculated as:

x_L＝min(x’₁,x’₂，x’₃,x’₄),x_R＝max(x’₁,x’₂，x’₃,x’₄)，y_u＝min(y’₁,y’₂，y’₃,y’₄)， y_D＝max(y’₁,y’₂，y’₃,y’₄)；

the seed template regions are: (x)_L,y_U),(x_R,y_U),(x_L,y_D),(x_R,y_D) The enclosed rectangular area is shown in fig. 6.

In step S3, after the dilation process of the template layout, a region buffer is performed, and as shown in fig. 7, the following strategy is used:

when the visual angle of the user changes, the region change trend direction is adopted for caching, and the size of the region is changed according to the visual angle change speed, wherein the calculation mode is as follows:

for the current view angle range, for the range center coordinate P_v(lon, lat) calculating a viewing angle range continuous region based on the viewing angle movement change direction and speed (as shown in FIG. 8)

lon_d＝lon+v*arc_x；

lat_d＝lat+v*arc_y；

P_d(lon_d，lat_d) And P_vThe continuous region between (lon, lat) is the region range needing transmission currently, and the view width is kept to be omega_H、ω_W；

For the transmission of panoramic picture data, particularly high-definition panoramic pictures, in order to solve the problem of smooth transmission, the invention adopts a real-time block transmission scheme to achieve the purposes of saving bandwidth and improving the browsing experience of the high-definition panoramic pictures.