阅读说明：本技术 图像采集方法、显示方法、装置、电子设备及存储介质 (Image acquisition method, display method, device, electronic equipment and storage medium ) 是由 查雄兴 于 2020-05-22 设计创作，主要内容包括：本发明公开了一种图像采集方法、显示方法、装置、电子设备及存储介质。本发明的VR图像采集方法,包括：接收VR设备的朝向和位置信息；根据VR设备的朝向和位置信息,采集全景视频图像中的部分像素,生成第一视频图像；将第一视频图像进行编码,并将编码后的视频图像发送给VR设备。通过接收VR设备的朝向和位置信息,并根据朝向和位置信息采集全景视频图像中的部分像素进行编码和传输,能够消息降低VR视频图像的文件大小,降低传输带宽。(The invention discloses an image acquisition method, a display device, electronic equipment and a storage medium. The VR image acquisition method comprises the following steps: receiving orientation and location information of a VR device; acquiring partial pixels in the panoramic video image according to the orientation and position information of the VR equipment to generate a first video image; and coding the first video image, and sending the coded video image to the VR equipment. By receiving the orientation and position information of the VR equipment and collecting partial pixels in the panoramic video image according to the orientation and position information for coding and transmitting, the file size of the VR video image can be reduced, and the transmission bandwidth is reduced.)

A VR image acquisition method, comprising:

receiving orientation and location information from a VR device;

acquiring partial pixels in a panoramic video image according to the orientation and position information of the VR equipment to generate a first video image;

and encoding the first video image, and sending the encoded video image to the VR equipment.

2. The VR image capture method of claim 1, wherein capturing a portion of pixels in a panoramic video image based on the orientation and position information of the device to generate a first video image comprises:

and acquiring partial pixels in the panoramic video image according to the orientation and the position information of the VR equipment and the visual range of human eyes to generate a first video image.

3. The VR image acquisition method of claim 1, wherein encoding the first video image and sending the encoded video image to the VR device comprises:

and encoding the first video image, and sending the encoded video image and the orientation and position information of the video image to the VR device.

A VR image display method, comprising:

sending the orientation and position information of the VR equipment to a host end;

receiving a first video image, wherein the first video image is generated by a host according to the orientation and position information acquisition of VR equipment;

decoding the first video image.

5. The VR image display method of claim 4, comprising: and receiving the orientation and position information of the video image while receiving the first video image, and decoding the first video image according to the orientation and position information of the VR equipment.

6. The VR image display method of claim 4, wherein the VR device receives the first video image and orientation and position information of the video image, decodes and renders the first video image, and comprises: and moving a huge screen and a camera in the VR equipment along with the orientation of the video image during rendering.

7. A VR image acquisition device comprising:

a receiving module to receive orientation and location information from a VR device;

the image generation module is used for acquiring partial pixels in a panoramic video image according to the orientation and position information of the VR equipment to generate a first video image;

and the coding sending module is used for coding the first video image and sending the coded video image to the VR equipment.

8. A VR image display apparatus, comprising:

the sending module is used for sending the orientation and position information of the VR equipment to the host end;

the image receiving module is used for receiving a first video image, and the first video image is generated by the host according to the orientation and position information acquisition of the VR equipment;

and the decoding module is used for decoding the first video image.

9. An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor when executing the program implementing:

the VR image acquisition method of any of claims 1 to 3;

alternatively, the first and second electrodes may be,

the VR image display method of any of claims 4 to 6.

10. A computer-readable storage medium storing computer-executable instructions for:

performing the VR image acquisition method of any of claims 1 to 3;

alternatively, the first and second electrodes may be,

performing the VR image display method of any of claims 4-6.

Technical Field

The invention relates to the field of virtual reality, in particular to an image acquisition method, a display device, electronic equipment and a storage medium.

Background

With the development of VR cloud games, the panoramic video acquired by the encoding end has high resolution, for example, the resolution of the panoramic video can reach 8K. In order to meet the requirement of high-quality display pictures, huge bandwidth and powerful decoders are needed, which has high requirements on network transmission bandwidth and software and hardware of devices of VR cloud game players, and therefore, the game implementation cost is high.

Disclosure of Invention

The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides an image acquisition method, a display device, an electronic device and a storage medium, which can effectively reduce VR image transmission bandwidth under effective bandwidth and hardware conditions.

In a first aspect, an embodiment of the present invention provides a VR image acquisition method, including: receiving orientation and location information of a VR device; acquiring partial pixels in the panoramic video image according to the orientation and position information of the VR equipment to generate a first video image; and coding the first video image, and sending the coded video image to the VR equipment.

The VR image acquisition method provided by the embodiment of the invention at least has the following beneficial effects: by receiving the orientation and position information of the VR equipment and collecting partial pixels in the panoramic video image according to the orientation and position information for coding and transmitting, the file size of the VR video image can be reduced, and the transmission bandwidth is reduced.

According to another embodiment of the invention, a VR image capture method for capturing a portion of pixels in a panoramic video image according to orientation and position information of a device to generate a first video image includes: and acquiring partial pixels in the panoramic video image according to the orientation and the position information of the VR equipment and the visual range of human eyes to generate a first video image.

According to another embodiment of the present invention, a VR image capture method for encoding a first video image and transmitting the encoded video image to a VR device includes: and encoding the first video image, and sending the encoded video image and the orientation and position information of the video image to the VR device.

In a second aspect, an embodiment of the present invention provides a VR image display method, including: sending the orientation and position information of the VR equipment to a host end; receiving a first video image, wherein the first video image is generated by a host according to the orientation and position information acquisition of VR equipment; the first video image is decoded.

The VR image display method provided by the embodiment of the invention at least has the following beneficial effects: the orientation and position information of the equipment is sent to the host end, the first video image only with partial pixels is received and decoded, the requirement of playing VR video on the hardware of the equipment end can be effectively reduced, and the bandwidth is saved.

According to the VR image display method of other embodiments of the invention, the first video image is received while the orientation and position information of the video image is received, and the first video image is decoded according to the orientation and position information of the VR device.

According to VR image display methods of further embodiments of the present invention, a VR device receiving a first video image and orientation and position information for the video image, decoding and rendering the video image, includes: and moving a huge screen and a camera in the VR equipment along with the orientation of the video image during rendering.

In a third aspect, an embodiment of the present invention provides a VR image acquisition apparatus, including: a receiving module to receive orientation and location information from a VR device; the image generation module is used for acquiring partial pixels in the panoramic video image according to the orientation and position information of the VR equipment and generating a first video image; and the coding sending module is used for coding the first video image and sending the coded video image to the VR equipment.

In a fourth aspect, an embodiment of the present invention provides a VR image display apparatus, including: the sending module is used for sending the orientation and position information of the VR equipment to the host end; the image receiving module is used for receiving a first video image, and the first video image is generated by the host end according to the orientation and position information acquisition of the VR equipment; and the decoding module is used for decoding the first video image.

In a fifth aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor when executing the program implementing: the VR image acquisition method of the first aspect; alternatively, the VR image display method as set forth in the second aspect.

In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions for: performing the VR image acquisition method of the first aspect; alternatively, the VR image display method as described in the second aspect is performed.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

FIG. 1 is a schematic flow chart of a VR image acquisition method in an embodiment of the invention;

fig. 2 is a schematic view of a field angle of a VR device in an embodiment of the present invention;

fig. 3 is a flowchart illustrating a VR image display method according to an embodiment of the present invention.

Detailed Description

The concept and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments to fully understand the objects, features and effects of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments, and those skilled in the art can obtain other embodiments without inventive effort based on the embodiments of the present invention, and all embodiments are within the protection scope of the present invention.

In the description of the embodiments of the present invention, if "a number" is referred to, it means one or more, if "a plurality" is referred to, it means two or more, if "greater than", "less than" or "more than" is referred to, it is understood that the number is not included, and if "greater than", "lower" or "inner" is referred to, it is understood that the number is included. If reference is made to "first" or "second", this should be understood to distinguish between features and not to indicate or imply relative importance or to implicitly indicate the number of indicated features or to implicitly indicate the precedence of the indicated features.

In a first aspect, an embodiment of the present invention provides a VR image acquisition method.

Referring to fig. 1, a schematic flow chart of a VR image acquisition method in an embodiment of the present invention is shown. The method specifically comprises the following steps:

s101, receiving orientation and position information of VR equipment;

s102, collecting partial pixels in the panoramic video image according to the orientation and position information of the VR equipment to generate a first video image;

s103, encoding the first video image, and sending the encoded video image to VR equipment.

In some specific embodiments, the orientation of the VR device is expressed in right-handed cartesian coordinates in three-dimensional space, and a three-dimensional coordinate system XYZ is set up in the scene where the VR device is located, pitch being a rotation around the X axis, also called pitch angle; yaw is the rotation about the Y axis, also called yaw; a roll is a rotation about the Z axis, also called the roll angle. And the position information of the VR device is coordinate information from the origin in a three-dimensional coordinate system XYZ. And corresponding pixel images are collected in the panoramic video image according to the orientation and position information of the VR device, namely according to the rotation angle of the VR device relative to the XYZ axes and the coordinate information relative to the origin.

In some embodiments, capturing a portion of pixels in the panoramic video image based on the orientation and position information of the device, generating a first video image, comprises: and acquiring partial pixels in the panoramic video image according to the orientation and the position information of the VR equipment and the visual range of human eyes to generate a first video image.

In some specific embodiments, the panoramic video image is a spherical 3D model, and the captured pixel region is a portion of pixels in the panoramic video image that have an origin at the origin of the 3D sphere and that fit the visual range of human eyes. As shown in fig. 2, in some embodiments, a virtual camera 201 is provided in the VR device, and the visual range of human eyes can be represented by the field angle FOV of the camera 201, and the partial pixel area in the captured panoramic video image is 202. For example, the horizontal viewing angle of the acquisition is 120 degrees, and the vertical viewing angle is 120 degrees, which can be adjusted according to the device.

In some embodiments, encoding the first video image and transmitting the encoded video image to the VR device includes: and encoding the first video image, and sending the encoded video image and the orientation and position information of the video image to the VR device.

In a second aspect, an embodiment of the present invention provides a VR image displaying method.

Referring to fig. 3, a flowchart of a VR image display method in an embodiment of the present invention is shown. The method specifically comprises the following steps:

s301, sending the orientation and position information of the VR equipment to a host end;

s302, receiving a first video image, wherein the first video image is generated by the host according to the orientation and position information acquisition of VR equipment;

and S303, decoding the first video image.

In some embodiments, the host side is a PC side or a server side where the VR game runs. The received first video image is generated by the host according to the orientation and position information acquisition of the VR device, in some specific embodiments, the orientation of the VR device is represented by right-hand cartesian coordinates of a three-dimensional space, a three-dimensional coordinate system XYZ is set in a scene where the VR device is located, and pitch rotates around an X axis and is also called a pitch angle; yaw is the rotation about the Y axis, also called yaw; a roll is a rotation about the Z axis, also called the roll angle. And the position information of the VR device is coordinate information from the origin in a three-dimensional coordinate system XYZ. And corresponding pixel images are collected in the panoramic video image according to the orientation and position information of the VR device, namely according to the rotation angle of the VR device relative to the XYZ axes and the coordinate information relative to the origin.

In some embodiments, a VR image display method includes: sending the orientation and position information of the VR equipment to a host end; receiving a first video image, wherein the first video image is generated by a host according to the orientation and position information acquisition of VR equipment; and receiving the orientation and position information of the video image while receiving the first video image, and decoding the first video image according to the orientation and position information of the VR device. And the orientation and position information of the VR equipment is the orientation and position information corresponding to the previous frame of image, the VR equipment is simulated in a VR scene at the host end by sending the orientation and position information to the host end, the position and the orientation of the VR equipment are moved in the scene, and then a part of pixel image is intercepted according to the field angle FOV.

In some embodiments, a VR device receives a first video image and orientation and position information for the video image, and decodes and renders, including: and moving a huge screen and a camera in the VR equipment along with the orientation of the video image during rendering. In some embodiments, the movement may be in terms of frame number or time, following the orientation of the video image.

In some embodiments, such as: VR equipment is first display all-in-one, and the customer end is cloud game server. At the initial moment, the position of the all-in-one machine is an original point position A (0,0,0), the direction information is 45 degrees upwards, the position information A (0,0,0) and the direction information (45 degrees upwards) are sent to a cloud game server, after the cloud game server receives the position information A (0,0,0) and the direction information (45 degrees upwards), the position information A (0,0,0) coordinates sent by the all-in-one machine are used as an original point, namely, the original point is placed at the central point of a 360-degree sphere scene, then the visual range of the all-in-one machine is intercepted according to the direction information (45 degrees upwards), after the image of the part of pixels is coded, the image is sent to the end of the all-in-one machine together with the position information A (0,0,0) and the direction information (45 degrees upwards) corresponding to the frame image of the all-in-one machine, the all-in-one machine decodes the received view screen image, and the macro screen and the camera in, and displaying the decoded image, then, moving the all-in-one machine to the left by 60 degrees, moving the all-in-one machine to the whole body downwards, changing the position information into B (0, -20,0), sending the direction and the position information to the server side in real time, intercepting the video image in the field angle range by the server side through the direction and the position information, and transmitting the video image back to the all-in-one machine side.

In some specific embodiments, when performing VR image decoding rendering, the juveniles and cameras in the VR device are moved to follow the orientation of the video image, that is, according to the original orientation and position information of the frame data, for example, at the time of T1, the VR all-in-one machine sends OR1 (including the orientation and position of the VR all-in-one machine) to the service end, the service end collects a frame of image data with the serial number of N1 according to OR1, sends the frame of image data N1 and OR1 to the VR all-in-one machine end, the VR all-in-one machine end is currently at the time of T2, the orientation and position are OR2, but one frame of image data at the time of T1 is decoded when rendering, and then the jujubes and the cameras are sequentially changed to the direction and position of OR1 for rendering and displaying, therefore, the problem that the user experience is influenced by the fact that the user watches VR images and lacks or does not correspond to the VR images due to the fact that the displayed images exceed the range of the huge screen and the camera can be avoided.

In a third aspect, an embodiment of the present invention provides a VR image acquisition apparatus, including:

a receiving module to receive orientation and location information from a VR device;

the image generation module is used for acquiring partial pixels in the panoramic video image according to the orientation and position information of the VR equipment and generating a first video image;

and the coding sending module is used for coding the first video image and sending the coded video image to the VR equipment.

In some embodiments, the image generation module further includes acquiring a portion of pixels in the panoramic video image according to the orientation of the VR device, the position information, and the visual range of human eyes to generate the first video image.

In some embodiments, the encoding sending module further includes encoding the first video image and sending the encoded video image and orientation and position information of the video image to the VR device.

In a fourth aspect, an embodiment of the present invention provides a VR image display apparatus, including:

the sending module is used for sending the orientation and position information of the VR equipment to the host end;

the image receiving module is used for receiving a first video image, and the first video image is generated by the host end according to the orientation and position information acquisition of the VR equipment;

and the decoding module is used for decoding the first video image.

In some embodiments, the image receiving module further comprises: and the video decoder is used for receiving the orientation and position information of the video image while receiving the first video image, and decoding the first video image according to the orientation and position information of the VR device.

In some embodiments, the VR image display device further comprises:

and the decoding rendering module is used for receiving the first video image and the orientation and position information of the video image, decoding and rendering the first video image, and moving the giant screen and the camera in the VR equipment along with the orientation of the video image during rendering.

In a fifth aspect, an embodiment of the present invention provides an electronic device, in some embodiments, the electronic device is a host end for running a VR game, and in some specific embodiments, the host end may be a PC apparatus or a server for running the VR game, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor when executing the program implementing: the VR image acquisition method of the first aspect.

In a sixth aspect, an embodiment of the present invention provides an electronic device, in some embodiments, the electronic device is a VR device, and in some specific embodiments, the VR device is a head-up or all-in-one machine, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor when executing the program implementing: the VR image display method as set forth in the second aspect.

In a seventh aspect, an embodiment of the present invention provides a VR image acquisition and display method, including: the VR image acquisition method of the first aspect and the VR image display method of the second aspect.

In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions for: performing the VR image acquisition method of the first aspect; alternatively, the VR image display method of the second aspect is performed.

The above-described embodiments of the apparatus are merely illustrative, wherein the units illustrated as separate components may or may not be physically separate, i.e. may be located in one place, or may also be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention. Furthermore, the embodiments of the present invention and the features of the embodiments may be combined with each other without conflict.