阅读说明：本技术 一种深度信息的量化和编码方法 (Depth information quantization and coding method ) 是由 邹川川 于 2021-08-09 设计创作，主要内容包括：本发明公开了一种深度信息的量化和编码方法。它具体包括如下步骤：用开启激光雷达的设备拍摄带有深度信息的视频；将深度信息量化为颜色编码数据；选择一条视频轨道,并在meta data中标记这条视频轨道用于存储深度信息；将量化后的颜色编码数据压缩为视频流保存在视频轨道中,并且在meta data中为该视频流做特定标记；读取meta data找到带标记的视频轨道,之后找到带特定标记的深度信息编码的那个视频流；将深度信息编码的那个视频流解码为颜色编码数据；将颜色编码数据量化还原为深度信息。本发明的有益效果是：能够在支持该颜色编码格式的设备上进行渲染；将深度信息量化后的数据编码成视频码流,以便在普通设备上传输和解码。(The invention discloses a method for quantizing and coding depth information. The method specifically comprises the following steps: shooting a video with depth information by using equipment for starting a laser radar; quantizing the depth information into color-coded data; selecting a video track, and marking the video track in meta data for storing depth information; compressing the quantized color coded data into a video stream, storing the video stream in a video track, and making a specific mark for the video stream in meta data; reading meta data to find a video track with a mark, and then finding a video stream coded by depth information with a specific mark; decoding the video stream encoded with the depth information into color-coded data; the color-coded data is quantized back to depth information. The invention has the beneficial effects that: capable of rendering on a device that supports the color coding format; and coding the data after the depth information quantization into a video code stream so as to be transmitted and decoded on common equipment.)

1. A method for quantizing and coding depth information is characterized by comprising the following steps:

(1) shooting a video with depth information by using equipment for starting a laser radar;

(2) the depth information is quantized to color coded data yuv, the quantization formula being: yuv = k D R, where k is a quantization factor calculated according to the depth information effective range and the yuv value range, D represents the depth information value, and R represents the yuv value range; the depth information effective range refers to the distance from equipment which can be sensed by the laser radar, the quantization factor refers to the calculation proportion of remapping the depth information to another range value, the depth information value refers to the distance between an actual object and a photographer, and the yuv value range refers to the value range which can be expressed by a unit yuv when a computer stores;

(3) selecting a video track, and marking the video track in meta data for storing depth information; the video track is used for storing digitized video information, and meta data is used for describing attribute information of a video file;

(4) compressing the quantized color coded data into a video stream, storing the video stream in a video track, and making a specific mark for the video stream in meta data;

(5) reading meta data to find a video track with a mark, and then finding a video stream coded by depth information with a specific mark;

(6) decoding the video stream encoded with the depth information into color-coded data;

(7) quantizing and restoring the color coded data into depth information, wherein the quantization restoration formula is as follows: d = 1/k × 1/R × yuv.

2. The method as claimed in claim 1, wherein in step (1), the method specifically comprises: and opening the laser radar function of the equipment, recording a video by using a camera on the equipment, and acquiring the real-time depth information of the object shot in the video.

3. The method as claimed in claim 2, wherein in step (2), the method further comprises: in the depth information acquired in real time, the depth information between 0 m and 5m from a photographer is quantized into color coding format data, and the quantized yuv data is directly rendered on a screen.

4. A method as claimed in claim 3, wherein in step (4), the method comprises: and compressing the quantized yuv data into a stream 1 video stream by using an HEVC video coding format, and deriving a general mp4 video file, wherein the video contains a recorded video stream 0 and a depth information video stream 1, and the coding and the storage of the depth information after quantization are completed.

5. The method as claimed in claim 4, wherein in step (5), the method further comprises: meta data information of an mp4 video file is acquired, a video track marked with depth information is read, and then stream 1 is determined to be a depth information video stream.

6. The method as claimed in claim 5, wherein in step (7), the method further comprises: according to the acquired depth information, different special effects are added into the stream 0 video stream, and the stereoscopic creation of the movie and television special effects is enriched.

Technical Field

The present invention relates to the field of image coding and decoding technology, and more particularly, to a depth information quantization and coding method.

Background

With the application of the laser radar to part of high-end mobile phones, the pictures or videos shot by the mobile phones can be accompanied by the distance between a photographer and an object, namely depth information; however, for depth information, the existing technology lacks a unified cross-platform depth information coding, decoding, storing and rendering method.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides a depth information quantization and coding method supporting depth information rendering.

In order to achieve the purpose, the invention adopts the following technical scheme:

a method for quantizing and coding depth information specifically comprises the following steps:

(1) shooting a video with depth information by using equipment for starting a laser radar;

(2) the depth information is quantized to color coded data yuv, the quantization formula being: yuv = k D R, where k is a quantization factor calculated according to the depth information effective range and the yuv value range, D represents the depth information value, and R represents the yuv value range; the depth information effective range refers to the distance from equipment which can be sensed by the laser radar, the quantization factor refers to the calculation proportion of remapping the depth information to another range value, the depth information value refers to the distance between an actual object and a photographer, and the yuv value range refers to the value range which can be expressed by a unit yuv when a computer stores;

(3) selecting a video track, and marking the video track in meta data for storing depth information; the video track is used for storing digitized video information, and meta data is used for describing attribute information of a video file;

(4) compressing the quantized color coded data into a video stream, storing the video stream in a video track, and making a specific mark for the video stream in meta data;

(5) reading meta data to find a video track with a mark, and then finding a video stream coded by depth information with a specific mark;

(6) decoding the video stream encoded with the depth information into color-coded data;

(7) quantizing and restoring the color coded data into depth information, wherein the quantization restoration formula is as follows: d = 1/k × 1/R × yuv.

The depth information quantization and coding and decoding method quantizes the depth information into color coded data according to a specific quantization standard, codes the color coded data into a video stream according to a video compression standard, decodes the video stream into the color coded data, quantizes and restores the color coded data into the depth information, can be applied to the storage of the depth information, transmits and renders between different devices, and uses the depth information and the video information to make scenes such as special effects. The quantization of the depth information can convert the depth information into color coded data under the condition of limited precision loss, the color coded data is coded in the existing color coding format, and the depth information can be rendered on a device supporting the color coding format. Wherein: the quantization method of the depth information uses different quantization strategies (such as uniform quantization and non-uniform quantization) to convert the depth information into general color coding format (such as yuv) data. The coding method of the depth information codes the data after the depth information quantization into a video code stream so as to be transmitted and decoded on common equipment.

Preferably, in the step (1), specifically: and opening the laser radar function of the equipment, recording a video by using a camera on the equipment, and acquiring the real-time depth information of the object shot in the video.

Preferably, in the step (2), specifically: in the depth information acquired in real time, the depth information between 0 m and 5m from a photographer is quantized into color coding format data, and the quantized yuv data is directly rendered on a screen.

Preferably, in the step (4), specifically: and compressing the quantized yuv data into a stream 1 video stream by using an HEVC video coding format, and deriving a general mp4 video file, wherein the video contains a recorded video stream 0 and a depth information video stream 1, and the coding and the storage of the depth information after quantization are completed.

Preferably, in the step (5), specifically: meta data information of an mp4 video file is acquired, a video track marked with depth information is read, and then stream 1 is determined to be a depth information video stream.

Preferably, in the step (7), specifically: according to the acquired depth information, different special effects are added into the stream 0 video stream, and the stereoscopic creation of the movie and television special effects is enriched.

The invention has the beneficial effects that: the method can be applied to the storage of depth information, the transmission and rendering among different devices, and scenes such as special effects and the like are made by using the depth information and the video information; capable of rendering on a device that supports the color coding format; and coding the data after the depth information quantization into a video code stream so as to be transmitted and decoded on common equipment.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

Detailed Description

The invention is further described with reference to the following figures and detailed description.

In the embodiment shown in fig. 1, a depth information quantization and coding method quantizes depth information into general color coding data by using a specific quantization strategy, compresses the depth information into a video by using a standard video compression technology, and implements functions such as parsing and rendering of depth information on devices of different operating systems and hardware. In order to improve the quantization precision of depth information and the universality of coding, the method selects a 10Bit YUV 420 color coding format. When compressing video streams, the HEVC standard is adopted for compression in order to improve the image quality and reduce the size of video files. Taking the example of adding a special effect to objects with different depths in a video by using depth information on an iPhone 12 Pro Max, a specific implementation example of the quantization and coding and decoding of the depth information is given, and the specific implementation example specifically comprises the following steps:

(1) shooting a video with depth information by using equipment for starting a laser radar; the method specifically comprises the following steps: and opening the laser radar function of the iPhone, recording a section of video by using a rear camera on the iPhone, and acquiring the real-time depth information of the object shot in the video.

(2) The depth information is quantized to color coded data yuv, the quantization formula being: yuv = k D R, where k is a quantization factor calculated according to the depth information effective range and the yuv value range, D represents the depth information value, and R represents the yuv value range; the method specifically comprises the following steps: in the depth information acquired in real time, the depth information between 0 and 5m from a photographer is quantized into color coding format data, namely 10Bit YUV 420 color coding format data by using a quantization formula, the maximum precision loss is 1/(2^10) × 5 and is about 0.00483m, and the quantized YUV data are directly rendered on a screen, and the fact that the closer to the photographer, the lower the brightness and the farther from the photographer, the higher the brightness is shown.

Wherein: the effective range of the depth information refers to a distance which can be sensed by the laser radar and is far away from equipment, for example, the distance can be sensed by 5 meters at the most, then the shot picture or video is finished, the values of all points are between 0 and 5 meters, and the distance between an object at the point and a photographer is represented. The quantization factor is a calculated scale for remapping the depth information to another range of values, such as yuv =100 depth information in linear quantization. 100 is the quantization factor, and when yuv =100 can be calculated by the quantization factor, the actual depth information is 1 meter. The calculation formulas may be different, and this factor ensures i can restore the information. The depth information value is, as described above, the distance between the actual object and the photographer, and a valid range is available due to the limited distance that can be perceived by the lidar device. The range of the yuv value is the maximum value that a unit yuv can represent when the computer is stored, for example, 10bit yuv, and the maximum value that can be represented is 10 powers of 2, namely the range is 0-1024.

(3) Selecting a video track, and marking the video track in meta data for storing depth information;

wherein: description of video tracks, such as a video file with sound, may have one or more video tracks and one or more audio tracks, i.e., for storing digitized image information and sound information, respectively. meta data is used to describe the attribute information of video or audio file, such as the video rating is 1920 × 1080, the video title, duration, singer name, etc. are often recorded in meta data. Since the depth information has been converted into yuv information by step (2), which is a color coding method and a very common format for image processing, the video stream actually stores yuv, but yuv can be restored to depth information in step (7).

(4) Compressing the quantized color coded data into a video stream, storing the video stream in a video track, and making a specific mark for the video stream in meta data; the method specifically comprises the following steps: the quantized yuv data is compressed into a stream 1 video stream by an HEVC video coding format, a general mp4 video file is derived, the video contains a recorded video stream 0 and a depth information video stream 1, the coding and the storage of the depth information after quantization are completed, and the video can be shared among different devices.

As described above, meta data is used to record data information of some files, such as a file name, depth track: stream 1, when decoded, knows that the depth information is stored in stream 1.

(5) Reading meta data to find a video track with a mark, and then finding a video stream coded by depth information with a specific mark; the method specifically comprises the following steps: the method comprises the steps of obtaining meta data information of an mp4 video file, reading a video track marked with depth information, and then determining stream 1 as a depth information video stream, wherein the steps can be carried out in different devices.

(6) Decoding the video stream encoded with the depth information into color-coded data; namely, stream 1 is decoded into 10-Bit YUV 420 color-coded format data.

(7) Quantizing and restoring the color coded data into depth information, wherein the quantization restoration formula is as follows: d = 1/k 1/R yuv; the method specifically comprises the following steps: according to the acquired depth information, different special effects are added into the stream 0 video stream, and the stereoscopic creation of the movie and television special effects is enriched.

The depth information is provided, the distance between the surrounding environment in the video and a video photographer can be read, the video photographer is really integrated into the video environment, a film and television special effect creator can create a special effect according to the information, and the special effect is more three-dimensional and vivid.

The depth information quantization and coding and decoding method quantizes the depth information into color coded data according to a specific quantization standard, codes the color coded data into a video stream according to a video compression standard, decodes the video stream into the color coded data, quantizes and restores the color coded data into the depth information, can be applied to the storage of the depth information, transmits and renders between different devices, and uses the depth information and the video information to make scenes such as special effects. The quantization of the depth information can convert the depth information into color coded data under the condition of limited precision loss, the color coded data is coded in the existing color coding format, and the depth information can be rendered on a device supporting the color coding format. Wherein: the quantization method of the depth information uses different quantization strategies (such as uniform quantization and non-uniform quantization) to convert the depth information into general color coding format (such as yuv) data. The coding method of the depth information codes the data after the depth information quantization into a video code stream so as to be transmitted and decoded on common equipment.