阅读说明：本技术 图像处理方法及装置 (Image processing method and device ) 是由 马祥 杨海涛 宋楠 张恋 于 2020-06-11 设计创作，主要内容包括：本申请公开了一种图像处理方法及装置,涉及图像处理领域,该方法提高了图像的编码效率,从而节省了存储图像的存储资源,以及节省了传输图像的传输带宽。该方法应用于编码装置。该方法包括：确定待编码比特；基于待编码比特的上下文,估计待编码比特的概率分布；其中,待编码比特的上下文包括：第一比特串和/或第二比特串中,与待编码比特对应的预设范围内的比特；其中,第二比特串用于表示第一子带中的另一个系数；根据待编码比特的概率分布,编码待编码比特,以得到待处理图像的编码信息。(The application discloses an image processing method and device, relates to the field of image processing, and improves the encoding efficiency of images, so that the storage resource for storing the images is saved, and the transmission bandwidth for transmitting the images is saved. The method is applied to an encoding device. The method comprises the following steps: determining bits to be coded; estimating the probability distribution of the bits to be coded based on the context of the bits to be coded; wherein the context of the bits to be encoded comprises: bits in a preset range corresponding to bits to be coded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first sub-band; and coding the bits to be coded according to the probability distribution of the bits to be coded so as to obtain the coding information of the image to be processed.)

1. An image processing method, comprising:

determining bits to be coded; the bit to be coded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices obtained by processing an image to be processed, and the first bit matrix is a three-dimensional bit matrix;

estimating a probability distribution of the bits to be coded based on a context of the bits to be coded; the context includes: bits in a preset range corresponding to the bits to be coded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first sub-band;

and coding the bits to be coded according to the probability distribution of the bits to be coded so as to obtain the coding information of the image to be processed.

2. The method of claim 1,

the second bit string is a binary bit string in a second bit matrix if the plurality of binary bit strings are respectively used to represent coefficients in different subbands; the second bit matrix is a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; alternatively, the first and second electrodes may be,

and if the plurality of binary bit strings are all used for representing the coefficients in the first sub-band, the second bit string is a binary bit string in the first bit matrix, and the binary bit string is located in a preset range of the first bit string.

3. The method of claim 1 or 2, wherein the context further comprises: bits in a preset range corresponding to the bits to be coded in a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing another coefficient in the second sub-band; wherein the content of the first and second substances,

if the plurality of binary bit strings are respectively used for representing coefficients in different sub-bands, the third bit string is a binary bit string in the first bit matrix, the fourth bit string is a binary bit string in a second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices, wherein the bit matrix is located in a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to a binary bit string located in a preset range of the first bit string; alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are each used to represent a coefficient in the first subband, the third bit string and the fourth bit string are each a binary bit string in a third bit matrix, the third bit matrix being a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to the third bit matrix, and the plurality of bit matrices are in one-to-one correspondence with a plurality of sub-bands.

4. The method according to any of claims 1 to 3, wherein prior to said determining bits to encode, the method further comprises:

acquiring a plurality of two-dimensional coefficient blocks corresponding to the image to be processed, wherein each two-dimensional coefficient block in the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients, and the plurality of coefficients are coefficients in different sub-bands respectively;

binarizing each coefficient of the plurality of two-dimensional coefficient blocks to obtain the plurality of bit matrices.

5. The method of claim 4, wherein binarizing each coefficient in the plurality of two-dimensional coefficient blocks comprises:

each coefficient in the plurality of two-dimensional coefficient blocks is binarized in a fixed-length binarization manner.

6. The method according to claim 4 or 5, wherein before the obtaining of the plurality of two-dimensional coefficient blocks corresponding to the image to be processed, the method further comprises:

acquiring the image to be processed, wherein the image to be processed comprises an image frame in a picture or a video;

the acquiring of the plurality of two-dimensional coefficient blocks corresponding to the image to be processed includes:

and partitioning and transforming the image to be processed to obtain a plurality of two-dimensional coefficient blocks.

7. The method of any one of claims 1 to 6, wherein the determining bits to be encoded comprises:

and determining the bits to be coded in the bit matrixes according to a preset coding sequence.

8. The method according to any of claims 1 to 7, wherein said estimating a probability distribution of the bits to be encoded based on the context of the bits to be encoded comprises:

and estimating the probability distribution of the bits to be coded through a probability estimation network based on the context of the bits to be coded.

9. The method according to any one of claims 1 to 8, wherein said encoding the bits to be encoded according to the probability distribution of the bits to be encoded comprises:

and entropy coding the bits to be coded according to the probability distribution of the bits to be coded.

10. An image processing method, comprising:

determining the position of a bit to be decoded; the bit to be decoded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices corresponding to the image to be processed, and the first bit matrix is a three-dimensional bit matrix;

estimating a probability distribution of the bits to be decoded based on a context of the bits to be decoded; the context includes: bits in a preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first sub-band;

and decoding the bits to be decoded to obtain the bit matrixes based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed.

11. The method of claim 10,

the second bit string is a binary bit string in a second bit matrix if the plurality of binary bit strings are respectively used to represent coefficients in different subbands; the second bit matrix is a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; alternatively, the first and second electrodes may be,

and if the plurality of binary bit strings are all used for representing the coefficients in the first sub-band, the second bit string is a binary bit string in the first bit matrix, and the binary bit string is located in a preset range of the first bit string.

12. The method of claim 10 or 11, wherein the context further comprises: bits in a preset range corresponding to the bits to be decoded in a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing another coefficient in the second sub-band; wherein the content of the first and second substances,

if the plurality of binary bit strings are respectively used for representing coefficients in different sub-bands, the third bit string is a binary bit string in the first bit matrix, the fourth bit string is a binary bit string in a second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices, wherein the bit matrix is located in a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to a binary bit string located in a preset range of the first bit string; alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are each used to represent a coefficient in the first subband, the third bit string and the fourth bit string are each a binary bit string in a third bit matrix, the third bit matrix being a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to the third bit matrix, and the plurality of bit matrices are in one-to-one correspondence with a plurality of sub-bands.

13. The method according to any one of claims 10 to 12, further comprising:

inversely binarizing binary bit strings in the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrixes; wherein each of the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients that are coefficients in different sub-bands, respectively;

and inversely transforming the two-dimensional coefficient blocks to obtain the image to be processed, wherein the image to be processed comprises an image frame in a picture or a video.

14. The method of claim 13, wherein said inverse binarizing the binary bit strings in the decoded bit matrices comprises:

and according to a fixed-length inverse binarization mode, inversely binarizing binary bit strings in the plurality of bit matrixes obtained by decoding.

15. The method according to any of claims 10 to 14, wherein the determining the position of the bit to be decoded comprises:

and determining the positions of the bits to be decoded in the bit matrixes according to a preset decoding sequence.

16. The method according to any of claims 10 to 15, wherein said estimating a probability distribution of said bits to be decoded based on a context of said bits to be decoded comprises:

estimating, by a probability estimation network, a probability distribution of the bits to be decoded based on a context of the bits to be decoded.

17. The method according to any one of claims 10 to 16, wherein the decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the coding information of the image to be processed comprises:

and entropy decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed.

18. An image processing apparatus, characterized in that the apparatus comprises:

a determining unit for determining bits to be encoded; the bit to be coded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices obtained by processing an image to be processed, and the first bit matrix is a three-dimensional bit matrix;

an estimating unit, configured to estimate a probability distribution of the bits to be coded based on a context of the bits to be coded; the context includes: bits in a preset range corresponding to the bits to be coded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first sub-band;

and the coding unit is used for coding the bits to be coded according to the probability distribution of the bits to be coded so as to obtain the coding information of the image to be processed.

19. The apparatus of claim 18,

the second bit string is a binary bit string in a second bit matrix if the plurality of binary bit strings are respectively used to represent coefficients in different subbands; the second bit matrix is a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; alternatively, the first and second electrodes may be,

and if the plurality of binary bit strings are all used for representing the coefficients in the first sub-band, the second bit string is a binary bit string in the first bit matrix, and the binary bit string is located in a preset range of the first bit string.

20. The apparatus of claim 18 or 19, wherein the context further comprises: bits in a preset range corresponding to the bits to be coded in a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing another coefficient in the second sub-band; wherein the content of the first and second substances,

if the plurality of binary bit strings are respectively used for representing coefficients in different sub-bands, the third bit string is a binary bit string in the first bit matrix, the fourth bit string is a binary bit string in a second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices, wherein the bit matrix is located in a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to a binary bit string located in a preset range of the first bit string; alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are each used to represent a coefficient in the first subband, the third bit string and the fourth bit string are each a binary bit string in a third bit matrix, the third bit matrix being a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to the third bit matrix, and the plurality of bit matrices are in one-to-one correspondence with a plurality of sub-bands.

21. The apparatus of any one of claims 18 to 20, further comprising:

an obtaining unit, configured to obtain, before the determining unit determines a bit to be encoded, a plurality of two-dimensional coefficient blocks corresponding to the image to be processed, where each of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients, and the plurality of coefficients are coefficients in different sub-bands respectively;

a binarization unit for binarizing each coefficient of the plurality of two-dimensional coefficient blocks to obtain the plurality of bit matrices.

22. The apparatus of claim 21,

the binarization unit is specifically configured to binarize each coefficient in the plurality of two-dimensional coefficient blocks in a fixed-length binarization manner.

23. The apparatus of claim 21 or 22,

the acquiring unit is further configured to acquire the image to be processed before acquiring the plurality of two-dimensional coefficient blocks corresponding to the image to be processed, where the image to be processed includes an image frame in a picture or a video;

the obtaining unit is further specifically configured to perform blocking and transformation on the image to be processed to obtain the multiple two-dimensional coefficient blocks.

24. The apparatus of any one of claims 18 to 23,

the determining unit is specifically configured to determine the bits to be encoded in the bit matrices according to a preset encoding order.

25. The apparatus of any one of claims 18 to 24,

the estimating unit is specifically configured to estimate, based on the context of the bits to be coded, a probability distribution of the bits to be coded through a probability estimation network.

26. The apparatus of any one of claims 18 to 25,

the encoding unit is specifically configured to perform entropy encoding on the bits to be encoded according to the probability distribution of the bits to be encoded.

27. An image processing apparatus, characterized in that the apparatus comprises:

a determining unit for determining a position of a bit to be decoded; the bit to be decoded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices corresponding to the image to be processed, and the first bit matrix is a three-dimensional bit matrix;

an estimating unit, configured to estimate a probability distribution of the bits to be decoded based on a context of the bits to be decoded; the context includes: bits in a preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first sub-band;

a decoding unit, configured to decode the bits to be decoded to obtain the bit matrices based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the coding information of the image to be processed.

28. The apparatus of claim 27,

the second bit string is a binary bit string in a second bit matrix if the plurality of binary bit strings are respectively used to represent coefficients in different subbands; the second bit matrix is a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; alternatively, the first and second electrodes may be,

and if the plurality of binary bit strings are all used for representing the coefficients in the first sub-band, the second bit string is a binary bit string in the first bit matrix, and the binary bit string is located in a preset range of the first bit string.

29. The apparatus of claim 27 or 28, wherein the context further comprises: bits in a preset range corresponding to the bits to be decoded in a third bit string and/or a fourth bit string, wherein the third bit string is used for representing one coefficient in a second sub-band, and the fourth bit string is used for representing another coefficient in the second sub-band; wherein the content of the first and second substances,

if the plurality of binary bit strings are respectively used for representing coefficients in different sub-bands, the third bit string is a binary bit string in the first bit matrix, the fourth bit string is a binary bit string in a second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices, wherein the bit matrix is located in a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to a binary bit string located in a preset range of the first bit string; alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are each used to represent a coefficient in the first subband, the third bit string and the fourth bit string are each a binary bit string in a third bit matrix, the third bit matrix being a bit matrix in the plurality of bit matrices that is within a preset range of the first bit matrix; the second sub-band is a sub-band corresponding to the third bit matrix, and the plurality of bit matrices are in one-to-one correspondence with a plurality of sub-bands.

30. The apparatus of any one of claims 27 to 29, further comprising:

the inverse binarization unit is used for inverse binarization of binary bit strings in the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrixes; wherein each of the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients that are coefficients in different sub-bands, respectively;

and the inverse transformation unit is used for inversely transforming the two-dimensional coefficient blocks to obtain the image to be processed, and the image to be processed comprises an image frame in a picture or a video.

31. The apparatus of claim 30,

the inverse binarization unit is specifically configured to inverse binarize binary bit strings in the multiple bit matrices obtained by decoding according to a fixed-length inverse binarization manner.

32. The apparatus of any one of claims 27 to 31,

the determining unit is specifically configured to determine, according to a preset decoding order, positions of the bits to be decoded in the plurality of bit matrices.

33. The apparatus of any one of claims 27 to 32,

the estimating unit is specifically configured to estimate, by a probability estimation network, a probability distribution of the bits to be decoded based on the context of the bits to be decoded.

34. The apparatus of any one of claims 27 to 33,

the decoding unit is specifically configured to perform entropy decoding on the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the coding information of the image to be processed.

35. An image processing apparatus, characterized in that the apparatus comprises: a memory for storing computer instructions and one or more processors for invoking the computer instructions to perform the method of any of claims 1-17.

36. A computer-readable storage medium, having stored thereon a computer program which, when run on a computer, causes the computer to perform the method of any one of claims 1 to 17.

37. A computer-readable storage medium, wherein encoded information obtained by the image processing method according to any one of claims 1 to 9 is stored on the computer-readable storage medium.

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method and apparatus.

Background

With the rapid development of internet science and technology and the increasing abundance of human physical and mental culture, the application requirements for images and videos in the internet are more and more, so that the data volume of high-definition images and high-definition videos is larger and larger. In addition, people have higher and higher requirements on image and video quality. For example, the spatial resolution of images is increasing, and in addition to the spatial resolution of video, the frame rate is also increasing.

After the images and videos are compressed and encoded by the prior art, a large amount of storage space is still required to store the compressed and encoded images and videos at the user end or the server end. Moreover, after the high-quality images and videos are compressed and encoded by the existing method, the efficiency is still low when the images and videos are transmitted in the internet with limited bandwidth. Therefore, how to improve the coding efficiency (or called compression efficiency) of images and videos, so as to relieve the pressure on storage and transmission of images and videos, is a technical problem to be solved urgently.

Disclosure of Invention

The application provides an image processing method and an image processing device, which improve the encoding efficiency of images (or videos), thereby saving the storage resource for storing the images (or videos) and saving the transmission bandwidth for transmitting the images (or videos).

In order to achieve the above purpose, the present application provides the following technical solutions:

in a first aspect, the present application provides an image processing method applied to an image processing apparatus. The method comprises the following steps: determining bits to be coded; the bit to be coded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices obtained after an image to be processed is processed, and the first bit matrix is a three-dimensional bit matrix. Estimating a probability distribution of the bits to be encoded based on the determined context of the bits to be encoded; here, the context of the bits to be encoded includes: bits in a preset range corresponding to bits to be coded in the first bit string and/or the second bit string; wherein the second bit string is used to represent another coefficient in the first subband. And coding the bits to be coded according to the probability distribution of the bits to be coded so as to obtain the coding information of the image to be processed.

By the method, the image processing device determines the context of the bits to be coded in a plurality of binary bit matrixes obtained after the image to be processed is processed, and estimates the probability distribution of the bits to be coded based on the context. Wherein the context comprises bits in a bit string (i.e. the second bit string) belonging to the same subband as the bit string (i.e. the first bit string) in which the bits to be encoded are located. By this method, the accuracy of the probability distribution of bits to be encoded estimated by the image processing apparatus can be improved. Furthermore, the image processing device encodes the bits to be encoded based on the estimated probability distribution of the bits to be encoded, which has higher accuracy than that of the prior art, thereby improving the encoding efficiency.

In a possible implementation, if the plurality of binary bit strings are respectively used to represent coefficients in different subbands, the second bit string is a binary bit string in a second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices, which is located within a preset range of the first bit matrix. Or, if the plurality of binary bit strings are all used to represent coefficients in the first subband, the second bit string is a binary bit string in the first bit matrix, which is located within a preset range of the first bit string.

By the possible implementation manner, if a plurality of bit matrixes obtained after processing the image to be processed are aggregated bit matrixes, that is, a plurality of binary bit strings in the first bit matrix are all used for representing coefficients in the first sub-band. In this way, the speed at which the image processing apparatus acquires the context of the bit to be encoded is increased, thereby increasing the encoding speed.

In another possible implementation manner, the context of the bits to be coded further includes: and the bits in the third bit string and/or the fourth bit string are in the preset range corresponding to the bits to be coded. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein the content of the first and second substances,

if a plurality of binary bit strings in the first bit matrix are used to represent coefficients in different sub-bands, respectively, the third bit string is a binary bit string in the first bit matrix and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a binary bit string located within a preset range of the first bit string. Alternatively, the first and second electrodes may be,

if a plurality of binary bit strings in the first bit matrix are all used for representing the coefficients in the first subband, i.e. the plurality of bit matrices correspond to the plurality of subbands one to one, then the third bit string and the fourth bit string are both binary bit strings in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second subband is the subband to which the third bit matrix corresponds.

By the possible implementation manner, if a plurality of bit matrixes obtained after processing the image to be processed are aggregated bit matrixes, that is, a plurality of binary bit strings in the first bit matrix are all used for representing coefficients in the first sub-band. In this way, the speed at which the image processing apparatus acquires the context of the bit to be encoded is increased, thereby increasing the encoding speed.

In another possible implementation manner, before the "determining bits to be encoded", the method further includes: and acquiring a plurality of two-dimensional coefficient blocks corresponding to the image to be processed. Each of the plurality of two-dimensional coefficient blocks includes a plurality of coefficients that are coefficients in different sub-bands, respectively. Each coefficient in the plurality of two-dimensional coefficient blocks is binarized to obtain the plurality of bit matrices described above.

In another possible implementation, the "binarizing each coefficient of the plurality of two-dimensional coefficient blocks" includes: and binarizing each coefficient in the plurality of two-dimensional coefficient blocks according to a fixed-length binarization mode.

Through the two possible implementation manners, the image processing device can obtain a plurality of binary bit matrixes corresponding to the image to be processed. In this way, the image processing apparatus can obtain the context of the bits to be encoded in the binarized bit matrix and estimate the probability distribution of the bits to be encoded with higher accuracy than in the prior art based on the context. Thereby improving coding efficiency.

In another possible implementation manner, before the "acquiring a plurality of two-dimensional coefficient blocks corresponding to an image to be processed", the method further includes: acquiring an image to be processed, wherein the image to be processed comprises an image frame in a picture or a video. The step of acquiring a plurality of two-dimensional coefficient blocks corresponding to the image to be processed includes: and partitioning and transforming the image to be processed to obtain a plurality of two-dimensional coefficient blocks.

In another possible implementation manner, the determining bits to be encoded includes: and determining bits to be coded in a plurality of bit matrixes according to a preset coding sequence.

In another possible implementation manner, the "estimating the probability distribution of the bits to be coded based on the context of the bits to be coded" includes: and estimating the probability distribution of the bits to be coded through a probability estimation network based on the context of the bits to be coded.

In another possible implementation manner, the "encoding the bits to be encoded according to the probability distribution of the bits to be encoded" includes: and entropy coding the bits to be coded according to the probability distribution of the bits to be coded.

In a second aspect, the present application provides an image processing method, which is applied to an image processing apparatus. The method comprises the following steps: the position of the bit to be decoded is determined. The bit to be decoded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices corresponding to the image to be processed, and the first bit matrix is a three-dimensional bit matrix. Based on the context of the bits to be decoded, the probability distribution of the bits to be decoded is estimated. Here, the context of the bit to be decoded includes: and the bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string. Wherein the second bit string is used to represent another coefficient in the first subband. And decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed to obtain a plurality of bit matrixes corresponding to the image to be processed.

In one possible implementation, if the plurality of binary bit strings are used to represent coefficients in different sub-bands, respectively, the second bit string is a binary bit string in a second bit matrix. Here, the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. Or, if the plurality of binary bit strings are all used to represent coefficients in the first subband, the second bit string is a binary bit string in the first bit matrix, which is located within a preset range of the first bit string.

In another possible implementation manner, the context of the bits to be decoded further includes: and the bits in the preset range corresponding to the bits to be decoded in the third bit string and/or the fourth bit string. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein the content of the first and second substances,

if the plurality of binary bit strings are used to represent coefficients in different sub-bands, respectively, the third bit string is a binary bit string in the first bit matrix, and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a binary bit string located within a preset range of the first bit string. Alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are all used for representing the coefficients in the first subband, i.e. the plurality of bit matrices correspond to the plurality of subbands one to one, then the third bit string and the fourth bit string are both binary bit strings in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second subband is the subband to which the third bit matrix corresponds.

In another possible implementation manner, the method further includes: and inversely binarizing binary bit strings in the plurality of bit matrixes obtained by decoding to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrixes. Wherein each of the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients that are coefficients in different sub-bands, respectively. And carrying out inverse transformation on the plurality of two-dimensional coefficient blocks to obtain the image to be processed. The image to be processed comprises an image frame in a picture or a video.

In another possible implementation manner, the "inverse binarization of binary bit strings in the decoded bit matrices" includes: and according to a fixed-length inverse binarization mode, inversely binarizing binary bit strings in the plurality of bit matrixes obtained by decoding.

In another possible implementation manner, the "determining the position of the bit to be decoded" includes: and determining the positions of the bits to be decoded in a plurality of bit matrixes corresponding to the images to be processed according to a preset decoding sequence.

In another possible implementation manner, the "estimating the probability distribution of the bits to be decoded based on the context of the bits to be decoded" includes: based on the context of the bits to be decoded, the probability distribution of the bits to be decoded is estimated by a probability estimation network.

In another possible implementation manner, the "decoding bits to be decoded based on the position of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoding information of the image to be processed" includes: and entropy decoding the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed.

For the above advantages of the second aspect and any possible implementation manner thereof, reference may be made to the description of the advantages of the first aspect and any possible implementation manner thereof, and details are not described here.

In a third aspect, the present application provides an image processing apparatus.

In one possible design, the image processing apparatus is configured to perform any one of the methods provided in the first aspect or the second aspect. The present application may divide the functional blocks of the image processing apparatus according to any one of the methods provided in the first aspect or the second aspect. For example, the functional blocks may be divided for the respective functions, or two or more functions may be integrated into one processing block. For example, the present application may divide the image processing apparatus into a determination unit, an estimation unit, an encoding unit, and the like according to the function, or the present application may divide the image processing apparatus into a determination unit, an estimation unit, a decoding unit, and the like according to the function. The descriptions of possible technical solutions and beneficial effects executed by the divided functional modules may refer to the technical solution provided by the first aspect or the corresponding possible design thereof, or may refer to the technical solution provided by the second aspect or the corresponding possible design thereof, and are not described herein again.

In another possible embodiment, the image processing apparatus includes: the memory is coupled to the one or more processors. The memory is adapted to store computer instructions which the processor is adapted to invoke to perform any of the methods as provided in the first aspect and any of its possible designs, or to perform any of the methods as provided in the second aspect and any of its possible designs.

In a fourth aspect, the present application provides a computer-readable storage medium, such as a computer non-transitory readable storage medium. Having stored thereon a computer program (or instructions) which, when run on an image processing apparatus, causes the image processing apparatus to perform any of the methods provided by any of the possible implementations of the first aspect or the second aspect described above.

In a fifth aspect, the present application provides a computer-readable storage medium having encoded information stored thereon, the encoded information being obtained according to any one of the image processing methods provided in any one of the possible implementations of the first aspect.

In a sixth aspect, the present application provides a computer program product which, when run on an image processing apparatus, causes any one of the methods provided by any one of the possible implementations of the first aspect or the second aspect to be performed.

In a seventh aspect, the present application provides a chip system, including: and the processor is used for calling and running the computer program stored in the memory from the memory and executing any one of the methods provided by the implementation modes in the first aspect or the second aspect.

It is understood that any one of the apparatuses, computer storage media, computer program products, or chip systems provided above can be applied to the corresponding methods provided above, and therefore, the beneficial effects achieved by the apparatuses, the computer storage media, the computer program products, or the chip systems can refer to the beneficial effects in the corresponding methods, and are not described herein again.

In the present application, the names of the above-mentioned image processing apparatuses do not limit the devices or functional modules themselves, and in actual implementation, these devices or functional modules may appear by other names. Insofar as the functions of the respective devices or functional modules are similar to those of the present application, they fall within the scope of the claims of the present application and their equivalents.

These and other aspects of the present application will be more readily apparent from the following description.

Drawings

Fig. 1 is a schematic diagram of an image processing system according to an embodiment of the present application;

fig. 2 is a schematic diagram of an image encoding apparatus and an image decoding apparatus applied to a terminal device according to an embodiment of the present application;

fig. 3 is a hardware configuration diagram of an image processing apparatus according to an embodiment of the present application;

fig. 4 is a first flowchart illustrating an image processing method according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram of an image partition provided in an embodiment of the present application;

FIG. 6 is a diagram of a two-dimensional coefficient block according to an embodiment of the present application;

fig. 7 is a schematic diagram of a three-dimensional bit matrix provided in an embodiment of the present application;

fig. 8 is a schematic diagram of a coding sequence of a plurality of bit matrices according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a binary bit string provided in an embodiment of the present application;

fig. 10 is a schematic diagram of a preset range of a first bit matrix according to an embodiment of the present application;

FIG. 11 is a first diagram illustrating a context model provided by an embodiment of the present application;

fig. 12 is a schematic diagram of a preset range of a first bit string provided in an embodiment of the present application;

FIG. 13a is a diagram illustrating a context model according to an embodiment of the present application;

fig. 13b is a first schematic diagram illustrating estimating a probability distribution of bits to be encoded according to an embodiment of the present application;

fig. 13c is a schematic diagram illustrating a second method for estimating a probability distribution of bits to be encoded according to an embodiment of the present disclosure;

fig. 14 is a schematic diagram of a sub-tape polymerization method according to an embodiment of the present disclosure;

fig. 15 is a flowchart illustrating a second image processing method according to an embodiment of the present application;

fig. 16 is a first schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure;

fig. 17 is a second schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 18 is a schematic structural diagram of a chip system according to an embodiment of the present disclosure;

fig. 19 is a schematic structural diagram of a computer program product according to an embodiment of the present application.

Detailed Description

In the embodiments of the present application, words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the embodiments of the present application, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless otherwise specified.

The term "at least one" in this application means one or more, and the term "plurality" in this application means two or more, for example, the plurality of second messages means two or more second messages. The terms "system" and "network" are often used interchangeably herein.

It is to be understood that the terminology used in the description of the various described examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various described examples and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "and/or" is an associative relationship that describes an associated object, meaning that three relationships may exist, e.g., A and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present application generally indicates that the former and latter related objects are in an "or" relationship.

It should also be understood that, in the embodiments of the present application, the size of the serial number of each process does not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

It should be understood that determining B from a does not mean determining B from a alone, but may also be determined from a and/or other information.

It will be further understood that the terms "comprises," "comprising," "includes," and/or "including," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also understood that the term "if" may be interpreted to mean "when" ("where" or "upon") or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined." or "if [ a stated condition or event ] is detected" may be interpreted to mean "upon determining.. or" in response to determining. "or" upon detecting [ a stated condition or event ] or "in response to detecting [ a stated condition or event ]" depending on the context.

It should be appreciated that reference throughout this specification to "one embodiment," "an embodiment," "one possible implementation" means that a particular feature, structure, or characteristic described in connection with the embodiment or implementation is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "one possible implementation" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

The embodiment of the application provides an image processing method, which can be used for improving the encoding efficiency of an image, thereby reducing the storage resource consumed by storing the image (or video) and reducing the transmission bandwidth consumed by transmitting the image (or video). The image processing method may include an image encoding method and an image decoding method.

The embodiment of the application also provides an image processing device which can comprise an image coding device and an image decoding device. The image encoding apparatus may be configured to execute the image encoding method described above, and the image decoding apparatus may be configured to execute the image decoding method described above.

Referring to fig. 1, an embodiment of the present application further provides an image processing system 10. As shown in fig. 1, the image processing system 10 includes an image encoding device 101 and an image decoding device 102. The image encoding apparatus 101 is configured to encode a bit matrix corresponding to an image to be processed to obtain an encoded code stream (which may also be referred to as a compressed code stream or a bit stream). The encoded code stream may be transmitted from the image encoding apparatus 101 to the image decoding apparatus 102 by a wired or wireless transmission method. The image decoding device 102 is configured to decode the received encoded code stream to obtain a bit matrix corresponding to the image to be processed, and restore the image to be processed based on the bit matrix. For a description of a bit matrix corresponding to an image to be processed, please refer to the following, which is not described herein.

The image encoding device and the image decoding device may be applied to a terminal device, and the terminal device may be a portable device such as a mobile phone, a tablet computer, and a wearable electronic device, may also be a device such as a vehicle-mounted device and an intelligent robot, and may also be a computing device such as a Personal Computer (PC), a Personal Digital Assistant (PDA), a netbook, and a server, which are not limited thereto.

The image encoding apparatus and the image decoding apparatus described above can be applied to a wireless device or a core network device that requires image transcoding.

For example, the image encoding apparatus provided in the embodiment of the present application may be an image encoder in the terminal device, the wireless device, or the core network device. The image decoding apparatus provided in the embodiment of the present application may be an image decoder in the terminal device, the wireless device, or the core network device. This is not limitative.

In the embodiment of the present application, a terminal device, a wireless device, or a core network device, in which an image encoding apparatus is deployed, is referred to as an encoding end device. A terminal device, a wireless device, or a core network device, in which the image decoding apparatus is disposed, is referred to as a decoding-end device. It is to be understood that the encoding side device and the decoding side device may be the same device or different devices, and are not limited thereto.

It should be understood that the encoding end device may further include a channel encoding device (e.g., a channel encoder), and the decoding end device may further include a channel decoding device (e.g., a channel decoder), which is not limited thereto. Wherein, the channel coding device can be used for carrying out channel coding on the image signal, and the channel decoding device can be used for carrying out channel decoding on the image signal.

Referring to fig. 2, fig. 2 is a schematic diagram of an application of the image encoding apparatus and the image decoding apparatus provided in the embodiment of the present application to a terminal device. As shown in fig. 2, the first terminal device 20 may include: an image coding apparatus 201 and a channel coding apparatus 202. The second terminal device 21 may include: image decoding apparatus 211 and channel decoding apparatus 212. The first terminal device 20 is connected with a wireless or wired first network communication device 22, the first network communication device 22 and a wireless or wired second network communication device 23 are connected through a digital channel, and the second terminal device 21 is connected with the wireless or wired second network communication device 23. The wireless or wired network communication device may be generally referred to as a signal transmission device, such as a communication base station, a data exchange device, and the like, which is not limited in this respect.

Referring to fig. 3, the embodiment of the present application provides a hardware configuration diagram of an image processing apparatus 30. The image processing device 30 may be the above-described image encoding device, or may be the above-described image decoding device. As shown in fig. 3, the image processing apparatus 30 includes a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, the memory 302, and the communication interface 303 may be connected by a bus 304.

The processor 301 is a control center of the image processing apparatus 30, and may be a Central Processing Unit (CPU), another general-purpose processor, or the like. Wherein a general purpose processor may be a microprocessor or any conventional processor or the like.

By way of example, processor 301 may include one or more CPUs, such as CPU 0 and CPU 1 shown in fig. 3.

The memory 302 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

In one possible implementation, the memory 302 may exist independently of the processor 301. A memory 302 may be coupled to the processor 301 through a bus 304 for storing data, instructions, or program code. The processor 301 can implement the image processing method provided by the embodiment of the present application when calling and executing the instructions or program codes stored in the memory 302.

In another possible implementation, the memory 302 may also be integrated with the processor 301.

A communication interface 303, configured to connect the image processing apparatus 30 with other devices (such as a wireless or limited first network communication device in fig. 2) through a communication network, where the communication network may be an ethernet, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), or the like. The communication interface 303 may include a receiving unit for receiving data, and a transmitting unit for transmitting data.

The bus 304 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an extended ISA (enhanced industry standard architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.

It is to be noted that the configuration shown in fig. 3 does not constitute a limitation of the image processing apparatus, and the image processing apparatus 30 may include more or less components than those shown in the drawing, or combine some components, or a different arrangement of components, in addition to the components shown in fig. 3.

The following describes an image processing method provided in an embodiment of the present application with reference to the drawings.

Referring to fig. 4, fig. 4 is a flowchart illustrating an image processing method according to an embodiment of the present application, where the method can be applied to an image encoding device in an image processing device. For convenience of description, the image encoding apparatus will be hereinafter abbreviated as an encoding apparatus. The method can comprise the following steps:

and S101, acquiring an image to be processed.

The image to be processed may be the whole original image, or may be a rectangular region in the whole original image, or a region with another shape in the whole original image, which is not limited in this respect.

Optionally, the encoding device may obtain a picture from a local or network and use the obtained picture as the original picture, or the encoding device obtains a video from a local or network and uses an image frame in the video as the original image, which is not limited herein.

It can be seen that when the encoding apparatus continuously encodes the image frames in the video, the encoding of the video is achieved.

And S102 (optional), dividing the image to be processed into a plurality of image blocks.

The encoding device may divide the image to be processed into a plurality of image blocks according to a preset image blocking rule.

Optionally, the encoding apparatus may equally divide the image to be processed into a plurality of image blocks according to a preset size. For example, the encoding device divides the image to be processed into T × S image blocks on average in size M × N. M, N, T, S are positive integers, and M and N may be the same or different. It can be seen that, when the values of T and S are both 1, the size of the image to be processed is M × N, i.e., the image to be processed does not need to be partitioned.

As shown in FIG. 5, T is 2, S is 4, and M and N are both 4. If the image 50 in fig. 5 is to be processed, the size of the image 50 is 8 × 16. The decoding apparatus may divide the image 50 into 2 × 4 image blocks with a size of 4 × 4 on average, and any image block 501 of the 2 × 4 image blocks is 4 × 4 in size.

It can be understood that the image block in the embodiment of the present application may be obtained by the encoding device partitioning the image to be processed, or may be transmitted to the encoding device after the image to be processed is partitioned into image blocks by other image processing devices (or other image processing apparatuses), which is not limited to this. The image blocks may also be referred to as sub-images, or sub-image blocks, etc.

And S103, acquiring a plurality of two-dimensional coefficient blocks corresponding to the image blocks.

Wherein any one of the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients. Each position in one two-dimensional coefficient block may represent a different sub-band (hereinafter, the sub-band is simply referred to as a sub-band in the embodiments of the present application). For a two-dimensional coefficient block, the coefficients included in the two-dimensional coefficient block are coefficients in different sub-bands, i.e., the coefficients and the sub-bands are in one-to-one correspondence. For example, if a two-dimensional coefficient block includes D × K coefficients, the D × K coefficients are coefficients in D × K different subbands, respectively, and the D × K coefficients and the D × K subbands correspond one-to-one. Wherein D and K are both positive integers. It will be appreciated that if the first sub-band is any one of the D x K sub-bands, then the plurality of two-dimensional coefficient blocks each include coefficients in one of the first sub-bands.

Illustratively, referring to FIG. 6, FIG. 6 shows a schematic diagram of a two-dimensional coefficient block 60. Therein, the two-dimensional coefficient block 60 includes 4 × 4 coefficients, and the 4 × 4 coefficients may be coefficients in 4 × 4 sub-bands (16 sub-bands in total). For example, the coefficient y (0,0) may be a coefficient in the subband f1, the coefficient y (0, 3) may be a coefficient in the subband f4, the coefficient y (3, 0) may be a coefficient in the subband f13, the coefficient y (3, 3) may be a coefficient in the subband f16, and so on. In the 4 × 4 coefficients, each coefficient may correspond to one subband, i.e., 4 × 4 coefficients correspond to 4 × 4 subbands.

Specifically, in a possible implementation manner, the encoding apparatus may transform the plurality of image blocks respectively, so as to obtain a plurality of two-dimensional coefficient blocks. For any image block of the plurality of image blocks, the encoding device transforms the image block to obtain one two-dimensional coefficient block. I.e. one image block corresponds to one two-dimensional coefficient block.

In another possible implementation manner, the encoding apparatus may transform the residual blocks corresponding to the plurality of image blocks, respectively, so as to obtain a plurality of two-dimensional coefficient blocks.

Specifically, for any one of the image blocks, the encoding apparatus may perform prediction on the image block to obtain a prediction block corresponding to the image block. Then, the encoding apparatus determines a residual block corresponding to the image block by calculating a difference between the image block and the prediction block. That is, one image block corresponds to one residual block, the embodiment of the present application obtains a corresponding prediction block for the encoding apparatus according to the plurality of image blocks, and determines a specific process of the residual block according to the image block and the prediction block, which is not described herein again.

Then, the encoding apparatus transforms the determined plurality of residual blocks to obtain a plurality of two-dimensional coefficient blocks. For any one of the plurality of residual blocks, the encoding apparatus transforms the residual block to obtain a two-dimensional coefficient block. I.e. one residual block corresponds to one two-dimensional coefficient block.

As an example, a process of obtaining a plurality of two-dimensional coefficient blocks by performing Discrete Cosine Transform (DCT) on each of the plurality of image blocks by an encoding apparatus will be described below.

Specifically, the encoding device may perform DCT transform on the plurality of image blocks with corresponding sizes according to the size of each of the plurality of image blocks, so as to obtain a plurality of two-dimensional coefficient blocks with corresponding sizes.

For example, if the size of each of the plurality of image blocks is 4 × 4, the encoding apparatus may perform 4 × 4 DCT transformation on each of the plurality of image blocks to obtain a plurality of two-dimensional coefficient blocks of 4 × 4 size.

For example, the encoding apparatus may perform 4 × 4 DCT transformation on a 4 × 4 image block X according to the following formula (1), so as to obtain a 4 × 4 transform coefficient matrix Y, i.e., the above two-dimensional coefficient block.

Formula (1):

wherein A and A^TIs a coefficient of a DCT transform, and A^TAre inverse matrixes of each other, a, b and c are A and A respectively^TCoefficient (2) of (1). y is_(0,0)To y_(3,3)Representing 16 coefficients in the transform coefficient matrix Y, the 16 coefficients being coefficients in 16 subbands, i.e. one subband for each coefficient and 16 different subbands for 16 coefficients.

It can be seen that 4 × 4 image blocks, after being transformed, result in a two-dimensional coefficient block comprising 4 × 4 coefficients. Therefore, if the size of each of the image blocks corresponding to the image to be processed is M × N, the encoding device transforms the image blocks to obtain each of the two-dimensional coefficient blocks, which each include M × N coefficients.

It can be seen that the encoding apparatus transforms the image block or the residual block, that is, transforms the information of the image block or the information of the residual block from the spatial domain to the frequency domain, and represents the information of the image block or the information of the residual block by different subbands and coefficients in different subbands in the frequency domain.

It should be understood that, in the embodiment of the present application, the above-mentioned multiple two-dimensional coefficient blocks may be obtained by transforming an image block or a residual block by an encoding apparatus, or may be obtained by transforming a graphic block or a residual block by another image processing apparatus (or another image processing device) to obtain multiple two-dimensional coefficient blocks, and then transmitting the multiple two-dimensional coefficient blocks to the encoding apparatus, which is not limited herein.

And S104, binarizing each coefficient in the obtained plurality of two-dimensional coefficient blocks to obtain a plurality of bit matrixes.

Specifically, the encoding apparatus may binarize each coefficient in the acquired plurality of two-dimensional coefficient blocks in a preset binarization manner to obtain a plurality of bit matrices.

Alternatively, the encoding apparatus may binarize each coefficient in the acquired plurality of two-dimensional coefficient blocks in a fixed-length binarization manner to obtain a plurality of bit matrices.

Wherein any one of the bit matrices (for example, the first bit matrix in the embodiment of the present application) is a three-dimensional bit matrix, and the first bit matrix includes a plurality of binary bit strings for representing a plurality of coefficients in a two-dimensional coefficient block (for simplicity of description, the "binary bit string" is hereinafter referred to simply as "bit string").

As an example, the encoding apparatus may binarize each coefficient in the acquired plurality of two-dimensional coefficient blocks into a bit string including 8 bits. Referring to fig. 6, if the value of the coefficient y (0,0) is 1, the coefficient y (0,0) may be binarized into a bit string of 00000001, i.e., the coefficient y (0,0) is represented by the bit string 00000001. If the value of the coefficient y (0, 3) is 128, the coefficient y (0, 3) may be binarized into a bit string of 10000000, i.e., the coefficient y (0, 3) is represented by the bit string of 10000000. If the value of the coefficient y (3, 3) is 255, the coefficient y (3, 3) can be binarized into a bit string of 11111111, i.e., the coefficient y (3, 3) can be represented by the bit string 11111111.

For a clearer understanding of the bit matrix in the embodiment of the present application, a bit matrix obtained by binarizing each coefficient in a two-dimensional coefficient block including 4 × 4 coefficients by the encoding apparatus will be described as an example. Referring to fig. 7, fig. 7 is a schematic diagram of a bit matrix 72 obtained after 8-bit fixed-length binarization of each coefficient in the two-dimensional coefficient block 60 including 4 × 4 coefficients in fig. 6.

As shown in fig. 7, the encoding apparatus binarizes each coefficient in the two-dimensional coefficient block 60 in 8-bit fixed length, i.e., one coefficient is represented by a bit string including 8 bits (bits are represented by black dots in fig. 7). As an example, the coefficient y (3, 0) in the two-dimensional coefficient block 60 may be represented by 8 bits in the bit string 73, and the coefficient y (0, 3) in the two-dimensional coefficient block 60 may be represented by 8 bits in the bit string 74. Thus, the 16 coefficients in the two-dimensional coefficient block 60 may correspond to 16 bit strings, and the 16 bit strings form a bit matrix 72 as shown in fig. 7, where the bit matrix 72 includes 4 × 8 (i.e., 128) bits.

It can be seen that, if the two-dimensional coefficient block includes M × N coefficients, when the encoding apparatus binarizes the two-dimensional coefficient block in a fixed length of P bits, the resulting bit matrix includes a bit string with a length of P and a number of M × N. Wherein P is a positive integer.

In the embodiment of the present application, the same position in a plurality of bit strings is referred to as a bit plane (bit plane). Therefore, bits located at the same position in the bit strings are bits of the same bit plane. For example, if each of the bit strings includes P bits, the P-th bit in each of the bit strings is a bit of the same bit plane. Wherein P is a positive integer, and 1. ltoreq. p.ltoreq.P.

Illustratively, as shown in fig. 7, plane 75 schematically illustrates one bit plane 75 in bit matrix 72. In the bit plane 75, the bit in the third bit among the 16 bit strings in the bit matrix 72 is included. For example, bit 731 in the third bit in bit string 73 and bit 741 in the third bit in bit string 74.

In addition, as can be seen from the above description, for any one of the two-dimensional coefficient blocks, the coefficients included in the two-dimensional coefficient block are coefficients in different sub-bands, that is, the coefficients correspond to the sub-bands one by one, and each of the two-dimensional coefficient blocks includes a coefficient in a first sub-band.

Thus, in one case, the encoding device may directly binarize the plurality of two-dimensional coefficient blocks to obtain a plurality of bit matrices. For any one of the bit matrices (e.g., the first bit matrix), the first bit matrix includes a plurality of bit strings for representing coefficients in different subbands. That is, for the first bit matrix, a plurality of binary bit strings included in the first bit matrix are respectively used to represent coefficients in different subbands. For example, if the two-dimensional coefficient block for obtaining the first bit matrix includes M × N coefficients, the M × N coefficients are coefficients in M × N subbands, respectively, that is, the M × N coefficients correspond to the M × N subbands one to one. Then, the first bit matrix includes M × N bit strings, and the M × N bit strings are respectively used to represent coefficients in M × N subbands. For example, referring to FIG. 7, bit string 73 may be used to represent coefficient y (3, 0) in sub-band f13, and bit string 74 may be used to represent coefficient y (0, 3) in sub-band f 4. That is, the plurality of bit strings in the first bit matrix correspond to the plurality of subbands one to one. Further, each of the plurality of bit matrices including the first bit matrix includes a bit string for representing a coefficient in the first subband.

In another case, the encoding apparatus may perform sub-band aggregation on the plurality of two-dimensional coefficient blocks before binarizing the plurality of two-dimensional coefficient blocks, so that a plurality of coefficients included in one two-dimensional coefficient block are each a coefficient in one sub-band, and coefficients in different two-dimensional coefficient blocks are coefficients in different sub-bands. Then, the encoding apparatus binarizes the two-dimensional coefficient blocks subjected to the aggregation processing to obtain bit matrices. For any one bit matrix (e.g., a first bit matrix) of the bit matrices, the first bit matrix includes a plurality of bit strings for representing coefficients in the same subband (e.g., a first subband), and bit strings in different bit matrices are for representing coefficients in different subbands. That is, one bit matrix corresponds to one subband, and a plurality of bit matrices correspond one-to-one to a plurality of subbands.

Or, the encoding device binarizes the two-dimensional coefficient blocks to obtain a plurality of initial bit matrices. Then, the encoding apparatus may perform subband aggregation on the plurality of initial bit matrices, so that one bit matrix (e.g., the first bit matrix) after aggregation includes a plurality of bit strings for representing coefficients in the same subband (e.g., the first subband). Bit strings in different bit matrices for representing coefficients in different sub-band sub-bands. That is, one bit matrix corresponds to one subband, and a plurality of bit matrices correspond one-to-one to a plurality of subbands.

Specifically, the specific process of performing subband aggregation on the multiple two-dimensional coefficient blocks and performing subband aggregation on the multiple initial bit matrices by the encoding apparatus may refer to the following, which is not described herein again.

And S105, determining bits to be coded in the bit matrixes according to a preset coding sequence.

Specifically, for a plurality of bit matrices, the coding order of the plurality of bit matrices may be: the bit matrices are encoded in a zigzag (zig-zag) order, or encoded in other orders, such as row-by-row, column-by-column, diagonal, horizontal-reverse, and vertical-reverse.

Referring to fig. 8, fig. 8 illustrates an encoding order diagram of a plurality of bit matrices. As shown in fig. 8, fig. 8 shows a top view of a 4 x 4 bit matrix, where each small square represents a top view of one bit matrix. As shown in fig. 8 (a), the encoding apparatus may sequentially encode the 4 × 4 bit matrices from the bit matrix 81 or the bit matrix 82 shown in fig. 8 (a) according to the zigzag trace indicated by the bold black line shown in fig. 8 (a). Alternatively, the encoding apparatus may sequentially encode the 4 × 4 bit matrices in a row-by-row track indicated by a bold black line shown in fig. 8 (a), starting from the bit matrix 81 or the bit matrix 82 shown in fig. 8 (b).

For different bit strings, the coding order of the different bit strings may be: different bit strings are encoded according to the frequency of the sub-band corresponding to the coefficient represented by each bit string. For example, a bit string representing a coefficient in a low frequency subband is encoded first, and a bit string representing a coefficient in a second low frequency subband is encoded, for example, without limitation.

For a single bit string, the coding order of the single bit string may be: the bits in a bit string are encoded according to where each bit in the bit string is located. For example, the lowest order bit in the bit string is encoded first, and the next lowest order bit in the bit string is encoded. Alternatively, the bit of the highest order in the bit string is encoded first, and the bit of the next highest order in the bit string is encoded second. The embodiments of the present application do not limit this.

Referring to fig. 9, the bit string 90 shown in fig. 9 includes 8 bits. The least significant bit in the bit string 90 is bit 0, the second least significant bit in the bit string 90 is bit 1, and so on, the most significant bit in the bit string 90 is bit 7. Therefore, when the encoding device encodes the bit string 90, the bit 0 of the lowest order bit can be encoded first, and the bits in the bit string 90 can be encoded one by one from the lower order bit to the upper order bit. Of course, the encoding device may encode the bit 7 of the highest order first, and encode the bits in the bit string 90 one by one sequentially from the higher order to the lower order. This is not limitative.

Based on the encoding order of the bit matrices, the encoding order of different bit strings, and the encoding order of a single bit string, several preset encoding orders for encoding each bit in the bit matrices are exemplarily listed below.

One possible pre-set coding order is: among the plurality of bit matrices, the encoding apparatus determines a bit string 11 corresponding to a subband having the lowest frequency in the bit matrix 1, starting with a first bit matrix (e.g., bit matrix 1), and encodes the least significant bit 11 of the bit string 11. Next, the encoding apparatus determines the bit matrix 2 to be encoded next according to the encoding order of the bit matrices (e.g., sub-encoding, line-by-line encoding, column-by-column encoding, etc.), determines the bit string 21 corresponding to the subband with the lowest frequency in the bit matrix 2, and encodes the bit 21 with the lowest order in the bit string 21. By analogy, after the encoding device has encoded the lowest bit in the bit string corresponding to the lowest-frequency subband in all bit matrices, the encoding device encodes the bit 12 of the next lowest bit in the bit string 11 corresponding to the lowest-frequency subband in the bit matrix 1, and then encodes the bit 22 of the next lowest bit in the bit string 11 corresponding to the lowest-frequency subband in the bit matrix 2. In this way, the encoding apparatus encodes the bits of the second lowest order in the bit string corresponding to the subband having the lowest frequency among all the bit matrices one by one. By analogy, when the encoding apparatus finishes encoding all bits in the bit string corresponding to the subband with the lowest frequency in all bit matrices, the encoding apparatus starts encoding the bits in the bit string 12 corresponding to the subband with the second lowest frequency in the bit matrix 1. In this way, the encoding apparatus can encode all bits in the bit string corresponding to all subbands in all bit matrices.

Another possible preset coding order is: the encoding apparatus determines a bit string 11 corresponding to a subband having the lowest frequency in the bit matrix 1, starting with a first bit matrix (e.g., bit matrix 1), encodes the lowest bit 11 in the bit string 11, and then encodes the next lower bit 12 in the bit string 11. By analogy, after the encoding device has encoded all bits in the bit string 11, the encoding device determines the bit matrix 2 to be encoded next according to the encoding order of the bit matrices (e.g., sub-encoding, line-by-line encoding, column-by-column encoding, etc.), determines the bit string 21 corresponding to the subband with the lowest frequency in the bit matrix 2, encodes the bit 21 with the lowest order in the bit string 21, and then encodes the bit 22 with the next lower order in the bit string. And so on until the encoding apparatus has encoded all the bits in the bit string 21. In this way, the encoding apparatus encodes all bits in the bit string corresponding to the subband having the lowest frequency among all bit matrices one by one. And in the same way, after the encoding device has encoded all the bits in the bit string corresponding to the sub-band with the lowest frequency in all the bit matrices, the encoding device encodes the bits in the bit string corresponding to the sub-band with the next lowest frequency in the bit matrix 1, then encodes the bits in the bit string corresponding to the sub-band with the next lowest frequency in the bit matrix 2, and so on until the encoding device has encoded all the bits in the bit string corresponding to all the sub-bands in all the bit matrices.

Yet another possible pre-set coding order is: the encoding apparatus starts with a first bit matrix (e.g., bit matrix 1), determines a bit string 11 corresponding to the subband with the lowest frequency in bit matrix 1, encodes the lowest bit 11 in bit string 11, and then encodes the next lower bit 12 in bit string 11 until the encoding apparatus has encoded all the bits in bit string 11. Next, the encoding apparatus determines a bit sequence 12 corresponding to the sub-band having the second lowest frequency in the bit matrix 1, encodes the least significant bit 11 in the bit sequence 12, and then encodes the second lowest significant bit 12 in the bit sequence 12 until the encoding apparatus has encoded all the bits in the bit sequence 12. By analogy, after the encoding device has encoded all bits in the bit matrix 1, the next bit matrix 2 to be encoded is determined according to the encoding sequence of the bit matrices (e.g., sub-encoding, line-by-line encoding, column-by-column encoding, etc.), and all bits in the bit matrix 2 are encoded until the encoding device has encoded all bits in bit strings corresponding to all subbands in all bit matrices.

It should be noted that the above several possible preset coding orders are only exemplary illustrations, and do not constitute a limitation on the preset coding order in the embodiments of the present application.

Based on the preset encoding order, the encoding apparatus determines the current bit to be encoded from the plurality of bit matrices determined in S104. For convenience of description, the bit to be encoded determined by the encoding apparatus in the following text is any one bit in a first bit string in the first bit matrix, the first bit string is used for representing a coefficient in the first subband, and the first bit matrix is any one bit matrix in the plurality of bit matrices.

And S106, acquiring the context of the bit to be coded.

Specifically, the context of the bits to be coded may include: and the bits in the preset range corresponding to the bits to be coded in the first bit string and/or the second bit string. Here, the second bit string is used to represent another coefficient in the first subband.

It is to be understood that the context of the bits to be encoded may be all bits in the first bit string and/or the second bit string within the preset range corresponding to the bits to be encoded, or may be a part of the bits in the first bit string and/or the second bit string within the preset range corresponding to the bits to be encoded, which is not limited herein.

If the context of the bit to be coded is a part of bits in the first bit string and/or the second bit string within a preset range corresponding to the bit to be coded, the coding apparatus may directly determine the position of each bit included in the context of the bit to be coded, without determining the position of each bit in the context through the preset range corresponding to the bit to be coded.

For the first bit string, the preset range corresponding to the bit to be coded may be a range having a preset distance from the position of the bit to be coded. For the second bit string, the preset range corresponding to the bit to be encoded may be a range that is based on a bit position of one bit plane located with the bit to be encoded and is a preset distance away from the bit position.

Alternatively, the preset distance may be measured by the number of bits. As an example, the position having the preset distance from the position of the bit to be coded may be a position having j bits from the position of the bit to be coded. In this way, the range with the preset distance from the position of the bit to be coded represents the range between the position with j bits away from the position of the bit to be coded and the position of the bit to be coded. Where j is a positive integer. In this way, the bits in the preset range corresponding to the bits to be coded, namely, the bits included between the position j bits away from the position of the bits to be coded and the position of the bits to be coded.

For example, the preset range corresponding to the bit to be coded in the first bit string is taken as an example for explanation.

The preset range corresponding to the bit to be coded in the first bit string may be a range with a preset distance between two sides with the position of the bit to be coded as the center. Alternatively, the preset range corresponding to the bit to be coded in the first bit string may be a range that takes the position of the bit to be coded as an end point and is a preset distance away from the position of the bit to be coded, and this is not limited. On the premise that the preset range does not exceed one bit matrix, the value of the preset distance is not specifically limited in the embodiment of the present application.

Referring to fig. 9, (a) in fig. 9 shows a preset range corresponding to a bit to be coded in the first bit string, which is a schematic diagram of ranges with a preset distance between two sides with a position of the bit to be coded as a center. As shown in (a) of fig. 9, the bit string 90 is a first bit string, and the bit to be coded is bit 4 in the bit string 90, the preset range corresponding to the bit to be coded may be a range centered on the position of bit 4, spaced apart from the position of bit 4 by L1 upward, and spaced apart from the position of bit 4 by L2 downward. As shown in fig. 9 (a), bit 3 and bit 5 are included in this range. Bit 3 and bit 5 are context bits of bit 4 to be coded in the bit string 90. It is to be understood that the values of L1 and L2 may be the same or different, and are not limited thereto.

Referring to fig. 9, (b) in fig. 9 shows a preset range corresponding to a bit to be coded in the first bit string, which is a range that has an end point at the position of the bit to be coded and is a preset distance away from the position of the bit to be coded. As shown in (b) of fig. 9, the bit string 90 is a first bit string, and the bit to be encoded is bit 4 in the bit string 90, the preset range corresponding to the bit to be encoded may be a range which is located at the position of bit 4 as an end point and is located at a distance L3 from the position of bit 4 downward. As shown in fig. 9 (b), bit 5 and bit 6 are included in this range. Then bit 5 and bit 6 are the context bits of bit 4 to be coded in the bit string 90. Of course, the preset range corresponding to the bit to be encoded may also be a range that is located at bit 4 as an end point and is separated from the position of bit 4 by L3 upward, which is not limited herein.

In the first case, if a plurality of bit strings in the first bit matrix are respectively used to represent coefficients in different subbands, the second bit string is a bit string in the second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices located within a preset range of the first bit matrix.

Among the bit matrices, the bit matrix located in the preset range of the first bit matrix may include a bit matrix adjacent to the first bit matrix, or may include a bit matrix whose distance from the first bit matrix is smaller than a preset distance, or may include a bit matrix in a preset area with the first bit matrix as a base point, which is not limited herein. It is easily understood that the number of bit matrices located within the first bit matrix preset range is at least one. Therefore, the number of the second bit matrices may be one or more. Accordingly, the second bit string may be one bit string or a plurality of bit strings.

Alternatively, the distance between the first bit matrices described above can be measured by the number of bit matrices. As an example, the bit matrix spaced apart from the first bit matrix by a preset distance may be a bit matrix spaced apart from the first bit matrix by q bit matrices. Thus, the bit matrix having a distance from the first bit matrix that is less than the predetermined distance, i.e., the bit matrix representing the predetermined distance from the first bit matrix, and the q bit matrices spaced apart from the first bit matrix. Wherein q is a positive integer.

Referring to fig. 10, fig. 10 is a schematic diagram illustrating that the preset range of the first bit matrix is a preset region with the first bit matrix as a base point. As shown in fig. 10, fig. 10 shows a top view of a 4 x 4 bit matrix, where each small square is used to represent a top view of one bit matrix. As shown in fig. 10 (a), if the bit matrix 101 is a first bit matrix, the dashed box area 1001 indicates a predetermined range of the first bit matrix in the 4 × 4 bit matrices. It can be seen that the preset range of the first bit matrix includes bit matrix 102, bit matrix 103, bit matrix 104, bit matrix 105, bit matrix 106, bit matrix 107, bit matrix 108, and bit matrix 109, for a total of 8 bit matrices. Alternatively, as shown in (b) of fig. 10, if the bit matrix 101 is a first bit matrix, the dashed box area 1002 in the 4 × 4 bit matrices represents a preset range of the first bit matrix, and in this case, the preset range of the first bit matrix includes bit matrices 102, 103, 104, 105, 106, 107, 108, 109, 110, and 111, and bit matrices 112, which total 11 bit matrices. Therefore, the second bit matrix may be any one of the 8 bit matrices or the 11 bit matrices, or both the 8 bit matrices or the 11 bit matrices are the second bit matrices.

It is understood that when extracting bits within a predetermined range corresponding to bits to be encoded from bit matrices within a predetermined range of a first bit matrix, bits within the predetermined range corresponding to the bits to be encoded in bit strings representing the first sub-band, a context model of the bits to be encoded can be constructed.

Referring to fig. 11, fig. 11 shows a schematic diagram of a context model 110 of bits to be encoded 1111. Wherein the bit string 111 represents a bit string in the first bit string that is within a preset range corresponding to the bits to be encoded. The bit strings in the context model 110, except the bit string 111, are all bit strings in the second bit string that are within a preset range corresponding to the bits to be encoded. It can be seen that the context model 110 comprises 8 second bit strings, it can be seen that the number of second bit matrices is also 8.

In the second case, the second bit string is a bit string of the first bit matrix different from the first bit string if a plurality of binary bit strings in the first bit matrix are each used to represent a coefficient in the first subband. The second bit string is a bit string in the first bit matrix, which is located within a preset range of the first bit string.

In the first bit matrix, the bit string located in the preset range of the first bit string may include a bit string adjacent to the first bit string, or may include a bit string whose distance from the first bit string is smaller than a preset distance, or may include a bit string in a preset area with the first bit string as a base point, which is not limited in this respect. It is easily understood that the number of bit strings located within the first bit string preset range is at least one. Therefore, the second bit string may be one bit string or a plurality of bit strings.

Alternatively, the distance between the first bit strings described above can be measured by the number of bit strings. As an example, the bit string having a preset distance from the first bit string may be a bit string having r bit strings apart from the first bit string. Thus, a bit string having a distance from the first bit string smaller than the preset distance means a bit string having a preset distance from the first bit string and r bit strings spaced from the first bit string. Wherein r is a positive integer.

As an example, referring to fig. 12, fig. 12 shows a schematic diagram in which, in the first bit matrix, the preset range of the first bit string is a preset area with the first bit string as a base point. As shown in fig. 12, fig. 12 shows a top view of a first bit matrix 120 comprising 4 x 4 bit strings, wherein each dot represents a top view of one bit string. As shown in fig. 12 (a), if a bit string 1201 (shown by a hollow dot) is a first bit string, a dashed-line frame region 121 indicates a preset range of the first bit string in the first bit matrix 120. It can be seen that the preset range of the first bit string includes 8 bit strings (shown as solid dots) other than the first bit string. Alternatively, as shown in fig. 12 (b), if the bit string 1201 is a first bit string, the dashed-line frame region 122 in the first bit matrix 120 indicates a preset range of the first bit string, and in this case, 11 bit matrices other than the first bit string are included in the preset range of the first bit string. Therefore, the second bit string may be any one of the 8 bit strings or the 11 bit strings, or both of the 8 bit strings or the 11 bit strings are the second bit string.

It can be seen that, in the first bit matrix, bits in a preset range of the first bit string, which are located in a preset range corresponding to bits to be coded, may form a context model of the bits to be coded.

Referring to fig. 13a, fig. 13a shows a schematic diagram of a context model 131 of a bit 1311 to be encoded in a first bit matrix 130. The bit string 1322 includes bits in a first bit string where the bits to be encoded 1311 are located, the bits being located within a preset range corresponding to the bits to be encoded. The context model 131 further includes bits within a preset range corresponding to bits to be encoded in 8 second bit strings located within a preset range of the first bit string, as shown in fig. 13 a. It can be seen that the length, width and height of the context model 131 may be less than or equal to the length, width and height of the first bit matrix.

Optionally, the context of the bit to be coded may include, in addition to the context described above: and the bits in the third bit string and/or the fourth bit string are in the preset range corresponding to the bits to be coded. Wherein the third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Here, the second subband is a subband different from the first subband.

For the description of the bits in the preset range corresponding to the bits to be coded in the third bit string and/or the fourth bit string, reference may be made to the description of the bits in the preset range corresponding to the bits to be coded in the first bit string and/or the second bit string, which is not repeated here.

In the first case, if a plurality of binary bit strings in the first bit matrix are used to represent coefficients in different subbands, respectively, the third bit string is a bit string in the first bit matrix, and the fourth bit string is a bit string in the second bit matrix. The second bit matrix is a bit matrix in the plurality of bit matrices, which is located in a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a bit string located within a preset range of the first bit string.

The second bit matrix is a bit matrix in the plurality of bit matrices, which is located within the preset range of the first bit matrix, and reference may be made to the above description, where the second bit matrix is a description of a bit matrix in the plurality of bit matrices, which is located within the preset range of the first bit matrix, and is not described herein again.

For the description of the bit string located in the preset range of the first bit string, reference may be made to the above description, where the second bit string is in the first bit matrix, and the description of the bit string located in the preset range of the first bit string is omitted here for brevity.

In the second case, if a plurality of binary bit strings in the first bit matrix are used to represent the coefficients in the first subband, i.e. the first bit matrix corresponds to the first subband, then the third bit string and the fourth bit string are both bit strings in the third bit matrix. The third bit matrix is a bit matrix in the preset range of the first bit matrix in the plurality of bit matrices, and the second subband is a subband corresponding to the third bit matrix.

For example, the third bit matrix is a description of a bit matrix located in a preset range of the first bit matrix in the plurality of bit matrices, and reference may be made to the above description, where the second bit matrix is a description of a bit matrix located in a preset range of the first bit matrix in the plurality of bit matrices, and details are not repeated here.

And S107, estimating the probability distribution of the bits to be coded based on the context of the bits to be coded.

After obtaining the context of the bit to be coded, the coding apparatus estimates the probability distribution of the bit to be coded through a probability estimation network (or a probability estimation model, a probability estimation module, etc.) based on the context.

In the above context, coded bits may be included, and uncoded bits may also be included. If the context includes uncoded bits, the encoding apparatus sets a default value for the uncoded bits, that is, the encoding apparatus performs default value setting processing on the uncoded bits. The default value may be, for example, "0" or "1", and is not limited thereto.

As an example, referring to fig. 11, in the context model 110 shown in fig. 11, open ellipses may represent coded bits, and filled perfect circles may represent uncoded bits, and for 4 uncoded bits shown in fig. 11, the encoding apparatus may set all the 4 uncoded bits to "0".

Then, the encoding apparatus inputs the context of the bits to be encoded, which have been processed by the uncoded bits, into a probability estimation network (or a probability estimation model, a probability estimation module, etc.) to estimate the probability distribution of the current bits to be encoded. The context of the bits to be coded for estimating the probability distribution of the bits to be coded includes the value of the coded bits and the default value set by the coding apparatus for the uncoded bits. As an example, the probability distribution of the current bit to be coded estimated by the coding apparatus may be: the probability that the bit to be encoded is encoded as "0" is 0.6, and the probability that the bit to be encoded is encoded as "1" is 0.4.

The probability estimation method used in the probability estimation network may be a deep learning network-based probability distribution estimation method. The deep learning network may be a Recurrent Neural Network (RNN), a Convolutional Neural Network (CNN), or the like, which is not limited thereto.

Referring to fig. 13b, fig. 13b is a schematic diagram illustrating the estimation of the probability distribution of bits to be encoded by the probability estimation network. As shown in fig. 13b, the encoding apparatus may input the context of the bits to be encoded into the probability estimation network 130b, so as to obtain the probability distribution of the bits to be encoded. The context of the bits to be encoded input into the probability estimation network 130b includes the value of the encoded bits and the default value set by the encoding apparatus for the uncoded bits.

As an example, if the probability estimation method used by the probability estimation network 130b shown in fig. 13b is a probability distribution estimation method based on a deep learning network, that is, the probability estimation network 130b shown in fig. 13b is implemented by a deep learning network. Then, as shown in fig. 13c, the encoding apparatus may input the context of the bits to be encoded into the deep learning network 130c, so as to obtain the probability distribution of the bits to be encoded. The context of the bit to be coded input into the deep learning network 130c includes the value of the coded bit and the default value set by the coding apparatus for the uncoded bit.

It should be noted that, in the embodiment of the present application, a specific implementation of the probability estimation network or the probability estimation model is not limited. It should be understood that any probability estimation network or probability estimation model that can estimate the probability distribution of bits to be encoded based on the context of the bits to be encoded in the present application is included in the embodiments of the present application.

In practical application, for convenient operation, the encoding device may set default values for bits to be encoded. Then, the encoding device inputs the context of the processed uncoded bits and the bits to be encoded with default values into a probability estimation network or a probability estimation model for estimating the probability distribution of the current bits to be encoded. That is, the encoding apparatus will include context model information of the bits to be encoded, and all the context model information is input into the probability estimation network or the probability estimation model to estimate the probability distribution of the current bits to be encoded.

It should be noted that, with the image processing method provided in the embodiment of the present application, the context of the bit to be encoded is obtained in the binary bit matrix, and the accuracy of estimating the probability distribution of the bit to be encoded according to the context is higher than the accuracy of estimating the probability distribution of the bit to be encoded in the prior art.

And S108, coding the bits to be coded according to the probability distribution of the bits to be coded so as to obtain the coding information of the image to be processed corresponding to the bit matrixes.

And the coding device carries out entropy coding on the current bit to be coded based on the estimated probability distribution of the current bit to be coded. For example, the encoding apparatus performs arithmetic coding (arithmetric coding) on a bit to be currently encoded.

When the encoding device is based on the image processing method provided by the embodiment of the application, after entropy encoding is performed on each bit in the bit matrixes, the encoding information (or called encoding code stream or compressed code stream or bit stream) of the image to be processed corresponding to the bit matrixes is obtained.

Thus, the encoding device completes encoding of a plurality of bit matrices corresponding to the image to be processed. By the method provided by the embodiment of the application, the probability distribution of the bit to be coded estimated by the coding device has high accuracy, so that the entropy coding efficiency is improved.

Next, a specific process of the encoding apparatus performing subband aggregation on the plurality of two-dimensional coefficient blocks will be described.

Specifically, in the multiple two-dimensional coefficient blocks obtained in S103, the encoding device aggregates coefficients in the same sub-band in each two-dimensional coefficient block to obtain a two-dimensional coefficient block corresponding to the sub-band. That is, the coefficients included in the two-dimensional coefficient block corresponding to the sub-band are all coefficients in the sub-band, i.e., the two-dimensional coefficient block corresponds to the sub-band.

As an example, if the number of the above-mentioned multiple two-dimensional coefficient blocks obtained is T × S, each two-dimensional coefficient block includes M × N coefficients, and M × N coefficients are coefficients in M × N sub-bands respectively, that is, M × N coefficients correspond to the M × N sub-bands one by one. Then, the encoding apparatus may extract coefficients in T × S first sub-bands from the T × S two-dimensional coefficient blocks to obtain a two-dimensional coefficient block corresponding to the first sub-band. Here, the first subband is any one of M × N subbands. And the two-dimensional coefficient block corresponding to the first sub-band is an aggregated two-dimensional coefficient block. In this way, the encoding device can acquire M × N aggregated two-dimensional coefficient blocks, which correspond one-to-one to M × N subbands. Each of the M × N aggregated two-dimensional coefficient blocks includes T × S coefficients, which are coefficients in the same sub-band.

Referring to fig. 14, T is 2, S is 4, and M and N are both 4 as an example. As shown in fig. 14, (a) in fig. 14 shows 2 × 4 two-dimensional coefficient blocks, and any one of the 2 × 4 two-dimensional coefficient blocks is Y1. Fig. 14 (b) shows a schematic diagram of a two-dimensional coefficient block Y1, and the two-dimensional coefficient block Y1 includes 4 × 4 coefficients, corresponding to 4 × 4 sub-bands. For example, the coefficient y1(0,0) may be a coefficient in the first sub-band and the coefficient y1(0,1) may be a coefficient in the second sub-band.

The encoding apparatus may aggregate coefficients in each sub-band of the 2 × 4 two-dimensional coefficient blocks shown in (a) in fig. 14, and may obtain 4 × 4 aggregated two-dimensional coefficient blocks shown in (c) in fig. 14. The 4 × 4 aggregated two-dimensional coefficient blocks correspond to 4 × 4 sub-bands. Of the 4 × 4 aggregated two-dimensional coefficient blocks, the two-dimensional coefficient block Y2 may be a two-dimensional coefficient block corresponding to the first sub-band. Fig. 14 (d) shows a schematic diagram of the two-dimensional coefficient block Y2. The two-dimensional coefficient block Y2 includes 2 × 4 coefficients, each of the 2 × 4 coefficients being a coefficient in the first sub-band.

In this way, the encoding apparatus can cause one two-dimensional coefficient block to include coefficients corresponding to the same sub-band by performing sub-band aggregation on the plurality of two-dimensional coefficient blocks acquired in S103. In this way, in a plurality of bit matrices obtained by the encoding apparatus binarizing each coefficient in the aggregated plurality of two-dimensional coefficient blocks, a plurality of bit strings in one bit matrix may be used to represent the coefficient in one sub-band. Therefore, when the coding device acquires the bit context to be coded, the speed of acquiring the context can be increased, and the speed of coding the bit matrix by the coding device is increased.

Next, a specific procedure of the encoding apparatus performing subband aggregation on the plurality of initial bit matrices described in S104 will be described.

Specifically, the encoding apparatus may obtain a plurality of initial bit matrices after binarizing each coefficient in the plurality of two-dimensional coefficient blocks acquired in S103. Wherein, an initial bit matrix comprises a plurality of bit strings respectively used for representing coefficients in different sub-bands.

Then, the encoding apparatus may aggregate bit strings used for representing coefficients in the same subband in each of the plurality of initial bit matrices to obtain a bit matrix corresponding to the subband. That is, a plurality of bit strings included in the bit matrix corresponding to the subband are all used to represent the coefficients in the subband, that is, the bit matrix corresponds to the subband.

As an example, if the number of the initial bit matrices is T × S, each initial bit matrix includes M × N bit strings, where the M × N bit strings are respectively used to represent coefficients in M × N subbands, and the M × N bit strings are in one-to-one correspondence with the M × N subbands. Then, the encoding apparatus may extract T × S bit strings for representing coefficients in the first subband from the T × S initial bit matrices to obtain a bit matrix corresponding to the first subband. Here, the first subband is any one of M × N subbands. And the bit matrix corresponding to the first sub-band is an aggregated bit matrix. In this way, the encoding apparatus can obtain M × N aggregated bit matrices, which correspond to M × N subbands one to one. Each of the M × N aggregated bit matrices includes T × S bit strings, and the T × S bit strings are all used to represent coefficients in the same subband.

If (a) in fig. 14 is regarded as a top view of 2 × 4 initial bit matrices, a small square represents one initial bit matrix, and any one of the 2 × 4 initial bit matrices is Y1. Fig. 14 (b) shows a top view of the initial bit matrix Y1, and one small square represents a top view of one bit string. As shown in (b) of fig. 14, the initial bit matrix Y1 includes 4 × 4 bit strings for representing coefficients in 4 × 4 subbands, respectively. For example, the bit string y1(0,0) may represent coefficients in a first sub-band, and the bit string y1(0,1) may represent coefficients in a second sub-band.

The encoding apparatus aggregates a plurality of bit strings in the 2 × 4 initial bit matrix shown in fig. 14 (a), and may obtain 4 × 4 aggregated bit matrices shown in fig. 14 (c), where one small square shown in fig. 14 (c) represents a top view of one aggregated bit matrix. In the 4 × 4 aggregated bit matrices, a plurality of bit strings included in any one aggregated bit matrix are all used to represent coefficients in the same subband. I.e., an aggregate sum bit matrix, corresponding to a subband. Of the 4 × 4 aggregated bit matrices, the bit matrix Y2 may be a bit matrix corresponding to the first subband. Fig. 14 (d) shows a schematic diagram of the bit matrix Y2, and a small square shown in fig. 14 (d) represents a top view of one bit string. The bit matrix Y2 includes 2 x 4 bit strings, each of the 2 x 4 bit strings being used to represent a coefficient in the first subband.

In this way, the encoding apparatus binarizes and aggregates the two-dimensional coefficient blocks acquired in S103, so that a plurality of bit strings in one bit matrix can be used to represent coefficients in the same sub-band. Therefore, when the coding device acquires the bit context to be coded, the speed of acquiring the context can be increased, and the speed of coding the bit matrix by the coding device is increased.

The above is an image processing method applied to an encoding device according to an embodiment of the present application, and an image processing method applied to a decoding device is described below.

Referring to fig. 15, fig. 15 is a flowchart illustrating an image processing method provided in an embodiment of the present application, where the method may be applied to an image decoding apparatus in an image processing apparatus. For convenience of description, an image decoding apparatus that executes the image processing method provided by the embodiment of the present application will be hereinafter abbreviated as a decoding apparatus.

The method can comprise the following steps:

s201, determining the position of a bit to be decoded in a plurality of bit matrixes corresponding to the image to be processed.

Specifically, the decoding apparatus may determine the positions of bits to be decoded in a plurality of bit matrices corresponding to the image to be processed according to a preset decoding order.

The decoding device may preset a preset decoding order, and the preset decoding order is the same as the encoding order adopted in the encoding device during encoding. In addition, the decoding device can also preset the information of the sizes and the like of a plurality of image blocks obtained after the image to be processed is partitioned. In this way, the decoding apparatus may determine the size of the bit matrix corresponding to a plurality of image blocks obtained by partitioning the image to be processed based on the sizes of the plurality of image blocks. It can be understood that the bit matrix determined by the decoding apparatus based on the sizes of a plurality of image blocks obtained after the image to be processed is partitioned may be a bit matrix aggregated by subbands (that is, one bit matrix corresponds to one subband, that is, a plurality of bit strings in one bit matrix are all used to represent coefficients in the same subband), or may be a bit matrix unaggregated by subbands (that is, one bit matrix corresponds to a plurality of subbands, that is, a plurality of bit strings in one bit matrix are used to represent coefficients in different subbands). Here, the description that one bit matrix corresponds to one subband and one bit matrix corresponds to a plurality of subbands may refer to the above, and is not described here again.

It can be understood that the decoding apparatus may further obtain size information of a plurality of image blocks obtained after the to-be-processed image is partitioned from the encoding information of the to-be-processed image. In this way, the decoding apparatus can determine the size of the bit matrix corresponding to the plurality of image blocks based on the size information of the plurality of image blocks acquired from the encoding information of the image to be processed.

The encoding information of the image to be processed may be encoding information of the image to be processed, which is acquired by the decoding apparatus from a local place in advance, or the encoding information of the image to be processed may be encoding information of the image to be processed, which is transmitted by another device (for example, the wireless or wired second network communication device 23 shown in fig. 2) and received by the decoding apparatus in real time. The embodiment of the present application does not limit the timing and manner of acquiring the encoding information of the image to be processed by the decoding apparatus.

The local encoding information of the image to be processed may be the encoding information of the image/video encoded by the local device through the methods in S101 to S108, or may be the encoding information of the image to be processed transmitted by another device (for example, the second network communication device 23 shown in fig. 2, which is wireless or wired) that is received and stored in advance by the local device, and is not limited thereto.

Then, the decoding apparatus may determine the position of the current bit to be decoded in the determined bit matrices according to the preset decoding order.

The process of determining the position of the current bit to be decoded in the multiple bit matrices by the decoding apparatus according to the decoding order may refer to the description of determining the bit to be coded in the multiple bit matrices according to the preset coding order in S105, which is not described herein again.

For convenience of description, the following description will be given taking as an example that the bit to be decoded is a bit at any position in a first bit string in a first bit matrix, the first bit string being used to represent one coefficient in a first subband, the first bit matrix being any one of the bit matrices in the plurality of bit matrices.

S202, obtaining the context of the bit to be decoded.

The context of the bits to be decoded may include: and the bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string. Wherein the second bit string is used to represent another coefficient in the first subband.

It is to be understood that the context of the bits to be decoded may be all bits in the first bit string and/or the second bit string within the preset range corresponding to the bits to be decoded, or may be a part of the bits in the first bit string and/or the second bit string within the preset range corresponding to the bits to be decoded, which is not limited herein.

If the context of the bit to be decoded is a part of bits in the first bit string and/or the second bit string within the preset range corresponding to the bit to be decoded, the decoding apparatus may directly output the position of each bit included in the context of the bit to be decoded, without determining the position of each bit in the context through the preset range corresponding to the bit to be decoded.

For the first bit string, the preset range corresponding to the bit to be decoded may be a range having a preset distance from the position of the bit to be decoded. For the second bit string, the preset range corresponding to the bit to be decoded may be a range that is based on a bit position of one bit plane located together with the bit to be decoded and is a preset distance away from the position.

For the description of the bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string, reference may be made to the description of the bits in the preset range corresponding to the bits to be coded in the first bit string and/or the second bit string in S106, and details are not repeated here.

In the first case, if a plurality of bit strings in the first bit matrix are respectively used to represent coefficients in different subbands, the second bit string is a bit string in the second bit matrix, and the second bit matrix is a bit matrix in the plurality of bit matrices located within a preset range of the first bit matrix.

In the second case, the second bit string is a bit string of the first bit matrix different from the first bit string if a plurality of binary bit strings in the first bit matrix are each used to represent a coefficient in the first subband. The second bit string is a bit string in the first bit matrix, which is located within a preset range of the first bit string.

Optionally, the context of the bit to be decoded may include, in addition to the context described above: and the bits in the preset range corresponding to the bits to be decoded in the third bit string and/or the fourth bit string. Wherein the third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Here, the second subband is a subband different from the first subband.

In the first case, if a plurality of binary bit strings in the first bit matrix are used to represent coefficients in different subbands, respectively, the third bit string is a bit string in the first bit matrix, and the fourth bit string is a bit string in the second bit matrix. Alternatively, the fourth bit string is a bit string in the first bit matrix, and the third bit string is a bit string in the second bit matrix. The second bit matrix is a bit matrix in the plurality of bit matrices, which is located in a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a bit string located within a preset range of the first bit string.

In the second case, if a plurality of binary bit strings in the first bit matrix are used to represent coefficients in the first subband, i.e. the first bit matrix corresponds to the first subband, then the third bit string and the fourth bit string are both bit strings in the third bit matrix. The third bit matrix is a bit matrix in the preset range of the first bit matrix in the plurality of bit matrices, and the second subband is a subband corresponding to the third bit matrix.

Specifically, for the description of the context of the bit to be decoded acquired by the decoding apparatus, reference may be made to the description of the context of the bit to be encoded acquired by the encoding apparatus in S106, which is not described herein again.

S203, estimating the probability distribution of the bits to be decoded based on the context of the bits to be decoded.

After obtaining the context of the bit to be decoded, the decoding device estimates the probability distribution of the bit to be decoded through a probability estimation network (or a probability estimation model, a probability estimation module, etc.) based on the context.

Further, in the above context, decoded bits may be included, and undecoded bits may also be included. If the context includes the undecoded bit, the decoding apparatus sets a default value for the undecoded bit, that is, the decoding apparatus performs default value setting processing on the undecoded bit. The default value may be, for example, "0" or "1", and is not limited thereto.

As an example, referring to fig. 11, in the context model 110 shown in fig. 11, an open ellipse may represent a decoded bit, a filled perfect circle may represent an undecoded bit, and for 4 undecoded bits shown in fig. 11, the decoding apparatus may set each of the 4 undecoded bits to "0".

Specifically, the decoding apparatus may refer to the description of estimating the probability distribution of the bits to be coded based on the context of the bits to be decoded and through a probability estimation network (or a probability estimation model, a probability estimation module, etc.) based on the context, where the description is omitted here. It should be noted that the context of the bits to be decoded for estimating the probability distribution of the bits to be decoded includes the value of the decoded bits and the default value set by the decoding apparatus for the bits not to be decoded.

In practical application, for convenient operation, the decoding device can set default values for bits to be decoded. Then, the decoding device inputs the context of the un-decoded bits and the bits to be decoded with default values into the probability estimation network or the probability estimation model for estimating the probability distribution of the current bits to be decoded. That is, the decoding apparatus will include context model information of the bits to be decoded, and all the context model information is input into the probability estimation network or the probability estimation model to estimate the probability distribution of the current bits to be decoded.

S204, decoding the bits to be decoded according to the positions of the bits to be decoded, the probability distribution of the bits to be decoded and the coding information of the image to be processed to obtain the bit matrixes.

The decoding apparatus may perform entropy decoding on the current bit to be decoded based on the position of the bit to be decoded, the estimated probability distribution of the current bit to be decoded, and the encoding information of the image to be processed. For example, the decoding apparatus performs arithmetic decoding (arithmetric decoding) on the bit to be currently decoded.

When the decoding device is based on the image processing method provided by the embodiment of the application, each bit in the bit matrixes is decoded through entropy, and then the bit matrixes are obtained.

And S205, performing inverse binarization on the plurality of decoded bit matrixes to obtain a plurality of two-dimensional coefficient blocks.

Wherein the decoding device is preset with an inverse binarization method corresponding to the binarization method used by the encoding device during encoding. For example, at S104, the encoding apparatus binarizes each coefficient in the two-dimensional coefficient block in a fixed-length binarization manner. Then, at the decoding end, the decoding apparatus inversely binarizes the bit matrix obtained by decoding in a manner inverse to the fixed-length binarization manner.

As an example, the encoding apparatus binarizes each coefficient in the two-dimensional coefficient block in an 8-bit fixed length binarization manner, and then the decoding apparatus restores continuous 8 bits in a plurality of decoded bit matrices to one coefficient. For example, the decoding apparatus restores successive bits 00000001 in the bit matrix to a coefficient having a value of 1, and restores successive bits 10000000 in the bit matrix to a coefficient having a value of 128. Thus, a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrices can be obtained.

For example, referring to FIG. 7, after 8-bit fixed length inverse binarization of the bit matrix 72 of FIG. 7, the two-dimensional coefficient block 60 shown in FIG. 6 may be obtained.

In S205, any one of the two-dimensional coefficient blocks finally obtained is a plurality of coefficients included in the two-dimensional coefficient block, which are coefficients in different sub-bands.

It should be noted that, if one of the bit matrices to be decoded is determined by the decoding apparatus to correspond to one subband, that is, a plurality of bit strings in one bit matrix are all used to represent coefficients in the same subband. For convenience of description, the embodiments of the present application refer to the multiple bit matrices decoded in this case as initial bit matrices. In order to allow a plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks obtained in S205 to be coefficients in different sub-bands, respectively. The decoding apparatus further performs the following operations:

in a possible implementation manner, the decoding apparatus may first perform inverse aggregation on a plurality of initial bit matrices obtained by decoding to obtain a plurality of target bit matrices. For any one of the target bit matrices, a plurality of bit strings included in the target bit matrix are respectively used to represent coefficients in different subbands. Then, the decoding apparatus inversely binarizes the plurality of target bit matrices, thereby obtaining the plurality of two-dimensional coefficient blocks. The plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks are coefficients in different sub-bands, respectively.

Specifically, the decoding apparatus may extract a bit string from each of the plurality of initial bit matrices, to obtain a plurality of bit strings, and use the obtained plurality of bit strings as a target bit matrix. In this way, the decoding apparatus can extract one bit string from each of the plurality of initial bit matrices by a plurality of times, thereby obtaining a plurality of target bit matrices. A plurality of bit strings included in one target bit matrix are used to represent coefficients in different sub-bands, respectively. The decoding device extracts the value of the number of times of the bit string from each initial bit matrix in the plurality of initial bit matrices, and the value is equal to the value of the number of the bit string included in one initial bit matrix in the plurality of initial bit matrices obtained by decoding.

For the same initial bit matrix, the bit string extracted by the decoding apparatus from the initial bit matrix at a time is different. In this way, when the decoding apparatus inversely binarizes the plurality of target bit matrices obtained by the inverse aggregation process, the plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks obtained are coefficients in different sub-bands, respectively.

As an example, if the number of the initial bit matrix is T × S, the initial bit matrix corresponds to T × S subbands. Any one of the T × S initial bit matrices includes M × N bit strings, and the M × N bit strings are used to represent M × N coefficients in the same subband. Then, the decoding apparatus may extract a bit string from each of the T × S initial bit matrices to obtain a target bit matrix. The target bit matrix comprises T S bit strings which are respectively used for representing coefficients in T S sub-bands. Since each initial bit matrix includes M × N bit strings, the decoding apparatus needs to perform M × N bit string extractions, so as to obtain M × N target bit matrices. And the T S bit strings included in any one of the M N target bit matrixes are respectively used for representing the coefficients in the T S sub-bands. It should be understood that the bit string extracted by the decoding apparatus from the initial bit matrix at a time is different for the same initial bit matrix.

In another possible implementation manner, the decoding apparatus may first perform inverse binarization on the plurality of initial bit matrices obtained by decoding to obtain a plurality of initial two-dimensional coefficient blocks. Then, the decoding device inversely aggregates the plurality of initial two-dimensional coefficient blocks to obtain the plurality of two-dimensional coefficient blocks. The plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks are coefficients in different sub-bands, respectively.

Specifically, the decoding apparatus may extract one coefficient from each of a plurality of initial two-dimensional coefficient blocks, and use the extracted plurality of coefficients as one target two-dimensional coefficient block. In this way, the decoding apparatus can obtain a plurality of target two-dimensional coefficient blocks by extracting one coefficient from each of a plurality of initial two-dimensional coefficient blocks a plurality of times. Thus, the plurality of coefficients included in one target two-dimensional coefficient block are coefficients in different sub-bands, respectively. The decoding device extracts the value of the coefficient times from each initial two-dimensional coefficient block in the initial two-dimensional coefficient blocks, and the value is equal to the coefficient value of the coefficient included in one initial two-dimensional coefficient block in the initial two-dimensional coefficient blocks.

The coefficients that the decoding device extracts from the initial two-dimensional coefficient block at a time are different for the same initial two-dimensional coefficient block. In this way, the decoding apparatus inverse-aggregates the plurality of initial two-dimensional coefficient blocks, and then, the plurality of coefficients included in any one of the plurality of two-dimensional coefficient blocks obtained are coefficients in different sub-bands.

As an example, if the number of the initial two-dimensional coefficient blocks is T × S, T × S subbands are corresponded. Any one of the T × S initial two-dimensional coefficient blocks includes M × N coefficients, and the M × N coefficients are M × N coefficients in the same sub-band. Then, the decoding apparatus may extract one coefficient from each of the T × S initial two-dimensional coefficient blocks to obtain a target two-dimensional coefficient block. The target two-dimensional coefficient block includes T × S coefficients, which are coefficients in T × S subbands, respectively. Since each initial two-dimensional coefficient block includes M × N coefficients, the decoding apparatus needs to perform M × N coefficient decimation, thereby obtaining M × N target two-dimensional coefficient blocks. Wherein, the T × S coefficients included in any one of the M × N target two-dimensional coefficient blocks are coefficients in the T × S subbands, respectively. It should be understood that the coefficients that the decoding apparatus extracts from the same initial two-dimensional coefficient block at a time are different for the same initial two-dimensional coefficient block.

S206, inversely transforming the two-dimensional coefficient blocks to obtain a plurality of image blocks, and restoring the image to be processed based on the image blocks.

The image blocks are obtained by dividing the image to be processed.

The decoding apparatus may preset an inverse transform method inverse to the transform method of the encoding apparatus, and inverse-transform the plurality of two-dimensional coefficient blocks by the inverse transform method to obtain the plurality of image blocks. Of course, if the coding information acquired in S201 is coding information obtained by coding a bit matrix corresponding to the residual block, the decoding apparatus inverse-transforms the plurality of two-dimensional coefficient blocks to obtain a plurality of residual blocks in S206. Here, the residual block is a plurality of residual blocks corresponding to a plurality of image blocks obtained by dividing the image to be processed. Further, the decoding apparatus restores a plurality of image blocks based on the plurality of residual blocks.

Then, the decoding device determines the number of image blocks corresponding to the image to be processed according to the size of the image to be processed, which is acquired from the encoding information of the image to be processed in advance, and the sizes of a plurality of image blocks obtained after the image to be processed is blocked. Therefore, the decoding device can restore the image to be processed according to the size of the image to be processed and the number of the image blocks corresponding to the image to be processed.

So far, the image processing method provided by the embodiment of the application realizes decoding of the coded information obtained by S101-108 coding.

In summary, in the image processing method provided in the embodiment of the present application, a plurality of two-dimensional coefficient blocks corresponding to an image block or a residual block are binarized, so as to obtain a plurality of corresponding three-dimensional bit matrices. Then, the encoding device obtains the context of the current bit to be encoded from the three-dimensional bit matrixes and estimates the probability distribution of the bit to be encoded based on the context. By the method provided by the embodiment of the application, the accuracy of estimating the probability distribution of the current bit to be coded is improved. Therefore, the encoding device can carry out entropy encoding on the current bit to be encoded based on the probability distribution which is estimated by the bit to be encoded and has higher accuracy than that of the prior art, so that the encoding efficiency of the image (or video) can be improved, the storage resource for storing the image (or video) is saved, and the transmission bandwidth for transmitting the image (or video) is saved.

The scheme provided by the embodiment of the application is mainly introduced from the perspective of a method. To implement the above functions, it includes hardware structures and/or software modules for performing the respective functions. Those of skill in the art would readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The embodiment of the present application may perform division of functional modules on the image processing apparatus according to the above method, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.

As shown in fig. 16, fig. 16 is a schematic structural diagram illustrating an image processing apparatus 160 according to an embodiment of the present disclosure. The image processing apparatus 160 may be configured to perform an image encoding method among the above-described image processing methods, for example, to perform the method shown in fig. 4. The image processing apparatus 160 may include a determination unit 161, an estimation unit 162, and an encoding unit 163, among others.

A determining unit 161 for determining the bits to be encoded. The bit to be coded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices obtained by processing an image to be processed, and the first bit matrix is a three-dimensional bit matrix. An estimating unit 162, configured to estimate a probability distribution of the bits to be coded based on the context of the bits to be coded. Here, the context of the bits to be encoded includes: and the bits in the preset range corresponding to the bits to be coded in the first bit string and/or the second bit string. Wherein the second bit string is used to represent another coefficient in the first subband. And an encoding unit 163, configured to encode the bits to be encoded according to the probability distribution of the bits to be encoded, so as to obtain encoding information of the image to be processed.

As an example, in conjunction with fig. 4, the determining unit 161 may be configured to perform S105, the estimating unit 162 may be configured to perform S107, and the encoding unit 163 may be configured to perform S108.

Optionally, if the plurality of binary bit strings are respectively used to represent coefficients in different subbands, the second bit string is a binary bit string in a second bit matrix. Here, the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. Or, if the plurality of binary bit strings are all used to represent coefficients in the first subband, the second bit string is a binary bit string in the first bit matrix, which is located within a preset range of the first bit string.

Optionally, the context of the bit to be coded further includes: and the bits in the third bit string and/or the fourth bit string are in the preset range corresponding to the bits to be coded. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein the content of the first and second substances,

if the plurality of binary bit strings are used to represent coefficients in different sub-bands, respectively, the third bit string is a binary bit string in the first bit matrix, and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix of the plurality of bit matrices that is within a preset range of the first bit matrix. The second sub-band is a sub-band corresponding to a binary bit string located within a preset range of the first bit string. Alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are all used for representing the coefficients in the first subband, i.e. the plurality of bit matrices correspond to the plurality of subbands one to one, then the third bit string and the fourth bit string are both binary bit strings in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices that is within a predetermined range of the first bit matrix, and the second subband is a subband corresponding to the third bit matrix.

Optionally, the image processing apparatus 160 further includes: an obtaining unit 164, configured to obtain a plurality of two-dimensional coefficient blocks corresponding to the image to be processed before the determining unit 161 determines the bits to be encoded. Wherein each of the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients, the plurality of coefficients being coefficients in different sub-bands, respectively. A binarization unit 165 for binarizing each coefficient in the plurality of two-dimensional coefficient blocks to obtain the plurality of bit matrices.

As an example, in conjunction with fig. 4, the obtaining unit 164 may be configured to perform S103, and the binarization unit 165 may be configured to perform S104.

Optionally, the binarization unit 165 is specifically configured to binarize each coefficient in the plurality of two-dimensional coefficient blocks in a fixed-length binarization manner.

As an example, in connection with fig. 4, the binarization unit 165 may be configured to perform S104.

Optionally, the obtaining unit 164 is further configured to obtain the image to be processed before obtaining the plurality of two-dimensional coefficient blocks corresponding to the image to be processed. The image to be processed comprises an image frame in a picture or a video. The obtaining unit 164 is further specifically configured to block and transform the image to be processed to obtain a plurality of two-dimensional coefficient blocks.

As an example, in connection with fig. 4, the obtaining unit 164 may be configured to perform S101 and S103.

Optionally, the determining unit 161 is specifically configured to determine bits to be encoded in the multiple bit matrices according to a preset encoding order.

As an example, in connection with fig. 4, the determining unit 161 may be configured to perform S105.

Optionally, the estimating unit 162 is specifically configured to estimate, based on the context of the bits to be coded, the probability distribution of the bits to be coded through a probability estimation network.

As an example, in connection with fig. 4, the estimation unit 162 may be configured to perform S107.

Optionally, the encoding unit 163 is specifically configured to perform entropy encoding on the bits to be encoded according to the probability distribution of the bits to be encoded.

As an example, in connection with fig. 4, the encoding unit 163 may be configured to perform S108.

For the detailed description of the above alternative modes, reference may be made to the foregoing method embodiments, which are not described herein again. In addition, for any explanation and beneficial effect description of the image processing apparatus 160 provided above, reference may be made to the corresponding method embodiment described above, and details are not repeated.

As an example, in connection with fig. 3, the determining unit 161, the estimating unit 162, the encoding unit 163, the obtaining unit 164, and the binarization unit 165 in the image processing apparatus 160 may be implemented by the processor 301 in fig. 3 executing program code in the memory 302 in fig. 3.

As shown in fig. 17, fig. 17 is a schematic structural diagram illustrating an image processing apparatus 170 according to an embodiment of the present disclosure. The image processing apparatus 170 may be used to execute an image decoding method among the above-described image processing methods, for example, to execute the method shown in fig. 15. The image processing apparatus 170 may include a determination unit 171, an estimation unit 172, and a decoding unit 173, among others.

A determining unit 171 for determining the position of the bit to be decoded. The bit to be decoded is any bit in a first bit string, the first bit string is any binary bit string in a plurality of binary bit strings included in a first bit matrix, the first bit string is used for representing a coefficient in a first sub-band, the first bit matrix is any bit matrix in a plurality of bit matrices corresponding to the image to be processed, and the first bit matrix is a three-dimensional bit matrix. An estimating unit 172, configured to estimate a probability distribution of the bits to be decoded based on the context of the bits to be decoded. Here, the context of the bits to be decoded includes: and the bits in the preset range corresponding to the bits to be decoded in the first bit string and/or the second bit string. Wherein the second bit string is used to represent another coefficient in the first subband. The decoding unit 173 is configured to decode the bits to be decoded to obtain a plurality of bit matrices based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoding information of the image to be processed.

As an example, in connection with fig. 15, the determining unit 171 may be configured to perform S201, the estimating unit 172 may be configured to perform S203, and the decoding unit 173 may be configured to perform S204.

Optionally, if the plurality of binary bit strings are respectively used to represent coefficients in different subbands, the second bit string is a binary bit string in a second bit matrix. Here, the second bit matrix is a bit matrix located within a preset range of the first bit matrix among the plurality of bit matrices. Or, if the plurality of binary bit strings are all used to represent coefficients in the first subband, the second bit string is a binary bit string in the first bit matrix, which is located within a preset range of the first bit string.

Optionally, the context of the bit to be decoded further includes: and the bits in the preset range corresponding to the bits to be decoded in the third bit string and/or the fourth bit string. The third bit string is used to represent one coefficient in the second subband and the fourth bit string is used to represent another coefficient in the second subband. Wherein the content of the first and second substances,

if the plurality of binary bit strings are used to represent coefficients in different sub-bands, respectively, the third bit string is a binary bit string in the first bit matrix, and the fourth bit string is a binary bit string in the second bit matrix. The second bit matrix is a bit matrix in a preset range of the first bit matrix in the plurality of bit matrices, and the second subband is a subband corresponding to a binary bit string in the preset range of the first bit string. Alternatively, the first and second electrodes may be,

if the plurality of binary bit strings are all used for representing the coefficients in the first subband, i.e. the plurality of bit matrices correspond to the plurality of subbands one to one, then the third bit string and the fourth bit string are both binary bit strings in the third bit matrix. The third bit matrix is a bit matrix of the plurality of bit matrices that is within a predetermined range of the first bit matrix, and the second subband is a subband corresponding to the third bit matrix.

Optionally, the image processing apparatus 170 further includes: and an inverse binarization unit 174, configured to inverse binarize the binary bit string in the plurality of decoded bit matrices to obtain a plurality of two-dimensional coefficient blocks corresponding to the plurality of bit matrices. Wherein each of the plurality of two-dimensional coefficient blocks comprises a plurality of coefficients that are coefficients in different sub-bands, respectively. And an inverse transformation unit 175, configured to inverse transform the plurality of two-dimensional coefficient blocks to obtain an image to be processed. The image to be processed comprises an image frame in a picture or a video.

As an example, in connection with fig. 15, the inverse binarization unit 174 may be configured to perform S205 and the inverse transformation unit 175 may be configured to perform S206.

Optionally, the inverse binarization unit 174 is specifically configured to inverse binarize binary bit strings in the multiple bit matrices obtained by decoding according to a fixed-length inverse binarization manner.

As an example, in connection with fig. 15, the inverse binarization unit 174 may be configured to perform S205.

Optionally, the determining unit 171 is specifically configured to determine positions of bits to be decoded in the multiple bit matrices according to a preset decoding order.

As an example, in connection with fig. 15, the determining unit 171 may be configured to perform S201.

Optionally, the estimating unit 172 is specifically configured to estimate, based on the context of the bits to be decoded, the probability distribution of the bits to be decoded through the probability estimation network.

As an example, in connection with fig. 15, the estimating unit 172 may be configured to perform S203.

Optionally, the decoding unit 173 is specifically configured to perform entropy decoding on the bits to be decoded based on the positions of the bits to be decoded, the probability distribution of the bits to be decoded, and the encoding information of the image to be processed.

As an example, in connection with fig. 15, the decoding unit 173 may be configured to perform S204.

For the detailed description of the above alternative modes, reference may be made to the foregoing method embodiments, which are not described herein again. In addition, for any explanation and beneficial effect description of the image processing apparatus 170 provided above, reference may be made to the corresponding method embodiment described above, and details are not repeated.

As an example, in connection with fig. 3, the determining unit 171, the estimating unit 172, the decoding unit 173, the inverse binarization unit 174, and the inverse transformation unit 175 in the image processing apparatus 170 may be implemented by the processor 301 in fig. 3 executing the program code in the memory 302 in fig. 3.

The embodiment of the present application further provides a chip system 180, as shown in fig. 18, where the chip system 180 includes at least one processor and at least one interface circuit. By way of example, when the system-on-chip 180 includes one processor and one interface circuit, then the one processor may be the processor 181 shown in solid line block in fig. 18 (or the processor 181 shown in dashed line block), and the one interface circuit may be the interface circuit 182 shown in solid line block in fig. 18 (or the interface circuit 182 shown in dashed line block). When the system-on-chip 180 includes two processors and two interface circuits, the two processors include the processor 181 shown in a solid line block in fig. 18 and the processor 181 shown in a dashed line block, and the two interface circuits include the interface circuit 182 shown in a solid line block in fig. 18 and the interface circuit 182 shown in a dashed line block. This is not limitative.

The processor 181 and the interface circuit 182 may be interconnected by wires. For example, the interface circuit 182 may be used to receive signals (e.g., to acquire a to-be-processed image, etc.). As another example, the interface circuit 182 may be used to send signals to other devices, such as the processor 181. Illustratively, the interface circuit 182 may read instructions stored in the memory and send the instructions to the processor 181. The instructions, when executed by the processor 181, may cause the image processing apparatus to perform the various steps in the embodiments described above. Of course, the chip system 180 may also include other discrete devices, which is not specifically limited in this embodiment.

Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions that, when executed on an image processing apparatus, cause the image processing apparatus to perform the steps performed by the image processing apparatus in the method flow shown in the above method embodiment.

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage medium in a machine-readable format or encoded on other non-transitory media or articles of manufacture.

Fig. 19 schematically illustrates a conceptual partial view of a computer program product comprising a computer program for executing a computer process on a computing device provided by an embodiment of the application.

In one embodiment, the computer program product is provided using a signal bearing medium 190. The signal bearing medium 190 may include one or more program instructions that, when executed by one or more processors, may provide the functions or portions of the functions described above with respect to fig. 4 or fig. 15. Thus, for example, one or more features described with reference to S101-S108 in FIG. 4, or with reference to S201-S206 in FIG. 15, may be undertaken by one or more instructions associated with the signal bearing medium 190. Further, the program instructions in FIG. 19 also describe example instructions.

In some examples, signal bearing medium 190 may comprise a computer readable medium 191 such as, but not limited to, a hard disk drive, a Compact Disc (CD), a Digital Video Disc (DVD), a digital tape, a memory, a read-only memory (ROM), a Random Access Memory (RAM), or the like.

In some embodiments, the signal bearing medium 190 may comprise a computer recordable medium 192 such as, but not limited to, memory, read/write (R/W) CD, R/W DVD, and the like.

In some implementations, the signal bearing medium 190 may include a communication medium 193 such as, but not limited to, a digital and/or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).

The signal bearing medium 190 may be conveyed by a wireless form of communication medium 193, such as a wireless communication medium conforming to the IEEE 1902.11 standard or other transmission protocol. The one or more program instructions may be, for example, computer-executable instructions or logic-implementing instructions.

In some examples, an image processing apparatus such as described with respect to fig. 4 or 15 may be configured to provide various operations, functions, or actions in response to one or more program instructions via the computer-readable medium 191, the computer-recordable medium 192, and/or the communication medium 193.

It should be understood that the arrangements described herein are for illustrative purposes only. Thus, those skilled in the art will appreciate that other arrangements and other elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used instead, and that some elements may be omitted altogether depending upon the desired results. In addition, many of the described elements are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, in any suitable combination and location.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented using a software program, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The processes or functions according to the embodiments of the present application are generated in whole or in part when the instructions are executed on and by a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). Computer-readable storage media can be any available media that can be accessed by a computer or can comprise one or more data storage devices, such as servers, data centers, and the like, that can be integrated with the media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.