阅读说明：本技术 可变宽度的人像精细抠图方法、装置、设备及存储介质 (Width-variable portrait fine matting method, device, equipment and storage medium ) 是由 韩智素 谌波 刘新科 玉珏 于 2021-08-20 设计创作，主要内容包括：本发明属于图像处理技术领域,具体涉及可变宽度的人像精细抠图方法、装置、设备及存储介质。该方法包括：S1、获取待抠图的原始图像,进行人像二值化分割预处理,分离得到人像绝对前景区域；S2、确定出所述人像绝对前景区域的头发区域和非头发区域,并分割得到头发区域；S3、获取所述人像绝对前景区域的确定边界上的每一个点p-(i),确定一个以该点为中心的正方形局部区域B-(i),并计算其局部最佳的未知区域的半径r-(i),沿着每个点的局部半径的边界绘出可变宽度的未知区域；S4、根据获取的所述原始图像和计算得到的所述人像绝对前景区域边缘的可变宽度的未知区域,在所述原始图像中进行软分割抠图操作,提取出前景目标。(The invention belongs to the technical field of image processing, and particularly relates to a variable-width portrait fine matting method, a variable-width portrait fine matting device, variable-width portrait fine matting equipment and a variable-width portrait fine matting storage medium. The method comprises the following steps: s1, obtaining the original image to be scratched, and carrying out image binarization segmentationProcessing and separating to obtain an absolute foreground region of the portrait; s2, determining a hair area and a non-hair area of the portrait absolute foreground area, and segmenting to obtain the hair area; s3, acquiring each point p on the determined boundary of the portrait absolute foreground area i Determining a square local area B centered on the point i And calculating the radius r of the locally optimal unknown region i Drawing an unknown region of variable width along the boundary of the local radius of each point; s4, according to the obtained original image and the unknown region with variable width of the edge of the absolute foreground region of the portrait, carrying out soft segmentation matting operation in the original image to extract a foreground object.)

1. A variable width portrait fine matting method is used for changing the unknown area radius of the hair area of the original image and generating an accurate trisection image to obtain a portrait matting result without obvious flaws; the method is characterized in that the method for finely matting the portrait with variable width comprises the following steps:

acquiring an original image to be scratched, carrying out portrait binaryzation segmentation pretreatment, and separating to obtain a portrait absolute foreground area;

determining a hair area and a non-hair area of the portrait absolute foreground area, and obtaining a segmentation result of the hair area through binary secondary segmentation, wherein the hair area is marked as R _ hair;

obtaining each point p on the determined boundary of the portrait absolute foreground area_iDetermining a square local area B centered on the point_iAnd calculating the radius r of the locally optimal unknown region_iDrawing an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm along the boundary of the local radius of each point;

according to the obtained original image and the unknown region with variable width of the edge of the absolute foreground region of the portrait obtained by calculation, carrying out soft segmentation matting operation in the original image to extract a foreground target;

wherein each point p on the boundary_iSquare local area B of_iIs locally optimized, the radius r of the unknown region_iComprises the following steps:

r_i＝r_max·min(H(B_i)，C(B_i)，D(B_i))

wherein r _ max is a preset maximum radius; h (B)_i) Is a linear interpolation between 0.1 and 1, and the square local area B_iProportional to the size of the hair region R _ hair intersection area; c (B)_i) For measuring background smoothness of local area, C (B)_i) Is a linear interpolation between 0.1 and 1; d (B)_i) For measuring the degree of separability of foreground and background colors in local regions, D (B)_i) Is a linear interpolation between 0.1 and 1.

2. The variable width portrait fine matting method as claimed in claim 1, characterized by: and carrying out binarization segmentation on the original image to be subjected to matting to further obtain a background area.

3. A variable width portrait fine matting method as claimed in claim 1 or 2, characterized by: the method for determining the hair area and the non-hair area of the portrait absolute foreground area comprises the following steps:

marking an area within a certain pixel of the top position of the head of the obtained portrait absolute foreground area in the vertical direction from the top end as a determined hair area;

and marking a rectangular area formed by connecting a center connecting line of the positions of the eyes of the obtained portrait absolute foreground area and the bottom position of the head as a non-hair area.

4. A variable width portrait fine matting method as claimed in claim 3, characterized by: before determining the hair region, the method further comprises the following steps:

acquiring the average brightness of an area within a certain pixel in the vertical direction from the top end of the head, and comparing the average brightness with a set threshold;

when the brightness is larger than the threshold value, judging that the original image is an abnormal image, namely an optical head condition, directly carrying out soft segmentation matting operation on the original image, and extracting a foreground target;

and when the brightness is smaller than the threshold value, judging that the original image is a normal image, marking an area within a certain pixel of the top vertical direction of the distance from the top of the head of the obtained portrait absolute foreground area as a determined hair area, and marking a rectangular area formed by connecting the center connecting line of the positions of the human eyes of the obtained portrait absolute foreground area and the bottom of the head as a non-hair area.

5. The variable width portrait fine matting method as claimed in claim 2, characterized by: the radius of the locally optimal unknown region is the square local region B_iThe cross area of the hair region R _ hair is larger than that of the square local region B_i1/3 of area, then the current square local region B is characterized_iIs in the hair region, said H (B)_i)＝1；

When the square local area B_iThe cross area of the hair region R _ hair is smaller than that of the square local region B_i1/10 areas, the current square local area B is characterized_iNot in the hair region, said H (B)_i) 0.1; each point p on the boundary_iIn the radius of the unknown region of (1), H (B)_i) Is a linear interpolation between 0.1 and 1.

6. The variable width portrait fine matting method as claimed in claim 2, characterized by: the parameter C (B) for measuring the background smoothness of the local area in the radius of the locally optimal unknown area_i) The calculation method of (2) comprises:

calculating a gradient map of the input image by using a sobel operator principle, and recording the gradient map as D;

carrying out binarization processing on the gradient map D, and reserving the pixel position with larger gradient;

in a square local area B_iWhen a pixel satisfies: (1) the image is marked as a background in the image binarization segmentation result; (2) if the gradient value is larger, marking the pixel as a large gradient background pixel;

counting the number of the background pixels with large gradient, and dividing the number by the size of the local area to obtain an area ratio;

when the obtained area ratio is larger than a set larger threshold value, C (B)_i) When the absolute background area is 0.1, the characterization of the absolute background area is complex;

c (B) when the obtained area ratio is smaller than another set smaller threshold value_i) 1, representing absolute background areas is simple;

each point p on the boundary_iIn the radius of the unknown region of (2), C (B)_i) Is a linear interpolation between 0.1 and 1.

7. The variable width portrait fine matting method as claimed in claim 2, characterized by: in the radius of the locally optimal unknown region, a parameter D (B) for measuring the separability of foreground and background colors of the local region_i) The calculation method of (2) comprises:

finding out the absolute foreground region in the square local region Bi, wherein the absolute foreground region is in the human image segmentation, the distance from the initial segmentation boundary is greater than a fixed threshold value, and marking the color as f₁，f₂，...，f_n；

In a square local area B_iFinding out the pixel set which is in the absolute background area of the human image segmentation and has the distance from the initial segmentation boundary larger than a fixed threshold value, and marking the color as b₁，b₂，...，b_m；

Respectively adopt f₁，f₂，...，f_nAnd b₁，b₂，...，b_mFitting Gaussian color model G_fAnd G_bCalculating normalized foreground probability

Taking the calculated normalized foreground probability as an average background pixel foreground probability

Foreground probability when background pixelD (B) is greater than a predetermined threshold value of the threshold interval_i) Setting the color of the representation absolute foreground area to be 0.1, wherein the color of the representation absolute foreground area is similar to that of the representation absolute background area;

foreground probability when background pixelD (B) when the threshold value is smaller than another preset smaller threshold value of the threshold interval_i) Setting the color of the representation absolute foreground area and the color of the representation absolute background area as 1, and easily distinguishing the representation absolute foreground area and the representation absolute background area;

each point p on the boundary_iIn the radius of the unknown region of (A), D (B)_i) Is a linear interpolation between 0.1 and 1.

8. The utility model provides a meticulous matting device of portrait of variable width which characterized in that: the width-variable portrait fine matting device adopts the width-variable portrait fine matting method of any one of claims 1 to 7 to realize the acquisition of portrait matting results without obvious flaws; the width-variable portrait fine matting device comprises:

the binarization segmentation module is used for carrying out portrait binarization segmentation pretreatment on the obtained original image to be subjected to matting, and separating to obtain a portrait absolute foreground region and an portrait absolute background region; the method is also used for carrying out binarization secondary segmentation on the hair area and the non-hair area determined in the absolute foreground area to obtain a segmentation result of the hair area;

an unknown region acquisition module for determining each point p on the boundary according to the hair region_iDetermining a square local area B centered on the point_iAnd calculating the radius r of the locally optimal unknown region_iDrawing an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm along the boundary of the local radius of each point;

and the soft segmentation matting module is used for carrying out soft segmentation matting operation in the original image according to the acquired original image and the unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm obtained by calculation so as to extract a foreground target.

9. A computer device comprising a memory and a processor, the memory storing at least one instruction, at least one program, set of codes, or set of instructions, wherein the processor when loading and executing the at least one instruction, at least one program, set of codes, or set of instructions implements the steps of the method of any one of claims 1 to 7.

10. A storage medium storing at least one instruction, at least one program, set of codes or set of instructions, wherein said at least one instruction, at least one program, set of codes or set of instructions, when loaded and executed by a processor, implements the steps of the method of any one of claims 1 to 7.

Technical Field

The invention belongs to the technical field of image processing, and particularly relates to a variable-width portrait fine matting method, a variable-width portrait fine matting device, variable-width portrait fine matting equipment and a variable-width portrait fine matting storage medium.

Background

In digital certificate photo applications, the foreground person and background are typically separated by automatic matting and the background is then replaced with a desired solid background (e.g., white or blue). The snap-fit technology is divided into a hard snap-fit technology and a soft snap-fit technology. In the hardsnap, each pixel is marked as a complete foreground or background, and the foreground-to-background transition is relatively hard, so that the method is suitable for foreground objects with relatively clear edges. The soft snap technique is called alpha matching, and can finely divide a large-area semitransparent area (such as a hair tip area) of a foreground edge. Because the portrait generally has more hair areas, the portrait can be better scratched by using the soft button drawing method, so that the final background replacement effect is more natural.

Existing soft matting techniques generally comprise the following three steps:

firstly, binary segmentation (binary segmentation) of a foreground and a background is carried out to determine an approximate outline of the foreground;

secondly, creating a trimap from the binary segmentation result, and dividing the trimap into three areas: absolute foreground, absolute background, unknown area of foreground edge.

And thirdly, performing finer foreground edge soft segmentation in the gray unknown region by using an alpha matching algorithm.

Wherein, the creation of the second step trimap is more critical. Studies have shown that trimap has a large impact on the final results. In general, trimaps are generated by fixed-size erosion and dilation operators. That is, the radius of the unknown region is the same at each location of the outline of the foreground object. There are two problems with doing so:

(1) the size of the radius of the unknown area required for the hair portion and the clothing portion is different for the portrait. The hair has more hair, and the whole hair can be covered by a larger unknown area; whereas the garment portion is a relatively sharp edge requiring a very small radius.

(2) Even in the hair area, if the background has a more complex texture, using a very large unknown area will cause many foreground estimation errors in the area, so that the final matting effect has obvious defects.

In order to solve similar problems, in some recent studies, deep learning methods are adopted to generate a reasonable trimap. However, the deep learning-based method has high requirements for training data, poor generalization capability, large computation amount, and complicated configuration at the mobile end.

Disclosure of Invention

In order to solve the problem that the radius of an unknown region of a trimap generated by a trimap created in the prior art is the same at each part of the external outline of a front scene, so that the final matting effect has obvious flaws, the invention provides a variable-width portrait fine matting method, a variable-radius unknown region is created according to the content of an image, and a relatively accurate trimap is generated, thereby obtaining a final portrait matting result without obvious flaws.

The invention is realized by adopting the following technical scheme:

a variable width portrait fine matting method is used for changing the unknown area radius of the hair area of the original image and generating an accurate trisection image to obtain a portrait matting result without obvious flaws; the width-variable portrait fine matting method comprises the following steps:

step one, obtaining an original image to be scratched, carrying out portrait binaryzation segmentation pretreatment, and separating to obtain a portrait absolute foreground area;

step two, determining a hair area and a non-hair area of the portrait absolute foreground area, and obtaining a segmentation result of the hair area through binary secondary segmentation, wherein the hair area is marked as R _ hair;

step three, acquiring each point p on the determined boundary of the portrait absolute foreground area_iDetermining a square local area B centered on the point_iAnd calculating the radius r of the locally optimal unknown region_iDrawing an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm along the boundary of the local radius of each point;

and fourthly, according to the acquired original image and the unknown region with variable width of the edge of the absolute foreground region of the portrait obtained by calculation, carrying out soft segmentation matting operation in the original image to extract a foreground target.

Wherein each point p on the boundary_iSquare local area B of_iIs locally optimized, the radius r of the unknown region_iComprises the following steps:

r_i＝r_max·min(H(B_i)，C(B_i)，D(B_i))

wherein r _ max is a preset maximum radius; h (B)_i) Is a linear interpolation between 0.1 and 1, and the square local area B_iProportional to the size of the hair region R _ hair intersection area; c (B)_i) For measuring background smoothness of local area, C (B)_i) Is a linear interpolation between 0.1 and 1; d (B)_i) For measuring the degree of separability of foreground and background colors in local regions, D (B)_i) Is a linear interpolation between 0.1 and 1.

Further, the binary segmentation of the original image to be subjected to matting also obtains a background area.

Further, the method for determining the hair area and the non-hair area of the portrait absolute foreground area comprises the following steps:

carrying out face key point detection on the obtained portrait absolute foreground area to obtain a head top position, a human eye position and a head bottom position;

marking an area within a certain pixel of the top position of the head of the obtained portrait absolute foreground area in the vertical direction from the top end as a determined hair area;

and marking a rectangular area formed by connecting a center connecting line of the positions of the eyes of the obtained portrait absolute foreground area and the bottom position of the head as a non-hair area.

Further, before determining the hair region, the method further comprises:

acquiring the average brightness of an area within a certain pixel in the vertical direction from the top end of the head, and comparing the average brightness with a set threshold;

when the brightness is larger than the threshold value, judging that the original image is an abnormal image, namely an optical head condition, directly carrying out soft segmentation matting operation on the original image, and extracting a foreground target;

and when the brightness is smaller than the threshold value, judging that the original image is a normal image, marking an area within a certain pixel of the top vertical direction of the distance from the top of the head of the obtained portrait absolute foreground area as a determined hair area, and marking a rectangular area formed by connecting the center connecting line of the positions of the human eyes of the obtained portrait absolute foreground area and the bottom of the head as a non-hair area.

Further, of the radii of the locally optimal unknown regions, the square local region B_iThe cross area of the hair region R _ hair is larger than that of the square local region B_i1/3 of area, then the current square local region B is characterized_iIs in the hair region, said H (B)_i)＝1；

When the square local area B_iThe cross area of the hair region R _ hair is smaller than that of the square local region B_i1/10 areas, the current square local area B is characterized_iNot in the hair region, said H (B)_i) 0.1; each point p on the boundary_iIn the radius of the unknown region，H(B_i) Is a linear interpolation between 0.1 and 1.

Further, in the radius of the locally optimal unknown region, a parameter C (B) for measuring the background smoothness of the local region_i) The calculation method of (2) comprises:

calculating a gradient map of the input image by using a sobel operator principle, and recording the gradient map as D;

carrying out binarization processing on the gradient map D, and reserving the pixel position with larger gradient;

in a square local area B_iWhen a pixel satisfies: (1) the image is marked as a background in the image binarization segmentation result; (2) if the gradient value is larger, marking the pixel as a large gradient background pixel;

counting the number of the background pixels with large gradient, and dividing the number by the size of the local area to obtain an area ratio;

when the obtained area ratio is larger than a set larger threshold value, C (B)_i) When the absolute background area is 0.1, the characterization of the absolute background area is complex;

c (B) when the obtained area ratio is smaller than another set smaller threshold value_i) 1, representing absolute background areas is simple;

each point p on the boundary_iIn the radius of the unknown region of (2), C (B)_i) Is a linear interpolation between 0.1 and 1.

Further, in the radius of the locally optimal unknown region, a parameter D (B) for measuring the degree of separability of foreground and background colors of the local region_i) The calculation method of (2) comprises:

in a square local area B_iFinding out the pixel set which is in the absolute foreground region of the human image segmentation and has the distance from the initial segmentation boundary larger than a fixed threshold value, and marking the color as f₁，f₂，...，f_n；

In a square local area B_iFinding out the pixel set which is in the absolute background area of the human image segmentation and has the distance from the initial segmentation boundary larger than a fixed threshold value, and marking the color as b₁，b₂，...，b_m；

Respectively adopt f₁，f₂，...，f_nAnd b₁，b₂，...，b_mFitting Gaussian color model G_fAnd G_bCalculating normalized foreground probability

Taking the calculated normalized foreground probability as an average background pixel foreground probability

Foreground probability when background pixelD (B) is greater than a predetermined threshold value of the threshold interval_i) Setting the color of the representation absolute foreground area to be 0.1, wherein the color of the representation absolute foreground area is similar to that of the representation absolute background area;

foreground probability when background pixelD (B) when the threshold value is smaller than another preset smaller threshold value of the threshold interval_i) Setting the color of the representation absolute foreground area and the color of the representation absolute background area as 1, and easily distinguishing the representation absolute foreground area and the representation absolute background area;

each point p on the boundary_iIn the radius of the unknown region of (A), D (B)_i) Is a linear interpolation between 0.1 and 1.

The invention also comprises a width-variable portrait fine matting device which adopts the width-variable portrait fine matting method to realize the acquisition of portrait matting results without obvious flaws; the width-variable portrait fine matting device comprises a binarization segmentation module, an unknown region acquisition module and a soft segmentation matting module.

The binarization segmentation module is used for carrying out portrait binarization segmentation pretreatment on the original image to be subjected to matting, and separating to obtain a portrait absolute foreground region and an portrait absolute background region; and the method is also used for carrying out binarization secondary segmentation on the hair area and the non-hair area determined in the absolute foreground area to obtain a segmentation result of the hair area.

The unknown region acquisition module is used for determining each point p on the boundary according to the hair region_iDetermining a square local area B centered on the point_iAnd calculating the radius ri of the local optimal unknown region, and drawing the unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm along the boundary of the local radius of each point.

The soft segmentation matting module is used for carrying out soft segmentation matting operation in the original image according to the acquired original image and an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm obtained through calculation, and extracting a foreground target.

The present invention also includes a computer device comprising a memory storing at least one instruction, at least one program, set of codes, or set of instructions, and a processor that implements the steps of the variable width portrait fine matting method when the at least one instruction, at least one program, set of codes, or set of instructions is loaded and executed.

The present invention also includes a storage medium having stored thereon at least one instruction, at least one program, set of codes, or set of instructions which, when loaded and executed by a processor, performs the steps of a variable width method of fine matting of images.

The technical scheme provided by the invention has the following beneficial effects:

the technical scheme provided by the invention is that an unknown area with variable radius is created according to the image content, the portrait segmentation is carried out aiming at the specific application scene of the certificate photo, a more accurate trimap can be generated according to the binarization segmentation result of the portrait, and the trimap with variable width is generated for the soft segmentation of the portrait, so that the final portrait matting result without obvious flaws is obtained. The invention considers the position of the hair area and the information of local background smoothness, thereby effectively avoiding obvious defects generated by matting and improving the success rate of replacing the background by the certificate photo. The method can be applied to software for electronic certificate photo shooting and post-processing. After the user takes the portrait photo by the mobile phone, the invention can be used for portrait matting and background replacement to generate high-quality certificate photo photos.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:

fig. 1 is a flowchart of a variable-width portrait fine matting method according to embodiment 1 of the present invention.

Fig. 2 is a schematic diagram of the effect of the matting operation in the variable-width portrait fine matting method in embodiment 1 of the present invention. (a) Inputting an original image; (b) automatic portrait binaryzation segmentation; (c) the width-variable ternary image, black, white and ash correspond to an absolute foreground, an absolute background and an unknown area respectively; (d) and finally, matting and changing the background result.

Fig. 3 is a flowchart of determining hair regions and non-hair regions in a variable width portrait fine matting method according to embodiment 1 of the present invention.

Fig. 4 is an effect diagram of dividing hair regions in the variable width portrait fine matting method in embodiment 1 of the present invention. (e) Determining a hair region (a semitransparent region) and a labeling region of other regions (black) in the binarized and segmented portrait region; (f) and obtaining a segmentation result of the hair area by using a segmentation algorithm.

Fig. 5 is a flowchart of determining an abnormal image and a normal image in the variable-width portrait fine matting method in embodiment 1 of the present invention.

Fig. 6 is a system block diagram of a variable-width fine portrait matting device in embodiment 5 of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The invention provides a variable-width fine portrait matting method, which is characterized in that an unknown area with variable radius is created according to image content to generate a more accurate trimap, so that a final portrait matting result without obvious flaws is obtained.

Example 1

As shown in fig. 1, the present embodiment provides a variable width portrait fine matting method for changing the unknown region radius of the hair region of the original image, generating an accurate trisection image to obtain a portrait matting result without obvious flaws; the width-variable portrait fine matting method comprises the following steps:

s101: and acquiring an original image to be subjected to image matting, performing image binarization segmentation pretreatment, and separating to obtain an image absolute foreground region.

In the embodiment of the present invention, the binary segmentation of the original image to be subjected to matting further obtains a background region.

Referring to fig. 1 and 2, an input portrait image is given as an original image, as shown in fig. 2 (a). First, a binary image segmentation algorithm is used to obtain a binary segmentation result, as shown in (b) of fig. 2. The binary portrait segmentation has a plurality of mature algorithms and a plurality of available cloud end APIs, such as the portrait segmentation API provided by face + +.

S102: and determining a hair area and a non-hair area of the absolute foreground area of the portrait, and obtaining a segmentation result of the hair area through binary secondary segmentation, wherein the hair area is marked as R _ hair.

In this step, the area of hair is continuously segmented from the binarized portrait foreground obtained in step S101. In the embodiment of the present invention, referring to fig. 3, the method for determining a hair region and a non-hair region of an absolute foreground region of a portrait includes:

s201, marking an area within a certain pixel of the top vertical direction of the distance from the top of the head of the obtained portrait absolute foreground area as a determined hair area;

s202, marking a rectangular area formed by connecting a center connecting line of the positions of human eyes of the obtained portrait absolute foreground area and the position of the bottom of the head as a non-hair area.

Before determining the hair area and the non-hair area of the portrait absolute foreground area, preferably, the top position of the head, the positions of the eyes and the bottom position of the head can be obtained in a mode of detecting key points of the face through the acquired portrait absolute foreground area, so that the hair area and the non-hair area can be more accurately distinguished.

In the present embodiment, first, it is assumed that the hair region is at the top of the portrait region, and a narrow region is determined at the topmost end of the portrait as the determined hair region. To deal with this particular case of an optical head, before determining the hair region, as shown in fig. 5, it further includes:

s211, acquiring the average brightness of an area within a certain pixel in the vertical direction from the top end of the head, and comparing the average brightness with a set threshold;

s212, when the brightness is larger than the threshold value, judging that the original image is an abnormal image, namely an optical head condition, and directly performing soft segmentation matting operation on the original image to extract a foreground target;

and S213, when the brightness is smaller than the threshold value, judging that the original image is a normal image, marking an area within a certain pixel of the top position of the head of the obtained portrait absolute foreground area in the vertical direction from the top end as a determined hair area, and marking a rectangular area formed by connecting a center connecting line of the positions of the eyes of the obtained portrait absolute foreground area and the bottom position of the head as a non-hair area.

Namely: the average brightness of the pixels in the area is checked first, when the brightness is greater than a threshold (e.g. 100, in the range of [0, 255]), the image is determined to be an abnormal image, the algorithm skips step 102 and step 103, directly generates a trimap with a small radius, and then enters step 104.

If the image is normal, a region other than hair is determined inside the portrait again in accordance with a predetermined template, as shown in (e) of fig. 4. Specifically, in the binarized and segmented portrait area, the top area (within 20 pixels from the top end in the vertical direction) is marked as a determined hair area, and referring to (f) of fig. 4, a rectangular area formed by connecting the centers of two human eyes (the positions of the human eyes can be detected through key points of the human face) and the bottom end of the head is marked as a non-hair area.

After determining the hair and non-hair regions, calculating a hair segmentation region by using a traditional image binarization segmentation algorithm GrabCT, and marking the region as R _ hair. The hair region is a subset of the portrait region.

S103: obtaining each point p on the determined boundary of the portrait absolute foreground area_iDetermining a square local area B centered on the point_iAnd calculating the radius r of the locally optimal unknown region_iAnd drawing an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm along the boundary of the local radius of each point.

In the present embodiment, among others, each point p on the boundary_iSquare local area B of_iIs locally optimized, the radius r of the unknown region_iComprises the following steps:

r_i＝r_max·min(H(B_i)，C(B_i)，D(B_i))

where r _ max is a preset maximum radius, e.g., 50 pixels.

Wherein, H (B)_i) Is a linear interpolation between 0.1 and 1, and the square local area B_iProportional to the size of the area of intersection of the hair region R _ hair. The radius of the locally optimal unknown region is the square local region B_iThe cross area of the hair region R _ hair is larger than that of the square local region B_i1/3 of area, then the current square local region B is characterized_iIs in the hair region, said H (B)_i) 1. When the square local area B_iThe cross area of the hair region R _ hair is smaller than that of the square local region B_i1/10 areas, the current square local area B is characterized_iNot in the hair region, said H (B)_i) 0.1; the boundaryEach point p on_iIn the radius of the unknown region of (1), H (B)_i) Is a linear interpolation between 0.1 and 1.

Namely: h (B)_i) Is a number between 0.1 and 1, and the square partial region B_iProportional to the size of the area of intersection of the hair region R _ hair. If the intersection area is greater than B_i1/3 of area, then indicate that the current local region is in the hair region, then H (B)_i) 1. If the intersection area is smaller than B_i1/10 of area, indicating that the current area is substantially absent from the hair area (B)_i) 0.1. H (B) if the intersection area is between the above two values_i) Also linearly interpolated between 0.1 and 1.

Wherein, C (B)_i) For measuring background smoothness of local area, C (B)_i) A linear interpolation between 0.1 and 1. The parameter C (B) for measuring the background smoothness of the local area in the radius of the locally optimal unknown area_i) The calculation method of (2) comprises: calculating a gradient map of the input image by using a sobel operator principle, and recording the gradient map as D; carrying out binarization processing on the gradient map D, and reserving the pixel position with larger gradient; in a square local area B_iWhen a pixel satisfies: (1) the image is marked as a background in the image binarization segmentation result; (2) if the gradient value is larger, marking the pixel as a large gradient background pixel; counting the number of the background pixels with large gradient, and dividing the number by the size of the local area to obtain an area ratio; when the obtained area ratio is larger than a set larger threshold value, C (B)_i) When the absolute background area is 0.1, the characterization of the absolute background area is complex; c (B) when the obtained area ratio is smaller than another set smaller threshold value_i) 1, representing absolute background areas is simple; each point p on the boundary_iIn the radius of the unknown region of (2), C (B)_i) Is a linear interpolation between 0.1 and 1.

Namely: c (B)_i) This parameter is mainly used to measure the background smoothness of the local area, and is also a number between 0.1 and 1. Specifically, if the texture of the local area background is complex, the front background degree separability is low, and C (B)_i) A relatively small value is taken. On the contrary, if the local area image is relatively smooth, the degree of separability of foreground and background is relatively high, C (B)_i) A value close to 1 is taken. Specifically, first, a gradient map of the input image is calculated by a sobel operator, which is denoted by D. And carrying out binarization processing on the D, and only keeping the pixel position with larger gradient. Finally, in the local area, if one pixel satisfies two conditions: (1) absolute background area in trimap; (2) if the gradient value is larger, the pixel is marked as a large gradient background pixel. The number of such pixels is counted and then divided by the size of the local area to obtain an area ratio. If the ratio is greater than a predetermined threshold, for example, 5%, C (B) indicating that the background is complicated_i) 0.1. If the ratio is less than a predetermined threshold, such as 0.5%, then the background is relatively simple, and C (B)_i) 1. If the ratio is in the middle, then C (B)_i) Linearly interpolating between 0.1 and 1.

Wherein D (B)_i) For measuring the degree of separability of foreground and background colors in local regions, D (B)_i) Is a linear interpolation between 0.1 and 1. In the radius of the locally optimal unknown region, a parameter D (B) for measuring the separability of foreground and background colors of the local region_i) The calculation method of (2) comprises: in a square local area B_iFinding out the pixel set which is in the absolute foreground region of the human image segmentation and has the distance from the initial segmentation boundary larger than a fixed threshold value, and marking the color as f₁，f₂，...，f_n(ii) a In a square local area B_iFinding out the pixel set which is in the absolute background area of the human image segmentation and has the distance from the initial segmentation boundary larger than a fixed threshold value, and marking the color as b₁，b₂，...，b_m(ii) a Respectively adopt f₁，f₂，...，f_nAnd b₁，b₂，...，b_mFitting Gaussian color model G_fAnd G_bCalculating normalized foreground probabilityTaking the calculated normalized foreground probability as an average background pixel foreground probabilityForeground probability when background pixelD (B) is greater than a predetermined threshold value of the threshold interval_i) Setting the color of the representation absolute foreground area to be 0.1, wherein the color of the representation absolute foreground area is similar to that of the representation absolute background area; foreground probability when background pixelD (B) when the threshold value is smaller than another preset smaller threshold value of the threshold interval_i) Setting the color of the representation absolute foreground area and the color of the representation absolute background area as 1, and easily distinguishing the representation absolute foreground area and the representation absolute background area; each point p on the boundary_iIn the radius of the unknown region of (A), D (B)_i) Is a linear interpolation between 0.1 and 1.

Namely: d (B)_i) This parameter is used to measure the degree of foreground and background color separation in a local region. Firstly at B_iFinding the pixel set in the trimap in the absolute foreground area, and marking the color as f₁，f₂，...，f_n. Similarly, can be at B_iFinding the pixel set in the trimap in the absolute background area, and marking the color as b₁，b₂，...，b_m. Respectively using f under the assumption that the background color distribution in the front of the local area is relatively simple₁，f₂，...，f_nAnd b₁，b₂，...，b_mFitting Gaussian color model G_fAnd G_b. For a background pixel b_iNormalized foreground probability can be calculatedThen, an average background pixel foreground probability is calculatedIf the probability is greater than a predetermined threshold (e.g., 0.2), indicating that the foreground and background colors are relatively similar in the local region, D (B) is determined_i) Set to 0.1. If the probability is less than another threshold, such as 0.01, then the background color is better differentiated before D (B)_i) Is set to 1. If the probability is between two thresholds, D (B)_i) Linearly interpolating between 0.1 and 1.

After the local radius of each point of the foreground boundary is calculated, the unknown region inside the trimap can be drawn along the boundary, as shown in fig. 2 (c).

S104: and according to the acquired original image and the unknown region with variable width of the edge of the absolute foreground region of the portrait obtained by calculation, carrying out soft segmentation matting operation in the original image to extract a foreground target.

With the input original image and the unknown area with the variable width of the calculated trimap, the existing alphamatting algorithm can be called to carry out the soft segmentation of the foreground. The algorithms commonly used include robust matching, KNN matching, etc.

In the embodiment of the present invention, in step 102, the hair region may be segmented by any binary image segmentation method. In step 104, after the unknown region with variable width of trimap is determined, any alphamatting algorithm can be used to realize the final soft segmentation result.

The technical scheme provided by the invention is that an unknown area with variable radius is created according to the image content, the portrait segmentation is carried out aiming at the specific application scene of the certificate photo, a more accurate trimap can be generated according to the binarization segmentation result of the portrait, and the trimap with variable width is generated for the soft segmentation of the portrait, so that the final portrait matting result without obvious flaws is obtained. The invention considers the position of the hair area and the information of local background smoothness, thereby effectively avoiding obvious defects generated by matting and improving the success rate of replacing the background by the certificate photo.

Example 2

As shown in fig. 6, in an embodiment of the present invention, there is provided a variable width portrait fine matting device comprising: the device comprises a binarization segmentation module, an unknown region acquisition module and a soft segmentation matting module.

The binarization segmentation module is used for carrying out portrait binarization segmentation pretreatment on the obtained original image to be subjected to matting, and separating to obtain a portrait absolute foreground region and an portrait absolute background region; and the method is also used for carrying out binarization secondary segmentation on the hair area and the non-hair area determined in the absolute foreground area to obtain a segmentation result of the hair area.

The unknown region acquisition module is used for determining each point p on the boundary according to the hair region_iDetermining a square local area B centered on the point_iAnd calculating the radius r of the locally optimal unknown region_iAnd drawing an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm along the boundary of the local radius of each point.

The soft segmentation matting module is used for carrying out soft segmentation matting operation in the original image according to the acquired original image and an unknown region with variable width of the edge of the portrait absolute foreground region in the trimap algorithm obtained through calculation, and extracting a foreground target.

The step of the variable-width portrait fine matting method is adopted when the variable-width portrait fine matting device is executed, and the variable-width portrait fine matting method can be applied to electronic certificate photo shooting and post-processing software. After the user takes the portrait photo by the mobile phone, the invention can be used for portrait matting and background replacement to generate high-quality certificate photo photos. Therefore, the operation process of the variable-width portrait fine matting device in the embodiment will not be described in detail.

Example 3

In an embodiment of the present invention, a computer device is provided, which comprises a memory in which at least one instruction, at least one program, set of codes, or set of instructions is stored, and a processor, which when loaded and executed carries out the steps of the above-mentioned method embodiments.

Example 4

In an embodiment of the present invention, a storage medium is provided, on which at least one instruction, at least one program, set of codes or set of instructions is stored, which when loaded and executed by a processor, implements the steps of the above-described method embodiments.

One of ordinary skill in the art will appreciate that all or part of the processes of the methods of the above embodiments may be implemented by at least one instruction, at least one program, code set, or instruction set, which may be stored in a non-volatile computer-readable storage medium, and the at least one instruction, at least one program, code set, or instruction set may include the processes of the embodiments of the methods described above when executed. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory.

In summary, the present invention can perform a second segmentation of one hair region based on the result of the binarized segmentation of the portrait, and then use a relatively large radius for the hair region and a relatively small radius for other regions (clothes, skin, etc.). And in each local hair area, the size of the radius is adjusted according to the local background complexity judgment result. The method does not need to use data for training, so the operation speed is high and the deployment is convenient.

The technical scheme provided by the invention is that an unknown area with variable radius is created according to the image content, the portrait segmentation is carried out aiming at the specific application scene of the certificate photo, a more accurate trimap can be generated according to the binarization segmentation result of the portrait, and the trimap with variable width is generated for the soft segmentation of the portrait, so that the final portrait matting result without obvious flaws is obtained. The invention considers the position of the hair area and the information of local background smoothness, thereby effectively avoiding obvious defects generated by matting and improving the success rate of replacing the background by the certificate photo. The method can be applied to software for electronic certificate photo shooting and post-processing. After the user takes the portrait photo by the mobile phone, the invention can be used for portrait matting and background replacement to generate high-quality certificate photo photos.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.