Video clipping method and device, terminal equipment and readable storage medium
阅读说明：本技术 视频裁剪方法、装置、终端设备以及可读存储介质 (Video clipping method and device, terminal equipment and readable storage medium ) 是由 刘思阳 于 2021-09-28 设计创作，主要内容包括：本发明提供了一种视频裁剪方法、装置、终端设备以及可读存储介质。该方法包括：获取待裁剪视频的每帧图片所对应的裁剪框,然后按照预设数量和每帧图片的时间顺序,对裁剪框进行分组,得到多组裁剪框,再剔除每一组裁剪框中的异常裁剪框,对剔除异常裁剪框后的每一组裁剪框进行均值计算,得到每一组裁剪框对应的多帧图片中第一帧图片的裁剪框,进而根据每一组中的第一帧图片的裁剪框,对相邻两组中的第一帧图片之间的每一帧图片的裁剪框进行差值计算,得到相邻两组中的第一帧图片之间的每一帧图片的裁剪框,最后根据每一帧图片的裁剪框,对待裁剪视频进行裁剪。所述方法能够避免连续多帧图片的裁剪框发生突变而造成的视频存在抖动的问题。(The invention provides a video clipping method, a video clipping device, terminal equipment and a readable storage medium. The method comprises the following steps: the method comprises the steps of obtaining cutting frames corresponding to each frame of picture of a video to be cut, grouping the cutting frames according to the preset number and the time sequence of each frame of picture to obtain multiple groups of cutting frames, then eliminating abnormal cutting frames in each group of cutting frames, carrying out mean value calculation on each group of cutting frames after the abnormal cutting frames are eliminated to obtain the cutting frame of a first frame of picture in multiple frames of pictures corresponding to each group of cutting frames, further carrying out difference value calculation on the cutting frame of each frame of picture between the first frames of pictures in two adjacent groups according to the cutting frame of the first frame of picture in each group to obtain the cutting frame of each frame of picture between the first frames of pictures in two adjacent groups, and finally cutting the video to be cut according to the cutting frame of each frame of picture. The method can avoid the problem of video jitter caused by abrupt change of the cropping frame of continuous multi-frame pictures.)
1. A method of video cropping, comprising:
acquiring an original cutting frame corresponding to each frame of picture of a video to be cut;
grouping the original cutting frames according to a preset number and the time sequence of each frame of picture to obtain a plurality of groups of original cutting frames;
rejecting abnormal clipping frames in each group of original clipping frames;
carrying out mean value calculation on each group of original cutting frames after the abnormal cutting frames are removed to obtain updated cutting frames of the first frame of pictures in the multi-frame pictures corresponding to each group of original cutting frames;
according to the updated cutting frame of the first frame picture in each group, calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups to obtain the updated cutting frame of each frame picture between the first frame pictures in the two adjacent groups;
and cutting the video to be cut according to the updated cutting frame of each frame of picture.
2. The method of video cropping according to claim 1, wherein culling the outlier crop box in each set of original crop boxes comprises:
carrying out mean value calculation on each group of original cutting frames to obtain a first average cutting frame corresponding to each group of original cutting frames;
and rejecting abnormal clipping frames in each group of original clipping frames according to a preset rejection condition and the first average clipping frame corresponding to each group of original clipping frames.
3. Video cropping method according to claim 2, characterized in that,
the preset rejection conditions comprise center rejection conditions, width rejection conditions and height rejection conditions, and the abnormal cutting frames comprise center abnormal cutting frames, width abnormal cutting frames and height abnormal cutting frames;
rejecting abnormal clipping frames in each group of original clipping frames according to a preset rejection condition and a first average clipping frame corresponding to each group of original clipping frames, and the method comprises the following steps:
according to the center removing condition and a first average cutting frame corresponding to each group of original cutting frames, removing the center abnormal cutting frame in each group of original cutting frames;
rejecting the width abnormal clipping frame in each group of original clipping frames according to the width rejection condition and the first average clipping frame corresponding to each group of original clipping frames;
and rejecting the height abnormal clipping frames in each group of original clipping frames according to the height rejection condition and the first average clipping frame corresponding to each group of original clipping frames.
4. The method of claim 3, wherein the culling the outlier center crop box in each of the plurality of original crop boxes based on the center culling condition and the first average crop box corresponding to each of the plurality of original crop boxes comprises:
determining a first distance value between the clipping center of the original clipping frame and the clipping center of a first average clipping frame corresponding to the original clipping frame of each group of original clipping frames;
determining the sum of the clipping width and the clipping height of a first average clipping frame corresponding to the group of original clipping frames to obtain a first rejection value;
when the first distance value is larger than the product of the first elimination value and a first preset parameter, determining the original cutting frame as the center abnormal cutting frame;
and removing the central abnormal clipping frame in each group of original clipping frames.
5. The method of claim 3, wherein the removing the width-abnormal crop box from each set of original crop boxes according to the width removal condition and the first average crop box corresponding to each set of original crop boxes comprises:
determining a first absolute distance value of the cutting width of the original cutting frame and the cutting width of a first average cutting frame corresponding to the original cutting frame of each group aiming at any original cutting frame in the original cutting frames;
when the first absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a second preset parameter, determining the original clipping frame as the width abnormal clipping frame;
and rejecting the width abnormal clipping frames in each group of original clipping frames.
6. The video cropping method of claim 3, wherein the culling of the outlier cropping frame of each original set of cropping frames based on the height culling criteria and the first average cropping frame corresponding to each original set of cropping frames comprises:
determining a second absolute distance value between the clipping height of the original clipping frame and the clipping height of a first average clipping frame corresponding to the original clipping frame of each group of original clipping frames;
when the second absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a third preset parameter, determining the original clipping frame as the height abnormal clipping frame;
and rejecting the height abnormal clipping boxes in each group of original clipping boxes.
7. A video cropping device, comprising:
the acquisition module is used for acquiring an original cutting frame corresponding to each frame of picture of a video to be cut;
the grouping module is used for grouping the original cutting frames according to the preset quantity and the time sequence of each frame of picture to obtain a plurality of groups of original cutting frames;
the rejection module is used for rejecting abnormal clipping frames in each group of original clipping frames;
the first obtaining module is used for carrying out mean value calculation on each group of original cutting frames after the abnormal cutting frames are removed to obtain updated cutting frames of the first frame of pictures in the multi-frame pictures corresponding to each group of original cutting frames;
the second obtaining module is used for calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups according to the updated cutting frame of the first frame picture in each group to obtain the updated cutting frame of each frame picture between the first frame pictures in the two adjacent groups;
and the cutting module is used for cutting the video to be cut according to the updated cutting frame of each frame of picture.
8. A video cropping device according to claim 1, characterized in that said culling module comprises:
the first obtaining submodule is used for carrying out mean value calculation on each group of original cutting frames to obtain a first average cutting frame corresponding to each group of original cutting frames;
and the eliminating sub-module is used for eliminating the abnormal cutting frames in each group of original cutting frames according to the preset eliminating conditions and the first average cutting frames corresponding to each group of original cutting frames.
9. The terminal equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the steps of the video cropping method of any of claims 1 to 6 when executing a program stored on a memory.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the video cropping method according to any one of claims 1 to 6.
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a video clipping method, an apparatus, a terminal device, and a readable storage medium.
Background
And intercepting the video frames of partial areas in the video to form a new video service scene. A crop box needs to be computed for each frame and then the content within the crop box is extracted in order to re-compose a new video. However, since the cropping frame is calculated independently for each frame, there are problems that the calculated cropping frame is large for a while and small for a while and is small for a while, and finally the resultant video is jittered and the video quality is very poor.
Disclosure of Invention
The invention aims to provide a video clipping method, a video clipping device, a terminal device and a readable storage medium. The specific technical scheme is as follows:
in a first aspect of the embodiments of the present invention, a video cropping method is provided first, where the method includes:
acquiring an original cutting frame corresponding to each frame of picture of a video to be cut;
grouping the original cutting frames according to a preset number and the time sequence of each frame of picture to obtain a plurality of groups of original cutting frames;
rejecting abnormal clipping frames in each group of original clipping frames;
carrying out mean value calculation on each group of original cutting frames after the abnormal cutting frames are removed to obtain updated cutting frames of the first frame of pictures in the multi-frame pictures corresponding to each group of original cutting frames;
according to the updated cutting frame of the first frame picture in each group, calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups to obtain the updated cutting frame of each frame picture between the first frame pictures in the two adjacent groups;
and cutting the video to be cut according to the updated cutting frame of each frame of picture.
Optionally, culling the abnormal crop box in each set of original crop boxes includes:
carrying out mean value calculation on each group of original cutting frames to obtain a first average cutting frame corresponding to each group of original cutting frames;
and rejecting abnormal clipping frames in each group of original clipping frames according to a preset rejection condition and the first average clipping frame corresponding to each group of original clipping frames.
Optionally, the preset rejection conditions include a center rejection condition, a width rejection condition and a height rejection condition, and the abnormal clipping boxes include a center abnormal clipping box, a width abnormal clipping box and a height abnormal clipping box;
rejecting abnormal clipping frames in each group of original clipping frames according to a preset rejection condition and a first average clipping frame corresponding to each group of original clipping frames, and the method comprises the following steps:
according to the center removing condition and a first average cutting frame corresponding to each group of original cutting frames, removing the center abnormal cutting frame in each group of original cutting frames;
rejecting the width abnormal clipping frame in each group of original clipping frames according to the width rejection condition and the first average clipping frame corresponding to each group of original clipping frames;
and rejecting the height abnormal clipping frames in each group of original clipping frames according to the height rejection condition and the first average clipping frame corresponding to each group of original clipping frames.
Optionally, rejecting the center abnormal clipping frame in each group of original clipping frames according to the center rejection condition and the first average clipping frame corresponding to each group of original clipping frames includes:
determining a first distance value between the clipping center of the original clipping frame and the clipping center of a first average clipping frame corresponding to the original clipping frame of each group of original clipping frames;
determining the sum of the clipping width and the clipping height of a first average clipping frame corresponding to the group of original clipping frames to obtain a first rejection value;
when the first distance value is larger than the product of the first elimination value and a first preset parameter, determining the original cutting frame as the center abnormal cutting frame;
and removing the central abnormal clipping frame in each group of original clipping frames.
Optionally, rejecting the width abnormal clipping box in each group of original clipping boxes according to the width rejection condition and the first average clipping box corresponding to each group of original clipping boxes includes:
determining a first absolute distance value of the cutting width of the original cutting frame and the cutting width of a first average cutting frame corresponding to the original cutting frame of each group aiming at any original cutting frame in the original cutting frames;
when the first absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a second preset parameter, determining the original clipping frame as the width abnormal clipping frame;
and rejecting the width abnormal clipping frames in each group of original clipping frames.
Optionally, rejecting a height abnormal clipping frame in each group of original clipping frames according to the height rejection condition and a first average clipping frame corresponding to each group of original clipping frames, includes:
determining a second absolute distance value between the clipping height of the original clipping frame and the clipping height of a first average clipping frame corresponding to the original clipping frame of each group of original clipping frames;
when the second absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a third preset parameter, determining the original clipping frame as the height abnormal clipping frame;
and rejecting the height abnormal clipping boxes in each group of original clipping boxes.
In a second aspect of the embodiments of the present invention, there is also provided a video cropping apparatus, including:
the acquisition module is used for acquiring an original cutting frame corresponding to each frame of picture of a video to be cut;
the grouping module is used for grouping the original cutting frames according to the preset quantity and the time sequence of each frame of picture to obtain a plurality of groups of original cutting frames;
the rejection module is used for rejecting abnormal clipping frames in each group of original clipping frames;
the first obtaining module is used for carrying out mean value calculation on each group of original cutting frames after the abnormal cutting frames are removed to obtain updated cutting frames of the first frame of pictures in the multi-frame pictures corresponding to each group of original cutting frames;
the second obtaining module is used for calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups according to the updated cutting frame of the first frame picture in each group to obtain the updated cutting frame of each frame picture between the first frame pictures in the two adjacent groups;
and the cutting module is used for cutting the video to be cut according to the updated cutting frame of each frame of picture.
Optionally, the culling module comprises:
the first obtaining submodule is used for carrying out mean value calculation on each group of original cutting frames to obtain a first average cutting frame corresponding to each group of original cutting frames;
and the eliminating sub-module is used for eliminating the abnormal cutting frames in each group of original cutting frames according to the preset eliminating conditions and the first average cutting frames corresponding to each group of original cutting frames.
Optionally, the preset rejection conditions include a center rejection condition, a width rejection condition and a height rejection condition, and the abnormal clipping boxes include a center abnormal clipping box, a width abnormal clipping box and a height abnormal clipping box;
the culling submodule comprises:
the first eliminating unit is used for eliminating the center abnormal cutting frame in each group of original cutting frames according to the center eliminating condition and the first average cutting frame corresponding to each group of original cutting frames;
the second eliminating unit is used for eliminating the width abnormal clipping frame in each group of original clipping frames according to the width eliminating conditions and the first average clipping frame corresponding to each group of original clipping frames;
and the third eliminating unit is used for eliminating the height abnormal cutting frames in each group of original cutting frames according to the height eliminating conditions and the first average cutting frames corresponding to each group of original cutting frames.
Optionally, the first rejecting unit comprises:
the first determining subunit is used for determining, for any one original clipping frame in each group of original clipping frames, a first distance value between the clipping center of the original clipping frame and the clipping center of a first average clipping frame corresponding to the group of original clipping frames;
a first obtaining subunit, configured to determine a sum of a clipping width and a clipping height of a first average clipping frame corresponding to the set of original clipping frames, to obtain a first culling value;
a second determining subunit, configured to determine, when the first distance value is greater than a product of the first culling value and a first preset parameter, that the original crop box is the center abnormal crop box;
and the first eliminating subunit is used for eliminating the central abnormal clipping frame in each group of original clipping frames.
Optionally, the second rejection unit comprises:
a third determining subunit, configured to determine, for any original clipping frame in each set of original clipping frames, a first absolute distance value between the clipping width of the original clipping frame and the clipping width of the first average clipping frame corresponding to the set of original clipping frames;
a fourth determining subunit, configured to determine, when the first absolute distance value is greater than a product of a clipping width of the first average clipping frame corresponding to the group of original clipping frames and a second preset parameter, that the original clipping frame is the width-abnormal clipping frame;
and the second eliminating subunit is used for eliminating the width abnormal clipping frame in each group of original clipping frames.
Optionally, the third rejecting unit comprises:
a fifth determining subunit, configured to determine, for any one of the first and second groups of original crop frames, a second absolute distance value between the clipping height of the original crop frame and the clipping height of the first average crop frame corresponding to the first group of original crop frames;
a sixth determining subunit, configured to determine, when the second absolute distance value is greater than a product of a clipping width of the first average clipping frame corresponding to the set of original clipping frames and a third preset parameter, that the original clipping frame is the height-abnormal clipping frame;
and the third eliminating subunit is used for eliminating the height abnormal clipping frames in each group of original clipping frames.
In a third aspect of the embodiments of the present invention, there is further provided a terminal device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
the processor is configured to implement the steps in the video cropping method according to the first aspect of the embodiment of the present invention when executing the program stored in the memory.
In yet another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium having stored therein instructions, which when run on a computer, cause the computer to perform the steps of any of the video cropping methods described above.
In yet another aspect of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the video cropping methods described above.
The video clipping method provided by the embodiment of the invention comprises the steps of firstly obtaining clipping frames corresponding to each frame of picture of a video to be clipped, then grouping the clipping frames according to the preset quantity and the time sequence of each frame of picture to obtain a plurality of groups of clipping frames, then eliminating abnormal clipping frames in each group of clipping frames, carrying out mean value calculation on each group of clipping frames after eliminating the abnormal clipping frames to obtain the clipping frame of the first frame of picture in a plurality of frames of picture corresponding to each group of clipping frames, further carrying out difference value calculation on the clipping frames of each frame of picture between the first frame of pictures in two adjacent groups according to the clipping frame of the first frame of picture in each group to obtain the clipping frame of each frame of picture between the first frame of pictures in two adjacent groups, and finally clipping the video to be clipped according to the clipping frame of each frame of picture. The method has the following technical effects:
the method comprises the steps of dividing cutting frames into groups according to the preset number and the time sequence of each frame of picture to obtain multiple groups of cutting frames, eliminating abnormal cutting frames in each group of cutting frames, carrying out mean value calculation on each group of cutting frames after the abnormal cutting frames are eliminated to obtain the cutting frame of a first frame of picture in multiple frames of pictures corresponding to each group of cutting frames, enabling each group of calculated cutting frames to be more accurate and uniform, avoiding the phenomenon that the calculated cutting frames are inaccurate due to extreme cutting frames under few conditions, and improving the accuracy of the obtained cutting frame of the first frame of picture in the multiple frames of pictures corresponding to each group of cutting frames.
And secondly, calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups according to the cutting frame of the first frame picture in each group to obtain the cutting frame of each frame picture between the first frame pictures in the two adjacent groups, so that the cutting frames in each group can be uniformly changed, the problem of video jitter caused by abrupt change of the cutting frames of continuous multi-frame pictures can be avoided on the premise of ensuring video cutting, and the quality of the cut synthesized video is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flow chart illustrating a method of video cropping according to an embodiment of the present application;
FIG. 2 is a process diagram of a video cropping method according to an embodiment of the present application;
FIG. 3 is a flow diagram illustrating a culling center exception crop box according to an embodiment of the present application;
FIG. 4 is a flow diagram illustrating a culling width exception crop box according to an embodiment of the present application;
FIG. 5 is a flow diagram illustrating a culling height anomaly crop box according to an embodiment of the invention;
fig. 6 is a block diagram illustrating a structure of a video cropping device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.
In a service scene of video composition, video frames of partial areas need to be cut from different videos so as to compose a new video.
One solution in the related art is: a crop box is calculated for each frame and the content within the crop box is then extracted to resynthesize the new video. However, since the cropping frame is calculated independently for each frame, there are problems that the calculated cropping frame is large for a while and small for a while and is small for a while, and finally the resultant video is jittered and the video quality is very poor.
In order to solve the problem that the synthesized video has jitter caused by the sudden change of the size of the cropping frame in the related art, the application provides a technical idea that: the method comprises the steps of grouping the cutting frames, then removing the abnormal cutting frames in each group of cutting frames, calculating to obtain the average cutting frame of each group of cutting frames after the abnormal cutting frames are removed, and then calculating the difference value of the cutting frames between the two groups of average cutting frame pairs so as to obtain the cutting frame of each frame of picture, thereby avoiding the problem that the video shakes due to the fact that the cutting frames of continuous multi-frame pictures are mutated.
The image processing method of the present application will be described in detail below. Fig. 1 is a flowchart illustrating a video cropping method according to an embodiment of the present application. Fig. 2 is a schematic process diagram of a video cropping method according to an embodiment of the present application, where a detection frame in fig. 2 is a cropping frame, and referring to fig. 1 and fig. 2, the video cropping method of the present application may include the following steps:
step S11: and acquiring an original cutting frame corresponding to each frame of picture of the video to be cut.
In this embodiment, a video to be cropped is a target video that needs to be cropped to synthesize a new video, and each frame of picture of the video to be cropped needs to be cropped.
Step S12: and grouping the original cutting frames according to the preset number and the time sequence of each frame of picture to obtain a plurality of groups of original cutting frames.
In this embodiment, in order to obtain a more accurate average crop box, a plurality of original crop boxes need to be grouped first, specifically, the plurality of original crop boxes may be grouped according to a preset number and a time sequence of each frame of picture, and since a video is time-ordered, the plurality of original crop boxes are grouped according to the time sequence of each frame of picture, so that a more reasonable crop box can be obtained, where the preset number may be an FPS value, and for example, the preset number may be 25 or 30. Every n crop boxes can be grouped into a group, and the grouping expression is as follows: si＝{b1，b2,...，bnN is a preset number, b is an original clipping frame, SiFor grouping, i is a group number, e.g. S1Is a first group, S2Is a second group, b1For the first original crop box within the group, bnThe nth crop box within the group.
Step S13: and rejecting abnormal clipping frames in each group of original clipping frames.
In this embodiment, in order to avoid the phenomenon that the calculated cropping frame is inaccurate due to the extreme cropping frame in a few cases, it is necessary to remove the abnormal cropping frame in each group of the original cropping frames, so as to improve the accuracy of the obtained cropping frame of the first frame picture in the multi-frame pictures corresponding to each group of the cropping frames.
In summary of the above embodiments, in one implementation, the removing the abnormal crop box in each set of original crop boxes may include the following steps:
step S131: and carrying out mean value calculation on each group of original clipping frames to obtain a first average clipping frame corresponding to each group of original clipping frames.
In this embodiment, each group of original crop frames is calculated to obtain a first average crop frame, and specifically, performing an average calculation on each group of original crop frames respectively includes: and respectively carrying out clipping center average value calculation, clipping width average value calculation and clipping height average value calculation on each group of original clipping frames so as to obtain a first average clipping frame corresponding to each group of original clipping frames. For example, the average cropping center of the ith group of crop boxes is ciAverage cutting width of wiAverage cutting height of hi。
Step S132: and rejecting abnormal clipping frames in each group of original clipping frames according to a preset rejection condition and the first average clipping frame corresponding to each group of original clipping frames.
In this embodiment, the preset rejection condition is a preset condition for determining whether an abnormal clipping frame exists in each group of original clipping frames, and when an original clipping frame in each group of original clipping frames and a first average clipping frame corresponding to the group of original clipping frames satisfy the preset rejection condition, it is determined that the original clipping frame is the abnormal clipping frame and needs to be rejected.
The method comprises the steps of obtaining a first average cutting frame corresponding to each group of original cutting frames by carrying out mean value calculation on each group of original cutting frames, and removing abnormal cutting frames in each group of original cutting frames according to preset removing conditions and the first average cutting frames corresponding to each group of original cutting frames, so that extreme cutting frames under few conditions can be removed well, the phenomenon that the calculated cutting frames are inaccurate due to the extreme cutting frames under few conditions is avoided, and the accuracy of the obtained cutting frames of the first frame picture in the multi-frame picture corresponding to each group of cutting frames is improved.
Specifically, the preset rejection conditions include a center rejection condition, a width rejection condition and a height rejection condition, and the abnormal clipping frames include a center abnormal clipping frame, a width abnormal clipping frame and a height abnormal clipping frame.
Rejecting abnormal clipping frames in each group of original clipping frames according to a preset rejection condition and a first average clipping frame corresponding to each group of original clipping frames, and the method comprises the following steps:
according to the center removing condition and a first average cutting frame corresponding to each group of original cutting frames, removing the center abnormal cutting frame in each group of original cutting frames;
rejecting the width abnormal clipping frame in each group of original clipping frames according to the width rejection condition and the first average clipping frame corresponding to each group of original clipping frames;
and rejecting the height abnormal clipping frames in each group of original clipping frames according to the height rejection condition and the first average clipping frame corresponding to each group of original clipping frames.
In this embodiment, the preset rejection conditions include a center rejection condition, a width rejection condition, and a height rejection condition, and as long as the original crop box satisfies any one of the rejection conditions, the original crop box can be determined to be an abnormal crop box, so that the abnormal crop box is rejected, and a more accurate average crop box can be obtained.
Specifically, the central rejection conditions are: dst (c)j，ci)＞αc(wi+hi) I.e. for SiAn original cutting frame b insidejIf dst (c)j，ci)＞αc(wi+hi) Then the original crop box bjThe box is cropped for exceptions.
Fig. 3 is a flowchart illustrating an embodiment of the present application for removing a center abnormal crop box, and with reference to fig. 3, in an implementation manner, by combining the above embodiments, the removing the center abnormal crop box in each group of original crop boxes according to the center removing condition and the first average crop box corresponding to each group of original crop boxes includes the following steps:
step S301: and determining a first distance value between the clipping center of the original clipping frame and the clipping center of a first average clipping frame corresponding to the original clipping frame in each original clipping frame group.
In the present embodiment, dst (c)j，ci) I.e. the first distance value between the clipping center of the original clipping frame and the clipping center of the first average clipping frame corresponding to the set of original clipping frames, dst is the distance between two points, cjFor the original cutting frame bjC cutting center ofiFor the ith set of original crop boxes SiThe cutting center of (1).
Step S302: and determining the sum of the clipping width and the clipping height of the first average clipping frame corresponding to the group of original clipping frames to obtain a first elimination value.
In this embodiment, the clipping width and the clipping height of the first average clipping frame corresponding to the set of original clipping frames are obtained first, and then added to obtain a first rejection value, that is, w in the above-mentioned central rejection conditioni+hi。
Step S303: and when the first distance value is larger than the product of the first elimination value and a first preset parameter, determining the original cutting frame as the center abnormal cutting frame.
In the present embodiment, the first preset parameter αcWhen the first distance value is larger than the product of the first elimination value and the first preset parameter, the original clipping frame is determined to be the central abnormal clipping frame, namely, the original clipping frame meets the central elimination condition, and the original clipping frame can be determined to be the central abnormal clipping frame.
Step S304: and removing the central abnormal clipping frame in each group of original clipping frames.
In this embodiment, the center abnormality determination is performed on the original clipping frames in each group, and after the original clipping frame is determined to be an abnormal clipping frame, the original clipping frame is removed from the group of original clipping frames, so that the phenomenon that the calculated clipping frame is inaccurate due to extreme clipping frames in a few cases is avoided.
The width elimination conditions are as follows: abs (w)j-wi)＞αwwiI.e. for SiAn original cutting frame insidebjIf abs (w)j-wi)＞αwwiThen the original crop box bjThe box is cropped for exceptions.
Fig. 4 is a flowchart of removing a width abnormal crop box according to an embodiment of the present application, and with reference to fig. 4, in an implementation manner, by combining the above embodiments, the removing the width abnormal crop box in each group of original crop boxes according to the width removal condition and the first average crop box corresponding to each group of original crop boxes includes the following steps:
step S401: and determining a first absolute distance value of the cutting width of the original cutting frame and the cutting width of a first average cutting frame corresponding to the original cutting frame in each group of original cutting frames.
In this embodiment, abs (w)j-wi) That is, the first absolute distance value between the cutting width of the original cutting frame and the cutting width of the first average cutting frame corresponding to the set of original cutting frames, abs is the absolute value of the difference between the two values, abs (wj-wi) In, wjOriginal cutting frame bkCutting width of, wiFor the ith set of original crop boxes SiThe cutting width of (1) is calculated by first calculating wjAnd wiAnd averaging the obtained difference values to obtain a first absolute distance value.
Step S402: and when the first absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a second preset parameter, determining the original clipping frame as the width abnormal clipping frame.
In the present embodiment, the second preset parameter αwWhen the first absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a second preset parameter, determining that the original clipping frame is an abnormal-width clipping frame, namely, the original clipping frame meets the width eliminating condition, and determining that the original clipping frame is the abnormal-width clipping frame.
Step S403: and rejecting the width abnormal clipping frames in each group of original clipping frames.
In this embodiment, the width abnormality determination is performed on the original clipping frames in each group, and after the original clipping frame is determined to be an abnormal clipping frame, the original clipping frame is removed from the group of original clipping frames, so that the phenomenon that the calculated clipping frame is inaccurate due to extreme clipping frames in a few cases is avoided.
The high rejection conditions were: abs (h)j-hi)＞αhhiI.e. for SiAn original cutting frame b insidejIf abs (h)j-hi)＞αhhiThen the original crop box is an abnormal crop box.
Fig. 5 is a flowchart illustrating an embodiment of removing a height abnormal crop box, and with reference to fig. 5, in an implementation manner, by integrating the above embodiments, according to a height removal condition and a first average crop box corresponding to each group of original crop boxes, a height abnormal crop box in each group of original crop boxes is removed, which includes the following steps:
step S501: and determining a second absolute distance value between the clipping height of the original clipping frame and the clipping height of the first average clipping frame corresponding to the original clipping frame in each group of original clipping frames.
In this embodiment, abs (h)j-hi>A second absolute distance value abs (h) between the clipping height of the original clipping frame and the clipping height of the first average clipping frame corresponding to the set of original clipping framesj-hi) In (h)jFor the original cutting frame bjCutting height of hiFor the ith set of original crop boxes SiThe cutting height of h is calculated firstjAnd hiAnd averaging the obtained difference values to obtain a second absolute distance value.
Step S502: and when the second absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a third preset parameter, determining the original clipping frame as the height abnormal clipping frame.
In the present embodiment, the third preset parameter αhWhen the second absolute distance value is larger than the product of the clipping width of the first average clipping frame corresponding to the group of original clipping frames and a third preset parameter, determining that the original clipping frame is a height abnormal clipping frame, namely, the original clipping frame meets the height elimination condition, and determining that the original clipping frame is the height abnormal clipping frame.
Step S503: and rejecting the height abnormal clipping boxes in each group of original clipping boxes.
In this embodiment, the height abnormality determination is performed on the original clipping frames in each group, and after the original clipping frame is determined to be an abnormal clipping frame, the original clipping frame is removed from the group of original clipping frames, so that the phenomenon that the calculated clipping frame is inaccurate due to extreme clipping frames in a few cases is avoided.
In the specific method for rejecting the abnormal crop box in each group of the original crop boxes, for a part of the original crop boxes, two or three of the center abnormal crop box, the width abnormal crop box and the height abnormal crop box may be simultaneously satisfied, and in order to simplify the determination process, the determination process and the rejection process may be performed in a certain order, for example, the execution order may be: the method comprises the steps of firstly executing the step of removing the center abnormal cutting frame, then removing the width abnormal cutting frame from each group of original cutting frames from which the center abnormal cutting frame is removed, and finally removing the height abnormal cutting frame from each group of original cutting frames from which the width abnormal cutting frame is removed, thereby obtaining a final removal result.
Step S14: and performing mean value calculation on each group of original cutting frames after the abnormal cutting frames are removed to obtain the updated cutting frame of the first frame of picture in the multi-frame pictures corresponding to each group of original cutting frames.
In this embodiment, performing mean value calculation on each group of original clipping frames from which the abnormal clipping frames are removed, including performing mean value calculation of clipping centers, mean value calculation of clipping widths, and mean value calculation of clipping heights, specifically, averaging the clipping centers of all remaining original clipping frames in the group to obtain an average clipping center; averaging the clipping widths of all the remaining original clipping frames in the group to obtain an average clipping width; averaging the clipping heights of all the remaining original clipping frames in the group to obtain an average clipping height; and the obtained average cutting frame is used as an updated cutting frame of a first frame of picture in the multi-frame pictures corresponding to the original cutting frame of the group, the cutting frames in each group are sequenced according to the time sequence, and the first frame of picture in the multi-frame pictures corresponding to the original cutting frame of the group is the picture corresponding to the minimum timestamp in the group.
Step S15: and according to the updated cutting frame of the first frame picture in each group, performing difference value calculation on the cutting frame of each frame picture between the first frame pictures in the two adjacent groups to obtain the updated cutting frame of each frame picture between the first frame pictures in the two adjacent groups.
According to the steps, the cutting frame corresponding to the first frame picture in each group is updated and arranged according to the time sequence, and a plurality of frame pictures are arranged between the first frame pictures in the two adjacent groups, so that the cutting frame corresponding to the remaining plurality of frame pictures is required to be updated in order to make the video obtained by cutting and synthesizing more smooth.
Specifically, the difference calculation is performed on the cropping frame of each frame picture between the first frame pictures in the two adjacent groups according to the updated cropping frame of the first frame picture in each group, and the specific calculation includes the difference calculation of the cropping center, the difference calculation of the cropping width and the difference calculation of the cropping height, for example, for the difference calculation of the cropping width, each group has 30 frames, 29 frames of pictures are still between the first frame pictures in the two adjacent groups, the cropping width of the updated cropping frame corresponding to the first frame picture in the previous group is 200, the cropping width of the updated cropping frame corresponding to the first frame picture in the next group is 230, the average difference between the two adjacent frames of pictures is calculated first, so that the difference of the cropping widths of the adjacent updated cropping frames in the group is equal, that is, the width of the updated cropping frame corresponding to each frame picture in the previous group is 200, and the width of the updated cropping frame picture in the previous group is 230, 201. 202 … … 229, calculating the difference value of the cropping frame of each picture between the first pictures in the two adjacent groups, so that the cropping frame in each group can be changed uniformly, and the problem of video jitter caused by abrupt change of the cropping frame of continuous multi-frame pictures can be avoided on the premise of ensuring video cropping, so as to improve the quality of the cropped and synthesized video.
Step S16: and cutting the video to be cut according to the updated cutting frame of each frame of picture.
In this embodiment, the updated cropping frame of each frame of picture can be obtained through the above steps, and then the video to be cropped can be cropped according to the updated cropping frame of each frame of picture, so that the problem that the cropping frame is large for a moment and small for a moment and is previous for a moment can be avoided, and the composite video can be prevented from shaking.
The image processing method of the present application has a plurality of technical effects as follows:
the method comprises the steps of dividing cutting frames into groups according to the preset number and the time sequence of each frame of picture to obtain multiple groups of cutting frames, eliminating abnormal cutting frames in each group of cutting frames, carrying out mean value calculation on each group of cutting frames after the abnormal cutting frames are eliminated to obtain the cutting frame of a first frame of picture in multiple frames of pictures corresponding to each group of cutting frames, enabling each group of calculated cutting frames to be more accurate and uniform, avoiding the phenomenon that the calculated cutting frames are inaccurate due to extreme cutting frames under few conditions, and improving the accuracy of the obtained cutting frame of the first frame of picture in the multiple frames of pictures corresponding to each group of cutting frames.
And secondly, calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups according to the cutting frame of the first frame picture in each group to obtain the cutting frame of each frame picture between the first frame pictures in the two adjacent groups, so that the cutting frames in each group can be uniformly changed, the problem of video jitter caused by abrupt change of the cutting frames of continuous multi-frame pictures can be avoided on the premise of ensuring video cutting, and the quality of the cut synthesized video is improved.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Based on the same inventive concept, an embodiment of the present invention provides an image processing apparatus 600. Referring to fig. 6, fig. 6 is a block diagram illustrating a video cropping device according to an embodiment of the present application. As shown in fig. 6, the apparatus 600 includes:
an obtaining module 601, configured to obtain an original clipping frame corresponding to each frame of a video to be clipped;
a grouping module 602, configured to group the original crop boxes according to a preset number and a time sequence of each frame of picture, so as to obtain multiple groups of original crop boxes;
a removing module 603, configured to remove an abnormal clipping frame in each set of original clipping frames;
a first obtaining module 604, configured to perform mean value calculation on each group of original crop frames from which the abnormal crop frame is removed, to obtain an updated crop frame of a first frame of picture in multiple frames of pictures corresponding to each group of original crop frames;
a second obtaining module 605, configured to perform difference calculation on the clipping frame of each frame picture between the first frame pictures in the two adjacent groups according to the updated clipping frame of the first frame picture in each group, to obtain an updated clipping frame of each frame picture between the first frame pictures in the two adjacent groups;
and the clipping module 606 is configured to clip the video to be clipped according to the updated clipping frame of each frame of the picture.
Optionally, the culling module comprises:
the first obtaining submodule is used for carrying out mean value calculation on each group of original cutting frames to obtain a first average cutting frame corresponding to each group of original cutting frames;
and the eliminating sub-module is used for eliminating the abnormal cutting frames in each group of original cutting frames according to the preset eliminating conditions and the first average cutting frames corresponding to each group of original cutting frames.
Optionally, the preset rejection conditions include a center rejection condition, a width rejection condition and a height rejection condition, and the abnormal clipping boxes include a center abnormal clipping box, a width abnormal clipping box and a height abnormal clipping box;
the culling submodule comprises:
the first eliminating unit is used for eliminating the center abnormal cutting frame in each group of original cutting frames according to the center eliminating condition and the first average cutting frame corresponding to each group of original cutting frames;
the second eliminating unit is used for eliminating the width abnormal clipping frame in each group of original clipping frames according to the width eliminating conditions and the first average clipping frame corresponding to each group of original clipping frames;
and the third eliminating unit is used for eliminating the height abnormal cutting frames in each group of original cutting frames according to the height eliminating conditions and the first average cutting frames corresponding to each group of original cutting frames.
Optionally, the first rejecting unit comprises:
the first determining subunit is used for determining, for any one original clipping frame in each group of original clipping frames, a first distance value between the clipping center of the original clipping frame and the clipping center of a first average clipping frame corresponding to the group of original clipping frames;
a first obtaining subunit, configured to determine a sum of a clipping width and a clipping height of a first average clipping frame corresponding to the set of original clipping frames, to obtain a first culling value;
a second determining subunit, configured to determine, when the first distance value is greater than a product of the first culling value and a first preset parameter, that the original crop box is the center abnormal crop box;
and the first eliminating subunit is used for eliminating the central abnormal clipping frame in each group of original clipping frames.
Optionally, the second rejection unit comprises:
a third determining subunit, configured to determine, for any original clipping frame in each set of original clipping frames, a first absolute distance value between the clipping width of the original clipping frame and the clipping width of the first average clipping frame corresponding to the set of original clipping frames;
a fourth determining subunit, configured to determine, when the first absolute distance value is greater than a product of a clipping width of the first average clipping frame corresponding to the group of original clipping frames and a second preset parameter, that the original clipping frame is the width-abnormal clipping frame;
and the second eliminating subunit is used for eliminating the width abnormal clipping frame in each group of original clipping frames.
Optionally, the third rejecting unit comprises:
a fifth determining subunit, configured to determine, for any one of the first and second groups of original crop frames, a second absolute distance value between the clipping height of the original crop frame and the clipping height of the first average crop frame corresponding to the first group of original crop frames;
a sixth determining subunit, configured to determine, when the second absolute distance value is greater than a product of a clipping width of the first average clipping frame corresponding to the set of original clipping frames and a third preset parameter, that the original clipping frame is the height-abnormal clipping frame;
and the third eliminating subunit is used for eliminating the height abnormal clipping frames in each group of original clipping frames.
An embodiment of the present invention further provides a terminal device, as shown in fig. 7. Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. Referring to fig. 7, the terminal device includes a processor 71, a communication interface 72, a memory 73 and a communication bus 74, wherein the processor 71, the communication interface 72 and the memory 73 communicate with each other through the communication bus 74;
a memory 73 for storing a computer program;
the processor 71, when executing the program stored in the memory 73, implements the following steps:
acquiring an original cutting frame corresponding to each frame of picture of a video to be cut;
grouping the original cutting frames according to a preset number and the time sequence of each frame of picture to obtain a plurality of groups of original cutting frames;
rejecting abnormal clipping frames in each group of original clipping frames;
carrying out mean value calculation on each group of original cutting frames after the abnormal cutting frames are removed to obtain updated cutting frames of the first frame of pictures in the multi-frame pictures corresponding to each group of original cutting frames;
according to the updated cutting frame of the first frame picture in each group, calculating the difference value of the cutting frame of each frame picture between the first frame pictures in the two adjacent groups to obtain the updated cutting frame of each frame picture between the first frame pictures in the two adjacent groups;
and cutting the video to be cut according to the updated cutting frame of each frame of picture.
Or the processor 71, when executing a program stored on the memory 73, performs the steps in the other method embodiments described above.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In yet another embodiment of the present invention, a computer-readable storage medium is further provided, which has instructions stored therein, and when the instructions are executed on a computer, the instructions cause the computer to execute the video cropping method described in any one of the above embodiments.
In yet another embodiment, a computer program product containing instructions is provided, which when run on a computer, causes the computer to perform the video cropping method of any of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.