阅读说明：本技术 运动矢量的确定方法、装置、电子设备和可读存储介质 (Method and device for determining motion vector, electronic equipment and readable storage medium ) 是由 张弓 于 2020-06-29 设计创作，主要内容包括：本申请涉及一种运动矢量的确定方法、装置、电子设备和可读存储介质。所述方法包括：获取第一运动矢量,其中,所述第一运动矢量与当前待插值块有位置交叠关系；根据与所述当前待插值块存在时空关联关系的候选插值块,确定所述候选插值块对应的第二运动矢量；根据所述第一运动矢量和所述第二运动矢量,确定目标运动矢量。上述方案,充分考虑到了与当前待插值块存在关联关系的运动矢量,根据这些运动矢量可以有效提高所确定的最优运动矢量的准确率。(The application relates to a method and a device for determining a motion vector, an electronic device and a readable storage medium. The method comprises the following steps: acquiring a first motion vector, wherein the first motion vector and a current block to be interpolated have a position overlapping relation; determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block having a space-time association relation with the current block to be interpolated; and determining a target motion vector according to the first motion vector and the second motion vector. According to the scheme, the motion vectors having the association relation with the current block to be interpolated are fully considered, and the accuracy of the determined optimal motion vector can be effectively improved according to the motion vectors.)

1. A method for determining a motion vector, comprising:

acquiring a first motion vector, wherein the first motion vector and a current block to be interpolated have a position overlapping relation;

determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block having a space-time association relation with the current block to be interpolated;

and determining a target motion vector according to the first motion vector and the second motion vector.

2. The method of claim 1, wherein the first motion vector and the second motion vector constitute a reference motion vector;

determining a target motion vector according to the first motion vector and the second motion vector includes:

determining a first evaluation coefficient corresponding to the first motion vector;

determining a second evaluation coefficient corresponding to the second motion vector;

and determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient.

3. The method according to claim 2, wherein when the first motion vector passes through the current block to be interpolated, it is determined that the first motion vector has a position overlapping relationship with the current block to be interpolated; the current block to be interpolated is positioned in the frame image to be interpolated;

the determining a first evaluation coefficient corresponding to the first motion vector includes:

acquiring a projection block of the first motion vector in the frame image to be interpolated;

obtaining a coincidence area according to the number of coincidence pixel points of the projection block and the current block to be interpolated;

and obtaining the first evaluation coefficient according to the overlapping area.

4. The method according to claim 3, wherein the deriving the first evaluation coefficient according to the overlapping area comprises:

determining a confidence level of the first motion vector;

and obtaining the first evaluation coefficient according to the overlapping area and the reliability.

5. The method of claim 4, wherein determining the confidence level of the first motion vector comprises:

determining front and rear matching blocks which have a projection relation with the first motion vector in front and rear adjacent original frame images of the frame image to be interpolated;

determining error evaluation information between the front and rear matching blocks;

determining a distance adjustment coefficient according to the distance between the front and back adjacent original frame images and the frame image to be interpolated;

acquiring the accuracy of the motion vector corresponding to the front and rear adjacent original frame images;

and obtaining the reliability according to at least one of the error evaluation information, the distance adjustment coefficient and the accuracy.

6. The method according to claim 2, wherein the candidate interpolation block includes a spatial neighborhood interpolation block, the motion vector corresponding to the spatial neighborhood interpolation block is the second motion vector, and the spatial neighborhood interpolation block and the current block to be interpolated are located in a frame image to be interpolated;

the determining a second evaluation coefficient corresponding to the second motion vector includes:

acquiring a spatial distance between the spatial neighborhood interpolation block and the current block to be interpolated;

and obtaining a second evaluation coefficient corresponding to the second motion vector according to the space distance.

7. The method according to claim 2, wherein the candidate interpolation block further includes a temporal neighborhood interpolation block, the motion vector corresponding to the temporal neighborhood interpolation block is the second motion vector, and the temporal neighborhood interpolation block and the current block to be interpolated are located in adjacent interpolation frame images;

the determining a second evaluation coefficient corresponding to the second motion vector includes:

acquiring the interval time of the interpolation frame image where the time neighborhood interpolation block and the current block to be interpolated are located;

and obtaining a second evaluation coefficient corresponding to the second motion vector according to the interval time.

8. The method according to claim 2, wherein determining the target motion vector from the reference motion vector based on the first evaluation coefficient and the second evaluation coefficient comprises:

normalizing the first evaluation coefficient and the second evaluation coefficient;

and determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient after normalization processing.

9. The method according to claim 8, wherein the second evaluation coefficient includes a spatial neighborhood evaluation coefficient and a temporal neighborhood evaluation coefficient; the motion vector corresponding to the spatial neighborhood evaluation coefficient is a spatial neighborhood motion vector, and the motion vector corresponding to the temporal neighborhood evaluation coefficient is a temporal neighborhood motion vector;

the determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient after the normalization processing includes:

respectively determining weighted values corresponding to the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient after normalization processing according to the quantity of the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector;

according to the weighted value, performing weighted summation operation on the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the time neighborhood evaluation coefficient after normalization processing;

and determining the target motion vector from the reference motion vector according to the result of the weighted sum operation.

10. The method according to any one of claims 2 to 9, wherein determining the target motion vector from the reference motion vector based on the first evaluation coefficient and the second evaluation coefficient comprises:

integrating the first evaluation coefficient and the second evaluation coefficient to accumulate the evaluation coefficients of the same motion vector in the first motion vector and the second motion vector to obtain a target evaluation coefficient;

and determining the reference motion vector with the maximum target evaluation coefficient as the target motion vector.

11. An apparatus for determining a motion vector, comprising:

the device comprises a first motion vector determining module, a second motion vector determining module and a interpolation module, wherein the first motion vector determining module is used for acquiring a first motion vector, and the first motion vector and a current block to be interpolated have a position overlapping relation;

the second motion vector determining module is used for determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block which has a space-time association relation with the current block to be interpolated;

and the target motion vector determining module is used for determining a target motion vector according to the first motion vector and the second motion vector.

12. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the computer program, when executed by the processor, causes the processor to perform the steps of the method according to any of claims 1 to 10.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 10.

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for determining a motion vector, an electronic device, and a computer-readable storage medium.

Background

In the processing of images or video, motion compensation is often required. At present, an interpolation frame is often blocked to obtain an interpolation block, and in a traditional fitting method for determining a motion vector, an optimal motion vector cannot be found, and robustness is low.

Disclosure of Invention

The embodiment of the application provides a method and a device for determining a motion vector, electronic equipment and a computer readable storage medium, which can effectively improve the accuracy of the determined optimal motion vector.

A method of motion vector determination, the method comprising:

acquiring a first motion vector, wherein the first motion vector and a current block to be interpolated have a position overlapping relation;

determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block having a space-time association relation with the current block to be interpolated;

and determining a target motion vector according to the first motion vector and the second motion vector.

An apparatus for determining a motion vector, the apparatus comprising:

the device comprises a first motion vector determining module, a second motion vector determining module and a interpolation module, wherein the first motion vector determining module is used for acquiring a first motion vector, and the first motion vector and a current block to be interpolated have a position overlapping relation;

the second motion vector determining module is used for determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block which has a space-time association relation with the current block to be interpolated;

and the target motion vector determining module is used for determining a target motion vector according to the first motion vector and the second motion vector.

An electronic device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

acquiring a first motion vector, wherein the first motion vector and a current block to be interpolated have a position overlapping relation;

determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block having a space-time association relation with the current block to be interpolated;

and determining a target motion vector according to the first motion vector and the second motion vector.

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

acquiring a first motion vector, wherein the first motion vector and a current block to be interpolated have a position overlapping relation;

determining a second motion vector corresponding to the candidate interpolation block according to the candidate interpolation block having a space-time association relation with the current block to be interpolated;

and determining a target motion vector according to the first motion vector and the second motion vector.

The method, the device, the electronic equipment and the storage medium for determining the motion vector acquire a first motion vector which has a position overlapping relation with a current block to be interpolated; determining a second motion vector corresponding to a candidate interpolation block according to the candidate interpolation block having a space-time association relation with the current block to be interpolated; and determining a target motion vector according to the first motion vector and the second motion vector. The motion vectors having the association relation with the current block to be interpolated are fully considered, and the accuracy of the determined optimal motion vector can be effectively improved according to the motion vectors.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a diagram illustrating an exemplary embodiment of a method for determining a motion vector;

FIG. 2 is a flow diagram of a method for motion vector determination in one embodiment;

FIG. 3 is a diagram illustrating the structure of an interpolated frame image and interpolated blocks according to an embodiment;

FIG. 4 is a diagram illustrating an embodiment of a first motion vector passing through a current block to be interpolated;

FIG. 5 is a diagram illustrating the structure of an interpolated frame image and interpolated blocks according to another embodiment;

FIG. 6 is a diagram illustrating a structure of an interpolated frame image and an interpolated block according to still another embodiment;

fig. 7 is a flowchart of a method of determining a motion vector in another embodiment;

fig. 8 is a flowchart of a method of determining a motion vector in still another embodiment;

fig. 9 is a block diagram showing the configuration of the motion vector determination apparatus in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first motion vector may be referred to as a second motion vector, and similarly, a second motion vector may be referred to as a first motion vector, without departing from the scope of the present application. Both the first motion vector and the second motion vector are motion vectors, but they are not the same motion vector.

The method for determining the motion vector provided by the embodiment of the invention can be applied to the electronic equipment shown in fig. 1. As shown in fig. 1, the electronic device includes a processor and a memory connected by a system bus. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program is executable by a processor for implementing a method for determining a motion vector provided in the following embodiments. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a Point of Sales (POS), a vehicle-mounted computer, and a wearable device. In addition, the electronic device may also be a server, and may be implemented by an independent server or a server cluster formed by a plurality of servers.

Those skilled in the art will appreciate that the architecture shown in fig. 1 is a block diagram of only a portion of the architecture associated with the subject application, and does not constitute a limitation on the servers to which the subject application applies, as a particular server may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In an embodiment, as shown in fig. 2, a method for determining a motion vector is provided, and this embodiment is illustrated by applying the method to the electronic device in fig. 1, and includes the following steps:

s201, a first motion vector is obtained, wherein the first motion vector and a current block to be interpolated have a position overlapping relation.

In inter-coding, the relative displacement between the current coding block and the best matching block in its reference picture is represented by a Motion Vector (MV). After the motion vector of a certain interpolation block is determined, the motion direction and distance of the certain interpolation block relative to the matching block in the reference frame image can be obtained, and the interpolation process of the interpolation block can be further completed. The frame image here may be a certain frame in the video.

When the video needs to be subjected to motion compensation processing, an interpolation frame image needs to be inserted into an original frame image of the video, a motion vector corresponding to an interpolation block in the interpolation frame image can be determined by a motion vector determination method, and the motion vector can represent the position corresponding relation between the corresponding interpolation block and a matching block in the original frame image. After the motion vector of each interpolation block in the interpolation frame image is determined, the position corresponding relation between each matching block in the original frame image and each interpolation block in the interpolation frame image can be known, the interpolation frame image is inserted between the original frame images according to the position corresponding relation, a smooth and complete new video can be formed, and at the moment, the motion compensation of the video can be considered to be completed.

The frame image to be interpolated may be divided into image blocks by manual designation or an adaptive method (for example, dividing the frame image to be interpolated into a set number of interpolation blocks according to the size thereof), and the image blocks are identical in size and shape. When performing motion compensation, each image block may be determined as a current block to be interpolated one by one according to a sequence from top to bottom and from left to right (of course, other sequences may also be used, for example, a sequence from bottom to top and from right to left, which is not described herein again), and then a target motion vector of each current block to be interpolated is determined. The target motion vector can refer to the optimal motion vector of the current block to be interpolated, and the definition of the video after interpolation can be ensured to the maximum extent by interpolating the current block to be interpolated through the target motion vector, and meanwhile, useful image information is kept in the frame image to be interpolated as far as possible.

After the target motion vector of the current block to be interpolated is determined, the next interpolation block can be determined as the current block to be interpolated in sequence, and then the target motion vector of the next interpolation block is determined until the target motion vectors of all interpolation blocks in the frame image to be interpolated are determined. And then, the interpolation of the frame image to be interpolated can be completed according to the target motion vectors, so that the motion compensation of the video where the frame image is located is completed.

Further, an interpolation block with an obtained motion vector may exist around the current block to be interpolated. For example: according to the calculation sequence, before the target motion vector of the current block to be interpolated is determined, the motion vector of an interpolation block positioned at the left side of the current block to be interpolated is already determined. The known motion vectors have a certain incidence relation with the current block to be interpolated, and the size and the direction of the target motion vector in the current block to be interpolated can be determined to a certain extent. Thus, the target motion vector of the current block to be interpolated can be determined from the known motion vectors around the current block to be interpolated.

The first motion vector is a motion vector having a position overlapping relation with the current block to be interpolated. The position overlapping relationship may refer to a case where the first motion vector and the current block to be interpolated overlap each other under a specific spatial condition, and it may be understood that a running path of the first motion vector passes through the current block to be interpolated. Specifically, in the frame image to be interpolated, a projection block corresponding to the first motion vector and a current block to be interpolated have a coincident pixel point.

S202, according to the candidate interpolation block having the space-time association relation with the current block to be interpolated, determining a second motion vector corresponding to the candidate interpolation block.

The spatio-temporal association refers to an association existing in time or space. The candidate interpolation block refers to an interpolation block which is associated with the current block to be interpolated in time or space. Further, for the association relationship in time, the candidate interpolation block and the current block to be interpolated may be respectively located in the same position of the adjacent frame image (for example, respectively located in the middle position of the frame image); for spatial correlation, the candidate interpolation block may be in a different position of the same frame image as the current block to be interpolated.

Further, after obtaining the candidate interpolation block, the electronic device determines a motion vector corresponding to the candidate interpolation block as a second motion vector.

S203, determining a target motion vector according to the first motion vector and the second motion vector.

In step, a target motion vector is determined based on the first motion vector and the second motion vector. An optimal motion vector may be determined from the first motion vector and the second motion vector, and the optimal motion vector may be determined as the target motion vector. The first motion vector and the second motion vector may be subjected to a certain integration operation (for example, a weighted sum operation), and the motion vector obtained by the integration operation may be determined as the target motion vector.

If the target motion vector of the current block to be interpolated is determined according to the first motion vector and the second motion vector, a correlation relationship between the first motion vector and the target motion vector and a correlation relationship between the second motion vector and the target motion vector need to be established, and then the target motion vector can be determined from the motion vectors (including the first motion vector and the second motion vector) according to the correlation relationship.

Further, the first motion vector and the second motion vector may be respectively calculated by a certain algorithm as evaluation coefficients of the target motion vector, the evaluation coefficients are used to evaluate a correlation between each motion vector and the target motion vector, and an optimal motion vector, that is, the target motion vector, may be determined from each motion vector according to the evaluation coefficients. The first motion vector or the second motion vector having the highest evaluation coefficient may be determined as the target motion vector.

In addition, the number of the first motion vector and the second motion vector may be more than one, and at this time, the optimal motion vector of the first motion vector and the optimal motion vector of the second motion vector may be determined, and then the target motion vector may be determined from the two optimal motion vectors.

The method for determining the motion vector in the embodiment fully considers the motion vector having an association relationship with the current block to be interpolated, specifically, determines the first motion vector having a position overlapping relationship with the current block to be interpolated, determines the second motion vector according to the candidate interpolation block having a space-time association relationship with the current block to be interpolated, and determines the target motion vector according to the first motion vector and the second motion vector.

In one embodiment, the first motion vector and the second motion vector are motion vectors having a certain spatial-temporal association with the current block to be interpolated. The electronic device may integrate the first motion vector and the second motion vector to obtain a reference motion vector. The reference motion vector may be used as a motion vector having an association relationship with the current block to be interpolated, and a target motion vector may be determined from the reference motion vector.

The integration process may be to combine the first motion vector and the second motion vector and treat the same motion vector as one motion vector, for example: the first motion vector comprises MV1/MV2/MV3, the second motion vector comprises MV3/MV4/MV5, and if the same motion vector MV3 exists in the first motion vector and the second motion vector, the two motion vectors are combined to obtain a reference motion vector MV1/MV2/MV3/MV4/MV 5.

The embodiment can remove the overlapped motion vector from the motion vector by integrating the motion vector, so that the determined target motion vector has higher accuracy, and the execution process of the target motion vector can be simplified.

In one embodiment, the correlation relationship between the known motion vector and the target motion vector may be characterized by an evaluation coefficient. Further, the evaluation coefficient (including the first evaluation coefficient and the second evaluation coefficient) is used to represent the possibility that a certain motion vector is the target motion quality, and specifically, may be a probability value (may also be referred to as a probability coefficient) for determining that a certain motion vector is the target motion vector.

Further, the first motion vector and the second motion vector constitute a reference motion vector; determining a target motion vector according to the first motion vector and the second motion vector includes: determining a first evaluation coefficient corresponding to the first motion vector; determining a second evaluation coefficient corresponding to the second motion vector; and determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient.

After determining the first motion vector, the electronic device determines a first evaluation coefficient corresponding to the first motion vector, and after determining the second motion vector, determines a second evaluation coefficient corresponding to the second motion vector. At this time, both the first evaluation coefficient of the first motion vector and the second evaluation coefficient of the second motion vector are known, and the target motion vector can be determined from the reference motion vector based on this.

Further, the first evaluation coefficient and the second evaluation coefficient characterize the possibility that the first motion vector and the second motion vector are the target motion vector, respectively. Based on this, the target evaluation coefficients of the reference motion vectors can be obtained by integrating the first evaluation coefficient and the second evaluation coefficient, and the reference motion vector whose target evaluation coefficient satisfies the condition is determined as the target motion vector.

Further, individual ones of the reference motion vectors may be both the first and second motion vectors. In this case, the first evaluation coefficient and the second evaluation coefficient corresponding to the same motion vector may be calculated (for example, accumulated, weighted and summed) to obtain the target evaluation coefficient of the motion vector.

Further, the reference motion vector whose target evaluation coefficient is larger than the set threshold may be determined as the target motion vector, or the reference motion vector whose target evaluation coefficient is the largest may be determined as the target motion vector.

The embodiment respectively determines the evaluation coefficients of the first motion vector and the second motion vector, and further determines the target motion vector from the reference motion vector according to the evaluation coefficients, so that the determination process is simple, and meanwhile, the optimal motion vector can be accurately and intuitively determined according to the comparison of the evaluation coefficient values.

In one embodiment, when the first motion vector passes through the current block to be interpolated, the first motion vector is determined to have a position overlapping relation with the current block to be interpolated; the current block to be interpolated is positioned in the frame image to be interpolated; the determining a first evaluation coefficient corresponding to the first motion vector includes: acquiring a projection block of the first motion vector in the frame image to be interpolated; obtaining a coincidence area according to the number of coincidence pixel points of the projection block and the current block to be interpolated; and obtaining the first evaluation coefficient according to the overlapping area.

The passing of the first motion vector through the current block to be interpolated may mean that a motion path of the first motion vector passes through the current block to be interpolated.

As shown in fig. 3, the current block 311 to be interpolated is located in a frame image 321 to be interpolated, and adjacent original frame images of the frame image to be interpolated are 322 and 323, respectively. There are corresponding image blocks 312 and 313 on adjacent original frame images 322 and 323, respectively, and the motion vector from the image block 312 to the image block 313 is the first motion vector (which may be a path formed by four dotted lines as in fig. 3). The projection block of the first motion vector in the frame image 321 to be interpolated is 314, and a coincidence region 331 (a region filled with a dotted line in fig. 3) exists between the current block 311 to be interpolated and the projection block 314. Further, the number of overlapping pixels included in the overlapping area 331 is determined, and the overlapping area can be obtained according to the number of overlapping pixels, for example: if the number of the coincident pixels is 100, the coincident area may be 100, and if the number of the coincident pixels is 200, the coincident area may be 200.

The coincidence area can reflect a position management relationship between the current block to be interpolated and the first motion vector, when the coincidence area is large, the correlation between the current block to be interpolated and the first motion vector can be considered to be high, the probability that the first motion vector is the optimal motion vector of the current block to be interpolated is high, when the coincidence area is small, the correlation between the current block to be interpolated and the first motion vector can be considered to be low, and the probability that the first motion vector is the optimal motion vector of the current block to be interpolated is low. Therefore, the first evaluation coefficient for which the first motion vector is the target motion vector can be determined according to the size of the overlapping area.

Further, in one embodiment, the first evaluation coefficient may be determined according to a ratio of the overlapping area to the total area of the current block to be interpolated. The ratio of the overlapping area to the total area of the current block to be interpolated can be directly used as a first evaluation coefficient, for example: if the ratio of the coincidence area to the total area of the current block to be interpolated is 0.2, the first evaluation coefficient may be 0.2, and if the ratio of the coincidence area to the total area of the current block to be interpolated is 1 (which may mean that the path of the first motion vector completely coincides with the current block to be interpolated, and the first motion vector is most likely to be the optimal motion vector), the first evaluation coefficient may be 1; or, after a certain operation (for example, multiplying by a preset coefficient) is performed on the ratio of the overlapping area to the total area of the current block to be interpolated, the result of the operation may be determined as the first evaluation coefficient.

In addition, fig. 4 shows the position overlapping relationship between the first motion vector (including the first motion vector 1/2/3/4) and the current block to be interpolated from another angle, and the size of the projection block represented by the first motion vector is consistent with the size of the current block to be interpolated as can be seen from fig. 4, the overlapping area between different first motion vectors and the current block to be interpolated may be different, and at this time, the first evaluation coefficient may be determined according to the size of the overlapping area. Further, as is apparent from fig. 4, when the area of coincidence between the first motion vector 1 and the current block to be interpolated is the largest, the first evaluation coefficient of the first motion vector 1 may be determined as the maximum value among the 4 first motion vectors.

According to the embodiment, the first evaluation coefficient is obtained according to the overlapping area of the current block to be interpolated and the projection block, the determination process is simple, the determination efficiency of the first evaluation coefficient can be effectively improved, and the determination efficiency of the target motion vector is further improved.

In one embodiment, the obtaining the first evaluation coefficient according to the overlapping area includes: determining a confidence level of the first motion vector; and obtaining the first evaluation coefficient according to the overlapping area and the reliability.

The reliability of the first motion vector may refer to the accuracy of the first motion vector, and may be determined by a motion vector associated with the first motion vector and a correlation characteristic of the frame image.

The coincidence area and the reliability may be normalized, the coincidence area and the reliability after the normalization may be accumulated, and a result of the accumulation may be determined as the first evaluation coefficient.

Further, in an embodiment, the determining the reliability of the first motion vector includes: determining front and rear matching blocks which have a projection relation with the first motion vector in front and rear adjacent original frame images of the frame image to be interpolated; determining error evaluation information between the front and rear matching blocks; determining a distance adjustment coefficient according to the distance between the front and back adjacent original frame images and the frame image to be interpolated; acquiring the accuracy of the motion vector corresponding to the front and rear adjacent original frame images; and obtaining the reliability according to at least one of the error evaluation information, the distance adjustment coefficient and the accuracy.

Other frame images may not be spaced between adjacent original frame images and frame images to be interpolated, or a certain number of frame images may be spaced (the number of frame images may be one, two, or even a plurality of). As shown in fig. 3, the adjacent original frame images 322 and 323 can be respectively understood as front and rear adjacent original frame images of the frame image 321 to be interpolated, and 312 and 313 can be respectively understood as front and rear matching blocks having a projection relation with the first motion vector.

The error evaluation information may refer to SAD (sum of absolute error), STAD (sum of absolute conversion error), and the like between the front and rear matching blocks.

For adjacent original frame images with different distances from the frame image to be interpolated, the matching blocks in the adjacent original frame images with the close distances tend to contain more information related to the current block to be interpolated, and therefore, the corresponding motion vectors are more likely to be the target motion vectors. Based on this, it is necessary to determine a distance adjustment coefficient according to the distance of adjacent original frame images to distinguish the degree of influence of the adjacent original frame images of different distances on the target motion vector. Further, when the distance between the adjacent original frame image and the frame image to be interpolated is short, the distance adjustment coefficient may be a larger value, and when the distance between the adjacent original frame image and the frame image to be interpolated is long, the distance adjustment coefficient may be a smaller value.

Further, the distance adjustment coefficients may be characterized by texture gradients, position penalties, and the like. The texture gradient may also be referred to as a texture level difference (texture gradient), which is a monocular visual cue of depth perception, and as the distance increases, the projection of any surface on the retina is sparse at near and dense at far, so that when the distance between the adjacent original frame image and the frame image to be interpolated is short, the texture gradient is a larger value, and when the distance between the adjacent original frame image and the frame image to be interpolated is far, the texture gradient is a smaller value, and the specific value may be determined according to actual situations, which is not limited by the embodiment of the present invention. The position penalty may refer to a coefficient determined according to a distance between adjacent original frame images and a frame image to be interpolated, and a determination method thereof is similar to a texture gradient and is not described herein again.

As described in the foregoing embodiment, in the motion compensation process, the front and rear adjacent original frame images are divided into image blocks, and each image block corresponds to a motion vector. After the motion vectors corresponding to the image blocks are calculated, the corresponding accuracy can be determined according to the actual interpolation effect of the motion vectors, if the interpolation effect is good, the accuracy is high, and if the interpolation effect is poor, the accuracy is low. In this embodiment, the motion vectors of the picture blocks in the adjacent original frame images are obtained, and the accuracy of the motion vectors is further determined, so as to determine the reliability of the first motion vector.

Further, the present embodiment obtains the reliability of the first motion vector based on the determined error evaluation information, distance adjustment coefficient, and accuracy. Specifically, one of the error evaluation information, the distance adjustment coefficient, and the accuracy may be used as the reliability, for example: determining the distance adjustment coefficient obtained by calculation as the reliability of the first motion vector; operations may also be performed on two or all of them, for example: after normalization, a summation (or weighted summation) operation is performed, and the result of the operation is determined as the reliability of the first motion vector.

Specifically, the measure of the reliability of the first motion vector may be that at least one of the following items is used as the reliability of the first motion vector: SAD, STAD, texture gradient, position penalty and accuracy of corresponding motion vectors of front and back adjacent original frame images of the first motion vector between corresponding matching blocks of the two original frame images.

In one embodiment, determining the first evaluation coefficient corresponding to the first motion vector includes: determining a confidence level of the first motion vector; and obtaining the first evaluation coefficient according to the reliability. According to the embodiment, the first evaluation coefficient is determined according to the reliability of the first motion vector, the overlapping area can be not considered, the determination process is simple, the determination efficiency of the first evaluation coefficient can be effectively improved, and the determination efficiency of the target motion vector is improved.

In one embodiment, the candidate interpolation blocks include a spatial neighborhood interpolation block, a motion vector corresponding to the spatial neighborhood interpolation block is the second motion vector, and the spatial neighborhood interpolation block and the current block to be interpolated are located in a frame image to be interpolated; the determining a second evaluation coefficient corresponding to the second motion vector includes: acquiring a spatial distance between the spatial neighborhood interpolation block and the current block to be interpolated; and obtaining a second evaluation coefficient corresponding to the second motion vector (which may be referred to as a spatial neighborhood motion vector, abbreviated as a spatial neighborhood MV at this time) according to the spatial distance.

And the spatial neighborhood interpolation block and the current block to be interpolated are both positioned in the frame image to be interpolated. The spatial distance between the two central pixels can be the distance between the central pixels, namely, the number of the pixels spaced between the two central pixels can be used as the spatial distance.

The embodiment determines the second evaluation coefficient of the motion vector corresponding to the spatial neighborhood interpolation block according to the distance of the spatial distance. Further, when the spatial distance is large, the information of the spatial neighborhood interpolation block related to the current block to be interpolated may be less, and at this time, the second evaluation coefficient may be set to a smaller value, so as to reduce the possibility that the corresponding motion vector becomes the target motion vector; when the spatial distance is smaller, the information of the spatial neighborhood interpolation block related to the current block to be interpolated may be more, and at this time, the second evaluation coefficient may be set to a larger value to increase the possibility that the corresponding motion vector becomes the target motion vector.

Furthermore, the spatial distance between each pixel point in the frame image to be interpolated and the central pixel point of the current block to be interpolated can be determined, the maximum spatial distance can be found out, when the second evaluation coefficient is determined according to the spatial distance, the determined spatial distance and the maximum spatial distance can be subjected to ratio operation, and the result of the ratio operation is determined as the second evaluation coefficient.

As shown in fig. 5, the current block to be interpolated 511 and the spatial neighborhood interpolation blocks (512 and 513) are located in the frame image to be interpolated 521, a spatial distance between a center pixel point of the current block to be interpolated 511 and a center pixel point of the spatial neighborhood interpolation block 512 is D1, and a spatial distance between the center pixel point of the current block to be interpolated 511 and a center pixel point of the spatial neighborhood interpolation block 513 is D2. Assuming that the maximum spatial distance between a pixel point in the frame image to be interpolated and the center pixel point of the current block to be interpolated is 100 pixel points, D1 is 20 pixel points, and D2 is 40 pixel points, it can be determined that the second evaluation coefficient of the motion vector corresponding to the spatial neighborhood interpolation block 512 is 0.2, and the second evaluation coefficient of the motion vector corresponding to the spatial neighborhood interpolation block 513 is 0.4. From the second evaluation coefficient, the spatial neighborhood interpolation block 513 is more likely to be the target motion vector than the spatial neighborhood interpolation block 512.

The method for determining a motion vector provided in the above embodiment determines the second evaluation coefficient of the motion vector corresponding to the interpolation block in the spatial neighborhood. When the target motion vector is determined, the motion vector which has a position overlapping relation with the current block to be interpolated is considered, the motion vector of the spatial neighborhood is considered, the target motion vector can be determined from the motion vectors as much as possible, and the determined target motion vector has high accuracy.

In an embodiment, the candidate interpolation block further includes a temporal neighborhood interpolation block, a motion vector corresponding to the temporal neighborhood interpolation block is the second motion vector, and the temporal neighborhood interpolation block and the current block to be interpolated are located in adjacent interpolation frame images; the determining a second evaluation coefficient corresponding to the second motion vector includes: acquiring the interval time of the interpolation frame image where the time neighborhood interpolation block and the current block to be interpolated are located; and obtaining a second evaluation coefficient corresponding to the second motion vector (which may be referred to as a temporal neighborhood motion vector, abbreviated as a temporal neighborhood MV at this time) according to the interval time.

The current block to be interpolated is located in the frame image to be interpolated, and the time neighborhood interpolation block may be located in an adjacent frame image of the frame image to be interpolated (the two may be directly adjacent to each other, or may be separated by a certain number of frame images). In some cases, the temporal neighborhood interpolation block and the current block to be interpolated may be located in the same position in the frame image (e.g., both in the middle of the frame image) or may be located in different positions.

The interval time of the interpolated Frame image may be determined according to a Frame rate (Frame rate) of the video. Where the frame rate is a frequency (rate) at which bitmap images in units of frames appear continuously on the display, for example: 16 frames per second. Frame rate and the number of frames between the Frame image to be interpolated and the adjacent Frame image are obtained, and the interval time can be determined according to the Frame Rate Conversion (FRC) technology.

Further, a mapping relation table may be preset between the interval time and the second evaluation coefficient. After the interval time is determined, the corresponding second evaluation coefficient can be obtained by inquiring the mapping relation table.

As shown in fig. 6, the current block to be interpolated 611 is located in the frame image to be interpolated 621, the temporal neighborhood interpolation block 612 is located in the adjacent interpolation frame image 622, and the temporal neighborhood interpolation block 613 is located in the adjacent interpolation frame image 623. The time distance between the center pixel point of the current block to be interpolated 611 and the center pixel point of the time neighborhood interpolation block 612 is T1, and the time distance between the center pixel point of the current block to be interpolated 611 and the center pixel point of the time neighborhood interpolation block 613 is T2. By querying the mapping relationship through T1 and T2, the second evaluation coefficients of the motion vectors corresponding to the temporal neighborhood interpolation block 612 and the temporal neighborhood interpolation block 613, respectively, can be determined.

In the method for determining a motion vector provided in the foregoing embodiment, the second evaluation coefficient of the motion vector corresponding to the interpolation block in the time neighborhood is determined in the time neighborhood. When the target motion vector is determined, the motion vector which has a position overlapping relation with the current block to be interpolated is considered, the motion vector of a time neighborhood is considered, the target motion vector can be determined from the motion vectors as much as possible, and the determined target motion vector has high accuracy.

In one embodiment, the process of determining the second evaluation coefficient of the second motion vector may be: and determining an evaluation coefficient corresponding to the spatial neighborhood motion vector, determining an evaluation coefficient corresponding to the temporal neighborhood motion vector, and integrating the two evaluation coefficients to obtain a second evaluation coefficient.

The spatial neighborhood motion vector refers to a motion vector corresponding to a spatial neighborhood interpolation block, and the temporal neighborhood motion vector refers to a motion vector corresponding to a temporal neighborhood interpolation block.

The method for determining a motion vector provided in the above embodiment combines the spatial neighborhood and the temporal neighborhood to determine the second evaluation coefficient, and considers various possible motion vectors as much as possible, so as to determine a more accurate target motion vector from the second evaluation coefficient, thereby ensuring the accuracy of the motion vector.

In one embodiment, the determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient includes: normalizing the first evaluation coefficient and the second evaluation coefficient; and determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient after normalization processing.

The normalization process is a process of normalizing the measurement units of the first evaluation coefficient and the second evaluation coefficient.

The target motion vector may be determined according to the magnitude of the normalized first evaluation coefficient and the normalized second evaluation coefficient. For example: and determining a maximum evaluation coefficient from the normalized first evaluation coefficient and the second evaluation coefficient, and determining a reference motion vector corresponding to the maximum evaluation coefficient as a target motion vector.

In this embodiment, the terminal performs normalization processing on the first evaluation coefficient and the second evaluation coefficient respectively to obtain the normalized first evaluation coefficient and the normalized second evaluation coefficient, and the normalized evaluation coefficients can be effectively compared in size, so that a target motion vector can be determined from a reference motion vector according to the size of the evaluation coefficients, and the determination efficiency of the target motion vector is improved.

In one embodiment, the second evaluation coefficient includes a spatial neighborhood evaluation coefficient and a temporal neighborhood evaluation coefficient; the motion vector corresponding to the spatial neighborhood evaluation coefficient is a spatial neighborhood motion vector, and the motion vector corresponding to the temporal neighborhood evaluation coefficient is a temporal neighborhood motion vector; the determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient after the normalization processing includes: respectively determining weighted values corresponding to the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient after normalization processing according to the quantity of the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector; according to the weighted value, performing weighted summation operation on the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the time neighborhood evaluation coefficient after normalization processing; and determining the target motion vector from the reference motion vector according to the result of the weighted sum operation.

Wherein, the determination of the weighted value may be: the larger the number of the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector is, the larger the corresponding weighted value is, and vice versa. The larger the number of motion vectors of a certain type is, the larger the influence of the motion vectors of the current block to be interpolated on the motion vectors of the corresponding type is, and the influence is represented by a weighted value.

And respectively carrying out weighted summation operation on the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector according to the weighted value, and fully considering the influence of the number of the motion vectors so that the finally determined target motion vector has higher accuracy.

In one embodiment, the determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient includes: integrating the first evaluation coefficient and the second evaluation coefficient to accumulate the evaluation coefficients of the same motion vector in the first motion vector and the second motion vector to obtain a target evaluation coefficient; and determining the reference motion vector with the maximum target evaluation coefficient as the target motion vector.

Specific examples are as follows: the first motion vectors comprise MV1/MV2/MV3, corresponding first evaluation coefficients 1/2,1/3,1/4, respectively, the second motion vectors comprise MV3/MV4/MV5, corresponding second evaluation coefficients 1/2,1/3,1/4, respectively. If the first motion vector and the second motion vector have the same motion vector MV3, the two motion vectors are combined to obtain a reference motion vector MV1/MV2/MV3/MV4/MV 5. For the motion vector of MV3, the corresponding first evaluation coefficient 1/4 and second evaluation coefficient 1/2 may be added to obtain 3/4. In summary, the target evaluation coefficients corresponding to the reference motion vectors are 1/2,1/3,3/4,1/3 and 1/4.

Further, in an embodiment, the determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient includes: integrating the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient to accumulate the evaluation coefficients of the same motion vector in the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient to obtain a target evaluation coefficient; and determining the reference motion vector with the maximum target evaluation coefficient as the target motion vector.

Specific examples are as follows: the first motion vector comprises MV1/MV2/MV3, the corresponding first evaluation coefficients are 1/2,1/3 and 1/4 respectively, the spatial neighborhood motion vector comprises MV3/MV4/MV5, the corresponding spatial neighborhood evaluation coefficients are 1/2,1/3 and 1/4 respectively, the temporal neighborhood motion vector comprises MV5/MV6/MV7, and the corresponding temporal neighborhood evaluation coefficients are 1/2,1/3 and 1/4 respectively. The first motion vector and the spatial neighborhood motion vector have the same motion vector MV3, the spatial neighborhood motion vector and the temporal neighborhood motion vector have the same motion vector MV5, and the reference motion vector obtained by combining the same motion vectors is MV1/MV2/MV3/MV4/MV5/MV6/MV 7. For the motion vector of MV3, the corresponding first evaluation coefficient 1/4 and second evaluation coefficient 1/2 may be accumulated to obtain an evaluation coefficient 3/4; for the motion vector of MV5, the corresponding first evaluation coefficient 1/4 and second evaluation coefficient 1/2 may be added to obtain an evaluation coefficient 3/4. In summary, the target evaluation coefficients corresponding to the reference motion vectors are 1/2,1/3,3/4,1/3,3/4,1/3 and 1/4.

In the above embodiment, all motion vectors are accumulated according to probability, the same motion vector is accumulated for evaluation coefficient, superposition operation is performed on the motion vectors which are overlapped in the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector, and the obtained target evaluation coefficient fully integrates various possible influence factors, so that the obtained target evaluation coefficient has higher accuracy. Furthermore, the motion vector with the maximum target evaluation coefficient is finally selected as the target motion vector of the current block to be interpolated, and the determined target motion vector has higher accuracy.

In one embodiment, as shown in fig. 7, a method for determining a motion vector is provided, which is described by taking the method as an example applied to the electronic device in fig. 1, and includes the following steps:

s701, when a first motion vector having a position overlapping relation with a current block to be interpolated exists, acquiring a projection block of the first motion vector in a frame image to be interpolated.

S702, obtaining the coincidence area according to the number of coincidence pixel points of the projection block and the current block to be interpolated.

And S703, determining front and back matching blocks which have a projection relation with the first motion vector in front and back adjacent original frame images of the frame image to be interpolated.

And S704, determining error evaluation information between the front matching block and the rear matching block.

S705, determining a distance adjustment coefficient according to the distance between the front and back adjacent original frame images and the frame image to be interpolated.

And S706, acquiring the accuracy of the motion vectors corresponding to the front and rear adjacent original frame images.

And S707, obtaining the reliability of the first motion vector according to at least one of the error evaluation information, the distance adjustment coefficient and the accuracy.

And S708, obtaining a first evaluation coefficient of the first motion vector according to the overlapping area and the reliability.

S709, obtaining a spatial distance between the spatial neighborhood interpolation block and the current block to be interpolated.

And S710, obtaining an evaluation coefficient of the motion vector corresponding to the spatial neighborhood interpolation block according to the spatial distance, and using the evaluation coefficient as a spatial neighborhood evaluation coefficient.

And S711, acquiring the interval time of the interpolation frame image where the time neighborhood interpolation block and the current block to be interpolated are located.

And S712, obtaining an evaluation coefficient of the motion vector corresponding to the time neighborhood interpolation block according to the interval time, and using the evaluation coefficient as the time neighborhood evaluation coefficient.

S713, integrating the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient to accumulate the evaluation coefficients of the same motion vector in the first motion vector, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient.

S714, normalization processing is carried out on the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient which are subjected to the normalization processing.

And S715, respectively determining weighted values corresponding to the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient after normalization processing according to the quantity of the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector.

And S716, performing weighted summation operation on the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the time neighborhood evaluation coefficient after the normalization processing according to the weighted value.

And S717, determining a target motion vector from the first motion vector, the spatial neighborhood motion vector and the temporal neighborhood motion vector according to the result of the weighted summation operation.

The method for determining the motion vector in the embodiment fully considers the motion vectors having an association relationship with the current block to be interpolated, and can effectively improve the accuracy of the determined optimal motion vector according to the motion vectors.

The present application further provides an application scenario, where the method for determining a motion vector is applied, and the method may be applied to the electronic device shown in fig. 1. Specifically, as shown in fig. 8, the application of the motion vector determination method in the application scenario is as follows:

the electronic device is provided with an MC module and an ME module. The MC module divides a frame image to be interpolated into blocks, and the ME module executes each step in the method for determining the motion vector and interpolates each interpolation block according to the determined motion vector. Further, the ME module performs the steps of:

1. statistics of passing MVs and corresponding records of passing probability coefficients.

When there are a plurality of existing MVs passing through the block to be interpolated currently, the MV is recorded as a passing MV. And setting the passing probability coefficient of the passing MV according to the characteristics of the passing MV through the area of the current block to be interpolated, the reliability of the passing MV and the like.

And recording the overlapping area of the projection block passing through the MV and the current block to be interpolated as the overlapping area S.

The SAD, STAD, texture gradient, position penalty between two corresponding matching blocks of the original frame image across the MV, or the accuracy of the original frame image corresponding to the MV is determined as the confidence level across the MV, denoted as k.

And respectively normalizing the coincidence area S and the reliability k of each passing MV and then performing superposition processing to obtain the passing probability coefficient of each passing MV.

2. Statistics of the spatial-temporal neighborhood MV, and recording of a spatial probability coefficient and a temporal probability coefficient.

If the interpolation block of the obtained MV exists in the space-time neighborhood, recording the MV of the interpolation block as the space-time neighborhood MV, and setting the probability coefficient of the space-time neighborhood MV according to the characteristics of the space, the time distance and the like of the current block to be interpolated and the space-time neighborhood interpolated block.

Specifically, for the spatial neighborhood, the spatial distance between the interpolation block corresponding to the corresponding spatial neighborhood MV and the current block to be interpolated is determined, and the spatial probability coefficient is determined according to the spatial distance. And for the time neighborhood, determining a time-adjacent interpolation block which has obtained the MV at the same position in a frame adjacent to the time line of the current block to be interpolated, determining the time distance between the time-adjacent interpolation block and the current block to be interpolated, and determining a time probability coefficient according to the time distance.

3. The probability coefficients are accumulated.

And accumulating all the MVs according to the probability, accumulating the probability coefficients by using the same MV to obtain a target probability coefficient, and finally selecting the MV with the maximum target probability coefficient as the target MV of the current block to be interpolated.

Specifically, the process of determining the target probability coefficient may be:

first, a unified measurement unit is performed. The penetration probability coefficient is based on the area, the space probability coefficient is based on the pixel number of the position distance, the time probability coefficient is based on the distance frame number, and the probability coefficients of three different dimensions are normalized in a certain mode to enable the measurement units to be the same.

Second, a metric weighting value is set. Setting according to the quantity of the MVs, the time neighborhood MVs and the space neighborhood MVs, wherein the larger the quantity of the MVs is, the larger the corresponding measurement weighted value is, and the smaller the quantity of the MVs is otherwise;

and finally, multiplying the normalized probability coefficient by a measurement weighted value, taking the pass probability coefficient as an example, multiplying the normalized pass probability coefficient by the measurement weighted value to obtain a target pass probability coefficient corresponding to the pass MV.

4. And completing the MV fitting of the current block to be interpolated, and completing the difference value of the current block to be interpolated according to the target MV obtained by fitting.

According to the technical scheme, the interpolation block MV selection strategy based on the maximum probability is adopted, and the MV with the maximum probability of occurrence in the space-time neighborhood MV and the passing MV is selected as the optimal MV of the current block to be interpolated, so that the selection accuracy of the interpolation block MV can be effectively improved, the robustness is high, and the performance of the MEMC algorithm is improved.

It should be understood that, although the various steps in the foregoing flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in the aforementioned flowcharts may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

In one embodiment, as shown in fig. 9, there is provided a motion vector determining apparatus 900, including:

a first motion vector determining module 901, configured to obtain a first motion vector, where the first motion vector and a current block to be interpolated have a position overlapping relationship.

A second motion vector determining module 902, configured to determine, according to a candidate interpolation block having a space-time association relationship with the current block to be interpolated, a second motion vector corresponding to the candidate interpolation block.

A target motion vector determining module 903, configured to determine a target motion vector according to the first motion vector and the second motion vector.

The device for determining the motion vector fully considers the motion vectors which are associated with the current block to be interpolated, and can effectively improve the accuracy of the determined optimal motion vector according to the motion vectors.

In one embodiment, the first motion vector and the second motion vector constitute a reference motion vector; a target motion vector determination module comprising: a first coefficient determining submodule, configured to determine a first evaluation coefficient corresponding to the first motion vector; a second coefficient determining submodule, configured to determine a second evaluation coefficient corresponding to the second motion vector; and the target motion vector determining sub-module is used for determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient.

In one embodiment, when the first motion vector passes through the current block to be interpolated, the first motion vector is determined to have a position overlapping relation with the current block to be interpolated; the current block to be interpolated is positioned in the frame image to be interpolated; a first coefficient determination submodule comprising: the projection block acquisition submodule is used for acquiring a projection block of the first motion vector in the frame image to be interpolated; the coincidence area determining submodule is used for obtaining the coincidence area according to the number of coincidence pixel points of the projection block and the current block to be interpolated; and the first coefficient determining submodule is used for obtaining the first evaluation coefficient according to the overlapping area.

In one embodiment, the first coefficient determination sub-module includes: a reliability determining unit configured to determine a reliability of the first motion vector; and the first coefficient determining unit is used for obtaining the first evaluation coefficient according to the overlapping area and the reliability.

In one embodiment, the credibility determination unit comprises: a matching block determining subunit, configured to determine, in previous and subsequent adjacent original frame images of the frame image to be interpolated, previous and subsequent matching blocks that have a projection relationship with the first motion vector; an error information determining subunit, configured to determine error evaluation information between the front and rear matching blocks; an adjustment coefficient determining subunit, configured to determine a distance adjustment coefficient according to a distance between the front and rear adjacent original frame images and the frame image to be interpolated; the accuracy acquisition subunit is used for acquiring the accuracy of the motion vectors corresponding to the front and rear adjacent original frame images; and the reliability determining subunit is used for obtaining the reliability according to at least one of the error evaluation information, the distance adjusting coefficient and the accuracy.

In one embodiment, the candidate interpolation blocks include a spatial neighborhood interpolation block, a motion vector corresponding to the spatial neighborhood interpolation block is the second motion vector, and the spatial neighborhood interpolation block and the current block to be interpolated are located in a frame image to be interpolated; a second coefficient determination submodule comprising: a spatial distance determining unit, configured to obtain a spatial distance between the spatial neighborhood interpolation block and the current block to be interpolated; and the coefficient determining unit is used for obtaining a second evaluation coefficient corresponding to the second motion vector according to the space distance.

In an embodiment, the candidate interpolation block further includes a temporal neighborhood interpolation block, a motion vector corresponding to the temporal neighborhood interpolation block is the second motion vector, and the temporal neighborhood interpolation block and the current block to be interpolated are located in adjacent interpolation frame images; a second coefficient determination submodule comprising: an interval time determining unit, configured to obtain an interval time between the time neighborhood interpolation block and an interpolation frame image in which the current block to be interpolated is located; and the second coefficient determining unit is used for obtaining a second evaluation coefficient corresponding to the second motion vector according to the interval time.

In one embodiment, the motion vector determination module includes: the normalization submodule is used for performing normalization processing on the first evaluation coefficient and the second evaluation coefficient; and the motion vector determining submodule is used for determining the target motion vector from the reference motion vector according to the first evaluation coefficient and the second evaluation coefficient after normalization processing.

In one embodiment, the second evaluation coefficient includes a spatial neighborhood evaluation coefficient and a temporal neighborhood evaluation coefficient; the motion vector corresponding to the spatial neighborhood evaluation coefficient is a spatial neighborhood motion vector, and the motion vector corresponding to the temporal neighborhood evaluation coefficient is a temporal neighborhood motion vector; a motion vector determination sub-module comprising: a weighted value determining unit, configured to determine weighted values corresponding to the first evaluation coefficient, the spatial neighborhood evaluation coefficient, and the temporal neighborhood evaluation coefficient after normalization processing according to the number of the first motion vector, the spatial neighborhood motion vector, and the temporal neighborhood motion vector; the weighted summation unit is used for carrying out weighted summation operation on the first evaluation coefficient, the spatial neighborhood evaluation coefficient and the temporal neighborhood evaluation coefficient after normalization processing according to the weighted value; a motion vector determination unit for determining the target motion vector from the reference motion vector according to a result of the weighted sum operation.

In one embodiment, the motion vector determination module includes: an evaluation coefficient determining submodule, configured to integrate the first evaluation coefficient and the second evaluation coefficient, so as to accumulate evaluation coefficients of the same motion vector in the first motion vector and the second motion vector to obtain a target evaluation coefficient; and the target motion vector determining unit is used for determining the reference motion vector with the maximum target evaluation coefficient as the target motion vector.

The division of the modules in the motion vector determination apparatus is merely for illustration, and in other embodiments, the motion vector determination apparatus may be divided into different modules as needed to complete all or part of the functions of the motion vector determination apparatus.

For the specific definition of the motion vector determination device, reference may be made to the above definition of the motion vector determination method, which is not described herein again. The respective modules in the motion vector determination apparatus described above may be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the electronic device, or can be stored in a memory in the electronic device in a software form, so that the processor can call and execute operations corresponding to the modules.

The respective modules in the motion vector determination apparatus provided in the embodiments of the present application may be implemented in the form of a computer program. The computer program may be run on a terminal or a server. Program modules constituted by such computer programs may be stored on the memory of the electronic device. Which when executed by a processor, performs the steps of the method described in the embodiments of the present application.

In one embodiment, an electronic device is further provided, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is also provided. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of the method for motion vector determination.

In an embodiment, a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of determining a motion vector is also provided.

Any reference to memory, storage, database, or other medium used herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.