阅读说明：本技术 一种从图像中提取平面的方法 (Method for extracting plane from image ) 是由 朱猛 于 2020-07-24 设计创作，主要内容包括：本发明公开了一种从图像中提取平面的方法,涉及图像处理技术领域,包括以下步骤：获取深度图像并进行像素窗口遍历；遍历中将二维图像坐标转为三维世界坐标,获取三维点集；三维点集中任意取若干点,求解三维平面方程；获取三维点集中所有点与求解的三维平面的距离,确定内点；筛选平面。本发明通过从深度图像和彩色图像中提取三维平面的方法,能够高效的利用深度图所提供的信息,快速准确的查找出图像中的三维平面,同时具有较强的抗噪声抗干扰能力,为三维测绘和增强现实等行业应用提供了良好的底层支持。(The invention discloses a method for extracting a plane from an image, which relates to the technical field of image processing and comprises the following steps: acquiring a depth image and traversing a pixel window; converting two-dimensional image coordinates into three-dimensional world coordinates in traversal to obtain a three-dimensional point set; randomly taking a plurality of points from the three-dimensional point set, and solving a three-dimensional plane equation; obtaining the distances between all points in the three-dimensional point set and the solved three-dimensional plane, and determining interior points; and (4) screening a plane. The method for extracting the three-dimensional plane from the depth image and the color image can efficiently utilize the information provided by the depth image, quickly and accurately find out the three-dimensional plane in the image, has stronger anti-noise and anti-interference capability, and provides good bottom support for the application of industries such as three-dimensional mapping, augmented reality and the like.)

1. A method of extracting a plane from an image, comprising the steps of:

acquiring a depth image and traversing a pixel window;

converting two-dimensional image coordinates into three-dimensional world coordinates in traversal to obtain a three-dimensional point set;

randomly taking a plurality of points from the three-dimensional point set, and solving a three-dimensional plane equation;

obtaining the distances between all points in the three-dimensional point set and the solved three-dimensional plane, and determining interior points;

and (4) screening a plane.

2. The method of claim 1, wherein the pixel window traversal comprises: a window of size 64 x 64 pixels traverses the depth map.

3. The method of claim 1, wherein the converting of the two-dimensional image coordinates into three-dimensional world coordinates comprises:

taking X as a two-dimensional coordinate on an imaging plane, and taking X as a three-dimensional world coordinate corresponding to X; c is the optical center of the camera; f is the focal point of the camera, which is expressed as:

wherein X, Y and Z are the coordinates of the current point in the three-dimensional world coordinate system, x and y are the coordinates of the current point in the two-dimensional image coordinate system, Z is the depth value of the current point, cx and cy are the coordinates of the optical center C in the two-dimensional image coordinate system, and fx and fy are the lengths of the camera focus F in the horizontal and vertical directions, respectively.

4. The method of claim 1, wherein the arbitrary points are arbitrary 3 points.

5. The method of extracting a plane from an image of claim 4, wherein solving the three-dimensional plane equation comprises:

the three-dimensional plane equation is expressed as:

Ax+By+Cz+D＝0

and respectively substituting the coordinates of the taken 3 points into a three-dimensional plane equation, fixing the coefficient D to be 1, and generating a three-element linear equation set:

Ax₀+By₀+Cz₀+1＝0

Ax₁+By₁+Cz₁+1＝0

Ax₂+By₂+Cz₂+1＝0

and solving to obtain the unique three-dimensional plane equation of the 3 points.

6. The method of claim 5, wherein the distance between all points in the three-dimensional point set and the solved three-dimensional plane comprises:

the point-to-face distance is expressed as:

and obtaining the distance between the rest points in the window and the calculated plane, wherein if the distance is smaller than a threshold value, the point is considered to be on the plane and is represented as an inner point, and if not, the point is an outer point.

7. The method of extracting a plane from an image as claimed in claim 6, further comprising the steps of:

and selecting the plane with the largest number of interior points as the optimal three-dimensional plane of the current window.

Technical Field

The invention relates to the technical field of image processing, in particular to a method for extracting a plane from an image.

Background

The image is the basis of human vision, is the objective reflection of natural scenes, and is an important source for human to know the world and human. Images are a kind of similarity, vivid description or portrayal of objective objects, and are the most commonly used information carriers in human social activities. Or image, is a representation of an objective object that contains information about the object being described.

At present, for a three-dimensional plane extraction method, due to the fact that calculation for generating a three-dimensional model is complex, accuracy is low, and the plane searching speed is low, real-time operation cannot be achieved all the time, and a stable and effective solution for the problem does not exist all the time.

Chinese patent application CN 101639355 discloses a "three-dimensional plane extraction method", which uses a known three-dimensional model and line segment characteristics therein, and uses a straight line to calculate a plane equation, so that the found plane is accurate and effective, but the scheme does not overcome the problem that a complex model is difficult to obtain, and meanwhile, the scheme based on a straight line has relatively large computation amount, and cannot perform real-time computation.

Chinese patent application CN 102945551B discloses a three-dimensional point cloud data plane extraction method based on graph theory, which extracts coplanar triangles by a nearest neighbor triangle connection method on the basis of known three-dimensional point clouds, so that the accuracy of the searched plane is extremely high, but the calculation amount of the scheme is extremely large, real-time processing can hardly be realized, and no good way exists for obtaining the three-dimensional point clouds.

Chinese patent application CN 105260737a discloses "a method for automatically extracting physical plane of laser scanning data fused with multi-scale features", in the scheme, plane information is extracted through data scanned by laser, and since laser scanning has very high precision and very high detection frequency, the accuracy of the extracted plane is very high, but this scheme utilizes expensive laser scanning equipment, and cannot be applied to consumer-grade markets.

In summary, how to overcome the deficiencies of the prior art in simple three-dimensional information acquisition and operation performance has become a key problem to be solved urgently in the current three-dimensional plane extraction technology.

An effective solution to the problems in the related art has not been proposed yet.

Disclosure of Invention

The invention provides a method for extracting a plane from an image, which can rapidly and accurately find the three-dimensional plane in the image by efficiently utilizing information provided by a depth map through a method for extracting the three-dimensional plane from the depth image and a color image, has stronger anti-noise and anti-interference capability, provides good bottom support for industry applications such as three-dimensional mapping and augmented reality, and overcomes the technical problems in the prior art.

The technical scheme of the invention is realized as follows:

a method of extracting a plane from an image, comprising the steps of:

step S1, obtaining a depth image and traversing a pixel window;

step S2, converting the two-dimensional image coordinate into a three-dimensional world coordinate in traversal to obtain a three-dimensional point set;

s3, randomly taking a plurality of points from the three-dimensional point set, and solving a three-dimensional plane equation;

step S4, obtaining the distances between all points in the three-dimensional point set and the solved three-dimensional plane, and determining interior points;

and step S5, screening planes.

Further, the pixel window traversal comprises: a window of size 64 x 64 pixels traverses the depth map.

Further, the converting of the two-dimensional image coordinates into three-dimensional world coordinates includes:

taking X as a two-dimensional coordinate on an imaging plane, and taking X as a three-dimensional world coordinate corresponding to X; c is the optical center of the camera; f is the focal point of the camera, which is expressed as:

Z＝z；

wherein X, Y and Z are the coordinates of the current point in the three-dimensional world coordinate system, x and y are the coordinates of the current point in the two-dimensional image coordinate system, Z is the depth value of the current point, cx and cy are the coordinates of the optical center C in the two-dimensional image coordinate system, and fx and fy are the lengths of the camera focus F in the horizontal and vertical directions, respectively.

Furthermore, the arbitrary points are 3 points.

Further, the solving of the three-dimensional plane equation includes:

the three-dimensional plane equation is expressed as:

Ax+By+Cz+D＝0

and respectively substituting the coordinates of the taken 3 points into a three-dimensional plane equation, fixing the coefficient D to be 1, and generating a three-element linear equation set:

Ax₀+By₀+Cz₀+1＝0

Ax₁+By₁+Cz₁+1＝0

Ax₂+By₂+Cz₂+1＝0

and solving to obtain the unique three-dimensional plane equation of the 3 points.

Further, the distances between all the points in the three-dimensional point set and the solved three-dimensional plane include:

the point-to-face distance is expressed as:

and obtaining the distance between the rest points in the window and the calculated plane, wherein if the distance is smaller than a threshold value, the point is considered to be on the plane and is represented as an inner point, and if not, the point is an outer point.

Further, the method comprises the following steps:

step S401, selecting the plane with the largest number of interior points as the optimal three-dimensional plane of the current window.

The invention has the beneficial effects that:

the method for extracting the plane from the image can quickly and accurately find the three-dimensional plane in the image by efficiently utilizing the information provided by the depth map through the method for extracting the three-dimensional plane from the depth image and the color image, has stronger anti-noise and anti-interference capability, and provides good bottom support for the application of industries such as three-dimensional mapping, augmented reality and the like.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a first schematic flow chart of a method for extracting planes from an image according to an embodiment of the present invention;

FIG. 2 is a second flowchart illustrating a method for extracting planes from an image according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a coordinate scene of a method for extracting a plane from an image according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.

According to an embodiment of the present invention, there is provided a method of extracting a plane from an image.

As shown in fig. 1 to 3, a method for extracting a plane from an image according to an embodiment of the present invention includes the steps of:

step S1, obtaining a depth image and traversing a pixel window;

step S2, converting the two-dimensional image coordinate into a three-dimensional world coordinate in traversal to obtain a three-dimensional point set;

s3, randomly taking a plurality of points from the three-dimensional point set, and solving a three-dimensional plane equation;

step S4, obtaining the distances between all points in the three-dimensional point set and the solved three-dimensional plane, and determining interior points;

and step S5, screening planes.

In addition, specifically, in step S1, a depth image is acquired and a pixel window traversal is performed; ways to obtain depth maps include structured light based depth maps, light time of flight based depth maps and multi-ocular vision based depth maps. The obtained depth map is in a two-dimensional matrix form, the point at each position in the matrix is a pixel point of the image, and the numerical value of the pixel point represents the physical distance between an object in the scene and the depth map device.

Further, wherein the pixel window traversal comprises: a window of size 64 x 64 pixels traverses the depth map.

The depth map is traversed using a window size of 64 x 64 pixels. The concrete reality is as follows: starting from the first pixel in the first row of the depth image, taking 64 pixels, jumping to the first pixel in the second row, starting to take 64 pixels, and so on until the line 64; after the completion, starting from the 65 th pixel of the first row, taking 64 pixels, and according to the rule, stopping until the 64 th row; thereafter, using the above rules, the entire image is traversed, at which time the image is divided into windows of 64 x 64 pixels in size.

Specifically, as shown in fig. 3, the converting of the two-dimensional image coordinates into three-dimensional world coordinates includes:

and converting the two-dimensional image coordinate into a three-dimensional world coordinate. According to the small-aperture camera model, x is a two-dimensional coordinate on an imaging plane, namely a coordinate in an image; x is a three-dimensional world coordinate corresponding to X; c is the optical center of the camera; f is the focal point of the camera, which is expressed as:

Z＝z；

wherein X, Y and Z are the coordinates of the current point in the three-dimensional world coordinate system, x and y are the coordinates of the current point in the two-dimensional image coordinate system, Z is the depth value of the current point, cx and cy are the coordinates of the optical center C in the two-dimensional image coordinate system, and fx and fy are the lengths of the camera focus F in the horizontal and vertical directions, respectively. Wherein, the arbitrary points are 3 points.

Wherein said solving three-dimensional plane equations comprises:

the three-dimensional plane equation is expressed as:

Ax+By+Cz+D＝0

and respectively substituting the coordinates of the taken 3 points into a three-dimensional plane equation, fixing the coefficient D to be 1, and generating a three-element linear equation set:

Ax₀+By₀+Cz₀+1＝0

Ax₁+By₁+Cz₁+1＝0

Ax₂+By₂+Cz₂+1＝0

and solving to obtain the unique three-dimensional plane equation of the 3 points.

Wherein, the distance between all points in the three-dimensional point set and the solved three-dimensional plane comprises:

the point-to-face distance is expressed as:

and obtaining the distance between the rest points in the window and the calculated plane, wherein if the distance is smaller than a threshold value, the point is considered to be on the plane and is represented as an inner point, and if not, the point is an outer point.

The method comprises the following steps:

step S401, selecting the plane with the largest number of interior points as the optimal three-dimensional plane of the current window.

In step S5, the size of the window is 64 × 64, so that the larger planes such as the wall surface and the floor surface in the image exist in the plurality of windows at the same time, and in this case, it is necessary to compare the optimal three-dimensional planes calculated for each window and remove the same planes and the smaller planes (the number of inner points is too small).

In summary, with the aid of the technical solution of the present invention, the method for extracting a plane from an image and the method for extracting a three-dimensional plane from a depth image and a color image can efficiently utilize information provided by a depth map to quickly and accurately find out the three-dimensional plane in the image, and meanwhile, the method has strong anti-noise and anti-interference capabilities, and provides good bottom layer support for industry applications such as three-dimensional mapping and augmented reality.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.