阅读说明：本技术 基于不可见投影二维码的室内定位方法 (Indoor positioning method based on invisible projection two-dimensional code ) 是由 韩亮 李谨杰 任章 于 2021-09-29 设计创作，主要内容包括：本发明公开一种基于不可见投影二维码的室内定位方法,包括：获取融合有二维码图像信息的用于展示待定位对象的任务场景的目标视频,该视频由多组图像按照相应顺序合并而成,每组图像中的各个图像按照相应顺序排列且各个图像之间的明暗分布变化规律对应于二维码图像信息,目标视频的帧率大于预定的人眼可识别的图像亮度最大变化频率；在将目标视频投影至室内预定区域的过程中,待定位对象获取每组图像中的各个图像之间的明暗分布变化规律,根据明暗分布变化规律获取对应的二维码图像,根据二维码图像获取自身位置信息。根据本发明,能够解决现有移动机器人室内定位方法的实施成本高和易受电磁干扰的问题,同时能够为室内场景带来丰富的可视化效果。(The invention discloses an indoor positioning method based on invisible projection two-dimensional codes, which comprises the following steps: acquiring a target video fused with two-dimensional code image information and used for displaying a task scene of an object to be positioned, wherein the video is formed by combining a plurality of groups of images according to a corresponding sequence, the images in each group of images are arranged according to the corresponding sequence, the light and shade distribution change rule among the images corresponds to the two-dimensional code image information, and the frame rate of the target video is greater than the preset maximum change frequency of the brightness of the images recognizable to human eyes; in the process of projecting a target video to an indoor preset area, an object to be positioned acquires a light and shade distribution change rule among all images in each group of images, acquires a corresponding two-dimensional code image according to the light and shade distribution change rule, and acquires position information of the object according to the two-dimensional code image. According to the method and the device, the problems that an existing indoor positioning method of the mobile robot is high in implementation cost and prone to electromagnetic interference can be solved, and meanwhile, rich visualization effects can be brought to indoor scenes.)

1. Indoor positioning method based on invisible projection two-dimensional code, which is characterized by comprising the following steps:

acquiring a target video fused with two-dimension code image information and used for displaying a task scene of an object to be positioned, wherein the target video is formed by combining a plurality of groups of images according to a corresponding sequence, the images in each group of images are arranged according to the corresponding sequence, the light and shade distribution change rule among the images in each group of images corresponds to the two-dimension code image information, and the frame rate of the target video is greater than the preset maximum change frequency of the image brightness recognizable to human eyes;

in the process of projecting the target video to an indoor preset area, the object to be positioned acquires a light and shade distribution change rule among all images in each group of images, acquires a corresponding two-dimensional code image according to the light and shade distribution change rule, and acquires position information of the object according to the two-dimensional code image.

2. The indoor positioning method based on the invisible projection two-dimensional code as claimed in claim 1, wherein the acquiring of the target video fused with the two-dimensional code image information and used for displaying the task scene of the object to be positioned comprises:

extracting each frame of image of an original video capable of displaying a task scene of the object to be positioned;

performing a predetermined operation on each extracted frame of image to obtain the plurality of groups of images;

and combining the multiple groups of images according to a preset combining sequence to obtain the target video.

3. The indoor positioning method based on the invisible projected two-dimensional code according to claim 2, wherein the predetermined operation includes:

adjusting the size of a target frame image to be operated according to the preset resolution of the target video;

performing color gamut conversion on the target frame image after the size adjustment;

copying the target frame image subjected to color gamut conversion for a preset number of times to obtain a group of images to be subjected to brightness adjustment;

and correspondingly adjusting the brightness of the corresponding position of each image in the group of images to be subjected to brightness adjustment according to the two-dimensional code image information, and performing color gamut inverse conversion on each image subjected to brightness adjustment to obtain a group of corresponding images for synthesizing the target video.

4. The indoor positioning method based on the invisible projection two-dimensional code according to claim 3, wherein the color gamut conversion of the target frame image after the size adjustment specifically comprises: converting the target frame image after size adjustment from an RGB color gamut to a CIELAB color gamut;

the group of images to be subjected to brightness adjustment comprises two target frame images subjected to color gamut conversion;

the correspondingly adjusting the brightness of the corresponding position of each image in the group of images to be brightness adjusted according to the two-dimensional code image information comprises:

acquiring hidden information of each position of each image according to the two-dimensional code image information;

and adding or subtracting preset light intensity to or from the original pixel at the corresponding position of each image in a Manchester coding mode according to the hidden information of each acquired position of each image.

5. The indoor positioning method based on the invisible projection two-dimensional code according to claim 4, wherein the step of acquiring a light and shade distribution change rule between the images in each group of images by the object to be positioned, acquiring the corresponding two-dimensional code image according to the light and shade distribution change rule, and acquiring the position information of the object according to the two-dimensional code image comprises the steps of:

the high-speed camera module carried by the object to be positioned sequentially acquires each image in each group of images;

the processor carried by the object to be positioned acquires the light and shade distribution change rule among the images according to the images in each group of acquired images,

acquiring a corresponding two-dimensional code image according to the light and shade distribution change rule,

acquiring the pixel coordinates of the corner points of the two-dimensional code and the id of the two-dimensional code according to the two-dimensional code image,

acquiring world coordinates of a two-dimension code corner according to the id of the two-dimension code and pre-acquired two-dimension code map information,

and acquiring the position information of the object to be positioned relative to a two-dimensional code map coordinate system according to the pixel coordinates of the two-dimensional code corner points, the world coordinates of the two-dimensional code corner points and a pre-acquired parameter matrix of the high-speed camera module.

6. The indoor positioning method based on the invisible projection two-dimensional code according to claim 5, wherein the processor obtains pixel coordinates of the two-dimensional code corner point and world coordinates of the two-dimensional code corner point based on a decoding algorithm;

the decoding algorithm comprises:

the current frame image F to be obtained^nowConverting from RGB color gamut to CIELAB color gamut, and extracting corresponding brightness channel image

Image of luminance channelWith the pre-acquired last frame image F^preCorresponding luminance channel imageAligning and displaying the aligned luminance channel imagesAnd luminance channel imageMaking difference to obtain new image F_b；

For new image F_bPerforming image preprocessing to obtain a two-dimensional code image;

detecting the two-dimensional code image to obtain pixel coordinates of the two-dimensional code corner points and id of the two-dimensional code;

and acquiring world coordinates of the corner points of the two-dimensional code according to the id of the two-dimensional code and the map information of the two-dimensional code.

7. According to claim 6The indoor positioning method based on invisible projection two-dimensional code is characterized in that the brightness channel image is used forWith the pre-acquired last frame image F^preCorresponding luminance channel imageThe alignment is realized by adopting an image alignment method based on sampling;

the image alignment method based on sampling specifically comprises the following steps:

image of luminance channelAnd luminance channel imageRespectively taking N rows at equal intervals along the horizontal axis to obtain a pixel row T_i ^now(i ═ 1.. times, N) and pixel column T_i ^pre(i＝1,...,N)；

If the luminance channel imageAnd luminance channel imageThe number of pixels of the longitudinal displacement between is m_yThen the luminance channel image is obtainedAnd luminance channel imageOptimum number of pixels of longitudinal displacement to be adjusted in alignmentThe requirements are as follows:

in the above formula, b is the predetermined maximum pixel shift amount, and h is the luminance channel imageAnd luminance channel imageHeight of (T)_i ^pre(0:h-m_y) And T_i ^pre(-m_yH) respectively representIth pixel column T of_i ^pre0 th to h-m_y-1 pixel and-m_yTo the h-1 th pixel, T_i ^now(m_yH) and T_i ^now(0:h+m_y) Respectively representIth pixel column T of_i ^nowM of_yTo the h-1 th pixel and 0 to the h + m_y-1 pixel.

8. The indoor positioning method based on the invisible projection two-dimensional code according to claim 7, characterized in that the image preprocessing comprises a normalization step, a median filtering step, a thresholding step, a morphological open operation step and a morphological close operation step which are sequentially performed;

the normalization step is used for converting the new image F_bIs limited to a luminance range of 0-255;

the median filtering step is used for removing a new image F_bThe noise of (2);

the thresholding link is used for filtering noiseRemoved new image F_bConverting the image into a binary black-and-white image;

the morphological open operation link is used for removing noise points of the binary black-and-white image;

and the morphology closed operation link is used for connecting the outer boundaries of the binarized black-and-white image after the noise points are removed so as to obtain a two-dimensional code image.

9. The indoor positioning method based on the invisible projection two-dimensional code of claim 8, wherein the detection of the two-dimensional code image to obtain the pixel coordinates of the two-dimensional code corner points and the id of the two-dimensional code is implemented based on a predetermined two-dimensional code detection toolkit.

10. The indoor positioning method based on the invisible projection two-dimensional code according to claim 9, wherein the processor performs pose calculation according to pixel coordinates of the two-dimensional code corner point, world coordinates of the two-dimensional code corner point and a pre-acquired parameter matrix of the high-speed camera module to acquire position information of the object to be positioned relative to a two-dimensional code map coordinate system;

the pose resolving is realized based on OpenCV software.

Technical Field

The invention belongs to the field of indoor positioning, and particularly relates to an indoor positioning method based on invisible projection two-dimensional codes.

Background

With the development of the robot technology and the artificial intelligence technology, the autonomous aerial robot industry develops rapidly, and plays an increasingly important role in the fields of logistics transportation, environment monitoring, farmland plant protection and the like. Because the cost is higher in outdoor experiments, the environment change is complex, the requirement on flight airspace is higher, and more researchers select to carry out flight tests indoors. However, the flight test developed indoors is often only a simple abstraction of a real task scene, and interaction with an external environment cannot be achieved (for example, a path planning task cannot acquire a road condition in real time). Meanwhile, the display effect of the simple scene is very limited.

In order to improve the interaction degree with the external environment and also for the display effect of the elevator robot platform, work gradually begins to combine Augmented Reality (AR) technology with mobile robot research. Reina et al propose a concept of a "virtual sensor", and use a virtual scene to output sensing information to a robot, thereby realizing interaction with an external environment. And Omidshafiei et al propose a new CPS robot research architecture, project task scenes on the ground, use OptiTrack motion capture device to carry out indoor positioning, realize the interaction of robot and environment and the visual presentation of information. Because there is no GNSS signal indoors, the above-mentioned platform combining AR and robot generally adopts a motion capture system or UWB device commonly used in indoor robot research to provide positioning information, the former is expensive, and the latter is susceptible to indoor electromagnetic interference. Meanwhile, the design of separation of positioning and display brings new problems of coordinate system alignment and the like, so that the operation is complicated, and the robot positioning and display device is not suitable for popularization in robot research institutions and education institutions.

Therefore, in order to reduce hardware cost and use threshold while maintaining visual effect, the applicant attempts to integrate positioning and display processes, so that the robot can obtain positioning information from the projected picture itself.

Disclosure of Invention

In view of this, the present invention provides an indoor positioning method based on invisible projected two-dimensional codes, which includes the following steps:

acquiring a target video fused with two-dimension code image information and used for displaying a task scene of an object to be positioned, wherein the target video is formed by combining a plurality of groups of images according to a corresponding sequence, the images in each group of images are arranged according to the corresponding sequence, the light and shade distribution change rule among the images in each group of images corresponds to the two-dimension code image information, and the frame rate of the target video is greater than the preset maximum change frequency of the image brightness recognizable to human eyes;

in the process of projecting the target video to an indoor preset area, the object to be positioned acquires a light and shade distribution change rule among images in each group of images, acquires a corresponding two-dimensional code image according to the light and shade distribution change rule, and acquires position information of the object to be positioned according to the two-dimensional code image;

the object to be positioned is a mobile robot.

Preferably, the acquiring of the target video fused with the two-dimensional code image information and used for displaying the task scene of the object to be positioned includes:

extracting each frame of image of an original video capable of displaying a task scene of the object to be positioned;

performing a predetermined operation on each extracted frame of image to obtain the plurality of groups of images;

and combining the multiple groups of images according to a preset combining sequence to obtain the target video, wherein the target video and the original video have the same duration.

Preferably, the predetermined operation includes:

adjusting the size of a target frame image to be operated according to the preset resolution of the target video;

performing color gamut conversion on the target frame image after the size adjustment;

copying the target frame image subjected to color gamut conversion for a preset number of times to obtain a group of images to be subjected to brightness adjustment;

and correspondingly adjusting the brightness of the corresponding position of each image in the group of images to be subjected to brightness adjustment according to the two-dimensional code image information, and performing color gamut inverse conversion on each image subjected to brightness adjustment to obtain a group of corresponding images for synthesizing the target video.

Preferably, the performing color gamut conversion on the resized target frame image specifically includes: converting the target frame image after size adjustment from an RGB color gamut to a CIELAB color gamut;

the group of images to be subjected to brightness adjustment comprises two target frame images subjected to color gamut conversion;

the correspondingly adjusting the brightness of the corresponding position of each image in the group of images to be brightness adjusted according to the two-dimensional code image information comprises:

acquiring hidden information of each position of each image according to the two-dimensional code image information;

and adding or subtracting preset light intensity to or from the original pixel at the corresponding position of each image in a Manchester coding mode according to the hidden information of each acquired position of each image.

Preferably, the acquiring, by the object to be positioned, a light and shade distribution change rule between images in each group of images, acquiring a corresponding two-dimensional code image according to the light and shade distribution change rule, and acquiring position information of the object to be positioned according to the two-dimensional code image includes:

the high-speed camera module carried by the object to be positioned sequentially acquires each image in each group of images;

the processor carried by the object to be positioned acquires the light and shade distribution change rule among the images according to the images in each group of acquired images,

acquiring a corresponding two-dimensional code image according to the light and shade distribution change rule,

acquiring the pixel coordinates of the corner points of the two-dimensional code and the id of the two-dimensional code according to the two-dimensional code image,

acquiring world coordinates of a two-dimension code corner according to the id of the two-dimension code and pre-acquired two-dimension code map information,

and acquiring the position information of the object to be positioned relative to a two-dimensional code map coordinate system according to the pixel coordinates of the two-dimensional code corner points, the world coordinates of the two-dimensional code corner points and a pre-acquired parameter matrix of the high-speed camera module.

Preferably, the processor acquires pixel coordinates of the two-dimensional code corner point and world coordinates of the two-dimensional code corner point based on a decoding algorithm;

the decoding algorithm comprises:

the current frame image F to be obtained^nowConverting from RGB color gamut to CIELAB color gamut, and extracting corresponding brightness channel image

Image of luminance channelWith the pre-acquired last frame image F^preCorresponding luminance channel imageAligning and displaying the aligned luminance channel imagesAnd luminance channel imageMaking difference to obtain new image F_b；

For new image F_bPerforming image preprocessing to obtain a two-dimensional code image;

detecting the two-dimensional code image to obtain pixel coordinates of the two-dimensional code corner points and id of the two-dimensional code;

and acquiring world coordinates of the corner points of the two-dimensional code according to the id of the two-dimensional code and the map information of the two-dimensional code.

Preferably, the luminance channel image isWith the pre-acquired last frame image F^preCorresponding luminance channel imageThe alignment is realized by adopting an image alignment method based on sampling;

the image alignment method based on sampling specifically comprises the following steps:

image of luminance channelAnd luminance channel imageRespectively taking N rows at equal intervals along the horizontal axis to obtain a pixel row T_i ^now(i ═ 1.. times, N) and pixel column T_i ^pre(i＝1,...,N)；

If the luminance channel imageAnd luminance channel imageThe number of pixels of the longitudinal displacement between is m_yThen the luminance channel image is obtainedAnd luminance channel imageOptimum number of pixels of longitudinal displacement to be adjusted in alignmentThe requirements are as follows:

in the above formula, b is the predetermined maximum pixel shift amount, and h is the luminance channel imageAnd luminance channel imageHeight of (T)_i ^pre(0:h-m_y) And T_i ^pre(-m_yH) divide intoOther representationIth pixel column T of_i ^pre0 th to h-my-1 th pixels and-m_yTo the h-1 th pixel, T_i ^now(m_yH) and T_i ^now(0:h+m_y) Respectively representIth pixel column T of_i ^nowM of_yTo the h-1 th pixel and 0 to the h + m_y-1 pixel.

Preferably, the image preprocessing comprises a normalization link, a median filtering link, a thresholding link, a morphology open operation link and a morphology close operation link which are sequentially performed;

the normalization step is used for converting the new image F_bIs limited to a luminance range of 0-255;

the median filtering step is used for removing a new image F_bThe noise of (2);

the thresholding link is used for removing noise points to form a new image F_bConverting the image into a binary black-and-white image;

the morphological open operation link is used for removing noise points of the binary black-and-white image;

and the morphology closed operation link is used for connecting the outer boundaries of the binarized black-and-white image after the noise points are removed so as to obtain a two-dimensional code image.

Preferably, the detecting the two-dimensional code image to obtain the pixel coordinates of the two-dimensional code corner points and the id of the two-dimensional code is implemented based on a predetermined two-dimensional code detection toolkit.

Preferably, the processor performs pose calculation according to the pixel coordinates of the two-dimensional code corner points, the world coordinates of the two-dimensional code corner points and a pre-acquired parameter matrix of the high-speed camera module to acquire position information of the object to be positioned relative to a two-dimensional code map coordinate system;

the pose resolving is realized based on OpenCV software.

The invention has the beneficial effects that:

the indoor positioning method based on the invisible projection two-dimensional code comprises the steps of firstly obtaining a target video fused with two-dimensional code image information and used for displaying a task scene of the mobile robot, and then projecting the target video to an indoor preset area. In the process of projecting the target video to an indoor preset area, the mobile robot to be positioned acquires a light and shade distribution change rule among all images in each group of images, acquires a corresponding two-dimensional code image according to the light and shade distribution change rule, and acquires position information of the mobile robot according to the two-dimensional code image. The indoor positioning method based on the invisible projection two-dimensional code not only can realize the indoor positioning of the mobile robot with high precision and low cost, but also can vividly show the task scene of the mobile robot through the projected picture, so that the indoor positioning method is suitable for unmanned aerial vehicle teaching and showing in places such as museums, schools and the like.

Additional features and advantages of the invention will be set forth in the detailed description which follows.

Drawings

The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.

Fig. 1 shows a flowchart of an implementation of an indoor positioning method based on invisible projected two-dimensional codes according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating an implementation system of an indoor positioning method based on invisible projected two-dimensional codes according to an embodiment of the present invention;

FIG. 3 shows a diagram of an encoding principle according to an embodiment of the invention;

fig. 4 is a diagram illustrating a positioning result of an indoor positioning method based on an invisible projected two-dimensional code according to an embodiment of the present invention.

Detailed Description

Preferred embodiments of the present invention will be described in more detail below. While the following describes preferred embodiments of the present invention, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Example (b): fig. 1 shows a flowchart of an implementation of an indoor positioning method based on an invisible projected two-dimensional code according to an embodiment of the present invention. With reference to fig. 1, the invisible projection two-dimensional code-based positioning method according to the embodiment of the present invention is described in detail by taking indoor positioning of an unmanned aerial vehicle as an example, and the invisible projection two-dimensional code-based indoor positioning method according to the embodiment of the present invention includes the following steps:

acquiring a target video fused with two-dimension code image information and used for displaying an unmanned aerial vehicle task scene, wherein the target video is formed by combining a plurality of groups of images according to a corresponding sequence, the images in each group of images are arranged according to the corresponding sequence, the light and shade distribution change rule among the images in each group of images corresponds to the two-dimension code image information, and the frame rate of the target video is greater than the preset maximum change frequency of the brightness of the images recognizable by human eyes;

in the process of projecting the target video to an indoor preset ground area, the unmanned aerial vehicle to be positioned acquires a light and shade distribution change rule among all images in each group of images, acquires a corresponding two-dimensional code image according to the light and shade distribution change rule, and acquires position information of the unmanned aerial vehicle according to the two-dimensional code image.

Further, in the embodiment of the present invention, the acquiring a target video fused with two-dimensional code image information and used for displaying an unmanned aerial vehicle task scene includes:

extracting each frame of image of an original video capable of displaying an unmanned aerial vehicle task scene;

performing a predetermined operation on each extracted frame of image to obtain the plurality of groups of images;

and combining the multiple groups of images according to a preset combining sequence to obtain the target video, wherein the target video and the original video have the same duration.

Still further, in an embodiment of the present invention, the predetermined operation includes:

adjusting the size of a target frame image to be operated according to the preset resolution of the target video;

performing color gamut conversion on the target frame image after the size adjustment;

copying the target frame image subjected to color gamut conversion for a preset number of times to obtain a group of images to be subjected to brightness adjustment;

and correspondingly adjusting the brightness of the corresponding position of each image in the group of images to be subjected to brightness adjustment according to the two-dimensional code image information, and performing color gamut inverse conversion on each image subjected to brightness adjustment to obtain a group of corresponding images for synthesizing the target video.

Still further, in the embodiment of the present invention, the performing color gamut conversion on the resized target frame image specifically includes: converting the target frame image after size adjustment from an RGB color gamut to a CIELAB color gamut;

the group of images to be subjected to brightness adjustment comprises two target frame images subjected to color gamut conversion;

the correspondingly adjusting the brightness of the corresponding position of each image in the group of images to be brightness adjusted according to the two-dimensional code image information comprises:

acquiring hidden information of each position of each image according to the two-dimensional code image information;

and adding or subtracting preset light intensity to or from the original pixel at the corresponding position of each image in a Manchester coding mode according to the hidden information of each acquired position of each image.

Furthermore, in the embodiment of the present invention, the acquiring, by the unmanned aerial vehicle to be positioned, a change rule of light and shade distribution between images in each group of images, acquiring a corresponding two-dimensional code image according to the change rule of light and shade distribution, and acquiring position information of the unmanned aerial vehicle according to the two-dimensional code image includes:

the airborne camera of the unmanned aerial vehicle sequentially acquires each image in each group of images;

the onboard computer of the unmanned aerial vehicle acquires the light and shade distribution change rule among the images according to the images in each group of acquired images,

acquiring a corresponding two-dimensional code image according to the light and shade distribution change rule,

acquiring the pixel coordinates of the corner points of the two-dimensional code and the id of the two-dimensional code according to the two-dimensional code image,

acquiring world coordinates of a two-dimension code corner according to the id of the two-dimension code and pre-acquired two-dimension code map information,

and acquiring the position information of the unmanned aerial vehicle relative to a two-dimensional code map coordinate system according to the pixel coordinates of the two-dimensional code angular points, the world coordinates of the two-dimensional code angular points and a pre-acquired parameter matrix of the airborne camera.

Still further, in the embodiment of the present invention, the on-board computer obtains the pixel coordinates of the two-dimensional code corner point and the world coordinates of the two-dimensional code corner point based on a decoding algorithm;

the decoding algorithm comprises:

the current frame image F to be obtained^nowConverting from RGB color gamut to CIELAB color gamut, and extracting corresponding brightness channel image

Image of luminance channelWith the pre-acquired last frame image F^preCorresponding luminance channel imageAligning and displaying the aligned luminance channel imagesAnd luminance channel imageMaking difference to obtain new image F_b；

For new image F_bPerforming image preprocessing to obtain a two-dimensional code image;

detecting the two-dimensional code image to obtain pixel coordinates of the two-dimensional code corner points and id of the two-dimensional code;

and acquiring world coordinates of the corner points of the two-dimensional code according to the id of the two-dimensional code and the map information of the two-dimensional code.

Still further, in the embodiments of the present invention, the luminance channel image isWith the pre-acquired last frame image F^preCorresponding luminance channel imageThe alignment is realized by adopting an image alignment method based on sampling;

the image alignment method based on sampling specifically comprises the following steps:

image of luminance channelAnd luminance channel imageRespectively taking N rows at equal intervals along the horizontal axis to obtain a pixel row T_i ^now(i ═ 1.. times, N) and pixel column T_i ^pre(i＝1,...,N)；

If the luminance channel imageAnd luminance channel imageThe number of pixels of the longitudinal displacement between is m_yThen the luminance channel image is obtainedAnd luminance channel imageOptimum number of pixels of longitudinal displacement to be adjusted in alignmentThe requirements are as follows:

in the above formula, b is the predetermined maximum pixel shift amount, and h is the luminance channel imageAnd luminance channel imageHeight of (T)_i ^pre(0:h-m_y) And T_i ^pre(-m_yH) respectively representIth pixel column T of_i ^pre0 th to h-m_y-1 pixel and-m_yTo the h-1 th pixel, T_i ^now(m_yH) and T_i ^now(0:h+m_y) Respectively representIth pixel column T of_i ^nowM of_yTo the h-1 th pixel and 0 to the h + m_y-1 pixel.

The number of pixels for the optimal lateral displacement is similar to the calculation method for the longitudinal displacement.

Still further, in the embodiment of the present invention, the image preprocessing includes a normalization step, a median filtering step, a thresholding step, a morphological open operation step, and a morphological close operation step, which are performed in sequence;

the normalization step is used for converting the new image F_bIs limited to a luminance range of 0-255;

the median filtering step is used for removing a new image F_bThe noise of (2);

the thresholding link is used for removing noise points to form a new image F_bConverting the image into a binary black-and-white image;

the morphological open operation link is used for removing noise points of the binary black-and-white image;

and the morphology closed operation link is used for connecting the outer boundaries of the binarized black-and-white image after the noise points are removed so as to obtain a two-dimensional code image.

Still further, in the embodiment of the present invention, the detecting the two-dimensional code image to obtain the pixel coordinates of the two-dimensional code corner point and the id of the two-dimensional code is implemented based on a predetermined two-dimensional code detection toolkit.

Furthermore, in the embodiment of the invention, the on-board computer performs pose calculation according to the pixel coordinates of the two-dimension code corner points, the world coordinates of the two-dimension code corner points and a pre-acquired on-board camera parameter matrix to acquire the position information of the unmanned aerial vehicle relative to a two-dimension code map coordinate system;

the pose resolving is realized based on OpenCV software.

Specifically, in the embodiment of the present invention, for the step of color gamut conversion, although it is best to convert the image from RGB to CIELAB color gamut according to the literature material description, the brightness adjustment by conversion to other color gamuts can achieve similar effects. For the frame rate of the input and output video and the frame rate of the airborne camera, adjustment is required according to the original video and hardware conditions. If the projector supports a refresh rate of 120fps, the output video frame rate is 120 fps. For image pre-processing, there may be different processing steps and sequences to achieve the same effect. Although experiments show that the attitude information obtained by the two-dimensional code has too large error, the error can be reduced and the attitude information can be reused perhaps by operations such as filtering and the like.

The indoor positioning method based on the invisible projection two-dimensional code of the embodiment of the invention fuses the two-dimensional code pattern into the original video content through the high-speed change of the brightness to generate a new video. When a projector projects a new video with rapidly changing brightness to the ground, due to the persistence of vision effect, human eyes can only see low-frequency original video content, and a high-frequency camera on the unmanned aerial vehicle can capture the two-dimensional code mark in an interframe difference mode (difference is made between two adjacent frames), so that the self pose is estimated. The invisible projection two-dimensional code-based indoor positioning method can realize indoor positioning of the unmanned aerial vehicle with low cost and high precision (within 5 cm), and can vividly show the task scene of the unmanned aerial vehicle through the projected picture, so that the invisible projection two-dimensional code-based indoor positioning method is suitable for unmanned aerial vehicle teaching and showing in places such as museums, schools and the like.

Fig. 2 is a schematic diagram illustrating an implementation system of an indoor positioning method based on an invisible projected two-dimensional code according to an embodiment of the present invention. Referring to fig. 2, an implementation system of the invisible projection two-dimensional code-based indoor positioning method according to the embodiment of the present invention is divided into a sending end and a receiving end. At a sending end, a Personal Computer (PC) firstly embeds two-dimensional code image information into a video through a coding algorithm, and finally projects the two-dimensional code image information to the ground through a projector. The projected picture comprises an original picture and a marker flashing at high frequency, due to the persistence of vision effect of human eyes, an observer can only see the original picture, and a high-speed camera on the unmanned aerial vehicle can shoot the information of the marker of the two-dimensional code, so that the information is processed at a receiving end. At a receiving end, the airborne computer firstly obtains 2-dimensional plane coordinates and 3-dimensional world coordinates of the two-dimensional code corner points through a decoding algorithm, then performs attitude estimation by combining camera parameters obtained by camera calibration, and finally obtains position information for positioning the unmanned aerial vehicle. The camera herein refers to a camera of an onboard camera.

The working principle of the implementation system of the invisible projection two-dimensional code-based indoor positioning method of the embodiment of the invention is described in more detail as follows:

a sending end:

the transmitting end mainly relates to an encoding algorithm. The encoding refers to a process of hiding two-dimensional code image information in a high frame rate display video in a form of light intensity change by using the flicker mixing characteristic of a human visual system. The flicker mixing characteristic means that the human eye receives only an average brightness for high speed light intensity variations with frequencies above 40-50 Hz. Most of the existing laser projectors have a vertical refresh rate of more than 48Hz and up to 120Hz, and a hardware basis is provided for a visual hiding algorithm.

Inputting: original video V_iThe two-dimensional code image M, the light intensity variation range delta L and the resolution w multiplied by h of the output video.

Step 1: from the original video V with the frame rate of 30fps_iOne frame is obtained, resized to w × h and matched to the projector.

Step 2: and color gamut conversion, which is used for converting the image from an RGB color gamut to a CIELAB color gamut to perform brightness adjustment.

Human eyes have different sensitivity degrees to brightness changes of different colors, so that brightness increase and decrease of different colors with the same visual perception should be ensured as much as possible during coding. Research shows that compared with color spaces such as RGB, HSL and YUV, the CIELAB color space is adopted to adjust the brightness to be most consistent with the visual characteristics of human eyes. CIELAB is a space with uniform appearance and is mainly divided into three channels of L, a and b, wherein L is a brightness channel, and a and b represent colors. The brightness adjustment performed in this color space can be distributed evenly over the RGB channels, so that the human eye perceives different colors as producing the same brightness adjustment. Therefore, the encoding algorithm first converts the RGB image into the CIELAB color space, then performs brightness adjustment in the L channel, and finally converts the RGB color space to the RGB color space for projection display.

And step 3: the current frame is copied to obtain a group of identical images.

And 4, step 4: and increasing and decreasing the brightness of the group of images according to the pixel information of the corresponding position of the two-dimensional code image M.

The principle of encoding is as in figure 3. And according to the hidden information of each image position, adding or subtracting the light intensity delta L to the original pixel at the corresponding position for information modulation. The hidden information is divided into 0 and 1, and the coding mode is similar, but the phase is different. For bit 0, the encoding scheme is LOW-HIGH, and bit 1 is HIGH-LOW. This encoding method is called manchester encoding, and can reduce the visibility of the two-dimensional code by ensuring that the image always flickers at high speed. The light intensity Δ L added or subtracted in the original image is measured according to the brightness of the projector and the ambient brightness, and should be reduced as much as possible to reduce the visibility on the premise that the two-dimensional code can be decoded.

And 5: a color gamut conversion, converting the set of images from a CIELAB color gamut to an RGB color gamut.

Step 6: the processed images are combined to produce a new 60fps section of video.

Although the embodiment of the present invention is exemplified by the setting of 30fps input and 60fps output, the algorithm is applicable to any case that the output frame rate is an integral multiple of the input frame rate.

Receiving end:

the receiving end mainly comprises two links of decoding algorithm and pose resolving. The decoding algorithm is responsible for identifying the two-dimensional code from the video shot by the onboard camera (120fps), acquiring the pixel coordinate and the two-dimensional code id of the corner point of the two-dimensional code, and obtaining the world coordinate by combining the map information of the two-dimensional code. And the pose calculation step obtains the position and the pose of the unmanned aerial vehicle relative to a map coordinate system according to the pixel coordinate, the world coordinate and the camera parameter. Because the attitude error is large, only the position information is obtained as a positioning result.

Decoding algorithm part

Inputting: one frame RGB image F acquired from a camera^now(ii) a Luminance channel image of previous frameTwo-dimensional code map C_map。

Step 1: image F^nowConvert from RGB color gamut to CIELAB color gamut and extract luminance channel

Step 2: and aligning the current image with the previous frame image to eliminate the jitter caused by the displacement of the unmanned aerial vehicle.

Because the frame rate of the camera shooting is very high (120fps), and the change frequency of the display content is far lower than the refresh frequency of the video, theoretically, the picture containing the two-dimensional code can be obtained only by making the difference between the brightness channels of two adjacent frames with brightness change. However, due to the motion of the drone, the degraded image has bright edges at the boundaries of the picture texture due to the displacement. The edges may block the two-dimensional code to block the waist, which seriously affects the detection effect. Therefore, it is necessary to shift the two frames of pictures up, down, left, and right to eliminate the edges as much as possible. Since the distance of translation is small, the effect on the positioning accuracy may be disregarded.

The algorithm adopts a sampling-based image alignment method by integrating the calculation speed and the alignment accuracy. For images of size w × h, if the images are to be aligned in the vertical direction, N columns are taken at equal intervals along the horizontal axis in the previous frame and the previous frame, respectively, to obtain a pixel column T_i ^pre(i ═ 1.., N) and T_i ^now(i ═ 1.., N). Assume that the number of pixels longitudinally displaced is m_yWhen alignedThe requirements are as follows:

in the above formula, b is the predetermined maximum pixel shift amount, and h is the luminance channel imageAnd luminance channel imageHeight of (T)_i ^pre(0:h-m_y) And T_i ^pre(-m_yH) respectively representIth pixel column T of_i ^pre0 th to h-m_y-1 pixel and-m_yTo the h-1 th pixel, T_i ^now(m_yH) and T_i ^now(0:h+m_y) Respectively representIth pixel column ofM of_yTo the h-1 th pixel and 0 to the h + m_y-1 pixel. Alignment along the transverse axis is similar. Considering the speed and the estimation accuracy together, N may be selected to be 3.

And step 3: the aligned images are subtracted to obtain a new image F_b。

And 4, step 4: saving the current luminance mapIs composed ofFor use in resolving the next frame.

And 5: and image preprocessing, including normalization, median filtering, thresholding, morphology opening operation and morphology closing operation.

Normalization is used for limiting the brightness range to 0-255, noise can be removed through median filtering, thresholding is conducted to change the image into a binary black-and-white image, small noise is continuously removed through morphological open operation, and the outer boundaries of the two-dimensional codes are connected through morphological closed operation to be detected.

Of these steps, it is most critical to select a threshold value for the thresholding operation. Only a reasonable threshold value can the two-dimension code shape be extracted. Edges caused by the movement of the body often exist in pairs of black and white, and black and white pixels of the two-dimensional code can be regarded as almost the same area, and the median of the whole image is selected as a threshold value.

Step 6: and detecting the two-dimensional code, and acquiring the pixel coordinate and the id of the corner point of the two-dimensional code. Many existing open source kits may be used here, such as aprilat 3, ArUco, OpenCV, etc.

And 7: according to the id of the two-dimensional code, combining the two-dimensional code map C_mapAnd calculating the world coordinates of the corner points of the two-dimensional code.

Pose resolving portion

And 8: and resolving the pose according to the pixel coordinate, the world coordinate and the camera matrix of the two-dimensional code corner point. The solving process belongs to the PnP problem solving process, and an OpenCV (open computational fluid dynamics) self-contained method such as IPPE (IPPE), EPNP (EPNP) and the like can be adopted.

Therefore, the onboard computer at the receiving end obtains the position information of the onboard computer relative to the projection two-dimensional code coordinate system. The precision can reach within 5cm through experimental verification.

The beneficial effects of the invisible projection two-dimensional code-based indoor positioning method of the embodiment of the invention are explained in detail as follows:

as shown in table 1, if compared to other indoor positioning approaches: compared with a Motion Capture System (Motion Capture System), the method has the advantages that the cost of Motion Capture is high, the cost of the method is low, only the cost of a projector is needed, and meanwhile, the positioning precision (within 5 cm) of the method can meet the requirements of most indoor unmanned aerial vehicle flight tasks. Compared with Ultra-wideband (UWB), the method of the embodiment of the invention has similar precision, but the method of the embodiment of the invention has more accurate high positioning, provides the possibility of scene visualization and has better display effect; meanwhile, the method of the embodiment of the invention does not depend on electromagnetic signals and has strong anti-electromagnetic interference capability. The low cost and the higher precision are determined by the positioning technology based on the two-dimension code, and the scene visualization is realized by a projector and the two-dimension code hiding technology together.

TABLE 1 comparison of the effects of the common positioning modes

If compare with traditional two-dimensional code locate mode: the traditional two-dimensional code positioning mode is to print all two-dimensional codes and attach the two-dimensional codes to the ground for positioning, and the arrangement and maintenance are complicated. Meanwhile, the black and white blocks presented by the two-dimensional code greatly influence the visual perception of audiences, and scene visualization cannot be performed. The method of the embodiment of the invention projects the two-dimensional code by the projector, has simple scene arrangement and no need of maintenance, and only needs to open the projector when in use. Meanwhile, the two-dimensional code is hidden in the rapidly flickering video picture through the persistence of vision effect, the impression of the audience on the projection picture is not influenced, and the visualization effect of the flight scene of the indoor unmanned aerial vehicle is greatly improved.

If contrast with the indoor unmanned aerial vehicle platform of augmented reality: these platforms typically employ a motion capture system + projector, UWB + projector solution. Compared with the schemes, the method of the embodiment of the invention only needs the projector as the external equipment, so that the cost is lower; in addition, the coordinate system adopted by the method of the embodiment of the invention is a projection picture coordinate system, so that the process of aligning the coordinate system once is omitted, and the operation is simpler.

The positioning result of the invisible projection two-dimensional code-based indoor positioning method of the embodiment of the invention is shown in fig. 4, and the statistical result is shown in table 2. The positioning accuracy of the method provided by the embodiment of the invention is similar to that of UWB, the error is 2-4 cm, and the positioning accuracy of the X axis and the Y axis is higher than that of the Z axis.

Index (I)	X axis	Y-axis	Z axis
				mAP (Rice)	0.0359	0.0201	0.0439
Variance (m)2)	0.0025	0.0007	0.0036

TABLE 2 statistical results of the method of the present example

According to the indoor positioning method based on the invisible projection two-dimensional code, only the minimum hardware expenditure (one projector) is needed to reduce the hardware cost, meanwhile, the position positioning precision required by an indoor flight task can be provided, and the watching experience of audiences is not influenced; introducing a Screen-Camera Communication (Screen-Camera Communication) technology in the field of computer vision, hiding two-dimensional code information which is not expected to be seen by audiences in a high-frequency flickering picture by utilizing the visual characteristics of human eyes, and completing a positioning task while not influencing the visual effect; the positioning precision of 2-4 cm can be realized under the condition of not adding filtering, and the position positioning requirements of most indoor robot tasks are met.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.