阅读说明：本技术 基于视觉感知的毫米波/太赫兹通信方法、装置和系统 (Millimeter wave/terahertz communication method, device and system based on visual perception ) 是由 高飞飞 张腾宇 张�浩 王鸣锦 于 2021-06-23 设计创作，主要内容包括：本发明提供一种基于视觉感知的毫米波/太赫兹通信方法、装置和系统,其中通信方法包括：利用相机实时采集图像信息以及移动端的深度信息；利用所述智能反射面根据所述图像信息识别出移动端在图像中的位置；基于所述移动端的深度信息,将移动端在图像中的位置转化为移动端的三维坐标；基于所述移动端的三维坐标所计算出的移动端所在实时位置的俯仰角和方位角,控制智能反射面的天线将信号波束指向移动端方向。本发明可以快速识别移动端并确定目标位置,并控制智能反射面的相控阵天线产生对应方向的俯仰角和方位角,可以使波束更快指向对应目标,提高识别移动端位置的能力,降低时延,实现实时波束跟踪。(The invention provides a millimeter wave/terahertz communication method, device and system based on visual perception, wherein the communication method comprises the following steps: acquiring image information and depth information of a mobile terminal in real time by using a camera; identifying the position of the mobile terminal in the image according to the image information by using the intelligent reflecting surface; converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal; and controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the mobile terminal based on the pitch angle and the azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal. The invention can quickly identify the moving end and determine the position of the target, and control the phased array antenna of the intelligent reflecting surface to generate the pitch angle and the azimuth angle in the corresponding direction, so that the beam can be more quickly pointed to the corresponding target, the capacity of identifying the position of the moving end is improved, the time delay is reduced, and the real-time beam tracking is realized.)

1. A millimeter wave/terahertz communication method based on visual perception is characterized by comprising the following steps:

acquiring image information and depth information of a mobile terminal in real time by using a camera;

identifying the position of the mobile terminal in the image according to the image information by using the intelligent reflecting surface;

converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;

and controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the mobile terminal based on the pitch angle and the azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal.

2. The millimeter wave/terahertz communication method based on visual perception according to claim 1, wherein the camera origin of coordinates is at a geometric center of a camera surface, and the origin of coordinates of the intelligent reflecting surface is approximately the same as the camera origin of coordinates;

the coordinates of the mobile terminal in the camera coordinate system are as follows:

(X_OQ,Y_OQ,Z_OQ)

the coordinate of the mobile terminal in the intelligent reflecting surface coordinate system is as follows:

the conversion formula of the coordinates of the mobile terminal in the camera coordinate system and the intelligent reflecting surface coordinate system is as follows:

the pitch angle θ of the moving end is obtained by the following formula:

azimuth angle of the moving endThe following equation is obtained:

the method further comprises the following steps: calculating the phase angle of the reflected wave beam corresponding to each antenna of the intelligent reflecting surface according to the following formula

In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.

3. The millimeter wave/terahertz communication method based on visual perception according to claim 2, wherein the method further comprises:

acquiring depth information of a base station end in real time by using a camera;

identifying the position of the base station end in the image according to the image information by using the intelligent reflecting surface;

converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;

and performing incident angle compensation on the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.

4. The millimeter wave/terahertz communication method based on visual perception according to claim 3, wherein the pitch angle and the azimuth angle of the real-time position of the base station end calculated based on the three-dimensional coordinates of the base station end comprise:

calculating the incident angle corresponding to each antenna of the intelligent reflecting surface according to the following formula:

in the above formula, the first and second carbon atoms are,at base station end pitch angle, θ₀Is the azimuth of the base station side.

5. The millimeter wave/terahertz communication method based on visual perception according to claim 4, wherein before the antenna for controlling the intelligent reflecting surface directs the signal beam to the direction of the moving end, the phase-shifted phase of the antenna for the intelligent reflecting surface is obtained by the following formula

Where, is, the dot multiplication, i.e. the multiplication of elements at corresponding positions of two matrices with the same dimension.

6. The millimeter wave/terahertz communication method based on visual perception according to claim 5, wherein the method further comprises:

recognizing the positions of the moving end and the intelligent reflecting surface by using the image acquired by the camera; when the direct radiation path between the base station end and the mobile end can be reached, the base station end judges the position of the mobile end according to the image collected by the camera, and then controls the array antenna to enable the position of the mobile end to be reachedThe wave beam points to the position of the mobile terminal, the phase angle matrix is inputWhen the direct radiation path is blocked, the base station end judges the position of the intelligent reflecting surface according to the image collected by the camera, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is

7. The millimeter wave/terahertz communication method based on visual perception according to claim 5 or 6, wherein the method further comprises:

the position of the base station end and the position of the intelligent reflecting surface are identified by using the image acquired by the camera, when the direct beam path between the base station end and the mobile end can be reached, the mobile end judges the position of the base station end according to the image acquired by the camera, then the array antenna is controlled to point the wave beam to the position of the base station end, and the input phase angle matrix isWhen the direct radiation path is blocked, the mobile terminal judges the position of the intelligent reflecting surface according to the image collected by the camera of the mobile terminal, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is

8. A millimeter wave/terahertz communication device based on visual perception is characterized by comprising: the system comprises a camera, a base station end, an intelligent reflecting surface and a mobile end;

the camera is used for acquiring image information and depth information of the mobile terminal in real time;

the intelligent reflecting surface identifies the position of the mobile terminal in the image according to the image information; the camera is also used for converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;

the intelligent reflecting surface is also used for controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the moving end based on the pitch angle and the azimuth angle of the real-time position of the moving end calculated by the three-dimensional coordinate of the moving end.

9. The millimeter wave/terahertz communication device based on visual perception according to claim 8, wherein the intelligent reflective surface identifies a position of a base station end in an image according to the image information; the camera is also used for converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;

and the intelligent reflecting surface is also used for compensating the incident angle of the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.

10. A millimeter wave/terahertz communication system based on visual perception, characterized by comprising the millimeter wave/terahertz communication device based on visual perception according to any one of claims 8 to 9.

Technical Field

The invention relates to the technical field of communication, in particular to a millimeter wave/terahertz communication method, device and system based on visual perception.

Background

Since birth, the communication technology industry has been following the moore's law of ' ten years and one generation ', and the whole mobile communication is developing in a direction of becoming faster, more stable and safer from the analog communication mode of the first generation mobile communication (1G) beginning in 1980, to the 2G taking TDMA as the core technology in about 1990, to the 3G age taking CDMA as the core technology in the beginning of the century, to the large-scale commercial use of the 4G mobile communication network taking OFDM as the core technology in about 2010. In 2020, the fifth generation mobile communication technology (5G) is called "5G yuan year" because of its advent, and has a good performance in scenarios such as enhanced mobile bandwidth communication (eMBB), ultra-reliable low-latency communication (URLLC), and large-scale machine type communication (mtc).

At present, the beam forming and tracking of the intelligent reflecting surface in the traditional high-frequency communication field basically adopts the following modes: pilot scanning and traversing all phases in the direction of the intelligent reflecting surface are used firstly to determine the direction of the moving end. When the mobile terminal moves, the traditional method needs to continuously send the signal-to-noise ratio near the direction of the pilot frequency scanning mobile terminal, the mobile terminal needs to feed back the value of the signal-to-noise ratio to the intelligent reflecting surface through an uplink channel, and then the intelligent reflecting surface selects the signal-to-noise ratio in the largest direction as the direction of the reflected wave beam; in the field of radar phased arrays, channels are mainly divided into communication channels and beacon channels, and the channel values of the beacon channels are analyzed by a difference power division network through a wave arrival algorithm, so that the signal source direction is judged.

The system has a large degree of dependence on the hardware of the intelligent reflecting surface, and when the energy of a scanning signal is weak or the energy of a beacon signal is difficult to detect, the direction of a mobile terminal is inconvenient to determine. And the pilot occupies more channel resources. When the number of mobile terminals is large, the calculation complexity also rises sharply, and the time delay is large.

Disclosure of Invention

The invention provides a millimeter wave/terahertz communication method, a device and a system based on visual perception, which are used for solving the technical defects in the prior art, realizing the purpose of judging the position of a target by using a spectrum, tracking the target and saving channel resources occupied by pilot frequency; when the number of the mobile terminals is large, the calculation complexity is not changed, and the time delay is small.

The invention provides a millimeter wave/terahertz communication method based on visual perception, which comprises the following steps:

acquiring image information and depth information of a mobile terminal in real time by using a camera; the camera is arranged on the intelligent reflecting surface;

identifying the position of the mobile terminal in the image according to the image information by using the intelligent reflecting surface;

converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;

and controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the mobile terminal based on the pitch angle and the azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal.

Preferably, the millimeter wave/terahertz communication method based on visual perception is characterized in that the origin of coordinates of the camera is at the geometric center of the camera surface, and the origin of coordinates of the intelligent reflecting surface is approximately the same as the origin of coordinates of the camera;

the coordinates of the mobile terminal in the camera coordinate system are as follows:

(X_OQ,Y_OQ,Z_OQ)

the coordinate of the mobile terminal in the intelligent reflecting surface coordinate system is as follows:

the conversion formula of the coordinates of the mobile terminal in the camera coordinate system and the intelligent reflecting surface coordinate system is as follows:

the pitch angle θ of the moving end is obtained by the following formula:

azimuth angle of the moving endThe following equation is obtained:

the method further comprises the following steps: calculating the phase angle of the reflected wave beam corresponding to each antenna of the intelligent reflecting surface according to the following formula

In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.

Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:

identifying the position of the base station end in the image according to the image information by using the intelligent reflecting surface;

converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;

and performing incident angle compensation on the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.

Preferably, the millimeter wave/terahertz communication method based on visual perception, wherein the pitch angle and the azimuth angle of the real-time position of the base station end, which are calculated based on the three-dimensional coordinates of the base station end, include:

calculating the incident angle corresponding to each antenna of the intelligent reflecting surface according to the following formula:

in the above formula, the first and second carbon atoms are,at base station end pitch angle, θ₀Is the azimuth of the base station side.

Preferably, in the millimeter wave/terahertz communication method based on visual perception, before the antenna controlling the intelligent reflecting surface directs the signal beam to the direction of the moving end, the phase shift phase of the antenna of the intelligent reflecting surface is obtained by the following formula

Where, is, the dot multiplication, i.e. the multiplication of elements at corresponding positions of two matrices with the same dimension.

Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:

recognizing the positions of the moving end and the intelligent reflecting surface by using the image acquired by the camera; when the direct beam path between the base station end and the mobile end can be reached, the base station end judges the position of the mobile end according to the image collected by the camera, and then controls the array antenna to point the beam to the position of the mobile end, and the input phase angle matrix isWhen the direct radiation path is blocked, the base station end judges the position of the intelligent reflecting surface according to the image collected by the camera, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is

Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:

the position of the base station end and the position of the intelligent reflecting surface are identified by using the image acquired by the camera, when the direct beam path between the base station end and the mobile end can be reached, the mobile end judges the position of the base station end according to the image acquired by the camera, then the array antenna is controlled to point the wave beam to the position of the base station end, and the input phase angle matrix isWhen the direct radiation path is blocked, the mobile terminal judges the position of the intelligent reflecting surface according to the image collected by the camera of the mobile terminal, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is

The invention also provides a millimeter wave/terahertz communication device based on visual perception, which comprises: the system comprises a camera, a base station end, an intelligent reflecting surface and a mobile end;

the camera is used for acquiring image information and depth information of the mobile terminal in real time; the camera is arranged on the intelligent reflecting surface;

the intelligent reflecting surface identifies the position of the mobile terminal in the image according to the image information; the camera is also used for converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;

the intelligent reflecting surface is also used for controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the moving end based on the pitch angle and the azimuth angle of the real-time position of the moving end calculated by the three-dimensional coordinate of the moving end.

Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the intelligent reflecting surface identifies the position of the base station end in the image according to the image information; the camera is also used for converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;

and the intelligent reflecting surface is also used for compensating the incident angle of the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.

Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the antenna of the intelligent reflecting surface is a passive phased array antenna, and/or the camera has a function of calculating the left-right distance and the depth of an object, and/or the camera is a binocular camera.

Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the camera coordinate origin is at the geometric center of the camera surface, and the coordinate origin of the intelligent reflecting surface is approximately the same as the camera coordinate origin.

The invention also provides a millimeter wave/terahertz communication system based on visual perception, which comprises the millimeter wave/terahertz communication device based on visual perception.

According to the millimeter wave/terahertz communication method, device and system based on visual perception, provided by the invention, the camera is used as visual assistance, the mobile terminal can be rapidly identified, the position of the target can be determined, the phased array antenna of the intelligent reflecting surface is controlled to generate the pitch angle and the azimuth angle in the corresponding direction, the wave beam can be enabled to be more rapidly directed to the corresponding target, the capability of identifying the position of the mobile terminal is improved, and the time delay is reduced. When the mobile terminal moves, the real-time position of the mobile terminal can be calculated, the intelligent reflecting surface is controlled to change the reflecting direction, and real-time beam tracking is realized.

Drawings

In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart of a millimeter wave/terahertz communication method based on visual perception provided by the invention;

FIG. 2 is a schematic diagram of spatial three-dimensional coordinates into which a picture taken by a camera provided by the present invention is translated;

fig. 3 is a schematic diagram of a coordinate system for reference when the intelligent reflecting surface is set to the pitch angle and the azimuth angle.

Fig. 4 is a schematic structural diagram of a third millimeter wave/terahertz communication device based on visual perception provided by the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention provides a millimeter wave/terahertz communication method based on visual perception, and referring to fig. 1, the method comprises the following steps:

s1, acquiring image information and depth information of the mobile terminal in real time by using the camera 10;

the cameras 10 may be disposed on the base station end 20, the intelligent reflective surface 30 or the mobile end 40, or two cameras 10 may be disposed on two ends of the base station end 20, the intelligent reflective surface 30 or the mobile end 40, or three cameras 10 may be disposed on the base station end 20, the intelligent reflective surface 30 or the mobile end 40, respectively, which may be implemented, where the cameras 10 are configured to collect image information in real time;

s2, recognizing the position of the mobile terminal 40 in the image according to the image information by using the intelligent reflecting surface 30;

the position of the mobile terminal 40 in the image can be identified according to the picture by using a traditional image processing technology based on the HSV imaging principle and an artificial intelligence technology of a deep neural network. At this time, the camera 10 is disposed on the smart reflective surface 30.

S3, converting the position of the mobile terminal 40 in the image into the three-dimensional coordinate of the mobile terminal 40 based on the depth information of the mobile terminal;

and S4, controlling the antenna of the intelligent reflecting surface 30 to point the signal beam to the direction of the mobile terminal 40 according to the pitch angle and the azimuth angle of the real-time position of the mobile terminal 40 calculated based on the three-dimensional coordinates of the mobile terminal 40.

The antenna of the intelligent reflecting surface 30 is a passive phased array antenna.

The hardware part required to be used in the millimeter wave/terahertz communication method based on visual perception comprises a camera 10, a multi-core server, a base station end 20 composed of a PXIe case, an FPGA coprocessor, a clock processing module, software defined radio and other equipment, an intelligent reflecting surface 30 and an intelligent reflecting surface 30 connected with the base station, and a mobile end 40 composed of unmanned vehicles, PCs, software defined radio modules connected with the PCs and other equipment. The camera 10 refers to the camera 10 and an upper computer thereof, and the camera 10 is located on the intelligent reflecting surface 30 and can acquire image information in real time. The upper computer connected to the camera 10 recognizes the position of the mobile terminal 40 (or the base station terminal 20) in the image according to the image by using a conventional image processing algorithm or a depth learning algorithm, obtains depth information of the mobile terminal 40 according to the camera 10, and converts a two-dimensional position in the image into a three-dimensional coordinate in a space by using an internal reference matrix of the camera 10. The upper computer of the camera 10 calculates the pitch angle and the azimuth angle of the position of the moving end 40 according to the three-dimensional coordinates, and sends the angle information to the intelligent reflection surface 30, so that the beam points to the direction. When the mobile terminal 40 moves, the camera 10 and the host machine thereof can calculate the real-time position of the mobile terminal 40, control the intelligent reflective surface 30 to change the reflective direction, and realize real-time beam tracking.

The communication between the base station and the mobile terminal 40 may follow 4G-LTE, 5G-NR protocols or others, the software may use LabVIEW, LabVIEW NXG or others, and the frequency band may be any high-frequency band such as millimeter wave and terahertz. The camera 10 may be a binocular camera 10 or others, and the hardware composition of the base station side 20 and the mobile terminal 40 may be the above hardware or others, and the present solution is not limited. Can be configured reasonably according to actual conditions or needs.

According to the millimeter wave/terahertz communication method based on visual perception, provided by the invention, the camera 10 is used as visual assistance, the mobile terminal 40 can be rapidly identified and the target position can be determined, and the phased array antenna of the intelligent reflecting surface 30 is controlled to generate the pitch angle and the azimuth angle in the corresponding direction, so that the wave beam can be more rapidly directed to the corresponding target, the capacity of identifying the position of the mobile terminal 40 is improved, and the time delay is reduced. When the mobile terminal 40 moves, the real-time position of the mobile terminal 40 can be calculated, and the intelligent reflecting surface 30 is controlled to change the reflecting direction, so that real-time beam tracking is realized.

The coordinate system referred to when the intelligent reflection surface 30 sets the pitch angle and the azimuth angle is shown in fig. 3. When looking after the intelligent reflecting surface 30, the origin is at the top right vertex of the intelligent reflecting surface 30, the X axis is downward, the Y axis is leftward, and the Z axis is forward.

In the coordinate system, θ is the included angle between the line OQ between the moving end 40 and the origin and the positive direction of the Z axis,is the included angle between the connecting line OS of the projection point of the mobile terminal 40 on the XOY plane and the origin and the positive direction of the X axis. WhereinPositive in the left half and negative in the right half of the X-axis. For ease of description and calculation, the following major issues are specified:

the camera 10 origin of coordinates is at the geometric center of the camera 10 surface;

the origin of coordinates of the intelligent reflecting surface 30 is approximately the same as the origin of coordinates of the camera 10;

the coordinate systems are all right-handed systems, and the X/Y/Z coordinates are all positive and negative. The moving tip 40 has a Z coordinate greater than 0 in any coordinate system, and the positive and negative of the X coordinate and the Y coordinate are not limited.

The upper computer converts the pictures taken by the camera 10 into three-dimensional space coordinates as shown in fig. 2. When viewed from behind the camera 10, the origin is at the location of the right eye of the camera 10, with the X-axis to the right, the Y-axis down, and the Z-axis forward. This is originally set by the camera 10, and three-dimensional coordinates of the position of the moving end 40 are obtained in this coordinate system.

Preferably, the millimeter wave/terahertz communication method based on visual perception is characterized in that the origin of coordinates of the camera 10 is at the geometric center of the surface of the camera 10, and the origin of coordinates of the intelligent reflecting surface 30 is approximately the same as the origin of coordinates of the camera 10;

the coordinates of the moving end 40 in the coordinate system of the camera 10 are:

(X_OQ,Y_OQ,Z_OQ)

the coordinates of the moving end 40 in the coordinate system of the intelligent reflecting surface 30 are as follows:

the transformation formula of the coordinates of the moving end 40 in the coordinate system of the camera 10 and the coordinate system of the intelligent reflection surface 30 is as follows:

to avoid unnecessary calculations, the algorithm is simplified, and all of the following equations are used only for (X)_OQ,Y_OQ,Z_OQ) Expressed as a known quantity.

The pitch angle θ of the movable end 40 is obtained by the following equation:

azimuth angle of the moving end 40The following equation is obtained:

the upper computer of the camera 10 calculates θ andthen, the method further comprises: calculating the phase angle of the reflected beam corresponding to each antenna of the intelligent reflecting surface 30 according to the following formula

In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.

Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:

identifying the position of the base station end 20 in the image according to the image information by using the intelligent reflecting surface 30;

converting the position of the base station end 20 in the image into a three-dimensional coordinate of the base station end 20 based on the depth information of the base station end;

and performing angle of incidence compensation on the antenna of the intelligent reflecting surface 30 based on the pitch angle and the azimuth angle of the real-time position of the base station end 20 calculated by the three-dimensional coordinates of the base station end 20.

Preferably, the millimeter wave/terahertz communication method based on visual perception, wherein the pitch angle and the azimuth angle of the real-time position of the base station end 20, which are calculated based on the three-dimensional coordinates of the base station end 20, include:

the incident angle corresponding to each antenna of the intelligent reflection surface 30 is calculated according to the following formula:

in the above formula, the first and second carbon atoms are,at a base station end of 20 pitch angles theta₀As the azimuth angle of the base station 20, M is the number of matrix rows, N is the number of matrix columns, λ is the wavelength of the electromagnetic wave, and d is the antenna spacing, as described above.

In this embodiment, when M is 8, and f is 30GHz, λ is 10mm, and d is 40 mm.

Preferably, in the millimeter wave/terahertz communication method based on visual perception, before the antenna controlling the intelligent reflection surface 30 directs the signal beam to the direction of the mobile terminal 40, the phase shift phase of the antenna of the intelligent reflection surface 30 is obtained by the following formula

Wherein, the expression dot multiplication is that the elements of the corresponding positions of the two matrixes with the same dimension are multiplied, and the result is still the matrix with the same dimension. Phase shifting phaseEach element of (a) is the phase of the antenna that needs phase shift at the corresponding position on the antenna array.

Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:

recognizing the positions of the mobile terminal and the intelligent reflecting surface 30 by using the image acquired by the camera 10, wherein the camera 10 is arranged at the base station terminal 20; when the direct path between the base station 20 and the mobile station 40 is accessibleIn this case, the base station 20 determines the position of the mobile terminal 40 according to the image collected by the camera 10, and then controls the array antenna to direct the beam to the position of the mobile terminal 40, where the input phase angle matrix isWhen the direct beam path is blocked, the base station 20 determines the position of the intelligent reflection surface 30 according to the image collected by the camera 10, and then controls the array antenna to direct the beam to the position of the intelligent reflection surface 30, wherein the input phase angle matrix is

Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:

the positions of the base station end 20 and the intelligent reflecting surface 30 are identified by using the image acquired by the camera 10, the camera 10 is arranged at the mobile end 40 at the moment, when the direct beam path between the base station end 20 and the mobile end 40 can be reached, the mobile end 40 judges the position of the base station end 20 according to the image acquired by the camera 10, then the array antenna is controlled to point the wave beam to the position of the base station end 20, and the input phase angle matrix isWhen the direct beam path is blocked, the mobile terminal 40 determines the position of the intelligent reflection surface 30 according to the image collected by the camera, and then controls the array antenna to direct the beam to the position of the intelligent reflection surface 30, wherein the input phase angle matrix is

The invention also provides a millimeter wave/terahertz communication device based on visual perception, which comprises: the system comprises a camera 10, a base station end 20, an intelligent reflecting surface 30 and a mobile end 40;

the camera 10 is used for acquiring image information and depth information of a mobile terminal in real time;

at this time, the camera 10 is disposed on the intelligent reflection surface 30;

the intelligent reflecting surface 30 identifies the position of the mobile terminal 40 in the image according to the image information; the camera 10 is further configured to convert the position of the mobile terminal 40 in the image into a three-dimensional coordinate of the mobile terminal 40 based on the depth information of the mobile terminal;

the intelligent reflecting surface 30 is further configured to control the antenna of the intelligent reflecting surface 30 to direct the signal beam to the direction of the moving end 40 based on the pitch angle and the azimuth angle of the real-time position of the moving end 40 calculated by the three-dimensional coordinates of the moving end 40.

Referring to fig. 4, when the camera 10 is located on the intelligent reflective surface 30, the intelligent reflective surface 30 identifies the real-time position of the mobile terminal 40 according to the image information, and controls the antenna of the intelligent reflective surface 30 to direct the second signal beam toward the mobile terminal 40.

The millimeter wave/terahertz communication device based on visual perception provided by the invention uses the camera 10 as visual assistance, can quickly identify the moving end 40 and determine the position of a target, and controls the phased array antenna of the intelligent reflecting surface 30 to generate a pitch angle and an azimuth angle in the corresponding direction, so that a wave beam can be more quickly directed to the corresponding target, the capacity of identifying the position of the moving end 40 is improved, and the time delay is reduced. The whole scheme is a concrete implementation and innovative application of perception and communication integration.

Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the intelligent reflecting surface 30 identifies the position of the base station end 20 in the image according to the image information; the camera 10 is further configured to convert the position of the base station end 20 in the image into three-dimensional coordinates of the base station end 20 based on the depth information of the base station end;

the intelligent reflecting surface 30 is further configured to perform an incident angle compensation on the antenna of the intelligent reflecting surface 30 based on the pitch angle and the azimuth angle of the real-time position of the base station end 20 calculated by the three-dimensional coordinates of the base station end 20.

Note the angle of incidence of the intelligent reflective surface 30Is fixed in general and can be input into the intelligent reflection after being measured manuallyThe surface 30 may be used to calculate the incident angle by recognizing the base station end 20 and determining the position thereof using the camera 10. Referring to fig. 4, when the camera 10 is located on the intelligent reflective surface 30, the intelligent reflective surface 30 identifies the real-time position of the base station 20 according to the image information, and performs the angle of incidence compensation on the antenna of the intelligent reflective surface 30.

The camera 10 needs to compensate the incident angle of the beam emitted from the base station 20 to the intelligent reflective surface 30, and then write the reflection angle reflected to the mobile terminal 40. The upper computer of the camera 10 is as followsWritten to the intelligent reflective surface 30.

Compared with the traditional method, the method has wider application scenes and lower time delay in the beam forming and tracking directions in the communication field, and is the specific realization and application of the perception communication integration.

Preferably, the millimeter wave/terahertz communication device based on visual perception is configured such that the antenna of the intelligent reflection surface 30 is a passive phased array antenna, and/or the camera 10 has a function of calculating a left-right distance and a depth of an object, and/or the camera 10 is a binocular camera 10. The camera 10 may be another camera having a function of calculating the left-right distance and the depth of the object, and preferably, the camera 10 takes a picture without being limited by a frequency band, so that the device is suitable for any high-frequency band such as millimeter waves and terahertz waves.

Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the origin of coordinates of the camera 10 is at the geometric center of the surface of the camera 10, and the origin of coordinates of the intelligent reflecting surface 30 is approximately the same as the origin of coordinates of the camera 10. Approximately the same as the origin of coordinates of the camera 10 means substantially close to the same state, since it is difficult to achieve exact identity, and therefore substantially the same degree is sufficient.

Preferably, the millimeter wave/terahertz communication device based on visual perception is provided, wherein the antenna of the mobile terminal 40 is an omnidirectional antenna or a horn antenna.

The invention also provides a millimeter wave/terahertz communication system based on visual perception, which comprises the millimeter wave/terahertz communication device based on visual perception, wherein the millimeter wave/terahertz communication device based on visual perception comprises: the system comprises a camera 10, a base station end 20, an intelligent reflecting surface 30 and a mobile end 40;

the camera 10 is used for acquiring image information and depth information of a mobile terminal in real time;

at this time, the camera 10 is disposed on the intelligent reflection surface 30;

the intelligent reflecting surface 30 identifies the position of the mobile terminal 40 in the image according to the image information; the camera 10 is further configured to convert the position of the mobile terminal 40 in the image into a three-dimensional coordinate of the mobile terminal 40 based on the depth information of the mobile terminal;

the intelligent reflecting surface 30 is further configured to control the antenna of the intelligent reflecting surface 30 to direct the signal beam to the direction of the moving end 40 based on the pitch angle and the azimuth angle of the real-time position of the moving end 40 calculated by the three-dimensional coordinates of the moving end 40.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.