阅读说明：本技术 移动终端设备的体温测量方法、移动终端设备及介质 (Body temperature measuring method of mobile terminal device, mobile terminal device and medium ) 是由 黄河 于 2020-06-16 设计创作，主要内容包括：本发明公开了一种移动终端设备的体温测量方法、移动终端设备及介质,移动终端设备包括温度检测单元和成像单元,方法包括：设定参考坐标系,参考坐标系包括X轴和Y轴；设定温度检测单元在参考坐标系中的坐标为第一坐标,温度检测单元用于沿第一方向实时检测人脸面部的温度,第一方向垂直于X轴和Y轴所在的平面；利用成像单元沿第一方向实时获取人脸面部的影像；确定人脸面部的测温点的影像在参考坐标系中的坐标为第二坐标；判断第二坐标是否与第一坐标是否重合或基本重合；以及,判断移动终端设备与测温点沿第一方向的距离是否在预设范围之内；若是,则输出温度检测单元检测到的温度信号。实现在移动终端设备上进行方便、快捷且精确的体温测量。(The invention discloses a body temperature measuring method of a mobile terminal device, the mobile terminal device and a medium, wherein the mobile terminal device comprises a temperature detecting unit and an imaging unit, and the method comprises the following steps: setting a reference coordinate system, wherein the reference coordinate system comprises an X axis and a Y axis; setting the coordinate of a temperature detection unit in a reference coordinate system as a first coordinate, wherein the temperature detection unit is used for detecting the temperature of the face of a human in real time along a first direction, and the first direction is vertical to the plane where the X axis and the Y axis are located; acquiring an image of the face of a human in real time along a first direction by using an imaging unit; determining the coordinates of the images of the temperature measuring points of the human face in the reference coordinate system as second coordinates; judging whether the second coordinate is coincident or basically coincident with the first coordinate; judging whether the distance between the mobile terminal equipment and the temperature measuring point along the first direction is within a preset range or not; if yes, outputting the temperature signal detected by the temperature detection unit. The convenient, quick and accurate body temperature measurement on the mobile terminal equipment is realized.)

1. A body temperature measuring method of a mobile terminal device including a temperature detecting unit and an imaging unit, the method comprising:

setting a reference coordinate system, wherein the reference coordinate system comprises an X axis and a Y axis;

setting the coordinate of the temperature detection unit in the reference coordinate system as a first coordinate, wherein the temperature detection unit is used for detecting the temperature of the face of a human in real time along a first direction, and the first direction is perpendicular to the plane where the X axis and the Y axis are located;

acquiring an image of the face in real time along the first direction by using the imaging unit;

determining the coordinates of the images of the temperature measuring points of the human face in the reference coordinate system as second coordinates;

judging whether the second coordinate is coincident or basically coincident with the first coordinate; judging whether the distance between the mobile terminal equipment and the temperature measuring point along the first direction is within a preset range or not;

and if so, outputting the temperature signal detected by the temperature detection unit.

2. The method according to claim 1, wherein the preset range is a depth of field of the temperature detection unit.

3. The method for measuring body temperature of mobile terminal device according to claim 2, wherein said depth of field is in a range of 0cm to 50 cm.

4. The method according to claim 1, wherein the first direction is parallel to an optical axis of the imaging unit.

5. The method for measuring body temperature of a mobile terminal device according to claim 1, further comprising, before outputting the temperature signal:

judging whether the temperature signal is an effective temperature signal;

when the second coordinate is coincident or basically coincident with the first coordinate and the distance between the mobile terminal and the temperature measuring point along the first direction is within a preset range, the detected temperature signal is an effective temperature signal; otherwise, it is an invalid temperature signal.

6. The method for measuring the body temperature of the mobile terminal device according to claim 1, wherein determining the coordinates of the image of the temperature measurement point of the face in the reference coordinate system as second coordinates comprises:

determining images of feature points of the human face according to the images of the human face, wherein the feature points and the temperature measuring points have a specific geometric position relation;

determining the coordinates of the images of the feature points in the reference coordinate system;

and calculating the second coordinate of the temperature measuring point image according to the coordinate of the image of the characteristic point in the reference coordinate system.

7. The method for measuring the body temperature of the mobile terminal device according to claim 6, wherein the temperature measuring point is a central area of a forehead;

the characteristic points are two eyes, and each eye is one characteristic point;

or, the feature points are two ears, and each ear is a feature point;

or the characteristic points comprise at least two points of the human face contour.

8. The method for measuring the body temperature of the mobile terminal device according to claim 6, wherein the temperature measuring point is an ear canal; the feature points include at least two points of an ear contour.

9. The method for measuring the body temperature of the mobile terminal device according to claim 1, wherein determining the coordinates of the image of the temperature measurement point of the face in the reference coordinate system as second coordinates comprises:

identifying a temperature measurement point image of the face image based on the characteristics of the temperature measurement points, wherein the characteristics of the temperature measurement points comprise: at least one of shape and image characteristics;

and determining the second coordinate of the temperature measuring point image in the reference coordinate system based on the identified temperature measuring point image.

10. The method according to claim 1, wherein the reference coordinate system comprises an image coordinate system, a camera coordinate system or a pixel coordinate system of the imaging unit.

11. The method according to claim 1, wherein setting the position of the temperature detection unit in the reference coordinate system as a first coordinate comprises:

and according to the position of the temperature detection unit in the mobile terminal, pre-storing a first coordinate of the temperature detection unit in the reference coordinate system.

12. The method for measuring the body temperature of the mobile terminal device according to claim 1, wherein the step of determining whether the distance between the mobile terminal and the temperature measuring point along the first direction is within a preset range comprises:

acquiring the distance between the temperature measurement point of the face and the mobile terminal in real time;

comparing the obtained distance with the preset range;

and judging whether the measured distance is within the preset range.

13. The method for measuring the body temperature of the mobile terminal device according to claim 12, wherein the step of obtaining the distance between the temperature measurement point of the face and the mobile terminal in real time comprises:

estimating the distance between the temperature detection unit and the temperature measurement point along the first direction according to the image of the characteristic point; alternatively, the first and second electrodes may be,

detecting the distance between the temperature measuring point and the mobile terminal equipment by using a preset distance sensor in the mobile terminal equipment;

alternatively, the first and second electrodes may be,

the image is a three-dimensional image;

and acquiring depth information of the face of the human face, and estimating the distance between the temperature detection unit and the temperature measurement point based on the depth information.

14. The method for measuring body temperature of mobile terminal device according to claim 1, wherein said temperature detecting unit is an infrared temperature sensor.

15. A mobile terminal device, characterized by comprising:

the imaging unit is used for acquiring an image of the face of a human face along a first direction;

a temperature detection unit for detecting a temperature of the face portion of the person in the first direction;

a storage unit that stores: information related to a reference coordinate system, the information related to the reference coordinate system including: an X axis and a Y axis, the first direction being perpendicular to a plane in which the X axis and the Y axis lie; the coordinate of the temperature detection unit in the reference coordinate system is a first coordinate;

the processing unit is used for determining the coordinates of the images of the temperature measuring points of the human face in the reference coordinate system as second coordinates;

the judging unit is used for judging whether the second coordinate is overlapped or basically overlapped with the first coordinate; judging whether the distance between the mobile terminal equipment and the temperature measuring point along the first direction is within a preset range or not;

and the display unit is used for displaying the temperature value detected by the temperature checking unit when the judgment result of the judging unit is yes.

16. The mobile terminal apparatus according to claim 15, wherein the judging unit includes:

the first sub-judgment unit is used for judging whether the second coordinate is coincident or basically coincident with the first coordinate;

the second sub-judgment unit is used for judging whether the distance between the mobile terminal equipment and the temperature measuring point along the first direction is within a preset range or not;

the output unit outputs a judgment signal, and when the judgment results of the first sub-judgment unit and the second sub-judgment unit are both yes, the judgment signal is an effective temperature signal; and when one of the judgment results of the first sub-judgment unit and the second sub-judgment unit is negative, the judgment signal is an invalid temperature signal.

17. The mobile terminal device according to claim 15, wherein the processing unit comprises:

the first determining subunit is used for determining an image of a feature point of the human face according to the image of the human face acquired by the imaging unit, and the storage unit also stores a specific geometric position relation between the feature point and the temperature measuring point;

a second determining subunit, configured to determine, according to the image of the feature point determined by the first determining unit, coordinates of the image of the feature point in the reference coordinate system;

and the third determining subunit is configured to obtain the geometric position relationship, and calculate the second coordinate of the temperature measurement point image according to the coordinate of the image of the feature point in the reference coordinate system and the geometric position relationship.

18. The mobile terminal device according to claim 17, wherein the temperature measurement point is a central area of a forehead;

the characteristic points are two eyes, and each eye is one characteristic point;

or, the feature points are two ears, and each ear is a feature point;

or the characteristic points comprise at least two points of the human face contour.

19. The mobile terminal device of claim 17, wherein the temperature measurement point is an ear canal; the feature points include at least two points of an ear contour.

20. The mobile terminal device according to claim 15, further comprising:

the temperature measuring point identification unit is used for identifying a temperature measuring point image in the face image according to the characteristics of the temperature measuring points and determining the second coordinate of the temperature measuring point image in the reference coordinate system based on the identified temperature measuring point image;

the temperature measuring points are characterized by comprising: at least one of shape and image characteristics.

21. The mobile terminal device according to claim 15, wherein the first direction is parallel to an optical axis of the imaging unit.

22. The mobile terminal device according to claim 15, wherein the preset range is a depth of field of the temperature detection unit.

23. The mobile terminal device of claim 22, wherein the depth of field is in a range of 0cm to 50 cm.

24. The mobile terminal device according to claim 15, characterized in that the reference coordinate system comprises an image coordinate system of the imaging unit.

25. The mobile terminal device according to claim 15, wherein the first coordinate is determined based on a position of the temperature detection unit in the mobile terminal device and is stored in the storage unit in advance.

26. The mobile terminal device according to claim 17, further comprising a distance obtaining unit configured to estimate a distance between the temperature detecting unit and the temperature measuring point along the first direction according to the image of the feature point;

alternatively, the first and second electrodes may be,

detecting the distance between the temperature measuring point and the mobile terminal equipment by using a preset distance sensor in the mobile terminal equipment;

alternatively, the first and second electrodes may be,

and when the image acquired by the imaging unit is a three-dimensional image, estimating the distance between the temperature detection unit and the temperature measurement point based on the acquired depth information of the face.

27. The mobile terminal device of claim 26, wherein the distance sensor is one of an infrared distance sensor, a TOF sensor, and a lidar.

28. The mobile terminal device according to claim 15, wherein the imaging unit comprises at least one of an optical monocular camera, an optical binocular camera, a TOF sensor, a 3D structured light sensor.

29. The mobile terminal device according to claim 15, wherein the temperature detection unit is an infrared temperature sensor.

30. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method for measuring a body temperature of a mobile terminal device according to any one of claims 1 to 14.

Technical Field

The invention belongs to the field of body temperature measurement, and particularly relates to a body temperature measurement method of a mobile terminal device, the mobile terminal device and a medium.

Background

In the aspect of existing body temperature measurement, a thermometer, a forehead thermometer and the like are commonly used, a special temperature measuring tool is needed for measurement, and the user is very inconvenient to carry the temperature measuring tool or carry out daily temperature measurement. Meanwhile, the existing infrared detection temperature measuring device is often inaccurate in measuring point position, for example, when a hand-held forehead temperature gun is used for measuring temperature of a human body, the measuring point position is only an approximate position, such as a wrist or a forehead, and the temperature measuring point cannot be accurately positioned by adopting a manual mode, so that the measured temperature accuracy is low, and manpower is wasted. The infrared body temperature measurement used in places with a large stream of people detects the temperature of the whole human body, so that the equipment is large in size, high in cost and inconvenient to use in places with relatively small stream of people.

Therefore, it is necessary to provide a portable and easy-to-use body temperature measuring method capable of accurately positioning a temperature measuring point.

Disclosure of Invention

In order to solve the above problems, embodiments of the present invention provide a body temperature measuring method for a mobile terminal device, and a medium, which are portable and easy to use and can accurately position a temperature measuring point.

In order to achieve the above object, in a first aspect, the present invention provides a body temperature measuring method of a mobile terminal device, the mobile terminal device including a temperature detecting unit and an imaging unit, the method including:

setting a reference coordinate system, wherein the reference coordinate system comprises an X axis and a Y axis;

setting the coordinate of the temperature detection unit in the reference coordinate system as a first coordinate, wherein the temperature detection unit is used for detecting the temperature of the face in real time along a first direction, and the first direction is perpendicular to the plane where the X axis and the Y axis are located;

acquiring an image of the face in real time along the first direction by using the imaging unit;

determining the coordinates of the images of the temperature measuring points of the human face in the reference coordinate system as second coordinates;

judging whether the second coordinate is coincident or basically coincident with the first coordinate; judging whether the distance between the mobile terminal and the temperature measuring point along the first direction is within a preset range or not;

and if so, outputting the temperature signal detected by the temperature detection unit.

In a second aspect, the present invention further provides a mobile terminal device, including:

the imaging unit is used for acquiring an image of the face of a human face along a first direction;

a temperature detection unit for detecting a temperature of the face portion of the person in the first direction;

a storage unit that stores: information related to a reference coordinate system, the information related to the reference coordinate system including: an X axis and a Y axis, the first direction being perpendicular to a plane in which the X axis and the Y axis lie; the coordinate of the temperature detection unit in the reference coordinate system is a first coordinate;

the processing unit is used for determining the coordinates of the images of the temperature measuring points of the human face in the reference coordinate system as second coordinates;

the judging unit is used for judging whether the second coordinate is overlapped or basically overlapped with the first coordinate; judging whether the distance between the mobile terminal and the temperature measuring point along the first direction is within a preset range or not;

and the display unit is used for displaying the temperature value detected by the temperature checking unit when the judgment result of the judging unit is yes.

In a third aspect, the present invention also proposes a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method for measuring a body temperature of a mobile terminal device of the first aspect.

The invention has the beneficial effects that:

the method comprises the steps of acquiring a face image in real time along a first direction through an imaging unit, detecting the temperature of the face along the first direction through a temperature detection unit, setting a first coordinate of the temperature detection unit in a reference coordinate system, wherein the first direction is vertical to an X axis and a Y axis of the reference coordinate system, determining a second coordinate of an image of a temperature measurement point in the reference coordinate system based on the acquired face image, judging whether the temperature measurement point is aligned with the temperature detection unit by judging whether the second coordinate and the first coordinate are overlapped or basically overlapped in the reference coordinate system, and determining the detection value of the temperature detection unit to be an effective temperature value when the second coordinate and the first coordinate are overlapped or basically overlapped, so that the accurate positioning of the temperature measurement point of the face on the mobile terminal device is realized, and the accurate measurement can be realized. Moreover, the method is used on the mobile terminal, so that people can conveniently measure the temperature at any time by using mobile terminal equipment such as a mobile phone, and the method is convenient and quick.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart illustrating a method for measuring a body temperature of a mobile terminal device according to a first embodiment of the present invention;

fig. 2 is a schematic structural diagram of a mobile terminal device according to a second embodiment of the present invention;

fig. 3 is a relationship intention of each functional unit in a mobile terminal device according to a second embodiment of the present invention;

fig. 4 is a reference diagram of a mobile terminal according to a second embodiment of the present invention;

description of reference numerals:

in fig. 2 and 3:

1. a mobile terminal device; 2. an imaging unit; 3. a temperature detection unit; 4. a storage unit; 5. a processing unit; 501. a first determining subunit; 502. a second determining subunit; 503. a third determining subunit; 504. a temperature measuring point identification unit; 6. a judgment unit; 601. a first sub-judgment unit; 602. a second sub-judgment unit; 603. an output unit; 7. a display unit; 8. measuring temperature points; 9. a distance acquisition unit; 10. a distance sensor.

Detailed Description

At present, specialized tools or equipment such as thermometers, handheld forehead thermometers or infrared imaging equipment are generally adopted for body temperature measurement, and it is very inconvenient for users to carry temperature measuring tools or measure temperature daily. The temperature detection unit is integrated on the mobile terminal device, so that the daily temperature measurement requirement of people can be met.

In order to realize accurate body temperature measurement, relating to the technical problem of aligning a temperature detection unit with a temperature measuring point, the invention establishes a reference coordinate system and gives a first coordinate of the temperature detection unit, acquires a face image of a human face and determines an image of the temperature measuring point through an imaging unit, calculating and determining a second coordinate of the temperature measuring point in the set reference coordinate system, wherein the direction of the light radiation received by the imaging unit is vertical to the plane of the reference coordinate system, and the transformed first and second coordinates can correspond to a projection of the true physical positions of the temperature sensing unit and the temperature sensing point in the first direction on the X-Y plane and have a specific and identical transformed relationship, then, by judging whether the first coordinate and the second coordinate are coincident or basically coincident, the technical problem of judging whether the temperature measuring point is aligned with the temperature detecting unit is solved, and accurate body temperature measurement can be realized on the mobile terminal device.

Preferred embodiments of the present invention will be described in more detail below. While the following describes preferred embodiments of the present invention, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein.

Example one

Fig. 1 is a flowchart illustrating a method for measuring a body temperature of a mobile terminal device according to a first embodiment of the present invention;

as shown in fig. 1, a method for measuring body temperature of a mobile terminal device, the mobile terminal device including a temperature detection unit and an imaging unit, the method includes:

step S1: setting a reference coordinate system, wherein the reference coordinate system comprises an X axis and a Y axis;

step S2: setting the coordinate of a temperature detection unit in a reference coordinate system as a first coordinate, wherein the temperature detection unit is used for detecting the temperature of the face of a human in real time along a first direction, and the first direction is vertical to the plane where the X axis and the Y axis are located;

step S3: acquiring an image of the face of a human in real time along a first direction by using an imaging unit;

step S4: determining the coordinates of the images of the temperature measuring points of the human face in the reference coordinate system as second coordinates;

step S5: judging whether the second coordinate is coincident or basically coincident with the first coordinate; judging whether the distance between the mobile terminal equipment and the temperature measuring point along the first direction is within a preset range or not;

step S6: if yes, outputting the temperature signal detected by the temperature detection unit.

The invention collects the face image in real time through the imaging unit, gives the first coordinate of the temperature detection unit in the reference coordinate system, determines the second coordinate of the image of the temperature measurement point in the reference coordinate system based on the collected face image, judges whether the second coordinate and the first coordinate are superposed or basically superposed in the reference coordinate system, and further judges whether the temperature detection unit and the temperature measurement point are aligned, when the temperature detection unit and the temperature measurement point are aligned, the detection value of the temperature detection unit is an effective temperature value, thereby realizing the accurate positioning of the temperature measurement point of the face on the mobile terminal equipment, and further realizing the accurate body temperature measurement. Moreover, the method is used on the mobile terminal, so that people can conveniently measure the temperature at any time by using mobile terminal equipment such as a mobile phone, and the method is convenient and quick.

Specific examples of the present invention will be described in detail below with reference to specific examples.

In this embodiment, an imaging unit and a temperature detection unit are integrated on a mobile terminal device in advance, and a temperature measurement point is set as a forehead central area of a face, wherein both a direction in which the imaging unit receives light radiation and a direction in which the temperature detection unit receives infrared radiation face a target to be measured.

Regarding step S1: a reference coordinate system is set, the reference coordinate system including an origin, an X-axis and a Y-axis.

In this embodiment, an image coordinate system (two-dimensional coordinate system) of the imaging unit may be set as a reference coordinate system, which is an X-axis and a Y-axis; the origin coincides with the optical axis of the imaging unit. In other embodiments, a camera coordinate system of the imaging unit may be set as a reference coordinate system, or a pixel coordinate system of the imaging unit may be set as a reference coordinate system, and the coordinate systems may all implement corresponding coordinate conversion according to lens parameters of the imaging unit, so as long as the reference coordinate system satisfies one of any coordinate systems belonging to the imaging unit itself, or the established reference coordinate system may have a known corresponding conversion relationship with any coordinate system of the imaging unit itself. In this embodiment, the image coordinate system is selected as the reference coordinate system for real-time description.

The purpose of setting the reference coordinate system is: the physical position of the temperature measuring point may be converted to a second coordinate in the reference coordinate system based on the image of the face obtained by the imaging unit, and the physical position of the temperature detecting unit may be converted to and stored in a first coordinate in the same reference coordinate system, for example, the coordinate (X) of the temperature detecting unit may be determined in the same reference coordinate system₀，Y₀) Coordinates of temperature measuring points (X)_t，Y_t) Since the direction of the light radiation received by the imaging unit is perpendicular to the plane of the X-axis and the Y-axis, and the converted first coordinate and the converted second coordinate can correspond to the projection of the real physical positions of the temperature detection unit and the temperature measuring point along the first direction on the X-Y plane and have a specific and same conversion relation, when (X is X) is zero₀，Y₀) And (X)_t，Y_t) When the reference coordinate system is coincident or basically coincident, the temperature detection unit is basically opposite to the temperature measurement point.

Regarding step S2: and setting the position of the temperature detection unit in the reference coordinate system as a first coordinate, wherein the temperature detection unit is used for detecting the temperature of the face of the detected object in real time along a first direction, and the first direction is vertical to the plane where the X axis and the Y axis are located.

In this embodiment, the first coordinates of the temperature detection unit in the reference coordinate system are stored in advance according to the position of the temperature detection unit in the mobile terminal device.

Specifically, since the temperature detection unit and the imaging unit are both fixedly arranged on the mobile terminal device, the position of the temperature detection unit on the mobile terminal device is fixed and unchanged, such as being arranged on the top of the mobile terminal device, i.e. the coordinates of the temperature detection unit are unchanged compared with the origin position of the reference coordinate system, therefore, the temperature detection unit and the imaging unit can be arranged on the mobile terminal device in a fixed manner, and the imaging unit can be arranged on the mobile terminal device in a fixed mannerThe physically fixed position based on the temperature detection unit is converted into a first coordinate in a reference coordinate system, the first coordinate being recorded as (X)₀，Y₀) In this embodiment, the temperature detection unit measures the temperature of the face of the person in a first direction, optionally, the first direction is parallel to the optical axis of the imaging unit.

Regarding step S3: acquiring an image of the face of a human in real time along a first direction by using an imaging unit; and step S4: and determining the position of the image of the temperature measuring point of the face in the reference coordinate system as a second coordinate.

The method for determining the second coordinate may include a first manner or a second manner, wherein the basic principle of the first manner is as follows: acquiring a two-dimensional or three-dimensional image of the face based on an imaging unit, identifying a feature point which has a specific geometric relationship with a temperature measuring point (forehead center) in the two-dimensional or three-dimensional image, and calculating a second coordinate of the temperature measuring point according to the coordinate of the feature point; the basic principle of the second mode is as follows: the method comprises the steps of acquiring a two-dimensional image or a three-dimensional image of a face based on an imaging unit, identifying the face based on a trained face intelligent identification algorithm model, directly positioning a temperature measuring point based on the position of the temperature measuring point marked in the model, and calculating a second coordinate of the temperature measuring point.

As will be described in detail below with respect to the first mode, the method for determining the second coordinate of the first mode includes:

step AS 41: determining images of feature points of the human face according to the images of the human face, wherein the feature points and the temperature measuring points have a specific geometric position relationship;

step AS 42: determining the coordinates of the images of the characteristic points in a reference coordinate system;

step AS 43: and calculating a second coordinate of the temperature measuring point image according to the coordinate of the image of the characteristic point in the reference coordinate system.

In one example, with respect to step AS41, when the measured temperature is forehead temperature, the temperature measurement point is the center region of the forehead, and the feature points may be two eyes, one for each eye. When the feature points are eyes, the feature information of the eyes can be stored in the mobile terminal device in advance, and the eyes can be identified by comparing the image of the face with the feature information of the eyes.

With respect to step AS42, after the feature points are identified, i.e., after the eyes are identified, the coordinates of the eyes in the reference coordinate system are determined by a certain algorithm.

In step AS42, the second coordinate of the temperature measuring point, i.e. the center of the forehead, can be calculated by a certain algorithm (e.g. triangulation algorithm) according to the specific geometric relationship between the eyes and the center area of the forehead.

In a specific application scenario, a face, namely a human face, can be identified through an AI face intelligent identification algorithm preset in a mobile terminal, a first characteristic point and a second characteristic point, namely two eyes, are further identified, and coordinates of the two eyes in an imaging coordinate system are calculated to be third coordinates (X) respectively₁，X₁) And fourth coordinate (X)₂，Y₂) Determining the coordinates of the central area of the forehead in an imaging coordinate system based on the coordinates of the eyes in the face image and the specific geometrical relationship between the eyes and the forehead, namely calculating the coordinates (X) of the temperature measuring point in the imaging coordinate system_t，Y_t) Wherein, because the optical parameters of the optical camera are fixed, the physical coordinate system of the face and the reference coordinate system (imaging coordinate system) have a fixed conversion relation, and finally, the coordinates of the temperature measuring point in the reference coordinate system, namely (X) are calculated through the fixed conversion relation_t，Y_t) The relative position relation between the real temperature measuring point and the mobile equipment can be corresponded.

The following scheme can be adopted for the identification of the feature points:

storing related characteristic point information of a temperature measuring point (forehead center) in a human face in a mobile terminal device, establishing a specific geometric position relation between the characteristic points and the forehead center position by acquiring a large number of human face images and marking positions of characteristic points of five sense organs, such as eyes, eyebrows, ears, mouth, nose, face contour and the like in each image, and forming a training data sample of an intelligent image algorithm (such as OPENCV and the like).

In another embodiment of the method, the feature points may also be binaural, one for each ear. The determination of the coordinates of the forehead temperature measurement point can be realized in a manner similar to the above-mentioned manner of using the coordinates of both eyes, and the method is the same as above, and is not described herein again.

In other embodiments of this method, the feature points may also include at least two points of the face contour, and as long as the two points are easy to identify and have a specific geometric position relationship with the forehead temperature measurement point, the second coordinate of the forehead temperature measurement point may also be determined in a manner similar to the above-described manner using the coordinates of both eyes, which is not described herein again.

The above method may also be used to measure ear temperature, in which case the point of measurement is the ear canal; the feature points include at least two points of the ear contour. The second coordinate of the ear canal temperature measurement point can be determined in a manner similar to the above-mentioned coordinate through both eyes, which is not described herein again.

It should be noted that, in the present invention, the face of the human face may be the front of the whole human face or the front of the partial human face including the feature points and the temperature measurement points, or may be the side of the human face or the side of the partial human face including the feature points and the temperature measurement points, and when the face of the human face or the front of the partial human face including the feature points and the temperature measurement points, the face of the human face may be used to measure the forehead temperature. When the human face side face or the partial human face side face comprising the characteristic points and the temperature measuring points is used, the imaging unit mainly shoots the outline of the human ear and prepares for the temperature measuring unit to measure the ear temperature.

As described in detail below with respect to the second mode, the method for determining the second coordinate of the second mode includes:

step BS41, identifying a temperature measuring point image of the face image based on the characteristics of the temperature measuring points, wherein the characteristics of the temperature measuring points comprise: at least one of shape and image characteristics;

and step BS42, determining a second coordinate of the temperature measuring point image in the reference coordinate system based on the identified temperature measuring point image.

In one example, regarding step BS41, based on the inherent features of the temperature measurement point itself, such as shape, image features, etc., the image of the temperature measurement point is directly located and identified in the acquired face;

in step BS42, after the image of the temperature measurement point is identified, the coordinates of the temperature measurement point in the reference system are directly calculated.

In a specific application scene, a two-dimensional or three-dimensional image of a human face is acquired in real time through an imaging unit, and after an integral image of the human face is acquired, the position of a temperature measuring point (such as a forehead) is directly positioned based on an intelligent recognition algorithm, so that a second coordinate of the temperature measuring point in a reference coordinate system is determined.

The following scheme can be adopted for the direct identification of the temperature measuring points:

storing feature information such as contour, shape and image related to a temperature measuring point (forehead) in a human face in a mobile terminal device, acquiring a large number of human face images, marking forehead area image area positions in the human face images, and forming training data samples of an intelligent image algorithm (such as OPENCV) so that when the images acquired by an imaging unit contain the human face, the positions of the forehead area can be automatically identified and positioned through the intelligent image algorithm, and then the positions of the forehead temperature measuring point can be directly determined.

In the above embodiment, the two-dimensional image of the face may be obtained by using a monocular optical camera or a two-sided optical camera, and the three-dimensional image of the face may be obtained by using a TOF sensor or a 3D structured light sensor as an imaging unit.

The principle of acquiring the three-dimensional image of the face in real time through the TOF sensor is as follows: the laser dot matrix is continuously transmitted to the face of the person through the TOF camera, the time of return of the received reflected light is recorded, the depth information of the face of the person is calculated based on the time of return of the reflected light, and the three-dimensional image of the detected object is obtained based on the TOF three-dimensional reconstruction algorithm.

Specifically, the TOF sensor includes a laser emission source and a CCD light receiver, and its basic principle is that the laser emission source emits a plurality of beams having a certain viewing angleAnd laser, wherein the duration of each laser beam is dt (from t1 to t2), each pixel of the CCD controls the period of the charge holding element of each pixel to collect the reflected light intensity by utilizing two synchronous trigger switches S1(t1 to t2) and S2(t2 to t2+ dt), and responses C1 and C2 are obtained. The distance L of the object from each pixel is 0.5 c dt c2/(c1+ c2), where c is the speed of light (this formula can remove the effect of the difference in the reflection characteristics of the reflectors on the distance measurement). The three-dimensional image of the face can be established through the existing TOF three-dimensional reconstruction algorithm based on the distance information of different laser points, and the forehead central area can be accurately identified according to the overall three-dimensional structure of the face based on the established three-dimensional image. Further, the second coordinate (X) of the temperature measuring point in the reference coordinate system can be easily calculated based on the position of the forehead temperature measuring point_t，Y_t). It should be noted that, performing face recognition by using a TOF sensor is an existing mature technology, and is easy to be implemented by those skilled in the art, and is not described herein again.

The principle of acquiring the three-dimensional image of the face in real time through the 3D structure optical camera is as follows: the continuous laser speckles (continuous non-pulse type surface light source) with structural characteristics are emitted to the face through the 3D structure optical camera, meanwhile, the images of the laser speckles are collected through the optical camera in the 3D structure optical camera, and the three-dimensional images of the detected object are obtained through a 3D structure optical three-dimensional reconstruction algorithm based on the images of the laser speckles. The basic principle of establishing a three-dimensional image by using a 3D structure optical camera is as follows: sending laser speckles to the face of the human face, and forming random diffraction spots after the laser irradiates a rough object or penetrates through ground glass. These speckles are highly random and will vary in pattern with distance, i.e., the speckle pattern will be different at any two locations in space. Therefore, as long as the structured light is applied to the space, the whole space is marked, and for the face in the space, the position of the face can be obtained only by identifying the speckle pattern on the face.

In the scheme, the spatial distribution of the speckle pattern needs to be calibrated as a light source in advance. E.g. by taking a reference plane at intervals of e.g. 10cm, and recording the speckle pattern on the reference plane at a recognizable distanceWithin range, multiple different speckle patterns may be recorded. When the measurement is carried out, a speckle image of a face is shot, the image and a plurality of prestored reference images are subjected to cross-correlation operation in sequence, and then a plurality of correlation degree images can be obtained, and the position of the face in the space can display a peak value on the correlation degree images. And performing superposition and interpolation operation on the peak values to obtain a three-dimensional image of the whole face. Then, the forehead central area can be accurately identified according to the integral three-dimensional structure of the face, and the second coordinate (X) of the temperature measuring point in the reference coordinate system can be easily calculated based on the position of the forehead temperature measuring point_t，Y_t). It should be noted that, performing face recognition based on a 3D structured light camera is a mature technology in the prior art, and is easy to be implemented by those skilled in the art, and is not described herein again.

Regarding step S5: judging whether the second coordinate and the first coordinate are overlapped or basically overlapped in the first direction; and judging whether the distance between the mobile terminal and the temperature measuring point along the first direction is within a preset range.

The method for judging whether the second coordinate is overlapped or basically overlapped with the first coordinate in the first direction comprises the following steps: based on the first coordinate (X)₀，Y₀) And a second coordinate (X)_t，Y_t) The X value and the Y value in the reference coordinate system can judge whether the first coordinate and the second coordinate are overlapped or basically overlapped in the reference coordinate system, and further judge whether the temperature measuring point (the center of the forehead) is aligned with the temperature detecting unit or not.

Judging whether the distance between the temperature measuring point and the temperature detecting unit is within a preset range or not, comprising the following steps:

s51: acquiring the distance between the temperature measurement point of the face and the mobile terminal in real time;

s52: comparing the measured distance with the preset range;

s53: and judging whether the measured distance is within the preset range.

The temperature detection unit adopts an existing infrared temperature sensor, the infrared temperature sensor comprises an infrared thermopile for receiving heat radiation of a face, and therefore whether the distance between a temperature measuring point and the temperature detection unit is within a preset range or not is judged, namely whether the distance is within the depth of field range of the infrared temperature sensor is judged, and optionally, the depth of field range is 0 cm-50 cm.

Regarding step S51, the manner of obtaining the distance between the temperature measurement point of the face and the mobile terminal in real time may be:

the first mode is as follows:

and estimating the distance between the temperature detection unit and the temperature measurement point along the first direction according to the image of the characteristic point. Specifically, when the imaging unit adopts an optical monocular camera, the distance between the face of the person and the imaging unit can be estimated by adopting the existing triangulation distance measurement method based on the lens focal length in the lens parameters of the optical monocular camera, and then the distance between the temperature detection unit and the temperature measurement point can be estimated.

When the imaging unit adopts an optical binocular camera, the distance measurement based on the optical binocular camera is generally used for short-distance high-precision measurement at present, the basic principle is that two images of a human face are simultaneously acquired through two cameras in the optical binocular camera, the distance between the human face and the human face is calculated based on the phase difference (parallax) of the human face in the two images, and then whether the distance between the mobile terminal and a temperature measuring point along a first direction is within a preset range is judged. It should be noted that the binocular camera ranging is a mature technology in the prior art, and those skilled in the art can easily implement the binocular camera ranging without further description.

The second mode is as follows:

and detecting the distance between the temperature measuring point and the mobile terminal equipment by using a preset distance sensor in the mobile terminal equipment.

Specifically, by integrating a distance sensor for assisting ranging on the mobile terminal for ranging, the distance sensor may be selected from a laser radar, a TOF sensor, an infrared distance sensor, or the like, and the distance sensor also performs ranging in the first direction. After the second coordinate of the forehead area on the face is identified to be basically coincident with the first coordinate of the temperature detection unit, the distance between the forehead area and the temperature measurement point (or the distance between the forehead area and the temperature measurement point in real time) is measured through a preset distance sensor, and then whether the distance between the temperature detection unit and the temperature measurement point along the first direction is within a preset range or not can be accurately judged. It should be noted that when the imaging unit is an optical monocular camera, it is preferable to use a distance sensor to perform auxiliary ranging, so as to achieve accurate ranging.

The third mode is as follows:

and when the image acquired by the imaging unit is a three-dimensional image, estimating the distance between the temperature detection unit and the temperature measurement point based on the acquired depth information of the face.

Specifically, when the image acquired by the imaging unit is a three-dimensional image, for example, when the imaging unit employs a TOF sensor, since depth information between the faces of the human faces has been acquired during the process of identifying the human faces and creating the three-dimensional image, the distance to the temperature measurement point can be directly estimated, that is, the distance between the temperature detection unit and the temperature measurement point can be estimated based on the time when the reflected light at the temperature measurement point returns in the three-dimensional image.

Alternatively, when the imaging unit is a 3D structure sensor, depth information of the face has already been acquired when a three-dimensional image of the face is created, and thus the distance from the temperature measurement point can be directly estimated. The distance between the imaging unit and the temperature measuring point is calculated based on the image information of the laser speckles at the temperature measuring point in the three-dimensional image, and then the distance between the temperature detecting unit and the temperature measuring point along the first direction is estimated. Regarding the ranging principle of the TOF sensor and the 3D structured light sensor, reference is made to the three-dimensional imaging principle of the TOF sensor and the 3D structured light sensor, which is not described herein again.

With respect to S52 and S53, comparing the calculated distance in the first direction between the temperature detection unit and the temperature measurement point with a preset range (i.e., a depth of field range of the temperature detection unit); and further judging whether the measured distance is within the preset range. For example, the depth of field range of the temperature detection unit in this embodiment is 0 to 50cm, when the measured or calculated distance is less than 50cm, it means that the temperature measurement point of the face or forehead is within the preset depth of field range (effective temperature measurement distance), and when the distance is greater than 50cm, it is determined that the depth of field range is exceeded.

Regarding step S6: if yes, outputting the temperature signal detected by the temperature detection unit.

Specifically, the screen of the mobile terminal device may be used to display related information and output the temperature signal monitored by the temperature detection unit.

Before outputting the temperature signal, the method further comprises the following steps:

judging whether the temperature signal is an effective temperature signal; when the second coordinate is coincident or basically coincident with the first coordinate and the distance between the mobile terminal and the temperature measuring point along the first direction is within a preset range, the detected temperature signal is an effective temperature signal; otherwise, it is an invalid temperature signal.

Specifically, when the temperature measuring point is accurately identified, the temperature measuring point and the temperature detecting unit are aligned, and the temperature measuring point is within the depth of field range of the temperature detecting unit, the detection value of the temperature measuring point (forehead) can be recorded as an effective temperature value in the mobile terminal device and displayed to the user, so that the accurate body temperature measurement is realized. If any one of the temperature measuring point is not accurately identified, the temperature measuring point is not aligned (the temperature measuring point and the temperature detecting unit are not basically overlapped in the first direction), and the distance between the temperature detecting unit and the temperature measuring point is not within the preset range, the measured temperature is an invalid temperature value, and the measured temperature value can not be measured or displayed.

Example two

Fig. 2 is a schematic structural diagram of a mobile terminal device according to a first embodiment of the present invention; fig. 3 is a flowchart of functional units in a mobile terminal device according to a second embodiment of the present invention.

Referring to fig. 2, a mobile terminal device 1 includes:

the imaging unit 2 is used for acquiring an image of a face of a person along a first direction;

a temperature detection unit 3 for detecting a temperature of the face of a person in a first direction;

the storage unit 4 stores: the related information of the reference coordinate system comprises: an X-axis, a Y-axis; the first direction is perpendicular to a plane in which the X axis and the Y axis lie; the temperature detection unit 3 is located at a first coordinate in a reference coordinate system;

the processing unit 5 is used for determining the position of the image of the temperature measuring point of the face in the reference coordinate system as a second coordinate;

the judging unit 6 is used for judging whether the second coordinate is overlapped or basically overlapped with the first coordinate; judging whether the distance between the mobile terminal 1 and the temperature measuring point 8 along the first direction is within a preset range;

and a display unit 7 for displaying the temperature value detected by the temperature inspection unit 3 when the judgment result of the judgment unit 6 is yes.

In this embodiment, the imaging unit 2 includes at least one of an optical monocular camera, an optical binocular camera, a TOF sensor, and a 3D structured light sensor. The face images acquired by the optical monocular camera and the optical binocular camera in real time are two-dimensional images, and the face images acquired by the TOF sensor and the 3D structured light sensor are three-dimensional images.

In this embodiment, the temperature detection unit 3 is an existing infrared temperature sensor, and the infrared temperature sensor includes an infrared thermopile for receiving thermal radiation from a face of a person. The preset range is a depth of field (effective thermometric distance) of the temperature detection unit 3, and optionally, the depth of field ranges from 0cm to 50 cm. It should be noted that the depth of field of the temperature detection unit 3 may be selected from temperature sensors (thermopile chips) with different depth of field ranges according to actual design requirements, and details are not described here.

Referring to fig. 3, in the present embodiment, the processing unit 5 includes:

the first determining subunit 501 is configured to determine an image of a feature point of the face according to the image of the face acquired by the imaging unit 2, and the storage unit 4 further stores a specific geometric position relationship between the feature point and the temperature measurement point 8;

a second determining subunit 502, configured to determine, according to the image of the feature point determined by the first determining unit, coordinates of the image of the feature point in the reference coordinate system;

the third determining subunit 503 is configured to obtain a geometric position relationship, and calculate a second coordinate of the temperature measurement point image according to the coordinate of the image of the feature point in the reference coordinate system and the geometric position relationship.

The judgment unit 6 includes:

a first sub-determining unit 601, configured to determine whether the second coordinate and the first coordinate coincide or substantially coincide in a reference coordinate system;

a second sub-determination unit 602, configured to determine whether a distance between the mobile terminal device 1 and the temperature measurement point 8 along the first direction is within a preset range;

an output unit 603 outputting a determination signal, wherein the determination signal is an effective temperature signal when the determination results of the first sub-determination unit and the second sub-determination unit are both yes; and when the judgment results of the first sub-judgment unit and the second sub-judgment unit are both negative, judging that the signal is an invalid temperature signal.

Wherein, the temperature measuring point 8 is the central area of the forehead, the characteristic points are two eyes, and each eye is a characteristic point; in other embodiments, the feature points may be two ears, each ear being a feature point, or the feature points may include at least two points of the face contour.

In another embodiment, the temperature measurement point is the ear canal; the feature points include at least two points of the ear contour.

Optionally, the method further comprises: a temperature measurement point identification unit 504, configured to identify a temperature measurement point 8 image of the face image according to the features of the temperature measurement point 8, and determine a second coordinate of the temperature measurement point 8 image in the reference coordinate system based on the identified temperature measurement point 8 image; the characteristics of the temperature measuring point 8 include: at least one of shape and image characteristics.

In the present embodiment, the reference coordinate system is the image coordinate system of the imaging unit 2. The reference coordinate system may also be the camera coordinate system or the pixel coordinate system of the imaging unit 2 in other embodiments.

In the present embodiment, the first coordinates are determined based on the position of the temperature detection unit 3 in the mobile terminal device 1 and stored in the storage unit 4 in advance; in other embodiments, the first coordinates are determined based on the physical relative position of the temperature detection unit 3 and the imaging unit 2 and stored in the storage unit 4 in advance.

In this embodiment, the device further includes a distance obtaining unit 9, which can estimate the distance between the temperature detection unit and the temperature measurement point along the first direction according to the image coordinate of the feature point according to the type of the sensor selected by the imaging unit; or, when a distance sensor is preset in the mobile terminal device, the distance between the temperature measuring point 8 and the mobile terminal device 1 is directly detected by using the distance sensor; alternatively, when the image acquired by the imaging unit 2 is a three-dimensional image, the distance between the temperature detection unit 3 and the temperature measurement point 8 is estimated based on the depth information of the acquired face. Alternatively, the distance sensor 10 may be an infrared distance sensor, a TOF sensor, a laser radar, or the like, which is preset on the top of the mobile device.

Further, when the optical monocular camera is selected, an infrared distance sensor is preferably configured to measure the distance between the temperature measuring points 8, and when the optical binocular camera is selected, the distance sensor is not configured, and the existing binocular distance measuring algorithm is directly utilized to measure the distance between the temperature measuring points 8; when a TOF sensor or a 3D structured light sensor is used as the imaging unit, the distance between the mobile terminal device 1 and the temperature measurement point 8 can be directly estimated without configuring a distance sensor because the acquired three-dimensional image of the face includes depth information. Reference is made to the first embodiment of the binocular ranging, the TOF sensor ranging and the 3D structured light sensor ranging principle, which is not described herein again.

In a specific application scenario, the mobile terminal device 1 may be an existing mobile terminal device, such as a mobile phone, a tablet computer, etc., the temperature detection unit 3 is integrated into the existing mobile terminal device in terms of hardware, then an existing optical camera, a binocular camera, or a TOF sensor, a 3D structured light camera, etc. in the mobile terminal device are used as the imaging unit 2, a memory chip (RAM \ ROM) in the mobile terminal device is used as the memory unit 4, and a screen of the mobile terminal device is used as the display unit 7 to display information such as a face image and a detected temperature value, etc., and in terms of software, functional modules such as a processing unit 5, a judgment unit 6, etc. are integrated into a processing chip system (CPU, GPU) of the mobile terminal device, and existing or developed intelligent face recognition, a judgment unit 6, etc. are integrated into a system of the mobile terminal device, And realizing image recognition of the face by using related algorithms such as three-dimensional reconstruction and the like, thereby realizing the scheme of the invention. According to the scheme of the present invention, a separate temperature detection terminal device may also be developed, and the imaging unit 2, the temperature detection unit 3, the display unit 5, the recognition processing unit 4, the image recognition algorithm, and the like are integrated to implement the scheme of the present invention.

Fig. 4 is a reference diagram of a mobile terminal device according to a second embodiment of the present invention.

Referring to fig. 4, the screen of the mobile terminal device 1 is used to display related information, which indicates the distance to move the object to be measured, so that the relative distance between the mobile terminal device 1 and the object to be measured is within the focal length of the imaging unit 2 or between the depths of field of the temperature detection unit 3, for example, the position of the mobile terminal device 1 is adjusted to be close to the human face, or the position of the human face is adjusted to be close to the mobile terminal device 1. If the person is close to the mobile terminal device 1, when the relative distance between the person and the mobile terminal device 1 is too close, the reminding information is displayed on the display screen of the mobile terminal device 1 to remind the person to be far away from the mobile terminal device 1, or when the relative distance between the person and the mobile terminal device 1 is too far, the reminding information is displayed on the display screen of the mobile terminal device 1 to remind the person to be close to the mobile terminal device 1. When the face is recognized, the second coordinate of the temperature measuring point 8 is calculated, the temperature measuring point 8 is accurately recognized, the temperature measuring point 8 and the temperature detecting unit 3 are aligned, and when the temperature measuring point 8 is within the depth of field range of the temperature detecting unit 3, the detection value of the temperature measuring point 8 (forehead) can be recorded as an effective temperature value in the mobile terminal device 1 and displayed to a user, so that the accurate body temperature measurement is realized. If the temperature measuring point is not accurately identified, the temperature is not aligned (the second coordinate of the temperature measuring point 8 and the first coordinate of the temperature detecting unit 3 are not overlapped or basically overlapped in the reference coordinate system), and the distance between the temperature detecting unit 3 and the temperature measuring point 8 is not within the preset range, the measured temperature is an invalid temperature value, and the measured temperature value is not measured or displayed.

EXAMPLE III

A non-transitory computer-readable storage medium according to an embodiment of the present invention stores computer instructions for causing a computer to execute the body temperature measurement method of a mobile terminal device according to the first embodiment.

A computer-readable storage medium according to an embodiment of the present invention has non-transitory computer-readable instructions stored thereon. The non-transitory computer readable instructions, when executed by a processor, perform all or a portion of the steps of a method of embodiments of the invention as previously described.

The computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROMs and DVDs), magneto-optical storage media (e.g., MOs), magnetic storage media (e.g., magnetic tapes or removable disks), media with built-in rewritable non-volatile memory (e.g., memory cards), and media with built-in ROMs (e.g., ROM cartridges).

In summary, the present invention collects a face image in real time through an imaging unit, and meanwhile, takes a coordinate system of the imaging unit as a reference coordinate system, sets a first coordinate of a temperature detection unit in the reference coordinate system, identifies a second coordinate of a temperature measurement point in the reference coordinate system based on the collected face image, and determines whether the second coordinate and the first coordinate coincide or substantially coincide in the reference coordinate system, and when the second coordinate and the first coordinate coincide or substantially coincide, a detection value of the temperature detection unit is an effective temperature value, thereby realizing accurate measurement of the temperature measurement point of the face on the mobile terminal device, and being capable of realizing convenient and easy-to-use accurate body temperature detection of the face.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.