阅读说明：本技术 信息检测装置、视频投影装置、信息检测方法以及视频投影方法 (Information detection device, video projection device, information detection method, and video projection method ) 是由 马见新友辉 于 2019-02-28 设计创作，主要内容包括：本发明的目的是提供一种用于检测关于视线方向、瞳孔位置等的信息的新技术。本技术提供了一种信息检测装置,包括：照射单元,所述照射单元用光照射眼球；扫描镜,所述扫描镜扫描来自所述眼球的反射光；以及检测单元,所述检测单元检测由所述扫描镜扫描的反射光。本技术还提供了一种图像投影装置,其包括上述组成元件。此外,本技术提供了一种信息检测方法,包括：用光照射眼球的照射步骤；使用扫描镜扫描来自眼球的反射光的扫描步骤；以及用于检测由扫描镜扫描的反射光的检测步骤。本技术还提供了一种包括所述步骤的图像投影方法。(An object of the present invention is to provide a new technique for detecting information about a sight line direction, a pupil position, and the like. The present technology provides an information detection apparatus, including: an irradiation unit that irradiates an eyeball with light; a scanning mirror that scans reflected light from the eyeball; and a detection unit that detects reflected light scanned by the scanning mirror. The present technology also provides an image projection apparatus including the above constituent element. In addition, the present technology provides an information detection method, including: an irradiation step of irradiating an eyeball with light; a scanning step of scanning reflected light from an eyeball using a scanning mirror; and a detection step for detecting the reflected light scanned by the scanning mirror. The present technology also provides an image projection method including the steps.)

1. An information detection apparatus comprising:

an irradiation unit that irradiates an eyeball with light;

a scanning mirror that scans reflected light from the eyeball; and

a detection unit that detects reflected light scanned by the scanning mirror.

2. The information detection apparatus according to claim 1, further comprising:

a controller that performs estimation processing of a pupil position or a gaze direction of the eyeball based on the reflected light detected by the detection unit.

3. The information detecting apparatus according to claim 2, wherein

The controller estimates a pupil position or a gaze direction of the eyeball based on a scanning oscillation angle of the scanning mirror and the reflected light detected by the detection unit.

4. The information detecting apparatus according to claim 2, wherein

The controller estimates a pupil position or a gaze direction of the eyeball based on a scanning oscillation angle of the scanning mirror and an intensity of the reflected light detected by the detection unit.

5. The information detecting apparatus according to claim 1, wherein

The irradiation unit is configured to irradiate the eyeball with the light without passing through the scanning mirror.

6. The information detecting apparatus according to claim 1, wherein

The irradiation unit is configured to irradiate the eyeball with the light via the scanning mirror.

7. The information detecting apparatus according to claim 1, wherein

The light for illuminating the eyeball is non-visible light.

8. The information detecting apparatus according to claim 1, wherein

The light for illuminating the eyeball is infrared light.

9. The information detecting apparatus according to claim 1, wherein

The light for illuminating the eyeball is a beam-like light.

10. A video projection apparatus comprising:

an irradiation unit that irradiates an eyeball with light;

a scanning mirror that scans reflected light from the eyeball;

a detection unit that detects reflected light scanned by the scanning mirror;

a controller that performs estimation processing on a pupil position of the eyeball based on the reflected light detected by the detection unit; and

a video display light irradiation unit that irradiates the pupil position estimated by the controller with video display light so that the video display light passes through the pupil position.

11. The video projection arrangement of claim 10 wherein

The video display light is concentrated near the pupil, and the retina is illuminated with the video display light.

12. The video projection arrangement of claim 10 wherein

Illuminating an eye with the video display light via the scanning mirror.

13. The video projection arrangement of claim 10 wherein

The video projection device is a glasses display.

14. An information detection method, comprising:

irradiating an eyeball with light;

scanning reflected light from the eyeball through a scanning mirror; and

reflected light scanned by the scan mirror is detected.

15. A video projection method, comprising:

irradiating an eyeball with light;

scanning reflected light from the eyeball through a scanning mirror;

detecting reflected light scanned by the scanning mirror;

estimating a pupil position of the eyeball based on the reflected light detected in the detecting step; and

illuminating the pupil position estimated in the estimating step with video display light such that the video display light passes through the pupil position.

Technical Field

The present technology relates to an information detection apparatus, a video projection apparatus, an information detection method, and a video projection method. More specifically, the present technology relates to an information detection device, a video projection device, an information detection method, and a video projection method capable of detecting a pupil position.

Background

In recent years, a technique of displaying a video superimposed on an external scene such as a real scene has been attracting attention. This technique is also known as Augmented Reality (AR) technique. One example of a product using this technology is a head mounted display. The head mounted display is used by being worn on the head of a user. In an image display method using a head mounted display, for example, light from the head mounted display illuminates the user's eyes in addition to light from the outside, whereby video is superimposed on an image of the outside for display.

In order to present a video to a user through a head mounted display, the user's gaze direction or pupil position may be grasped. Regarding a technique for grasping a gaze direction or pupil position of a user, for example, patent document 1 below describes a scanning display device characterized by incorporating a gaze detection apparatus that detects the gaze direction of the user while sharing an optical system of the scanning display device. In addition, a pupil detection apparatus described in the following patent document 2 includes: a detection unit that detects, as a reflected light beam intensity signal, an intensity of a light beam reflected on an eyeball surface among light beams incident on the eyeball surface; and a processing unit that acquires a pupil position based on a change in intensity of the reflected light beam indicated by the intensity signal output from the detection unit. In addition, a focal length control apparatus for a camera described in the following patent document 3 is characterized by including in the camera: a line-of-sight detecting means for detecting which part of the viewfinder the photographer is looking at based on a line-of-sight direction of the photographer; means for confirming a line-of-sight direction of the photographer in a viewfinder field of view (viewfinder field) detected by the line-of-sight detecting means, and for setting a focal length value of the camera based on a result of the confirmation.

CITATION LIST

Patent document

Patent document 1: japanese patent application publication No.2003-029198

Patent document 2: japanese patent application publication No.2006-

Patent document 3: japanese patent application laid-open No. HEI 05-317260

Disclosure of Invention

Technical problem

The head mounted display is used by being worn on the head of a user, and therefore further miniaturization is desired. It is also desirable to further reduce the power consumption of head mounted displays.

It is an object of the present technology to provide a new technology for detecting information such as a gaze direction or a pupil position. In particular, it is an object of the present technique to reduce the size and/or reduce the power consumption of the apparatus for detecting such information.

Technical scheme for problems

The present technology provides an information detection apparatus, including: an irradiation unit that irradiates an eyeball with light; a scanning mirror that scans reflected light from the eyeball; and a detection unit that detects reflected light scanned by the scanning mirror.

According to one embodiment of the present technology, the information detection apparatus may further include a controller that performs estimation processing of a pupil position or a gaze direction of the eyeball based on the reflected light detected by the detection unit.

According to an embodiment of the present technology, the controller may estimate a pupil position or a gaze direction of the eyeball based on a scanning oscillation angle of the scanning mirror and the reflected light detected by the detection unit.

According to one embodiment of the present technology, the controller may estimate a pupil position or a gaze direction of the eyeball based on a scanning oscillation angle of the scanning mirror and an intensity of the reflected light detected by the detection unit.

According to an embodiment of the present technology, the irradiation unit may be configured to irradiate the eyeball with the light without passing through the scanning mirror.

According to an embodiment of the present technology, the irradiation unit may be configured to irradiate the eyeball with the light via the scanning mirror.

According to one embodiment of the present technology, the light used to illuminate the eyeball may be non-visible light.

According to one embodiment of the present technology, the light used to illuminate the eye may be infrared light.

According to one embodiment of the present technology, the light for illuminating the eyeball may be a beam-like light.

Furthermore, the present technology provides a video projection apparatus comprising: an irradiation unit that irradiates an eyeball with light; a scanning mirror that scans reflected light from the eyeball; a detection unit that detects reflected light scanned by the scanning mirror; a controller that performs estimation processing on a pupil position of the eyeball based on the reflected light detected by the detection unit; and a video display light irradiation unit that irradiates the pupil position estimated by the controller with video display light so that the video display light passes through the pupil position.

In accordance with one embodiment of the present technique, the video display light may be concentrated near the pupil and the retina may be illuminated with the video display light.

According to one embodiment of the present technology, an eyeball may be illuminated with the video display light via the scanning mirror.

According to one embodiment of the present technology, the video projection device may be a glasses display.

In addition, the present technology provides an information detection method, including: irradiating an eyeball with light; scanning reflected light from the eyeball through a scanning mirror; and detecting reflected light scanned by the scan mirror.

In addition, the present technology provides a video projection method, comprising: irradiating an eyeball with light; scanning reflected light from the eyeball through a scanning mirror; detecting reflected light scanned by the scanning mirror; estimating a pupil position of the eyeball based on the reflected light detected in the detecting step; and illuminating the pupil position estimated in the estimating step with video display light such that the video display light passes through the pupil position.

Advantageous effects of the invention

In the present technique, reflected light from the eyeball is detected via a scanning mirror. Therefore, the size and power consumption of the detection unit can be reduced.

Note that the effects exerted by the present technology are not necessarily limited to the effects described herein, and any of the effects described herein may be exerted.

Drawings

FIG. 1 is a schematic diagram of an information detection apparatus according to the present technology.

Fig. 2 is a diagram showing an example of a scanning line of the scanning mirror.

Fig. 3 is a schematic diagram of an information detection apparatus according to the present technology.

FIG. 4 is a schematic diagram of a video projection device in accordance with the present technology.

FIG. 5 is a schematic diagram of a video projection device in accordance with the present technology.

FIG. 6 is a schematic diagram of a video projection device in accordance with the present technology.

Fig. 7 is a diagram showing a flow example of an information detection method according to the present technology.

Fig. 8 is a diagram showing a flow example of a video projection method according to the present technology.

Fig. 9 is a diagram illustrating an example of a head mounted display according to the present technology.

Fig. 10 is a diagram illustrating an example of a head mounted display according to the present technology.

Fig. 11 is a diagram showing a configuration example of an information detection apparatus according to the present technology.

Fig. 12 is a graph showing the relationship between light intensity and scan oscillation angle.

Fig. 13 is a diagram showing a configuration example of an information detection apparatus according to the present technology.

Fig. 14 is a graph showing the relationship between light intensity and scan oscillation angle.

Fig. 15 is a diagram showing a configuration example of an information detection apparatus according to the present technology.

Fig. 16 is a graph showing the relationship between light intensity and scan oscillation angle.

Fig. 17 is a diagram showing a configuration example of a video projection apparatus according to the present technology.

Fig. 18 is a diagram showing a configuration example of a video projection apparatus according to the present technology.

Fig. 19 is a diagram showing a configuration example of a video projection apparatus according to the present technology.

Detailed Description

Suitable examples for carrying out the present technique are described below. Note that the embodiments described below show representative embodiments of the present technology, and the scope of the present technology is not limited to these embodiments. Note that the description of the present technology is given in the following order.

1. First embodiment (information detecting apparatus)

(1) Description of the first embodiment

(2) First example of the first embodiment (information detecting apparatus)

(3) Second example of the first embodiment (information detecting apparatus)

2. Second embodiment (video projection apparatus)

(1) Description of the second embodiment

(2) First example of the second embodiment (video projection apparatus)

(3) Second example of the second embodiment (video projection apparatus)

(4) Third example of the second embodiment (video projection apparatus)

3. Third embodiment (information detecting method)

(1) Description of a third embodiment

(2) Example of the third embodiment (information detecting method)

4. Fourth embodiment (video projection method)

(1) Description of a fourth embodiment

(2) Example of the fourth embodiment (video projection method)

5. Constitution example of device

6. Examples of the invention

(1) Example 1 (simulation of the first example of the first embodiment)

(2) Example 2 (simulation of the second example of the first embodiment)

(3) Example 3 (simulation of the second example of the first embodiment)

(4) Example 4 (simulation of the first example of the second embodiment)

(5) Example 5 (simulation of the second example of the second embodiment)

(6) Example 6 (simulation of the third example of the second embodiment)

1. First embodiment (information detecting apparatus)

(1) Description of the first embodiment

An information detection device according to the present technology includes: an irradiation unit that irradiates an eyeball with light; a scanning mirror that scans reflected light from the eyeball; and a detection unit that detects reflected light scanned by the scanning mirror. That is, the reflected light from the eyeball reaches the detection unit via the scanning mirror. As a result, information about the eyeball, for example, information about the line of sight direction, the pupil position, and the like, can be detected based on the detected reflected light.

In addition, as described above, since the reflected light from the eyeball reaches the detection unit via the scanning mirror, the scanning mirror is driven, whereby the reflected light from various positions of the eyeball can be detected by one light detection element, for example. Therefore, the size of the detection unit can be reduced. In addition, since the number of light detection elements constituting the detection unit of the apparatus according to the present technology can be small, the power consumption of the apparatus is small.

In addition, as described above, since the reflected light from the eyeball reaches the detection unit via the scanning mirror, a noise component such as ambient light or stray light can be eliminated.

In the sight line detection apparatus described in patent document 1, for example, four infrared sensors are provided on a concave mirror in front of the eyes to detect infrared rays reflected on the eyeballs. Each of the plurality of infrared sensors detects infrared rays reflected on the eyeball. Then, the apparatus detects the position of the eyeball based on the plurality of detection values. However, in such a detection method, the detection accuracy is insufficient in some cases. In addition, since the apparatus includes a plurality of infrared sensors, power consumption thereof is large.

In the information detection apparatus according to the present technology, light reflected on the eyeball reaches the detection unit via the scanning mirror. That is, the information detection apparatus according to the present technology can detect reflected light from various positions of the eyeball. Therefore, the information detection device according to the present technology has high detection accuracy. Further, the information detection apparatus according to the present technology can detect reflected light from various positions of the eyeball by using, for example, one light detection element by driving the scanning mirror. Therefore, the power consumption of the information detection apparatus according to the present technology is small.

The pupil detection apparatus described in patent document 2 scans infrared rays and irradiates the eyes with the infrared rays. Patent document 2 discloses a pupil detection unit that receives infrared light reflected on an eye by a condensing lens. However, the device is susceptible to ambient or stray light when infrared light is received through the collector lens.

In the focal length control apparatus described in patent document 3, the infrared light reflected on the eyeball is also condensed by the light receiving lens. Thus, the device is also susceptible to ambient or stray light.

When performing light detection using the information detection apparatus according to the present technology, reflected light from the eyeball reaches the detection unit via the scanning mirror. That is, the angle of light reaching the detection unit is limited. Therefore, using the information detection apparatus according to the present technology can result in reducing the influence of noise components (e.g., ambient light or stray light).

(2) First example of the first embodiment (information detecting apparatus)

Hereinafter, an example of an information detection apparatus according to the present technology will be described with reference to fig. 1. FIG. 1 is a schematic diagram of an information detection apparatus according to the present technology. In the schematic view, the traveling direction of light for information detection by using the information detecting apparatus is indicated by an arrow.

As shown in fig. 1, the information detection apparatus 100 includes an irradiation unit 101, a light guide unit 102, a scanning mirror 103, a detection unit 104, and a controller 105.

The irradiation unit 101 irradiates the eyeball 150 with light 161. The light to be emitted 161 is used to detect information about the eyeball. The light to be emitted 161 is advantageously non-visible light, more advantageously infrared light. If the light to be emitted is non-visible light, in particular infrared light, the burden on the eyes in the detection of information is reduced. In addition, such light reduces the impact on the outside scene or video recognized by the user.

For example, if the information detection apparatus 100 is a glasses-shaped apparatus, for example, the irradiation unit 101 may be attached to any position of a frame or a lens of the glasses-shaped apparatus, or may be formed as a part of the frame or the lens.

The irradiation unit 101 may be disposed such that the light 161 passes through the light guiding unit 102 and reaches the eyeball 150. For example, the irradiation unit 101 may be disposed on a surface opposite to an eyeball-side surface of a rim or a lens of the eyeglass-shaped device. Alternatively, the irradiation unit 101 may be disposed such that the light 161 reaches the eyeball 150 without passing through the light guiding unit 102. In this case, for example, the irradiation unit 101 may be disposed on the rim of the glasses-like device or on the eyeball side of the lens. The irradiation unit 101 may be, for example, a planar light source.

Advantageously, the eye is illuminated with light 161 as parallel light. This makes more accurate information detection possible. For example, in a case where the irradiation unit 101 is configured such that the light 161 passes through the light guiding unit 102 and reaches the eyeball 150, the light guiding unit 102 may collimate the light 161. Alternatively, for example, in the case where the irradiation unit 101 is provided such that the light 161 reaches the eyeball 150 without passing through the light guiding unit 102, the irradiation unit 101 itself may be configured to output the light 161 as parallel light.

The eyeball 150 is irradiated with the light 161 emitted by the irradiation unit 101 only through the light guiding unit 102. That is, the eyeball 150 is irradiated with the light 161 without the scanning mirror 103. Therefore, the emitted light 161 is not affected by the reflectance of the scan mirror 103.

The light guide unit 102 may have a characteristic of transmitting the light 161 emitted from the irradiation unit 101. In addition, the light guiding unit 102 may have a characteristic of reflecting the reflected light 162 from the eyeball toward the scanning mirror 103. Examples of the light guide unit having such characteristics may include a holographic optical element.

In the case where the light emitted from the irradiation unit 101 is not parallel light, the light guide unit 102 may advantageously have an optical characteristic of collimating the light.

For example, in the case where the information detection apparatus 100 is a glasses-shaped apparatus, the light guide unit 102 may be a lens itself of the glasses-shaped apparatus, or may be configured as a part of the lens, for example.

The scan mirror 103 can be, for example, a micro-electro-mechanical system (MEMS) mirror. The scanning mirror 103 scans the reflected light 162 from the eyeball reflected by the light guiding unit 102. The scanning mirror 103 is driven to change the orientation of the surface of the scanning mirror 103, whereby reflected light 162 reflected at various positions of the eyeball 150 is reflected by the scanning mirror 103 and proceeds to the detection unit 104.

The scanning mirror 103 may be configured such that the reflected light 162 reflected by the light guiding unit 102 reaches the detecting unit 104. In order for the reflected light 162 reflected by the light guide unit 102 to reach the scanning mirror 103, an optical system may be appropriately provided on the path of the reflected light between the light guide unit 102 and the scanning mirror 103. In addition, in order to allow the reflected light 162 reflected by the scanning mirror 103 to reach the detection unit 104, an optical system may be appropriately provided on the path of the reflected light between the scanning mirror 103 and the detection unit 104.

For example, in the case where the information detection apparatus 100 is a glasses-shaped apparatus, the scanning mirror 103 may be attached to an arbitrary position of a temple (temple)111 of the glasses-shaped apparatus, or may be provided to be included in the temple.

In the present technology, scanning may include guiding reflected light reflected at various reflection positions of the eyeball to a predetermined light detection element by a scanning mirror. For example, in the present technology, scanning includes guiding reflected light reflected at each reflection position from a certain point of the eyeball to another point thereof to the light detection element by driving the scanning mirror. The reflection position of the reflected light guided to the light detection element may be moved in one dimension or may also be moved in two dimensions. Advantageously, the reflection position of the reflected light guided to the light detection element can be moved two-dimensionally.

This makes it possible to grasp information about the eyeball more accurately.

FIG. 2 shows an example of a scan line of the scan mirror 103. As shown in fig. 2, by driving the scanning mirror 103, the position of the reflected light scanned by the scanning mirror is moved from the position "a" to the position "b". Next, the position of the reflected light scanned by the scanning mirror is moved from position "c" to position "d". Similarly, the position of the reflected light scanned by the scan mirror moves from position "e" to position "f", from position "g" to position "h", from position "i" to position "j", and from position "k" to position "l". In this way, the reflection position of the reflected light scanned by the scanning mirror 103 can be moved two-dimensionally. The reflected light reflected on the eyeball (or pupil) 150 is detected on the scan line from the position "e" to the position "f" and on the scan line from the position "g" to the position "h". In response to a change in the position of the eyeball 150, the scanning line or the position on the scanning line that detects the reflected light reflected on the eyeball 150 can be changed.

The detection unit 104 detects the reflected light 162 reflected by the scan mirror 103. The detection unit 104 includes, for example, a photodiode. The kind of the detection unit 104 may be appropriately selected by those skilled in the art according to the kind of light to be detected. For example, if the reflected light 162 is infrared light, the detection unit 104 includes an infrared sensor.

For example, the detection unit 104 may detect the intensity of the reflected light 162. The intensity of the reflected light 162 differs depending on the position of the eyeball 150 that reflects the reflected light 162. For example, the intensity of reflected light reflected at the pupil portion is different from the intensity of reflected light reflected at the iris portion. In addition, the intensity of reflected light reflected on the surface of the eyeball (e.g., cornea) is stronger than the intensity of scattered light reflected on the fundus oculi (e.g., retina) of the eyeball. Accordingly, the pupil position and/or the line-of-sight direction can be estimated based on the orientation or the scanning oscillation angle of the surface of the scanning mirror 103 and the light intensity detected by the detection unit 104.

Further, in another embodiment, the detection unit 104 may generate an image based on the detected reflected light 162. For example, the detection unit 104 may include an image sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or a Charge Coupled Device (CCD), particularly an infrared image sensor, to generate an image based on the reflected light 162. For example, the controller 105 performs image processing on an image generated based on the reflected light 162, whereby information about the eyeball (e.g., pupil position or line of sight direction) can be obtained.

The detection unit 104 may be connected to a controller 105. The controller 105 may process information about the light detected by the detection unit 104, such as the intensity.

For example, in the case where the information detection apparatus 100 is a glasses-shaped apparatus, the detection unit 104 may be attached to an arbitrary position of a temple of the glasses-shaped apparatus, or may be provided to be included in the temple.

The controller 105 may obtain information about the eyeball (e.g., pupil position, line of sight direction, or rotation angle of the eyeball) based on the reflected light (e.g., intensity of the reflected light, etc.) detected by the detection unit 104. For example, the controller 105 may perform estimation processing of the pupil position or the gaze direction of the eyeball based on the reflected light.

In addition, a controller 105 may be connected to the scan mirror 103 to enable control of the scan mirror 103. For example, the controller 105 can drive the scan mirror 103 within a predetermined scan oscillation angle. The controller 105 can obtain information related to the scanning mirror 103, such as information regarding the orientation of the surface of the scanning mirror, the angle of scanning oscillation, and the like.

Advantageously, the controller 105 may obtain information about the eyeball based on the information about the scanning mirror 103 and the information about the reflected light detected by the detection unit 104. For example, the controller 105 may perform estimation processing on, for example, a pupil position, a line-of-sight direction, a rotation angle of an eyeball, or the like, based on the scanning oscillation angle of the scanning mirror 103 and the reflected light (in particular, the intensity of the reflected light) detected by the detection unit 104.

The controller 105 may include a processor, such as a Central Processing Unit (CPU), and memory, such as Random Access Memory (RAM) and/or Read Only Memory (ROM). The memory may store a program or the like for causing the apparatus to execute the information detection method or the video projection method according to the present technology. The processor may implement the functions of the controller 105.

The controller 105 may control the irradiation of light by the irradiation unit 101. For example, the controller 105 may turn on or off the irradiation of light by the irradiation unit 101, or may change the intensity or type of light to be emitted. Further, the controller 105 may control the driving of the scan mirror 103. For example, the controller 105 can change the scan oscillation angle of the scan mirror 103. Further, the controller 105 may control the detection by the detection unit 104. For example, the controller 105 may turn on or off the detection operation by the detection unit 104.

For example, when the information detection apparatus 100 is a glasses-shaped apparatus, the controller 105 may be provided to be included in a temple of the glasses-shaped apparatus. Alternatively, the controller 105 may be provided in a device different from the glasses-shaped device and may be connected to the glasses-shaped device by wire or wirelessly.

For example, if the surface of the scanning mirror 103 is oriented in the direction shown in the left side of fig. 1, the reflected light (including, for example, surface reflected light or cornea reflected light) passing through the position 171 on the eyeball 150 is reflected by the scanning mirror 103 and detected by the detection unit 104. If the orientation of the surface of the scanning mirror 103 is changed to the direction shown on the right side of fig. 1, the reflected light (e.g., scattered light reflected on the iris) passing through the position 172 on the eyeball 150 is reflected by the scanning mirror 103 and detected by the detection unit 104. The intensity of the reflected light passing through the position 171 and the intensity of the reflected light passing through the position 172 are different from each other. Thus, the reflection position on the eyeball 150 of the reflected light reaching the detection unit 104 differs depending on the orientation of the surface of the scanning mirror 103. Therefore, if the scanning mirror 103 is driven, information on reflected light reflected at various positions on the eyeball 150 can be obtained.

The obtained information related to the reflected light corresponds to, for example, the orientation of the surface of the scanning mirror 103 or the scanning oscillation angle. Therefore, information on the eyeball 150 can be obtained based on the obtained information on the light and the orientation of the surface of the scanning mirror 103 or the scanning oscillation angle.

In addition, unlike the case of the information detection device 300 described below in "(3) the second example (information detection device)" of the first embodiment, light from the irradiation unit 101 of the information detection device 100 is emitted onto the eyeball without a scanning mirror or a half mirror. Therefore, the utilization efficiency of light from the irradiation unit 101 of the information detection apparatus 100 can be higher than that of light from the irradiation unit 301 of the information detection apparatus 300. Further, the information detection apparatus 100 is not easily affected by stray light caused by the scanning mirror or the half mirror.

(3) Second example of the first embodiment (information detecting apparatus)

Hereinafter, another example of the information detection apparatus according to the present technology will be described with reference to fig. 3. Fig. 3 is a schematic diagram of an information detection apparatus according to the present technology. In the schematic view, the traveling direction of light for information detection by using the information detecting apparatus is indicated by an arrow.

As shown in fig. 3, the information detection apparatus 300 includes an irradiation unit 301, a light guide unit 302, a scanning mirror 303, a detection unit 304, and a controller 305. The information detection apparatus 300 further includes a half mirror 306. Among these constituent elements, the scanning mirror 303, the detection unit 304, and the controller 305 are the same as the scanning mirror 103, the detection unit 104, and the controller 105 described above in "(2) the first example (information detection apparatus) of the first embodiment". The information detection apparatus 100 is configured to irradiate the eyeball with light emitted from the irradiation unit 101 without the scanning mirror 103, and the information detection apparatus 300 irradiates the eyeball with light emitted from the irradiation unit 301 via the scanning mirror 303. Therefore, the irradiation unit 301 and the light guiding unit 302 are mainly described below.

The irradiation unit 301 is configured to irradiate the eyeball with light 361 via the scanning mirror 303. That is, the light 361 is scanned and emitted toward the eyeball.

Advantageously, the light 361 may be emitted as beam-like light (in particular, a laser beam) from the irradiation unit 301. More advantageously, the illumination unit 301 emits beam-like infrared light. The irradiation unit 301 may emit, for example, infrared laser light. This makes it possible to detect reflected light from the pupil section more clearly, for example. Therefore, the pupil position can be estimated more accurately.

The light 361 emitted by the irradiation unit 301 is reflected by the half mirror 306 to reach the scanning mirror 303, further reflected by the scanning mirror 303 to reach the light guiding unit 302, and further reflected by the light guiding unit 302 to reach the eyeball.

For example, in the case where the information detection apparatus 300 is a glasses-shaped apparatus, the irradiation unit 301 may be attached to any position of a temple of the glasses-shaped apparatus. Alternatively, the irradiation unit 301 may be provided to be included in the temple. In addition, the irradiation unit 301 may be connected to the controller 305 to be controllable by the controller 305.

The half mirror 306 may have a characteristic of reflecting light from the irradiation unit 301 and transmitting reflected light from the eyeball. The detection unit 304 detects reflected light 362 reflected by the scanning mirror 303 and transmitted through the half mirror 306.

The light guiding unit 302 may have a characteristic of reflecting the light 361 emitted from the irradiation unit 301 and reflecting the reflected light 362 from the eyeball. The light guide unit 302 reflects the reflected light 362 from the eyeball toward the scan mirror 303. Examples of the element having the above-described characteristics may include a holographic optical element.

The light guiding unit 302 may be, for example, a grating or a grating lens. In the case where the light guiding unit 302 is a grating, the light guiding unit 302 may selectively reflect only light having a predetermined wavelength. In the case where the light guiding unit 302 is a grating lens, the light guiding unit 302 may selectively reflect only light having a predetermined wavelength and concentrate the light toward the eyeball.

In addition, the information detection apparatus 300 may further include other optical elements, such as a collimator lens, between the scanning mirror 303 and the light guide unit 302.

The scan mirror 303 may be, for example, a MEMS mirror. As described above, the scanning mirror 303 scans the light 361 emitted by the irradiation unit 301, and also scans the reflected light 362 reflected on the eyeball. That is, in the information detection apparatus 300 in the present example, the reflected light from the eyeball is scanned by the scanning mirror 303 that scans the illumination light.

For example, if the surface of the scanning mirror 303 is oriented in the direction shown on the left side of fig. 3, the reflected light (e.g., surface reflected light or cornea reflected light) passing through the position 371 on the eyeball is reflected by the scanning mirror 303 and detected by the detector 304. If the orientation of the surface of scanning mirror 303 changes to the direction shown on the right side of FIG. 1, the reflected light that passes through location 372 on the eyeball (e.g., the scattered light reflected on the iris) is reflected by scanning mirror 303 and detected by detector 304. The intensity of the reflected light passing through the position 371 and the intensity of the reflected light passing through the position 372 are different from each other. Thus, the reflection position of the reflected light reaching the detection unit 304 on the eyeball differs depending on the orientation of the surface of the scanning mirror 303. Therefore, if the scanning mirror 303 is driven, information on reflected light reflected at various positions on the eyeball can be obtained.

The obtained information related to light corresponds to, for example, the orientation of the surface of the scanning mirror 303 or the scanning oscillation angle. Therefore, various information on the eyeball can be obtained based on the obtained information on the light and the orientation of the surface of the scanning mirror 303 or the scanning oscillation angle.

The information detection apparatus 300 includes a common optical system (e.g., a scanning mirror) on the path of light emitted toward the eyeball and reflected light. Therefore, the information detection apparatus 300 can be made smaller than the information detection apparatus 100 described above in "(2) the first example (information detection apparatus) of the first embodiment".

In addition, light from the irradiation unit 301 of the information detection apparatus 300 is emitted toward the eyeball via the half mirror 306 and the scanning mirror 303. Therefore, the light use efficiency may be lower than that of the information detection apparatus 100 described above in "(2) the first example (information detection apparatus)" of the first embodiment. However, if the light transmittance or reflectance of each of the half mirror 306 and the scanning mirror 303 is increased, the utilization efficiency of the light can be improved.

2. Second embodiment (video projection apparatus)

(1) Description of the second embodiment

The video projection apparatus according to the present technology includes, in addition to the constituent elements described above in "1. first embodiment (information detection apparatus)", a video display light irradiation unit that irradiates the pupil position estimated by the controller with video display light so that the video display light passes through the pupil position.

The video projection apparatus according to the present technology exerts the effects described above in "1. first embodiment (information detection apparatus)". Further, the video projection apparatus according to the present technology exhibits an effect that the video display light can be emitted toward an appropriate position.

(2) First example of the second embodiment (video projection apparatus)

Hereinafter, an example of a video projection apparatus according to the present technology is explained with reference to fig. 4. FIG. 4 is a schematic diagram of a video projection device in accordance with the present technology. In the schematic diagram, the traveling direction of light for information detection by the video projection apparatus and the direction of video display light are indicated by arrows of broken lines.

As shown in fig. 4, the video projection apparatus 400 includes an illumination unit 401, a light guide unit 402, a scanning mirror 403, a detection unit 404, and a controller 405. The video projection apparatus 400 further includes a video display light illumination unit 420. Among these constituent elements, the irradiation unit 401, the scanning mirror 403, the detection unit 404, and the controller 405 are the same as the irradiation unit 101, the scanning mirror 103, the detection unit 104, and the controller 105 described above in "(2) the first example (information detection apparatus) of the first embodiment" of part 1. Therefore, information (e.g., pupil position) as described in "(2) the first example (information detecting means) of the first embodiment" of the section 1 can be detected.

The video projection apparatus 400 is an apparatus in which the video display light irradiation unit 420 is added to the information detection apparatus 100 described above in "(2) the first example (information detection apparatus) of the first embodiment of section 1. Hereinafter, the light guiding unit 402 and the video display light irradiating unit 420 are mainly described.

The light guide unit 402 may have a characteristic of transmitting illumination light emitted from the illumination unit 401 and reflecting reflected light from the eyeball. In the case where the light emitted from the irradiation unit 401 is not parallel light, it is advantageous that the light guide unit 402 may have an optical characteristic of collimating the light.

Further, the light guiding unit 402 may have a characteristic of reflecting the video display light emitted by the video display light irradiation unit 420. Advantageously, the light guiding unit 402 reflects the video display light so that the video display light emitted from the video display light irradiation unit 420 is condensed near the pupil and emitted toward the retina. That is, the light guiding unit 402 may diffract the video display light so that the video display light travels straight through the pupil. This enables the video to be presented to the user by means of a so-called maxwellian view (maxwellian view). Thus, a clear video can be presented to the user.

In the present technology, the video display light may be condensed near (e.g., on) the pupil, or may be deviated from the pupil by several to several tens of mm (e.g., 1 to 20mm, particularly, 2 to 15mm) in the optical axis direction. A maxwell view can be achieved even if the focal point is not on the pupil as in the latter case. The shift of the focus in the optical axis direction makes it difficult for the user to lose the video even if the video shifts. More specifically, the video display light may be concentrated on the pupil, in the lens, or between the corneal surface and the pupil.

For example, in the case where the video projection apparatus 400 is a glasses-like apparatus, the light guiding unit 102 may be, for example, a lens itself of the glasses-like apparatus. Alternatively, the light guiding unit 102 may be constructed as a part of a lens.

The video display light irradiation unit 420 emits video display light toward the light guiding unit 402. The video display light to be emitted may be emitted radially. The video display light irradiation unit 420 may be controlled by the controller 405. For example, the controller 405 may control the video display light irradiation unit 420 to emit video display light so that the video display light is condensed at the pupil position estimated by the controller 405. This makes it possible to present the video to the user more reliably.

The video display light may be, for example, light that may be used to present video to a user through a Maxwell view. The video display light may be light emitted by a Light Emitting Diode (LED) or a Cathode Ray Tube (CRT), for example.

For example, in the case where the video projection apparatus 400 is a glasses-shaped apparatus, the video display light irradiation unit 420 may be attached to a temple portion of the glasses-shaped apparatus, for example. Alternatively, the video display light irradiation unit 420 may be included in the temple portion. A video is presented to the user through the video display light irradiation unit 420, whereby the video is superimposed on an external scene seen by the user through the lens of the glasses-like device.

(3) Second example of the second embodiment (video projection apparatus)

Another example of a video projection apparatus according to the present technology will be described with reference to fig. 5. FIG. 5 is a schematic diagram of a video projection device in accordance with the present technology. In this schematic diagram, the traveling direction of light for information detection according to the present technology and the traveling direction of video display light are indicated by arrows of broken lines.

As shown in fig. 5, the video projection apparatus 500 includes an illumination unit 501, a light guide unit 502, a scanning mirror 503, a detection unit 504, and a controller 505. The video projection apparatus 500 further includes a half mirror 506 and a video display light irradiation unit 520.

Among these constituent elements, the irradiation unit 501, the light guiding unit 502, the scanning mirror 503, the detecting unit 504, the controller 505, and the half mirror 506 correspond to the irradiation unit 301, the light guiding unit 302, the scanning mirror 303, the detecting unit 304, the controller 305, and the half mirror 306 described above in "(3) the second example (information detecting apparatus) of the first embodiment" of part 1. Therefore, information (e.g., pupil position) as described above in "(3) the second example (information detecting means) of the first embodiment" of the section 1 can be detected.

The video projection apparatus 500 may also be an apparatus in which the video display light irradiation unit 520 is added to the information detection apparatus 300 described above in section 1 "(3) the second example (information detection apparatus) of the first embodiment". Next, the light guiding unit 502 and the video display light irradiation unit 520 are mainly described.

The light guiding unit 502 may have a characteristic of reflecting the illumination light emitted from the irradiation unit 501 and reflecting the reflected light from the eyeball.

In addition, the light guiding unit 502 may have a characteristic of reflecting the video display light emitted by the video display light irradiation unit 520. Advantageously, the light guiding unit 502 reflects the video display light so that the video display light emitted from the video display light irradiation unit 520 is condensed near the pupil and emitted toward the retina. That is, the light guiding unit 502 may diffract the video display light such that the video display light travels straight through the pupil. This allows the video to be presented to the user through a so-called maxwell view. Thus, a clear video can be presented to the user.

The video display light irradiation unit 520 emits video display light toward the light guiding unit 402. The video display light to be emitted may be emitted radially. The video display light irradiation unit 520 may be controlled by the controller 505. For example, the controller 505 may control, for example, the video display light irradiation unit 520 to emit video display light so that the video display light is condensed at the pupil position estimated by the controller 505. This makes it possible to present the video to the user more reliably. The video display light may be, for example, light that may be used to present video to a user through a Maxwell view. The video display light may be light emitted by an LED or CRT, for example.

For example, in the case where the video projection apparatus 500 is a glasses-shaped apparatus, the video display light irradiation unit 520 may be attached to a temple portion of the glasses-shaped apparatus, for example. Alternatively, the video display light irradiation unit 520 may be included in the temple portion. A video is presented to the user through the video display light irradiation unit 520, whereby the video is superimposed on an external scene seen by the user through the lens of the glasses-like device.

(4) Third example of the second embodiment (video projection apparatus)

Another example of a video projection apparatus according to the present technology will be described with reference to fig. 6. FIG. 6 is a schematic diagram of a video projection device in accordance with the present technology. In this schematic diagram, the traveling direction of light for information detection by the video projection apparatus and the traveling direction of video display light are indicated by arrows of broken lines.

As shown in fig. 6, the video projection apparatus 600 includes an illumination unit 601, a light guide unit 602, a scanning mirror 603, a detection unit 604, and a controller 605. The video projection device 600 also includes a half mirror 606.

The irradiation unit 601 is configured to irradiate the eyeball 650 with illumination light 661 via the scanning mirror 603. That is, the illumination light 661 is scanned and emitted toward the eyeball.

Advantageously, the illumination light 661 can be emitted as a beam-like light from the illumination unit 601. More advantageously, the irradiation unit 601 emits a beam-like infrared light, in particular an infrared laser beam.

The illumination light 661 emitted by the irradiation unit 601 is reflected by the half mirror 606 to reach the scanning mirror 603, further reflected by the scanning mirror 603 to reach the light guiding unit 602, and further reflected by the light guiding unit 602 to reach the eyeball 650.

The irradiation unit 601 is configured to emit image display light 663 in addition to the illumination light 661. For example, the illumination light 661 and the video display light 663 may be output by the illumination unit 601 in a multiplexed state. Video is presented to the user via video display light 663.

The irradiation unit 601 may, for example, output laser light in which video display light and illumination light are multiplexed, particularly laser light in which red, green, and blue laser light and infrared laser light are multiplexed.

For example, in the case where the video projection apparatus 600 is a glasses-shaped apparatus, the irradiation unit 601 may be attached to any position of a temple of the glasses-shaped apparatus. Alternatively, the irradiation unit 601 may be provided to be included in the temple. The video is presented to the user through the illumination unit 601, whereby the video is superimposed on the external scene seen by the user through the lenses of the glasses-like device.

The light guiding unit 602 may have a characteristic of reflecting the illumination light 661 and the video display light 663 emitted from the illumination unit 601 and reflecting the reflected light 662 from the eyeball. Advantageously, the light guiding unit 602 may have the following characteristics: the illuminating light 661 is reflected to reach a position to be scanned on the eyeball, and the video display light 663 is reflected to be condensed at the pupil position to illuminate the retina (in particular, through the center of the pupil). Optical elements known in the art may be used as the light guiding unit having these characteristics. Examples of the optical element include a holographic optical element.

For example, as shown on the left side of fig. 6, when the illumination light 661 and the video display light 663 reach the position 673 of the light guiding unit 602, both the illumination light 661 and the video display light 663 are reflected to pass through the center 671 of the pupil. For example, as shown on the right side of fig. 6, when the illumination light 661 and the video display light 663 reach the position 674 in the light guiding unit 602, the illumination light 661 is reflected to be emitted toward the position 672 deviated from the pupil center, and the video display light 663 is reflected to pass through the center 671 of the pupil.

The scanning mirror 603 may be, for example, a MEMS mirror. As described above, the scanning mirror 603 scans the illumination light 661 and the video display light 663 (or a combination thereof) emitted by the illumination unit 601, and also scans the reflected light 662 reflected on the eyeball 650. That is, in the video projection apparatus 600 according to the present example, the reflected light from the eyeball and the video display light are also scanned by the scanning mirror that scans the illumination light.

When the eyeball is irradiated with illumination light, reflected light is generated. When scanning the reflected light, information about the eyeball can be obtained.

In addition, the video display light is scanned on the retina, thereby presenting a video to the user. Video display light is focused by the scan, for example, near the pupil, and emitted toward the retina. That is, the video display light travels straight through the pupil. This allows the video to be presented to the user through a so-called maxwell view. Thus, a clear video can be presented to the user.

For example, if the surface of the scanning mirror 603 is oriented in the direction shown on the left side of fig. 6, the reflected light (for example, surface reflected light or cornea reflected light) passing through the center 671 of the eyeball 650 is reflected by the light guiding unit 602 and the scanning mirror 603, passes through the half mirror 606, and is detected by the detector 604. If the orientation of the surface of the scanning mirror 603 is changed to the direction shown on the right side of fig. 6, the reflected light (including, for example, scattered light reflected on the iris) passing through the position 672 of the eyeball 650 is reflected by the light guiding unit 602 and the scanning mirror 603, transmitted through the half mirror 606, and detected by the detector 604. In this way, the position of the reflected light on the eyeball 650 that reaches the detection unit 604 differs depending on the orientation of the surface of the scanning mirror 603, and the intensity of the reflected light differs depending on the position of the reflected light on the eyeball. Therefore, if the scanning mirror 603 is driven, information on reflected light reflected at various positions on the eyeball 650 can be obtained.

The detection unit 604 detects reflected light 662 reflected by the scanning mirror 603 and transmitted through the half mirror 606.

Controller 605 may process information related to the reflected light detected by detector 604 (e.g., the intensity of the reflected light). For example, the controller 605 may obtain information about the eyeball (e.g., a rotation angle of the eyeball, a pupil position, and a line of sight direction) based on the information about the reflected light. Advantageously, the controller 605 may perform an estimation process of the pupil position of the eyeball based on the reflected light detected by the detection unit 604.

In addition, the controller 605 may be configured to control the scanning mirror 603. For example, the controller 605 can drive the scan mirror 603 within a predetermined scan oscillation angle. The controller 605 can obtain information about the scanning mirror 603, e.g., information about the orientation of the surface of the scanning mirror, the scanning oscillation angle, and the like.

Advantageously, the controller 605 may obtain information about the eyeball based on the information about the scanning mirror 603 and the information about the reflected light detected by the detection unit 604. For example, the controller 605 may estimate a pupil position, a line of sight direction, or a rotation angle of the eyeball based on the scanning oscillation angle of the scanning mirror 603 and the reflected light (e.g., the intensity of the reflected light) detected by the detection unit 604.

Further, the controller 605 may control the irradiation unit 601 to adjust the video display light to be emitted, based on the acquired information about the eyeball. In this way, the video may be displayed at an appropriate location in the user's field of view.

The controller 605 may include a processor such as a CPU and a memory such as RAM and/or ROM. The memory may store a program or the like for causing the apparatus to execute the information detection method or the video projection method according to the present technology. The processor may implement the functions of the controller 605.

The controller 605 may control illumination of the illumination light and/or the video display light of the illumination unit 601. For example, the controller 605 may turn on or off the illumination of the illumination light and/or the video display light of the illumination unit 601. Alternatively, the controller 605 may change the intensity or type of illumination light and/or video display light to be emitted. Further, the controller 605 may control the driving of the scan mirror 603. For example, the controller 605 can change the scan oscillation angle of the scan mirror 603. Further, the controller 605 may control the detection of the detection unit 604. For example, the controller 605 may turn on or off the detection operation by the detection unit 604.

The half mirror 606 may have a characteristic of reflecting the illumination light 661 and the video display light 663 (or a combination thereof) and transmitting the reflected light 662. This allows only the reflected light 662 to selectively reach the detection unit 604.

In the video projection apparatus 600, as described above, the illumination unit 601 is configured to be able to emit the illumination light and the video display light, and therefore, the apparatus can be downsized. Further, since the optical system (e.g., scanning mirror) on the path of the illumination light, the video display light, and the reflected light is shared, the apparatus can be further miniaturized.

3. Third embodiment (information detecting method)

(1) Description of a third embodiment

The present technology also provides an information detection method, including: irradiating an eyeball with light; scanning reflected light from an eyeball through a scanning mirror; and detecting reflected light scanned by the scanning mirror.

The information detection method according to the present technology can detect information of the eyeball as described above in section 1.

(2) Example of the third embodiment (information detecting method)

Hereinafter, an example of an information detection method according to the present technology will be described with reference to fig. 1 and 7. Fig. 7 is a diagram showing a flow example of an information detection method according to the present technology.

In step S101, the information detection apparatus 100 starts the information detection process according to the present technology.

In step S102, the controller 105 causes the irradiation unit 101 to irradiate the eyeball with illumination light. Advantageously, the emitted light is non-visible light, and more advantageously it is infrared light. For example, the illumination light may be emitted only when the pupil position needs to be detected. Alternatively, the illumination light may be emitted at predetermined time intervals.

In step S103, the controller 105 drives the scanning mirror 103 so that the scanning mirror 103 scans the reflected light generated by irradiating the illumination light on the eyeball in step S102. As a result of the scanning, the reflected light reflected at each of the various positions of the eyeball proceeds to the detection unit 104.

In step S104, the controller 105 causes the detection unit 104 to detect the reflected light scanned in step S103. As a result of the detection, information about the reflected light, for example, the intensity of the reflected light, can be obtained.

In step S105, the controller 105 obtains information about the eyeball based on the information about the reflected light obtained in step S104 and the information about the scanning in step S103.

In step S106, the information detection apparatus 100 ends the information detection processing according to the present technology.

The above-described processing may be performed by, for example, an information detection apparatus or a video projection apparatus according to the present technology. For more detailed operations of the components of these apparatuses in the above steps, reference is made to the above "1. first embodiment (information detection apparatus)" and "2. second embodiment (video projection apparatus)".

4. Fourth embodiment (video projection method)

(1) Description of a fourth embodiment

The present technology also provides a video projection method, including: irradiating an eyeball with light; scanning reflected light from an eyeball by using a scanning mirror; detecting reflected light scanned by the scanning mirror; estimating a pupil position of the eyeball based on the reflected light detected in the detecting step; and illuminating the pupil position estimated in the estimating step with video display light so that the video display light passes through the pupil position.

The video projection method according to the present technology can detect information of the eyeball as described above in section 1. Further, the video projection method according to the present technology can emit video display light toward an accurate position based on information of the detected eyeball.

(2) Example of the fourth embodiment (video projection method)

Hereinafter, an example of a video projection method according to the present technology is explained with reference to fig. 6 and 8. Fig. 8 is a diagram showing a flow example of a video projection method according to the present technology.

In step S201, the video projection apparatus 600 starts the video projection processing according to the present technology.

In step S202, the controller 605 causes the irradiation unit 601 to irradiate the eyeball with illumination light. Advantageously, the light to be emitted is non-visible light, more advantageously infrared light. For example, the illumination light may be emitted only when the pupil position needs to be detected. Alternatively, the illumination light may be emitted at predetermined time intervals.

In step S203, the controller 605 drives the scanning mirror 603 so that the scanning mirror 603 scans the reflected light generated by irradiating illumination light onto the eyeball in step S202. As a result of the scanning, the reflected light reflected at each of various positions of the eyeball proceeds to the detection unit 604.

In step S204, the controller 605 causes the detection unit 604 to detect the reflected light scanned in step S203. As a result of the detection, information about the reflected light, for example, the intensity of the reflected light, can be obtained.

In step S205, the controller 605 obtains information about the eyeball based on the information about the reflected light obtained in step S204 and the information about the scanning in step S203.

In step S206, the controller 605 causes the irradiation unit 601 to emit video display light adjusted based on the information on the eyeball (for example, pupil position) obtained in step S205. Thereby, the eyeball is irradiated with the video display light, so that the user can recognize the video.

In step S207, the video projection apparatus 600 ends the video projection processing according to the present technology.

The above processing may be performed by, for example, a video projection apparatus according to the present technology. For more detailed operations of each component of the apparatus in each step described above, please refer to the above "2. second embodiment (video projection apparatus)".

5. Constitution example of device

A specific example of a video projection apparatus according to the present technology is explained below with reference to fig. 9 and 10.

Fig. 9 shows a head mounted display (hereinafter referred to as HMD)900 as an example of a video projection apparatus according to the present technology. The HMD 900 is a glasses-like device, i.e., a glasses display.

The main constituent elements for information detection according to the present technology are provided in the nose pad 951 of the HMD 900. That is, in the nosepiece 951 of the HMD 900, an irradiation unit 901, a half mirror 906, a scanning mirror 903, a detection unit 904, and a controller 905 are provided. Note that the controller 905 may be provided not in the nose pad 951 but in, for example, the temple 953.

In a temple 953 of the HMD 900, there are provided main constituent elements for presenting a video to a user based on information detected according to the present technology. That is, the temple 953 of the HMD 900 is provided with the video display light irradiation unit 920. The video display light irradiation unit 920 includes an output unit 921, a scanning mirror 922, a lens 923, and an optical system 924. The video display light irradiation unit 920 may be connected to the controller 905.

In addition, the light guide unit 902 is provided to a lens 952 of the HMD 900.

The illumination light emitted from the irradiation unit 901 is reflected by the half mirror 906 and the scanning mirror 903, and reaches the light guiding unit 902. The illumination light is further reflected by the light guide unit 902 and reaches the eyeball. The reflected light from the eyeball is reflected by the light guide unit 902, scanned by the scanning mirror 903, and transmitted through the half mirror 906. The reflected light transmitted through the half mirror 906 is detected by the detection unit 904. Information on the eyeball is detected based on information on the detected reflected light and, for example, information on the scanning mirror 903. Based on the information about the eyeball, the video display light irradiation unit 920 emits video display light to enable the user to accurately recognize the video.

Fig. 10 shows another example of an HMD that is a video projection apparatus in accordance with the present technique. The HMD1000 shown in fig. 100 has a glasses-like shape, i.e., may also be referred to as a glasses display.

A main component for performing information detection according to the present technology and a main component for presenting a video to a user based on information detected according to the present technology are provided in the temple 1053 of the HMD 1000. Furthermore, the HMD1000 has a common optical system on the path of illumination light to the eyeball, reflected light from the eyeball, and video display light. In other words, an irradiation unit 1001, a scanning mirror 1003, a detection unit 1004, and a controller 1005 are provided in a temple 1053 of the HMD 1000. A temple 1053 of the HMD1000 is provided with a half mirror 1006, a lens 1007, and an optical system (e.g., a mirror) 1008. In addition, a light guide unit 1002 is provided in a lens 1052 of the HMD 1000.

Illumination light emitted from the irradiation unit 1001 reaches the light guiding unit 1002 via the half mirror 1006, the optical system 1008, and the scanning mirror 1003. The illumination light is further reflected by the light guide unit 1002 and reaches the eyeball. The reflected light from the eyeball is reflected by the light guiding unit 1002, scanned by the scanning mirror 1003, passed through the optical system 1008, and transmitted through the half mirror 1006. The reflected light transmitted through the half mirror 1006 is detected by the detection unit 1004. Information on the eyeball is detected based on information on the detected reflected light and information on the scanning mirror 1003, for example. Based on the information about the eyeball, the irradiation unit 1001 emits video display light to enable the user to accurately recognize the video.

6. Examples of the invention

(1) Example 1 (simulation of the first example of the first embodiment)

Information detection by the information detection apparatus described above in "(2) the first example (information detection apparatus) of the first embodiment" of section 1 was simulated using an optical studio (trademark) (manufactured by Zemax llc.). Fig. 11 shows the configuration of the simulation apparatus. The apparatus 1100 shown in fig. 11 includes a collimated light source 1101 as an irradiation unit, a hologram element 1102 as a light guiding unit, a collimator lens 1110, a scanning mirror 1103, and a detection unit 1104.

The light intensity of the reflected light from the eyeball detected by the detection unit 1104 is acquired by simulation in the case where the eyeball is directed to the front with respect to the pupil portion and in the case where the scanning oscillation angle of the scanning mirror 1103 is-10 degrees to 10 degrees. Fig. 15 shows the relationship between the scanning oscillation angle and the light intensity.

In addition, assuming that the eyeball is oriented toward the front at 0 degrees, the light intensity of the reflected light detected when the scanning oscillation angle is-10 to 10 degrees is similarly acquired in the case where the eyeball is rotated by 5 degrees or 10 degrees. Fig. 12 also shows the relationship between the light intensity acquired when the eyeball is rotated by 5 degrees or 10 degrees and the scan oscillation angle.

As shown in fig. 12, the boundaries (hereinafter referred to as edges; denoted by a, b, and c in fig. 12) between the scanning oscillation angle when the light intensity exceeding 0 is obtained and the scanning oscillation angle when the light intensity is 0 are different between the cases where the rotation of the eyeball is 0 degrees, 5 degrees, and 10 degrees, and the edges move according to the rotation angle of the eyeball. Therefore, the rotation angle of the eyeball, the pupil position, or the line-of-sight direction can be estimated based on the relationship between the scanning oscillation angle and the light intensity.

In addition, comparing the rotation of the eyeball at 0 degree, 5 degrees, and 10 degrees, the distribution of light intensity shifts according to the rotation angle of the eyeball. Therefore, the rotation angle of the eyeball, the pupil position, or the line-of-sight direction can also be estimated based on the distribution of the light intensity with respect to the scanning oscillation angle.

The above simulation is a result obtained when the scanning oscillation angle of the scanning mirror is changed to a one-dimensional direction. Therefore, by changing the scanning oscillation angle of the scanning mirror to a two-dimensional direction, information about the eyeball (e.g., the rotation angle of the eyeball, the pupil position, and the line-of-sight direction) can be estimated more accurately.

(2) Example 2 (simulation of the second example of the first embodiment)

Information detection by the information detection apparatus described above in "(3) the second example (information detection apparatus) of the first embodiment" of section 1 was simulated using an optical studio (trademark) (manufactured by Zemax llc.). Fig. 13 shows the configuration of the simulation apparatus. The apparatus 1300 shown in fig. 13 includes an illumination unit 1301, a hologram element (grating) 1302 as a light guiding unit, a collimator lens 1310, a scanning mirror 1303, and a detection unit 1304. In addition, the device 1300 includes a half mirror 1305.

In the case where the eyeball is directed to the front with respect to the pupil portion and in the case where the scanning oscillation angle of the scanning mirror 1303 is-10 degrees to 10 degrees, the light intensity of the reflected light from the eyeball detected by the detection unit 1304 is acquired by simulation. Fig. 14 shows the relationship between the scanning oscillation angle and the light intensity.

In addition, assuming that the eyeball is oriented to the front at 0 degrees, in the case where the eyeball is rotated by 5 degrees or 10 degrees, the light intensity detected when the scanning oscillation angle is-10 degrees to 10 degrees is similarly acquired. Fig. 14 also shows the relationship between the light intensity acquired when the eyeball is rotated by 5 degrees or 10 degrees and the scan oscillation angle.

As shown in fig. 14, the edges (a, b, and c in fig. 14) are different between the cases where the rotation of the eyeball is 0 degrees, 5 degrees, and 10 degrees, and the edges move with the rotation angle of the eyeball. Therefore, the rotation angle of the eyeball, the pupil position, or the line-of-sight direction can be estimated based on the relationship between the scanning oscillation angle and the light intensity.

Comparing the rotation of the eyeball with 0 degree, 5 degrees and 10 degrees, the distribution of the light intensity shifts according to the rotation angle of the eyeball. Therefore, the rotation angle of the eyeball, the pupil position, or the line-of-sight direction can also be estimated based on the distribution of the light intensity with respect to the scanning oscillation angle.

In addition, in each case where the rotation of the eyeball is 0 degree, 5 degrees, and 10 degrees, a prominent peak (d in fig. 14) can be confirmed. The peaks are due to both surface reflected light on the cornea and scattered light on the retina. In addition, the region around the peak (e in fig. 14) is caused by scattered light on the retina. Accordingly, based on the peak and/or the region, information about the eyeball (e.g., pupil position, gaze direction, or rotation angle of the eyeball) can be more accurately estimated.

In the case of using the apparatus 1300, unlike the case of using the apparatus 1100 of example 1, a peak and/or a region can be observed. This is mainly due to the use of beam-like light (particularly laser light) as illumination light to be emitted toward the eyeball. Therefore, by using the irradiation unit for irradiating the eyeball with the beam-like light as the irradiation unit for irradiating the eyeball with light, information relating to the eyeball can be obtained more accurately.

(3) Example 3 (simulation of the second example of the first embodiment)

Information detection by the information detection apparatus described above in "(3) the second example (information detection apparatus) of the first embodiment" of section 1 was simulated using an optical studio (trademark) (manufactured by Zemax llc.). Fig. 15 shows the configuration of the simulation apparatus. The apparatus 1500 shown in fig. 15 includes an illumination unit 1501, a hologram element (grating lens) 1502 as a light guide unit, a collimator lens 1510, a scanning mirror 1503, and a detection unit 1504. In addition, the device 1500 includes a half mirror 1505.

Unlike example 2, the hologram element 1502 is a grating lens, i.e., has a light condensing property. Due to this condensing characteristic, the illumination light from the illumination unit is condensed toward the pupil.

The light intensity of the reflected light from the eyeball detected by the detection unit 1504 is acquired by simulation in the case where the eyeball is directed to the front with respect to the pupil portion and in the case where the scanning oscillation angle of the scanning mirror 1503 is-10 degrees to 10 degrees. Fig. 16 shows the relationship between the scanning oscillation angle and the light intensity.

In addition, assuming that the eyeball is oriented to the front at 0 degrees, in the case where the eyeball is rotated by 5 degrees or 10 degrees, the light intensity detected when the scanning oscillation angle is-10 degrees to 10 degrees is similarly acquired. Fig. 16 also shows the relationship between the light intensity acquired when the eyeball is rotated by 5 degrees or 10 degrees and the scan oscillation angle.

As shown in fig. 16, the edges (a, b, and c in fig. 16) are different between the cases where the rotation of the eyeball is 0 degrees, 5 degrees, and 10 degrees, and the edges move with the rotation angle of the eyeball. Therefore, the rotation angle of the eyeball, the pupil position, or the line-of-sight direction can be estimated based on the relationship between the scanning oscillation angle and the light intensity.

Further, comparing the rotation of the eyeball at 0 degree, 5 degrees, and 10 degrees, the distribution of the light intensity shifts according to the rotation angle of the eyeball. Therefore, the rotation angle of the eyeball, the pupil position, or the line-of-sight direction can also be estimated based on the distribution of the light intensity.

In each case where the rotation of the eyeball is 0 degrees, 5 degrees, and 10 degrees, the peak of the protrusion (d in fig. 16) can be confirmed. The peaks are due to both surface reflected light on the cornea and scattered light on the retina. In addition, the region around the peak (e in fig. 16) is caused by scattered light on the retina. Accordingly, based on the peak and/or the region, information about the eyeball (e.g., pupil position, gaze direction, or rotation angle of the eyeball) can be more accurately estimated.

In the case of using the apparatus 1500, unlike the case of using the apparatus 1100 of example 1, a peak and/or a region can be observed. This is mainly due to the use of beam-like light (particularly laser light) as illumination light to be emitted toward the eyeball. Therefore, by using the irradiation unit for irradiating the eyeball with the beam-like light as the irradiation unit for irradiating the eyeball with light, information relating to the eyeball can be obtained more accurately.

Although the rotation angles of the eyeballs in examples 2 and 3 are the same, the intervals of the three peaks and the intervals of the three edges observed when the apparatus 1500 is used are wider than those when the apparatus 1300 is used in example 2. That is, even at the same rotational angle, the resolution is higher when using the device 1500. This difference is due to the fact that: because the apparatus 1500 uses a grating lens, the illumination light is focused to the eyeball, whereas because the apparatus 1300 uses a grating, the illumination light is not focused to the eyeball. Therefore, by condensing the illumination light to the eyeball with the grating lens, it is believed that information about the eyeball can be obtained with higher accuracy.

Furthermore, the three edges observed when using the device 1300 are sharper than the edges observed when using the device 1500. Therefore, in order to obtain information about the eyeball based on the edge, it is considered preferable to use an irradiation unit that makes the illumination light go straight with respect to the eyeball without condensing the illumination light.

The intensity in region "e" (scattered light on the retina) observed when device 1300 is in use is higher than the intensity obtained when device 1500 is in use. This difference can be attributed to the fact that: since the illumination light does not go straight toward the eyeball when the apparatus 1500 is used, the intensity of the scattered light on the retina detected by the detection unit is low. Therefore, in order to obtain information on the eyeball based on the intensity of the scattered light on the retina, it is preferable to consider using an irradiation unit that irradiates the eyeball with illumination light serving as parallel light without condensing the illumination light on the pupil.

(4) Example 4 (simulation of the first example of the second embodiment)

The information detection of the video projection apparatus described above in "(2) the first example (video projection apparatus) of the second embodiment" of section 2 was simulated using an optical studio (trademark) (manufactured by Zemax llc.). Fig. 17 shows the configuration of the simulation apparatus. As shown in fig. 17, the video projection apparatus 1700 includes an illumination unit 1701, a light guide unit 1702, a scanning mirror 1703, and a detection unit 1704. The video projection apparatus 1700 further includes a video display light irradiation unit 1720.

The video projection apparatus 1700 is an apparatus as an embodied video display light irradiation unit of the video projection apparatus 400 described above in "(2) the first example (video projection apparatus)" of the second embodiment. That is, the video display light irradiation unit 1720 includes an output unit 1721, a scan mirror 1722, and a lens 1723. The output unit 1721 outputs video display light. The direction, intensity, and the like of the output video display light are adjusted by the optical system 1724. The output video display light is scanned by a scanning mirror 1722. The scanned video display light is collimated by lens 1723 and reaches light directing unit 1702. In this way, the video display light irradiation unit 1720 collimates the scanned video display light and irradiates the light guiding unit 1702 with the video display light.

The light guiding unit 1702 reflects the video display light so that the video display light is condensed near the pupil. That is, the light guiding unit 1702 is a reflective grating lens for video display light. The video display light reflected by the light guiding unit 1702 passes straight through the pupil (particularly, the center of the pupil) and then reaches the retina. As a result, the user can recognize the video by the video display light.

In the case where the eyeball is irradiated with the video display light by the video display light irradiation unit 1720, the case where the eyeball faces the front with respect to the light guiding unit, and the scan oscillation angle of the scan mirror 1703 is-10 degrees to 10 degrees, the light intensity of the reflected light from the eyeball detected by the detection unit 1704 is acquired by simulation.

In addition, assuming that the eyeball is oriented toward the front at 0 degrees, the light intensity of the reflected light detected when the scanning oscillation angle is-10 to 10 degrees is similarly acquired in the case where the eyeball is rotated by 5 degrees or 10 degrees. The simulation results were similar to those obtained in example 1 above. Therefore, it can be understood that information about the eyeball can be detected also in the case where the video display light is emitted by the video display light irradiation unit.

(5) Example 5 (simulation of the second example of the second embodiment)

The information detection of the video projection apparatus described above in "(3) the second example (video projection apparatus) of the second embodiment" of part 2 was simulated using an optical studio (trademark) (manufactured by Zemax llc.). Fig. 18 shows the configuration of the simulation apparatus. As shown in fig. 18, the video projection apparatus 1800 includes an irradiation unit 1801, a light guide unit 1802, a scanning mirror 1803, and a detection unit 1804. The video projection apparatus 1800 further includes a half mirror 1806 and a video display light irradiation unit 1820.

The video projection apparatus 1800 is an apparatus as an embodied video display light irradiation unit of the video projection apparatus 500 described in "(3) second example (video projection apparatus)" of the second embodiment. That is, the video display light irradiation unit 1820 includes an output unit 1821, a scanning mirror 1822, and a lens 1823. The output unit 1821 outputs video display light. The direction, intensity, and the like of the output video display light are adjusted by the optical system 1824. The output video display light is scanned by a scanning mirror 1822. The scanned video display light is collimated by lens 1823 and reaches light directing unit 1802. In this way, the video display light irradiation unit 1820 collimates the scanned video display light and irradiates the light guide unit 1802 with the video display light.

The light guide unit 1802 reflects the video display light so that the video display light is condensed near the pupil. That is, the light guide unit 1802 is a reflective grating lens for video display light. The video display light reflected by the light guide unit 1802 passes straight through the pupil (particularly, the center of the pupil) and then reaches the retina. As a result, the user can recognize the video by the video display light.

In addition, video projection apparatus 1800 includes a lens 1825 between scan mirror 1803 and light directing unit 1802. The illumination light scanned by the scanning mirror 1803 is collimated by the lens 1825 to reach the light guide unit 1802. Then, the light guide unit 1802 reflects the collimated illumination light, and the illumination light reaches the eyeball. That is, the light guide unit 1802 is a reflection grating for illumination light.

In the case where the eyeball is irradiated with the video display light by the video display light irradiation unit 1820, the eyeball is directed toward the front with respect to the light guiding unit, and the scanning oscillation angle of the scanning mirror 1803 is-10 degrees to 10 degrees, the light intensity of the reflected light from the eyeball detected by the detection unit 1804 is acquired by simulation.

In addition, assuming that the eyeball is oriented toward the front at 0 degrees, the light intensity of the reflected light detected when the scanning oscillation angle is-10 to 10 degrees is similarly acquired in the case where the eyeball is rotated by 5 degrees or 10 degrees.

The simulation results were similar to those obtained in example 2 above. Therefore, it can be understood that information about the eyeball can be detected also in the case where the video display light is emitted by the video display light irradiation unit.

(6) Example 6 (simulation of the third example of the second embodiment)

The information detection of the video projection apparatus described above in "(4) the third example (video projection apparatus)" of the second embodiment of section 2 was simulated using an optical studio (trademark) (manufactured by Zemax llc.). Fig. 19 shows the configuration of the simulation apparatus. As shown in fig. 19, the video projection apparatus 1900 includes an illumination unit 1901, a light guide unit 1902, a scanning mirror 1903, and a detection unit 1904. The video projection device 1900 further includes a half mirror 1906 and a lens 1907.

The video projection apparatus 1900 has a similar configuration to the video projection apparatus 600 described above in "(4) the third example (video projection apparatus)" of the second embodiment.

Note that the video projection apparatus 1900 and the video projection apparatus 600 are different from each other in that the configurations of the irradiation unit and the detection unit are exchanged, and in the former, the half mirror 1906 transmits the illumination light and the video display light and reflects the reflected light. The video projection apparatus 1900 is also different in that: a lens 1907 is provided between the scanning mirror 1903 and the light guide unit 1902. The illumination light scanned by the scanning mirror 1903 is collimated by the lens 1907 to reach the light guiding unit 902. Then, the light guide unit 902 reflects the collimated illumination light, which reaches the eyeball. That is, the light guide unit 1902 is a reflection grating for illumination light.

In the case where the eyeball 1950 is irradiated with the video display light via the scanning mirror 1903 by the irradiation unit 1901, the case where the eyeball 1950 faces the front with respect to the light guiding unit, and the case where the scanning oscillation angle of the scanning mirror 1903 is-10 degrees to 10 degrees, the light intensity of the reflected light from the eyeball 1950 detected by the detection unit 1904 is acquired by simulation.

In addition, assuming that the eyeball 1950 is directed forward at 0 degrees, in the case where the eyeball 1950 rotates 5 degrees or 10 degrees, the light intensity of the reflected light detected when the scanning oscillation angle is-10 degrees to 10 degrees is similarly acquired.

The simulation results were similar to those obtained in example 2 above. Therefore, it can be understood that information relating to the eyeballs can be detected also in the case where the eyeballs are illuminated with the video display light on the same path as the illumination light by the video display light illumination unit.

The present technology may have the following configuration.

[1] An information detection apparatus comprising:

an irradiation unit that irradiates an eyeball with light;

a scanning mirror that scans reflected light from the eyeball; and

a detection unit that detects reflected light scanned by the scanning mirror.

[2] The information detection apparatus according to [1], further comprising:

a controller that performs estimation processing of a pupil position or a gaze direction of the eyeball based on the reflected light detected by the detection unit.

[3] The information detecting apparatus according to [2], wherein

The controller estimates a pupil position or a gaze direction of the eyeball based on a scanning oscillation angle of the scanning mirror and the reflected light detected by the detection unit.

[4] The information detecting apparatus according to [2], wherein

The controller estimates a pupil position or a gaze direction of the eyeball based on a scanning oscillation angle of the scanning mirror and an intensity of the reflected light detected by the detection unit.

[5] The information detecting apparatus according to any one of [1] to [4], wherein

The irradiation unit is configured to irradiate the eyeball with the light without passing through the scanning mirror.

[6] The information detecting apparatus according to any one of [1] to [5], wherein

The irradiation unit is configured to irradiate the eyeball with the light via the scanning mirror.

[7] The information detecting apparatus according to any one of [1] to [6], wherein

The light for illuminating the eyeball is non-visible light.

[8] The information detecting apparatus according to any one of [1] to [7], wherein

The light for illuminating the eyeball is infrared light.

[9] The information detecting apparatus according to any one of [1] to [8], wherein

The light for illuminating the eyeball is a beam-like light.

[10] A video projection apparatus comprising:

an irradiation unit that irradiates an eyeball with light;

a scanning mirror that scans reflected light from the eyeball;

a detection unit that detects reflected light scanned by the scanning mirror;

a controller that performs estimation processing on a pupil position of the eyeball based on the reflected light detected by the detection unit; and

a video display light irradiation unit that irradiates the pupil position estimated by the controller with video display light so that the video display light passes through the pupil position.

[11] The video projection apparatus according to [10], wherein

The video display light is concentrated near the pupil, and the retina is illuminated with the video display light.

[12] The video projection apparatus according to [10] or [11], wherein

Illuminating an eye with the video display light via the scanning mirror.

[13] The video projection device of any of [10] to [12], wherein

The video projection device is a glasses display.

[14] An information detection method, comprising:

irradiating an eyeball with light;

scanning reflected light from the eyeball through a scanning mirror; and

reflected light scanned by the scan mirror is detected.

[15] A video projection method, comprising:

irradiating an eyeball with light;

scanning reflected light from the eyeball through a scanning mirror;

detecting reflected light scanned by the scanning mirror;

estimating a pupil position of the eyeball based on the reflected light detected in the detecting step; and

illuminating the pupil position estimated in the estimating step with video display light such that the video display light passes through the pupil position.

List of reference symbols

100,300 information detection device

101,301 irradiation unit

102,302 light guide unit

103,303 scanning mirror

104,304 detection unit

105,305 controller