Intraoral 3D scanner with multiple miniature cameras and miniature pattern projectors

文档序号：1663383 发布日期：2019-12-31 浏览：32次中文

阅读说明：本技术 用多个***头和微型图案投射器的口内3d扫描仪 (Intraoral 3D scanner with multiple miniature cameras and miniature pattern projectors ) 是由奥弗·萨菲尔优素福·阿缇亚 A·罗尼斯基 N·马克梅尔 S·奥泽罗夫塔勒·维科尔埃于 2019-06-24 设计创作，主要内容包括：用于口内扫描的装置包括具有探针的细长手持杆。一个或更多个光投射器和两个或更多个摄像头设置在探头内。每个光投射器具有使用衍射或折射来形成光图案的图案生成光学元件。每个摄像头可以被配置为聚焦在距离与摄像头传感器相距最远的透镜1mm到30mm之间处。还描述了其他应用。(An apparatus for intraoral scanning includes an elongated hand held wand with a probe. One or more light projectors and two or more cameras are disposed within the probe. Each light projector has pattern generating optical elements that use diffraction or refraction to form a light pattern. Each camera may be configured to focus between 1mm and 30mm from the lens furthest from the camera sensor. Other applications are also described.)

1. An apparatus for intraoral scanning, the apparatus comprising:

an elongated hand-held wand including a probe at a distal end of the elongated hand-held wand;

one or more light projectors, each light projector comprising at least one light source and a pattern generating optical element, wherein each light projector is configured to project a light pattern defined by a plurality of projector rays when the light source is activated;

two or more cameras, each of the two or more cameras including a camera sensor having an array of pixels, wherein each of the two or more cameras is configured to capture a plurality of images depicting at least a portion of the projected light pattern on the intraoral surface; and

one or more processors configured to:

accessing calibration data that associates camera rays corresponding to pixels on a camera sensor of each of the two or more cameras with projector rays of the plurality of projectors;

determining, using the calibration data, an intersection of a projector ray and a camera ray corresponding to at least a portion of the projected light pattern, wherein the intersection of the projector ray and the camera ray is associated with a three-dimensional point in space;

identifying a three-dimensional location of the projected light pattern based on the light pattern that the two or more cameras agree that there is a projector ray projection at certain intersection points; and

the identified three-dimensional locations are used to generate a digital three-dimensional model of the intraoral surface.

2. The apparatus of claim 1, wherein the light pattern comprises a plurality of light spots, and wherein each of the plurality of projector rays corresponds to a light spot of the plurality of light spots.

3. The apparatus of claim 2, wherein each projector ray corresponds to a respective path of a pixel on a camera sensor of a respective one of the two or more cameras, and wherein to identify a three-dimensional location, the one or more processors run a corresponding algorithm:

for each projector light ray i, identifying, for each detected light spot j on the camera sensor path corresponding to the projector light ray i, how many other cameras detect, on their respective camera sensor paths corresponding to the projector light ray i, a respective light spot k corresponding to a respective camera light ray that intersects the projector light ray i and the camera light ray corresponding to the detected light spot j, wherein the projector light ray i is identified as a particular projector light ray that generates the detected light spot j for which the largest number of other cameras detect the respective light spot k; and

the respective three-dimensional position on the interior surface of the mouth is calculated as the intersection of the projector ray i and the respective camera ray corresponding to the detected spot j and the respective detected spot k.

4. The apparatus of claim 3, wherein to identify a three-dimensional location, the one or more processors are further to:

the projector light i and the corresponding camera light corresponding to the detected light spot j and the corresponding detected light spot k are not considered again; and

the corresponding algorithm is run for the next projector ray i.

5. The apparatus of claim 3, further comprising:

a temperature sensor;

wherein the one or more processors are further to:

receiving temperature data from a temperature sensor, wherein the temperature data is indicative of a temperature of at least one of the one or more light projectors or the two or more cameras; and

based on the temperature data, selecting between a plurality of sets of stored calibration data corresponding to a plurality of respective temperatures, each set of stored calibration data indicating for the respective temperature (a) projector light corresponding to each projected light spot from each of the one or more projectors, and (b) camera light corresponding to each pixel on the camera sensor of each of the one or more cameras.

6. The apparatus of claim 2, wherein the light pattern comprises a non-coded structured light pattern, and wherein the plurality of light spots comprises an approximately uniform distribution of discrete unconnected light spots.

7. The apparatus of claim 2, wherein the plurality of light spots comprises a first subset of light spots having a first wavelength and a second subset of light spots having a second wavelength, and wherein the calibration data comprises first calibration data of the first wavelength and second calibration data of the second wavelength.

8. The apparatus of claim 1, further comprising:

a target having a plurality of regions;

wherein:

each light projector having at least one region of the target in its illumination field;

each camera has at least one region of interest in its field of view;

a plurality of the regions of the target are in a field of view of one of the cameras and in an illumination field of one of the projectors; and

the one or more processors are further configured to:

receiving data from the two or more cameras indicating a position of the target relative to the light pattern;

comparing the received data to a stored calibration position of the target, wherein a difference between (i) the received data indicative of the position of the target and (ii) the stored calibration position of the target represents an offset of the projector light and the camera light from their respective calibration values; and

the offset of the projector light and the camera light is taken into account in the identification of the three-dimensional position.

9. The apparatus of claim 1, wherein the one or more processors are further configured to:

driving each of the one or more light projectors to project a light pattern at an interior surface of the mouth; and

driving each of the two or more cameras to capture the plurality of images.

10. The apparatus of claim 1 wherein the one or more light projectors comprise a plurality of structured light projectors, wherein each of the plurality of structured light projectors is for projecting a distribution of respective discrete unconnected light points on an intraoral surface simultaneously or at different times.

11. The apparatus of claim 1, wherein the pattern generating optical element comprises a diffractive optical element or a refractive optical element.

12. The apparatus of claim 1, wherein the two or more cameras are configured to focus at an object focal plane that is positioned between about 1mm to about 30mm from a camera lens that is farthest from a camera sensor.

13. An apparatus for intraoral scanning, the apparatus comprising:

an elongated hand-held wand including a probe at a distal end of the elongated hand-held wand;

one or more light projectors, each light projector comprising:

at least one light source configured to generate light when activated; and

a pattern generating optical element, wherein the pattern generating optical element is configured to generate a light pattern when light is transmitted through the pattern generating optical element; and

two or more cameras, each of the two or more cameras comprising a camera sensor and one or more lenses, wherein each of the two or more cameras is configured to capture a plurality of images depicting at least a portion of the projected light pattern on the intraoral surface, wherein each camera is configured to focus at an object focal plane positioned between about 1mm to about 30mm from a lens of the one or more lenses furthest from the camera sensor.

14. The apparatus of claim 13, wherein the one or more light projectors are disposed within a probe, and wherein the two or more cameras are disposed within a probe.

15. The apparatus of claim 14, wherein:

the one or more light projectors include at least two light projectors, the two or more cameras include at least four cameras;

the at least two light projectors and a majority of the at least four cameras are arranged in at least two rows each substantially parallel to a longitudinal axis of the probe, the at least two rows including at least a first row and a second row;

a distal-most camera along the longitudinal axis and a proximal-most camera along the longitudinal axis of the at least four cameras are positioned such that their optical axes are at an angle of 90 degrees or less relative to each other from a line of sight perpendicular to the longitudinal axis; and

the cameras in the first row and the cameras in the second row are positioned such that from a line of sight coaxial with the longitudinal axis of the probe, the optical axes of the cameras in the first row are at an angle of 90 degrees or less relative to the optical axes of the cameras in the second row.

16. The apparatus of claim 15, wherein:

the remaining cameras of the at least four cameras, except for the most distal camera and the most proximal camera, have optical axes substantially parallel to the longitudinal axis of the probe; and

each of the at least two rows includes an alternating sequence of projectors and cameras.

17. The apparatus of claim 16, wherein the at least four cameras comprise at least five cameras, wherein the at least two light projectors comprise at least five light projectors, wherein the most proximal component in the first row is a light projector, and wherein the most proximal component in the second row is a camera.

18. The apparatus of claim 15, wherein:

the distal-most camera along the longitudinal axis and the proximal-most camera along the longitudinal axis are positioned such that their optical axes are at an angle of 35 degrees or less relative to each other from a line of sight perpendicular to the longitudinal axis; and

the cameras in the first row and the cameras in the second row are positioned such that from a line of sight coaxial with the longitudinal axis of the probe, the optical axes of the cameras in the first row are at an angle of 35 degrees or less relative to the optical axes of the cameras in the second row.

19. The apparatus of claim 13, wherein the light pattern is defined by a plurality of projector rays, the apparatus further comprising one or more processors configured to:

accessing calibration data that associates camera rays corresponding to pixels on a camera sensor of each of the two or more cameras with projector rays of the plurality of projector rays;

determining, using the calibration data, an intersection of a projector ray and a camera ray corresponding to a portion of the projected light pattern, wherein the intersection of the projector ray and the camera ray is associated with a three-dimensional point in space;

the identified three-dimensional locations are used to generate a digital three-dimensional model of the intraoral surface.

20. The apparatus of claim 13 wherein the one or more structured light projectors are each configured to generate a distribution of discrete unconnected light points at all planes between 1mm and 30mm from the pattern generating optical element.

21. The apparatus of claim 13 wherein each of the one or more structured light projectors has an illumination field of about 45 degrees to about 120 degrees, and wherein each of the two or more cameras has a field of view of about 45 to about 120 degrees.

22. The apparatus of claim 13, wherein the pattern generation optical element is configured to generate the light pattern using at least one of diffraction or refraction, and wherein the pattern generation optical element has a luminous flux efficiency of at least 90%.

23. The apparatus of claim 13, further comprising:

at least one uniform light projector configured to project white light onto the intraoral surface, wherein at least one of the two or more cameras is configured to capture a two-dimensional color image of the intraoral surface using illumination from the uniform light projector.

24. The apparatus of claim 13, wherein the pattern generating optical element comprises a Diffractive Optical Element (DOE).

25. The apparatus of claim 24, wherein the DOE is divided into a plurality of sub-DOE sheets arranged in an array, wherein each sub-DOE sheet generates a distribution of respective discrete unconnected light points in a different region of the illumination field, such that when the light source is activated, a distribution of discrete unconnected light points is generated.

26. The apparatus of claim 13, wherein each of the one or more projectors comprises an additional optical element disposed between the light source and the pattern generating optical element, the additional optical element configured to generate a bessel beam from light transmitted through the additional optical element, wherein the light pattern comprises a pattern of discrete unconnected light spots, the light spots maintaining a diameter of less than 0.06mm through each inner surface of a geometric sphere centered on the pattern generating optical element and having a radius between 1mm and 30 mm.

27. The apparatus of claim 26, wherein the additional optical element comprises an axicon lens.

28. The apparatus of claim 13, wherein the light pattern comprises a distribution of discrete unconnected light points, wherein, for each orthogonal plane in the illumination field, a ratio of illuminated area to non-illuminated area is 1:150-1: 16.

29. An apparatus for intraoral scanning, the apparatus comprising:

an elongated hand-held wand including a probe at a distal end of the elongated hand-held wand;

one or more light projectors disposed within the probe, each light projector comprising:

a light source configured to generate light when activated;

a first optical element configured to generate a Bessel beam from light transmitted through the first optical element;

a pattern generating optical element, wherein the pattern generating optical element is configured to generate a light pattern when the Bessel light beam is transmitted through the pattern generating optical element; and

two more cameras, each camera including a camera sensor and objective optics including one or more lenses, wherein each camera is configured to capture a plurality of images depicting at least a portion of the projected light pattern on the interior intraoral surface.

30. The apparatus of claim 29, wherein the light pattern comprises a pattern comprising discrete unconnected light spots that maintain a substantially uniform size at any orthogonal plane positioned between about 1mm and about 30mm from the pattern generating optical element.

31. The apparatus of claim 30, wherein the spots have a diameter of less than 0.06mm through each inner surface of a geometric sphere centered on the pattern generating optical element and having a radius between 1mm and 30 mm.

32. The apparatus of claim 29, wherein the first optical element comprises an axicon lens.

33. The apparatus of claim 32, wherein the axicon lens has an axicon angle of 0.2-2 degrees.

34. The apparatus of claim 29, further comprising: a beam shaping optical element between the first optical element and the light source, wherein the beam shaping optical element comprises a collimating lens.

35. The apparatus of claim 29, wherein each camera is configured to focus at an object focal plane positioned between about 1mm to about 30mm from a lens of the one or more lenses furthest from the camera sensor.

36. The apparatus of claim 29 wherein each of the one or more structured light projectors has an illumination field of about 45 degrees to about 120 degrees, wherein each of the two or more cameras has a field of view of about 45 degrees to about 120 degrees, and wherein the light source comprises at least one laser diode.

37. The apparatus of claim 29, wherein at least one of the two or more cameras is a light field camera.

38. An apparatus for intraoral scanning, the apparatus comprising:

an elongated hand-held wand including a probe at a distal end of the elongated hand-held wand;

at least two light projectors disposed within the probe, each light projector comprising:

at least one light source configured to generate light when activated; and

at least four cameras disposed within the probe, each of the at least four cameras comprising a camera sensor and one or more lenses, wherein each of the at least four cameras is configured to capture a plurality of images depicting at least a portion of the projected light pattern on the intraoral surface;

wherein a majority of the at least two light projectors and the at least four cameras are arranged in at least two rows each substantially parallel to a longitudinal axis of the probe, the at least two rows including at least a first row and a second row.

39. The apparatus of claim 38, wherein:

40. The apparatus of claim 39, wherein:

the remaining cameras of the at least four cameras, except for the most distal camera and the most proximal camera, have optical axes substantially parallel to the longitudinal axis of the probe; and

each of the at least two rows may include an alternating sequence of projectors and cameras.

41. The apparatus of claim 40, wherein the at least four cameras comprise at least five cameras, wherein the at least two light projectors comprise at least five light projectors, wherein the most proximal component in the first row is a light projector, and wherein the most proximal component in the second row is a camera.

42. The apparatus of claim 39, wherein:

the cameras in the first row and the cameras in the second row are positioned such that from a line of sight coaxial with the longitudinal axis of the probe, the optical axes of the cameras in the first row are at an angle of 35 degrees or less relative to the optical axes of the cameras in the second row.

43. The apparatus of claim 38, wherein the at least four cameras have a combined field of view along the longitudinal axis of 25-45mm and a field of view along the z-axis of 20-40mm corresponding to a distance from the probe.

44. The apparatus of claim 38, wherein the pattern generation optical element is configured to generate the light pattern using at least one of diffraction or refraction, and wherein the pattern generation optical element has a luminous flux efficiency of at least 90%.

45. The apparatus of claim 38, wherein each of the at least two projectors comprises an additional optical element disposed between the light source and the pattern generating optical element, the additional optical element configured to generate a bessel beam from light transmitted through the additional optical element, wherein the light pattern comprises a pattern of discrete unconnected light points, the pattern maintaining a diameter of less than 0.06mm through each inner surface of a geometric sphere centered on the pattern generating optical element and having a radius between 1mm and 30 mm.

46. The apparatus of claim 45 wherein the additional optical element comprises an axicon lens.

Technical Field

The present invention relates generally to three-dimensional imaging, and more particularly to intraoral three-dimensional imaging using structured light illumination.

Background

Dental impressions of the intraoral three-dimensional surfaces of a subject (e.g., teeth and gums) are used to plan dental treatment. A conventional dental impression is made using a dental impression tray filled with impression material (e.g. PVS or alginate) of the subject's bite. The impression material then cures into a negative impression of the teeth and gums, from which a three-dimensional model of the teeth and gums can be formed.

Digital dental impressions utilize intraoral scans to generate three-dimensional digital models of the intraoral three-dimensional surface of a subject. Digital intraoral scanners typically use structured light three-dimensional imaging. The surface of the subject's teeth may be highly reflective and somewhat translucent, which may reduce contrast in the structured light pattern reflected from the teeth. Thus, to improve capture of intraoral scans, when using digital intraoral scanners that utilize structured light three-dimensional imaging, prior to scanning, the subject's teeth are often coated with opaque powder to facilitate contrast of the structured light pattern to a useful level, for example, to convert a surface to a scattering surface. Although some advances have been made in intraoral scanners that utilize structured light three-dimensional imaging, there may be additional advantages.

Disclosure of Invention

The use of structured light three-dimensional imaging may lead to "correspondence problems" in which the correspondence between the light spots in the structured light pattern and the light spots seen by the camera observing the pattern needs to be determined. One technique to address this problem is based on projecting "coded" light patterns and imaging the illuminated scene from one or more viewpoints. The emitted light pattern is encoded such that portions of the light pattern are unique and distinguishable when captured by the camera system. Since the pattern is encoded, the correspondence between the image points and the points of the projected pattern can be found more easily. The decoded points can be triangulated and three-dimensional information recovered.

Applications of the present invention include systems and methods relating to a three-dimensional intraoral scanning device including one or more cameras and one or more pattern projectors. For example, certain applications of the present invention may involve an intraoral scanning device having multiple cameras and multiple pattern projectors.

Other applications of the invention include methods and systems for decoding structured light patterns.

Other applications of the present invention may involve systems and methods for three-dimensional intraoral scanning using non-coded structured light patterns. For example, the non-coded structured light pattern may comprise a uniform pattern of dots.

For example, in some particular applications of the present invention, an apparatus for intraoral scanning is provided that includes an elongated hand-held wand having a probe at a distal end. During scanning, the probe may be configured to enter the oral cavity of the subject. One or more light projectors (e.g., micro-structured light projectors) and one or more cameras (e.g., micro-cameras) are coupled to a rigid structure disposed within the distal end of the probe. Each structured light projector emits light using a light source, such as a laser diode. Each light projector may be configured to project a light pattern defined by a plurality of projector light rays when the light source is activated. Each camera may be configured to capture a plurality of images depicting at least a portion of the light pattern projected on the interior intraoral surface. In some applications, the structured light projector may have an illumination field of at least 45 degrees. Alternatively, the illumination field may be less than 120 degrees. Each structured light projector may also include pattern generating optical elements. The pattern generating optical element may generate the light pattern using diffraction and/or refraction. In some applications, the light pattern may be a distribution of discrete unconnected light points. Optionally, when the light source (e.g. a laser diode) is activated to emit light through the pattern generating optical element, the light pattern maintains a distribution of discrete unconnected light points at all planes between 1mm and 30mm from the pattern generating optical element. In some applications, the pattern generating optical elements of each structured light projector may have a luminous flux efficiency, i.e., the proportion of light falling on the pattern generator that enters the pattern, of at least 80% (e.g., at least 90%). Each camera includes a camera sensor and objective optics including one or more lenses.

In some applications, a laser diode light source and diffractive and/or refractive pattern generating optical elements may provide certain advantages. For example, the use of a laser diode and diffractive and/or refractive pattern generating optical elements may help maintain an energy efficient structured light projector to prevent the probe from heating up during use. In addition, these components may help reduce costs by not requiring active cooling within the probe. For example, modern laser diodes may use less than 0.6 watts of power while continuously emitting at high brightness (e.g., as compared to modern Light Emitting Diodes (LEDs)). These modern laser diodes may use even less power when pulsed according to some applications of the present invention, for example, less than 0.06 watts when pulsed at a 10% duty cycle (although for some applications the laser diode may use at least 0.2 watts while continuously emitting at high brightness, and even less power when pulsed, for example, at least 0.02 watts when pulsed at a 10% duty cycle). Furthermore, the diffractive and/or refractive pattern generating optical element may be configured to emit light with most, if not all, of the light (e.g., as compared to a mask that blocks some light from reaching the object).

In particular, a diffraction and/or refraction based pattern generating optical element generates a pattern by diffraction, refraction, or interference of light, or any combination of the above, rather than light modulation by a transparent or transmissive mask. In some applications, this may be advantageous because the light flux (through) efficiency (the proportion of light entering the pattern to light falling on the pattern generator) is close to 100% (e.g. at least 80%, e.g. at least 90%) regardless of the "area based duty cycle" mode. In contrast, the luminous flux efficiency of a transparent mask or transmissive mask pattern generating optical element is directly related to the "area-based duty cycle". For example, for a desired "area based duty cycle" of 100:1, the pass through efficiency of the mask based pattern generator would be 1%, while the efficiency of the diffraction and/or refraction based pattern generating optical elements remains close to 100%. Furthermore, the light collection efficiency of the laser is at least 10 times higher than an LED with the same total light output, since the laser inherently has a smaller emission area and divergence angle, resulting in brighter output illumination per unit area. The high efficiency of the laser and diffractive and/or refractive pattern generator can help achieve a thermally efficient configuration that limits significant probe temperature rise during use, thereby reducing costs by potentially eliminating or limiting the need for active cooling within the probe. While laser diodes and DOEs may be particularly preferred in some applications, their use alone or in combination is not essential. Other light sources including LEDs and pattern generating elements including transparent and transmissive masks may be used in other applications.

In some applications, to improve image capture of an intraoral scene under structured light illumination without using contrast enhancement means such as coating teeth with opaque powder, the inventors have realized that the distribution of discrete unconnected dots (e.g., rather than lines) can provide an improved balance between improving pattern contrast while maintaining a useful amount of information. In some applications, the unconnected spots have a uniform (e.g., constant) pattern. In general, denser structured light patterns may provide more surface sampling, higher resolution, and better stitching of corresponding surfaces obtained from multiple image frames. However, too dense a structured light pattern may lead to more complex correspondence problems, since there are a higher number of spots to solve. In addition, denser structured light patterns may have lower pattern contrast due to more light in the system, which may be caused by a combination of (a) stray light that reflects off of somewhat smooth surfaces of the tooth and may be captured by the camera, and (b) penetration (percolation), i.e., some light enters the tooth, reflects along multiple paths within the tooth, and then exits the tooth in many different directions. As described further below, methods and systems are provided for addressing the corresponding problems presented by the distribution of discrete unconnected light points. In some applications, the discrete unconnected light points from each projector may be non-encoded.

In some applications, the field of view of each camera may be at least 45 degrees, for example, at least 80 degrees (e.g., 85 degrees). Optionally, the field of view of each camera may be less than 120 degrees, for example less than 90 degrees. For some applications, one or more cameras have a fisheye lens or other optics that provide up to a 180 degree viewing angle (viewing).

In any case, the fields of view of the various cameras may be the same or different. Similarly, the focal lengths of the various cameras may be the same or different. The term "field of view" of each camera as used herein refers to the diagonal field of view of each camera. Furthermore, each camera may be configured to focus at an object focal plane between 1mm and 30mm, e.g. at least 5mm and/or less than 11mm, e.g. 9mm-10mm, from the lens furthest away from the respective camera sensor. Similarly, in some applications, the illumination field of each structured light projector may be at least 45 degrees and optionally less than 120 degrees. The inventors have recognized that the large field of view achieved by combining the individual fields of view of all cameras can improve accuracy due to a reduced amount of image stitching errors, especially in the edentulous areas where the gum surface is smooth and there may be less sharp high resolution three-dimensional features. Having a larger field of view enables large smooth features, such as the general curvature of the tooth, to appear in each image frame, which improves the accuracy of stitching the various surfaces obtained from a plurality of such image frames. In some applications, the entire combined field of view of the various cameras (e.g., intraoral scanners) is between about 20mm and about 50mm along the longitudinal axis of the elongated hand-held wand, and about 20-40mm along the z-axis, where the z-axis may correspond to depth. In other applications, the field of view may be at least 20mm, at least 25mm, at least 30mm, at least 35mm, or at least 40mm along the longitudinal axis. In some embodiments, the combined field of view may vary with depth (e.g., with scan distance). For example, at a scan distance of about 4mm, the field of view may be about 40mm along the longitudinal axis, and at a scan distance of about 14mm, the field of view may be about 45mm along the longitudinal axis. If most of the motion of the intraoral scanner is done with respect to the long axis (e.g., longitudinal axis) of the scanner, the overlap between scans can be substantial. In some applications, the field of view of the combined cameras is discontinuous. For example, an intraoral scanner may have a first field of view separated from a second field of view by a fixed interval. The fixed spacing may be, for example, along the longitudinal axis of the elongated hand-held wand.

In some applications, a method is provided for generating a digital three-dimensional image of an intraoral surface. It should be noted that a "three-dimensional image" as a phrase used in the present application is based on a three-dimensional model (e.g., a point cloud) from which an image of the interior surface of a three-dimensional mouth is constructed. The resulting image, while typically displayed on a two-dimensional screen, contains data relating to the three-dimensional structure of the scanned object and thus can typically be manipulated to display the scanned object from different views and perspectives. In addition, data from the three-dimensional image may be used to make a physical three-dimensional model of the scanned object.

For example, one or more structured light projectors may be driven to project a distribution of discrete unconnected light points on an intraoral surface, and one or more cameras may be driven to capture the projected images. The image captured by each camera may include at least one spot of light.

Each camera comprises a camera sensor having an array of pixels, for each pixel there being a respective ray originating from that pixel in three-dimensional space, the direction of the ray being directed towards the object being imaged; when imaged on the sensor, each point along a particular one of these rays will fall on its corresponding respective pixel on the sensor. As used throughout this application, including in the claims, the term used herein is "camera light". Similarly, for each projected light point from each projector, there is a corresponding projector light ray. Each projector light ray corresponds to a respective pixel path on at least one camera sensor, i.e. if the camera sees a light spot projected by a particular projector light ray, that light spot must be detected by a pixel on the particular path of the pixel corresponding to that particular projector light ray. (a) The values of the camera light corresponding to each pixel on the camera sensor of each camera and (b) the values of the projector light corresponding to each projected light spot from each projector may be stored during a calibration process, as described below.

Based on the stored calibration values, the processor may be configured to run a corresponding algorithm to identify the three-dimensional position of each projected light point on the surface. For a given projector ray, the processor "looks" at the corresponding camera sensor path on one of the cameras. Each detected light spot along the camera sensor path will have a camera ray that intersects a given projector ray. The intersection points define three-dimensional points in space. The processor then searches among the camera sensor paths on the other cameras corresponding to the given projector ray and identifies how many other cameras have also detected a light spot on their respective camera sensor paths corresponding to the given projector ray, the camera ray of which intersects a three-dimensional point in space. As used throughout this application, a camera is considered to "agree" that a light spot is located at the same three-dimensional point in space if two or more cameras detect a light spot whose respective camera rays intersect a given projector ray at that three-dimensional point. Thus, the processor may identify the three-dimensional location of the projected light pattern based on the light pattern that two or more cameras agree that there is a projector ray projected at certain intersections. This process is repeated for additional spots along the camera sensor path and the maximum number of spots that the camera "agrees" are identified as the spots projected onto the surface from a given projector light. Thus, a three-dimensional position on the surface is calculated for the spot.

Once the location of a particular light spot on the surface is determined, the projector light that projected the light spot and all camera lights corresponding to the light spot may be disregarded and the corresponding algorithm may be run again for the next projector light. Finally, the identified three-dimensional locations can be used to generate a digital three-dimensional model of the intraoral surface.

In another example, a method of generating a digital three-dimensional model of an intraoral surface may include projecting a pattern of discrete unconnected light spots onto the intraoral surface of a patient using one or more light projectors in a probe disposed distal to an intraoral scanner, wherein the pattern of discrete unconnected light spots is non-coded. The method may further include capturing a plurality of images of the projected pattern of unconnected light points using two or more cameras disposed in the probe, decoding the plurality of images of the projected pattern to determine three-dimensional surface information of the intraoral surface, and generating a digital three-dimensional model of the intraoral surface using the three-dimensional surface information. Decoding the plurality of images can include accessing calibration data that associates camera rays corresponding to pixels on a camera sensor of each of the two or more cameras with a plurality of projector rays, wherein each of the plurality of projector rays is associated with one of the discrete unconnected light spots. The decoding may further include determining, using the calibration data, intersections of the projector rays and the camera rays corresponding to the pattern of projected discrete unconnected light points, wherein the intersections of the projector rays and the camera rays are associated with three-dimensional points in space. Decoding may also include identifying the three-dimensional location of the pattern of discrete unconnected light points based on two or more cameras agreeing that there are discrete unconnected light points projected by the projector light at certain intersections.

There is therefore provided, in accordance with some applications of the present invention, apparatus for intraoral scanning, the apparatus including:

an elongated hand-held wand including a probe at a distal end of the hand-held wand;

a rigid structure disposed within the probe distal end;

one or more structured light projectors coupled to the rigid structure; and

one or more cameras coupled to the rigid structure.

In some applications, each structured light projector may have an illumination field of 45-120 degrees. Alternatively, one or more structured light projectors may use a laser diode light source. In addition, the structured light projector may include beam shaping optics. Further, the structured light projector may comprise pattern generating optical elements.

The pattern generating optical element may be configured to generate a distribution of discrete unconnected light points. When a light source (e.g. a laser diode) is activated to emit light through the pattern generating optical element, a distribution of discrete unconnected light points may be generated at all planes between 1mm and 30mm from the pattern generating optical element. In some applications, the pattern generating optical element (i) generates the distribution using diffraction and/or refraction. Optionally, the pattern generating optical element has a luminous flux efficiency of at least 90%.

Further, in some applications, each camera may (a) have a field of view of 45-120 degrees. The camera may include a camera sensor and objective optics including one or more lenses. In some applications, the camera may be configured to focus at an object focal plane between 1mm and 30mm from the lens furthest from the camera sensor.

For some applications, each of the one or more cameras is configured to focus at an object focal plane between 5mm and 11mm from the lens that is furthest from the camera sensor.

For some applications, the pattern generating optical element of each of the one or more projectors is configured to generate a distribution of discrete unconnected light points at all planes between 4mm and 24mm from the pattern generating optical element when a light source (e.g., a laser diode) is activated to emit light through the pattern generating optical element.

For some applications, each of the one or more cameras is configured to focus at an object focal plane between 4mm and 24mm from the lens that is furthest from the camera sensor.

For some applications, each structured light projector has an illumination field of 70-100 degrees.

For some applications, each camera has a field of view of 70-100 degrees.

For some applications, each camera has a field of view of 80-90 degrees.

For some applications, the apparatus further includes at least one uniform light projector configured to project white light onto the object being scanned, and at least one camera is configured to capture a two-dimensional color image of the object using illumination from the uniform light projector.

For some applications, the beam shaping optics include a collimating lens.

For some applications, the structured light projectors and cameras are positioned such that each structured light projector faces an object placed outside of a wand (wand) in its illumination field. Alternatively, each camera may face objects outside the wand placed in its field of view. Furthermore, in some applications, at least 20% of the discrete unconnected light points are located in the field of view of at least one camera.

For some applications, the height of the probe is 10-15mm, where light enters the probe through a lower surface (or sensing surface) of the probe and the height of the probe is measured from the lower surface of the probe to an upper surface of the probe opposite the lower surface.

For some applications, the one or more structured light projectors are exactly one structured light projector, and the one or more cameras are exactly one camera.

For some applications, the pattern generating optical element comprises a Diffractive Optical Element (DOE).

For some applications, each DOE is configured to generate a distribution of discrete unconnected light points such that, when the light source is activated to emit light through the DOE, the ratio of illuminated area to non-illuminated area is 1:150-1:16 for each orthogonal plane in the illumination field.

For some applications, each DOE is configured to generate a distribution of discrete unconnected light points such that, when the light source is activated to emit light through the DOE, the ratio of illuminated area to non-illuminated area is 1:64-1:36 for each orthogonal plane in the illumination field.

For some applications, the one or more structured light projectors are a plurality of structured light projectors. In some applications each spot generated by a particular DOE has the same shape. Optionally, the shape of the light spot generated by at least one DOE is different from the shape of the light spot generated by at least one other DOE.

For some applications, each of the one or more projectors includes an optical element disposed between the beam shaping optical element and the DOE, the optical element configured to generate a Bessel (Bessel) beam when the laser diode is activated to emit light through the optical element such that the discrete unconnected spots remain less than 0.06mm in diameter through each inner surface of a sphere centered on the DOE and having a radius between 1mm and 30 mm.

For some applications, the optical element is configured to generate a Bessel beam when the laser diode is activated to emit light through the optical element such that the discrete unconnected spots remain less than 0.02mm in diameter through each inner surface of a geometric sphere centered on the DOE and having a radius between 1mm and 30 mm.

For some applications, each of the one or more projectors includes an optical element disposed between the beam shaping optical element and the DOE. The optical element may be configured to generate a Bessel (Bessel) beam when the light source is activated to emit light through the optical element such that the discrete unconnected spots remain small in diameter throughout the depth range. For example, in some applications discrete unconnected spots can be kept to a diameter of less than 0.06mm by each orthogonal plane that is between 1mm and 30mm from the DOE.

For some applications, the optical element is configured to generate a Bessel (Bessel) beam when the laser diode is activated to emit light through the optical element such that the discrete unconnected spots remain less than 0.02mm in diameter through each orthogonal plane that is between 1mm and 30mm from the DOE.

For some applications, the optical element is configured to generate a Bessel (Bessel) beam when the laser diode is activated to emit light through the optical element such that the discrete unconnected spots remain less than 0.04mm in diameter through each orthogonal plane that is between 4mm and 24mm from the DOE.

For some applications, the optical element is an axicon lens.

For some applications, the axicon lens is a diffractive axicon lens.

For some applications, the optical element is an annular aperture.

For some applications, the one or more structured light projectors are a plurality of structured light projectors, and the light sources of at least two structured light projectors are configured to emit light of two different wavelengths, respectively.

For some applications, the light sources of the at least three structured-light projectors are configured to emit light at three different wavelengths, respectively.

For some applications, the light sources of the at least three structured light projectors are configured to emit red, blue, and green light, respectively.

In some applications, the light source includes a laser diode.

For some applications, the one or more cameras are a plurality of cameras coupled to a rigid structure such that an angle between two respective optical axes of at least two cameras is 0-90 degrees.

For some applications, the angle between the two respective optical axes of the at least two cameras is 0-35 degrees.

For some applications, the one or more structured light projectors are a plurality of structured light projectors coupled to a rigid structure such that an angle between two respective optical axes of at least two structured light projectors is 0-90 degrees.

For some applications, the angle between the two respective optical axes of the at least two structured light projectors is 0-35 degrees.

For some applications, each camera has a plurality of discrete preset focus positions at each of which the camera is configured to focus at a respective object focal plane.

For some applications, each camera includes an autofocus actuator configured to select a focus position from a discrete preset focus position.

For some applications, each of the one or more cameras includes an optical aperture phase mask configured to extend a depth of focus (depth of focus) of the camera such that an image formed by each camera remains in focus at all object distances between 1mm and 30mm from a lens farthest from the camera sensor.

For some applications, the optical aperture phase mask is configured to extend the depth of focus of the cameras such that the image formed by each camera remains in focus at all object distances between 4mm and 24mm from the lens that is furthest from the camera sensor.

For some applications, each of the one or more cameras is configured to capture images at a frame rate of 30-200 frames per second.

For some applications, each of the one or more cameras is configured to capture images at a frame rate of at least 75 frames per second.

For some applications, each of the one or more cameras is configured to capture images at a frame rate of at least 100 frames per second.

For some applications, the laser diode of each of the one or more projectors is configured to emit an elliptical beam of light. The beam-shaping optical element of each of the one or more projectors may comprise a collimating lens. Optionally, the pattern generating optical element comprises a Diffractive Optical Element (DOE) which is divided into a plurality of sub-DOE sheets arranged as an array. Each sub-DOE sheet may generate a respective distribution of discrete unconnected light spots in a different region of the illumination field, such that the distribution of discrete unconnected light spots is generated when the light source is activated to emit light through the segmented DOE.

For some applications, the collimating lens may be configured to generate an elliptical beam having a major axis of 500-.

For some applications, when the laser diode is activated to emit light through the segmented DOE, the array of sub-DOE sheets may be positioned to be contained within the elliptical beam.

For some applications, the cross-section of each sub-DOE sheet is square with sides of 30-75 microns length, and the cross-section is perpendicular to the optical axis of the DOE.

For some applications, multiple sub-DOE sheets are arranged in a rectangular array, including 16-72 sub-DOE sheets and having a longest dimension of 500-800 microns.

For some applications, the collimating lens and the segmented DOE are a single optical element, a first side of the optical element includes the collimating lens, and a second side of the optical element, opposite the first side, includes the segmented DOE.

For some applications, the at least one light source of each of the one or more projectors is a plurality of laser diodes. In some applications, multiple laser diodes may be configured to emit light at the same wavelength.

For some applications, multiple laser diodes may be configured to emit light at different wavelengths.

For some applications, the plurality of laser diodes are two laser diodes configured to emit light at two different wavelengths, respectively.

For some applications, the plurality of laser diodes are three laser diodes configured to emit light at three different wavelengths, respectively.

For some applications, three laser diodes are configured to emit red, blue, and green light, respectively.

For some applications:

the beam-shaping optical element of each of the one or more projectors comprises a collimating lens, and

the pattern generating optical element includes a composite diffractive periodic structure having a periodic structure feature size of 100-400 nm.

For some applications, the collimating lens and the complex diffractive periodic structure are a single optical element, a first side of the optical element includes the collimating lens, and a second side of the optical element, opposite the first side, includes the complex diffractive periodic structure.

For some applications, the apparatus further includes an axicon lens disposed between the collimating lens and the composite diffractive periodic structure, the axicon lens having an axicon head angle of 0.2-2 degrees.

For some applications, the collimating lens has a focal length of 1.2-2 mm.

For some applications:

the beam-shaping optical element of each of the one or more projectors comprises a collimating lens, and

the pattern generating optical element includes a microlens array having a numerical aperture of 0.2-0.7.

For some applications, the microlens array is a hexagonal microlens array.

For some applications, the microlens array is a rectangular microlens array.

For some applications, the collimating lens and the microlens array are a single optical element, a first side of the optical element includes the collimating lens, and a second side of the optical element, opposite the first side, includes the microlens array.

For some applications, the apparatus further includes an axicon lens disposed between the collimating lens and the microlens array, the axicon lens having an axicon head angle of 0.2-2 degrees.

For some applications, the collimating lens has a focal length of 1.2-2 mm.

For some applications:

the beam-shaping optical element of each of the one or more projectors comprises a collimating lens, the collimating lens having a focal length of 1.2-2mm,

each of the one or more projectors includes an aperture ring disposed between a collimating lens and a pattern generating optical element, an

The pattern generating optical element includes a composite diffractive periodic structure having a periodic structure feature size of 100-400 nm.

For some applications:

the beam shaping optical element of each of the one or more projectors includes a lens (a) disposed between the laser diode and the pattern generating optical element, and (b) having a planar surface on a first side of the lens and an aspheric surface on a second side of the lens opposite the first side, the aspheric surfaces configured to generate a bessel beam directly from a diverging beam when the laser diode is activated to emit the diverging beam through the lens and the pattern generating optical element such that discrete unconnected light spots have a substantially uniform size at any orthogonal plane between 1mm and 30mm from the pattern generating optical element.

For some applications, the aspheric surface of the lens is configured to generate the bessel beam directly from the diverging beam when the laser diode is activated to emit the diverging beam through the lens and the pattern generating optical element such that the discrete unconnected spots have a substantially uniform size at any orthogonal plane between 4mm and 24mm from the pattern generating optical element.

For some applications, the pattern generating optical element includes a composite diffractive periodic structure having a periodic structure feature size of 100-400 nm.

For some applications, the pattern generating optical element includes a microlens array having a numerical aperture of 0.2-0.7.

For some applications:

(a) the beam-shaping optical element includes an aspheric surface on a first side of the lens, and (b) a planar surface on a second side of the lens opposite the first side is shaped to define a pattern-generating optical element, an

The aspheric surface is configured to generate a bessel beam directly from the divergent beam when the laser diode is activated to emit the divergent beam through the lens, such that the bessel beam is split into an array of discrete bessel beams when the laser diode is activated to emit the divergent beam through the lens, such that the discrete unconnected spots have a substantially uniform size at all planes between 1mm and 30mm from the lens.

For some applications, the planar surface of the lens is shaped to define a pattern-generating optical element such that the bessel beams are split into an array of discrete bessel beams when the laser diode is activated to emit a divergent beam through the lens such that the discrete unconnected spots have a substantially uniform size at all planes between 4mm and 24mm from the pattern-generating optical element.

For some applications, the apparatus and method may further comprise:

at least one temperature sensor coupled to the rigid structure and configured to measure a temperature of the rigid structure; and

a temperature control unit.

The temperature control circuit may be configured to (a) receive data from the temperature sensor indicative of a temperature of the rigid structure, and (b) activate the temperature control unit based on the received data. The temperature control unit and circuitry may be configured to maintain the probe and/or rigid structure at a temperature between 35 and 43 degrees celsius.

For some applications, the temperature control unit is configured to maintain the probe at a temperature between 37 and 41 degrees celsius.

For some applications, the temperature control unit is configured to prevent a temperature change of the probe from exceeding a threshold temperature change.

For some applications, the apparatus further comprises:

an object, such as a diffuse reflector, includes a plurality of regions disposed within a probe such that:

(a) each projector has at least one diffuse reflector area in its illumination field,

(b) each camera has at least one diffuse reflector region in its field of view, and (c) a plurality of diffuse reflector regions in the field of view of one of the cameras and in the field of illumination of one of the projectors.

In some applications, the temperature control circuit may be configured to (a) receive data from the camera indicating a position of the diffuse reflector relative to the distribution of discrete unconnected light points, (b) compare the received data to a stored calibrated position of the diffuse reflector, (i) a difference between the received data indicating the position of the diffuse reflector and (ii) the stored calibrated position of the diffuse reflector indicates a change in temperature of the probe, and (c) adjust the temperature of the probe based on the comparison of the received data to the stored calibrated position of the diffuse reflector.

There is also provided, in accordance with some applications of the present invention, a method for generating a digital three-dimensional image, the method comprising:

driving each of the one or more structured light projectors to project a distribution of discrete unconnected light points on an intraoral three-dimensional surface;

driving each of the one or more cameras to capture an image, the image comprising at least one spot of light, each of the one or more cameras comprising a camera sensor, the camera sensor comprising an array of pixels;

based on the stored calibration values, which indicate (a) camera rays corresponding to each pixel on the camera sensor of each of the one or more cameras, and (b) projector rays corresponding to each projected light spot from each of the one or more projectors, such that each projector ray corresponds to a respective pixel path on the at least one camera sensor:

using a processor, running a corresponding algorithm:

(1) for each projector light ray i, identifying, for each detected light spot j on the camera sensor path corresponding to the light ray i, how many other cameras detect, on their respective camera sensor paths corresponding to the light ray i, a respective light spot k corresponding to a respective camera light ray that intersects the light ray i and the camera light ray corresponding to the detected light spot j, such that the light ray i is identified as the particular projector light ray that generated the detected light spot j for which the largest number of other cameras detected the respective light spot k; and

(2) the respective three-dimensional position on the interior surface of the mouth is calculated as the intersection of the projector ray i and the respective camera ray corresponding to the detected spot j and the respective detected spot k. .

For some applications, running the corresponding algorithm using the processor further comprises, after step (1), using the processor to:

the projector light i and the respective camera light corresponding to the detected light spot j and the respective detected light spot k are not considered again; and

the corresponding algorithm is run again for the next projector ray i.

For some applications, driving each of the one or more structured light projectors to project a distribution of discrete unconnected light points includes driving each structured light projector to project 400-3000 discrete unconnected light points onto an intraoral three-dimensional surface.

For some applications, driving each of the one or more structured light projectors to project a distribution of discrete unconnected light points comprises driving a plurality of structured light projectors each to project a distribution of discrete unconnected light points, wherein:

(a) at least two structured light projectors configured to emit light of different wavelengths, an

(b) For each wavelength, the stored calibration values represent camera light corresponding to each pixel on the camera sensor.

For some applications, driving each of the one or more structured light projectors to project a distribution of discrete unconnected light points includes driving a plurality of structured light projectors each to project a distribution of discrete unconnected light points, wherein each light point projected from a particular structured light projector has the same shape and the shape of the light point projected from at least one structured light projector is different from the shape of the light point projected from at least one other structured light projector.

For some applications, the method further comprises:

driving at least one uniform light projector to project white light onto an intraoral three-dimensional surface; and

driving at least one camera to capture a two-dimensional color image of a three-dimensional surface within the mouth using illumination from the uniform light projector.

For some applications, the method further includes running, using the processor, a surface reconstruction algorithm that combines at least one image captured using illumination from the structured light projector with a plurality of images captured using illumination from the uniform light projector to generate a three-dimensional image of the intraoral three-dimensional surface.

For some applications, driving each of the one or more structured light projectors includes driving a plurality of structured light projectors to simultaneously project a distribution of respective discrete unconnected light points on an intraoral three-dimensional surface.

For some applications, driving each of the one or more structured-light projectors includes driving a plurality of structured-light projectors to project respective discrete unconnected light spots on the intraoral three-dimensional surface at different respective times.

For some applications, driving the plurality of structured light projectors to project the respective discrete unconnected light spots on the intraoral three-dimensional surface at different respective times includes driving the plurality of structured light projectors to project the respective discrete unconnected light spots on the intraoral three-dimensional surface in a predetermined sequence.

For some applications, driving the plurality of structured light projectors to project respective discrete unconnected light spots on the intraoral three-dimensional surface at different respective times includes:

driving at least one structured light projector to project a distribution of discrete unconnected light points on an intraoral three-dimensional surface; and

it is determined during the scan which of the plurality of structured light projectors is next driven to project the distribution of discrete unconnected light points.

For some applications:

driving each of the one or more structured light projectors includes driving exactly one structured light projector to project a distribution of discrete unconnected light points on an intraoral three-dimensional surface.

For some applications, driving each of the one or more cameras includes driving the one or more cameras at a frame rate of 30-200 frames per second such that each captures an image.

For some applications, driving the one or more cameras includes driving the one or more cameras at a frame rate of at least 75 frames per second such that each captures an image.

For some applications, driving the one or more cameras includes driving the one or more cameras at a frame rate of at least 100 frames per second such that each captures an image.

For some applications, using the processor includes selecting between sets of stored calibration data corresponding to a plurality of respective temperatures of the structured light projector and the camera based on data received from the temperature sensors indicative of temperatures of the structured light projector and the camera, each set of stored calibration data indicating for a respective temperature (a) a projector light line corresponding to each projected light spot from each of the one or more projectors, and (b) a camera light line corresponding to each pixel on the camera sensor of each of the one or more cameras.

For some applications, using the processor includes interpolating between the plurality of sets of stored calibration data based on data received from the temperature sensors indicative of temperatures of the structured light projector and the camera to obtain calibration data for temperatures between respective temperatures corresponding to each set of calibration data.

For some applications:

driving each of the one or more cameras includes driving each of the one or more cameras to capture an image that also includes at least one region of a diffuse reflector having a plurality of regions such that:

(a) each projector has at least one diffuse reflector area in its illumination field,

The processor may be configured to (a) receive data from the camera indicating a position of the diffuse reflector relative to the distribution of discrete unconnected light points, (b) compare the received data to a stored calibration position of the diffuse reflector, (i) a difference between the received data indicating the position of the diffuse reflector and (ii) the stored calibration position of the diffuse reflector indicates a deviation of the projector light and the camera light from their respective stored calibration values, and (c) run a corresponding algorithm based on the deviations of the projector light and the camera light from their respective stored calibration values.

In some embodiments, such as any of those described above or throughout the specification, combined structure illumination using light field imaging may provide high dynamic range three-dimensional imaging. The fringe pattern may be projected onto the scene and modulated by the scene depth. The structured light field can then be detected using a light field recording device. The structured light field contains information about the ray direction and the phase-coded depth, by means of which the scene depth can be estimated from different directions. Multi-directional depth estimation can effectively achieve highly dynamic three-dimensional imaging.

Applications of the present invention may also include systems and methods relating to three-dimensional intraoral scanning devices including one or more light field cameras and one or more pattern projectors. For example, in some embodiments, an intraoral scanning device is provided. The device may include an elongated hand-held wand including a probe at a distal end. The probe may have a proximal end and a distal end. During intraoral scanning, the probe may be placed in the oral cavity of a subject. According to some applications of the present invention, the structured light projector and the light field camera may be disposed at a proximal end of the probe and the mirror disposed at a distal end of the probe. The structured light projector and the light field camera can be positioned to face the mirror, and the mirror is positioned to (a) reflect light from the structured light projector directly onto the object being scanned, and (b) reflect light from the object being scanned to the light field camera.

A structured light projector in the proximal end of the probe includes a light source. In some applications, the light source may have an illumination field of at least 6 degrees and/or less than 30 degrees. The structured light projector may focus light from the light source at a projector focal plane that is at least 30mm and/or less than 140mm from the light source. The structured light projector may further include a pattern generator disposed in an optical path between the light source and the projector focal plane, the pattern generator generating a structured light pattern at the projector focal plane when the light source is activated to emit light through the pattern generator.

In some applications, the light field camera in the proximal end of the probe may have a field of view of at least 6 degrees and/or less than 30 degrees. The light field camera may be focused at a camera focal plane that is at least 30mm and/or less than 140mm from the light field camera. The light field camera may also include a light field camera sensor that includes (i) an image sensor that includes an array of sensor pixels, and (ii) a microlens array disposed in front of the image sensor such that each microlens lens is disposed on a sub-array of sensor pixels. An objective lens disposed in front of the light field camera sensor forms an image of the scanned object onto the light field camera sensor.

According to some applications of the present invention, one or more structured light projectors and one or more light field cameras are disposed at a distal end of the probe. The structured light projectors and light field cameras are positioned such that each structured light projector directly faces an object placed outside the wand in its illumination field and each camera directly faces an object placed outside the wand in its field of view. At least 40% of the projected structured light pattern from each projector is in the field of view of at least one camera.

One or more structured light projectors in the distal end of the probe each include a light source. In some applications, the respective structured light projectors may each have an illumination field of at least 60 degrees and/or less than 120 degrees. Each structured light projector may focus light from the light source at a projector focal plane that is at least 30mm and/or less than 140mm from the light source. Each structured light projector may further include a pattern generator disposed in an optical path between the light source and the projector focal plane, the pattern generator generating a structured light pattern at the projector focal plane when the light source is activated to emit light through the pattern generator.

In some applications, the one or more light field cameras in the distal end of the probe may each have a field of view of at least 60 degrees and/or less than 120 degrees. Each light field camera may be focused at a camera focal plane that is at least 3mm and/or less than 40mm from the light field camera. Each light field camera may also include a light field camera sensor that includes (i) an image sensor including an array of sensor pixels, and (ii) a microlens array disposed in front of the image sensor such that each microlens is disposed on a sub-array of sensor pixels. An objective lens disposed in front of each light field camera sensor forms an image of the scanned object onto the light field camera sensor.

There is therefore provided, in accordance with some applications of the present invention, apparatus for intraoral scanning, the apparatus including:

(A) an elongated hand-held wand including a probe at a distal end of the hand-held wand, the probe having a proximal end and a distal end;

(B) structured light projector, set up the near-end at the probe, this structured light projector:

(a) with an illumination field of 6 to 30 degrees,

(b) comprises a light source, an

(c) Is configured to focus light from the light source at a projector focal plane between 30mm and 140mm from the light source, and

(d) including a pattern generator disposed in an optical path between the light source and the projector focal plane, the pattern generator configured to generate a structured-light pattern at the projector focal plane when the light source is activated to emit light through the pattern generator.

(a) having a field of view of 6 to 30 degrees,

(b) configured to focus at a camera focal plane between 30mm and 140mm from the light field camera,

(c) including a light field camera sensor that includes (i) an image sensor including an array of sensor pixels, and (ii) a microlens array disposed in front of the image sensor such that each microlens is disposed over a sub-array of sensor pixels, an

(d) Comprises an objective lens arranged in front of the light field camera sensor and configured to form an image of a scanned object onto the light field camera sensor; and

(D) a mirror disposed at a distal end of the hand-held wand,

the structured light projector and the light field camera are positioned to face the mirror, and the mirror is positioned to (a) reflect light from the structured light projector directly onto the object being scanned, and (b) reflect light from the object being scanned into the light field camera.

For some applications, the light source includes a Light Emitting Diode (LED), and the pattern generator includes a mask.

For some applications, the light source includes a laser diode.

For some applications, the pattern generator includes a Diffractive Optical Element (DOE) configured to generate the structured light pattern as a distribution of discrete unconnected light points.

For some applications, the pattern generator includes a refractive microlens array.

For some applications, the height of the probe is 14-17mm, the width of the probe is 18-22mm, the height and width defining a plane perpendicular to the longitudinal axis of the rod, light enters the probe through a lower surface of the probe, and the height of the probe is measured from the lower surface of the probe to an upper surface of the probe opposite the lower surface.

For some applications, the apparatus is configured for use with an output apparatus, the apparatus further comprising:

a control circuit configured to:

(a) the structured light projector is driven to project a structured light pattern onto an object external to the wand,

(b) driving a light field camera to capture a light field generated by a structured-light pattern reflected from an object, the light field including (i) an intensity of the structured-light pattern reflected from the object, and (ii) a direction of a light ray; and

at least one computer processor configured to reconstruct a three-dimensional image of a surface of the scanned object based on the captured light field and output the image to an output device.

For some applications:

(a) the object external to the wand is a tooth within the mouth of the subject,

(b) the control circuit is configured to drive the light field camera to capture a light field generated by a structured light pattern reflected from the tooth in the absence of powder on the tooth, and

(c) the computer processor is configured to reconstruct a three-dimensional image of the tooth based on the light field captured without powder on the tooth and output the image to an output device.

For some applications, the subarray of each sensor pixel in a central area of the image sensor includes 10-40% fewer pixels than the subarray of each sensor pixel in a peripheral area of the image sensor, the central area of the image sensor including at least 50% of the total number of sensor pixels.

For some applications, a depth at which (a) each microlens on the sub-array of sensor pixels disposed in the peripheral region of the image sensor is configured to focus is 1.1-1.4 times greater than a depth at which (b) each microlens on the sub-array of sensor pixels disposed in the central region of the image sensor is configured to focus.

There is also provided, in accordance with some applications of the present invention, apparatus, including:

(A) an elongated hand-held wand including a probe at a distal end of the hand-held wand, the probe having a proximal end and a distal end;

(B) one or more structured light projectors disposed at the distal end of the probe, each structured light projector:

(a) with an illumination field of 60 to 120 degrees,

(b) comprises a light source, an

(c) Is configured to focus light from the light source at a projector focal plane between 3mm and 40mm from the light source, and

(d) a pattern generator disposed in an optical path between the light source and the projector focal plane, the pattern generator configured to generate a structured-light pattern at the projector focal plane when the light source is activated to emit light through the pattern generator; and

(a) having a field of view of 60 to 120 degrees,

(b) configured to focus at a camera focal plane between 3mm and 40mm from the light field camera,

(d) Comprises an objective lens arranged in front of the light field camera sensor and configured to form an image of a scanned object onto the light field camera sensor; and

the structured light projectors and light field cameras are positioned such that (a) each structured light projector directly faces an object placed outside the wand in its illumination field, (b) each camera directly faces an object placed outside the wand in its field of view, and (c) at least 40% of the structured light pattern from each projector is located in the field of view of at least one camera.

For some applications, the height of the probe is 10-14mm, the width of the probe is 18-22mm, the height and width defining a plane perpendicular to the longitudinal axis of the rod, light enters the probe through a lower surface of the probe, and the height of the probe is measured from the lower surface of the probe to an upper surface of the probe opposite the lower surface.

For some applications, the one or more structured light projectors are exactly one structured light projector, and the one or more structured light field cameras are exactly one light field camera.

For some applications, the one or more structured light projectors are a plurality of structured light projectors and the one or more light field cameras are a plurality of light field cameras.

For some applications, the apparatus is configured for use with an output apparatus, the apparatus further comprising:

a control circuit configured to:

(a) driving each of the one or more structured light projectors to project a structured light pattern onto an object external to the wand,

(b) driving one or more light field cameras to capture a light field generated by a structured-light pattern reflected from an object, the light field including (i) an intensity of the structured-light pattern reflected from the object, and (ii) a direction of a light ray; and

at least one computer processor configured to reconstruct a three-dimensional image of a surface of the scanned object based on the captured light field and output the image to an output device.

For some applications:

at least one of the one or more structured light projectors is a monochromatic structured light projector configured to project a monochromatic structured light pattern onto a scanned object,

at least one of the one or more light field cameras is a monochromatic light field camera configured to capture a light field generated by a monochromatic structured light pattern reflected from an object being scanned, an

The device also includes: (a) a light source configured to emit white light onto an object being scanned; and (b) a camera configured to capture a two-dimensional color image of the scanned object.

For some applications, the monochromatic structured light projector is configured to project a structured light pattern at a wavelength of 420-470 nm.

There is also provided, in accordance with some applications of the present invention, apparatus, including:

(A) an elongated hand-held wand including a probe at a distal end of the hand-held wand, the probe having a proximal end and a distal end;

(B) structured light projector, set up the near-end at the probe, this structured light projector:

(a) the device is provided with an irradiation field,

(b) comprises a light source, an

(a) having a field of view,

(b) configured to focus at a camera focal plane,

(d) Comprises an objective lens arranged in front of the light field camera sensor and configured to form an image of a scanned object onto the light field camera sensor; and

(D) a mirror disposed at a distal end of the hand-held wand,

the structured light projector and the light field camera are positioned to face the mirror, and the mirror is positioned to (a) reflect light from the structured light projector directly onto the scanned object, and (b) reflect light from the scanned object into the light field camera.

There is also provided, in accordance with some applications of the present invention, apparatus, including:

(A) an elongated hand-held wand including a probe at a distal end of the hand-held wand, the probe having a proximal end and a distal end;

(B) one or more structured light projectors disposed at the distal end of the probe, each structured light projector:

(a) the device is provided with an irradiation field,

(b) comprises a light source, an

(a) having a field of view,

(b) configured to focus at a camera focal plane,

(d) Comprises an objective lens arranged in front of the light field camera sensor and configured to form an image of a scanned object onto the light field camera sensor; and

The present invention will be more fully understood from the following detailed description, taken together with the accompanying drawings, in which the application thereof is illustrated.

Drawings

FIG. 1 is a schematic illustration of a hand-held wand (hand-held wand) in which a plurality of structured light projectors and cameras are disposed within a probe at the distal end of the hand-held wand in accordance with some applications of the present invention;

2A-B are schematic diagrams of positioning configurations of a camera and a structured light projector, respectively, according to some applications of the present invention;

FIG. 2C is a chart depicting a number of different configurations of the positions of a structured light projector and a camera in a probe, in accordance with some applications of the present invention;

FIG. 3 is a schematic view of a structured light projector according to some applications of the present invention;

FIG. 4 is a schematic diagram of a structured light projector projecting a distribution of discrete unconnected light points onto a plurality of object focal planes in accordance with some applications of the present invention;

5A-B are schematic diagrams of structured light projectors including beam shaping optics and additional optics disposed between the beam shaping optics and the pattern generating optics according to some applications of the present invention;

6A-B are schematic diagrams of a structured light projector projecting discrete unconnected light spots and a camera sensor detecting the light spots in accordance with some applications of the present invention;

FIG. 7 is a flow chart summarizing a method for generating a digital three-dimensional image, in accordance with some applications of the present invention;

FIG. 8 is a flow chart summarizing a method for performing certain steps in the method of FIG. 7, in accordance with some applications of the present invention;

FIGS. 9, 10, 11 and 12 are schematic diagrams depicting simplified examples of the steps of FIG. 8, in accordance with some applications of the present invention;

FIG. 13 is a flow chart summarizing further steps in a method for generating a digital three-dimensional image according to some applications of the present invention;

FIGS. 14, 15, 16 and 17 are schematic diagrams depicting simplified examples of the steps of FIG. 13, in accordance with some applications of the present invention;

FIG. 18 is a schematic view of a probe including a diffuse reflector according to some applications of the present invention;

19A-B are schematic diagrams of a cross-section of a structured light projector and a light beam emitted by a laser diode showing a pattern generating optical element disposed in the optical path of the light beam in accordance with some applications of the present invention;

FIGS. 20A-E are schematic illustrations of microlens arrays for use as pattern generating optical elements in structured light projectors according to some applications of the present invention;

21A-C are schematic diagrams of a compound 2-D diffractive periodic structure for use as a pattern generating optical element in a structured light projector according to some applications of the present invention;

22A-B are schematic diagrams illustrating a single optical element having an aspheric first side and a planar second side opposite the first side and a structured light projector including the optical element according to some applications of the present invention;

23A-B are schematic diagrams of axicon lenses and structured light projectors comprising axicon lenses according to some applications of the present invention;

24A-B are schematic diagrams illustrating optical elements having aspheric surfaces on a first side and flat surfaces on a second side opposite the first side and structured light projectors including optical elements according to some applications of the present invention;

FIG. 25 is a schematic view of a single optical element in a structured light projector according to some applications of the present invention;

26A-B are schematic diagrams of structured light projectors with more than one laser diode according to some applications of the present invention;

27A-B are schematic diagrams of different ways of combining laser diodes of different wavelengths according to some applications of the present invention;

FIG. 28A is a schematic view of a hand-held wand with a structured light projector and a light field camera disposed at the proximal end of the hand-held wand and a mirror disposed within a probe at the distal end of the hand-held wand in accordance with some applications of the present invention;

FIG. 28B is a schematic view of the hand-held wand of FIG. 28A showing the probe within the mouth of a subject, in accordance with some applications of the present invention;

29A-B are schematic diagrams of structured light projectors according to some applications of the present invention;

FIG. 30 is a schematic illustration of a light field camera and a captured three-dimensional object according to some applications of the present invention;

FIG. 31 is a schematic view of a handheld wand with a structured light projector and a light field camera disposed within a probe at a distal end of the handheld wand in accordance with some applications of the present invention; and

FIG. 32 is a schematic view of a hand-held wand in which a plurality of structured light projectors and light field cameras are disposed within a probe at the distal end of the hand-held wand, according to some applications of the present invention.

Detailed Description

Referring now to fig. 1, fig. 1 is a schematic illustration of an elongated hand-held wand 20 for intraoral scanning according to some applications of the present invention. The plurality of structured light projectors 22 and the plurality of cameras 24 are coupled to a rigid structure 26, the rigid structure 26 being disposed within a probe 28 at a distal end 30 of the hand-held wand. In some applications, probe 28 enters the subject's mouth during an intraoral scan.

For some applications, the structured light projectors 22 are positioned within the probe 28 such that each structured light projector 22 faces an object 32 positioned outside the hand held wand 20 that is placed in its illumination field, rather than positioning the structured light projectors in the proximal end of the hand held wand and illuminating the object by reflecting light through mirrors and then onto the object. Similarly, for some applications, cameras 24 are positioned within probe 28 such that each camera 24 faces an object 32 positioned outside of hand-held wand 20 in its field of view, rather than positioning the camera in the proximal end of the hand-held wand and viewing the object by reflecting light into the camera through a mirror. This positioning of the projector and camera within the probe 28 enables the scanner to have an overall large field of view while maintaining a low profile probe.

In some applications, the height H1 of the probe 28 is less than 15mm, and the height H1 of the probe 28 is measured from the lower surface 176 (sensing surface) through which reflected light from the scanned object 32 enters the probe 28 to the upper surface 178 opposite the lower surface 176. In some applications, the height H1 is between 10-15 mm.

In some applications, cameras 24 each have a large field of view β (beta) of at least 45 degrees, e.g., at least 70 degrees, e.g., at least 80 degrees, e.g., 85 degrees. In some applications, the field of view may be less than 120 degrees, e.g., less than 100 degrees, e.g., less than 90 degrees. In experiments conducted by the inventors, the field of view β (beta) of each camera, between 80 and 90 degrees, was found to be particularly useful as it provides a good balance between pixel size, field of view and camera overlap, optical quality and cost. The camera 24 may include a camera sensor 58 and objective optics 60 including one or more lenses. To enable near focus imaging, the camera 24 may focus at an object focal plane 50, the object focal plane 50 being between 1mm and 30mm, e.g. between 4mm and 24mm, e.g. between 5mm and 11mm, e.g. 9mm-10mm, from the lens that is furthest away from the camera sensor. In experiments conducted by the inventors, it was found that an object focal plane 50 between 5mm and 11mm from the lens furthest from the camera sensor is particularly useful because it is easy to scan the teeth at this distance and because the focus of most tooth surfaces is good. In some applications, camera 24 may capture images at a frame rate of at least 30 frames per second, for example, at a frame rate of at least 75 frames per second, for example, at a frame rate of at least 100 frames per second. In some applications, the frame rate may be less than 200 frames per second.

As described above, the large field of view achieved by combining the respective fields of view of all cameras may improve accuracy due to a reduced amount of image stitching errors, particularly in the edentulous areas where the gum surface is smooth and there may be fewer sharp high resolution three-dimensional features. Having a larger field of view enables large smooth features, such as the general curvature of the tooth, to appear in each image frame, which improves the accuracy of stitching the various surfaces obtained from a plurality of such image frames.

Similarly, the structured light projectors 22 may each have a large illumination field α (alpha) of at least 45 degrees, for example, at least 70 degrees. In some applications, the illumination field α (alpha) may be less than 120 degrees, for example, less than 100 degrees. Other features of the structured light projector 22 are described below.

For some applications, to improve image capture, each camera 24 has a plurality of discrete preset focus positions at each of which the camera is focused at a respective object focal plane 50. Each camera 24 may include an autofocus actuator that selects a focal position from discrete preset focal positions to improve a given image capture. Additionally or alternatively, each camera 24 includes an optical aperture phase mask (optical aperture phase mask) that extends the depth of focus of the camera such that the image formed by each camera remains in focus over all object distances between 1mm and 30mm, e.g., between 4mm and 24mm, e.g., between 5mm and 11mm, e.g., 9mm to 10mm, from the lens furthest from the camera sensor.

In some applications, the structured light projector 22 and the cameras 24 are coupled to the rigid structure 26 in a closely packed and/or alternating manner such that (a) a major portion of each camera's field of view overlaps with the field of view of an adjacent camera, (b) a major portion of each camera's field of view overlaps with the illumination field of an adjacent projector. Optionally, at least 20%, e.g., at least 50%, e.g., at least 75%, of the projected light pattern is in the field of view of the at least one camera at an object focal plane 50, the object focal plane 50 being at least 4mm from the lens that is furthest from the camera sensor. Due to the different configurations of the projector and cameras possible, some of the projected patterns may never be seen in the field of view of any camera, and some of the projected patterns may be blocked from view by the object 32 during scanning when the scanner is moved.

The rigid structure 26 may be a non-flexible structure to which the structured light projector 22 and the camera 24 are coupled to provide structural stability to the optics within the probe 28. The incorporation of all projectors and all cameras into a common rigid structure helps to maintain the geometric integrity of the optics of each structured light projector 22 and each camera 24 under varying environmental conditions (e.g., under mechanical stress that may be caused by the subject's mouth). In addition, the rigid structure 26 helps maintain stable structural integrity and positioning of the structured light projector 22 and the camera head 24 relative to each other. As described further below, controlling the temperature of rigid structure 26 helps maintain the geometric integrity of the optics over a wide range of ambient temperatures as probe 28 enters and exits the subject's mouth or as the subject breathes during scanning.

Referring now to FIGS. 2A-B, FIGS. 2A-B are illustrations of a positioning arrangement for a camera 24 and a structured light projector 22, respectively, according to some applications of the present inventionSchematic illustration of the device. For some applications, to improve the overall field of view and illumination field of the intraoral scanner, the camera 24 and structured light projector 22 are positioned such that they do not all face in the same direction. For some applications, such as shown in fig. 2A, the plurality of cameras 24 are coupled to the rigid structure 26 such that an angle θ (theta) between two respective optical axes 46 of at least two cameras 24 is 90 degrees or less, e.g., 35 degrees or less. Similarly, for some applications, such as shown in FIG. 2B, the plurality of structured light projectors 22 are coupled to the rigid structure 26 such that an angle between two respective optical axes 48 of at least two of the structured light projectors 22(phi) is 90 degrees or less, for example 35 degrees or less.

Referring now to FIG. 2C, FIG. 2C is a diagram depicting a number of different configurations of the positions of the structured light projector 22 and the camera 24 in the probe 28 according to some applications of the present invention. The structured light projector 22 is represented by a circle in fig. 2C and the camera 24 is represented by a rectangle in fig. 2C. Note that a rectangle is used to represent a camera, since typically, the field of view β (beta) of each camera sensor 58 and each camera 24 has a 1: 2, in the longitudinal direction. Column (a) of fig. 2C shows a bird's eye view of various configurations of the structured light projector 22 and the camera 24. The x-axis marked in the first row of column (a) corresponds to the central longitudinal axis of probe 28. Column (b) shows a side view of camera head 24 in various configurations as viewed from a line of sight coaxial with the central longitudinal axis of probe 28. Similar to fig. 2A, column (b) of fig. 2C shows cameras 24 positioned such that optical axes 46 are at an angle of 90 degrees or less, e.g., 35 degrees or less, relative to each other. Column (c) shows a side view of camera head 24 in various configurations as viewed from a line of sight perpendicular to the central longitudinal axis of probe 28.

Typically, the most distal (toward the positive x-direction in fig. 2C) and most proximal (toward the negative x-direction in fig. 2C) cameras 24 are positioned such that their optical axes 46 are rotated slightly inward, e.g., at an angle of 90 degrees or less, e.g., 35 degrees or less, relative to the next closest camera 24. The more centrally located cameras 24, i.e., neither the most distal camera 24 nor the most proximal camera 24, are positioned so as to face directly outward of the probe, with their optical axes 46 substantially perpendicular to the central longitudinal axis of the probe 28. It should be noted that in row (xi), the projector 22 is located at the most distal position of the probe 28, so the optical axis 48 of that projector 22 is directed inwardly, allowing a greater number of spots 33 projected from that particular projector 22 to be seen by more cameras 24.

In general, the number of structured light projectors 22 in the probe 28 may range from two (e.g., as shown in row (iv) of FIG. 2C) to six (e.g., as shown in row (xii)). In general, the number of cameras 24 in probe 28 may range from four (e.g., as shown in rows (iv) and (v)) to seven (e.g., as shown in row (ix)). Note that the various configurations shown in fig. 2C are by way of example only, and not by way of limitation, and the scope of the present invention includes additional configurations not shown. For example, the scope of the invention includes more than five projectors 22 located in the probe 28 and more than seven cameras located in the probe 28.

In an exemplary application, an apparatus for intraoral scanning (e.g., an intraoral scanner) includes an elongated hand-held wand including: a probe located at a distal end of the elongated hand-held wand; at least two light projectors disposed within the probe; and at least four cameras arranged in the probe. Each light projector may include: at least one light source configured to generate light when activated; and a pattern generating optical element configured to generate a light pattern when emitting light through the pattern generating optical element. Each of the at least four cameras may include a camera sensor and one or more lenses, wherein each of the at least four cameras is configured to capture a plurality of images depicting at least a portion of the light pattern projected on the interior surface of the mouth. The majority of the at least two light projectors and the at least four cameras may be arranged in at least two rows, each row being substantially parallel to the longitudinal axis of the probe, the at least two rows comprising at least a first row and a second row.

In a further application, a distal-most camera along the longitudinal axis and a proximal-most camera along the longitudinal axis of the at least four cameras are positioned such that their optical axes are at an angle of 90 degrees or less relative to each other from a line of sight perpendicular to the longitudinal axis. The cameras in the first row and the cameras in the second row may be positioned such that from a line of sight coaxial with the longitudinal axis of the probe, the optical axes of the cameras in the first row are at an angle of 90 degrees or less relative to the optical axes of the cameras in the second row. The remaining portions of the at least four cameras, except for the most distal camera and the most proximal camera, have optical axes that are substantially parallel to the longitudinal axis of the probe. Each of the at least two rows may include an alternating sequence of light projectors and cameras.

In a further application, the at least four cameras comprise at least five cameras, the at least two light projectors comprise at least five light projectors, the most proximal component in the first row is a light projector, and the most proximal component in the second row is a camera.

In a further application, the distal-most camera along the longitudinal axis and the proximal-most camera along the longitudinal axis are positioned such that their optical axes are at an angle of 35 degrees or less relative to each other from a line of sight perpendicular to the longitudinal axis. The cameras in the first row and the cameras in the second row may be positioned such that from a line of sight coaxial with the longitudinal axis of the probe, the optical axes of the cameras in the first row are at an angle of 35 degrees or less relative to the optical axes of the cameras in the second row.

In a further application, the at least four cameras may have a combined field of view along the longitudinal axis of 25-45mm and a field of view along the z-axis of 20-40mm, corresponding to the distance from the probe.

Referring now to fig. 3, fig. 3 is a schematic diagram of a structured light projector 22 according to some applications of the present invention. In some applications, the structured light projector 22 includes a laser diode 36, beam shaping optics 40, and pattern generation optics 38, the pattern generation optics 38 generating a distribution 34 of discrete unconnected light points (discussed further below with reference to fig. 4). In some applications, the structured light projector 22 may be configured to generate the distribution 34 of discrete unconnected light spots at all planes between 1mm and 30mm, for example between 4mm and 24mm, from the pattern generating optical element 38 when the laser diode 36 emits light through the pattern generating optical element 38. For some applications, the distribution 34 of discrete unconnected light spots is focused on one plane located between 1mm and 30mm, e.g., between 4mm and 24mm, while all other planes located between 1mm and 30mm, e.g., other planes between 4mm and 24mm, still contain discrete unconnected light spots. While the above description is described as using laser diodes, it should be understood that this is an exemplary and non-limiting application. Other light sources may be used in other applications. Furthermore, although described as projecting a pattern of discrete unconnected spots, it should be understood that this is an exemplary and non-limiting application. Other light patterns or arrays, including but not limited to lines, grids, checkerboards, and other arrays, may be used in other applications.

The pattern generation optical element 38 may be configured to have a light conversion efficiency (i.e., the ratio of light entering the pattern to total light falling on the pattern generation optical element 38) of at least 80%, for example, at least 90%.

For some applications, the respective laser diodes 36 of each structured light projector 22 emit light at different wavelengths, i.e., the respective laser diodes 36 of at least two structured light projectors 22 emit light at two different wavelengths, respectively. For some applications, the respective laser diodes 36 of the at least three structured light projectors 22 each emit light at three different wavelengths. For example, red, blue and green laser diodes may be used. For some applications, the respective laser diodes 36 of at least two structured light projectors 22 each emit light at two different wavelengths. For example, in some applications, six structured light projectors 22 are provided within the probe 28, three of which include blue laser diodes and three of which include green laser diodes.

Referring now to fig. 4, fig. 4 is a schematic diagram of a structured light projector 22 that projects a distribution of discrete unconnected light points onto multiple object focal planes in accordance with some applications of the present invention. The scanned object 32 may be one or more teeth or other intraoral objects/tissues within the subject's mouth. The somewhat translucent and smooth nature of the tooth may affect the contrast of the projected structured light pattern. For example, (a) some light hitting a tooth may scatter to other regions within the intraoral scene, resulting in some amount of stray light, and (b) some light may penetrate the tooth and subsequently exit the tooth at any other point. Thus, to improve image capture of an intraoral scene under structured light illumination without using contrast enhancing means such as coating teeth with opaque powder, the inventors have recognized that a sparse distribution 34 of discrete unconnected spots may provide an improved balance between reducing the amount of projected light while maintaining a useful amount of information. The sparsity of distribution 34 can be characterized by the following ratio of (a) to (b):

(a) the illuminated area on the orthogonal plane 44 in the illumination field alpha (alpha), i.e., the sum of the areas of all the projected spots 33 on the orthogonal plane 44 in the illumination field alpha (alpha),

(b) the non-illuminated area on the orthogonal plane 44 in the illumination field α (alpha). In some applications, the sparsity ratio may be at least 1:150 and/or less than 1:16 (e.g., at least 1:64 and/or less than 1: 36).

In some applications, each structured light projector 22 projects at least 400 discrete unconnected light spots 33 onto an intraoral three-dimensional surface during scanning. In some applications, each structured light projector 22 projects less than 3000 discrete unconnected light spots 33 onto the intraoral surface during scanning. In order to reconstruct a three-dimensional surface from the sparse distribution 34 of projections, the correspondence between the individual projected spots 33 and the spots detected by the camera 24 must be determined, as described further below with reference to fig. 7 to 19.

For some applications, pattern-generating optical element 38 is a Diffractive Optical Element (DOE)39 (fig. 3) that generates distribution 34 of discrete unconnected spots 33 when laser diode 36 emits light through the DOE onto object 32. As used herein throughout the application, including in the claims, a spot is defined as a small area of light having any shape. For some applications, the respective DOEs 39 of the differently structured light projector 22 generate spots having different respective shapes, i.e. each spot 33 generated by a particular DOE39 has the same shape, and the shape of a spot 33 generated by at least one DOE39 is different from the shape of a spot 33 generated by at least one other DOE 39. For example, some of the DOEs 39 may generate a circular spot 33 (such as shown in fig. 4), some of the DOEs 39 may generate a square spot, and some of the DOEs 39 may generate an elliptical spot. Alternatively, some DOEs 39 may generate a pattern of connected or unconnected lines.

Referring now to fig. 5A-B, fig. 5A-B are schematic illustrations of a structured light projector 22 according to some applications of the present invention, the structured light projector 22 including a beam-shaping optical element 40 and additional optical elements disposed between the beam-shaping optical element 40 and the pattern-generating optical element 38 (e.g., DOE 39). Optionally, the beam shaping optical element 40 is a collimating lens 130. The collimating lens 130 may be configured to have a focal length of less than 2 mm. Alternatively, the focal length may be at least 1.2 mm. For some applications, an additional optical element 42 disposed between beam shaping optical element 40 and pattern generating optical element 38 (e.g., DOE 39) generates a bessel beam when laser diode 36 emits light through optical element 42. In some applications, the bessel beams are transmitted through DOE39 such that all discrete unconnected spots 33 remain small in diameter (e.g., less than 0.06mm, such as less than 0.04mm, such as less than 0.02mm), pass through a series of orthogonal planes 44 (e.g., each orthogonal plane is located between 1mm and 30mm from DOE39, such as between 4mm and 24mm from DOE39, etc.). In the context of the present patent application, the diameter of the spot 33 is defined as the full width at half maximum (FWHM) of the spot intensity.

Although all the spots described above are smaller than 0.06mm, some spots having diameters close to the upper end of these ranges (e.g. only slightly smaller than 0.06mm or 0.02mm) and also close to the edges of the illumination field of the projector 22 may be elongated when they intersect in a geometrical plane orthogonal to the DOE 39. For this case, it is useful to measure their diameters when they intersect the inner surface of a geometric sphere centered on DOE39 and having a radius between 1mm and 30mm, which corresponds to the distance from the respective orthogonal plane of DOE39 of between 1mm and 30 mm. As used throughout this application, including in the claims, the term "geometry" is intended to relate to a theoretical geometric configuration (e.g., a plane or sphere) and is not part of any physical device.

For some applications, when the bessel beam is transmitted through the DOE39, in addition to a spot less than 0.06mm in diameter, a spot 33 greater than 0.06mm in diameter is generated.

For some applications, the optical element 42 is an axicon lens 45, such as that shown in fig. 5A and described further below with reference to fig. 23A-B. Alternatively, the optical element 42 may be an annular aperture ring (annular aperture ring)47, such as shown in fig. 5B. Maintaining a small diameter spot improves the three-dimensional resolution and accuracy of the overall depth of focus. Without the optical element 42 (e.g., axicon lens 45 or annular aperture ring 47), the size of the spot 33 can change, e.g., become larger, as you move away from the plane of best focus due to diffraction and defocus.

Referring now to fig. 6A-B, fig. 6A-B are schematic illustrations of a structured light projector 22 projecting discrete unconnected light spots 33 and a camera sensor 58 detecting light spots 33' according to some applications of the present invention. For some applications, a method is provided for determining correspondence between projected spots 33 on the intraoral surface and detected spots 33' on respective camera sensors 58. Once the correspondence is determined, a three-dimensional image of the surface is reconstructed. Each camera sensor 58 has an array of pixels, with a corresponding camera ray 86 for each pixel. Similarly, for each projected spot 33 from each projector 22, there is a corresponding projector light ray 88. Each projector light ray 88 corresponds to a respective path 92 of pixels on at least one camera sensor 58. Thus, if the camera sees a spot 33 'projected by a particular projector light ray 88, that spot 33' will necessarily be detected by a pixel on a particular path 92 corresponding to that particular projector light ray 88 pixel. With particular reference to fig. 6B, a correspondence between each projector light ray 88 and a corresponding camera sensor path 92 is shown. Projector light rays 88 'correspond to camera sensor path 92', projector light rays 88 "correspond to camera sensor path 92", and projector light rays 88 "'correspond to camera sensor path 92"'. For example, if a particular projector ray 88 projects a spot of light into a dust-filled space, a dust line in the air will be illuminated. This dust line detected by the camera sensor 58 will follow the same path on the camera sensor 58 as the camera sensor path 92 corresponding to the particular projector light line 88.

During the calibration process, calibration values are stored based on camera rays 86 corresponding to pixels on the camera sensor 58 of each camera 24 and projector rays 88 corresponding to the projected spots 33 from each structured light projector 22. For example, calibration values are stored for (a) a plurality of camera rays 86 corresponding to a respective plurality of pixels on the camera sensor 58 of each camera 24, and (b) a plurality of projector rays 88 corresponding to a respective plurality of projected spots 33 from each structured light projector 22.

For example, the following calibration procedure may be used. A high-precision point target, such as a black point on a white background, is irradiated from below, and an image of the target is taken with all the cameras. The point target is then moved vertically towards the camera, i.e. along the z-axis to the target plane. The point centers of all points in all corresponding z-axis positions are computed to create a three-dimensional grid of points in space. The distortion and camera pinhole model are then used to find the pixel coordinates of each three-dimensional location of the corresponding point center, thus defining for each pixel a camera ray as a ray originating from a pixel oriented towards the corresponding point center in the three-dimensional grid. The camera rays corresponding to the pixels between the grid points may be interpolated. The camera calibration process described above is repeated for all of the respective wavelengths of the individual laser diodes 36 such that for each wavelength, included in the stored calibration values is a camera ray 86 corresponding to each pixel on each camera sensor 58.

After the camera 24 is calibrated and all camera light 86 values are stored, the structured light projector 22 may be calibrated as follows. A flat featureless target is used and the structured light projector 22 is turned on one at a time. Each spot is located on at least one camera sensor 58. Since the camera 24 is now calibrated, the three-dimensional spot position of each spot is calculated by triangulation based on images of the spots in a number of different cameras. The above process is repeated with featureless targets located at a plurality of different z-axis positions. Each projected light point on the featureless target will define projector light rays originating from the projector in space.

Referring now to fig. 7, fig. 7 is a flow chart summarizing a method for generating a digital three-dimensional image according to some applications of the present invention. In steps 62 and 64 of the method outlined in fig. 7, each structured light projector 22 is driven to project a distribution 34 of discrete unconnected light spots 33 on an intraoral three-dimensional surface, and each camera 24 is driven to capture an image comprising at least one light spot 33. Based on the stored calibration values indicative of (a) the camera light rays 86 corresponding to each pixel on the camera sensor 58 of each camera 24, and (b) the projector light rays 88 corresponding to each projected spot 33 from each structured light projector 22, a corresponding algorithm is run in step 66 using the processor 96 (fig. 1), as will be further described below with reference to fig. 8-12. Once the correspondence is resolved, the three-dimensional position on the intraoral surface is calculated in step 68 and used to generate a digital three-dimensional image of the intraoral surface. Furthermore, capturing an intraoral scene using multiple cameras 24 provides a signal with improved noise in capture by a factor of the square root of the number of cameras.

Referring now to fig. 8, fig. 8 is a flow chart summarizing the corresponding algorithm of step 66 in fig. 7, according to some applications of the present invention. Based on the stored calibration values, all projector rays 88 and all camera rays 86 corresponding to all detected light spots 33' are mapped (step 70), and all intersection points 98 (fig. 10) of at least one camera ray 86 and at least one projector ray 88 are identified (step 72). Fig. 9 and 10 are schematic diagrams of simplified examples of steps 70 and 72, respectively, of fig. 8. As shown in fig. 9, three projector rays 88 are mapped together with eight camera rays 86, corresponding to a total of eight detected spots 33' on the camera sensor 58 of the camera 24. As shown in fig. 10, sixteen intersection points 98 are identified.

In steps 74 and 76 of fig. 8, the processor 96 determines a correspondence between the projected spots 33 and the detected spots 33' in order to identify the three-dimensional position of each projected spot 33 on the surface. FIG. 11 is a schematic diagram depicting steps 74 and 76 of FIG. 8 using the simplified example described in the previous paragraph. For a given projector ray i, the processor 96 "looks" at the corresponding camera sensor path 90 on the camera sensor 58 of one of the cameras 24. Each detected spot j along the camera sensor path 90 will have a camera ray 86 that intersects a given projector ray i at an intersection point 98. The intersection 98 defines a three-dimensional point in space. The processor 96 then "looks" at the camera sensor paths 90 'corresponding to the given projector ray i on the respective camera sensors 58' of the other cameras 24 and identifies how many other cameras 24 have also detected a respective light point k on their respective camera sensor paths 90 'corresponding to the given projector ray i, the camera ray 86' of which intersects the same three-dimensional point in space defined by the intersection point 98. This process is repeated for all detected spots j along the camera sensor path 90 and the maximum number of cameras 24 "agreeing" spots j are identified as spots 33 (fig. 12) projected onto the surface from a given projector ray i. That is, the projector light i is identified as the particular projector light 88 that generates the detected spot j for which the highest number of other cameras detected the corresponding spot k. Thus, the three-dimensional position of the spot 33 on the surface is calculated.

For example, as shown in fig. 11, all four cameras detect respective light points on respective camera sensor paths corresponding to projector ray i, with their respective camera rays intersecting projector ray i at intersection point 98, intersection point 98 being defined as the intersection point of camera ray 86 and projector ray i corresponding to detected light point j. Thus, all four cameras are said to "agree" that there is a spot 33 projected by the projector ray i at the intersection 98. However, when the process is repeated for the next spot j ', none of the other cameras detect a respective spot on their respective camera sensor paths corresponding to the projector ray i, and its respective camera ray intersects the projector ray i at an intersection point 98', which intersection point 98 'is defined as the intersection point of the camera ray 86 "(corresponding to the detected spot j') and the projector ray i. Thus, only one camera is said to "agree" that there is a spot 33 projected by the projector light ray i at the intersection 98', while four cameras "agree" that there is a spot 33 projected by the projector light ray i at the intersection 98. Thus, the projector ray i is identified as the particular projector ray 88 (fig. 12) that generated the detected spot j by projecting the spot 33 onto the surface at the intersection 98. According to step 78 of fig. 8, and as shown in fig. 12, a three-dimensional position 35 on the interior surface of the mouth is calculated at an intersection point 98.

Referring now to fig. 13, fig. 13 is a flow chart summarizing other steps in a corresponding algorithm according to some applications of the present invention. Once the position 35 on the surface is determined, the projection ray i of the projection spot j, and all the camera rays 86 and 86' corresponding to the spot j and the corresponding spot k are not considered (step 80), and the corresponding algorithm is run again for the next projector ray i (step 82). Fig. 14 depicts a simplified example described above after the removal of a particular projector ray i that projects a spot 33 at a position 35. The corresponding algorithm is then run again for the next projector ray i, according to step 82 in the flowchart of fig. 13. As shown in fig. 14, the remaining data shows that three cameras "agree" that there is a spot 33 at the intersection 98, the intersection 98 being defined by the intersection of the camera ray 86 corresponding to the detected spot j and the projector ray i. Thus, as shown in fig. 15, the three-dimensional position 37 is calculated at the intersection 98.

As shown in fig. 16, once the three-dimensional location 37 on the surface is determined, the projected ray i that projects spot j, and all camera rays 86 and 86' corresponding to spot j and the corresponding spot k, are not considered. The remaining data shows that there is a spot 33 projected by the projector ray i at the intersection 98 and that the three-dimensional position 41 on the surface is calculated at the intersection 98. As shown in fig. 17, according to a simplified example, the three projected spots 33 of the three projector rays 88 of the structured light projector 22 have now been located at three-dimensional locations 35, 37 and 41 on the surface. In some applications, each structured light projector 22 projects 400-3000 spots 33. Once all projector rays 88 correspondences have been resolved, a reconstruction algorithm may be used to reconstruct a digital image of the surface using the calculated three-dimensional positions of the projected spots 33.

Reference is again made to fig. 1. For some applications, there is at least one uniform light projector 118 bonded to the rigid structure 26. The uniform light projector 118 emits white light onto the object 32 being scanned. At least one camera (e.g., one of the cameras 24) is configured to capture a two-dimensional color image of the object 32 using illumination from the uniform light projector 118. The processor 96 may run a surface reconstruction algorithm that combines at least one image captured using illumination from the structured light projector 22 with a plurality of images captured using illumination from the uniform light projector 118 to generate a three-dimensional image of the intraoral three-dimensional surface. Using a combination of structured light and uniform illumination enhances the overall capture of the intraoral scanner and may help reduce the number of options that the processor 96 needs to consider when running the corresponding algorithm.

For some applications, multiple structured light projectors 22 are driven simultaneously to project their respective distributions 34 of discrete unconnected light spots 33 on an intraoral three-dimensional surface. Alternatively, a plurality of structured light projectors 22 may be driven to project their respective distributions 34 of discrete unconnected light spots 33 on the intraoral three-dimensional surface at different respective times, for example, in a predetermined order, or in an order that is dynamically determined during scanning. Alternatively, for some applications, a single structured light projector 22 may be driven to project the distribution 34.

Dynamically determining which structured light projector 22 to activate during a scan may improve the overall signal quality of the scan, as some structured light projectors may have better signal quality in some areas within the oral cavity relative to other areas. For example, when scanning the upper jaw (maxillary region) of a subject, a red projector tends to have better signal quality than a blue projector. In addition, regions of the oral cavity that are difficult to see may be encountered during scanning, such as areas of missing teeth or narrow crevices between large teeth. In these types of cases, dynamically determining which structured light projector 22 to activate during scanning allows for activating a particular projector that may have a better line of sight.

For some applications, different structured light projectors 22 may be configured to focus at different object focal planes. Dynamically determining which structured light projector 22 to activate during a scan allows for the activation of particular structured light projectors 22 according to their respective object focal planes depending on the distance from the region currently being scanned.

For some applications, all data points acquired at a particular time are used as a rigid point cloud (rigid point cloud), and a plurality of such point clouds are captured at a frame rate of more than 10 captures per second. The multiple point clouds are then stitched together using a registration algorithm (e.g., Iterative Closest Point (ICP)) to create a dense point cloud. A surface reconstruction algorithm may then be used to generate a representation of the surface of the object 32.

For some applications, at least one temperature sensor 52 is coupled to rigid structure 26 and measures the temperature of rigid structure 26. A temperature control circuit 54 disposed within the hand-held wand 20 (a) receives data from the temperature sensor 52 indicative of the temperature of the rigid structure 26, and (b) activates the temperature control unit 56 in response to the received data. A temperature control unit 56 (e.g., a PID controller) maintains the probe 28 at a desired temperature (e.g., between 35 and 43 degrees celsius, between 37 and 41 degrees celsius, etc.). Keeping the probe 28 at 35 degrees celsius or higher, for example, 37 degrees celsius or higher reduces fogging of the glass surface of the hand-held wand 20, and when the probe 28 enters the oral cavity, the structured light projector 22 projects light through the glass surface of the hand-held wand 20, and the camera 2 observes, the oral cavity being typically at about 37 degrees celsius or higher. Keeping the probe 28 below 43 degrees, e.g., below 41 degrees celsius, prevents discomfort or pain.

In addition, to use the stored calibration values for the camera light and projector light during scanning, temperature variations of the camera 24 and structured light projector 22 may be prevented, thereby preserving the geometric integrity of the optics. Changes in temperature may cause the length of the probe 28 to change due to thermal expansion, which in turn may cause corresponding camera and projector position shifts. Distortions may also be generated due to different types of stresses that may accumulate within the probe 28 during such thermal expansion, resulting in an offset in the angles of the respective camera and projector light. Within cameras and projectors, geometric changes may also be generated due to temperature changes. For example, DOE39 may expand and change the projected pattern, temperature changes may affect the refractive index of the camera lens, or temperature changes may change the wavelength emitted by laser diode 36. Thus, in addition to maintaining probe 28 at a temperature within the above-described range, temperature control unit 56 may also prevent the temperature of probe 28 from varying by more than 1 degree when using hand-held wand 20, thereby maintaining the geometric integrity of the optics disposed within probe 28. For example, if temperature control unit 56 maintains probe 28 at a temperature of 39 degrees Celsius, temperature control unit 56 will further ensure that the temperature of probe 28 is not below 38 degrees Celsius or above 40 degrees Celsius during use.

For some applications, probe 28 is maintained at its controlled temperature by using a combination of heating and cooling. For example, the temperature control unit 56 may include a heater, such as a plurality of heaters, and a cooler, such as a thermoelectric cooler. If the temperature of probe 28 falls below 38 degrees Celsius, a heater may be used to increase the temperature of probe 28, and if the temperature of probe 28 is above 40 degrees Celsius, a thermoelectric cooler may be used to decrease the temperature of probe 28.

Alternatively, for some applications, probe 28 is maintained at its controlled temperature by using only heating and not cooling. The use of the laser diode 36 and diffractive and/or refractive pattern generating optical elements helps to maintain an energy efficient structured light projector, limiting the probe 28 from heating up during use; the laser diode 36 may use less than 0.2 watts of power while emitting at high brightness, and the diffractive and/or refractive pattern generating optical element utilizes all of the emitted light (e.g., as opposed to a mask that prevents some rays of light from striking the object). However, external ambient temperatures, such as those encountered within the oral cavity of a subject, may cause heating of probe 28. To overcome this, heat may be drawn from the probe 28 by a thermally conductive element 94 (e.g., a heat pipe) disposed within the hand-held wand 20 such that a distal end 95 of the thermally conductive element 94 is in contact with the rigid structure 26 and a proximal end 99 is in contact with a proximal end 100 of the hand-held wand 20. Thus, heat is transferred from the rigid structure 26 to the proximal end 100 of the hand piece 20. Alternatively or additionally, a fan disposed in the handle area 174 of the wand 20 may be used to draw heat away from the probe 28.

For some applications, alternatively or additionally, processor 96 may select between sets of calibration data corresponding to different temperatures, respectively, in order to maintain geometric integrity of the optics by preventing temperature changes of probe 28 from exceeding threshold changes in temperature. For example, the threshold change may be 1 degree celsius. Based on the data indicative of the temperatures of the structured light projector 22 and the camera 24 received from the temperature sensor 52, the processor 96 may select between multiple sets of stored calibration data corresponding to multiple respective temperatures of the structured light projector 22 and the camera 24, each set of stored calibration data indicating for the respective temperature (a) a projector light corresponding to each projected light spot from each of the one or more projectors, and (b) a camera light corresponding to each pixel on the camera sensor of each of the one or more cameras. If processor 96 only accesses stored calibration data for a particular plurality of temperatures, processor 96 may interpolate between sets of stored calibration data based on data received from temperature sensor 52 to obtain calibration data corresponding to temperatures between respective temperatures for each set of calibration data.

Referring now to fig. 18, fig. 18 is a schematic illustration of a probe 28 according to some applications of the present invention. For some applications, the probe 28 also includes a target, such as a diffuse reflector 170, the diffuse reflector 170 having a plurality of regions 172 disposed within the probe 28 (or adjacent the probe 28, as shown in FIG. 18). In some applications, (a) each structured light projector 22 may have at least one region 172 of the diffuse reflector 170 in its illumination field, (b) each camera 24 has at least one region 172 of the diffuse reflector 170 in its field of view, and (c) multiple regions 172 of the diffuse reflector 170 are in the field of view of the camera 24 and in the illumination field of the structured light projector 22. Alternatively or additionally, to maintain geometric integrity of the optics by preventing temperature changes of probe 28 from exceeding a threshold temperature change, processor 96 may (a) receive data from camera 24 indicative of the position of the diffuse reflector relative to distribution 34 of discrete unconnected light spots 33, (b) compare the received data to a stored calibrated position of diffuse reflector 170, wherein a difference between (i) the received data indicative of the position of diffuse reflector 170 and (ii) the stored calibrated position of diffuse reflector 170 is indicative of an offset of projector ray 88 and camera ray 86 from their respective stored calibration values, and (c) run a corresponding algorithm based on the offset of projector ray 88 and camera ray 86.

Alternatively or additionally, a difference between (i) the received data indicative of the position of the diffuse reflector 170 and (ii) the stored calibrated position of the diffuse reflector 170 may be indicative of a change in temperature of the probe 28. In this case, the temperature of probe 28 may be adjusted based on a comparison of the received data of diffuse reflector 170 with the stored calibration position.

Several applications of the structured light projector 22 are described below.

Referring now to fig. 19A-B, fig. 19A-B are schematic illustrations of a cross-section of a structured light projector 22 and a light beam 120 emitted by a laser diode 36, showing a pattern generating optical element 38 disposed in the optical path of the light beam, according to some applications of the present invention. In some applications, each laser diode 36 emits an elliptical beam 120 having an elliptical cross-section with (a) a major axis of at least 500 microns and/or less than 700 microns and (b) a minor axis of at least 100 microns and/or less than 200 microns. For some applications a small area beam splitter may be used to generate a tightly focused array of spots, for example a DOE with a side length of less than 100 microns may be used in order to keep the projected spots 33 tightly focused throughout the focus range of interest. However, such a small DOE will only utilize a portion of the light emitted by the elliptical laser beam 120.

Thus, for some applications, pattern-generating optical element 38 is a segmented DOE 122, which segmented DOE 122 is segmented into a plurality of sub-DOE sheets 124 arranged in an array. The array of sub-DOE sheets 124 is positioned such that (a) is contained within the elliptical beam 120, and (b) utilizes a high percentage, e.g., at least 50%, of the light emitted by the elliptical beam 120. In some applications, the array is a rectangular array, includes at least 16 and/or less than 72 sub-DOE sheets 124, and has a longest dimension of at least 500 microns and/or less than 800 microns. Each sub-DOE sheet 124 may have a square cross-section with sides of a length of at least 30 microns and/or less than 75 microns, the cross-section being taken perpendicular to the optical axis of the DOE.

Each sub-DOE sheet 124 generates a respective distribution 126 of discrete unconnected spots 33 in a different region 128 of the illumination field. For such an application of the structured light projector 22, the distribution 34 of discrete unconnected light spots 33 as described above with reference to fig. 4 is a combination of the respective distributions 126 generated by the respective sub-DOE sheets 124. Fig. 19B shows the orthogonal plane 44 on which the respective distributions 126 of discrete unconnected spots 33 are shown, each respective distribution 126 being located in a different region 128 of the illumination field. Since each sub-DOE sheet 124 is responsible for a different region 128 of the illumination field, each sub-DOE sheet 124 has a different design in order to point its respective distribution 126 in a different direction and avoid beam crossing to avoid overlap between the projected spots 33.

Referring now to fig. 20A-E, fig. 20A-E are schematic illustrations of microlens array 132 as pattern-generating optical element 38, according to some applications of the present invention. The microlens array can be used as a spot generator because it is periodic and the profile variation of each lens in the array is wavelength-scaled. The pitch of the microlens array 132 is adjusted to obtain the desired angular spacing (angular pitch) between the spots. As described above, the Numerical Aperture (NA) of the microlens array 132 is adjusted to provide a desired angular illumination field. In some applications, the NA of microlens array 132 is at least 0.2 and/or less than 0.7. The microlens array 132 may be, for example, a hexagonal microlens array as shown in fig. 20C, or a rectangular microlens array as shown in fig. 20E.

The structured light projector 22 having a microlens array 132 as the pattern generating optical element 38 may include a laser diode 36, a collimating lens 130, an aperture, and a microlens array 132. The aperture defines a smaller input beam diameter to maintain a tightly focused spot at a close focal length from the microlens array 132, e.g., at least 1mm and/or less than 30mm, e.g., at least 4mm and/or less than 24 mm. Fig. 20B shows collimated laser beams illuminating microlens array 132, which then generates diverging beams 134, whose interference generates array of spots 33, e.g., distribution 34 (fig. 20D). For some applications, the aperture is a chrome film (chrome film) applied to the laser diode side of the collimating lens 130. Alternatively, for some applications, the aperture is a chrome film disposed on the collimating lens side of the microlens array 132. In some applications, the aperture may span a distance of at least 10 times the pitch of the microlens array 132 and have a diameter of at least 50 microns and/or less than 200 microns.

Referring now to fig. 21A-C, fig. 21A-C are schematic diagrams of a composite two-dimensional diffractive periodic structure 136 (e.g., a diffraction grating such as a Dammann (Dammann) grating) as a pattern generating optical element 38 according to some applications of the present invention. The composite diffractive periodic structure 136 can have a periodic structure feature size 137 of at least 100nm and/or less than 400 nm. A large illumination field as described above can be obtained by small sub-features of about 300 nm. The period of the composite diffractive periodic structure 136 can be adjusted to provide a desired angular separation of the projected beams.

The structured light projector 22 having the complex diffractive periodic structure 136 as the pattern generating optical element 38 may include a laser diode 36, a collimating lens 130, an aperture, and the complex diffractive periodic structure 136. The aperture defines a smaller input beam diameter to maintain a tightly focused spot at a close focal length, e.g., at least 1mm and/or less than 30mm, e.g., at least 4mm and/or less than 24mm, from the complex diffractive periodic structure 136. For some applications, the aperture is a chrome film on the periodic structure features of the composite diffractive periodic structure 136. In some applications, the aperture may span a distance of at least 10 periods of the composite diffractive periodic structure 136 and have a diameter of at least 50 microns and/or less than 200 microns.

For some applications, beam shaping optics 40 (such as shown in FIG. 3) is a collimating lens 130 disposed between laser diode 36 and pattern generating optics 38. With respect to the application described above with reference to fig. 19A-B, 20A-E and 21A-C, the collimating lens 130 may be disposed between the laser diode 36 and the segmented DOE 122 (fig. 19A), between the laser diode 36 and the microlens array 132 (fig. 20A), and between the laser diode 36 and the composite diffractive periodic structure 136 (fig. 21A).

Referring now to fig. 22A-B, fig. 22A-B are schematic diagrams illustrating a single optical element 138 and a structured light projector 22 including the optical element 138, the single optical element 138 having an aspheric first side and a planar second side opposite the first side, in accordance with some applications of the present invention. For some applications, the collimating lens 130 and the pattern generating optical element 38 may be fabricated as a single optical element 138, the first aspheric side 140 of which collimates the light emitted from the laser diode 36, and the second planar side 142 of which generates the distribution 34 of discrete unconnected spots 33. The planar side 142 of the single optical element 138 may be shaped to define the DOE39, the segmented DOE 122, the microlens array 132, or the composite diffractive periodic structure 136.

Referring now to fig. 23A-B, fig. 23A-B are schematic illustrations of axicon lens 144 and structured light projector 22 including axicon lens 144 according to some applications of the present invention. Axicon lenses are known to generate bessel beams, which are beams focused within a desired depth range according to the input beam diameter and the axicon angle. For some applications, axicon lens 144, which has a nose angle γ (gamma) of at least 0.2 degrees and/or less than 2 degrees, is disposed between collimating lens 130 and pattern-generating optical element 38. When the laser diode 36 emits light through the axicon lens 144, the axicon lens 144 generates a focused Bessel beam 146. The focused Bezier beam 146 is split by the pattern-generating optics 38 into a number of beams 148, each beam 148 being an exact copy of the Bezier beam 146 generated by the axicon lens 144. The pattern generating optical element 38 may be a DOE39, a microlens array 132, or a complex diffractive periodic structure 136.

Referring now to fig. 24A-B, fig. 24A-B are schematic diagrams illustrating an optical element 150 and a structured light projector 22 including the optical element 150, the optical element 150 having an aspheric surface 152 on a first side and a planar surface on a second side opposite the first side, according to some applications of the present invention. For some applications, the collimating lens 130 and the axicon lens 144 may be fabricated as a single optical element 150. When the laser diode 36 emits light through the optical element 150, the aspheric surface 152 of the single optical element 150 generates a bessel beam directly from the diverging beam. The distribution 34 of discrete unconnected spots 33 is then generated as the light travels through the pattern generating optical element 38, such that the discrete unconnected spots 33 have a substantially uniform size at any orthogonal plane between 1mm and 30mm, for example between 4mm and 24mm, from the pattern generating optical element 38. The pattern generating optical element 38 may be a DOE39, a microlens array 132, or a complex diffractive periodic structure 136. As used throughout this application, including in the claims, a spot having a "substantially uniform size" means that the size of the spot does not vary by more than 40%.

Referring now to fig. 25, fig. 25 is a schematic view of a single optical element 154 in the structured light projector 22 according to some applications of the present invention. For some applications, a single optical element 154 may perform the functions of a collimating lens, an axicon lens, and a pattern generating optical element. The single optical element 154 includes an aspheric surface 156 on a first side and a planar surface 158 on a second side opposite the first side. When the laser diode 36 emits a diverging beam through the single optical element 154, the aspheric surface 156 generates a bessel beam directly from the diverging beam. The planar surface 158 is shaped to define the pattern generating optical element 38 and thus to divide the bessel beams into an array of discrete bessel beams 160 so as to generate the distribution 34 of discrete unconnected spots 33 such that the discrete unconnected spots 33 have a substantially uniform size at any orthogonal plane between 1mm and 30mm, for example between 4mm and 24mm, from the single pattern optical element 154. The planar surface 158 may be shaped to define the DOE39, the microlens array 132, or the complex diffractive periodic structure 136.

Referring now to fig. 26A-B, fig. 26A-B are schematic diagrams of structured light projectors 22 having more than one light source (e.g., laser diode 36) according to some applications of the present invention. When a laser diode is used, laser speckle (laser spot) may generate spatial noise. Speckle effect (speckle effect) is the result of interference of many waves of the same frequency but different phase and amplitude. When all the waves are superimposed, the composite wave is a wave whose amplitude varies randomly over the beam profile. For some applications, speckle effects may be reduced by combining multiple laser diodes 36 of the same wavelength. Different lasers with the same wavelength are not coherent with each other, so combining them into the same space or the same diffractive beam splitter 162 will reduce speckle by a factor of at least the square root of the number of different laser diodes 36.

The beam splitter 162 may be a standard 50/50 splitter that reduces the efficiency of the two beams to below 50%, or a Polarizing Beam Splitter (PBS) that maintains greater than 90% efficiency. For some applications, each laser diode 36 may have its own collimating lens 130, as shown in FIG. 26A. Alternatively, a plurality of laser diodes 36 may share a collimating lens 130 disposed between the beam splitter 162 and the pattern generating optical element 38, as shown in fig. 26B. The pattern generating optical element 38 may be a DOE39, a segmented DOE 122, a microlens array 132, or a complex diffractive periodic structure 136.

As described above, the sparse distribution 34 improves capture by providing an improved balance between reducing the amount of light projected while maintaining a useful amount of information. For some applications, multiple laser diodes 36 with different wavelengths may be combined in order to provide a higher density pattern without reducing trapping. For example, each structured light projector 22 may include at least two, e.g., at least three, laser diodes 36, the laser diodes 36 emitting light at different respective wavelengths. Although the projected spots 33 may in some cases almost overlap, the color discrimination capability of the camera sensor can be used to spatially resolve spots of different colors. Alternatively, red, blue and green laser diodes may be used. All of the structured light projector configurations described above may be implemented using a plurality of laser diodes 36 in each structured light projector 22.

Referring now to fig. 27A-B, fig. 27A-B are schematic diagrams of different ways of combining laser diodes of different wavelengths according to some applications of the present invention. Two or more lasers of different wavelengths may be combined into the same diffractive element using a fiber coupler 164 (fig. 27A) or a laser combiner 166 (fig. 27B). For laser combiner 166, the combining element may be a dichroic bi-directional or tri-directional dichroic combiner. Within each structured light projector 22, all of the laser diodes 36 emit light through a common pattern generating optical element 38 at the same time or at different times. The individual laser beams may hit slightly different locations in the pattern generating optical element 38 and generate different patterns. These patterns do not interfere with each other due to different colors, different pulse times or different angles. The use of fiber coupler 164 or laser combiner 166 allows laser diode 36 to be located in a remote housing 168. A distal housing 168 may be provided at the proximal end of the hand-held wand 20 to allow for a smaller probe 28.

For some applications, the structured light projector 22 and the camera 24 may be disposed in the proximal end 100 of the probe 28.

The following description relates primarily to applications of the invention including light field cameras.

Referring now to fig. 28A, fig. 28A is a schematic diagram of an intraoral scanner 1020 in accordance with some applications of the present invention. The intraoral scanner 1020 includes an elongated hand-held wand 1022 having a probe 1028 at a distal end 1026 of the hand-held wand 1022. The probe 1028 has a distal end 1027 and a proximal end 1024. As used throughout this application, including in the claims, the proximal end of the hand-held wand is defined as the end of the hand-held wand closest to the user's hand when the user is holding the hand-held wand in the ready-to-use position, and the distal end of the hand-held wand is defined as the end of the hand-held wand furthest from the user's hand when the user is holding the hand-held wand in the ready-to-use position.

For some applications, a single structured light projector 1030 is disposed in the proximal end 1024 of the probe 1028, a single light field camera 1032 is disposed in the proximal end 1024 of the probe 1028, and a mirror 1034 is disposed in the distal end 1027 of the probe 1028. The structured light projector 1030 and the light field camera 1032 are positioned to face a mirror 1034, and the mirror 1034 is positioned to reflect light from the structured light projector 1030 directly onto the scanned object 1036 and from the scanned object 1036 into the light field camera 1032.

The structured light projector 1030 includes a light source 1040. In some applications, the structured light projector 1030 may have an illumination field ψ (psi) of at least 6 degrees and/or less than 30 degrees. In some applications, the structured light projector 1030 focuses light from the light source 1040 at a projector focal plane 1038 (e.g., as shown in fig. 29A-B) that is at least 30mm and/or less than 140mm from the light source 1040. The structured light projector 1030 can have a pattern generator 1042 disposed in the optical path between the light source 1040 and the projector focal plane 1038. When the light sources 1040 are activated to emit light through the pattern generator 1042, the pattern generator 1042 generates a structured light pattern at the projector focal plane 1038.

Light field camera 1032 may have a field of view ω (omega) of at least 6 degrees and/or less than 30 degrees. Light field camera 1032 can be focused at a camera focal plane 1039 (e.g., as shown in fig. 30) that is at least 30mm and/or less than 140mm from light field camera 1032. The light field camera 1032 has a light field camera sensor 1046, the light field camera sensor 1046 including an image sensor 1048 and a microlens array 1050 disposed in front of the image sensor 1048, the image sensor 1048 including an array of pixels, such as a CMOS image sensor, the microlens array 1050 having each microlens 1050 disposed over a sub-array 1052 of sensor pixels. Additionally, light field camera 1032 has an objective lens 1054 disposed in front of light field camera sensor 1048, which objective 1054 forms an image of scanned object 1036 onto light field camera sensor 1046.

The intra-oral scanner 1020 can include control circuitry 1056 that (a) drives the structured light projector 1030 to project a structured light pattern onto an object 1036 external to the hand-held wand 1022, and (b) drives the light field camera 1032 to capture a light field generated by the structured light pattern reflected from the object 1036. The structured light field contains information about the intensity and ray direction of the structured light pattern reflected by the object 1036. The light field also contains information about the phase-coded depth, by means of which the scene depth can be estimated from different directions. Using information from the captured light field, the computer processor 1058 may reconstruct a three-dimensional image of the surface of the object 1036 and may output the image to an output device 1060, such as a monitor. Note that the computer processor 1058 is shown in fig. 28A, 31, and 32 as being external to the handheld wand 1022 in an illustrative, but non-limiting manner. For other applications, the computer processor 1058 may be disposed within the handheld wand 1022.

In some applications, the object 1036 being scanned is at least one tooth within the subject's mouth. As noted above, dentists often coat a subject's teeth with opaque powders in order to improve image capture when using a digital intraoral scanner. The light field camera 1032 in the intraoral scanner 1020 can capture the light field from the structured light pattern reflected from the teeth in the absence of such powder on the teeth, enabling a simpler digital intraoral scanning experience.

When the structured light projector 1030 and the light field camera 1032 are disposed in the proximal end 1024 of the probe 1028, the size of the probe 1028 is limited by the angle at which the mirror 1034 is positioned. In some applications, the height H2 of the probe 1028 is less than 17mm, the width W1 of the probe 1028 is less than 22mm, and the height H2 and the width W1 define a plane perpendicular to the longitudinal axis 1067 of the hand-held bar 1022. Further, the height H2 of the probe 1028 is measured from the lower surface 1070 (scanning surface) through which reflected light from the scanned object 1036 enters the probe 1028 to an upper surface 1072 opposite the lower surface 1070. In some applications, height H2 is between 14-17 mm. In some applications, the width W1 is between 18-22 mm.

Referring now to fig. 29A, fig. 29A is a schematic diagram of a structured light projector 1030 having a laser diode 1041 as a light source 1040 in accordance with some applications of the present invention. For some applications, the pattern generator 1042 may be a Diffractive Optical Element (DOE) 1043. The laser diode 1041 may emit light through a collimator 1062, which is then transmitted through the DOE 1043 to generate a structured light pattern as a distribution of discrete unconnected light points. As an alternative to DOE 1043, pattern generator 1042 may be an array of refractive microlenses disposed in the optical path (configuration not shown) between laser diodes 1041 and the projector focal plane.

Referring now to FIG. 29B, FIG. 29B is a schematic diagram of a structured light projector 1030 having a Light Emitting Diode (LED)1064 as the light source 1040 and a mask 1066 as the pattern generator 1042.

Referring now to fig. 30, fig. 30 is a schematic diagram of a light field camera 1032 showing a light field camera sensor 1046 and a captured three-dimensional object 1036, in accordance with some applications of the present invention. For some applications, the optical parameters of light field camera 1032 may be selected such that (a) light reflected from a foreground (forkround) 1075 of object 1036 is focused onto a central region 1074 of the light field camera sensor, and (b) light reflected from a background (background)1077 of object 1036 is focused onto a peripheral region 1076 of light field camera sensor 1046. In some applications, the peripheral region 1076 may be directed more frequently toward objects further away, such as the gums, than toward objects closer, such as the teeth, when scanning an intraoral scene.

The central region 1074 of the light field camera sensor 1046 may have a higher spatial resolution than the peripheral region 1076 of the light field camera sensor 1046. For example, each sub-array 1052 in the central region 1074 of the image sensor 1048 may have 10-40% fewer pixels than each sub-array 1052 in the peripheral region 1076, i.e., the microlenses in the central region 1074 may be smaller than the microlenses in the peripheral region 1076. Smaller microlenses allow for more microlenses per unit area in the central region 1074. Thus, the central region 1074 of the light field camera sensor 1046 may have higher spatial resolution due to the increased ratio of microlenses per unit area. In some applications, the central region 1074 may include at least 50% of the total number of sensor pixels.

While the central region 1074 has a higher spatial resolution than the peripheral region 1076, the peripheral region 1076 may have a higher depth resolution than the central region 1074 and may be set to focus at a greater object distance than the central region 1074. The larger microlenses in the peripheral region 1076 of the light field camera sensor 1046 are configured to focus at a greater depth than the smaller microlenses in the central region 1074. For example, each microlens 1050 disposed over a subarray 1052 of sensor pixels in the peripheral region 1076 of the image sensor 1048 may be configured to focus at a depth that is 1.1-1.4 times greater than a depth at which each microlens 1050 disposed over a subarray 1052 of sensor pixels in the central region 1074 of the image sensor 1048 is configured to focus.

Thus, the higher spatial resolution of the central region 1074 may allow for capturing the foreground 1075 of the object 1036 at a higher spatial resolution than the background 1077 of the object 1036, e.g., when scanning an intraoral scene of a subject, teeth may be captured at a higher spatial resolution than the regions surrounding the teeth, while the more distant focus and greater depth resolution of the peripheral region 1076 may allow for capturing the background 1077, e.g., the toothless regions and gums surrounding the teeth in the foreground 1075.

Referring now to fig. 31, fig. 31 is a schematic illustration of an intra-oral scanner 1020 having a light field camera 1032 and a structured light projector 1030 disposed in a distal end 1027 of a probe 1028 according to some applications of the present invention. For some applications, exactly one structured light projector 1030 and exactly one light field camera 1032 are disposed in the distal end 1027 of the probe 1028. The structured light projector 1030 may be positioned to directly face an object 1036 located outside of the hand-held wand 1022 placed in its field of illumination. Thus, light projected from the structured light projector 1030 will fall on the object 1036 without any optical redirection, e.g., reflection from a mirror to redirect the light, as described above with reference to fig. 28A. Similarly, the light field camera 1032 can be positioned to directly face an object 1036 located outside of the hand wand 1022 that is placed in its field of view. Thus, light reflected from object 1036 will enter light field camera 1032 without any optical redirection, e.g., reflection from a mirror to redirect the light, as described above with reference to fig. 28A.

Positioning the structured light projector 1030 in the distal end 1027 of the probe 1028 can allow the illumination field ψ (psi) of the structured light projector 1030 to be wider, for example, at least 60 degrees and/or less than 120 degrees. Positioning the structured light projector 1030 in the distal end 1027 of the probe 1028 may also allow the structured light projector 1030 to focus light from the light source 1040 at a projector focal plane that is at least 3mm and/or less than 40mm from the light source 1040.

Positioning light field camera 1032 in distal end 1027 of probe 1028 may allow the field of view ω (omega) of light field camera 1032 to be wider, e.g., at least 60 degrees and/or less than 120 degrees. Positioning the light field camera 1032 in the distal end 1027 of the probe 1028 can also allow the light field camera 1032 to focus at a camera focal plane that is at least 3mm and/or less than 40mm from the light source 1040. In some applications, the illumination field ψ (psi) of the structured light projector 1030 and the field of view ω (omega) of the light field camera 1032 overlap such that at least 40% of the projected structured light pattern from the structured light projector 1030 is in the field of view ω (omega) of the light field camera 1032. Similar to as described above with reference to fig. 30, when intraoral scanner 1020 has a single light field camera 1032 disposed in distal end 1027 of probe 1028, the optical parameters of light field camera sensor 1046 may be selected such that a central region of light field camera sensor 1090 has a higher resolution than peripheral regions of light field camera sensor 1046.

Positioning the structured light projector 1030 and the light field camera 1032 in the distal end 1027 of the probe 1028 can make the probe 1028 smaller because the mirror 1034 is not used in this configuration. In some applications, the height H3 of the probe 1028 is less than 14mm, the width W2 of the probe 1028 is less than 22mm, and the height H3 and the width W2 define a plane perpendicular to the longitudinal axis 1067 of the hand-held bar 1022. In some applications, the height H3 is between 10-14 mm. In some applications, the width W2 is between 18-22 mm. As described above, the height H2 of the probe 1028 is measured from (a) the lower surface 1070 (scanning surface) through which reflected light from the scanned object 1036 enters the probe 1028 to (b) the upper surface 1072 opposite the lower surface 1070. The control circuit 1056 may (a) drive the structured light projector 1030 to project a structured light pattern onto an object 1036 external to the handheld wand 1022, and (b) drive the light field camera 1032 to capture a light field generated by the structured light pattern reflected from the object 1036. Using information from the captured light field, the computer processor 1058 can reconstruct a three-dimensional image of the surface of the object 1036 and output the image to an output device 1060, such as a monitor.

Referring now to fig. 32, fig. 32 is a schematic illustration of an intra-oral scanner 1020 having a plurality of structured light projectors 1030 and a plurality of light field cameras 1032 disposed in a distal end 1027 of a probe 1028 according to some applications of the present invention. Having multiple structured light projectors and multiple light field cameras may increase the overall field of view of the intraoral scanner 1020, which may enable capture of multiple objects 1036, e.g., capture of multiple teeth and areas around the teeth, e.g., an edentulous area in the mouth of a subject. In some applications, the plurality of illumination fields ψ (psi) overlap with the corresponding plurality of fields of view ω (omega) such that at least 40% of the projected structured light pattern from each structured light projector 1030 is in the field of view ω (omega) of at least one light field camera 1032. The control circuitry 1056 may (a) drive the plurality of structured light projectors 1030 to project a structured light pattern onto an object 1036 external to the handheld wand 1022, and (b) drive the plurality of light field cameras 1032 to capture light fields generated by the plurality of structured light patterns reflected from the object 1036. Using information from the captured light field, the computer processor 1058 can reconstruct a three-dimensional image of the surface of the object 1036 and output the image to an output device 1060, such as a monitor.

For some applications, at least one of the structured light projectors 1030 can be a monochromatic structured light projector that projects a monochromatic structured light pattern onto the scanned object 1036. For example, a monochromatic structured light projector may project a blue structured light pattern at wavelengths of 420-470 nm. At least one of light field cameras 1032 may be a monochromatic light field camera that captures a light field generated by a monochromatic structured light pattern reflected from scanned object 1036. The intraoral scanner 1020 may also include a light source that emits white light onto the object 1036 and a camera that captures a two-dimensional color image of the object 1036 under illumination by the white light. The computer processor 1058 can combine (a) the information captured from the monochromatic light field with (b) at least one two-dimensional color image of the object 1036 to reconstruct a three-dimensional image of the surface of the object 1036. The computer processor 1058 may then output the image to an output device 1060, such as a monitor.

Any of the above-described devices may be used to perform a method of generating image data (e.g., image data of an intraoral surface). In one example embodiment, a method includes generating respective light patterns by one or more light projectors disposed in a probe of an intraoral scanner. Generating a light pattern by a light projector of the one or more light projectors may include generating light by the light projector, focusing the light at a projector focal plane, and generating a light pattern from the light at the projector focal plane by a pattern generator. The method may further include projecting respective light patterns of the one or more light projectors toward an intraoral surface disposed within an illumination field of the one or more light projectors. The method may also include receiving, by one or more light field cameras disposed in the probe, a light field generated by at least a portion of the respective light pattern reflected from the intraoral surface. The method may also include generating, by one or more light field cameras, a plurality of images depicting the light field, and transmitting the plurality of images to a data processing system.

In some embodiments, one or more light projectors and one or more light field cameras are disposed at the distal end of the probe, and the one or more light projectors and the one or more light field cameras are positioned such that (a) each light projector directly faces the intraoral surface, (b) each light field camera directly faces the intraoral surface, and (c) at least 40% of the light pattern from each light projector is within the field of view of the at least one light field camera.

In some embodiments, one or more light projectors and a light field camera are disposed at the proximal end of the probe. For such embodiments, the method may further include reflecting the respective light pattern onto the intraoral surface using a mirror, and reflecting the light field reflected from the intraoral surface into the one or more light field cameras using the mirror.

In some applications of the present invention, the method may be performed by any of the described apparatus for intraoral scanning (e.g., an intraoral scanner and/or a data processing system such as computer processor 1058) to generate a digital three-dimensional model of an intraoral surface. In one embodiment, the method includes driving one or more light projectors of an intraoral scanner to project a light pattern on an intraoral surface. The method also includes driving one or more light field cameras of the intraoral scanner to capture a plurality of images depicting a light field generated by a light pattern projected from at least a portion of the intraoral surface, wherein the light field contains information about the intensity of the light pattern reflected from the intraoral surface and the direction of the light rays. The method also includes receiving a plurality of images depicting at least a portion of the projected light pattern on the intraoral surface and generating a digital three-dimensional model of the intraoral surface using information from the captured light field depicted in the plurality of images.

In one application, at least 40% of the light pattern from each light projector is in the field of view of at least one of the one or more light field cameras. In one application, each light projector is a structured light projector having an illumination field of 60-120 degrees, and wherein the projector focal plane is between 3mm and 40mm from the light source. In one application, each light field camera has a field of view of 60-120 degrees and is configured to focus at a camera focal plane between 3mm and 40mm from the light field camera. In one application, the plurality of images includes images from a plurality of light field cameras. In one application, the light field also contains information about the phase-coded depth, by means of which the depth can be estimated from different directions. In one application, the method further includes receiving a plurality of two-dimensional color images of the interior surface of the mouth and determining color data for a digital three-dimensional model of the interior surface of the mouth based on the plurality of two-dimensional color images.

Applications of the invention described herein may take the form of a computer program product accessible from a computer-usable or computer-readable medium (e.g., non-transitory computer-readable medium) providing program code for use by or in connection with a computer or any instruction execution system (e.g., processor 96 or processor 1058). For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. In some applications, the computer-usable or computer-readable medium is a non-transitory computer-usable or computer-readable medium.

Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a Random Access Memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk read only memories (CD-ROMs), compact disks read/write (CD-rs/ws), and DVDs. For some applications, cloud storage and/or storage in a remote server is used.

A data processing system suitable for storing and/or executing program code will include at least one processor (e.g., processor 96 or processor 1058) coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution. The system can read the instructions of the present invention on the program storage device and follow these instructions to execute the method of the application of the present invention.

Network adapters may be coupled to the processor to enable the processor to become coupled to other processors or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the C programming language or similar programming languages.

It should be understood that the methods described herein may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer (e.g., processor 96 or processor 1058) or other programmable data processing apparatus, create means for implementing the functions/acts specified in the methods described in this application. These computer program instructions may also be stored in a computer-readable medium (e.g., a non-transitory computer-readable medium) that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the method described in the application. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the methods described in this application.

Processor 96 and processor 1058 are typically hardware devices programmed with computer program instructions to produce a corresponding special purpose computer. For example, a computer processor typically functions as a dedicated three-dimensional surface reconstruction computer processor when programmed to perform the methods described herein. Generally, the operations described herein performed by a computer processor transform the physical state of a memory, which is an actual physical object, to have a different magnetic polarity, charge, etc., depending on the technology of the memory used.

Alternatively, the processor 96 may take the form of a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), or a neural network implemented on a special-purpose chip.

It will be appreciated by persons skilled in the art that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not in the prior art.

82页详细技术资料下载

上一篇：一种医用注射器针头装配设备

下一篇：兼具牙签、牙间刷及牙线棒结构的三合一洁牙棒

Intraoral 3D scanner with multiple miniature cameras and miniature pattern projectors

相关技术

网友询问留言