阅读说明：本技术 具有电子仿真透明度的显示器组件 (Display assembly with electronically emulated transparency ) 是由 M·A·拉姆金 K·M·灵根贝格 J·D·拉姆金 于 2019-01-23 设计创作，主要内容包括：在一个实施例中,一种电子显示组件包括：电路板；位于所述电路板的第一侧的第一微透镜层,和位于所述电路板的与第一微透镜层相反的一侧的第二微透镜层。第一微透镜层包括第一多个微透镜,以及所述第二微透镜层包括第二多个微透镜。电子显示组件还包括与第一微透镜层相邻的图像传感器层,和与第二微透镜阵列相邻的显示层。所述图像传感器层包括传感器像素,所述传感器像素用于检测通过所述第一微透镜的进来光,以及所述显示层包括用于通过所述第二微透镜发射光的显示像素。所述电子显示组件通过以与检测到的通过所述第一微透镜的进来光的角度对应的角度从所述第二微透镜发射光来仿真透明度。(In one embodiment, an electronic display assembly comprises: a circuit board; a first microlens layer on a first side of the circuit board, and a second microlens layer on a side of the circuit board opposite the first microlens layer. The first microlens layer includes a first plurality of microlenses and the second microlens layer includes a second plurality of microlenses. The electronic display assembly also includes an image sensor layer adjacent the first microlens layer, and a display layer adjacent the second microlens array. The image sensor layer includes sensor pixels for detecting incoming light through the first microlenses, and the display layer includes display pixels for emitting light through the second microlenses. The electronic display component simulates transparency by emitting light from the second microlenses at an angle corresponding to the angle of detected incoming light through the first microlenses.)

1. An electronic display assembly comprising:

a circuit board;

a first microlens layer on a first side of the circuit board, the first microlens layer including a first plurality of microlenses;

a second microlens layer on a side of the circuit board opposite the first microlens layer, the second microlens layer comprising a second plurality of microlenses;

An image sensor layer adjacent to the first microlens layer, the image sensor layer comprising a plurality of sensor pixels configured to detect incoming light through the first plurality of microlenses;

a display layer adjacent to a second microlens array, the display layer comprising a plurality of display pixels configured to emit light through the second plurality of microlenses;

a logic cell layer coupled to the circuit board, the logic cell layer comprising one or more logic cells configured to simulate transparency by directing signals from the plurality of sensor pixels to the plurality of display pixels to emit light from the second plurality of microlenses at an angle corresponding to the angle of the detected incoming light through the first plurality of microlenses.

2. The electronic display assembly of claim 1, wherein:

the first plurality of microlenses facing a first direction; and

the second plurality of microlenses are oriented in a second direction 180 degrees from the first direction.

3. The electronic display assembly of claim 1, wherein:

the image sensor layer is disposed within the first microlens layer; and

The display layer is disposed within the second microlens layer.

4. The electronic display assembly of claim 1, wherein the circuit board is flexible.

5. The electronic display assembly of claim 1, wherein simulating transparency comprises emitting light from the second plurality of microlenses such that an image is displayed that matches an image that would be seen in the absence of the electronic display assembly.

6. The electronic display assembly of claim 1, wherein the logic cell layer is located between the image sensor layer and the circuit board.

7. The electronic display assembly of claim 1, wherein the logic cell layer is located between the display layer and the circuit board.

8. The electronic display assembly of claim 1, wherein each of the first and second plurality of microlenses comprises a three-dimensional shape, the collimating lens being located at one end of the three-dimensional shape, the three-dimensional shape comprising:

a triangular polyhedron;

a rectangular cuboid;

a pentagonal polyhedron;

a hexagonal polyhedron;

a heptagonal polyhedron; or

An octagonal polyhedron.

9. The electronic display assembly of claim 8, wherein each of the first and second plurality of microlenses further comprises a plurality of opaque walls configured to prevent light from leaking into adjacent microlenses.

10. An electronic display assembly comprising:

a circuit board;

a first microlens layer on a first side of the circuit board, the first microlens layer including a first plurality of microlenses;

a second microlens layer on a side of the circuit board opposite the first microlens layer, the second microlens layer comprising a second plurality of microlenses;

an image sensor layer adjacent to the first microlens layer, the image sensor layer comprising a plurality of sensor pixels configured to detect incoming light through the first plurality of microlenses; and

a display layer adjacent to a second microlens array, the display layer comprising a plurality of display pixels configured to emit light through the second plurality of microlenses;

wherein the electronic display component is configured to simulate transparency by emitting light from the second plurality of microlenses at an angle corresponding to the angle of detected incoming light through the first plurality of microlenses.

11. The electronic display assembly of claim 10, wherein:

the first plurality of microlenses facing a first direction; and

the second plurality of microlenses are oriented in a second direction 180 degrees from the first direction.

12. The electronic display assembly of claim 10, wherein:

the image sensor layer is disposed within the first microlens layer; and

the display layer is disposed within the second microlens layer.

13. The electronic display assembly of claim 10, wherein the circuit board is flexible.

14. The electronic display assembly of claim 10, wherein simulating transparency comprises emitting light from the second plurality of microlenses such that an image is displayed that matches an image that would be seen in the absence of the electronic display assembly.

15. The electronic display assembly of claim 10, wherein each of the first and second plurality of microlenses comprises a three-dimensional shape, the collimating lens being located at one end of the three-dimensional shape, the three-dimensional shape comprising:

a triangular polyhedron;

a rectangular cuboid;

a pentagonal polyhedron;

a hexagonal polyhedron;

a heptagonal polyhedron; or

An octagonal polyhedron.

16. The electronic display assembly of claim 15, wherein each of the first and second plurality of microlenses further comprises a plurality of opaque walls configured to prevent light from leaking into adjacent microlenses.

17. A method of manufacturing an electronic display, the method comprising:

forming a plurality of unit attachment locations on the circuit board, each unit attachment location corresponding to one of the plurality of display units and one of the plurality of sensor units;

coupling a plurality of sensor units to a first side of a circuit board, each sensor unit coupled to a respective one of the unit attachment locations; and

coupling a plurality of display units to a second side of the circuit board opposite the first side, each display unit coupled to a respective one of the unit attachment locations;

coupling a first plurality of microlenses to the plurality of sensor units; and

coupling a second plurality of microlenses to the plurality of display units.

18. The method of manufacturing an electronic display of claim 17, further comprising coupling a plurality of logic units between the circuit board and the plurality of display units.

19. The method of manufacturing an electronic display of claim 17, further comprising coupling a plurality of logic units between the circuit board and the plurality of sensor units.

20. A method of manufacturing an electronic display according to claim 17, wherein each microlens in the first and second plurality of microlenses comprises:

A three-dimensional shape with a collimating lens located at one end of the three-dimensional shape, the three-dimensional shape comprising:

a triangular polyhedron;

a rectangular cuboid;

a pentagonal polyhedron;

a hexagonal polyhedron;

a heptagonal polyhedron; or

An octagonal polyhedron; and

a plurality of opaque walls configured to prevent light from leaking into adjacent microlenses.

Technical Field

The present disclosure relates generally to light field displays and cameras, and more particularly, to display assemblies having electronically emulated transparency.

Background

Electronic displays are used in a variety of applications. For example, displays are used in smart phones, laptops, and digital cameras. In addition to electronic displays, some devices (such as smartphones and digital cameras) may also include image sensors. Although some cameras and electronic displays capture and reproduce light fields separately, light field displays and light field cameras are typically not integrated with each other.

Disclosure of Invention

In one embodiment, an electronic display assembly comprises: a circuit board; a first microlens layer on a first side of the circuit board, and a second microlens layer on a side of the circuit board opposite the first microlens layer. The first microlens layer includes a first plurality of microlenses and the second microlens layer includes a second plurality of microlenses. The electronic display assembly also includes an image sensor layer adjacent to the first microlens layer. The image sensor layer includes a plurality of sensor pixels configured to detect incoming light through the first plurality of microlenses. The electronic display assembly also includes a display layer adjacent to the second microlens array. The display layer includes a plurality of display pixels configured to emit light through the second plurality of microlenses. The electronic display assembly also includes a logic cell layer coupled to the circuit board. The logic cell layer includes one or more logic cells configured to simulate transparency by directing signals from the plurality of sensor pixels to the plurality of display pixels to emit light from the second plurality of microlenses at an angle corresponding to the angle of the detected incoming light through the first plurality of microlenses.

In another embodiment, an electronic display assembly includes a circuit board and a first microlens layer on a first side of the circuit board. The first microlens layer includes a first plurality of microlenses. The electronic display assembly also includes a second microlens layer located on an opposite side of the circuit board from the first microlens layer. The second microlens layer includes a second plurality of microlenses. The electronic display assembly also includes an image sensor layer adjacent to the first microlens layer. The image sensor layer includes a plurality of sensor pixels configured to detect incoming light through the first plurality of microlenses. The electronic display assembly also includes a display layer adjacent to the second microlens array. The display layer includes a plurality of display pixels configured to emit light through the second plurality of microlenses. The electronic display component is configured to simulate transparency by emitting light from the second plurality of microlenses at an angle corresponding to the angle of detected incoming light through the first plurality of microlenses.

In another embodiment, a method of manufacturing an electronic display comprises: the method includes forming a plurality of unit attachment locations on a circuit board, coupling a plurality of sensor units to a first side of the circuit board, and coupling a plurality of display units to a second side of the circuit board opposite the first side. Each unit attachment location corresponds to one of the plurality of display units and one of the plurality of sensor units. Each sensor unit is coupled to a respective one of the unit attachment locations, and each display unit is coupled to a respective one of the unit attachment locations. The method of manufacturing an electronic display also includes coupling a first plurality of microlenses to the plurality of sensor units and coupling a second plurality of microlenses to the plurality of display units.

The present disclosure provides several technical advantages. Some embodiments provide a complete and accurate re-creation of the target light field while maintaining a light weight and comfortable wearing for the user. Some embodiments provide a thin electronic system that provides both opacity and controllable one-way emulated transparency, as well as digital display capabilities such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). Some embodiments provide a direct sensor-to-display system that uses direct association of input pixels to corresponding output pixels to avoid the need for image transformation. For some systems, this reduces complexity, cost and power requirements. Some embodiments provide an intra-layer signal processing architecture that provides locally distributed processing of large amounts of data (e.g., 160k of image data or more), thereby avoiding bottlenecks and performance, power, and transmission line issues associated with existing solutions. Some embodiments use a microlens layer with an array of plenoptic cells to accurately capture and display to a viewer an amount of light. Plenoptic elements include opaque element walls to eliminate optical crosstalk between elements, thereby improving the accuracy of the reproduced light field.

Some embodiments provide a three-dimensional electronic device through a geodesic facet. In such embodiments, the flexible circuit board with an array of small rigid surfaces (e.g., display and/or sensor facets) may be formed into any 3D shape that is particularly helpful to accommodate the narrow radius of curvature (e.g., 30-60mm) required for a head-mounted near-eye wrap display. Some embodiments provide a distributed multi-screen array for high-density displays. In such embodiments, an array of custom sized and shaped small high resolution microdisplays (e.g., display facets) is formed and then assembled on a larger flexible circuit board, which can then be formed into a 3D shape (e.g., a hemispherical surface). Each microdisplay can operate independently of any other display, thereby providing a large array of many high resolution displays with unique content on each high resolution display such that the entire assembly together forms essentially a single very high resolution display. Some embodiments provide a distributed multi-aperture camera array. Such embodiments provide an array of custom sized and shaped small image sensors (e.g., sensor facets), all assembled on a larger flexible circuit board that is then formed into a 3D (e.g., hemispherical) shape. Each discrete image sensor can operate independently of any other image sensor so as to provide a large array of many apertures, capturing unique content on each aperture, such that the entire assembly becomes essentially a seamless, extremely high resolution multi-node camera.

Other technical advantages will be readily apparent to one skilled in the art from the figures 1A through 42, their description, and the claims. Moreover, while specific advantages have been enumerated above, various embodiments may include all, some, or none of the enumerated advantages.

Drawings

For a more complete understanding of the present disclosure and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

1A-1C illustrate reference scenes with various three-dimensional (3D) objects and various viewing positions, in accordance with certain embodiments;

2A-2C illustrate viewing the 3D object of FIGS. 1A-1C through a transparent panel according to some embodiments;

3A-3C illustrate viewing of the 3D object of FIGS. 1A-1C through a camera image panel according to some embodiments;

4A-4C illustrate viewing the 3D object of FIGS. 1A-1C through a simulated transparency electronic panel according to some embodiments;

5A-5C illustrate viewing the 3D object of FIGS. 1A-1C from an alternative angle through the camera image panel of FIGS. 3A-3C, in accordance with certain embodiments;

6A-6C illustrate viewing the 3D object of FIGS. 1A-1C from an alternative angle through the simulated transparency electronic panel of FIGS. 4A-4C, in accordance with certain embodiments;

FIG. 7 illustrates a cross-sectional view of a simulated transparency assembly in accordance with certain embodiments;

FIG. 8 illustrates an exploded view of the simulated transparency component of FIG. 7 in accordance with certain embodiments;

FIG. 9 illustrates a method of manufacturing the simulated transparency assembly of FIG. 7 in accordance with certain embodiments;

FIG. 10 illustrates a direct sensor to display system that may be used by the simulated transparency component of FIG. 7 in accordance with certain embodiments;

FIG. 11 illustrates a method of manufacturing the direct sensor to display system of FIG. 10, in accordance with certain embodiments;

12-13 illustrate various intra-layer signal processing structures that may be used by the simulated transparency component of FIG. 7, in accordance with certain embodiments;

FIG. 14 illustrates a method of manufacturing the intralayer signal processing system of FIGS. 12-13 in accordance with certain embodiments;

FIG. 15 represents a plenoptic cell (plenoptic cell) component that may be used by the simulated transparency component of FIG. 7, in accordance with certain embodiments;

figure 16 shows a cross-section of a portion of the all-optical primitive assembly of figure 15 in accordance with certain embodiments;

17A-17C show cross-sections of a portion of the plenoptic element assembly of FIG. 15 with various incoming light fields, in accordance with certain embodiments;

18A-18B illustrate a method of fabricating the all-optical cellular component of FIG. 15, in accordance with certain embodiments;

figures 19A-19B illustrate another method of fabricating the all-optical cellular component of figure 15 in accordance with certain embodiments;

Figures 20-21 represent a full-gloss cellular component that can be fabricated by the method of figures 18A-19B, in accordance with certain embodiments;

FIG. 22 illustrates a flexible circuit board that may be used by the simulated transparency assembly of FIG. 7 in accordance with certain embodiments;

FIG. 23 shows additional details of the flexible circuit board of FIG. 22 in accordance with certain embodiments;

FIG. 24 illustrates data flow through the flexible circuit board of FIG. 22 in accordance with certain embodiments;

FIG. 25 illustrates a method of manufacturing an electronic assembly using the flexible circuit board of FIG. 22, in accordance with certain embodiments;

FIG. 26 shows a cross-sectional view of a curved multi-display array according to some embodiments;

FIG. 27 shows an exploded view of the curved multi-display array of FIG. 26 according to some embodiments;

28-29 illustrate logic and display facets of the curved multi-display array of FIG. 26 in accordance with certain embodiments;

FIG. 30 illustrates a back side of the flexible circuit board of FIG. 22 in accordance with certain embodiments;

FIG. 31 illustrates data flow through the flexible circuit board of FIG. 30 in accordance with certain embodiments;

FIG. 32 illustrates the flexible circuit board of FIG. 30 having been formed into a hemispherical shape in accordance with certain embodiments;

FIG. 33 shows data flow through the flexible circuit board of FIG. 32 in accordance with certain embodiments;

Fig. 34 illustrates an array of logic facets that have been formed into a hemispherical shape in accordance with certain embodiments;

FIG. 35 illustrates communication between the logical facets of FIG. 34 in accordance with certain embodiments;

FIG. 36 illustrates a method of fabricating the curved multi-display array of FIG. 26 according to some embodiments;

figure 37 shows a cross-sectional view of a curved multi-camera array according to some embodiments;

FIGS. 38-39 illustrate exploded views of the curved multi-camera array of FIG. 37, in accordance with certain embodiments;

FIG. 40 illustrates a rear view of the flexible circuit board of FIG. 32 in accordance with certain embodiments;

FIG. 41 illustrates data flow through the flexible circuit board of FIG. 40 in accordance with certain embodiments; and

figure 42 illustrates a method of fabricating the curved multi-camera array of figure 37, in accordance with certain embodiments.

Detailed Description

Electronic displays are used in a variety of applications. For example, displays are used in smart phones, laptops, and digital cameras. In addition to electronic displays, some devices (such as smartphones and digital cameras) may also include image sensors. However, devices with displays and image sensors are typically limited in their ability to accurately capture and display a plenoptic environment.

To address the problems and limitations associated with existing electronic displays, embodiments of the present disclosure provide various electronic components for capturing and displaying light fields. Fig. 1A-9 relate to display assemblies with electronic artificial transparency, fig. 10-11 relate to direct camera-display systems, fig. 12-14 relate to intra-layer signal processing, fig. 15-21 relate to plenoptic imaging systems, fig. 22-25 relate to three-dimensional (3D) electronic device distribution through a geodesic facet, fig. 26-36 relate to distributed multi-screen arrays for high-density displays, and fig. 37-42 relate to distributed multi-aperture camera arrays.

To facilitate a better understanding of the present disclosure, the following examples of certain embodiments are given. The following examples should not be read to limit or define the scope of the present disclosure. Embodiments of the present disclosure and their advantages are best understood by referring to fig. 1A-42, wherein like numerals are used for like and corresponding parts.

1A-9 illustrate various aspects of a component having electronically emulated transparency, in accordance with certain embodiments. In general, the electronic components shown in detail in fig. 7-8 may be used in different applications to provide features such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). For VR applications, there is a need for a digital display that can completely replace the real world field of view, similar to how a standard computer monitor blocks the view of the scene behind it. However, for AR applications, a digital display capable of overlaying data over a real world field of view is required, such as a heads-up display for pilots in modern cockpits. MR applications require a combination of both. Typical systems for providing some or all of these features are unsatisfactory for a number of reasons. For example, typical solutions do not provide for accurate or complete re-creation of the target light field. As another example, existing solutions are often bulky and uncomfortable for the user.

To address the problems and limitations associated with existing electronic displays, embodiments of the present disclosure provide a thin electronic system that provides both opacity and controllable one-way emulated transparency as well as digital display capabilities. From one side, the surface appears opaque, but from the opposite side, the surface can appear completely transparent, appear completely opaque, function as a digital display, or any combination of these. In some embodiments, simultaneous plenoptic sensing and display technologies are combined within a single layered structure to form a surface that appears one-way visually transparent. The system may include multiple layers of electronics and optics to manually recreate transparency that may be enhanced and/or digitally controlled. The individual image sensor pixels on one side may be spatially arranged to match the position of the display pixels on the opposite side of the assembly. In some embodiments, all of the electronic drive circuitry and some of the display logic circuitry may be sandwiched between the sensor layer and the display layer, and the output signal of each sensor pixel may be directed by the circuitry to the corresponding display pixel on the opposite side. In some embodiments, such centrally processed signals are aggregated with incoming signals from an array of plenoptic imaging sensors on the opposite side, and processed according to the following mode of operation. In VR mode, the external video source replaces the user's view of the external world completely with the incoming view from the video in preference to the camera data. In AR mode, an external video source is overlaid on the camera data, resulting in a combined view of the outside world and the view from the video (e.g., the video data is simply added to the scene). In MR mode, an external video source is blended with the camera data, allowing virtual objects to appear to interact with real objects in the real world, changing the virtual content to appear to be integrated with the real environment through object occlusion, lighting, etc.

With sensor pixels on one side of the assembly and display pixels on the other side, and with pixel-to-pixel alignment between the camera and the display, some embodiments combine stacked transparent High Dynamic Range (HDR) sensors and display pixels into a single structure. Both the sensor and display pixel arrays may be focused through multiple sets of microlenses to capture and display a four-dimensional light field. This means that: a full field of view of the real world is captured on one side of the assembly and electronically rendered on the other side, allowing partial or complete changes in the incoming image while maintaining image sharpness, brightness, and sufficient angular resolution so that the display side appears transparent even when viewed at oblique angles.

Fig. 1A-6C are provided to represent the difference between the electronic simulated transparency provided by embodiments of the present disclosure and a typical camera image (such as a current camera image displayed through a camera viewfinder or using a smart phone). 1A-1C illustrate reference scenes with various 3D objects 110 (i.e., 110A-C) and front viewing positions, according to some embodiments. Fig. 1A is a top view of the arrangement of the 3D object 110 and the front viewing direction of the 3D object 110. Fig. 1B is a perspective view of the same arrangement and front viewing direction of the 3D object 110 as fig. 1A. Fig. 1C is a front view of the 3D object 110 taken from the position shown in fig. 1A and 1B. It can be seen that the field of view of the 3D object 110 in fig. 1C is the normal expected field of view of the 3D object 110 (i.e., the field of view of the 3D object 110 is completely unchanged because there is nothing between the observer and the 3D object 110).

2A-2C illustrate viewing the 3D object 110 of FIGS. 1A-1C through a transparent panel 210 according to some embodiments. The transparent panel 210 may be, for example, a piece of transparent glass. Fig. 2A is a top view of the front viewing direction of the 3D object 110 through the transparent panel 210, and fig. 2B is a perspective view of the same arrangement and front viewing direction of the 3D object 110 as fig. 2A. Fig. 2C is a front view of the 3D object 110 obtained through the transparent panel 210 from the position shown in fig. 2A and 2B. It can be seen that the field of view of the 3D object 110 through the transparent panel 210 in fig. 2C is the normal expected field of view of the 3D object 110 (i.e., the field of view of the 3D object 110 is not changed at all because the viewer is looking through the transparent panel 210). In other words, the field of view of the 3D object 110 through the transparent panel 210 in fig. 2C is the same as the field of view in fig. 1C with no object between the viewer and the 3D object 110 (i.e., a "perceived" transparency). In other words, the edges of the projected image on the transparent panel 210 are aligned with the field of view of the actual 3D object 110 behind the transparent panel 210 to create a field of view aligned image 220A of the 3D object 110A, a field of view aligned image 220B of the 3D object 110B, and a field of view aligned image 220C of the 3D object 110C.

3A-3C illustrate viewing of the 3D object 110 of FIGS. 1A-1C through a camera image panel 310 according to some embodiments. The camera image panel 310 may be, for example, a camera viewfinder or a display of a smartphone that displays its current camera image. In these images, the camera image panel 310 is at an angle (e.g., 30 degrees) with respect to the viewer to represent how such a system does not provide realistic artificial transparency. Fig. 3A is a top view of the front viewing direction of the 3D object 110 through the camera image panel 310, and fig. 3B is a perspective view of the same arrangement and front viewing direction of the 3D object 110 as fig. 3A. Fig. 3C is a front view of the 3D object 110 through the camera image panel 310 taken from the position shown in fig. 3A and 3B. It can be seen that the field of view of the 3D object 110 in fig. 3C through the camera image panel 310 is different from the field of view of the 3D object 110 through the transparent panel 210. Here, the camera image panel 310 redirects the line of sight perpendicular to the camera image panel 310, thereby not displaying perceived transparency (i.e., the image on the camera image panel 310 is not aligned with the field of view, but instead describes the image acquired by the redirected line of sight). In other words, the edges of the projected image on the camera image panel 310 are not aligned with the field of view of the actual 3D object 110 behind the camera image panel 310. This is represented by the misaligned image 320A of 3D object 110A and the misaligned image 320B of 3D object 110B on camera image panel 310 in FIG. 3C.

Fig. 4A-4C illustrate viewing of the 3D object 110 of fig. 1A-1C through a simulated transparency electronic panel 410, according to some embodiments. In these images, the simulated transparency panel 410 is at an angle (e.g., 30 degrees) with respect to the viewer to represent how the simulated transparency panel 410 provides a realistic simulated transparency unlike the camera image panel 310. Fig. 4A is a top view of a front viewing direction of the 3D object 110 through the simulated transparency panel 410, and fig. 4B is a perspective view of the same arrangement and front viewing direction of the 3D object 110 as fig. 4A. Fig. 4C is a front view of 3D object 110 through simulated transparency panel 410 taken from the position shown in fig. 4A and 4B. It can be seen that the field of view of the 3D object 110 in fig. 4C through the simulated transparency panel 410 is different from the field of view of the 3D object 110 through the camera image panel 310, but similar to the field of view of the 3D object 110 through the transparent panel 210. Here, the simulated transparency panel 410 does not redirect the viewer's line of sight through the simulated transparency panel 410, but rather allows them to remain nearly unchanged and thereby provide simulated transparency (i.e., the image on the simulated transparency panel 410 is aligned with the field of view, like the transparent panel 210). Like the transparent panel 210, the edges of the projected image on the simulated transparency panel 410 are aligned with the field of view of the actual 3D object 110 behind the simulated transparency panel 410 to create a field of view aligned image 220A of the 3D object 110A, a field of view aligned image 220B of the 3D object 110B, and a field of view aligned image 220C of the 3D object 110C.

Fig. 5A-5C show the 3D object 110 of fig. 1A-1C viewed through the camera image panel 310 of fig. 3A-3C, but from an alternative angle. In these images, the camera image panel 310 is at different 30 degrees angles relative to the viewer to further illustrate how such a system does not provide realistic artificial transparency. As in fig. 3A-3C, the edges of the projected image on the camera image panel 310 are not aligned with the field of view of the actual 3D object 110 behind the camera image panel 310. This is represented by the misaligned image 320C of 3D object 110C and the misaligned image 320B of 3D object 110B on camera image panel 310 in fig. 5C.

Fig. 6A-6C show the 3D object 110 of fig. 1A-1C viewed through the simulated transparency electronic panel 410 of fig. 4A-4C, but from an alternative angle. As in fig. 4A-4C, the edges of the projected image on the simulated transparency panel 410 in fig. 6C are aligned with the field of view of the actual 3D object 110 behind the simulated transparency panel 410 to create a field of view aligned image 220B of the 3D object 110B and a field of view aligned image 220C of the 3D object 110C.

As shown above in fig. 4A-4C and 6A-6C, the simulated transparency panel 410 provides a field-of-view aligned image 220 of the 3D object 110 behind the simulated transparency panel 410, thereby providing electronic simulated transparency. Fig. 7-8 show an exemplary embodiment of a simulated transparency panel 410. Fig. 7 illustrates a cross-sectional view of an emulated transparency component 710, which may be emulated transparency panel 410, and fig. 8 illustrates an exploded view of emulated transparency component 710 of fig. 7, in accordance with some embodiments.

In some embodiments, simulated transparency component 710 comprises two microlens arrays 720 (i.e., sensor-side microlens array 720A and display-side microlens array 720B), image sensor layer 730, circuit board 740, and electronic display layer 760. Typically, incoming light field 701 enters sensor-side microlens array 720A, where incoming light field 701 is detected by image sensor layer 730. Electronically replicated outgoing light field 702 is then generated by electronic display layer 760 and projected through display-side microlens array 720B. As explained in more detail below, the unique arrangement and features of simulated transparency component 710 allow it to provide electronic simulated transparency via electronically replicated outgoing light field 702, as well as other features described below. Although a particular shape of the simulated transparency component 710 is shown in fig. 7-8, the simulated transparency component 710 may have any suitable shape (including any polygonal or non-polygonal shape) and flat and non-flat configurations.

Microlens array 720 (i.e., sensor-side microlens array 720A and display-side microlens array 720B) is typically a multi-layer microlens. In some embodiments, each microlens of the microlens array 720 is a plenoptic cell 1510 as described in more detail below with reference to fig. 15. Typically, each microlens of sensor-side microlens array 720A is configured to capture and direct a portion of incoming light field 701 to a pixel within image sensor layer 730. Similarly, each microlens of display-side microlens array 720B is configured to emit a portion of the electronically replicated outgoing light field 702 generated by the pixels of electronic display layer 760. In some embodiments, each microlens of sensor-side microlens array 720A and display-side microlens array 720B has a 3D shape, with a collimating lens located at one end of the 3D shape. The 3D shape may be, for example, a triangular polyhedron, a rectangular cuboid, a pentagonal polyhedron, a hexagonal polyhedron, a heptagonal polyhedron, or an octagonal polyhedron. In some embodiments, each microlens of sensor-side microlens array 720A and display-side microlens array 720B includes opaque walls, such as cell walls 1514 (discussed below with reference to fig. 15), configured to prevent light from leaking into adjacent microlenses. In some embodiments, each microlens of sensor-side microlens array 720A and display-side microlens array 720B additionally or alternatively includes a light incidence angle suppression coating (such as filter layer 1640 described below) to prevent light from leaking into adjacent microlenses.

In some embodiments, the microlenses of sensor-side microlens array 720A are arranged to face a first direction, and the microlenses of display-side microlens array 720B are arranged to face a second direction that is 180 degrees relative to the first direction. In other words, some embodiments of the simulated transparency component 710 include a sensor-side microlens array 720A, the sensor-side microlens array 720A being arranged just opposite the display-side microlens array 720B. In other embodiments, any other orientation of sensor-side microlens array 720A and display-side microlens array 720B is possible.

In general, image sensor layer 730 includes a plurality of sensor pixels configured to detect incoming light field 701 after incoming light field 701 passes through sensor-side microlens array 720A. In some embodiments, the image sensor layer 730 includes an array of sensor cells 735 (e.g., sensor cells 735A-C, as shown in fig. 8). Each sensor unit 735 may be a defined portion (e.g., a particular area, such as a portion of a rectangular grid) of the image sensor layer 730 or a particular number or pattern of sensor pixels within the image sensor layer 730. In some embodiments, each sensor unit 735 corresponds to a particular logic unit 755 of the logic unit layer 750, described below. In some embodiments, the image sensor layer 730 is coupled to or otherwise immediately adjacent to the sensor-side microlens array 720A. In some embodiments, the image sensor layer 730 is located between the sensor-side microlens array 720A and the circuit board 740. In other embodiments, the image sensor layer 730 is located between the sensor-side microlens array 720A and the logic cell layer 750. In some embodiments, other suitable layers may be included in the simulated transparency component 710 on either side of the image sensor layer 730. Additionally, although a particular number and pattern of sensor units 735 are shown, any suitable number (including only one) and pattern of sensor units 735 may be used.

Circuit board 740 is any suitable rigid or flexible circuit board. Generally, the circuit board 740 includes various pads and traces that provide electrical connections between the various layers of the simulated transparency assembly 710. As an example, in embodiments including circuit board 740, circuit board 740 may be positioned between image sensor layer 730 and logic cell layer 750 as shown in fig. 7-8 to provide electrical connections between image sensor layer 730 and logic cell layer 750. In other embodiments, circuit board 740 may be positioned between logic cell layer 750 and electronic display layer 760 to provide an electrical connection between logic cell layer 750 and electronic display layer 760. In some embodiments, circuit board 740 includes an array of unit attachment locations 745 (e.g., unit attachment locations 745A-C, as shown in FIG. 8). Each unit attachment location 745 may be a defined portion (e.g., a particular area, such as a portion of a rectangular grid) of circuit board 740, and may include a plurality of pads (e.g., Ball Grid Array (BGA) pads) and/or vias. In some embodiments, each cell attachment location 745 corresponds to a particular sensor cell 735 of image sensor layer 730 and a particular display cell 765 of electronic display layer 760 (e.g., cell attachment location 745A corresponds to sensor cell 735A and display cell 765A) and is configured to allow electrical communication between the corresponding particular sensor cell 735 and the particular display cell 765.

Logic unit layer 750 provides optional/additional logic and/or processing for emulation transparency component 710. In general, logic cell layer 750 emulates transparency by directing signals from the plurality of sensor pixels of image sensor layer 730 to the plurality of display pixels of electronic display layer 760, thereby emitting electronically replicated outgoing light field 702 from display-side microlens array 720B at an angle corresponding to the angle of incoming light field 701 detected by sensor-side microlens array 720A. By emitting an electronically replicated outgoing light field 702 from display-side microlens array 720B at an angle corresponding to the angle of incoming light field 701 detected by sensor-side microlens array 720A, an image is displayed that matches the image that would be seen if simulated transparency component 710 were not present (i.e., simulated transparency). In some embodiments, logic cell layer 750 includes an array of logic cells 755 (e.g., logic cells 755A-C, as shown in fig. 8). Each logical unit 755 can be a defined portion (e.g., a particular area, such as a portion of a rectangular grid) of the logical unit layer 750. In some embodiments, each logic unit 755 is a separate physically rigid unit that is later joined or coupled to other logic units 755 in order to form a logic unit layer 750. In some embodiments, each logic cell 755 corresponds to a particular sensor cell 735 of image sensor layer 730 and a particular display cell 765 of electronic display layer 760 (e.g., logic cell 755A corresponds to (and is electrically coupled to) sensor cell 735A and display cell 765A). In some embodiments, logic unit layer 750 is located between circuit board 740 and electronic display layer 760. In other embodiments, the logic cell layer 750 is located between the image sensor layer 730 and the circuit board 740. In some embodiments, other suitable layers may be included in the emulated transparency component 710 on either side of the logical unit layer 750. Additionally, although a particular number and pattern of logic cells 755 are shown, any suitable number (including zero or only one) and pattern of logic cells 755 may be used.

In general, electronic display layer 760 includes a plurality of display pixels configured to generate electronically replicated outgoing light field 702 and project electronically replicated outgoing light field 702 through display-side microlens array 720B. In some embodiments, electronic display layer 760 comprises an array of display elements 765 (e.g., display elements 765A-C, as shown in FIG. 8). Each display unit 765 may be a defined portion (e.g., a particular area, such as a portion of a rectangular grid) of electronic display layer 760 or a particular number or pattern of display pixels within electronic display layer 760. In some embodiments, each display unit 765 corresponds to a particular logical unit 755 of logical unit tier 750. In some embodiments, electronic display layer 760 is coupled to or otherwise immediately adjacent to display-side microlens array 720B. In some embodiments, electronic display layer 760 is located between display-side microlens array 720B and circuit board 740. In other embodiments, electronic display layer 760 is located between display-side microlens array 720B and logic cell layer 750. In some embodiments, other suitable layers may be included in artificial transparency component 710 on either side of electronic display layer 760. In addition, although a particular number and pattern of display units 765 are shown, any suitable number (including only one) and pattern of display units 765 may be used.

In some embodiments, the sensor pixels of the Image sensor layer 730 may be sensor pixels 1800 as described in FIGS. 18-20 and their associated description in U.S. patent application No. 15/724,027, entitled "StackedRankine Pixel Structures for Image Sensors," which is hereby incorporated by reference in its entirety. In some embodiments, the display pixels of Electronic display layer 760 are display pixels 100 as described in FIGS. 1-4 and their associated description in U.S. patent application No. 15/724,004 entitled "StackedTranssparent Pixel Structures for Electronic Displays," which is hereby incorporated by reference in its entirety.

Although fig. 7-8 describe the emulated transparency component 710 as having an array of sensors, displays, and electronics, other embodiments may have a "single cell" structure. Additionally, while the illustrated embodiment of simulated transparency component 710 depicts unidirectional simulated transparency (i.e., allowing capture of an incoming light field 701 from a single direction and display of a corresponding electronically replicated outgoing light field 702 in the opposite direction), other embodiments may include arrangements and combinations of simulated transparency components 710 that allow bi-directional transparency.

FIG. 9 illustrates a method 900 of manufacturing the simulated transparency component 710 of FIG. 7, in accordance with certain embodiments. The method 900 may begin at step 910, where a plurality of unit attachment locations are formed on a circuit board at step 910. In some embodiments, the circuit board is circuit board 740 and the unit attachment location is unit attachment location 145. In some embodiments, each cell attachment location corresponds to one of a plurality of display cells (such as display cell 765) and one of a plurality of sensor cells (such as sensor cell 735).

At step 920, a plurality of sensor units are coupled to a first side of the circuit board. In some embodiments, the sensor unit is a sensor unit 735. In some embodiments, each sensor unit is coupled to a respective one of the unit attachment locations of step 910 in step 920. In some embodiments, the sensor cells are first formed into an image sensor layer (such as image sensor layer 730), and the image sensor layer is coupled to the first side of the circuit board in this step.

At step 930, a plurality of display units are coupled to a second side of the circuit board opposite the first side. In some embodiments, the display unit is display unit 765. In some embodiments, each display unit is coupled to a respective one of the unit attachment locations. In some embodiments, the display unit is first formed as a display layer (such as electronic display layer 760), and the display layer is coupled to the second side of the circuit board in this step.

At step 940, a first plurality of microlenses is coupled to the plurality of sensor units of step 920. In some embodiments, the microlenses are plenoptic elements 1510. In some embodiments, the microlenses are first formed into a microlens array layer (such as sensor-side microlens array 720A), and the microlens array layer is coupled to the sensor cells.

At step 950, a second plurality of microlenses is coupled to the plurality of display units of step 930. In some embodiments, the microlenses are plenoptic elements 1510. In some embodiments, the microlenses are first formed into a microlens array layer (such as display-side microlens array 720B), and the microlens array layer is coupled to the display unit. After step 950, method 900 may end.

In some embodiments, method 900 may additionally include coupling a plurality of logic units between the circuit board of step 910 and the plurality of display units of step 930. In some embodiments, the logic unit is logic unit 755. In some embodiments, the plurality of logic units are coupled between the circuit board and the plurality of sensor units of step 920.

Particular embodiments may repeat one or more steps of method 900, where appropriate. Although this disclosure describes and illustrates particular steps of method 900 as occurring in a particular order, this disclosure contemplates any suitable steps of method 900 occurring in any suitable order (e.g., any temporal order). Further, while this disclosure describes and represents an exemplary simulated transparency assembly manufacturing method including particular steps of method 900, this disclosure contemplates any suitable simulated transparency assembly manufacturing method including any suitable steps, which may include all, some, or none of the steps of method 900, where appropriate. Additionally, although this disclosure describes and illustrates particular components, devices, or systems performing particular steps of method 900, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of method 900.

FIG. 10 illustrates a direct sensor to display system 1000 that may be implemented by the simulated transparency component of FIG. 7, in accordance with certain embodiments. In general, FIG. 10 represents how an embodiment of the simulated transparency component 710 uses direct association of input pixels to corresponding output pixels. In some embodiments, this is accomplished by using a layered approach such that image sensor layer 730 and electronic display layer 760 are located near each other and mounted on opposite sides of a shared substrate (e.g., circuit board 740) as shown in fig. 7-8. Signals from image sensor layer 730 may be propagated directly to electronic display layer 760 through circuit board 740 (and in some embodiments, through circuit board 740 and logic unit layer 750). The logic cell layer 750 provides simple processing with optional inputs for any necessary control or reinforcement. Typical electronic sensor/display pairs (e.g., digital cameras) do not represent a one-to-one correspondence because the display is not directly coupled to the input sensor and therefore requires some degree of image transformation. However, certain embodiments of the present disclosure enable a one-to-one mapping between input and output pixels (i.e., the sensor pixel and display pixel layouts are the same), thereby avoiding the need for any image transformation. This reduces the complexity and power requirements of the emulated transparency component 710.

As shown in fig. 10, each sensor unit 735 is directly coupled to a corresponding display unit 765. For example, the sensor unit 735A may be directly coupled to the display unit 765A, the sensor unit 735B may be directly coupled to the display unit 765B, and so on. In some embodiments, the signaling between the sensor unit 735 and the display unit 765 may be any suitable differential signaling, such as Low Voltage Differential Signaling (LVDS). More specifically, each sensor unit 735 may output a first signal in a particular format (e.g., LVDS) corresponding to the incoming light field 701. In some embodiments, the first signal is sent via a corresponding logic unit 755, which in turn sends a second signal to the display unit 765 in the same format as the first signal (e.g., LVDS). In other embodiments, the first signal is sent directly from the sensor unit 735 to the display unit 765 (e.g., the sensor unit 735 and the display unit 765 are coupled directly to opposite sides of the circuit board 740). The display unit 765 receives the second signals from the logic unit 755 (or the first signals directly from the sensor unit 735 via the circuit board 740) and uses them to generate the outgoing light field 702.

Because no conversion is required in the signaling between the sensor unit 735 and the display unit 765, the emulated transparency component 710 may provide many benefits over a typical display/sensor combination. First, a signal processor is not required to convert the signal from the sensor unit 735 to the display unit 765. For example, an off-board signal processor is not required to perform image transformation between the sensor unit 735 and the display unit 765. This reduces the space, complexity, weight, and cost requirements of the simulated transparency component 710. Second, the emulated transparency component 710 may provide greater resolution than would normally be available for a display/sensor combination. By directly coupling the sensor unit 735 with the display unit 765 and without requiring any processing or transformation of data between the units, the resolution of the sensor unit 735 and the display unit 765 may be much greater than would normally be available. Additionally, the simulated transparency component 710 may provide different resolutions at the sensor unit 735 and the display unit 765 at any particular time. That is, a particular sensor unit 735 and a corresponding display unit 765 may have a particular resolution at a particular time that is different from the other sensor units 735 and display units 765, and the resolution of each sensor unit 735 and display unit 765 may change at any time.

In some embodiments, each particular sensor pixel of the sensor unit 735 is mapped to a single display pixel of the corresponding display unit 765, and the display pixel displays light corresponding to light captured by its mapped sensor pixel. This is best shown in fig. 17A-17B. As one example, each central sensing pixel 1725 of a particular plenoptic cell 1510 of sensor-side microlens array 720A (e.g., bottom plenoptic cell 1510 of sensor-side microlens array 720A in fig. 17A) is mapped to a central display pixel 1735 of a corresponding plenoptic cell 1510 of display-side microlens array 720B (e.g., bottom plenoptic cell 1510 of display-side microlens array 720B in fig. 17A). As another example, each top sensing pixel 1725 of a particular plenoptic cell 1510 of sensor-side microlens array 720A (e.g., top plenoptic cell 1510 of sensor-side microlens array 720A in fig. 17B) is mapped to a bottom display pixel 1735 of a corresponding plenoptic cell 1510 of display-side microlens array 720B (e.g., top plenoptic cell 1510 of display-side microlens array 720B in fig. 17B).

In some embodiments, the sensor unit 735 is coupled directly to the circuit board 740, while the display unit 765 is coupled to the logic unit 755 (which in turn is coupled to the circuit board 740), as shown in fig. 8. In other embodiments, the display unit 765 is coupled directly to the circuit board 740, while the sensor unit 735 is coupled to the logic unit 755 (which in turn is coupled to the circuit board 740). In other embodiments, both the sensor unit 735 and the display unit 765 are coupled directly to the circuit board 740 (i.e., without any intermediate logic unit 755). In such embodiments, the sensor unit 735 and the display unit 765 are coupled to opposite sides of the circuit board 740 at a unit attachment location 745 (e.g., the sensor unit 735A and the display unit 765A are coupled to opposite sides of the circuit board 740 at a unit attachment location 745A).

FIG. 11 illustrates a method 1100 of manufacturing the direct sensor-to-display system 1000 of FIG. 10, in accordance with certain embodiments. The method 1100 may begin at step 1110 where a plurality of unit attachment locations are formed on a circuit board at step 1110. In some embodiments, the circuit board is circuit board 740 and the unit attachment location is unit attachment location 745. In some embodiments, each unit attachment location corresponds to one of the plurality of display units and one of the plurality of sensor units. The display unit may be the display unit 765 and the sensor unit may be the sensor unit 735. In some embodiments, each particular cell attachment location includes a BGA pad configured to be coupled to one of the plurality of sensor cells and/or one of the plurality of logic cells. In some embodiments, each particular cell attachment location includes a plurality of interconnect pads configured to electrically couple the particular cell attachment location to one or more adjacent cell attachment locations. In some embodiments, the cell attachment locations are arranged in a plurality of columns and a plurality of rows, as shown in fig. 8.

At step 1120, a plurality of sensor units are coupled to a first side of the circuit board. In some embodiments, each sensor unit is coupled to a respective one of the unit attachment locations of step 1110. At step 1130, a plurality of display units are coupled to a second side of the circuit board opposite the first side. In some embodiments, each display unit is coupled to a respective one of the unit attachment locations of step 1110 such that each particular one of the plurality of sensor pixel units is mapped to a corresponding one of the plurality of display pixel units. By mapping each particular sensor pixel cell to one of the display pixel cells, the display pixel of each particular display pixel cell of the plurality of display pixel cells is configured to display light corresponding to light captured by the sensor pixel of its mapped sensor pixel cell. After step 1130, the method 1100 may end.

Particular embodiments may repeat one or more steps of method 1100 where appropriate. Although this disclosure describes and illustrates particular steps of method 1100 as occurring in a particular order, this disclosure contemplates any suitable steps of method 1100 occurring in any suitable order (e.g., any temporal order). Moreover, while this disclosure describes and represents an exemplary direct sensor-to-display system manufacturing method that includes particular steps of method 1100, this disclosure contemplates any suitable direct sensor-to-display system manufacturing method including any suitable steps, which may include all, some, or none of the steps of method 1100, where appropriate. Additionally, although this disclosure describes and illustrates particular components, devices, or systems performing particular steps of method 1100, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of method 1100.

Fig. 12-13 represent various intra-layer signal processing structures that may be used by the simulated transparency component 710 of fig. 7, in accordance with certain embodiments. In general, the architecture of fig. 12-13 uses a layer of digital logic (e.g., logic cell layer 750) sandwiched between the camera and the display (i.e., sandwiched between image sensor layer 730 and electronic display layer 760). These architectures allow for locally distributed processing of large amounts of data (e.g., 160k of image data or more), thereby avoiding the bottlenecks and performance, power, and transmission line issues associated with typical architectures. Human visual acuity represents a large amount of data that must be processed in real time. Typical imaging systems propagate a single data stream to/from a high performance processor (e.g., CPU or GPU), which may or may not serialize data for manipulation. The bandwidth required for this scheme at human 20/20 visual acuity far exceeds the bandwidth of any known transmission protocol. Typical systems also use a master controller that is responsible for processing all incoming/outgoing data or managing the allocation to smaller processing nodes. In any event, all data must be transmitted off-system/off-chip, manipulated, and then returned to the display device. However, this typical scheme cannot handle the large amount of data required for human visual acuity. However, embodiments of the present disclosure take advantage of the faceted nature of the sensor/display combination as described herein to disperse and localize the signal processing. This enables real-time digital image processing that was previously impractical.

As shown in fig. 12-13, some embodiments of artificial transparency component 710 include a layer of logic cells 750, logic cell layer 750 containing the necessary logic to manipulate input signals from image sensor layer 730 and provide output signals to electronic display layer 760. In some embodiments, the logic cell layer 750 is located between the image sensor layer 730 and the circuit board 740, as shown in fig. 12. In other embodiments, logic cell layer 750 is located between circuit board 740 and electronic display layer 760, as shown in FIG. 13. In general, logic unit layer 750 is a dedicated image processing layer that is capable of mixing input signals directly from image sensor layer 730 and performing one or more mathematical operations (e.g., matrix transformations) on the input signals before outputting the resulting signals directly to electronic display layer 760. Since each logical unit 755 of the logical unit layer 750 is responsible for only its associated facet (i.e., sensor unit 735 or display unit 765), the data of a particular logical unit 755 can be manipulated without significantly impacting system level I/O. This effectively avoids the need to parallelize any incoming sensor data for centralized processing. The distributed approach enables the simulated transparency component 710 to provide a number of features such as magnification/zoom (each facet applies a scaling transform to its input), vision correction (each facet applies a simulated optical transform, compensating for common vision problems such as myopia, hyperopia, astigmatism, etc.), color blindness correction (each facet applies a color transform, compensating for common color blindness problems), polarization (each facet applies a transform that simulates wave polarization, allowing glare reduction), and dynamic range reduction (each facet applies a transform that darkens high intensity regions (e.g., the sun) and lightens low intensity regions (e.g., shadows)). In addition, long transmission lines may not be needed since any data transformation remains confined to the logic cell layer 750 of each facet. This avoids problems with crosstalk, signal integrity, etc. In addition, since the disclosed embodiments do not require optical transparency (but instead utilize simulated transparency), there is no functional impact to placing an opaque handling layer between the sensor and the display facet.

In some embodiments, the logic cell layer 750 includes discrete logic cells (e.g., transistors) formed directly on the circuit board 740. For example, standard photolithographic techniques may be used to form the logic cell layer 750 directly on the circuit board 740. In other embodiments, each logic unit 755 is a separate Integrated Circuit (IC) that is coupled to a sensor facet or a display facet or directly to the circuit board 740. As used herein, "facet" refers to a discrete unit that is separately manufactured and subsequently coupled to the circuit board 740. For example, "display facet" may represent a cell comprising a combination of electronic display layer 760 and display-side microlens array 720B, and "sensor facet" may represent a cell comprising a combination of image sensor layer 730 and sensor-side microlens array 720A. In some embodiments, the display facet may include a single display unit 765, or it may include multiple display units 765. Similarly, the sensor facet may comprise a single sensor unit 735, or it may comprise a plurality of sensor units 735. In some embodiments, logic 755 may be included in a sensor facet or a display facet. In embodiments where logic 755 is a separate IC coupled directly to the display or sensor facet (rather than formed directly on circuit board 740), any suitable technique, such as a 3D IC design with through silicon vias, may be used to couple the ICs of logic 755 to the wafer of facets.

In some embodiments, the logic unit layer 750 is an Application Specific Integrated Circuit (ASIC) or an Arithmetic Logic Unit (ALU), rather than a general purpose processor. This allows the logic cell layer 750 to be power efficient. In addition, this allows the logic cell layer 750 to operate without cooling, further reducing the cost and power requirements of the emulated transparency component 710.

In some embodiments, the logic unit 755 is configured to communicate using the same protocol as the sensor unit 735 and the display unit 765. For example, in embodiments where the logic 755 is a discrete IC, the IC may be configured to interface with the sensor and display facetSame protocol (e.g., LVDS or built-in integrated circuit (I)²C) ) communication. This eliminates the problem of having to switch between the sensor and the display facet, thereby reducing power and cost.

In some embodiments, logic unit layer 750 performs one or more operations on signals received from image sensor layer 730 before sending output signals to electronic display layer 760. For example, logic unit layer 750 may transform the received signals from image sensor layer 730 to include enhanced information for display on electronic display layer 760. This may be used, for example, to provide AR to the viewer. In some embodiments, logic unit layer 750 may completely replace the received signals from image sensor layer 730 with alternative information for display on electronic display layer 760. This may be used, for example, to provide VR to the observer.

Fig. 14 illustrates a method 1400 of fabricating the intra-layer signal processing system of fig. 12-13, in accordance with certain embodiments. The method 1400 may begin at step 1410, where a plurality of sensor units are coupled to a first side of a circuit board at step 1410. In some embodiments, the sensor unit is a sensor unit 735, and the circuit board is a circuit board 740. In some embodiments, each sensor unit is coupled to one of a plurality of unit attachment locations (such as unit attachment location 745). Each sensor unit includes a plurality of sensor pixels.

At step 1420, a plurality of display cells are formed. In some embodiments, the display unit is a combination of a display unit 765 and a logic unit 755. Each display cell may be formed by combining the electronic display and logic cells into a single 3D integrated circuit using through silicon vias. Each display unit includes a plurality of display pixels.

At step 1430, the plurality of display cells of step 1420 are coupled to a second side of the circuit board opposite the first side. In some embodiments, each logic unit is coupled to a respective one of the unit attachment locations. After step 1430, the method 1400 may end.

Particular embodiments may repeat one or more steps of method 1400 where appropriate. Although this disclosure describes and illustrates particular steps of the method 1400 as occurring in a particular order, this disclosure contemplates any suitable steps of the method 1400 occurring in any suitable order (e.g., any temporal order). Moreover, while this disclosure describes and represents an exemplary intra-layer signal processing system fabrication method including particular steps of method 1400, this disclosure contemplates any suitable intra-layer signal processing system fabrication method including any suitable steps, which may include all, some, or none of the steps of method 1400, where appropriate. Additionally, although this disclosure describes and illustrates particular components, devices, or systems performing particular steps of method 1400, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of method 1400.

Fig. 15-17C represent various views of an array 1500 of plenoptic cells 1510 that can be used within microlens arrays 720A-B of the simulated transparency component 710. Fig. 15 shows a plenoptic component 1500, fig. 16 shows a cross-section of a portion of the plenoptic component 1500 of fig. 15, and fig. 17A-17C show cross-sections of a portion of the plenoptic component 1500 of fig. 15 with various incoming and outgoing light fields.

Standard electronic displays typically include a planar arrangement of pixels that form a two-dimensional rasterized image, conveying two-dimensional data in an intrinsic manner. One limitation is that: the planar image cannot be rotated in order to perceive different perspectives within the scene being transmitted. In order to clearly view this image, the viewer's eye or the camera's lens must be focused on the screen, regardless of what is depicted within the image itself. In contrast, a certain amount of light entering the eye from the real world allows the eye to naturally focus on any point within the certain amount of light. Because the rays from the scene naturally enter the eye, this plenoptic "field" of light contains rays from the scene, rather than a virtual image focused by an external lens at a single focal plane. Although existing light field displays may be able to replicate this phenomenon, they provide a significant tradeoff between spatial and angular resolution, resulting in a perceived amount of light that appears blurred or insufficient in detail.

To overcome the problems and limitations associated with existing light field displays, embodiments of the present disclosure provide a coupled light field capture and display system that is capable of recording and subsequently electronically recreating an incoming full amount of light. Both capture and display processing are accomplished by the arrangement of plenoptic cells 1510 responsible for recording or displaying the smaller field of view of the larger composite image. Each plenoptic cell 1510 of the sensor itself comprises a dense cluster of image sensor pixels, and each plenoptic cell of the display itself comprises a dense cluster of display pixels. In both cases, light rays entering the sensor cell or exiting the display cell are focused by one or more transparent lenslets 1512 to produce a precisely tuned distribution of nearly collimated light rays. This essentially records the incoming light field and reproduces it on the opposite side of the assembly. More specifically, for the sensor, the amount of light entering the lens (or series of lenses) of this cell is focused on the image pixels such that each pixel concentrates light from only one direction, as determined by its position within the cell and the outline of the lens. This allows for rasterized encoding of various angle rays within the light field, the number of pixels in the primitive determining the angular resolution of the recording. For a display, the light emitted from a pixel is focused by the same lens (or series of lenses) to create an amount of light that matches the image recorded by the sensor, as well as any electronic enhancements or changes (e.g., from the logic cell layer 750 described above). The cone of light emitted from this cell contains a subset of rays at a sufficient angular separation to enable the formation of a light field for the viewer, with each output ray direction being determined by the position of its originating pixel within the cell and the profile of the lens.

Plenoptic cell 1510 can be used by both sensor-side microlens array 720A and display-side microlens array 720B. For example, a plurality of plenoptic cells 1510A can be included in the sensor-side microlens array 720A, and each plenoptic cell 1510A can be coupled to an image sensor 1520 or otherwise adjacent to the image sensor 1520. The image sensor 1520 may be part of the image sensor layer 730 and may include a sensor pixel array 1525, the sensor pixel array 1525 including sensing pixels 1725. Similarly, a plurality of plenoptic cells 1510B can be included in the display-side microlens array 720B, and each plenoptic cell 1510B can be coupled to the display 1530 or otherwise adjacent to the display 1530. Display 1530 may be part of electronic display layer 760 and may include an array of display pixels 1625, display pixels 1625 including display pixels 1735. The sense pixel 1725 can be a sensor pixel 1800 as described in FIGS. 18-20 and their associated description in U.S. patent application No. 15/724,027, entitled "Stacked distributed Pixel structures for Image Sensors," which is hereby incorporated by reference in its entirety. Display Pixel 1735 may be display Pixel 100 as described in FIGS. 1-4 and their associated description in U.S. patent application No. 15/724,004 entitled "Stacked Transparent Pixel Structures for electronic displays," which is hereby incorporated by reference in its entirety.

In some embodiments, the all-optical primitive 1510 includes transparent lenslets 1512 and primitive walls 1514. In particular, the all-light cell 1510A includes transparent lenslets 1512A and cell walls 1514A, and the all-light cell 1510B includes transparent lenslets 1512B and cell walls 1514B. In some embodiments, transparent lenslets 1512 comprise a 3D shape with a collimating lens at one end of the 3D shape. For example, as shown in FIG. 15, the transparent lenslets 1512 may be rectangular cuboids with a collimating lens located at one end of the rectangular cuboid. In other embodiments, the 3D shapes of transparent lenslets 1512 may be triangular polyhedrons, pentagonal polyhedrons, hexagonal polyhedrons, heptagonal polyhedrons, octagonal polyhedrons, cylinders, or any other suitable shape. Each plenoptic cell 1510A includes an input field of view (FOV)1610 (e.g., 30 degrees), and each plenoptic cell 1510B includes an output FOV1620 (e.g., 30 degrees). In some embodiments, the input FOV 1610 matches the output FOV1620 of the corresponding plenoptic primitive 1510.

Transparent lenslets 1512 may be formed of any suitable transparent optical material. For example, transparent lenslets 1512 may be formed of a polymer, silica glass, or sapphire. In some embodiments, transparent lenslets 1512 may be formed of a polymer, such as polycarbonate or acrylic. In some embodiments, transparent lenslets 1512 may be replaced with waveguides and/or photonic crystals in order to capture and/or generate optical fields.

Typically, cell walls 1514 are barriers for preventing optical crosstalk between adjacent all-optical cells 1510. The cell walls 1514 may be formed of any suitable material that is opaque to visible light when hardened. In some embodiments, the cell walls 1514 are formed from a polymer. The use of the cell walls 1514 to prevent optical crosstalk is described in more detail below with reference to fig. 17A and 17C.

In some embodiments, the image sensor 1520 includes or is coupled to backplane circuitry 1630a and the display 1530 includes or is coupled to backplane circuitry 1630 b. Typically, backplane circuitry 1630a-B provides electrical connections to allow image data to flow from image sensor 1520 to display 1530. In some embodiments, backplane circuitry 1630a and backplane circuitry 1630b are opposing sides of a single backplane. In some embodiments, backplane circuitry 1630a and backplane circuitry 1630b are circuit boards 740.

In some embodiments, filter layers 1640 may be included at one or both ends of transparent lenslets 1512 to limit light entry or exit to particular angles of incidence. For example, a first filter layer 1640A may be included at the protruding ends of transparent lenslets 1512, and/or a second filter layer 1640B may be included at the opposite ends of transparent lenslets 1512. Similar to cell walls 1514, such coatings or films may also limit image leakage between adjacent transparent lenslets 1512 to acceptable amounts. A filter layer 1640 may be used in addition to or instead of the cell walls 1514.

Each of fig. 17A-17C represents a cross-sectional view of seven adjacent plenoptic cells 1510 of a sensor-side microlens array 720A and a corresponding display-side microlens array 720B. These figures show how the incoming light field 701 is captured by the image sensor 1520 and electronically replicated on the display 1530 to emit nearly the same light field. In fig. 17A, the incoming light field 1710 from an object directly in front of the sensor plenoptic element 1510 is focused on the central sensing pixel 1725 by the transparent lenslets 1512 of the sensor plenoptic element 1510. The corresponding light is then sent by the corresponding center display pixel 1735 of the corresponding display plenoptic 1510. The transmitted light is focused by transparent lenslets 1512 of the display plenoptic 1510 and emitted as an emitted light field 1711. The emitted light field 1711 matches exactly the zero-degree source light field (i.e., the incoming light field 1710). Additionally, emitted light rays that would otherwise penetrate into the neighboring display plenoptic cell 1510 that strike cell walls 1514 at location 1740 are blocked by opaque cell walls 1514, thereby preventing optical crosstalk.

In fig. 17B, the incoming light field 1720 from an object fourteen degrees off the axis of the sensor plenoptic cell 1510 is focused by transparent lenslets 1512 of the sensor plenoptic cell 1510 on top sensing pixels 1725. The corresponding light is then sent by the corresponding opposing (i.e., bottom) display pixel 1735 of the corresponding display plenoptic 1510. The transmitted light is focused by transparent lenslets 1512 of the display plenoptic 1510 and emitted as an emitted light field 1721. The emitted light field 1721 exactly matches the 14-degree source light field (i.e., the incoming light field 1720).

In fig. 17C, the incoming light field 1730 from an object 25 degrees off the axis of the sensor plenoptic cell 1510 is focused completely on the cell walls 1514 by the transparent lenslets 1512 of the sensor plenoptic cell 1510. Because the incoming light field 1730 is focused entirely on the cell walls 1514 of the sensor plenoptic cell 1510, rather than on the sensing pixels 1725, no corresponding light is sent by the corresponding display plenoptic cell 1510. In addition, incoming light rays striking cell walls 1514 at location 1750 that would otherwise penetrate into an adjacent sensor plenoptic cell 1510 are blocked by opaque cell walls 1514, thereby preventing optical crosstalk.

Fig. 18A-18B illustrate a method of fabricating the all-optical cell assembly of fig. 15, in accordance with certain embodiments. In fig. 18A, a microlens array (MLA) plate 1810 is formed or obtained. The MLA plate 1810 includes a plurality of lenslets, as shown in the figure. In fig. 18B, a plurality of grooves 1820 are cut to a predetermined depth around each of the plurality of lenslets of the MLA plate 1810. In some embodiments, the notch 1820 may be cut using multiple passes to achieve a desired depth. In some embodiments, the groove 1820 may be cut using laser ablation, etching, photolithographic processing, or any other suitable method. After the grooves 1820 are cut to a desired depth, they are filled with a material configured to prevent light from leaking through the grooves 1820. In some embodiments, the material is any light absorbing (e.g., carbon nanotubes) or opaque material (e.g., non-reflective opaque material or colored polymer) when hardened. The resulting all-optical primitive assembly after the groove 1820 is filled and allowed to harden is shown in fig. 20-21.

Figures 19A-19B illustrate another method of fabricating the all-optical cellular component of figure 15, in accordance with certain embodiments. In fig. 19A, a pre-formed grid structure 1830 with gaps 1840 is obtained or formed. The grid structure 1830 is made of any suitable material as described above for the cell walls 1514. The grid structure 1830 may be formed by any suitable method, including but not limited to additive manufacturing and ablation of elemental material.

In fig. 19B, the gap 1840 is filled with an optical polymer 1850. Optical polymer 1850 can be any suitable material as described above for transparent lenslets 1512. After the gaps 1840 are filled with the optical polymer 1850, a final lens profile is created using molding or ablation. Examples of plenoptic components obtained after forming the lens are shown in fig. 20-21.

Fig. 22-23 illustrate a flexible circuit board 2210 that may be used as a circuit board 740 by the artificial transparency assembly 710 of fig. 7, according to some embodiments. In general, wrapping an electronic device around a 3D shape (such as a spherical or hemispherical surface) is an important task. While various examples of flexible and even stretchable circuitry currently exist, there are several obstacles to overcome when placing such electronics on small radius (e.g., 30-60mm) spherical or hemispherical surfaces. For example, bending of a flexible electronic device substrate in one direction does not inherently indicate compliance with compound curvature, as the torsional forces required for such curvature can damage the involved thin films. As another example, there remains a problem with the degree of stretchability and longevity of currently available stretchable electronic devices.

To address the problems and limitations of the current solutions, embodiments of the present disclosure provide a 3D (e.g., spherical or hemispherical) electronic device manufacturing method using a geodesic facet solution that includes an array of small rigid surfaces arranged on a single flexible circuit. In some embodiments, the flexible circuit is cut into a particular mesh shape, then wrapped into a 3D shape (e.g., a spherical or hemispherical shape) and locked into place to prevent wear from repeated bending. The method is particularly useful for accommodating the narrow radius of curvature (e.g., 30-60mm) required for head mounted near-eye wrap displays. In some embodiments, the assembly includes a single base flexible printed circuit system layer, with the rigid sensor and display arrays disposed on opposite sides of the flexible circuit. The entire assembly, including the sensor and display layers, can be fabricated by standard planar semiconductor processes (e.g., spin coating, photolithography, etc.). The rigid electronics layer may be etched to form individual sensors and display cells (i.e., "facets"), then connected to the flexible circuitry by connection pads and bonded by patterned conductive and non-conductive adhesives. This allows the flexible circuitry to fold slightly at the edges between the rigid facets. In some embodiments, planar fabrication is followed, and the fully cured functional electronics stack is formed into the desired final 3D shape using one side of the final rigid polymer housing as a mold. In this way, the array of rigid electronic device facets are not deformed, but simply fall into place in their mold, with the flexible circuitry bending at defined creases/gaps to match the interior of the housing facets. The assembly may finally be covered and sealed with the opposing mating sides using a rigid housing.

Embodiments of the present disclosure are not limited to only spherical or hemispherical shapes, but such shapes are certainly contemplated. The disclosed embodiments may be formed in any compound curvature or any other rotational shape. Additionally, the disclosed embodiments can be formed with any non-uniform curvature as well as non-curved (i.e., flat) surfaces.

Fig. 22 shows the flexible circuit board 2210 in two different states: a flat flexible circuit board 2210A and a 3D shaped flexible circuit board 2210B. The flexible circuit board 2210 includes facet locations 2220, the facet locations 2220 generally being locations where facets (e.g., the sensor facet 3735, the display facet 2665, or the logic facet 2655 discussed below) may be mounted on the flexible circuit board 2210. In some embodiments, the flexible circuit board 2210 includes a gap 2215. As shown at the bottom of fig. 22, when the flexible circuit board 2210 is flat, at least some of the facet locations 2220 are separated from one or more adjacent facet locations 2220 by one or more gaps 2215. As shown at the top of fig. 22, when the flexible circuit board 2210 is formed into a 3D shape, the gap 2215 may be substantially eliminated, thereby forming a continuous surface across at least some of the facets coupled at the facet location 2220 (e.g., a continuous sensing surface across multiple sensor facets 3735 or a continuous display surface across multiple display facets 2665).

In general, facet locations 2220 may have any shape. In some embodiments, the facet locations 2220 have a polygonal shape (e.g., triangular, square, rectangular, pentagonal, hexagonal, heptagonal, or octagonal). In some embodiments, facet locations 2220 are all the same. However, in other embodiments, the facet locations 2220 all share the same polygonal shape (e.g., are all hexagons), but have different sizes. In some embodiments, the facet locations 2220 have different shapes (e.g., some are rectangular and some are hexagonal). Any suitable shape for facet location 2220 may be used.

In some embodiments, the facet locations 2220 are arranged in columns 2201. In some embodiments, the facet locations 2220 are additionally or alternatively arranged in rows 2202. Although a particular pattern of facet locations 2220 is shown, any suitable pattern of facet locations 2220 may be used.

Fig. 23 shows additional details of a flexible circuit board 2210 according to some embodiments. In some embodiments, each facet location 2220 includes pads and/or vias for coupling the sensor or display facets to the flexible circuit board 2210. By way of example, some embodiments of the flexible circuit board 2210 include BGA pads 2240 at each facet location 2220. Any suitable pattern and number of pads/vias may be included at each facet location 2220.

In general, each particular facet location 2220 is configured to send signals between a particular sensor facet coupled to that particular facet location and a particular display facet coupled to the opposite side of that particular facet location. For example, a particular facet location 2220 may have a sensor facet 3735 coupled to one side and a display facet 2665 coupled to its opposite side. The particular facet location 2220 provides the necessary electrical connections to allow signals from the sensor facet 3735 to pass directly to the display facet 2665, thereby enabling the display facet 2665 to display light corresponding to the light captured by the sensor facet 3735.

In some embodiments, wiring traces 2230 are included on the flexible circuit board 2210 to electrically connect the facet locations 2220. For example, a wire trace 2230 may be connected to the interconnect pad 2250 of each facet location 2220 to electrically connect adjacent facet locations 2220. In some embodiments, facet locations 2220 are connected in series via wiring traces 2230. For example, fig. 24 shows a serial data stream through a flexible circuit board 2210 in accordance with some embodiments. In this example, each facet location 2220 is assigned a unique identifier (e.g., "1," "2," etc.), and data flows serially through the facet locations 2220 via wire traces 2230, as shown in the figure. In this manner, each facet location 2220 may be addressed by a single processor or logic unit using its unique identifier. Any suitable addressing scheme and data flow pattern may be used.

Fig. 25 illustrates a method 2500 of manufacturing an electronic assembly using the flexible circuit board 2210 of fig. 22, in accordance with some embodiments. At step 2510, a plurality of facet locations are formed on the flexible circuit board. In some embodiments, the facet locations are facet locations 2220 and the flexible circuit board is a flexible circuit board 2210. Each facet location corresponds to one of the plurality of sensor facets and one of the plurality of display facets. The sensor facet may be sensor facet 3735 and the display facet may be display facet 2665. In some embodiments, the plurality of facet locations are arranged as a plurality of facet columns, such as column 2201. In some embodiments, the plurality of facet locations are additionally or alternatively arranged as a plurality of rows of facets, such as row 2202.

At step 2520, the flexible circuit board of step 2510 is cut or otherwise formed into a pattern that allows the flexible circuit board to be later formed into a 3D shape, such as a spherical or hemispherical shape. When the flexible circuit board is flat, at least some of the facet locations are separated from one or more adjacent facet locations by a plurality of gaps (such as gap 2215). The plurality of gaps are substantially eliminated when the flexible circuit board is formed into a 3D shape.

At step 2530, the electronic assembly is assembled by coupling a first plurality of rigid facets to a first side of a flexible circuit board. The first plurality of rigid facets may be sensor facets 3735 or display facets 2665. Each rigid facet is coupled to a respective one of the facet locations. In some embodiments, the first plurality of rigid facets are coupled to connection pads of a first side of a flexible circuit board using a patterned conductive and non-conductive adhesive.

In some embodiments, the first plurality of rigid facets of step 2530 are rigid sensor facets (such as sensor facet 3735), and method 2500 further includes coupling a plurality of rigid display facets (such as display facet 2665) to a second side of the flexible circuit board opposite the first side. In this case, each particular facet location is configured to send a signal between a particular rigid sensor facet electrically coupled to the particular facet location and a particular rigid display facet electrically coupled to the same particular facet location. This allows light from the particular rigid display facet corresponding to the light captured by the corresponding rigid sensor facet to be displayed.

At step 2540, the assembled electronic component is formed into a desired 3D shape. In some embodiments, this step comprises: the flexible circuit board with its coupled rigid facets is placed on one side of a rigid housing having the desired shape. This allows the rigid facets to fall into defined spaces in the housing and the flexible circuit board to bend at defined creases/gaps between the rigid facets. After placing the flexible circuit board with its coupled rigid facets on one side of the rigid housing, the opposing mating side of the rigid housing may be attached to the first side, thereby sealing the assembly into a desired shape.

Particular embodiments may repeat one or more steps of method 2500 where appropriate. Although this disclosure describes and illustrates particular steps of method 2500 as occurring in a particular order, this disclosure contemplates any suitable steps of method 2500 occurring in any suitable order (e.g., any temporal order). Further, while this disclosure describes and represents an exemplary method of manufacturing an electronic assembly using a flexible circuit board, this disclosure contemplates any suitable method of manufacturing an electronic assembly using a flexible circuit board, which may include all, some, or none of the steps of method 2500, where appropriate. Additionally, although this disclosure describes and illustrates particular components, devices, or systems performing particular steps of method 2500, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of method 2500.

FIGS. 26-36 illustrate a distributed multi-screen array for high density displays according to some embodiments. In general, to provide a near-eye display capable of simulating the entire field of view of a single human eye, a high dynamic range image display having a resolution order of magnitude greater than that of current common display screens is required. Such a display should be able to provide a light field display with an angular resolution and a spatial resolution sufficient to accommodate 20/20 human visual acuity. This is a large amount of information equivalent to a total horizontal pixel count of 100K to 200K. These displays should also surround the entire field of view of a human eye (approximately 160 deg. horizontally and 130 deg. vertically). To present binocular vision, a pair of such displays would be required that span the entire curved surface around each eye. However, typical displays available today do not meet these requirements.

To address these and other limitations of current displays, embodiments of the present disclosure provide an array of custom sized and shaped small high-resolution microdisplays (e.g., display facets 2665), all of which are formed and then assembled on a larger flexible circuit board 2210, which flexible circuit board 2210 may be formed into a 3D shape (e.g., a hemispherical surface). The microdisplay can be mounted inside the hemispherical circuitry where another layer containing an array of TFT logic cells (e.g., logic cells 755) can be included to handle all power and signal management. Typically, one logic unit 755 may be included for each microdisplay. Each microdisplay acts as a separate unit, displaying data from the logic unit behind it. Any additional information (e.g., such as external video for AR, VR, or MR applications) may be passed to the entire array via the central control processor. In some embodiments, the external data signals are advanced from one microdisplay to the next in sequence as a packetized multiplexed stream, while the TFT logic for each display determines the source and portion of the read signal. This allows each cell to operate independently of any other display, providing a large array of many high resolution displays with unique content on each high resolution display, such that the entire assembly together forms essentially a single very high resolution display.

To meet the requirements of resolution, color definition, and luminance output, each microdisplay may have a unique high-performance pixel architecture. For example, each microdisplay screen may include an array of display pixels 100 as described in FIGS. 1-4 and their associated description in U.S. patent application No. 15/724,004 entitled "Stacked Transparent Pixel structures for Electronic Displays," which is hereby incorporated by reference in its entirety. The microdisplay screen can be assembled on the same substrate using any suitable method. Such simultaneous manufacturing using standard semiconductor layering and photolithography processes virtually eliminates the overhead and costs associated with the production and packaging of many individual screens, greatly increasing affordability.

Fig. 26 illustrates a cross-sectional view of a curved multi-display array 2600 in accordance with certain embodiments. FIG. 26 is essentially the back side of the flexible circuit board 2210B of FIG. 22, with the addition of a logic facet 2655 and a display facet 2665 coupled to the flexible circuit board 2210B at facet locations 2220. In general, each logical facet 2655 is an individual logical unit 755 from the logical unit layer 750. Similarly, each display facet 2665 is an individual display unit 765 of display layer 760 that is coupled to a portion of microlens array 720.

In some embodiments, each individual logic facet 2655 is coupled to a flexible circuit board 2210, and each individual display facet 2665 is then coupled to one of the logic facets 2655. In other embodiments, each logic facet 2655 is first coupled to one of the display facets 2665, and the combined facet is then coupled to the flexible circuit board 2210. In such an embodiment, the combined logic facet 2655 and display facet 2665 may be referred to as a display facet 2665 for simplicity. As used herein, "display facet" may represent two embodiments (i.e., individual display facet 2665 or a combination of display facet 2665 and logic facet 2655).

In general, each display facet 2665 can be individually addressable (e.g., addressed by a central control processor, not shown), and an array of display facets 2665 can represent a dynamic heterogeneous set that forms a single collective. In other words, multi-display array 2600 provides a tiled electronic display system that displays images through individual display facets 2665 that together form a complete whole. Each individual display facet 2665 can provide a variety of different display resolutions and can be customized on the fly to run different resolutions, color ranges, frame rates, etc. For example, one display facet 2665 may have a 512x512 display resolution while an adjacent display facet 2665 (of the same size) has a 128x128 display resolution, where the former represents a higher density of image data. In this example, the two displays are different, but individually controllable and work in concert to form a single display image.

The array of display facets 2665 as a whole can employ any curved or flat surface structure. For example, display facets 2665 may be formed as hemispherical surfaces, cylindrical surfaces, long spherical surfaces (elongated spherical surfaces), or any other shaped surface.

The logic facet 2655 and the display facet 2665 may have any suitable shape. In some embodiments, the shapes of the logic facet 2655 and the display facet 2665 match each other and match the shape of the facet location 2220. In some embodiments, logic facet 2655 and display facet 2665 have polygonal shapes, such as triangular, quadrilateral, pentagonal, hexagonal, heptagonal, or octagonal. In some embodiments, some or all of logic facets 2655 and display facets 2665 have non-polygonal shapes. For example, the display facets 2665 at the edges of the flexible circuit board 2210 may not be polygonal, as they may have curved cuts to improve the aesthetics of the overall assembly.

In addition to having a selectable/controllable display resolution, each display facet 2665 may also have a color range selectable from a plurality of color ranges and/or a frame rate selectable from a plurality of frame rates in some embodiments. In such an embodiment, the display facets 2665 of a particular flexible circuit board 2210 may be configured to provide different frame rates and different color ranges. For example, one display facet 2665 may have a particular color range while another display facet 2665 has a different color range. Similarly, one display facet 2665 may have a particular frame rate while the other display facet 2665 has a different frame rate.

Fig. 27 shows an exploded view of the curved multi-display array 2600 of fig. 26, and fig. 28-29 show additional details of the logic facet 2655 and the display facet 2665 according to some embodiments. As shown in these figures, each logic facet 2655 may include an interconnect pad 2850, and interconnect pad 2850 may be electrically coupled to interconnect pad 2250 of an adjacent logic facet 2655. This may enable the display facets 2665 to be coupled in series via the wiring traces 2230. In addition, each logic facet 2655 may include a pad 2840 having a pattern that matches pad 2940 of the back side of display facet 2665. This allows the logic facet 2655 and display facet 2665 to be coupled together using any suitable technique in the art. In some embodiments, pads 2840 and 2940 are BGA pads or any other suitable surface mount pads.

Fig. 30 and 32 show the back side of the flexible circuit board 2210 of fig. 22, and show similar details as described with reference to fig. 23. Fig. 31 and 33 represent serial data flow through the flexible circuit board 2210 and show similar details as described with reference to fig. 24. Fig. 34 illustrates an array of logic facets 2655 that have been formed into hemispherical shapes, in accordance with certain embodiments. In this figure, the flexible circuit board 2210 and the display facet 2665 have been removed for clarity. Fig. 35 illustrates communication between the logical facets 2655 of fig. 34 in accordance with certain embodiments. As shown in this figure, each logic facet 2655 may communicate with an adjacent logic facet 2655 using interconnect pads 2850. In addition, each logical facet 2655 may have a unique identification, as shown in fig. 35. This allows each logical facet 2655 to be uniquely addressed by, for example, a central processing unit.

FIG. 36 illustrates a method 3600 of fabricating the curved multi-display array of FIG. 26, according to some embodiments. The method 3600 may begin at step 3610 where a plurality of facet locations are formed on a circuit board at step 3610. In some embodiments, the facet locations are facet locations 2220 and the circuit board is a flexible circuit board 2210. In some embodiments, each facet position corresponds to one of a plurality of display facets (such as display facet 2665).

At step 3620, the flexible circuit board is cut or otherwise formed into a pattern that allows the flexible circuit board to be later formed into a 3D shape. When the flexible circuit board is flat, at least some of the facet locations are separated from one or more adjacent facet locations by a plurality of gaps (such as gap 2215). The plurality of gaps are substantially eliminated when the flexible circuit board is formed into a 3D shape.

At step 3630, a plurality of logic facets are coupled to a first side of the flexible circuit board. Each logical facet is coupled to a respective one of the facet locations of step 3610. At step 3640, a plurality of display facets are coupled to a respective one of the plurality of logic facets of step 3630. In an alternative embodiment, the display facet may be mounted to the logic facet of step 3630 at the wafer level prior to coupling the logic facet to the first side of the flexible circuit board. At step 3650, the assembled electronic display assembly is formed into a 3D shape. In some embodiments, this step may be similar to step 2540 of method 2500 described above. After step 3650, the method 3600 may end.

Particular embodiments may repeat one or more steps of method 3600 where appropriate. Although this disclosure describes and illustrates particular steps of method 3600 as occurring in a particular order, this disclosure contemplates any suitable steps of method 3600 occurring in any suitable order (e.g., any temporal order). Further, while this disclosure describes and represents an exemplary method of manufacturing a curved multi-display array, this disclosure contemplates any suitable method of manufacturing a curved multi-display array that may include all, some, or none of the steps of method 3600, where appropriate. Additionally, although this disclosure describes and represents particular components, devices, or systems performing particular steps of method 3600, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of method 3600.

Fig. 37-42 illustrate a distributed multi-aperture camera array 3700 according to some embodiments. Typically, in order to capture the entire light field of the entire field of view of a single human eye, a large high dynamic range image sensor with a resolution much higher than currently available is required. Such an image sensor would enable a light field camera with angular and spatial resolution sufficient to accommodate 20/20 human visual sensitivity. This is a large amount of information equivalent to a total horizontal pixel count of 100K to 200K. Such a multi-aperture image sensor must also encompass the entire field of view of a human eye (approximately 160 deg. horizontally and 130 deg. vertically). To image binocular vision, a pair of such cameras across the entire curved surface around each eye is required. Typical image sensor assemblies available today do not meet these requirements.

To overcome these and other limitations of typical image sensors, embodiments of the present disclosure provide an array of custom sized and shaped small image sensors, all assembled on a larger flexible circuit board 2210, the flexible circuit board 2210 being formed into a 3D (e.g., hemispherical) shape. An image sensor (e.g., sensor facet 3735) is mounted to the outside of the flexible circuit board 2210, where another layer containing an array of TFT logic cells (e.g., logic cells 755) may be provided to handle all power and signal management — one logic cell is provided for each display. Each image sensor acts as a discrete unit, passing the readout data to its subsequent logic (in embodiments that include logic), where it is processed and routed accordingly (e.g., in some embodiments, to the corresponding display facet 2665). This allows each sensor facet 3735 to operate independently of any other sensor facet 3735, providing a large array of many apertures, capturing unique content on each aperture, so that the entire assembly becomes essentially a seamless, very high resolution multi-node camera. It should be noted that although in some embodiments the image sensors may pass data to their companion logic, the functionality of the image sensors themselves does not necessarily require logic coupling.

To meet the requirements of resolution, color definition, and luminance output, each microsensor may have a unique high-performance pixel architecture. For example, each microsensor can comprise an array of sensor pixels 1800 as described in FIGS. 18-20 and their associated description in U.S. patent application No. 15/724,027 entitled "Stacked transmissive Pixel structures for Image Sensors," which is hereby incorporated by reference in its entirety. The microsensors can be assembled on the same substrate using any suitable method. Such simultaneous manufacturing using standard semiconductor layering and photolithography processes virtually eliminates the overhead and costs associated with the production and packaging of many individual screens, greatly increasing affordability.

Another characteristic of some embodiments of the distributed multi-aperture camera array 3700 is built-in depth perception based on disparity between different plenoptic primitives. The images produced by the primitives on opposite sides of a given sensor can be used to calculate an offset in image detail, where the offset distance is directly related to the proximity of the detail to the sensor surface. This scene information can be used by the central processor when overlaying any enhanced video signal, resulting in AR/MR content being placed in front of the viewer at the appropriate depth. The information can also be used for various artificial focus blur and depth sensing tasks, including simulating depth of field, spatial edge detection, and other visual effects.

Fig. 37 shows a cross-sectional view of a distributed multi-aperture camera array 3700 according to some embodiments. Fig. 37 is essentially the flexible circuit board 2210B of fig. 22, with the addition of a sensor facet 3735 coupled to the flexible circuit board 2210B at a facet location 2220. In some embodiments, each sensor facet 3735 is an individual sensor cell 735 from the image sensor layer 730.

In some embodiments, each individual sensor facet 3735 is coupled to a flexible circuit board 2210. In other embodiments, each individual sensor facet 3735 is coupled to one of the logic facets 2655 that have been coupled to the flexible circuit board 2210. In other embodiments, each logic facet 2655 is first coupled to one of the sensor facets 3735, and the combined facet is then coupled to the flexible circuit board 2210. In such an embodiment, the combined logic facet 2655 and sensor facet 3735 may be referred to as sensor facet 3735 for simplicity. As used herein, "sensor facet" may represent two embodiments (i.e., an individual sensor facet 3735 or a combination of sensor facet 3735 and logic facet 2655).

In general, each sensor facet 3735 can be individually addressed (e.g., addressed by a central control processor, not shown), and a batch of sensor facets 3735 can represent a dynamic heterogeneous set that forms a single collective. In other words, the distributed multi-aperture camera array 3700 provides a tiled electronic sensor system that provides images captured through the individual sensor facets 3735 that together form a complete whole. Each individual sensor facet 3735 can capture images at a number of different resolutions and can be customized on the fly to capture different resolutions, color ranges, frame rates, etc. For example, one sensor facet 3735 may have 512x512 capture resolution while an adjacent sensor facet 3735 (of the same size) has 128x128 capture resolution, where the former represents higher density image data. In this example, the two sensors are different, but individually controllable and work in concert to capture a single light field.

The array of sensor facets 3735 as a whole can employ any curved or flat surface structure. For example, the sensor facet 3735 may be formed as a hemispherical surface, a cylindrical surface, a long spherical surface, or any other shaped surface.

The sensor facet 3735 can have any suitable shape. In some embodiments, the shape of the sensor facet 3735 matches the shape of the display facet 2665 and the shape of the facet location 2220. In some embodiments, the sensor facet 3735 has a polygonal shape, such as a triangle, quadrilateral, pentagon, hexagon, heptagon, or octagon. In some embodiments, some or all of the sensor facets 3735 have a non-polygonal shape. For example, the sensor facets 3735 at the edges of the flexible circuit board 2210 may not be polygonal, as they may have curved cuts to improve the aesthetics of the overall assembly.

In addition to having a selectable/controllable resolution, each sensor facet 3735 can also have a color range that is selectable from a plurality of color ranges and/or a frame rate that is selectable from a plurality of frame rates in some embodiments. In such an embodiment, the sensor facets 3735 of a particular flexible circuit board 2210 may be configured to provide different frame rates and different color ranges. For example, one sensor facet 3735 may have a particular color range while another sensor facet 3735 has a different color range. Similarly, one sensor facet 3735 can have a particular frame rate while another sensor facet 3735 has a different frame rate.

Fig. 38-39 show exploded views of the distributed multi-aperture camera array 3700 of fig. 37, in accordance with certain embodiments. As shown in these figures, each sensor facet 3735 may include pads 3940 having a pattern that matches pads 2240 on the flexible circuit board 2210 or pads 2940 on the logic facet 2655. This allows the sensor facet 3735 to be coupled to the logic facet 2655 or the flexible circuit board 2210 using any suitable technique in the art. In some embodiments, pads 3940 are BGA pads or any other suitable surface mount pads. Fig. 40-40 show similar views of the flexible circuit board 2210 as shown in fig. 23-24, except that: the flexible circuit board 2210 has been formed into a 3D shape.

Fig. 42 illustrates a method 4200 of manufacturing a distributed multi-aperture camera array 3700, according to some embodiments. The method 4200 may begin at step 4210 where a plurality of facet locations are formed on a circuit board at step 4210. In some embodiments, the facet locations are facet locations 2220 and the circuit board is a flexible circuit board 2210. In some embodiments, each facet location corresponds to one of a plurality of sensor facets (such as sensor facet 3735).

At step 4220, the flexible circuit board is cut or otherwise formed into a pattern that allows the flexible circuit board to be later formed into a 3D shape. When the flexible circuit board is flat, at least some of the facet locations are separated from one or more adjacent facet locations by a plurality of gaps (such as gap 2215). The plurality of gaps are substantially eliminated when the flexible circuit board is formed into a 3D shape.

At step 4230, a plurality of sensor facets are coupled to a first side of a flexible circuit board. Each sensor facet is coupled to a respective one of the facet locations of step 4210. At step 4240, the assembled electronic camera assembly is formed into a 3D shape. In some embodiments, this step may be similar to step 2540 of method 2500 described above. After step 4240, method 4200 may end.

Particular embodiments may repeat one or more steps of method 4200 where appropriate. Although this disclosure describes and illustrates particular steps of method 4200 as occurring in a particular order, this disclosure contemplates any suitable steps of method 4200 occurring in any suitable order (e.g., any temporal order). Further, while this disclosure describes and represents an exemplary method of manufacturing a distributed multi-aperture camera array, this disclosure contemplates any suitable method of manufacturing a distributed multi-aperture camera array that may include all, some, or none of the steps of method 4200, where appropriate. Additionally, although this disclosure describes and represents particular components, devices, or systems performing particular steps of method 4200, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of method 4200.

Here, "or" is inclusive rather than exclusive, unless otherwise indicated explicitly or by context. Thus, herein, "a or B" means "A, B or both," unless the context clearly indicates otherwise or indicates otherwise by context. Further, "and" are both common and individual unless otherwise indicated explicitly or by context. Thus, herein, "a and B" means "a and B, collectively or individually," unless the context clearly indicates otherwise or indicates otherwise by context.

The scope of the present disclosure includes all changes, substitutions, variations, alterations, and modifications to the exemplary embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of the present disclosure is not limited to the exemplary embodiments described or illustrated herein. Moreover, although the present disclosure describes and illustrates various embodiments herein as including particular components, elements, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would understand. Further, reference in the appended claims to an apparatus or system or a component of an apparatus or system adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function includes the apparatus, system, component, whether or not it or the particular function is activated, enabled, or unlocked, so long as the apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative.

Although the present disclosure describes and illustrates various embodiments herein as including particular components, elements, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would understand.

Further, reference in the appended claims to an apparatus or system or a component of an apparatus or system adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function includes the apparatus, system, component, whether or not it or the particular function is activated, enabled, or unlocked, so long as the apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative.