阅读说明：本技术 包括指纹传感器的显示装置及驱动该显示装置的方法 (Display device including fingerprint sensor and method of driving the same ) 是由 李淳奎 金昣佑 金正学 赵宰亨 于 2021-04-23 设计创作，主要内容包括：提供了一种显示装置及一种驱动显示装置的方法,所述显示装置包括：显示面板,包括其中显示图像的显示区域；以及指纹传感器,设置在显示面板的第一表面上并且包括用于感测用户的指纹的感测区域,其中,显示装置被构造为通过使用针对感测区域的部分的数据来认证用户的指纹。(Provided are a display device and a method of driving the display device, the display device including: a display panel including a display area in which an image is displayed; and a fingerprint sensor disposed on the first surface of the display panel and including a sensing area for sensing a fingerprint of the user, wherein the display device is configured to authenticate the fingerprint of the user by using data for a portion of the sensing area.)

1. A display device, the display device comprising:

a display panel including a display area in which an image is displayed; and

a fingerprint sensor disposed on a first surface of the display panel and including a sensing area for sensing a fingerprint of a user,

wherein the display device is configured to authenticate the fingerprint of the user by using data for a portion of the sensing region.

2. The display device according to claim 1, wherein the sensing region is included in the display region.

3. The display device according to claim 2, wherein a size of the sensing region is equal to or greater than 80% of a size of the display region.

4. The display device according to claim 1, wherein the portion of the sensing region corresponds to a touch point where the user's finger is in contact with the display panel.

5. The display device according to claim 4, further comprising:

a touch sensor configured to recognize a touch of the user.

6. The display device according to claim 5, wherein the touch sensor is provided on a second surface of the display panel.

7. The display device of claim 1, wherein the portion of the sensing region corresponds to a singular point of the fingerprint.

8. The display device of claim 7, wherein the singular point comprises a core point, a triangular point, a spine end, a junction, a bifurcation point, or an aperture.

9. The display device of claim 7, wherein the display device is further configured to obtain coordinates of the singular point.

10. The display device of claim 9, wherein the coordinates of the singular points are acquired using artificial intelligence techniques.

11. The display device of claim 1, wherein the display device is further configured to:

reading out data for a first area contacted by the user's finger, and

data for a second region corresponding to a singular point of the fingerprint is acquired.

12. The display device according to claim 11, wherein the second region has a size different from that of the first region.

13. The display device according to claim 1, further comprising:

a first memory for storing the data of the portion of the sensing region; and

a second memory storing previously obtained fingerprint information of the user.

14. The display device according to claim 13, further comprising:

a processor configured to match the data stored in the first memory with the fingerprint information stored in the second memory.

15. A method of driving a display device, the method comprising:

performing a fingerprint sensing operation by sensing a sensing area contacted by a finger of a user with a fingerprint sensor included in the display device;

generating data at the fingerprint sensor and providing the data to a readout circuit, wherein the data is based on a sensing signal corresponding to a first region that is part of the sensing region;

performing an image processing operation on the data; and

performing a matching operation in which an image processed by performing the image processing operation is compared with a previously stored fingerprint image.

16. The method of claim 15, wherein the first area corresponds to a touch point contacted by the finger of the user.

17. The method of claim 16, further comprising:

determining whether singular points are included in the first region or whether a number of singular points equal to or greater than a reference number are included in the first region.

18. The method of claim 17, further comprising:

generating data based on sensing signals corresponding to a second region corresponding to the singular point, and transmitting the data based on the sensing signals corresponding to the second region to a memory.

19. The method of claim 18, wherein the second region is included in the first region.

20. The method of claim 16, further comprising:

acquiring coordinates of singular points of the fingerprint of the user; and

crop the image processed by performing the image processing operation based on the coordinates of the singular points.

Technical Field

Exemplary embodiments of the present invention relate to a display device including a fingerprint sensor and a method of driving the same.

Background

Electronic devices employ authentication techniques as security measures to protect stored information. Authentication is the process of identifying and verifying the identity of a system or person in a secure manner. In addition to traditional usernames and passwords, biometric information of the user is being used for this security requirement.

The biometric-information-based authentication technique may include a process of acquiring biometric information (e.g., fingerprint, iris, voice, face, blood vessel, etc.) from a user and a process of determining whether the user is an authorized user by comparing the acquired biometric information with previously registered biometric information of the user. Fingerprint recognition is currently one of the most well known biometrics, and is by far the most widely used biometric solution for authentication on computer systems. Fingerprinting techniques may enhance security of user devices and may facilitate providing various application services (such as mobile payment, etc.).

Display devices such as smart phones and wearable devices in which fingerprint sensors for sensing a user's fingerprint are mounted are examples of widely used electronic devices having biometric authentication.

Disclosure of Invention

According to an exemplary embodiment of the present invention, there is provided a display device including: a display panel including a display area in which an image is displayed; and a fingerprint sensor disposed on the first surface of the display panel and including a sensing area for sensing a fingerprint of the user, wherein the display device is configured to authenticate the fingerprint of the user by using data for a portion of the sensing area.

The sensing region may be included in the display region.

The size of the sensing area may be equal to or greater than 80% of the size of the display area.

The portion of the sensing region may correspond to a touch point where a finger of the user makes contact with the display panel.

The display device may further include a touch sensor configured to recognize a touch of a user.

The touch sensor may be disposed on the second surface of the display panel.

The portions of the sensing region may correspond to singular points of the fingerprint.

Singularities may include core points, trigon points, ridge ends, intersections, bifurcations, or apertures.

The display device may also be configured to acquire coordinates of the singular point.

Artificial Intelligence (AI) techniques may be used to obtain the coordinates of the singular points.

The display device may be further configured to: reading out data for a first area contacted by a finger of a user; and data for a second region corresponding to a singular point of the fingerprint is acquired.

The second region may have a size different from that of the first region.

The display device may further include: a first memory for storing data of a portion of the sensing region; and a second memory storing previously obtained fingerprint information of the user.

The display device may further include a processor configured to match data stored in the first memory with fingerprint information stored in the second memory.

According to an exemplary embodiment of the present invention, there is provided a method of driving a display device, the method including: performing a fingerprint sensing operation by sensing a sensing area contacted by a finger of a user with a fingerprint sensor included in the display device; generating data at the fingerprint sensor and providing the data to the readout circuitry, wherein the data is based on a sensing signal corresponding to a first region that is part of a sensing region; performing an image processing operation on the data; and performing a matching operation in which an image processed by performing the image processing operation is compared with a previously stored fingerprint image.

The first region may correspond to a touch point contacted by a finger of the user.

The method may further comprise: it is determined whether singular points are included in the first region or whether a number of singular points equal to or greater than a reference number are included in the first region.

The method may further comprise: data is generated based on the sensing signal corresponding to the second region corresponding to the singular point, and the data based on the sensing signal corresponding to the second region is transmitted to the memory.

The second region may be included in the first region.

The method may further comprise: acquiring coordinates of singular points of a fingerprint of a user; and clipping an image processed by performing an image processing operation based on the coordinates of the singular points.

According to an exemplary embodiment of the present invention, there is provided a display device including: a display panel including a display area in which an image is displayed; a fingerprint sensor disposed on the display panel and including a sensing area for sensing a fingerprint of a user; and a processor configured to authenticate a fingerprint of a user by using data read out from only a portion of the sensing region.

The portion of the sensing region may include a touch point contacted by a finger of the user.

According to an exemplary embodiment of the present invention, there is provided a display device including: a display panel including a display area in which an image is displayed; a fingerprint sensor disposed on the display panel and including a sensing area, wherein the fingerprint sensor is configured to sense a first portion of the sensing area contacted by a finger of a user and to generate data based on a sensing signal corresponding to the first portion of the sensing area; and a processor configured to authenticate the fingerprint of the user by processing an image of the fingerprint of the user and comparing the processed image with a pre-stored fingerprint image.

The first portion of the sensing region may be less than the entire sensing region.

Drawings

Fig. 1 is a top plan view schematically illustrating a display device according to an exemplary embodiment of the present invention.

Fig. 2 is a schematic cross-sectional view of a display device according to an exemplary embodiment of the present invention, taken along line I-I' of fig. 1.

Fig. 3 is a block diagram for explaining a process of authenticating a fingerprint according to an exemplary embodiment of the present invention.

Fig. 4 is a top plan view schematically illustrating a display device according to an exemplary embodiment of the present invention.

Fig. 5 is a schematic cross-sectional view of a display device according to an exemplary embodiment of the present invention, taken along line II-II' of fig. 4.

Fig. 6 is a flowchart illustrating a method of driving a display device according to an exemplary embodiment of the present invention.

Fig. 7 is a view illustrating an example of a state in which the display apparatus performs the first functional step in fig. 6 according to an exemplary embodiment of the present invention.

Fig. 8 is a view illustrating an example of a state in which a display apparatus performs the fingerprint requesting step in fig. 6 according to an exemplary embodiment of the present invention.

Fig. 9 is a view illustrating an example of a state in which a user touches the display device with a finger for describing the user touch step in fig. 6 according to an exemplary embodiment of the present invention.

Fig. 10 is a view illustrating a display device for describing a first area data readout step in fig. 6 according to an exemplary embodiment of the present invention.

Fig. 11 is a flowchart illustrating a method of driving a display device according to an exemplary embodiment of the present invention.

Fig. 12 is a view illustrating a display device for describing a second area data readout step in fig. 11 according to an exemplary embodiment of the present invention.

Fig. 13 is a view illustrating a singular point in a fingerprint of a user according to an exemplary embodiment of the present invention.

Fig. 14 is a view illustrating a display apparatus for describing the third region acquiring/transmitting step in fig. 11 according to an exemplary embodiment of the present invention.

Fig. 15 is a view illustrating a display apparatus for describing the third region acquiring/transmitting step in fig. 11 according to an exemplary embodiment of the present invention.

Fig. 16 is a flowchart illustrating a method of driving a display device according to another exemplary embodiment of the present invention.

Fig. 17 is a view illustrating a display device for describing a third area data readout step in fig. 16 according to an exemplary embodiment of the present invention.

Fig. 18 is a flowchart illustrating a method of driving a display device according to an exemplary embodiment of the present invention.

Detailed Description

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. However, the embodiments described herein may be implemented in various ways, and thus should not be limited to the embodiments described herein.

The case where a first element or layer is referred to as being "disposed on" a second element or layer may include the case where the first element or layer is directly disposed on the second element or layer or the case where an additional element or layer is disposed between the first element or layer and the second element or layer.

In this disclosure, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise.

Throughout the specification, the same or similar reference numerals may be used to denote the same components.

For convenience of description, the size and thickness of each component shown in the drawings are illustrated, and thus, the present invention is not limited thereto. For example, in the drawings, the thickness of layers and regions may be exaggerated for clarity in illustrating the layers and regions.

Fig. 1 is a top plan view schematically illustrating a display device according to an exemplary embodiment of the present invention.

Referring to fig. 1, a display device 1 includes a display surface IS in which an image IS displayed. The display surface IS may include a display area DA in which an image IS displayed and a non-display area NDA adjacent to the display area DA. The non-display area NDA may surround the display area DA or be positioned on less than all sides of the display area DA.

The display area DA includes pixels and may include a plurality of emission areas each configured to emit light of a predetermined color. Further, the display area DA may be used as a detection means for detecting an external environment (e.g., a touch or a fingerprint).

A sensing region FSA configured to recognize a fingerprint of a user may be provided in the display apparatus 1. When the fingerprint surface of the user's finger touches on the sensing area FSA, the display device 1 can recognize the fingerprint by imaging the fingerprint of the touch sensing area FSA.

In an exemplary embodiment of the present invention, the sensing region FSA may overlap at least a portion of the display region DA with respect to a planar position thereof. For example, the sensing region FSA may be included in the display region DA. In other words, a portion of the display area DA or the entire display area DA may overlap the sensing area FSA.

In an exemplary embodiment of the present invention, the size of the sensing area FSA may occupy 80% or more of the size of the display area DA. In an exemplary embodiment of the present invention, the size of the sensing area FSA is equal to or smaller than the size of the display area DA, but may be equal to or larger than 15 × 15mm²(length x width).

The non-display area NDA is an area in which an image is not displayed. The display area DA may have a rectangular shape. The non-display area NDA may surround the display area DA in a plan view. A speaker module, a camera module, a sensor module, etc. may be disposed in the non-display area NDA. In an alternative, these modules may not be disposed in the non-display area NDA. Here, the sensor module may include at least one of an illumination sensor, a proximity sensor, an infrared sensor, an ultrasonic sensor, and a vision sensor.

However, the present invention is not limited thereto, and the shape of the display area DA and the shape of the non-display area NDA may be different from those shown in fig. 1.

Fig. 2 is a schematic cross-sectional view of a display device according to an exemplary embodiment of the present invention, taken along line I-I' of fig. 1. Fig. 3 is a block diagram for explaining a process of authenticating a fingerprint according to an exemplary embodiment of the present invention.

Hereinafter, when the first member is represented as being disposed on the front surface (or front surface) of the second member, this may indicate that the first member is disposed (e.g., stacked) on the first surface of the second member, and when the first member is represented as being disposed on the rear surface (or back surface) of the second member, this may indicate that the first member is disposed on the second surface of the second member, the second surface being opposite to the first surface. In other words, disposing the first member on the front surface and disposing the first member on the rear surface may be understood as disposing the first member in an opposite direction relative to the second member.

Referring to fig. 2, the display device 1 includes a display panel DP, a touch sensor TS, a fingerprint sensor FS, a window member WD, and a case member HU.

The display apparatus 1 may further include a plurality of memories 210 and 220 and a processor 300 (as shown in fig. 3). In an exemplary embodiment of the present invention, the plurality of memories 210 and 220 and the processor 300 may be mounted on the display panel DP or the fingerprint sensor FS in the form of a chip.

The display panel DP may be any of various types of display panels such as an organic light emitting display panel, a liquid crystal display panel, a field emission display panel, an electrophoretic display panel, a quantum dot emission display panel, and a micro (or nano) LED display panel.

The display panel DP may include a plurality of transistors and a plurality of light emitting elements. The at least one transistor and the at least one light emitting element may form a pixel.

The display panel DP may form the display area DA described above.

In an exemplary embodiment of the present invention, the touch sensor TS may be disposed on the front surface of the display panel DP. For example, the touch sensor TS may be disposed on the first surface of the display panel DP. The touch sensor TS may include a plurality of sensing electrodes. The sensing electrodes may sense body part touches, hovers, gestures, proximity, etc. of the user. The sensing electrode may be configured in various forms according to various types (such as a resistive type, a capacitive type, an electromagnetic type, an optical type, etc.). For example, when the sensing electrode is configured as a capacitance type, the sensing electrode may be configured as a self capacitance type, a mutual capacitance type, or the like.

In some exemplary embodiments of the present invention, when the sensing electrodes are configured as a self-capacitance type, the respective sensing electrodes are individually driven, and a sensing signal corresponding to a capacitance formed by the respective sensing electrodes and the body of the user may be provided to a corresponding bonding wire of the bonding wires.

In various exemplary embodiments of the present invention, when the sensing electrodes are configured as a mutual capacitance type, the touch driving signal may be received through a bonding line corresponding to a first sensing electrode, and the touch sensing signal may be transmitted through a bonding line corresponding to a second sensing electrode forming a mutual capacitance with the first sensing electrode. When the body of the user is closer to the front surface of the display panel DP, the mutual capacitance between the first and second sensing electrodes may change, thereby causing a difference in a sensing signal indicating that the user is in contact with the display panel DP.

In an exemplary embodiment of the present invention, the fingerprint sensor FS may be disposed on the rear surface of the display panel DP. For example, the fingerprint sensor FS may be disposed on a second surface of the display panel DP, the second surface of the display panel DP being opposite to the first surface of the display panel DP. However, the present invention is not limited to the location of the fingerprint sensor FS. In an exemplary embodiment of the present invention, the fingerprint sensor FS may be disposed on another surface of the display panel DP.

The fingerprint sensor FS may form the sensing area FSA described above.

Referring additionally to fig. 3, in an exemplary embodiment of the present invention, the fingerprint sensor FS may sense light emitted from the light emitting elements and reflected from valleys between ridges of a fingerprint through the image sensor 110 included in the fingerprint sensor FS. Here, the image sensor 110 may be an optical fingerprint sensor. Further, without being limited thereto, the image sensor 110 may be an ultrasonic image sensor, an infrared image sensor, or a capacitive fingerprint image sensor.

The fingerprint sensor FS may be a semiconductor chip or a semiconductor package, and may be attached to the rear surface of the display panel DP. For example, the image sensor 110 may be a semiconductor chip or a semiconductor layer in which a plurality of photoelectric conversion elements (e.g., photodiodes, phototransistors, photogates, clamped (pinned) photodiodes, etc.) are formed. In an exemplary embodiment of the present invention, the image sensor 110 may be a semiconductor layer such as a complementary metal-oxide-semiconductor (CMOS) image sensor (CIS) or a Charge Coupled Device (CCD).

The fingerprint sensor FS includes a plurality of sensing pixels, and each of the sensing pixels senses light reflected from a different area in a fingerprint and generates an electrical signal corresponding to the sensed light. Each of the sensing pixels may generate an electrical signal corresponding to light reflected from ridges in the fingerprint, or may generate an electrical signal corresponding to light reflected from valleys between ridges in the fingerprint. The amount of light sensed by the image sensor 110 may vary according to the shape of a fingerprint from which the light is reflected, and may generate an electrical signal (e.g., a sensing signal) having different levels according to the sensed amount of light. In other words, the electrical signals from the plurality of sensing pixels may include luminance information (or image information). As such, the region corresponding to each of the sensing pixels may be determined as a ridge or a valley by the processing operation for the electrical signals, and the entire fingerprint image may be formed by combining the determined ridge information and the valley information.

In the optical sensing method, fingerprint information may be acquired using light emitted from a plurality of pixels of the display panel DP and reflected from a fingerprint. The accuracy and the acquisition speed of the light information acquired by the fingerprint sensor FS may vary according to the method of driving the display panel DP.

The fingerprint sensor FS may comprise a readout circuit 120 coupled to the image sensor 110 via readout lines RL1, RL2 to RLn. The readout circuit 120 may scan the sensing signal through the readout lines RL1, RL2 to RLn. Here, n is a natural number.

The readout circuit 120 may generate sensing data based on the sensing signal and transmit the sensing data to the first memory 210 or the second memory 220 disposed outside the fingerprint sensor FS. The sensing data may be converted into fingerprint data by image processing.

Each of the first memory 210 and the second memory 220 may store one or more pieces of fingerprint data. For example, the first memory 210 may store previously registered fingerprint information of the user, and the second memory 220 may store fingerprint data acquired from the fingerprint sensor FS when a process of authenticating a fingerprint is performed.

Each of the first memory 210 and the second memory 220 may be at least one of storage media such as a nonvolatile memory element (such as a Read Only Memory (ROM), a programmable ROM (prom), an erasable programmable ROM (eprom), an electrically erasable programmable ROM (eeprom), a Hard Disk Drive (HDD), or a flash memory), a volatile memory element (such as a Random Access Memory (RAM), and a cache), but is not limited to these examples.

The processor 300 may comprise functionality to match fingerprint information stored in the first memory 210 with fingerprint information stored in the second memory 220. The processor 300 may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application Processor (AP), a Display Digital Interface (DDI), a display driver Integrated Circuit (IC), etc., but is not limited thereto.

The processor 300 and the first and second memories 210 and 220 may be a single chip or may be separate chips. Further, the present invention is not limited to two memories, and the first memory 210 and the second memory 220 may be a single memory performing the function of each of the first memory 210 and the second memory 220.

The case member HU forms a rear surface and an edge surface of the display device 1, and the window member WD may form a front surface of the display device 1. The case member HU and the window member WD may cover the display panel DP, the touch sensor TS, and the fingerprint sensor FS. For example, the touch sensor TS, the display panel DP, and the fingerprint sensor FS may be provided in the housing member HU. The case member HU and the window member WD may protect various components of the display device 1 from external physical impact (e.g., scratch) and the like. The window member WD may be a location contacted by an input tool (such as a user's finger, etc.) for actual sensing of a fingerprint.

The display area DA may overlap at least a portion of the display panel DP. In this specification, the term "overlap" may indicate that two components are overlapped when viewed from the thickness direction of the display device 1.

The sensing area FSA may overlap at least a part of the fingerprint sensor FS. As described above, the sensing region FSA may be included in the display region DA.

It will be appreciated that there is also provided, in accordance with an exemplary embodiment of the invention, a computer readable storage medium for storing a computer program. The computer-readable storage medium stores a computer program that, when executed by the processor 300, causes the processor 300 to perform the image processing method described above. The computer readable storage medium is any data storage device that can store data which can be read by a computer system. Examples of computer readable storage media include RAM, ROM, and the like. The computer readable storage medium may be one of the first memory 210 and the second memory 220, or may be included in the processor 300.

Fig. 4 is a top plan view schematically illustrating a display device according to an exemplary embodiment of the inventive concept. Fig. 5 is a schematic cross-sectional view of a display device according to an exemplary embodiment of the present invention, taken along line II-II' of fig. 4.

Referring to fig. 4 and 5, the display device 2 according to the present embodiment is different from the display device 1 according to the embodiment of fig. 1 and 2 in that: the sensing region FSA and the display region DA are set to the same region. In other words, in fig. 4 and 5, the sensing region FSA has the same position and size as the display region DA.

In the present embodiment, the size of the sensing region FSA may be the same as the size of the display region DA. In other words, when the user touches any area in the display area DA, which is an area for displaying an image, with his or her finger, the display device 2 not only recognizes the user touch but also senses the fingerprint of the user.

Since the functions of the respective members in the display device 2 and the positional relationship therebetween are similar to those of the members of the display device 1 and the positional relationship therebetween, a repetitive description will be omitted.

Hereinafter, a method of driving the display device including the fingerprint authentication process will be described. In a driving method to be described below, the display device may be the display device 1 and the display device 2 described with reference to fig. 1 to 4.

Fig. 6 is a flowchart illustrating a method of driving a display device according to an exemplary embodiment of the present invention. Fig. 7 is a view illustrating an example of a state in which the display apparatus performs the first functional step in fig. 6 according to an exemplary embodiment of the present invention. Fig. 8 is a view illustrating an example of a state in which a display apparatus performs the fingerprint requesting step in fig. 6 according to an exemplary embodiment of the present invention. Fig. 9 is a view illustrating an example of a state in which a user touches the display device with his or her finger for describing the user touch step in fig. 6 according to an exemplary embodiment of the present invention. Fig. 10 is a view illustrating a display device for describing a first area data readout step in fig. 6 according to an exemplary embodiment of the present invention.

Referring to fig. 6, in an exemplary embodiment of the present invention, a method of driving a display device may include a first function performing step S110, a fingerprint authentication requesting step S120, a user touching step S130, a sensing area fingerprint sensing step S140, a first area data reading out step S150, an image processing step S160, and a matching step S170. In the present embodiment, the steps are described as being sequentially performed according to the flowchart, but it is understood that the order of the steps may be changed, some of the steps may be omitted, or another step may be further included between the steps.

Referring also to fig. 7, the first function performing step S110 corresponds to a step in which the display device performs the first function before an event requiring fingerprint authentication occurs. For example, the first function may correspond to a standby mode state of the display device (as shown in fig. 7), a state in which the display device is locked with its screen closed, or a state of the display device before a program requiring fingerprint authentication is executed.

Referring additionally to fig. 8, the fingerprint authentication request step S120 corresponds to a step in which an event requesting fingerprint identification occurs in the display device while the display device is performing the first function. In this case, the user is prompted to touch the display device with a finger to authenticate the fingerprint. For example, in a case where the display area DA is in the display open state, the display device may display a phrase, an image, or the like for prompting the user to touch the screen with a finger on the display screen. For example, as shown in FIG. 8, the phrase may be "touching the screen to identify a fingerprint".

Here, the fingerprint sensor FS may be activated to sense a fingerprint of the user. According to an exemplary embodiment of the present invention, the driving state of the pixels overlapping the sensing area FSA may be changed when the fingerprint sensor FS is activated.

For example, when the display area DA is maintained in a display on state before the fingerprint sensor FS is activated, the respective pixels of the display area DA may be driven at the first frequency. After activation of the fingerprint sensor FS, at least the pixels overlapping the sensing area FSA may be driven at a second frequency different from the first frequency.

As an example, when the display area DA is maintained in a display on state before the fingerprint sensor FS is activated, the respective pixels of the display area DA may be driven with a first brightness. After activation of the fingerprint sensor FS, at least the pixels overlapping the sensing area FSA may be driven with a second brightness different from the first brightness.

Referring also to fig. 9, the user touch step S130 corresponds to a state in which the input tool is in contact with the window member WD of the display device or is located very close to the window member WD of the display device.

For example, the user touching step S130 may be a step in which the finger 10 of the user is in contact with one region of the window member WD that overlaps the sensing region FSA. When the user's finger 10 comes into contact with the sensing area FSA of the display device, as shown, the content displayed on the screen of the display device changes and the process of authenticating the fingerprint may start. For example, the content displayed on the screen of the display device may be "fingerprint authentication" indicating that a fingerprint authentication process is in progress. When the user's finger 10 is in contact with the outside of the sensing region FSA or when the sensing region FSA is not touched, the display device determines that the user does not want fingerprint recognition. In this case, the display apparatus may return to the first function performing step S110 or the fingerprint authentication requesting step S120.

When the display device determines that the user touch is made within the sensing region FSA under the user touch step S130, the display device may perform the sensing region fingerprint sensing step S140. The sensing area fingerprint sensing step S140 corresponds to a step in which the fingerprint sensor FS included in the display device senses the sensing area FSA.

In an exemplary embodiment of the present invention, the fingerprint sensor FS may acquire sensing signals spread over the entire sensing area FSA through the image sensor 110 included therein.

The fingerprint sensor FS may read out data generated based on the acquired sensing signal.

Referring also to fig. 10, the first area data readout step S150 corresponds to a step in which the image sensor 110 generates data based on a sensing signal corresponding to the first area SA1 among sensing signals acquired throughout the entire sensing area FSA, and reads out data to be supplied to the readout circuit 120.

The first area SA1 may correspond to a touch point TP where a user touch is made. For example, the first area SA1 may be larger than and surround the touch point TP. In an exemplary embodiment of the present invention, the first area SA1 may include the touch point TP and may have a size smaller than that of the sensing area FSA. According to an exemplary embodiment of the present invention, the first area SA1 may have a size equal to or smaller than the size of an actual area contacted by a user's finger.

In an exemplary embodiment of the present invention, the fingerprint sensor FS may perform analog-to-digital conversion on the sensing signal corresponding to the first area SA1, and may transmit a digital signal corresponding to the sensing signal of the first area SA1 to the first memory 210 or the second memory 220. In other words, the fingerprint sensor FS may convert the analog sensing signal into a digital signal to be transmitted to the first memory 210 or the second memory 220. The first memory 210 or the second memory 220 may store digital signals corresponding to the sensing signals of the first area SA 1. The digital signal may include fingerprint feature information (such as minutiae).

When a digital signal corresponding to the sensing signal of the first area SA1 is stored in the first memory 210 or the second memory 220, the image processing step S160 may be performed.

The image processing step S160 corresponds to a step in which the processor 300 processes an image based on a digital signal corresponding to the sensing signal of the first area SA 1.

For example, the image processing may include a process of converting an image into a gray scale based on a digital signal corresponding to the sensing signal of the first area SA1, or the like, performed by the processor 300. The sensing region fingerprint sensing step S140 may include the following processes: in sensing a fingerprint, a fingerprint image is formed by setting brightness or depth using light information input to the fingerprint sensor FS when the fingerprint is reflected from the screen of the display panel DP.

As an example, the image processing may include scaling, rotation, and/or translation performed by the processor 300 on an image transformed based on a digital signal corresponding to the sensing signal of the first area SA 1.

As another example, the image processing may include converting, by the processor 300, a coordinate system of frequency domain information included in the image transformed based on the digital signal corresponding to the sensing signal of the first region SA1 into a polar coordinate system using Log Polar Transform (LPT). For example, LPT may be performed on the magnitude of a Fast Fourier Transform (FFT) image derived by FFT. For example, the FFT may be applied to an image to which the LPT is applied to obtain an FFT image. Then, the processor 300 performs phase correlation between the images, thereby detecting a peak value as a result of performing the phase correlation. The position of the detected peak may indicate rotation information or scaling information related to the image.

The matching step S170 corresponds to a step in which the processor 300 matches an image processed based on a digital signal corresponding to the sensing signal of the first area SA1 with an image containing previously stored fingerprint information of the user. Hereinafter, an image processed based on a digital signal corresponding to the sensing signal of the first area SA1 may be referred to as an input fingerprint image, and an image containing previously stored fingerprint information of the user may be referred to as an enrolled fingerprint image.

In an exemplary embodiment of the present invention, the processor 300 may calculate a similarity between the registered feature information of the registered fingerprint image of the user previously stored in the first memory 210 and the input feature information of the input fingerprint image stored in the second memory 220. For example, the feature information may include at least one of detail information extracted based on a frequency domain conversion method, Scale Invariant Feature Transform (SIFT) information, and phase information.

According to an exemplary embodiment of the present invention, the size of the registered fingerprint image of the user previously stored in the first memory 210 may be different from the size of the input fingerprint image stored in the second memory 220. For example, because the registered fingerprint images of the user previously stored in the first memory 210 were generated based on the sensing area FSA, and the input fingerprint images stored in the second memory 220 were generated based on the first area SA1, they may have different sizes.

For example, the processor 300 may calculate a score representing a degree to which the input fingerprint image matches the registered fingerprint image based on a pattern similarity between the registered fingerprint image and the input fingerprint image and based on a similarity between feature information of the registered fingerprint image and feature information of the input fingerprint image. However, the score may be based on only one of a pattern similarity between the registered fingerprint image and the input fingerprint image and a similarity between the feature information of the registered fingerprint image and the feature information of the input fingerprint image. The processor 300 may determine that the authentication is successful when the score is equal to or greater than a predetermined threshold, and may determine that the authentication is failed when the score is less than the predetermined threshold.

In the present embodiment, since the steps from the image processing step S160 to the matching step S170 are performed using the signals corresponding to the sensing signals of the first region SA1, which is a part of the sensing region FSA, instead of the entire sensing region FSA, the processor 300 can quickly determine whether the authentication is successful or not, as compared to when the steps S160 to S170 are performed using the signals corresponding to the sensing signals of the entire sensing region FSA.

According to the exemplary embodiment of the present invention illustrated in fig. 1 to 10, the display device 1 may include: a display panel DP including a display area DA in which an image is displayed; and a fingerprint sensor FS disposed on the first surface of the display panel DP and including a sensing area FSA for sensing a fingerprint of a user, wherein the display apparatus 1 is configured to authenticate the fingerprint of the user by using data for a portion of the sensing area FSA.

Next, a method of driving a display device according to an exemplary embodiment of the present invention will be described. Hereinafter, the description of the same components as those in fig. 1 to 10 may be omitted, and the same or similar reference numerals may be used therefor.

Fig. 11 is a flowchart illustrating a method of driving a display apparatus according to another exemplary embodiment of the inventive concept. Fig. 12 is a view illustrating a display device for describing a second area data readout step in fig. 11 according to an exemplary embodiment of the present invention. Fig. 13 is a view illustrating a singular point in a fingerprint of a user according to an exemplary embodiment of the present invention. Fig. 14 is a view illustrating a display apparatus for describing the third region acquiring/transmitting step in fig. 11 according to an exemplary embodiment of the present invention. Fig. 15 is a view illustrating a display apparatus for describing the third region acquiring/transmitting step in fig. 11 according to an exemplary embodiment of the present invention.

Referring to fig. 11, the method of driving the display device according to the present embodiment is different from the embodiment of fig. 6 in that: the first area data reading step S150 is omitted, and a second area data reading step S151, a third area acquisition/transfer step S152, a singular point inclusion checking (singular point inclusion checking) step S153, and a third area moving step S154 are further included.

In other words, in an exemplary embodiment of the present invention, the method of driving the display device may include a first function performing step S110, a fingerprint authentication requesting step S120, a user touching step S130, a sensing area fingerprint sensing step S140, a second area data readout step S151, a third area acquiring/transmitting step S152, a singular point inclusion checking step S153, a third area moving step S154, an image processing step S160, and a matching step S170.

The fingerprint sensor FS may read out data generated based on the sensing signal acquired at the sensing-area fingerprint sensing step S140.

Referring also to fig. 12, the second area data readout step S151 corresponds to a step in which the image sensor 110 generates data based on a sensing signal corresponding to the second area SA2 among sensing signals acquired throughout the entire sensing area FSA, and reads out data to be supplied to the readout circuit 120.

The second area SA2 may be set based on the touch point TP where the user touches. In an exemplary embodiment of the present invention, the second area SA2 may include the touch point TP and may have a size smaller than that of the sensing area FSA. According to an exemplary embodiment of the present invention, the second region SA2 may have a size smaller than that of the sensing region FSA, and may have a size equal to or greater than that of the first region SA1 described above.

The third area acquiring/transferring step S152 corresponds to a step in which the readout circuit 120 acquires data corresponding to the third area SA3 among the data acquired under the second area data readout step S151, and transfers the data to the first memory 210 or the second memory 220.

For example, referring to fig. 14, the third area SA3 may be set to an area smaller than the second area SA2 and included in the second area SA 2. For example, the third area SA3 may be entirely included in the second area SA 2. According to an exemplary embodiment of the present invention, the third area SA3 may be the same size as that of the first area SA1 described above.

Under the third region acquiring/transmitting step S152, the fingerprint sensor FS may perform analog-to-digital conversion on the sensing signal corresponding to the third region SA3, and may transmit a digital signal corresponding to the sensing signal of the third region SA3 to the first memory 210 or the second memory 220. The first memory 210 or the second memory 220 may store digital signals corresponding to the sensing signals of the third area SA 3.

Referring additionally to fig. 13, the user fingerprints 10_ FP have unique features, and each user fingerprint 10_ FP may include various singular points SP. For example, the user fingerprint 10_ FP may include singular points SP as shown (such as core points and triangular points (deltas)). In addition, the user fingerprint 10_ FP may further include various singular points SP (such as ridge ends, intersections, bifurcations, pores, etc.) included in the fingerprint.

The singular point SP may or may not be included in the third area SA3, depending on the position of the third area SA 3. In fig. 14, an example in which the singular point SP is included in the third area SA3 is shown.

The singular point includes a step in which the checking step S153 corresponds to a step in which the readout circuit 120 or the processor 300 determines whether the data includes the singular points SP of the user fingerprint (or the singular points SP whose number is equal to or greater than the reference number) based on the data acquired at the third area acquisition/transmission step S152.

In an exemplary embodiment of the present invention, the display apparatus may determine whether the singular point SP is included using image processing and an Artificial Intelligence (AI) technique based on deep learning at the singular point inclusion checking step S153. The display device may include a deep learning algorithm. Deep learning algorithms may include deep belief networks, autoencoders, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), deep Q-networks, and the like. The deep learning algorithm listed in the present disclosure is only an example, and thus, the present invention is not limited thereto. Using the deep learning algorithm, the display device may determine whether the singular point SP is included based on the AI technique trained for the singular point SP through the database of user fingerprints 10_ FP. Therefore, the display apparatus can quickly determine whether the singular point SP is included at the singular point inclusion checking step S153.

In the singular point inclusion checking step S153, when it is determined that the data acquired in the third region acquisition/transmission step S152 includes the singular points SP described above (or the number of singular points SP equal to or greater than the reference number), the readout circuit 120 transfers the data acquired in the third region acquisition/transmission step S152 to the first memory 210 or the second memory 220, and may sequentially execute the image processing step S160 and the matching step S170.

Referring also to fig. 15, in the singular point inclusion checking step S153, when it is determined that the data acquired in the third region acquisition/transmission step S152 does not include the singular points SP described above (or includes the singular points SP whose number is less than the reference number), the third region moving step S154 may be performed. In fig. 15, an example in which the singular point SP is not included in the third area SA3 is shown.

The third area moving step S154 corresponds to a step in which the third area SA3 is moved to another third area SA 3'. For example, the third area SA3 may be moved to another third area SA 3' in an attempt to capture enough singular points SP for subsequent image processing.

Then, the third area acquisition/transmission step S152 and the singular point inclusion checking step S153 may be performed again based on the moved third area SA 3'.

In the singular point inclusion checking step S153, when it is determined that the data acquired based on the shifted third region SA 3' in the third region acquiring/transmitting step S152 does not include the singular points SP described above (or includes the singular points SP whose number is smaller than the reference number), the third region shifting step S154 is performed again. In contrast, at the singular point inclusion checking step S153, when it is determined that the data acquired at the third region acquisition/transmission step S152 includes the singular points SP described above (or the singular points SP whose number is equal to or greater than the reference number), the image processing step S160 and the matching step S170 may be sequentially performed.

In the present embodiment, since the steps from the image processing step S160 to the matching step S170 are performed using the signal corresponding to the sensing signal of the third area SA3, which is a part of the sensing area FSA, instead of the entire sensing area FSA, the processor 300 can quickly determine whether the authentication is successful or not, as compared to when the steps S160 and S170 are performed using the signal corresponding to the sensing signal of the entire sensing area FSA. Further, since the processor 300 performs fingerprint authentication using the singular point SP, security can be improved.

Fig. 16 is a flowchart illustrating a method of driving a display device according to another exemplary embodiment of the present invention. Fig. 17 is a view illustrating a display device for describing a third area data readout step in fig. 16 according to an exemplary embodiment of the present invention.

Referring to fig. 16, steps S110 to S140 correspond to the steps described above with respect to fig. 6. In fig. 16, the fingerprint sensor FS may read out data generated based on the sensing signal acquired at the sensing-area fingerprint sensing step S140.

Referring to fig. 16 and 17, the third area data readout step S155 corresponds to a step in which the image sensor 110 generates data based on a sensing signal corresponding to the third area SA3 having the singular point SP at the center thereof among sensing signals acquired throughout the entire sensing area FSA, and reads out data to be supplied to the readout circuit 120. Here, the third area SA3 may be an area set based on the singular point SP, the position of which is determined using image processing and an AI technique based on deep learning.

Subsequently, the image processing step S160 and the matching step S170 may be sequentially performed. These steps may correspond to the steps described above with respect to fig. 6.

In the present embodiment, since the steps from the image processing step S160 to the matching step S170 are performed using the signal corresponding to the sensing signal of the third area SA3, which is a part of the sensing area FSA, instead of the entire sensing area FSA, the processor 300 can quickly determine whether the authentication is successful or not, as compared to when the steps S160 and S170 are performed using the signal corresponding to the sensing signal of the entire sensing area FSA. Further, since the processor 300 performs fingerprint authentication using the singular point SP, security can be improved.

Fig. 18 is a flowchart illustrating a method of driving a display device according to an exemplary embodiment of the present invention.

Referring to fig. 18, in the present embodiment, the method of driving the display device may include a first function performing step S110, a fingerprint authentication requesting step S120, a user touching step S130, a sensing area fingerprint sensing step S140, a second area data readout step S151, an image processing step S160, a singular point coordinate acquiring step S156, an image cropping step S161, and a matching step S170. Referring to fig. 18, steps S110-S140 may correspond to the steps described above with respect to fig. 6.

After data is generated based on the sensing signal corresponding to the second area SA2 and the data is read out and supplied to the readout circuit 120 in the second area data readout step S151, the image processing step S160 and the singular point coordinate acquisition step S156 may be performed, respectively.

The singular point coordinate acquisition step S156 corresponds to a step in which the coordinates of the singular point SP are acquired based on data generated from the sensing signal corresponding to the second area SA 2. In an exemplary embodiment of the present invention, the display apparatus may acquire the coordinates of the singular point SP using a singular point detection algorithm. For example, the singular point SP detection algorithm is an algorithm configured to divide an image into blocks (e.g., patches) in a grid form based on a sensing signal corresponding to the second area SA2 and generate a unique window coordinate value of each of the blocks.

In an exemplary embodiment of the present invention, the display apparatus may acquire the coordinates of the singular point SP using the image processing and the AI technique based on the deep learning described above.

The image cropping step S161 corresponds to the following steps: the image acquired at the image processing step S160 is clipped to fit the area corresponding to the third area SA3 described above, based on the coordinates of the singular point SP calculated at the singular point coordinate acquisition step S156.

The display device may perform the matching step S170 by using the image cropped under the image cropping step S161 as an input fingerprint image.

According to an exemplary embodiment of the present invention, the display apparatus may rapidly perform fingerprint authentication, and security of the display apparatus may be improved.

While the present invention has been described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as set forth in the following claims.