阅读说明：本技术 芯片缺陷检测方法及系统 (Chip defect detection method and system ) 是由 柏钧蓝 于 2019-10-08 设计创作，主要内容包括：本公开涉及芯片缺陷检测方法及图像处理系统,该方法包括：获取所述待检测芯片的快照,对快照执行多重滤波以消除附着在快照上的环境干扰图像,基于每一层滤波后的图像确定快照中包含的芯片图像的角点；当角点数等于4时,以角点为轮廓基准,从快照中生成芯片真实图像,以及对芯片真实图像执行自适应二值化处理以标识芯片缺陷。(The present disclosure relates to a chip defect detection method and an image processing system, the method including: acquiring a snapshot of the chip to be detected, performing multiple filtering on the snapshot to eliminate an environmental interference image attached to the snapshot, and determining corner points of the chip image contained in the snapshot based on each layer of filtered image; and when the number of the corner points is equal to 4, generating a chip real image from the snapshot by taking the corner points as the outline reference, and performing self-adaptive binarization processing on the chip real image to identify the chip defects.)

1. A method for detecting chip defects, comprising:

acquiring a snapshot of a chip to be detected, wherein the snapshot comprises an image of the chip and an environmental interference image shot at the same time when the snapshot is generated;

removing the environmental interference image imaged together with the image of the chip by performing multiple filtering on the snapshot and recording each re-filtered image;

determining corner points of the chip image contained in the snapshot based on the filtered image;

when the number of the determined corner points is equal to 4, generating a chip real image from the snapshot by taking the corner points as a contour reference;

and performing self-adaptive binarization processing on the real chip image to identify chip defects.

2. The method of claim 1, wherein if the determined number of corner points is less than 4, it is determined that the snapshot of the chip currently to be detected is invalid; and if the number of the determined angular points is more than 4, determining that the chip to be detected has chip defects.

3. The method of claim 1, wherein performing multiple filtering on the snapshot comprises:

and performing downsampling Gaussian filtering on the snapshot by using a K-layer Gaussian pyramid filter to obtain K filtered images, wherein the K filtered images and the snapshot form a K + 1-layer pyramid image, the snapshot is positioned on the 1 st layer of the pyramid image, and K is larger than 1.

4. The method of claim 3, wherein determining corner points of the chip image based on the filtered image comprises:

(1) for the K +1 layer image in the pyramid image, the following processing is performed:

determining an angular point of the K +1 layer image;

if the number of the corner points of the K + 1-th layer image is more than or equal to 4, mapping the determined corner points to the K-th layer image;

(2) for each of the K-th to 1-th layer images in the pyramid image, performing the following processing:

and establishing a corner point search area corresponding to the current layer image and searching for corner points in the established search area so as to determine the corner points of the current layer image, wherein the corner point search area of the current layer image is generated by taking points mapped from the determined corner points of the previous layer image as centers and expanding a preset number of pixels outwards along four directions, and the size of the corner point search area of each layer of images from the K layer image to the 1 st layer image is different.

5. The method of claim 4, wherein the size of the corner search area of the K-th through 1-th layer images is determined by: f. of_iWhere 1 ≦ i ≦ K, F is a predetermined value smaller than the K +1 th layer image size, F represents the number of pixels extending outward in any of the four directions, and sqrt () represents an open square function.

6. The method of any of claims 1-5, wherein generating a chip real image from the snapshot with the corner as a contour reference when the number of corners equals 4 further comprises:

establishing a quadrilateral area with the four corner points as outline references,

judging whether the quadrilateral area is a rectangle, wherein if the quadrilateral area is not the rectangle, the method further comprises the following steps:

performing perspective conversion on the quadrangular image to generate the chip real image having a rectangular shape.

7. The method of claim 6, wherein performing adaptive binarization processing on the chip real image comprises:

performing illumination compensation on the real chip image, and performing binarization on the real chip image subjected to the illumination compensation to form a binarized image;

and establishing a communication area in the binary image to mark the chip defects.

8. An image processing system comprising:

the chip to be detected comprises a user display interface, a first storage interface and a second storage interface, wherein the user display interface is configured to receive a snapshot of the chip to be detected according to user instructions, and the snapshot comprises an environmental interference image which is shot at the same time when the snapshot is generated;

an image processing unit configured to perform the method of one of claims 1 to 7 to determine whether the chip is defective;

wherein the user display interface is further configured to graphically display the real image of the chip marked with the chip defect.

9. A computer readable medium having stored thereon machine readable instructions which, when executed by a machine, cause the machine to carry out the method of any one of claims 1 to 7.

10. An apparatus for chip defect detection, comprising means for implementing the steps in the method of one of claims 1 to 7.

Technical Field

The invention relates to a chip detection technology, in particular to a chip surface damage defect detection method.

Background

In the chip manufacturing process, usually thousands of chips are manufactured on a single wafer, and each chip needs to be separated from the wafer independently for subsequent processes such as UV irradiation, for example, each chip is cut from the wafer by using a diamond blade rotating at high speed. Therefore, during the dicing process, there is a high possibility that damage may be caused to the chips, particularly, near the edge and corner of each chip where the dicing operation is performed, causing chip defects, such as operation damage occurring in the upper left corner of the chip as shown in the partially enlarged schematic view of the chip shown by the black area in fig. 1. Chip defects may also include physical damage such as scratches occurring on the chip surface for other reasons. Therefore, it is necessary to screen out such chips in practice to avoid entering the next process and even entering the hands of the customer.

At present, the defects of the chips are mainly detected in a manual mode, quality testing personnel check a large number of chip images by naked eyes, and because the chip cut damage is very small and is usually less than 50 micrometers, an operator has to manually amplify the images to find the defects, so that the method is not accurate and occupies a large amount of human resources.

Disclosure of Invention

The invention provides a scheme for automatically detecting the chip defects, which can quickly and accurately remove the defective chips without an operator checking the images.

According to an aspect of the present invention, there is provided a method for detecting a chip defect, comprising: acquiring a snapshot of a chip to be detected, wherein the snapshot comprises an image of the chip and an environmental interference image shot at the same time when the snapshot is generated; removing the environmental interference image imaged together with the chip by performing multiple filtering on the snapshot, and recording each filtered image; determining corner points of the chip image contained in the snapshot based on the filtered image; when the number of the corner points is equal to 4, generating a chip real image from the snapshot by taking the corner points as outline reference; and performing self-adaptive binarization processing on the real chip image to identify chip defects.

According to another aspect of the present invention, there is provided an image processing system comprising: the user display interface is configured to receive a snapshot of the chip to be detected according to user instructions; the image processing unit is configured to determine whether the chip has defects by the chip defect detection method; wherein the user display interface is further configured to graphically display the real image of the chip marked with the occurrence location of the chip defect.

According to yet another aspect of the invention, there is provided a computer readable medium having stored thereon machine readable instructions which, when executed by a machine, cause the machine to perform the method of the invention.

By using the scheme of the invention, the complicated operations of manual operation such as opening images, amplifying, moving, judging, recording and the like are avoided, the detection accuracy is improved, the detection time is shortened and the efficiency is improved.

Drawings

FIG. 1 illustrates a schematic diagram of an exemplary chip corner damage;

FIG. 2 schematically illustrates a chip snapshot;

FIG. 3 shows a flow diagram of a chip defect detection method according to an embodiment of the invention;

FIG. 4A illustrates an exemplary Gaussian pyramid filtering diagram;

FIG. 4B schematically illustrates a pyramid image filtered through various layers;

FIG. 5 shows a flow chart of corner detection according to an embodiment of the invention;

FIGS. 6A and 6B are schematic diagrams illustrating chip defects represented by connected regions;

FIG. 7 shows an exemplary image processing system schematic;

FIG. 8 shows a schematic diagram of a user interface of an exemplary image processing system.

Detailed Description

The method and system provided by the embodiment of the invention are explained in detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

After the chips are cut from the wafer, the chips need to be placed on a test platform for testing, and thus a chip pick and place head (pick and place head) is typically operated on the production line to pick and place the chips on the test platform. In one embodiment of the invention, a camera is used to capture a chip when it is picked up by a chip pick-and-place head to generate a chip snapshot. Therefore, some structural or pattern features on the pick-and-place head may inevitably be shot together and reflected in the chip snapshot. As shown in fig. 2, where the entire chip snapshot is represented by CSS, representing the image taken for the pick-and-place head that is picking up the chip, the black rectangular box represents the image of the chip being picked up, represented by CI. It can be seen that in this snapshot CSS, in addition to the chip image CI, a two-dimensional code QR image on the surface of the gripper head and an image of the calibration holes used for positioning during the gripping process are included, which severely disturb the chip image CI. Obviously, the contour of the CSS, the QR code, the calibration hole image, etc. are all disturbances to the chip image CI, which is referred to herein as an environmental disturbance image. According to the detection technology of the invention, the chip image CI needs to be automatically extracted from the snapshot CSS, so that the interference image such as the two-dimensional code QR needs to be eliminated.

FIG. 3 shows a flowchart of a defect detection method according to one embodiment of the invention. As shown in the figure, in step 301, a snapshot CSS of a chip to be detected is obtained, and for convenience of description, it is referred to as CSS hereinafter₀For example, the snapshot may be captured in real time by a camera when the chip is captured, or may be selectively input from a snapshot of the chip stored in a specified location such as a database file.

In step 303, CSS is performed by checking snapshot₀Multiple, e.g., K-fold filtering, is performed to eliminate environmental interference images imaged with the chip image CI, such as the QR code or calibration hole image shown in fig. 2, where the determination of the K value is aimed at substantial elimination of the interference images and subsequent corner detection algorithms may be implemented. At the same time, recording the filtered image each time, thereby obtaining a filtered image CSS₁、CSS₂、...CSS_K. In a preferred embodiment of the invention, Gaussian is usedThe interference images such as QR codes are regarded as Gaussian noise, so that the Gaussian low-pass filter can be utilized to gradually eliminate the noise by gradient down-sampling, thereby generating a series of Gaussian pyramid image sets which are arranged in a pyramid shape, have gradually reduced resolution and are derived from the same original image, wherein the bottom layer of the pyramid is an image to be processed such as a snapshot CSS₀While the top layer is a low resolution approximation image such as CSS_K. Fig. 4A shows a schematic diagram of the effect of such gaussian pyramid filtering. Thus CSS of input snapshot by using K-level Gaussian pyramid filter₀The whole pyramid, namely the pyramid image of the K +1 layer can be obtained by continuously iterating the steps. As an example, fig. 4B schematically shows a 4-layer pyramid image diagram obtained after filtering 3 times when K is included, wherein CSS is used₀Representing a snapshot of the chip, at the lowest level of the pyramid, and CSS₁、CSS₂、CSS₃Then represents each image obtained after 1, 2, 3 times of filtering. It is noted here that the images shown in fig. 4B are not drawn to scale. As can be seen in the figure, the CSS is the last filtered image₃The traces of the QR codes and the calibration holes of the interference images are basically eliminated. It should be noted that the 3-layer filtering is only schematically illustrated here to substantially eliminate the interference image, but obviously, in practice, more filtering may be needed to eliminate the interference image, which depends on the filter used and the imaging quality of each interference image, so that the specific filtering times can be predetermined according to the practical application and used in the subsequent large number of chip detections. In one example, 7-layer filtering may be employed to eliminate the interference image.

It should be noted that, the QR code and the calibration aperture are only used as the source of the interference image, but the invention is not limited thereto, and the interference image may also be from other intentionally set sources or unintentionally present sources, but filtered layer by layer after gaussian pyramid filtering.

CSS to snapshot is done in step 303₀After Gaussian pyramid filtering, an example is obtainedThe multilayer pyramid image as shown in FIG. 4B, with CSS at the highest layer₃The interference image is substantially eliminated in the image. Thus, the process returns and continues with step 305. In step 305, a snapshot CSS is determined based on the filtered image created in the pyramid image₀Corner points of the center chip image CI. According to the invention, in order to conveniently position the four corner points of the chip image CI, the global information can be obtained most when the chip image CI is positioned at a high pyramid layer. Therefore, according to the embodiment of the present invention, the corner positions of the image of the highest layer are searched from the pyramid image of the highest layer, the to-be-searched regions of the next layer image for the corners are determined by using the corner positions, and the corners of the next layer image are searched in the to-be-searched regions. Therefore, the interference corner points can be effectively removed until the four corner points of the target chip image CI are determined finally, or a defective chip can be directly found in the corner point searching process.

Fig. 5 shows a flow diagram of corner detection implemented at step 305 according to an embodiment of the present invention. For convenience of description, the 4-layer pyramid image shown in fig. 4B is used as an example for illustration.

In step 501: the corner points of the top-level image are determined using a corner point search algorithm, in this example, the CSS of the layer 4 image is first determined₃Each corner point of (a). After finding the corner points of the layer image, in step 503, it is determined for the image CSS₃Whether the number of found corner points is greater than or equal to 4. If less than 4, it can be determined that the current snapshot image CSS₀If the image is not a valid image, the corner point detection process is exited, an error is returned, and the step 301 is returned to obtain the next chip snapshot to be detected. If in step 503 it is determined that the number of corner points found is greater than or equal to 4, step 505 is entered, the n corner points C to be found in step 501 for the layer 4 image₄-1、C₄-2、C₄-3、…C₄N is mapped into the next layer image, i.e. layer 3 image. Here the subscript 4 represents the layer 4 image. Furthermore, here 'mapping' means that corner points determined for the layer 4 image are marked in situ into the layer 3 image.

At step 507, for the next layer image, to map from the previous layer imageAnd establishing corner searching areas of the image of the layer by taking the coming corner as a center, and searching the corner of the image of the layer in the searching areas. Specifically, in this example, in the third layer image CSS₂In order to correspond to the corner point C of the 4 th layer₄-1、C₄-2、C₄-3、…C₄N is spread in four directions with the mapping point of-n as the center₃Obtaining n rectangular expansion frames T by one pixel unit₃-1、T₃-2、T₃-3、…T₃N as the corner search area of the layer 3 image. Thereby, the image CSS₂Within the determined n corner search areas, corner points within each search area are determined.

According to the embodiment of the invention, the determination of the number f of the expanded pixels is related to the hierarchy of the pyramid. In one example, for the ith layer image, it extends the number of pixels f_iCan be determined by the following formula:

f_i＝F/sqrt(i),

where i is not less than 1 and not more than K, sqrt () represents a square-open function, and F is a predetermined value smaller than the K +1 th layer image size and makes corner search regions established for two adjacent corners not overlap. As an example, F is 10-30, preferably 18.5.

In step 509, a search area T is determined₃-1、T₃-2、T₃-3、…T₃-n is the number of corner points searched in each search area is 0. If 0, the area is considered invalid, the search area is deleted in step 510 and corner detection in other search areas continues. Otherwise, go to step 511, determine and record the CSS for the current layer image₂Found corner points. It should be noted here that in step 511, if a certain search area T exists₃If the number of corner points found in i is greater than 0, the center positions of all corner point positions found in the area are used as new corner point positions. It is assumed here that CSS is applied to the image₂M new corner points are determined and marked as C₃-1，C₃-2，C₃-3，…,C₃-m, wherein m ≦ n.

At step 513, it is determined whether the current layer picture is the last layer filtered picture, which in this example isWhether or not to reach CSS₁(Note that CSS is used herein₀Is an original image and therefore not a filtered image). If not, the step 505 and 511 are repeatedly executed to determine the corner points of the other lower layer images respectively. For example, in this example, after determining that the layer 3 image is completed, it is still necessary to continue to determine the CSS of the layer 2 image₁Each corner point of (a). For this purpose, the CSS is determined for the layer 2 image₁At the corner point, as shown in step 505 and 507, to obtain the CSS from the third layer image₂M corner points C₃-1，C₃-2，C₃-3，…C₃-m mapped points are centered and spread f in four directions₂One pixel unit to retrieve m rectangular expansion frames T₂-1、T₂-2、T₂-3、…T₂-m as layer 2 image CSS₁Then proceeds to determine the corners in the m search areas according to steps 509, 510, 511. It is assumed here that CSS is applied to the second layer image₁Determines p new corner points, denoted as C₂-1，C₂-2，C₂-3，…,C₂-p, wherein p ≦ m.

In step 513, when it is determined that the last layer filtered image CSS is currently performed₁If yes, step 515 is entered, and the layer 1 picture, i.e. the original picture CSS, is continuously determined₀The corner points of (a). In determining CSS₀Similar to before, to CSS from layer 2 images₁P corner points C₂-1，C₂-2，C₂-3，…,C₂P mapped point centered expansion f in four directions₁Pixel unit to generate p rectangular extension frames as the first layer image CSS₀Corner point search area T₁-1、T₁-2、T₁-3、…,T₁P, then according to the process shown in step 509 and 511, removing the search areas where no corner is found and determining one or more corners in each of the remaining search areas. It is to be noted here that, unlike the processing for the other layer images in the pyramid image, in step 511, the first layer image CSS is determined₀If more than one corner point in a certain search area is found, then the corner points in the search area are reservedAll these corner points found by the search area are not determined as the center points of these corner points as in the other layer images. It is assumed here that CSS is performed for the first layer image in step 515₀It is determined that there are corner points for a total of q search areas, then these search areas and the corner points for each search area are stored, and then the q search areas and the corner point information therein are returned in step 517, the process returns to fig. 3, and the process continues to step 307.

In the above embodiments of the present invention, the corner search algorithm for searching the corners of each layer image may be implemented in various manners, for example, implemented by the HARRIS corner search algorithm.

In step 307, CSS is performed on the image at the first layer₀The following judgment processing is executed between the determined corner searching area and the corner: judging first layer image CSS₀Whether the number q of corner search areas remaining in (a) is equal to 4, wherein if q is equal to<4, then the CSS of the snapshot is determined₀Not a valid image, so the step 301 is returned to obtain the next chip snapshot to be detected. If q is>4, indicating that there are a plurality of corner points in the current chip, i.e. there is significant corner damage in the chip, so as shown in fig. 3, the process directly proceeds to step 317, and directly proceeds to the image, e.g. the snapshot CSS₀The search areas and the corner points of the search areas are identified so that corner defects can be clearly seen. If it is determined that the number q of remaining corner search areas is exactly equal to 4, step 308 is entered.

In step 308, it is further determined whether each search area exactly contains 1 corner point, and if a certain search area contains more than 1 corner point, it indicates that the chip has a damage defect at the position of the search area, so that the subsequent processing is finished, and the process directly proceeds to step 317, where a plurality of corner points in the search area are identified on the image as a chip defect. If each of the 4 search regions contains exactly 1 corner point, it is determined that four corner points C exist in the chip image CI₀-1，C₀-2，C₀-3，C₀-4, go to step 309.

In step 309, the four corner points C are used₀-1，C₀-2，C₀-3，C₀-4 as profile reference, defining a quadrilateral area. In one example, the quadrilateral area may be considered as a chip real image CI. However, it is understood that the quadrilateral area may not be a rectangle because the optical axis of the camera is not perfectly perpendicular to the real chip, or the ray of the optical axis does not pass through the center point of the chip exactly, i.e. the image of the quadrilateral area is distorted with respect to the real chip. Thus, in another example, it may be further determined in step 309 whether the quadrilateral area is a standard rectangle, and if not, proceeding to step 311, the quadrilateral is converted to a rectangle, and the rectangular area is filled with corresponding pixel values, for example, interpolation techniques may be used to predict each filled pixel, thereby generating the chip real image CI'. As an example, a perspective transformation technique may be adopted to determine the transformed chip actual image CI ' by finding a perspective transformation matrix using four vertices of a quadrangle and four vertices of a new rectangle, wherein the transformed chip actual image CI ' may be determined by selecting the four vertices of the maximum bounding rectangle containing the four vertices of the quadrangle as the four points of the new rectangle used to determine the perspective transformation matrix, and calculating the chip actual image CI ' based on the image CI after finding the perspective transformation matrix, including the four updated vertices C ' of the chip actual image CI '₀-1，C′₀-2，C′₀-3，C′₀-4. After obtaining a new image CI 'using a perspective transformation in step 311, the actual chip area CI' can be taken from the CSS snapshot₀And cutting out the steel wire completely, and carrying out subsequent step processing. If it is determined in step 309 that the quadrilateral area is substantially a standard rectangle, then the subsequent processing is performed directly, bypassing step 311.

In step 313, adaptive binarization processing is performed on the actual image CI' to find defects existing in the chip. Since the chip surface, edge texture, and the like having no defects are uniform, and appear as a completely uniform color such as black when imaged, for example, and the texture structure is inevitably destroyed when there is a cutting defect or surface damage, the chip surface, edge texture, and the like inevitably appear as a pixel different from a normal pixel such as black, for example, white, after being subjected to binarization processing. The present invention is based on this in an attempt to find possible defects. Here, a binarization algorithm may be used, for example, an OTSU algorithm which is a global threshold method or a Sauvola algorithm which is a local threshold method may be used. Thus, with the OTSU algorithm or the Sauvola algorithm, a binarized image can be generated based on the actual image CI'.

In another embodiment of the present invention, to achieve better effect, the influence that the uneven illumination may have on the binarization is further considered here, so before the binarization processing is performed, the illumination contrast elimination processing is performed on the actual image CI', as shown by the dashed square in fig. 3. In one example, background compensation techniques may be employed to achieve contrast cancellation. Since the background of the image may be non-uniform, the luminance background of the image may be determined in advance and then used to perform background compensation on the original image, thereby obtaining a background-compensated image CI ″. And then, carrying out binarization processing on the image CI' after illumination contrast compensation by using an OSTU binarization method or a Sauvola algorithm.

In the binarized image, a white area may be defined as a defective area and a black area as a normal area. In order to present the defect regions more intuitively, the method of the present invention may further comprise step 315 of identifying the defect locations by establishing connected regions. As an example, the binary image may be divided into different connected regions by using an 8-near-point method, where 8-near-point connection refers to: if a pixel is connected to other pixels in the top, bottom, left, right, top left corner, bottom left corner, top right corner or bottom right corner, they are considered connected, and the area with a value of 1, i.e. white, is the defect location, so that the defect can be marked in the image CI ″. The image CI ″ labeling the defect location and/or information indicating that the current chip is defective may then be output in step 317, fig. 6A and 6B exemplarily showing the chip defect labeling the white connected region. As another example, CSS, which is a snapshot image directly output, may also be selected₀And the defects represented by the near-by region connectivity are marked thereon.

The above examples illustrate the inventionThe method of the embodiment of the invention realizes the automatic detection of the chip defects. Fig. 7 shows an image processing system 700 for implementing chip detection using the above method according to an embodiment of the present invention. The system 700 comprises a user interface 701, an image processing unit 702 and a storage 703. The user interface 701 is used for receiving a user instruction and inputting a snapshot CSS of a chip to be detected according to the user instruction₀. The image processing unit 702 is configured to implement the chip defect detection methods discussed herein by executing program code to determine whether a chip is defective and output information indicating whether a defect is present to the user interface 701 for viewing by a user. In addition, the user display interface 701 is further configured to graphically display an image marked with the occurrence position of the chip defect, such as the binarized image CI "or a CSS including a region in which the chip region CI has been updated to the binarized image CI ″₀. In another embodiment of the present invention, the user interface may be further configured to include other command button options, as shown in fig. 8, the user interface 701 includes buttons for specifying input and output paths of the chip snapshot image to be detected, for example, an input folder button, a select input folder button, an output folder button, and a select output folder button. Wherein the input folder button is used to open a designated input folder path, which may be designated by the select input folder button, and to select a snapshot of the chip to be tested from among snapshots stored in the storage device 703. The output folder button is used to open a designated output folder in the storage device 703, which may be specified by selecting the output folder button to specify a storage path, to view a binarized image or updated snapshot stored therein, which is marked with a defect after the completion of defect detection by the image processing unit 702, for subsequent viewing.

The user interface 701 further includes a start button, and after the input source and the storage path of the image are specified, the start button is clicked to instruct the image processing unit 702 to execute the chip defect detection process. In addition to these operation buttons, the user interface 701 may include other prompt information such as the elapsed time of image processing and the image currently being processed. After the image processing unit 702 completes the image defect detection, it may store the binarized image or the update snapshot in a designated output folder in the storage device 703 and instruct the user interface 701 to display the detection result in an updated display interface, such as whether a defect exists, the position where the defect exists, and the like, in an image manner.

It is to be noted herein that, although the embodiments of the present invention have been described in conjunction with the above, the present invention is not limited thereto. For example, the image processing system herein may include a processor, an electronic device, a hardware device, an electronic component, a logic circuit, a memory, software code, firmware code, etc., or any combination thereof. Those of skill would further appreciate that the various illustrative modules and method steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. The image processing system in a logic sense is formed by reading a corresponding computer program instruction in the nonvolatile memory into the memory through the processor and running the computer program instruction by taking software implementation as an example.

Another embodiment of the invention provides a machine-readable medium having stored thereon machine-readable instructions which, when executed by a computer, cause the computer to perform any of the methods disclosed herein. In particular, a system or apparatus may be provided which is provided with a machine-readable medium on which software program code implementing the functionality of any of the embodiments described above is stored and which causes a computer of the system to read and execute machine-readable instructions stored in the machine-readable medium. In this case, the program code itself read from the machine-readable medium can realize the functions of any of the above-described embodiments, and thus the machine-readable code and the machine-readable medium storing the machine-readable code form part of the present invention.

It should be noted that not all steps in the above flows are necessary, and some steps may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.

While the invention has been shown and described in detail in the drawings and in the preferred embodiments, it is not intended to limit the invention to the embodiments disclosed, and it will be apparent to those skilled in the art that various combinations of the code auditing means in the various embodiments described above may be used to obtain further embodiments of the invention, which are also within the scope of the invention.