阅读说明：本技术 振动检测系统 (Vibration detection system ) 是由 麦可·柯比 中野晶太 宗像広志 于 2020-09-03 设计创作，主要内容包括：一种振动检测系统(100),对表面为非镜面的超声波焊头(12)及焊针(13)的振动进行检测,包括：激光光源(20),对超声波焊头(12)及焊针(13)照射平行激光(21)；照相机(30),包括对平行激光(21)所照射的超声波焊头(12)及焊针(13)的图像进行拍摄的拍摄元件(31)；以及图像处理装置(40),对照相机(30)所拍摄的图像进行处理而显示振动产生部位。(A vibration detection system (100) for detecting vibration of an ultrasonic horn (12) and a welding pin (13) having a non-specular surface, comprising: a laser light source (20) that irradiates the ultrasonic horn (12) and the welding pin (13) with parallel laser light (21); a camera (30) including an imaging element (31) for imaging an image of the ultrasonic horn (12) and the welding pin (13) irradiated with the parallel laser beam (21); and an image processing device (40) for processing the image captured by the camera (30) and displaying the vibration-generating region.)

1. A vibration detection system for detecting vibration of an observation target object whose surface is a non-mirror surface, comprising:

a laser light source that irradiates the observation target with laser light;

a camera having an imaging element for capturing an image of the observation target irradiated with the laser beam to obtain an image; and

and an image processing device for processing the image captured by the camera and displaying the vibration generation part.

2. The vibration detection system of claim 1, wherein the vibration detection system is configured to detect vibration of the vehicle

An exposure time at the time of photographing by the camera is longer than a vibration cycle of the observation target object, and an image including an interference pattern generated by interference of the laser light reflected on the surface of the observation target object is acquired;

the image processing device specifies a vibration-generating pixel based on a deviation between an image of the observation target object including the interference pattern in the non-vibration state and an image including the interference pattern in the vibration state acquired by the camera, and outputs an observation image including a display corresponding to the vibration-generating pixel specified as the image of the observation target object.

3. The vibration detection system of claim 2, wherein the vibration detection system is configured to detect vibration in a vehicle

The image processing apparatus maintains the designation of the pixel as a vibration generating pixel when a predetermined number of other vibration generating pixels are present in a predetermined range around the designated vibration generating pixel, and cancels the designation of the pixel as a vibration generating pixel when the predetermined number of vibration generating pixels are absent in the predetermined range.

4. Vibration detection system according to claim 2 or 3, characterized in that

The laser light source irradiates the observation target with a parallel laser beam having a single wavelength.

Technical Field

The present invention relates to a vibration detection system for detecting vibration of an observation target, and more particularly to a vibration detection system for detecting vibration of an observation target having a non-mirror surface.

Background

In a wire bonding apparatus, when ultrasonic vibration of a tool such as a bonding pin is observed, a method using a laser doppler vibrometer is often used (for example, see patent document 1).

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2013-125875

Disclosure of Invention

Problems to be solved by the invention

In recent years, it has been required to detect vibration on a two-dimensional surface of an observation target in real time. However, in the method described in patent document 1, the measurement portion of the vibration is limited to the point or line irradiated with the laser beam, and the vibration on the two-dimensional surface cannot be observed in real time.

Therefore, an object of the present invention is to detect vibration on a two-dimensional surface of an observation target in real time.

Means for solving the problems

The vibration detection system of the present invention is a vibration detection system for detecting vibration of an observation target object whose surface is a non-mirror surface, the vibration detection system including: a laser light source that irradiates an observation target with laser light; a camera including an imaging element for capturing an image of an observation target irradiated with laser light; and an image processing device for processing the image captured by the camera and displaying the vibration generation part.

As described above, since the vibration generation region is specified based on the two-dimensional image captured by the camera, the vibration on the two-dimensional surface of the observation target object can be detected in real time.

In the vibration detection system of the present invention, the exposure time at the time of photographing by the camera is longer than the vibration cycle of the observation object, and an image including an interference pattern generated by interference of laser light reflected on the surface of the observation object is acquired; the image processing apparatus may specify a vibration generation pixel based on a deviation between an image of the observation target object including the interference pattern in the non-vibration state and an image including the interference pattern in the vibration state acquired by the camera, and output the observation image including a display corresponding to the vibration generation pixel specified as the image of the observation target object.

When an observation target having a non-specular surface is irradiated with laser light, an interference pattern generated by interference of the laser light due to non-specular reflection appears on the surface of an imaging element of a camera. The image capturing element of the camera acquires an image of the interference pattern. Since the exposure time of the camera at the time of imaging is longer than the vibration cycle of the observation target, the camera acquires an image of a wobbling interference pattern when the observation target vibrates. If the image of the interference pattern is shaken, the intensity of the luminance of the pixel changes compared with the case of non-shaking. Therefore, by designating a pixel in which the intensity of the luminance at the time of vibration changes from the intensity of the luminance at the time of non-vibration as a vibration generation pixel and outputting an observation image including a display corresponding to the vibration generation pixel designated as the image of the observation object, it is possible to visualize and display the vibration portion of the observation object.

In the vibration detection system of the present invention, the image processing apparatus may maintain the designation of the pixel as the vibration generating pixel when a predetermined number of other vibration generating pixels exist in a predetermined range around the designated vibration generating pixel, and may cancel the designation of the pixel as the vibration generating pixel when the predetermined number of vibration generating pixels do not exist in the predetermined range.

This suppresses the pixels that do not actually vibrate due to noise from being designated as vibration-generating pixels, and can detect vibration with high accuracy.

In the vibration detection system of the present invention, the laser light source may irradiate the observation target with the parallel laser light of a single wavelength.

By irradiating a parallel laser beam of a single wavelength, an interference pattern of the laser beam caused by non-specular reflection appears more clearly, and a speckle pattern photographed by a camera becomes more clearly. This enables detection of vibration with higher accuracy.

ADVANTAGEOUS EFFECTS OF INVENTION

The invention can detect the vibration on the two-dimensional surface of the observation object in real time.

Drawings

Fig. 1 is a system diagram showing a configuration of a vibration detection system according to an embodiment.

Fig. 2 is a schematic diagram showing a state where parallel laser light reflected on the surface of the welding pin is incident on an imaging element of the camera.

Fig. 3 is a schematic diagram showing an image taken by the camera.

Fig. 4 is a schematic diagram showing a pixel of an imaging element of a camera.

Fig. 5 is a flowchart showing image processing by the image processing apparatus.

Fig. 6 is a schematic diagram showing an observation image output to a monitor.

Detailed Description

Hereinafter, the vibration detection system 100 according to the embodiment will be described with reference to the drawings. In the following description, the vibration detection system 100 will be described by detecting the vibration of the ultrasonic horn 12 or the needle 13 of the wire bonding apparatus 10 as an object to be observed.

First, referring to fig. 1, a wire bonding apparatus 10 including an ultrasonic horn 12 or a horn 13 as an observation target object will be briefly described. The wire bonding apparatus 10 includes: a bonding arm 11, an ultrasonic horn 12, a welding pin 13, an ultrasonic oscillator 14, and a bonding platform 16.

The ultrasonic horn 12 has a welding pin 13 attached to a front end thereof and an ultrasonic oscillator 14 attached to a rear end thereof. The ultrasonic horn 12 is ultrasonically vibrated by ultrasonic vibration generated by the ultrasonic oscillator 14, and ultrasonically vibrates the welding pin 13. The ultrasonic horn 12 is connected to the joining arm 11, and the welding pin 13 is driven in a direction approaching and separating from the joining surface 16 by a driving mechanism not shown. The bonding stage 16 adsorbs and fixes a substrate 18 having a semiconductor element 17 mounted on a surface thereof. Wire bonding apparatus 10 presses the tip of bonding pin 13 against the electrode of semiconductor element 17 by a driving mechanism, not shown, to bond wire 15 to the electrode of semiconductor element 17, and then moves bonding pin 13 onto the electrode of substrate 18 to press the tip of bonding pin 13 against the electrode of substrate 18 to bond wire 15 to the electrode of substrate 18. Thus, wire bonding apparatus 10 connects the electrodes of semiconductor element 17 and the electrodes of substrate 18 by loop wire 19. Therefore, in the joining operation, the ultrasonic horn 12 and the welding pin 13 are ultrasonically vibrated. The vibration detection system 100 according to the embodiment detects and displays the vibration on the two-dimensional surface of the ultrasonic horn 12 or the welding pin 13. The surface of the ultrasonic horn 12 or the horn 13 is non-specular and has fine irregularities on the surface.

As shown in fig. 1, the vibration detection system 100 includes: a laser light source 20, a camera 30, and an image processing device 40.

The laser light source 20 converts the laser light of a single wavelength output from the laser oscillator into parallel laser light 21 by using a beam expander (beam expander), and irradiates the ultrasonic horn 12 or the welding pin 13 with the parallel laser light 21 of the single wavelength. The camera 30 includes an imaging element 31 for taking a two-dimensional image of the ultrasonic horn 12 or the welding pin 13 irradiated to the parallel laser light 21. The image processing device 40 processes the two-dimensional image captured by the camera 30 to specify a vibration generation region, and outputs and displays a two-dimensional observation image 12e and an observation image 13e (see fig. 6) in which the display of the vibration region is different from that of the other regions, on the monitor 50. The image processing apparatus 40 is a computer including a processor 41 and a memory 42 for performing information processing therein.

Next, the operation of the vibration detection system 100 according to the embodiment will be described with reference to fig. 2 to 6.

As shown in fig. 2, since the surface 13a of the bonding pin 13 is non-specular and fine irregularities exist on the surface 13a, when the parallel laser beam 21 is irradiated to the surface 13a of the bonding pin 13, the parallel laser beam 21 is reflected in random directions on the surface 13a of the bonding pin 13. The reflected laser beams 22 caused by the non-specular reflection interfere with each other, and an interference pattern of the reflected laser beams 22 appears on the surface of the imaging element 31 of the camera 30.

Since the interference pattern includes a bright portion where light is intensified and a dark portion where light is weakened, the imaging element 31 of the camera 30 acquires an image 13c of a speckle pattern as an interference pattern, as shown in fig. 3, the image 13c including a plurality of bright portions 33 and dark portions 34 appearing on the surface of the image 13b of the welding pin 13.

Therefore, when the ultrasonic horn 12 and the pins 13 are imaged by the camera 30, the camera 30 acquires an image 12b of the ultrasonic horn 12 including a speckle pattern and an image 13b of the pins 13 including a speckle pattern as shown in a field of view 32 of fig. 13. The images 12b and 13b are images including interference patterns.

The exposure time of the camera 30 at the time of shooting is longer than the vibration cycle of the ultrasonic vibration of the ultrasonic horn 12 and the welding pin 13. Therefore, when the ultrasonic horn 12 and the horn 13 are ultrasonically vibrated, the image 12b of the ultrasonic horn 12 including the speckle pattern and the image 13b of the horn 13 including the speckle pattern on the imaging device 31 are fluctuated as indicated by arrows 91 and 92 in the exposure in the region where the vibration peak is formed. On the other hand, in the area of the vibrating node, even if the ultrasonic horn 12 and the horn 13 are ultrasonically vibrated, the image 12b and the image 13b on the imaging element 31 do not fluctuate during exposure.

In the region where the images 12b and 13b are fluctuated during exposure, the intensity of the luminance of the pixels 36 of the imaging element 31 is changed from the intensity of the luminance in a stationary state or a non-vibrating state in which the ultrasonic horn 12 and the horn 13 are not ultrasonically vibrated. In an example, the intensity of the luminance of the pixel 36 is increased in a region where the peak of the vibration is located, as compared with the case of non-vibration.

On the other hand, when the images 12b and 13b do not move during exposure of the node that has been vibrated, the images 12b and 13b are substantially the same as the images 12a and 13b in which the ultrasonic horn 12 and the welding pin 13 are in a stationary state or a non-vibrating state. Therefore, in the area of the vibration nodes where the images 12b and 13b do not fluctuate during exposure, the intensity of the brightness of the pixels 36 of the imaging element 31 is substantially the same as the intensity of the brightness in the stationary state or the non-vibration state where the ultrasonic horn 12 and the horn 13 do not vibrate ultrasonically.

Therefore, as shown in fig. 4, the processor 41 of the image processing apparatus 40 designates, as the vibration generating pixel 37, a pixel 36 in which the intensity of the luminance at the time of the ultrasonic vibration changes from the intensity at the time of rest or at the time of non-vibration in which the ultrasonic vibration is not performed. Here, the intensity of the luminance is a degree of the detected luminance of the pixel 36, and may be expressed by 256 steps of 0 to 255, for example.

The image processing apparatus 40 performs the following processing on each pixel 36 of the image frame 35, which is a region of the two-dimensional image of the field of view 32 on which image processing is performed at a time, to specify the vibration generating pixel 37. In the following description, the coordinates (x, y) indicated by the reference numerals denote the coordinates (x, y) of the two-dimensional image frame 35, and for example, the pixel 36(x, y) denotes the pixel 36 of the coordinates (x, y).

As shown in step S101 of fig. 5, the processor 41 of the image processing apparatus 40 reads the image frame 35v at the time of ultrasonic vibration and the image frame 35S at the time of still, from the two-dimensional image at the time of ultrasonic vibration and the two-dimensional image at the time of still or non-vibration acquired from the camera 30 stored in the memory 42.

As shown in step S102 of fig. 5, the processor 41 calculates an average value Ia (x, y) of the intensity Iv (x, y) of the ultrasonic vibration luminance and the intensity Is (x, y) of the stationary luminance in each pixel 36(x, y).

Average value Ia (x, y) ═ Iv (x, y) + Is (x, y) ]/2

As shown in step S103 of fig. 5, the processor 41 calculates an average value in the image frame 35 of the absolute value of the deviation between the intensity Iv (x, y) of the luminance at the time of the ultrasonic vibration and the average value Ia (x, y) in each pixel 36(x, y) as the absolute deviation average value.

Average absolute deviation average value | Iv (x, y) -Ia (x, y) | on the image frame 35

As shown in step S104 of fig. 5, the processor 41 calculates the quartic value NIave (x, y) of the normalized pixel intensity using the following (equation 1).

Nlave (x, y) [ | Iv (x, y) -Ia (x, y) |/absolute deviation average value]⁴- - (formula 1)

As shown in step S105 of fig. 5, when NIave (x, y) is equal to or greater than 1, the processor 41 determines that the change in the intensity of the luminance of the pixel 36(x, y) is valid, proceeds to step S106 of fig. 5, designates the pixel 36(x, y) as the vibration generation pixel 37(x, y), and proceeds to step S107. If the processor 41 determines in step S107 that all the pixels 36(x, y) of the image frame 35 are not to be processed, the process returns to step S104 to perform the processing of the next pixel 36(x, y). On the other hand, if the processor 41 determines No in step S105 of fig. 5, the process returns to step S104 to perform the process of the next pixel 36(x, y). The processor 41 calculates NIave (x, y) among all the pixels 36(x, y) of the image frame 35 to specify the shake-generating pixel 37(x, y) of the image frame 35, determines YES in step S107 of fig. 5, and proceeds to step S108 of fig. 5.

In step S108 of fig. 5, the processor 41 checks whether or not only a predetermined number of other vibration generating pixels 37(x1, y1) are present in a predetermined range around one vibration generating pixel 37(x, y). For example, a square array of 5 × 5 pixels 36 centered around the vibration-generating pixel 37(x, y) may be set as a predetermined range, and it may be checked whether or not seven to eight other vibration-generating pixels 37(x1, y1) are present therein. If it is determined as YES in step S108 of fig. 5, it is determined that the intensity of the luminance of the pixel 36(x, y) has changed due to the ultrasonic vibration, and the process proceeds to step S109 of fig. 5, where the pixel 36(x, y) is maintained as the vibration-generating pixel 37(x, y).

On the other hand, in the case where seven to eight other vibration-generating pixels 37(x1, y1) are not present in the array, the change in intensity of the luminance of the pixel 36(x, y) is determined not to be caused by vibration, and the process proceeds to step S110 in fig. 5, where the pixel 36(x, y) is not designated as the vibration-generating pixel 37(x, y).

Processor 41 then causes the designation of vibration-producing pixels 37(x, y) to be determined. The processor 41 performs the processing described above in each image frame 35, and determines the designation of the vibration generation pixel 37(x, y) for all the pixels 36(x, y) of the imaging element 31.

As shown in fig. 6, the processor 41 outputs an observation image 12e and an observation image 13e including a display corresponding to the designated vibration generation pixel 37(x, y) to the image of the ultrasonic horn 12 and the welding pin 13, thereby visualizing and displaying the vibration on the two-dimensional surface of the ultrasonic horn 12 and the welding pin 13.

The observation images 12e and 13e can be in various forms, but in fig. 5, for example, red dots 52 are superimposed on the portions corresponding to the vibration generation pixels 37 of a general image obtained by irradiating non-interference light such as an electric lamp to the ultrasonic horn 12 and the welding pin 13, and displayed. According to this display method, a large number of red dots 52 are displayed in the region that becomes the peak of the vibration, and substantially no red dots are displayed in the portion that becomes the node of the vibration. In the example shown in fig. 5, it is understood that a peak of vibration exists in the ultrasonic horn 12 showing a large number of red dots 52, the middle portion of the change in the diameter of the horn 13, and the tip portion of the horn 13, and the other portion is a node of vibration.

As described above, the vibration detection system 100 according to the embodiment processes the two-dimensional image of the ultrasonic horn 12 or the needle 13 and displays the two-dimensional image as the two-dimensional observation image 12e or the observation image 13e, so that the vibration on the two-dimensional surface of the ultrasonic horn 12 or the needle 13 can be detected in real time.

In the above description, the vibration detection system 100 is described as detecting the vibration of the ultrasonic horn 12 and the horn 13 of the wire bonding apparatus 10, but may be applied to the detection of the vibration of other portions of the wire bonding apparatus 10.

For example, when bonding the wire bonding apparatus 10 shown in fig. 1, the semiconductor element 17 can be irradiated with the parallel laser beam 21 to detect vibration of the semiconductor element 17. When the vibration of semiconductor element 17 is large, the vibration energy from bonding pin 13 is consumed by the vibration other than bonding, and it can be determined that the bonding is not performed satisfactorily. Similarly, it is possible to detect whether or not the vibration of the substrate 18 is large, and if the vibration of the substrate 18 is large, the vibration energy from the bonding pin 13 is consumed by the vibration other than the bonding, and it can be determined that the bonding is not performed satisfactorily.

The vibration detection system 100 can be applied to a device other than the wire bonding apparatus 10, for example, a chip bonding apparatus or the like, and can detect vibration of each part of another semiconductor manufacturing apparatus.

In the above description, the laser light source 20 is described as irradiating the observation target with the parallel laser light 21 of a single wavelength, but the present invention is not limited thereto, and the laser light may have a small width in wavelength or may be irradiated with a laser light other than the parallel light. Further, the intensity of the laser light may vary somewhat. In the above description, the image of the interference pattern is described as a speckle pattern including a plurality of bright portions 33 and dark portions 34, but the pattern is not limited thereto, and may be other patterns such as stripes.

When the vibration direction of the observation target is not one direction, the plurality of laser light sources 20 and the plurality of cameras 30 are prepared, the observation target is irradiated with laser light from multiple directions, and images are captured from multiple directions by the plurality of cameras 30, whereby the vibration in multiple directions can be detected.

Description of the symbols

10: wire bonding device

11: joint arm

12: ultrasonic welding head

12a, 13b, 13 c: image of a person

12e, 13 e: observation image

13: welding pin

13 a: surface of

14: ultrasonic oscillator

15: lead wire

16: joint platform

17: semiconductor device with a plurality of semiconductor chips

18: substrate

19: loop line

20: laser light source

21: parallel laser

22: reflected laser

30: camera with a camera module

31: imaging element

32: visual field

33: ming dynasty part

34: dark part

35. 35v, 35 s: image frame

36: pixel

37: vibration generating pixel

40: image processing apparatus

41: processor with a memory having a plurality of memory cells

42: memory device

50: monitor with a display

52: red dot

100 vibration detection system.