Multi-view image acquisition and preprocessing system and method under complex illumination condition
阅读说明：本技术 一种复杂光照条件的多目图像采集、预处理系统及方法 (Multi-view image acquisition and preprocessing system and method under complex illumination condition ) 是由 王健 李清洵 居旻 吕琦 于 2019-10-18 设计创作，主要内容包括：本发明公开了一种复杂光照条件的多目图像采集、预处理系统及方法,用于在实际应用中实现对物体进行全方位多角度、复杂光照条件的图像信息的获取,并对获取的图像进行一系列的预处理优化,为后续应用提供有效的图像数据集。本图像采集系统由硬件部分和软件部分两块组成,硬件部分包括整个装置的箱体结构、图像采集模块、光照模块以及通信模块,软件部分包括硬件部分的通信连接和消息传递的实现。系统通过接收读取输入的命令,按照要求高效并且低难度地实现对物体的多目图像采集和存储。针对本采集系统提供了图像预处理的过程,能够高效处理和优化所采集的图像,便于后续更好地利用图像。(The invention discloses a multi-view image acquisition and preprocessing system and method under complex illumination conditions, which are used for acquiring image information of an object under the complex illumination conditions in all directions and at multiple angles in practical application, and performing a series of preprocessing optimization on the acquired image to provide an effective image data set for subsequent application. The image acquisition system consists of a hardware part and a software part, wherein the hardware part comprises a box body structure of the whole device, an image acquisition module, an illumination module and a communication module, and the software part comprises the communication connection of the hardware part and the realization of message transmission. The system efficiently acquires and stores the multi-purpose image of the object with low difficulty according to requirements by receiving a reading input command. The image preprocessing process is provided for the acquisition system, the acquired image can be efficiently processed and optimized, and the image can be better utilized subsequently.)
1. The utility model provides a many meshes image acquisition module of complicated illumination condition which characterized in that: the system comprises an acquisition box body structure, an image acquisition module, an illumination module and a communication module; the bottom of the inner side of the box body structure is provided with a clamping groove for fixing a measured object; the illumination module comprises a plurality of strip-shaped light sources parallel to the edges of the rectangular box body; the strip-shaped light sources are uniformly distributed on the inner side surface of the box body structure; the image acquisition module comprises a plurality of camera modules distributed near the strip-shaped light source; the communication module is connected with the communication terminals of the upper computer and the camera modules.
2. The multi-view image acquisition module for complex lighting conditions of claim 1, wherein: the communication terminal of the camera module preferably selects the raspberry pie, and the raspberry pies are connected to the upper computer server through the relay raspberry pie through distributed connection.
3. The multi-view image acquisition module for complex lighting conditions of claim 2, wherein: the communication terminal realizes distributed storage of distributed images based on a Redis database.
4. The multi-view image acquisition module for complex lighting conditions according to claim 1 or 2, wherein: the camera module is a digital video camera.
5. An image acquisition and preprocessing method using the multi-view image acquisition module with complex illumination condition of claim 1, characterized in that: the method comprises the following steps:
step 1, according to directory information automatically stored during acquisition, images acquired by camera modules at different angles can automatically carry out minimum rectangular approximation and cutting on an object imaging boundary according to the imaging conditions of the angles, the occupation ratio of a meaningless background part in the images is reduced to the maximum extent, and a target region ROI is acquired;
step 2, obtaining a plurality of boundary points of the target object as feature points through feature extraction, selecting a picture as a reference picture for reference, establishing a mathematical transformation model between the images through matching of the feature points, realizing registration of the object in the images and finishing correction;
step 3, calculating and storing the brightness of the reference image, automatically performing image enhancement on all images shot under the light source condition by taking the brightness of the reference image as a reference, and finally adjusting the brightness of the images to be consistent;
step 4, carrying out foreground detection on the shot object; collecting a large number of images of each camera module when no strip-shaped object is placed so as to realize background modeling; through the detection and extraction of the outline, the single color filling of the background part is realized, and the interference of the background is removed;
and 5, detecting the feature points of the processed images through an SIFT algorithm, matching the feature points, removing mismatching point pairs, and establishing a mathematical transformation model between the images, so that the images are spliced, and the complete image information of the collected object is obtained.
Technical Field
The invention relates to the technical field of image detection, in particular to a multi-view image acquisition and preprocessing system and method under complex illumination conditions.
Background
With the rapid development of computer information technology, the fields of machine vision, image processing and the like are gradually emerging. Machine vision technology is a branch of artificial intelligence, which is an discipline that uses a robot to replace human eyes to measure, collect and process a target object, and simulates human eyes and brains to sense and recognize. Machine vision is applied to a plurality of fields such as medicine, military affairs and the like, and plays an increasingly important role.
In various industrial applications, automatic image acquisition and acquisition are the first step of digital image processing in computer vision, so that research and design of an image acquisition system is the basis of development of all vision systems and is an indispensable important component. The image acquisition system is used for converting visual images and characteristics of an object into data which can be processed by a computer.
At present, many researchers provide various image acquisition systems for various specific application scenes and put into use, but most of the systems acquire objects only based on monocular cameras or binocular cameras, and mechanical structures are added to assist in acquiring all-directional images, so that the physical realization difficulty of image acquisition is increased, more manpower and financial resources are consumed, and the efficiency is low. In addition, many existing acquisition systems only perform image acquisition and perform pre-processing of the acquired images simultaneously with the acquisition to achieve a preliminary optimization of the image set, which may introduce unnecessary errors if the acquired image set is subsequently used for applications with high accuracy requirements, such as defect detection.
The image acquisition is used as an important link of machine vision, and the research of a better image acquisition system and a better preprocessing method has important significance. The multi-view vision detection technology is an image processing technology which obtains images of the same target from different angles through a plurality of vision sensors, and performs corresponding processing operations such as matching, splicing, fusing, segmenting and the like on the collected images to finally obtain target image information. The image acquisition scheme based on the multi-view vision can obtain richer image data and information and improve the image acquisition efficiency and precision, so that the multi-view vision method is adopted to realize the acquisition system and is used for realizing the acquisition of the image information of all-around and multiple angles on an object in practical application.
Disclosure of Invention
The purpose of the invention is as follows: in order to realize multi-directional information acquisition of images, the invention provides a multi-view image acquisition and preprocessing system and method under complex illumination conditions, which can acquire rich object image information and more effective image results, and realize the acquisition of three-dimensional multi-angle information of an object by using the multi-view image acquisition system under certain illumination conditions and optimize the acquired images.
The technical scheme is as follows: in order to achieve the purpose, the invention adopts the technical scheme that:
a multi-view image acquisition module with complex illumination conditions comprises an acquisition box body structure, an image acquisition module, an illumination module and a communication module; the bottom of the inner side of the box body structure is provided with a clamping groove for fixing a measured object; the illumination module comprises a plurality of strip-shaped light sources parallel to the edges of the rectangular box body; the strip light sources are uniformly distributed on the inner side surface of the box body structure; the image acquisition module comprises a plurality of camera modules distributed near the strip-shaped light source; the communication module is connected with the communication terminals of the upper computer and the camera modules.
Furthermore, the communication terminal of the camera module preferably selects a raspberry group, and the raspberry groups are connected to the upper computer server through the relay raspberry groups through distributed connection.
Further, the communication terminal realizes distributed storage of images based on Redis.
Further, the camera module is a digital video camera.
An image acquisition and preprocessing method of a multi-view image acquisition module utilizing the complex illumination condition comprises the following steps:
and 5, detecting the feature points of the processed images through an SIFT algorithm, matching the feature points, removing mismatching point pairs, and establishing a mathematical transformation model between the images, so that the images are spliced, and the complete image information of the collected object is obtained.
Has the advantages that:
1. the image acquisition system of the invention can acquire images in three-dimensional and multi-direction, and one image acquisition camera module can only acquire images at one angle and a limited visual field.
2. The invention is efficient for image acquisition, and because a large number of images need to be acquired for other purposes under normal conditions, the invention adopts a multi-view camera to capture images of an object from multiple angles in consideration of the efficiency of image acquisition.
3. According to the invention, each camera module of the multi-view image acquisition system can automatically classify and store the acquired images according to the installation position, the installation angle, the shooting time, the number of the shot object and the illumination condition during each shooting, so that the specific acquisition information of all the images can be visually obtained according to the stored catalog and the name, and great convenience is provided for subsequent image processing.
4. The invention reduces the physical realization difficulty of multi-angle image acquisition, can reduce the cost and improve the acquisition efficiency. In addition, the realization of the specific operation mechanism of the invention depends on the publish/subscribe mode of Redis, thereby greatly improving the efficiency of data acquisition and storage. The images are preprocessed, so that possible interference or errors generated in the image acquisition process are reduced to a great extent, and the availability of an image set is improved.
Drawings
FIG. 1 is a schematic diagram of the geometry and numbering of an acquired object
FIG. 2 is a schematic view of the geometry of an image acquisition system
FIG. 3 is a diagram of a communication module of the image acquisition system
FIG. 4 is a diagram of a bridge connection architecture
FIG. 5 is a schematic diagram of Redis database and subscription/publication structure
FIG. 6 is a diagram illustrating image cropping effects
FIG. 7 is a schematic diagram of image registration effect
FIG. 8 is a flow chart of the operation of the image acquisition system
FIG. 9 is a flow chart of image acquisition
FIG. 10 is a flow chart of image cropping
FIG. 11 is a flowchart of image registration rectification
FIG. 12 is a flowchart of image brightness calibration
FIG. 13 is a flow chart of image background detection and fill-in
Fig. 14 is an image stitching flowchart.
Detailed Description
The present invention will be further described with reference to the accompanying drawings.
The invention uses an image acquisition system to acquire images of a strip-shaped object. The object has a geometry as shown in fig. 1, except for the bottom, a total of 5 faces, 4 side edges and 2 upper side edges. The image acquisition system of the present invention can be used to acquire images of the edges and faces of an object, as shown in fig. 2. The system comprises an acquisition box body structure, an image acquisition module, an illumination module and a communication module; the bottom of the inner side of the box body structure is provided with a clamping groove for fixing a measured object; the illumination module comprises a plurality of strip-shaped light sources parallel to the edges of the rectangular box body; the strip light sources are uniformly distributed on the inner side surface of the box body structure; the image acquisition module comprises a plurality of camera modules distributed near the strip-shaped light source; the communication module is connected with the communication terminals of the upper computer and the camera modules.
The number, the position and the angle of the camera modules are set according to the shape of a target object, specifically, in order to obtain an image of an observation object without a dead angle by the setting principle of the camera modules, the setting position and the number of actual cameras are limited by the visual angle and the image distortion condition of the camera, 8 camera modules are distributed in an 8-edge shape around the system, and each camera module is responsible for the image within a 60-degree visual angle. Therefore, for the requirement that the test object in the system is rectangular and the observation surface is required, 19 camera modules are arranged, and the number, the positions and the angles of the acquisition modules are comprehensively considered for the shapes and the sizes of other observation objects. Considering that the acquisition system of the invention needs to realize the communication of image transmission, the terminals of 19 camera modules are realized by selecting raspberry groups. The raspberry pie has small volume and low cost, has the function of image acquisition and the communication function, and can realize the control of the LED light source by controlling the potential of the GPIO port of the raspberry pie.
The system camera uses a digital camera. Because the transmission distance of the system is short, the digital camera can directly process the digital signals, and therefore faster image output can be realized under the condition of short-distance transmission.
The system light source selects an LED lamp. In the artificial light source, the LED lamp has the advantages of long service life, low cost, computer controllability and the like, and the strip-shaped LED lamp is selected according to the geometric characteristics of an object needing to acquire an image. When the shot object is placed parallel to the light source, the illumination intensity of the LED lamp reaching the surface of the object is uniform. For each surface of the object, the light receiving device receives three illumination angles of front light, back light and front oblique light. When the image is acquired, the optimal illumination condition needs to be determined through continuous adjustment and experiment so as to obtain the best imaging result.
The communication module of the system is realized by connecting a common server case of an upper computer and 19 raspberry pies as shown in fig. 3. Considering that the raspberry group can only realize communication through a USB or a network port, and the number of the raspberry groups used in the system is large, 4 3B version raspberry groups capable of communicating through the USB and the network port and 15 ZERO version raspberry groups capable of communicating only through the USB are selected for use in the system, and the relay function of communication between the ZERO version and the upper computer server is realized through the 3B version, so that the connection of a distributed system is realized.
As shown in fig. 4-5, the software portion of the image acquisition system is a distributed system.
(1) The communication control between each camera module and the upper computer is realized through a network bridge. Fig. 4 partially shows a schematic diagram of bridge connections between one raspberry pi 3B in a connection and 4 raspberry pi ZERO connected thereto. The raspberry pie is based on a Linux operating system, so that the configuration of the distributed system is completed in the Linux environment.
(2) The acquisition command of the image and the image transmission in the system are realized based on Redis, and the specific structure is shown in FIG. 5. The host computer maintains a message channel, 19 raspberries subscribe to the channel, and the host computer issues commands according to a communication protocol through the channel. After the light source condition, the camera needing to finish shooting and the number of the current shooting object are specified, all the raspberries receive the message from the channel and then read the instruction, and finish the realization of the corresponding light source condition and the acquisition of the image according to the specific requirement. The upper computer can acquire images from all the distributed terminals through a persistence mechanism of Redis. The invention can realize the cataloging storage of the collected images according to the serial numbers of the specific shot surfaces or edges and the shooting angles, can mark the information such as specific illumination conditions, shooting time, object serial numbers and the like, is convenient for subsequent preprocessing, and greatly improves the utilization efficiency of the collected images.
The method for realizing image acquisition and preprocessing by using the image acquisition system mainly comprises the following steps:
and 5, detecting the characteristic points of the processed images through a sift algorithm, matching the characteristic points, removing mismatching point pairs, and establishing a mathematical transformation model between the images, so that the images are spliced, and the complete image information of the collected object is obtained.
The method for specifically acquiring the target region ROI in
(1) and (3) completely building each module of the image acquisition system, and placing an object to be acquired in the clamping groove.
(2) And finishing the configuration of Redis, and sending an image acquisition command to a channel by the upper computer. The format of the issuing command agreed by the issuing/subscribing message mechanism is "# + the number of the LED lamp needing to be turned on + the raspberry pi number # needing to be subjected to image acquisition. The LED lamps and the raspberry pies are numbered in the collecting system box body, the numbers of the LED lamps are from A to L, and the numbers of the raspberry pies are from a to s.
(3) And according to the serial number in the command, the system automatically finishes the acquisition of the image according to the requirement, and uniformly names and stores the acquired object images in different categories according to different shooting angles and shooting positions.
The image cropping operation of the image acquisition system comprises the following steps:
and respectively positioning the target region ROI in the image acquired by each camera module.
The vertex coordinates of the target region ROI are extracted.
And cutting and storing the images acquired by the camera modules in batches according to the corresponding ROI coordinates.
The image registration and correction steps in
(1) and selecting one image as a reference image for the images acquired at all angles.
(2) Extracting feature points (such as SIFT features) of the image, and obtaining feature points of the outline, the contour and the ROI region in the image.
(3) And establishing the correlation of the matched characteristic points between each image and the reference image.
(4) And finishing the registration of the images according to the matched feature points to realize the image rectification.
The image brightness calibration step in
(1) and selecting one image as a reference image for the images acquired under the illumination conditions.
(2) The luminance value of the reference image is calculated.
(3) And by image enhancement, the brightness of all images acquired under corresponding illumination conditions is adjusted to be consistent by taking the brightness of the reference image as a reference.
The image background detection and filling part operation steps in
(1) the target object is not placed in the image acquisition system.
(2) And each camera module acquires a plurality of images from the background of the acquisition system under each specified illumination condition, realizes background modeling on each pixel point of the image and constructs a background model.
(3) And matching each pixel sampling value of the acquired image with the corresponding position of the background, realizing foreground detection and acquiring the acquired object and the background in the image.
(4) And carrying out image processing on the background to realize single color filling.
The multi-image stitching operation in step 5 is as shown in fig. 14:
(1) after the processing of the steps, the detection of the feature points is carried out by utilizing an SIFT algorithm aiming at the multi-view image acquired by a certain object under a certain specified light source condition.
(2) And matching the characteristic points extracted from the images collected at different angles, determining the position relation between the images, and acquiring the characteristic point pairs matched with each other between the images.
(3) And removing mismatching point pairs in the feature point pairs through RANSAC.
(4) And establishing a mathematical transformation model between the images according to the characteristic corresponding relation to complete coordinate transformation.
(5) The image areas to be spliced are fused, and the splicing of the images is completed, so that the complete image information of the collected object is obtained.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.