阅读说明：本技术 超声波诊断装置和方法以及图像处理装置和方法 (Ultrasonic diagnostic apparatus and method, and image processing apparatus and method ) 是由 陈颀 唐喆 简伟健 于 2018-07-31 设计创作，主要内容包括：一种能够使图像配准的操作变得容易的超声波诊断装置、超声波诊断方法、图像处理装置以及图像处理方法。本发明的超声波诊断装置具备：生成部,基于通过超声波的收发而得到的反射波信号来生成第一图像；取得部,取得第二图像,该第二图像是通过超声波诊断装置以外的医用图像诊断装置而生成的图像；配准部,对第一图像和第二图像进行配准；以及显示部,对配准后得到的图像进行显示,超声波诊断装置的特征在于,配准部在规定的范围内离散地设定多个第一图像与第二图像的相对位置,计算与多个相对位置分别对应的第一图像与第二图像的相似度,基于相似度的计算结果来更新多个相对位置,再次计算与更新后的多个相对位置分别对应的相似度。(An ultrasonic diagnostic apparatus, an ultrasonic diagnostic method, an image processing apparatus, and an image processing method are provided, which can facilitate the operation of image registration. An ultrasonic diagnostic apparatus of the present invention includes: a generation unit that generates a first image based on a reflected wave signal obtained by transmission and reception of ultrasonic waves; an acquisition unit configured to acquire a second image generated by a medical image diagnostic apparatus other than the ultrasonic diagnostic apparatus; a registration unit that registers the first image and the second image; and a display unit that displays the images obtained by the registration, wherein the registration unit discretely sets relative positions of the plurality of first images and the second images within a predetermined range, calculates a similarity between the first image and the second image corresponding to each of the plurality of relative positions, updates the plurality of relative positions based on a calculation result of the similarity, and recalculates the similarity corresponding to each of the plurality of updated relative positions.)

1. An ultrasonic diagnostic apparatus is provided with:

a generation unit that generates a first image based on a reflected wave signal obtained by transmission and reception of ultrasonic waves;

an acquisition unit that acquires a second image generated by the medical image diagnostic apparatus;

a registration unit configured to register the first image and the second image; and

a display unit for displaying the images obtained by the registration,

the ultrasonic diagnostic apparatus described above is characterized in that,

the registration unit discretely sets relative positions of the plurality of first images and the second image within a predetermined range, calculates a similarity between the first image and the second image corresponding to each of the plurality of relative positions, updates the plurality of relative positions based on a result of the calculation of the similarity, and recalculates the similarity corresponding to each of the plurality of updated relative positions.

2. The ultrasonic diagnostic apparatus according to claim 1,

the registration unit reduces the predetermined range in accordance with a region included in the first image or a purpose of diagnosis of the first image, and performs the registration using the reduced range.

3. The ultrasonic diagnostic apparatus according to claim 1,

the relative position is at least one of a rotation angle and a movement distance of the first image with respect to the second image.

4. The ultrasonic diagnostic apparatus according to claim 1,

the first image is any one of a two-dimensional image, a three-dimensional image, and a moving image, and the second image is any one of a three-dimensional image and a moving image.

5. The ultrasonic diagnostic apparatus according to any one of claims 1 to 4,

as a preprocessing for calculating the similarity, the registration unit performs enhancement processing of at least the structure depicted in the first image.

6. The ultrasonic diagnostic apparatus according to any one of claims 1 to 4,

the registration unit calculates an optimal similarity and an optimal relative position corresponding to the optimal similarity using one or more relative positions having a high similarity among the calculation results of the similarities,

the registration unit updates the plurality of relative positions using the calculation result of the similarity and the calculation result of the optimal similarity.

7. The ultrasonic diagnostic apparatus according to claim 6,

the registration unit calculates a similarity between the first image and the second image corresponding to the plurality of relative positions, respectively, using a global optimization algorithm,

the registration section calculates the optimal similarity and the optimal relative position using a local optimization algorithm,

the registration unit updates the plurality of positions of the second image using the global optimization algorithm.

8. The ultrasonic diagnostic apparatus according to claim 7,

the above global optimization algorithm is any one of a particle swarm algorithm and a genetic algorithm,

the local optimization algorithm is any one of a downhill simplex algorithm, a powell algorithm, a gradient descent algorithm, a conjugate gradient algorithm, a quasi-newton algorithm, and a levenberg-marquardt algorithm.

9. The ultrasonic diagnostic apparatus according to any one of claims 1 to 4,

the registration unit repeatedly executes a process of calculating the similarity between the first image and the second image corresponding to each of the plurality of relative positions, updating the plurality of relative positions based on the calculation result of the similarity, and calculating the similarity corresponding to each of the updated plurality of relative positions again until the similarity reaches a predetermined degree.

10. An ultrasonic diagnostic method includes:

a generation step of generating a first image based on a reflected wave signal obtained by transmission and reception of ultrasonic waves;

an acquisition step of acquiring a second image generated by a medical image diagnostic apparatus;

a registration step of registering the first image and the second image; and

a display step of displaying the images obtained after the registration,

the ultrasonic diagnostic method is characterized in that,

in the registration step, a similarity between the first image and the second image corresponding to each of the relative positions is calculated while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, the plurality of positions are updated based on the calculation result of the similarity, and the similarity is calculated again using the plurality of updated positions.

11. An image processing apparatus includes:

a first acquisition unit that acquires an ultrasound image as a first image;

a second acquisition unit configured to acquire a second image generated by the medical image diagnostic apparatus or the image processing apparatus; and

a registration unit that registers the first image and the second image,

the image processing apparatus as described above is characterized in that,

the registration unit calculates a similarity between the first image and the second image corresponding to each of the relative positions while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, updates the plurality of positions based on a calculation result of the similarity, and recalculates the similarity using the plurality of updated positions.

12. An image processing method includes:

a first acquisition step of acquiring an ultrasound image as a first image;

a second acquisition step of acquiring a second image generated by the medical image diagnosis apparatus or the image processing apparatus; and

a registration step of registering the first image and the second image,

the above-mentioned image processing method is characterized in that,

the registration step calculates a similarity between the first image and the second image corresponding to each of the relative positions while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, updates the plurality of positions based on a calculation result of the similarity, and recalculates the similarity using the plurality of updated positions.

Technical Field

The present invention relates to an ultrasonic diagnostic apparatus, an ultrasonic diagnostic method, an image processing apparatus, and an image processing method.

Background

In a conventional ultrasonic diagnostic apparatus or medical image processing apparatus, a doctor can be effectively assisted in recognizing a target disease such as a tumor by registering (registering) an ultrasonic image (ultrasound) with a medical image such as a CT (Computed Tomography) image or a MRI (Magnetic resonance imaging) image. Furthermore, because ultrasound images have the characteristics of real-time performance and convenience, image processing methods that fuse (fusion) ultrasound waves with MRI or CT images are popular.

In brief, registration between an ultrasound image and a CT/MRI image refers to aligning the position of the ultrasound image with the position of the CT/MRI image by searching for an accurate image Rotation (Rotation) amount and Translation (Translation) amount. Registration between ultrasound images and CT/MRI images is in fact a process of finding an optimal mapping of ultrasound image space to CT/MRI image space.

Generally, due to the limited scanning range Of an ultrasound scanning probe, the data Of an ultrasound image may be scanned from various scanning angles and only contain a small Field Of View (FOV), e.g. a partial volume Of an organ. Therefore, it is necessary to search the rotation amount and the translation amount of the image in a large search range to perform registration of the ultrasound image and the CT/MRI image.

As a technique for narrowing an image search range during registration, feature extraction is generally performed on two or more target images, preliminary registration is performed using the extracted features, and then further registration within a local range is performed on the target images by a local optimization algorithm.

Disclosure of Invention

In order to solve the above-described problems, an ultrasonic diagnostic apparatus, an ultrasonic diagnostic method, an image processing apparatus, and an image processing method capable of facilitating the operation of image registration have been proposed.

An ultrasonic diagnostic apparatus of the present invention includes: a generation unit that generates a first image based on a reflected wave signal obtained by transmission and reception of ultrasonic waves; an acquisition unit configured to acquire a second image generated by a medical image diagnostic apparatus; a registration unit configured to register the first image and the second image; and a display unit that displays the images obtained by the registration, wherein the registration unit discretely sets relative positions of the plurality of first images and the second images within a predetermined range, calculates a similarity between the first image and the second image corresponding to each of the plurality of relative positions, updates the plurality of relative positions based on a result of the calculation of the similarity, and recalculates the similarity corresponding to each of the plurality of updated relative positions.

An ultrasonic diagnostic method according to the present invention includes: a generation step of generating a first image based on a reflected wave signal obtained by transmission and reception of ultrasonic waves; an acquisition step of acquiring a second image generated by a medical image diagnostic apparatus; a registration step of registering the first image and the second image; and a display step of displaying the images obtained after the registration, wherein in the registration step, a similarity between the first image and the second image corresponding to each of the relative positions is calculated while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, the plurality of positions are updated based on a calculation result of the similarity, and the similarity is calculated again using the plurality of updated positions.

An image processing apparatus according to the present invention includes: a first acquisition unit that acquires an ultrasound image as a first image; a second acquisition unit configured to acquire a second image generated by the medical image diagnostic apparatus or the image processing apparatus; and a registration unit that registers the first image and the second image, wherein the registration unit calculates a similarity between the first image and the second image corresponding to each of the relative positions while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, updates the plurality of positions based on a calculation result of the similarity, and recalculates the similarity using the plurality of updated positions.

An image processing method according to the present invention includes: a first acquisition step of acquiring an ultrasound image as a first image; a second acquisition step of acquiring a second image generated by the medical image diagnosis apparatus or the image processing apparatus; and a registration step of registering the first image and the second image, wherein the registration step calculates a similarity between the first image and the second image corresponding to each of the relative positions while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, updates the plurality of positions based on a calculation result of the similarity, and recalculates the similarity using the plurality of updated positions.

Effects of the invention

In the ultrasonic diagnostic apparatus or method according to the present invention, relative positions of the plurality of first images and the plurality of second images are set discretely within a predetermined range, a similarity between the first image and the second image is calculated for each of the plurality of relative positions, and the plurality of relative positions are updated based on a calculation result of the similarity.

Therefore, compared with the conventional image registration method for performing preliminary registration through feature extraction, the method reduces the dependency on the extraction of features of blood vessels, surfaces and the like in the image, thereby reducing the requirements on the number of acquired ultrasonic images, the acquisition mode and the like and facilitating the image registration processing. Moreover, the condition that final registration fails due to low accuracy of feature extraction is avoided, and the accuracy of image registration is improved.

Drawings

Fig. 1 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to a first embodiment.

Fig. 2 is a flowchart showing image registration performed by the ultrasonic diagnostic apparatus according to the first embodiment.

Fig. 3 is a flowchart showing the registration optimization process in fig. 2.

Fig. 4 is a flowchart showing a modification of the registration optimization process in fig. 2.

Fig. 5 is a schematic diagram showing an example of registration between an ultrasound image and a CT image.

Fig. 6 is a schematic diagram showing an example of registration between an ultrasound image and an MRI image.

Fig. 7 is a flowchart showing image registration performed by the ultrasonic diagnostic apparatus according to the second embodiment.

Detailed Description

The following describes in detail an embodiment of an ultrasonic diagnostic apparatus according to the present invention with reference to the drawings. The embodiments described in the present invention are merely examples, and are not limited to the configurations described in the embodiments.

(first embodiment)

Fig. 1 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to a first embodiment of the present invention.

An ultrasonic diagnostic apparatus 10 of the present invention includes a generating unit 11, an acquiring unit 12, a registering unit 13, and a display unit 14.

The generator 11 obtains a reflected wave signal by transmitting and receiving an ultrasonic wave, and generates an ultrasonic image as a first image based on the reflected wave signal. More specifically, the ultrasonic diagnostic apparatus 10 further includes a probe (not shown) for transmitting and receiving ultrasonic waves, and the probe transmits the received reflected waves to the generating unit 11, so that the generating unit 11 generates an ultrasonic image. The ultrasound image in the present embodiment is a three-dimensional stereoscopic image.

The acquisition unit 12 acquires a second image. The second image is an image generated by a medical image diagnostic apparatus such as an ultrasonic diagnostic apparatus, a CT diagnostic apparatus, or an MRI diagnostic apparatus. The second image in the present embodiment will be described by taking a CT image generated by a CT apparatus or an MRI image generated by an MRI apparatus as an example. The acquisition unit 12 may acquire the second image from the medical image diagnosis apparatus via a network (internet or local area network), for example. The second image in the present embodiment is a three-dimensional stereoscopic image.

The registration unit 13 registers the first image generated by the generation unit 11 and the second image acquired by the acquisition unit 12. The specific operation of registration will be described in detail later.

The display unit 14 displays the registered image. The display mode can display a fused image of the first image and the second image, or can display the registered first image and the registered second image side by side.

Next, an image registration operation performed by the ultrasonic diagnostic apparatus 10 according to the first embodiment will be described with reference to fig. 2.

First, in step S101, the acquisition unit 12 of the ultrasonic diagnostic apparatus 10 acquires a CT image or an MRI image as a second image.

Then, in step S102, the operator scans a target position of the organ desired to be examined. In the scanning process, the user does not need to consider the scanning condition of the organ characteristics in the ultrasonic image, and only needs to scan the ultrasonic wave to the target position. For example, in the case where the target position of the organ to be examined is the entire liver or a certain region of the liver, the operator only needs to ensure that the target position is scanned, and does not need to ensure that the features of the liver, such as blood vessels and surfaces, are clearly scanned. In addition, the execution order of step S101 and step S102 may be interchanged.

Next, in step S103, the registration unit 13 performs enhancement processing on the first image and the second image. Specifically, the structure depicted in the first image and the second image is enhanced. The enhancement processing includes noise filtering processing, intensity and contrast enhancement processing, and blood vessel enhancement processing.

Next, in step S104, the registration section 13 performs preliminary registration of the first image and the second image.

As mentioned above, the first and second images are both three-dimensional stereo images, so registering the two requires rotation and/or translation of at least one of the images. The conversion parameters of the image include rotation parameters, translation parameters and the like. The rotation parameters include the rotation angle of the X-axis, the rotation angle of the Y-axis, and the rotation angle of the Z-axis. The translation parameters include a distance of movement along the X-axis, a distance of movement along the Y-axis, and a distance of movement along the Z-axis.

Since the optimization capability of the registration optimization algorithm in the subsequent step S106 is strong, the requirement of the preliminary registration in this step on the registration accuracy is low, and only the first image and the second image need to be simply registered.

A simple registration may align only the center points of the first and second images (in other words, center the ultrasound image at the center of the CT/MRI image), and set the initial values of the rotation angles of the X, Y, and Z axes to 0 °, and the initial values of the movement distances of the X, Y, and Z axes to 0 mm.

Further, the initial values of the conversion parameters (the rotation angle and the moving distance) may also be set to the average values. For example, the initial values of the respective conversion parameters are set based on the average value of the past registration results.

Next, in step S105, the registration unit 13 sets a search range for registration. In the present invention, since the precision of the preliminary registration in step S103 is low, the search range of the registration needs to be set to the global search range. The global search range means that the rotation angle is defined as a global angle range, and the movement distance is also defined as a global translation range.

The global search scope is defined, for example, as follows:

rotation of an X axis: minus 180 to plus 180 DEG

Rotation of the Y axis: minus 180 to plus 180 DEG

Rotation of the Z axis: minus 180 to plus 180 DEG

X-axis translation: -W/2- + W/2

Y-axis translation: -H/2- + H/2

Z-axis translation: -D/2- + D/2

Where W is the larger of the length of the first image in the X direction and the length of the second image in the X direction, H is the larger of the length of the first image in the Y direction and the length of the second image in the Y direction, and D is the larger of the length of the first image in the Z direction and the length of the second image in the Z direction. W, H and D are in units of pixels or millimeters, for example.

For example, assuming that W, H and D both equal 200mm, the above global search range is defined as follows:

rotation of an X axis: minus 180 to plus 180 DEG

Rotation of the Y axis: minus 180 to plus 180 DEG

Rotation of the Z axis: minus 180 to plus 180 DEG

X-axis translation: -100 mm- +100mm

Y-axis translation: -100 mm- +100mm

Z-axis translation: -100 mm- +100mm

Next, in step S106, the registration unit 13 performs registration optimization processing of the first image and the second image. Specifically, relative positions of a plurality of first images and a plurality of second images are set discretely within a predetermined range, the degree of similarity between the first image and the second image corresponding to each of the plurality of relative positions is calculated, the plurality of relative positions are updated based on the calculation result of the degree of similarity, and the degree of similarity corresponding to each of the plurality of updated relative positions is recalculated. When the similarity reaches a prescribed level or the relative position is not updated any more, the optimization processing ends.

The "similarity between the first image and the second image" may be calculated in various ways, and may be calculated based on a similarity measure such as a gradient value, a gray scale value, or an image correlation of the image. The "predetermined range" herein refers to the global search range set in step S105.

Next, in step S107, the display unit 14 displays the registration result. When the registration optimization process converges, the first image and/or the second image are searched for the optimal rotation angle and movement distance. These values are used as a transformation matrix (transformation parameters) to transform the CT/MRI image or the ultrasound image, and the image obtained after registration is displayed.

Next, the detailed operation of the registration optimization processing in step S106 will be described with reference to fig. 3.

In the registration optimization processing of the present embodiment, the algorithm of the objective function is a similarity calculation function that uses a metric value in which a gradient and a gradation are combined as optimization. The final goal of the optimization method is to find the maximum of the objective function.

In step S201, a plurality of relative positions of the first image and the second image are set discretely within a predetermined range.

Next, in step S202, the similarity between the first image and the second image corresponding to the plurality of relative positions is calculated.

Next, in step S203, the calculated plurality of similarities are sorted.

Then, in step S204, local optimization processing is performed using the N relative positions with high similarity among the plurality of similarities calculated in step S202. Here, N is 7, for example.

The local optimization processing in the present embodiment employs a Downhill simplex (Downhill simplex) algorithm. The downhill simplex algorithm is an algorithm that is optimized in a local small range with respect to input parameter data. A detailed description of the optimization process of the downhill simplex algorithm is omitted.

In step S204, the N relative positions with high similarity among the plurality of similarities are used as input parameter data of the downhill simplex algorithm to initialize the simplex, and the optimization result of the downhill simplex algorithm thus calculated becomes the current optimal similarity (population optimal value) among the plurality of relative positions and the optimal relative position corresponding to the optimal similarity.

Next, in step S205, any one of the plurality of relative positions is updated using the result of the local optimization processing. In other words, any one of the plurality of relative positions is replaced with the optimal relative position corresponding to the optimal similarity obtained in step S204.

Next, in step S206, the registration unit 13 updates each of the plurality of relative positions using the plurality of similarities calculated in step S202 and the optimal similarity calculated in step S205.

For example, for each of the plurality of relative positions, the relative position is optimized (updated) in consideration of a historical optimum value of the relative position itself and a current optimum value (i.e., a current optimum similarity) of the plurality of relative positions. Of course, the manner of optimizing (updating) the plurality of relative positions is not limited thereto.

Next, in step S207, it is determined whether or not the optimization result converges, in other words, whether or not the similarity reaches a predetermined degree. Each optimization method comprises an objective function, the function value is enabled to reach the maximum value, the objective function is the optimization objective, and when the larger objective value cannot be found through continuous optimization, the optimization result at the moment can be judged to be convergence. The phrase "the degree of similarity reaches a predetermined degree" herein means that the degree of similarity reaches a predetermined convergence degree. For example, the difference between the similarity calculated after the relative position is updated M-th and the similarity calculated after the relative position is updated M-1-th is smaller than a prescribed value (e.g., 10) ^-4) In the case of (3), it is assumed that the similarity reaches a predetermined convergence degree. Further, after the similarity is determined to have reached the predetermined degree, the relative positions of the first image and the second image are not updated.

If no in step S207, the process returns to step S202, and steps S202 to S206 are repeatedly executed. On the other hand, in the case where it is determined to be yes in step S207, the registration optimization processing ends.

The following describes the technical effects of the present embodiment.

In the present embodiment, step S201, step S202, step S203, step S206, and step S207 employ a global Optimization algorithm, i.e., a Particle Swarm Optimization (PSO). Step S204 adopts a local optimization algorithm, i.e., a downhill simplex algorithm.

In the Particle swarm algorithm, each relative position in the above embodiment is also referred to as a Particle (Particle), each Particle is an individual optimization point having properties such as rotation and translation, and its adaptive value is determined by an optimization function, and the direction and distance of movement are determined by the velocity. The particle swarm algorithm first initializes a group of random particles and then finds the optimal solution through iteration. In each iteration, each particle updates itself by tracking individual historical optima and the entire population to find a global optimum at the current time to find the optimal particle in the search space, and uses the rotation and/or translation of the optimal particle as the final transformation parameters.

In the present embodiment, one or more relative positions with high similarity calculated in the global optimization algorithm are substituted into the local optimization algorithm (step S203), and the obtained current optimal similarity is fed back to the global optimization algorithm (step S205). The speed of obtaining the optimal value by the registration optimization processing can be accelerated through the local optimization algorithm.

By combining the global optimization algorithm and the local optimization algorithm, the global optimization capability and the local optimization capability of the algorithm are improved, so that the first image and the second image only need to be simply registered before registration optimization processing is carried out.

Moreover, the mode of combining the global optimization algorithm and the local optimization algorithm can deal with a larger search range, and the condition that the registration result is inaccurate due to the small search range of the registration optimization processing can be avoided.

Moreover, the image registration method of the invention reduces the dependency of extracting the features of the blood vessels, the surfaces and the like in the image, so that in the process of ultrasonic scanning, a user does not need to consider the scanning condition of the organ features in the ultrasonic image, and only needs to scan the ultrasonic to the target position. Therefore, the requirements on the number of ultrasonic image acquisition, the acquisition mode and the like are reduced, and the convenience of ultrasonic scanning operation is improved.

In addition, the enhancement processing on the image in step S103 can improve the accuracy of the similarity calculation, and thus can improve the precision of the registration optimization processing in step S106. For example, it is advantageous to improve the accuracy of the optimization result by enhancing the gradient by the image enhancement processing using the gradient value when performing the similarity calculation of the first image and the second image in step S106.

Of course, other similarity measures besides gradient values or gray scale values may be used.

(modification of the first embodiment)

In the first embodiment, the global optimization algorithm employs a particle swarm algorithm, and the local optimization algorithm employs a downhill simplex algorithm, but is not limited thereto.

The global optimization algorithm may also use other algorithms such as a Genetic algorithm (Genetic algorithm), and the local optimization algorithm may also use any one of a Powell algorithm, a Gradient descent (Gradient device) algorithm, a Conjugate Gradient algorithm, a Quasi-Newton (Quasi-Newton) algorithm, and a Levenberg-Marquardt algorithm.

In the following modification of the first embodiment, an example is shown in which the global optimization algorithm uses a genetic algorithm and the local optimization algorithm uses a gradient descent algorithm. This modification differs from the first embodiment only in the registration optimization processing (step S106), and the following description focuses on the difference.

Fig. 4 is a flowchart showing a modification of the registration optimization process in fig. 2.

Among the steps shown in fig. 4, steps S301 to S303 and S305 are substantially the same as steps S201 to S203 and S205 in the first embodiment, and redundant description is omitted here.

In step S304, the 1 relative position with the highest similarity among the plurality of similarities is used as the input parameter data of the gradient descent algorithm, and the optimization result of the gradient descent algorithm calculated by this becomes the current optimal similarity among the plurality of relative positions and the optimal relative position corresponding to the optimal similarity. A detailed description of the calculation process of the gradient descent algorithm is omitted here.

Further, after step S305, selection (selection), intersection (Crossing), and Mutation (Mutation) are performed for the current plurality of relative positions based on the contents of the genetic algorithm, and then the plurality of relative positions obtained after the processing are updated.

Similarly, the above modification can also obtain the technical effects of the first embodiment.

Fig. 5 is a schematic diagram showing an example of registration between an ultrasound image and a CT image. Fig. 6 is a schematic diagram showing an example of registration between an ultrasound image and an MRI image.

Fig. 5 (a) shows the CT image acquired in step S101, (b) shows the ultrasound image acquired in step S102, and (c) shows the registration result in step S107.

When the registration optimization process converges, the CT/MRI image or the ultrasound image is transformed using the searched optimal rotation angle and movement distance as a transformation matrix (transformation parameters). In this case, only one of the images may be transformed, or both of the images may be transformed at the same time.

Fig. 5 (c) shows an example in which both images are transformed in order to observe the target region more clearly, that is, the region marked with x in (a). In fig. 5 (c), the two images are not displayed in a fused manner, but the first and second images after registration are displayed side by side. In addition, the CT image in (c) also displays a fan shape corresponding to the boundary of the upper ultrasound image, so that both can be observed in a more convenient manner.

Fig. 6 (a) shows the MRI image acquired in step S101, (b) shows the ultrasound image acquired in step S102, and (c) shows the registration result in step S107.

Fig. 6 (c) also shows an example in which both images are transformed in order to observe the target region more clearly, as in fig. 5. Fig. 6 (c) also shows a mode in which the registered first image and second image are displayed side by side. The MRI image in fig. 6 (c) also shows a fan shape corresponding to the boundary of the upper ultrasound image.

(second embodiment)

Next, a second embodiment of the present invention will be described.

In the second embodiment, the registration unit 13 reduces a predetermined range (global search range) according to a region included in the first image or the purpose of diagnosis of the first image, and performs registration using the reduced range. Also, the step of image enhancement is omitted in the second embodiment.

Fig. 7 is a flowchart showing image registration performed by the ultrasonic diagnostic apparatus 10 according to the second embodiment. Among the steps shown in fig. 7, the steps other than step S403 to step S405 are the same as those in the first embodiment, and overlapping description is omitted here.

In step S403, a target position is determined on the CT/MRI image. The process of determining the target position may be determined by an operator, for example, in a case where a tumor is displayed in a lower right partial region of the liver on the CT/MRI image, the operator determines the lower right partial region of the liver as the target region.

The operation of determining the target region may be automatically performed by a technique such as image recognition.

In step S404, preliminary registration is performed based on the decided target position. Unlike the simple registration in the first embodiment, the first image and the second image may be aligned with the target position or a position close to the target position as the center, and the initial values of the rotation angles of the X-axis, the Y-axis, and the Z-axis may be set to angles at which the target position is easily ultrasonically scanned. Because there is a high possibility that a clear image of the target position is obtained at an angle at which the ultrasonic scanning of the target position is easy.

In step S405, a search range for registration is set based on the target position. The global search range is narrowed in accordance with the target position. For example, some rotation angles and regions where the target range is difficult to clearly observe are removed.

In the case where a partial region at the lower right of the liver is determined as the target region, the global search range may be defined as follows, for example:

rotation of an X axis: minus 180 to plus 180 DEG

Rotation of the Y axis: minus 180 to plus 180 DEG

Rotation of the Z axis: minus 60 degrees to plus 180 degrees

X-axis translation: -W/2 to 0

Y-axis translation: -H/2- + H/2

Z-axis translation: -D/2 to 0

Where W is the larger of the length of the first image in the X direction and the length of the second image in the X direction, H is the larger of the length of the first image in the Y direction and the length of the second image in the Y direction, and D is the larger of the length of the first image in the Z direction and the length of the second image in the Z direction. W, H and D are in units of pixels or millimeters, for example.

In the second embodiment, the first image and the second image are preliminarily registered based on the target position, and the accuracy of the preliminary registration can be improved. Moreover, the search range of the registration optimization processing is narrowed based on the target position, so that the calculation amount can be effectively reduced, and the time required by the registration optimization processing can be reduced.

(other modification example)

While the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments may be implemented in various other ways, and various omissions, substitutions, and changes may be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

The concept of the present invention can also be applied to other image processing apparatuses. For example, an image processing apparatus includes: a first acquisition unit that acquires an ultrasound image as a first image; a second acquisition unit configured to acquire a second image generated by the medical image diagnostic apparatus or the image processing apparatus; and a registration unit that registers the first image and the second image, wherein the registration unit calculates a similarity between the first image and the second image corresponding to each of the relative positions while changing the relative position between the first image and the second image using a plurality of positions set discretely for the second image, updates the plurality of positions based on a calculation result of the similarity, and recalculates the similarity using the plurality of updated positions.

The ultrasonic diagnostic apparatus and the image processing apparatus according to the present invention may be incorporated in a medical device as a circuit capable of realizing the above-described functions, or may be distributed as a program executable by a computer or other electronic devices, stored in a storage medium such as a magnetic disk (floppy (registered trademark), hard disk, or the like), an optical disk (CD-ROM, DVD, or the like), a magneto-optical disk (MO), or a semiconductor memory, and executed via a processor of the computer or the electronic devices.

In the above embodiments, the description has been made on the case where both the first image and the second image are three-dimensional stereoscopic images, and actually, the first image may be any of a two-dimensional image, a three-dimensional image, and a moving image, and the second image may be any of a three-dimensional image and a moving image. Further, the type of the second image may be an ultrasound image generated by an ultrasound apparatus.

In step S103, the registration unit 13 performs enhancement processing on both the first image and the second image, and may actually perform enhancement processing only on the first image or the second image or not perform enhancement processing at all as necessary.

In each embodiment, the registration unit 13 repeatedly executes the patch registration optimization process until the degree of similarity reaches a predetermined degree. In fact, even if the registration optimization process of the global optimization algorithm and the local optimization algorithm combination of the present invention is performed only once, a faster and more accurate registration result than the prior art can be obtained, and the occurrence of inaccurate registration result due to a small search range of the registration optimization process can be avoided.

In the second embodiment, an example in which the search range is narrowed based on the target position is described. In addition, other anatomical structures such as ribs may be used to narrow the search.

For example, the search range for the rotation angle is-180 ° +180 °, usually without limitation, but if we have rib information, the search range for the rotation angle should follow the rib structure and the probe should be located between 2 ribs. In this case, the rotation angle at which the target region cannot be scanned is excluded, and the search range of the rotation angle may be set to a value smaller than 360 °. In addition, information of the ribs may be obtained through other images, and thus the action of limiting the search range based on the rib information may be automatically completed.