阅读说明：本技术 超声波摄像装置和使用该摄像装置的手术辅助系统及方法 (Ultrasonic imaging device, and surgery assistance system and method using same ) 是由 田中智彦 池田贞一郎 今井亮 竹岛启纯 于 2021-03-16 设计创作，主要内容包括：本发明涉及超声波摄像装置和使用该摄像装置的手术辅助系统及方法。提供能得到处于摄像范围外的器件的位置信息并将该位置信息与超声波图像一起提示给用户的超声波摄像装置以及手术辅助系统。手术辅助系统包含：超声波摄像装置；固定于要插入到被检测体内的治疗器具的超声波产生源；和显示超声波图像以及超声波产生源的位置的显示装置。超声波摄像装置具备推定超声波产生源的位置信息的位置推定部,位置推定部对在摄像区域外由超声波产生源发出的超声波产生的栅格伪影(假像)进行解析,来推定超声波产生源相对于摄像区域的位置信息。(The present invention relates to an ultrasonic imaging apparatus, and a surgery assistance system and method using the same. Provided are an ultrasonic imaging apparatus and a surgery assistance system, which can obtain position information of a device outside an imaging range and present the position information to a user together with an ultrasonic image. The operation assistance system includes: an ultrasonic imaging device; an ultrasonic wave generation source fixed to a therapeutic device to be inserted into a subject; and a display device for displaying the ultrasonic image and the position of the ultrasonic wave generation source. The ultrasonic imaging apparatus includes a position estimation unit that estimates positional information of an ultrasonic wave generation source, and the position estimation unit analyzes a grid artifact (false image) generated by an ultrasonic wave emitted from the ultrasonic wave generation source outside an imaging region to estimate positional information of the ultrasonic wave generation source with respect to the imaging region.)

1. An ultrasonic imaging apparatus is characterized by comprising:

a transmission unit that transmits an ultrasonic signal, which has been formed into a beam, to a subject via an ultrasonic probe;

a receiving unit that performs beam forming and receives reflected ultrasonic waves from an imaging region to which the ultrasonic signal is irradiated;

an image forming unit that creates an ultrasonic image of the imaging region based on the reflected ultrasonic wave received by the receiving unit; and

a position estimating unit that estimates position information of an ultrasonic wave generation source inserted into the subject using the ultrasonic wave generated by the ultrasonic wave generation source received by the receiving unit at the time of non-transmission of the ultrasonic signal by the transmitting unit,

the position estimating unit analyzes an artifact generated by the ultrasonic wave emitted from the ultrasonic wave generation source outside the imaging region, and estimates position information of the ultrasonic wave generation source with respect to the imaging region.

2. The ultrasonic imaging apparatus according to claim 1,

the position information includes any one of a depth of the ultrasonic wave generation source, a distance to the imaging region, and an arrival direction to the imaging region.

3. The ultrasonic imaging apparatus according to claim 1,

the position estimating unit analyzes the inclination of the artifact, and estimates the arrival direction of the ultrasonic wave generation source using the inclination of the artifact.

4. The ultrasonic imaging apparatus according to claim 1,

the position estimating unit includes a recognizer that learns a relationship between the artifact and position information of the ultrasonic wave generation source.

5. The ultrasonic imaging apparatus according to claim 1,

the ultrasonic wave generation source includes:

a piezoelectric element for generating ultrasonic waves by applying a voltage,

the transmission unit transmits an electric signal to the piezoelectric element when the ultrasonic signal is not transmitted.

6. The ultrasonic imaging apparatus according to claim 1,

the ultrasonic wave generation source includes:

a photoacoustic element that generates an ultrasonic wave by irradiation of light; and

a light generation unit that irradiates light to the photoacoustic element and transmits a trigger signal to the ultrasonic imaging apparatus,

the ultrasonic imaging apparatus further includes:

and a control unit that receives the trigger signal from the light generation unit and controls the transmission and reception of the ultrasonic waves by the transmission unit and the reception unit.

7. The ultrasonic imaging apparatus according to claim 1,

the ultrasonic imaging apparatus further includes:

and a control unit configured to control a phase of beam forming by the reception unit so that beam forming is performed according to the estimated position information, using the position information of the ultrasonic wave generation source estimated by the position estimation unit.

8. The ultrasonic imaging apparatus according to claim 7,

the position estimating unit re-estimates the position information of the ultrasonic wave generation source using the ultrasonic wave signal received by the receiving unit after the control of the control unit.

9. The ultrasonic imaging apparatus according to claim 7,

the position estimating unit creates an image of the ultrasonic wave generation source using the ultrasonic wave signal received by the receiving unit after the control of the control unit, and displays the image on a display device.

10. The ultrasonic imaging apparatus according to claim 1,

the ultrasonic imaging apparatus further includes:

and a display image forming unit configured to form a display image indicating the position information of the ultrasonic wave generation source estimated by the position estimating unit.

11. The ultrasonic imaging apparatus according to claim 10,

the display image forming unit forms a display image including an ultrasonic image of an imaging region and the positional information on the same screen.

12. A surgical assistance method for monitoring a surgical instrument inserted into a body of a patient using an ultrasonic imaging device,

repeatedly transmitting and receiving ultrasonic signals in a predetermined imaging range to form and display an ultrasonic image,

generating ultrasonic waves from an ultrasonic wave generation source fixed to the surgical instrument at an interval of ultrasonic wave transmission and reception, imaging an ultrasonic wave signal received by the ultrasonic imaging device, and analyzing an artifact of the ultrasonic wave generation source outside an imaging range,

and estimating and presenting positional information of the ultrasonic wave generation source with respect to the imaging range.

13. A surgical assistance method according to claim 12,

information including at least one of a direction in which the ultrasonic wave generation source approaches the imaging range, a distance from the imaging range, and a distance in a depth direction of an ultrasonic beam is presented as the positional information together with the ultrasonic image of the imaging range.

14. A surgical assistance method according to claim 12,

after estimating the positional information of the ultrasonic wave generation source, changing the phase of beam formation when receiving an ultrasonic wave signal so that the ultrasonic wave generation source is included in an imaging range, and receiving the ultrasonic wave signal from the ultrasonic wave generation source,

an image created based on the received ultrasonic signal is presented.

15. A surgical assistance system, comprising:

an ultrasonic imaging apparatus including an imaging unit that transmits an ultrasonic signal beam-formed to a subject via an ultrasonic probe, receives a reflected ultrasonic signal from a region irradiated with the ultrasonic signal, and images an ultrasonic image of the subject;

an ultrasonic wave generation source fixed to a therapeutic device to be inserted into a subject; and

a display device that displays the ultrasonic image and a position of the ultrasonic wave generation source,

the ultrasonic imaging apparatus includes:

a position estimating unit that receives an ultrasonic wave from an ultrasonic wave generation source inserted into the subject via the ultrasonic probe and estimates position information of the ultrasonic wave generation source,

the position estimating unit analyzes an artifact generated by the ultrasonic wave emitted from the ultrasonic wave generation source outside the imaging region, and estimates position information of the ultrasonic wave generation source with respect to the imaging region.

Technical Field

The present invention relates to a surgical support technique combining a photoacoustic technique and an ultrasonic imaging apparatus, and more particularly to a technique of inserting a surgical instrument such as a catheter into a patient and monitoring the movement of the surgical instrument when the surgical instrument is moved to a surgical target site.

Background

In the operation for treating the angiostenosis, catheter treatment which is less burdened on the patient than the open chest operation is widely used. Visual recognition is an example of a problem in performing surgery in an extremely small area such as a catheter. A small-diameter device such as a guide wire is mainly visually recognized by using X-ray fluoroscopy, but the positional relationship between an affected part of a living body and the device is unclear. Particularly in intravascular treatment, it is important to know the positional relationship between the blood vessel wall and the guide wire, which is directly related to the surgical performance, and to achieve high visual recognition. Although there are some attempts to solve this problem by using an ultrasonic imaging apparatus during surgery, it is not easy to trace the distal end of the guide wire with ultrasonic waves, and a skilled echographer separate from the operator is required, and therefore, it has not become widespread.

In view of this, the following techniques are also proposed: an ultrasonic transmitter is attached to the distal end of the catheter, and the distal end of the catheter is detected by receiving ultrasonic waves from the ultrasonic transmitter and imaging the ultrasonic waves in an ultrasonic imaging device. For example, patent document 1 discloses a technique disclosed as follows: the position of the ultrasonic transmitter is detected while maintaining a state directly below the ultrasonic probe, based on the posture detected by the posture detecting unit attached to the ultrasonic probe.

Documents of the prior art

Patent document

Patent document 1: JP 2015-139465

In the technique of combining the photoacoustic technique and the ultrasonic imaging apparatus to grasp the position of the device such as patent document 1, the device tip provided with the photoacoustic means is visualized when it enters the imaging range. Therefore, it is not possible to know from which direction and how much the device approaches before the device enters the imaging range. That is, it is not possible to predict where the device will appear in the ultrasonic image of the affected part until the device approaches the affected part. In addition, in general, a blood vessel has many branches, and in catheter treatment, it is necessary to advance a device to a predetermined affected part while passing through these branches, but in the conventional method, even if the advancing direction of the device is deviated halfway, it is impossible to grasp the situation from an image.

Disclosure of Invention

The present invention addresses the problem of providing an ultrasonic imaging apparatus that can obtain positional information of a device outside an imaging range and present the positional information to a user together with an ultrasonic image.

In order to solve the above problem, an ultrasonic imaging apparatus according to the present invention uses a grid artifact (false image) generated by an ultrasonic wave from outside an imaging range, and thereby grasps position information when an ultrasonic wave generation source attached to a device is outside the imaging range.

That is, an ultrasonic imaging apparatus according to the present invention includes: a transmission unit that transmits an ultrasonic signal, which has been formed into a beam, to a subject via an ultrasonic probe; a receiving unit that performs beam forming and receives reflected ultrasonic waves from an imaging region to which the ultrasonic signal is irradiated; an image forming unit that creates an ultrasonic image of the imaging region based on the reflected ultrasonic wave received by the receiving unit; and a position estimation unit that estimates position information of an ultrasonic wave generation source inserted into the subject using the ultrasonic wave received by the reception unit at the time of non-transmission of the ultrasonic signal by the transmission unit, wherein the position estimation unit analyzes an artifact generated by the ultrasonic wave generation source outside the imaging region, and estimates position information of the ultrasonic wave generation source with respect to the imaging region.

The surgical assistance system of the present invention includes: the ultrasonic imaging apparatus having the above-described configuration; an ultrasonic wave generation source fixed to a therapeutic device to be inserted into a subject; and a display device that displays the ultrasonic image and a position of the ultrasonic wave generation source.

In addition, a surgical assistance method according to the present invention is a surgical assistance method for monitoring a surgical instrument inserted into a body of a patient using an ultrasonic imaging device, the surgical assistance method including repeatedly transmitting and receiving an ultrasonic signal to and from a predetermined imaging range, forming an ultrasonic image and displaying the image, generating an ultrasonic wave from an ultrasonic wave generation source fixed to the surgical instrument at an interval between the transmission and reception of the ultrasonic wave, imaging the ultrasonic signal received by the ultrasonic imaging device, analyzing an artifact of the ultrasonic wave generation source outside the imaging range, estimating positional information of the ultrasonic wave generation source with respect to the imaging range, and presenting the information.

Effects of the invention

An ultrasonic imaging apparatus performs imaging under beam forming conditions in which a predetermined region is an imaging region, that is, a predetermined region is subjected to phase adjustment, and in the ultrasonic imaging apparatus, ultrasonic waves emitted from outside the imaging region are imaged as artifacts similar to grating (hereinafter, referred to as GL) artifacts, and the appearance of the GL artifacts changes depending on the positional relationship with the imaging region. In the present invention, the relationship between the ultrasonic wave generation source, which is the generation source, and the imaging range can be estimated by analyzing the grid (hereinafter referred to as GL) artifact, and particularly, the arrival direction to the imaging range can be estimated. By presenting the arrival direction together with the ultrasonic image, which is an image of the imaging region, the user can know whether or not the ultrasonic wave generation source is correctly traveling to the imaging region reflected as an image, and can know from which direction the ultrasonic wave generation source is traveling to the target site, and can predict the travel of the ultrasonic wave generation source.

Drawings

Fig. 1 is a diagram showing an outline of a surgical support system according to an embodiment.

Fig. 2 is a diagram illustrating a flow of operations of the surgical support system according to the embodiment.

Fig. 3 is a diagram showing the overall configuration of the ultrasonic imaging apparatus according to embodiment 1.

Fig. 4 is a sectional view showing the structure of the front end of the device.

Fig. 5 is a diagram for explaining the operation of the ultrasonic imaging apparatus according to embodiment 1.

Fig. 6 is a diagram illustrating GL artifacts.

Fig. 7(a) and (B) are views each showing an example of a display screen of the ultrasonic imaging apparatus according to embodiment 1.

Fig. 8 is a diagram showing the overall configuration of an ultrasonic imaging apparatus according to embodiment 2.

Fig. 9 is a diagram for explaining the operation of the ultrasonic imaging apparatus according to embodiment 2.

Fig. 10 is a diagram illustrating an operation of a modification.

Fig. 11 is a diagram showing a display example of a modification.

Description of the reference numerals

1: operation auxiliary system

10: ultrasonic imaging apparatus

11: transmitting part

12: receiving part

13: input unit

14: display unit

20: ultrasonic wave generating source

25: light generating unit (light generating device)

40: display device

150: signal processing unit

151: ultrasonic image forming unit

152: position estimating unit

153: display image generating unit

155: control unit

Detailed Description

Embodiments of an ultrasonic imaging apparatus and a surgical support system including the ultrasonic imaging apparatus according to the present invention will be described below.

As shown in fig. 1, the operation support system 1 includes: an ultrasonic imaging device 10 connected to the ultrasonic probe 3; an ultrasonic wave generation source 20 fixed to the device 2; and a display device 40 for displaying an image or the like formed by the ultrasonic imaging device 10. In the present specification, a therapeutic device such as a balloon catheter or a microcatheter, or a surgical device to be inserted into a body such as a guide wire for transporting the therapeutic device to a target site is collectively referred to as a device.

The operation support system 1 is an apparatus as follows: after the device 2 is inserted into the body (blood vessel) of the subject (patient) 30 and reaches the affected part, for example, in catheter treatment for performing treatment such as dilation of a stenotic part of the blood vessel and removal of thrombus, assistance is performed so that the device 2 can reliably reach the affected part such as the stenotic part. Therefore, ultrasonic waves are generated at regular intervals from the ultrasonic wave generation source 20 fixed to the device 2, and the ultrasonic waves are detected by the ultrasonic imaging apparatus 10 to estimate the position of the ultrasonic wave generation source 20. The ultrasonic imaging apparatus 10 also transmits ultrasonic waves to the subject 30 and receives reflected ultrasonic waves when the ultrasonic wave is not emitted from the ultrasonic wave generation source 20, and acquires an ultrasonic image of the inside of the subject.

The ultrasonic wave generation source 20 includes an element that generates ultrasonic waves by applying energy such as light or electricity. Specifically, a photoacoustic material such as a piezoelectric element that emits ultrasonic waves by vibrating with a high-frequency signal, or a dye that emits ultrasonic waves by receiving light can be used. The ultrasonic wave generation source 20 is connected to a device that drives the ultrasonic wave generation source at a substantial portion of the device 2. In the case where the ultrasonic wave generation source is a photoacoustic material, the light generation device 25 is connected to an optical fiber. In the case of the piezoelectric element, a dedicated power supply can be used, but the piezoelectric element can be driven by a signal from a transmission unit of the ultrasonic imaging apparatus 10 by being connected to the transmission unit.

The ultrasonic imaging apparatus 10 can use an apparatus having the same configuration as that of a conventional ultrasonic imaging apparatus, but the signal processing system thereof includes a means for estimating the position of the ultrasonic wave generation source 20 (fig. 3: position estimation unit 152). However, a signal processing device or a computer different from the ultrasonic imaging device 10 may perform a part or all of the functions of the position estimating unit. In this case, the ultrasonic imaging apparatus 10 includes an interface unit for exchanging data with such a signal processing apparatus or a computer. Data can be exchanged by a known means such as exchange via wire, wireless, network, or medium.

An ultrasonic image of the inside of a subject can be obtained by transmitting an ultrasonic beam, forming and receiving reflected ultrasonic waves reflected at various positions inside the subject, and determining an imaging range based on the beam formation and the irradiation angle of the ultrasonic beam. On the other hand, the ultrasonic wave emitted from the ultrasonic wave generation source 20 spreads in a wave shape regardless of the imaging range, and reaches the ultrasonic probe. When beamforming matching the beamforming at the time of transmission is performed at the time of reception, if the ultrasonic wave generation source 20 is within the imaging range, an image of the shape of the ultrasonic wave generation source 20 can be obtained from the ultrasonic wave emitted therefrom. If the ultrasonic wave generation source 20 is in a dot shape, a dot-shaped image can be obtained. Conventionally, the positional relationship between the affected part and the device in the real space is grasped from the position of the image of the ultrasonic wave generation source 20 in the imaging range. However, when the ultrasonic wave generation source 20 is outside the imaging range, even if the ultrasonic wave from the ultrasonic wave generation source reaches the receiving portion, it is not possible to draw the image as the shape of the ultrasonic wave generation source 20.

In ultrasonic imaging, a phenomenon such as grid artifact is known. The grating lobe is a virtual component that spreads laterally to the central axis with respect to the original ultrasonic beam, and an image formed by a reflected wave from the grating lobe is a GL artifact. The GL artifact is premised on the presence of a grating lobe, but even if the ultrasound is spontaneous, which is not a reflected wave, it appears as an image when the conditions for generating GL are satisfied. Conditions for generating GL are determined by the arrangement interval d of the transducers constituting the ultrasonic probe and the wavelength λ, (sin θ is nd/λ, n: 1, 2, …, θ is the direction of the grating lobe) in a predetermined relationship. The artifact appearing at this time is an artifact generated according to the same principle as that of GL, and is therefore referred to as a GL artifact in the present specification.

In the mode of representing the GL artifact of the spontaneous ultrasonic wave, the generation position and inclination thereof are changed depending on the position of the ultrasonic wave generation source. In the ultrasonic imaging apparatus 10 according to the present embodiment, the position estimating unit 152 analyzes the GL artifact to estimate the position of the ultrasonic wave generation source outside the imaging range.

Therefore, in the operation support system 1, as shown in fig. 2, the ultrasonic imaging device 10 is controlled so as to operate in 2 operation modes (S21). One mode is an imaging mode for acquiring an ultrasonic image of the subject, and the other mode is a device monitoring mode (hereinafter, simply referred to as a monitoring mode) for receiving the spontaneous ultrasonic wave from the ultrasonic wave generation source 20 and forming an image. In the imaging mode, ultrasound is transmitted and received to and from the imaging region under the set imaging conditions (S22), an ultrasound image such as a B-mode image is created (S23), and the ultrasound image is displayed on the display device 40 (S24). In the monitoring mode, transmission in the imaging mode is set to non-Operation (OFF), and ultrasonic waves are generated from the ultrasonic wave generation source 20 (S25). The ultrasonic imaging apparatus 10 continues reception even when ultrasonic waves are generated (S26), and creates an image.

The position estimating unit 152 analyzes GL artifacts appearing in an image created by the ultrasound from the ultrasound generating source 20 (S27), calculates the positional information, inclination, and the like thereof, and displays the analysis result on the display device 40 (S28). The mode of the analysis result displayed on the display device 40 is not particularly limited, and may be, for example, a mode in which the positional relationship with respect to the imaging region can be grasped together with the ultrasonic image displayed in step S23.

The switching between the imaging mode and the monitoring mode (S21) may be performed by the control unit in the ultrasound imaging apparatus 10, or may be performed by transmitting a trigger signal from the ultrasound generation source 20 side to the ultrasound imaging apparatus 10 and using the trigger signal. While the operator is continuously advancing the device 2 in the body (blood vessel) of the subject, the imaging mode and the monitoring mode of the ultrasonic imaging apparatus 10 are alternately repeated, and the image of the imaging region including the affected part and the device position information are displayed on the display apparatus 40, whereby the operator can confirm the state in which the device is continuously advancing in the imaging region.

Based on the above outline of the operation support system, a specific embodiment of the device position estimation will be described below. In the drawings referred to in the following embodiments, elements having the same function are denoted by the same reference numerals, and redundant description is omitted.

< embodiment 1>

An embodiment in which a photoacoustic material is used as the ultrasonic wave generation source 20 will be described.

Fig. 3 shows the overall configuration of the operation support system 1 and the ultrasonic imaging apparatus 10 according to the present embodiment.

The ultrasound imaging apparatus 10 has a configuration similar to that of a general ultrasound imaging apparatus, except that a function (position estimating unit 152) for estimating the position of an ultrasound generation source is added, and includes, as shown in fig. 3: a transmission unit 11 for transmitting an ultrasonic signal to the ultrasonic probe 3; a receiving unit 12 that receives the reflected wave (RF signal) detected by the ultrasonic probe 3 and performs processing such as phase adjustment and addition; and a signal processing unit 150 for processing the RF signal received by the receiving unit 12. The ultrasonic imaging apparatus 10 further includes: a control unit 155 for controlling the entire apparatus and its accessories; an input unit 13 for a user to input conditions and instructions required for image capturing; a display unit 14 (fig. 1: display device 40) for displaying an ultrasonic image acquired by the ultrasonic imaging device 10, a GUI (graphical User Interface), and the like; and a memory 16 for storing an image or the like as a result of processing by the signal processing section 150.

The signal processing unit 150 includes: an ultrasonic image forming unit 151 for creating an ultrasonic image using the signal received by the receiving unit 12; a position estimating unit 152 that estimates information on the position of the ultrasonic wave generating source 20 using an ultrasonic signal transmitted from the ultrasonic wave generating source 20 and received by the receiving unit 12 via the ultrasonic probe 3; and a display image forming unit 153 for generating an image to be displayed on the display unit 14. Here, when it is necessary to distinguish between an ultrasonic signal that is a reflected wave of a transmitted ultrasonic beam among the ultrasonic signals received by the receiving unit 12 and an ultrasonic signal from the ultrasonic wave generation source 20, the former is referred to as an image signal and the latter is referred to as a monitor signal.

The ultrasonic image forming unit 151 creates an image of the ultrasonic wave generating source 20 (referred to as a monitoring image) using the monitoring signal in addition to an ultrasonic image of the subject such as a B-mode image. The position estimating unit 152 estimates the position and direction (collectively referred to as position information) of the ultrasound wave generation source 20 using the monitoring image.

Some or all of the functions of each unit and the control unit 155 constituting the signal processing unit 150 can be realized by uploading software for programming the functions to a computer having a CPU, a GPU, and a memory. In the example shown in fig. 3, the functions of the signal processing unit 150 and the control unit 155 are collectively shown as one processing unit 15. However, a part or all of the functions of each unit may be realized by hardware such as an electronic circuit, an ASIC, and an FPGA. Further, a part of the functions of the processing unit 15 may be realized by a computer independent from the ultrasonic imaging apparatus 10.

The ultrasonic probe 3 can be any of various ultrasonic probes 3 such as a 1D array probe in which a large number of transducer elements are arrayed in a one-dimensional direction, a 1D3 array probe having a 2 to 3-column array in a direction orthogonal to the array direction of the 1D array probe, and a 2D array probe having a large number of array arrays in a two-dimensional direction.

The ultrasonic wave generation source 20 is made of a material that receives laser light and thermally insulates and expands to emit ultrasonic waves such as photoacoustic signals, for example, a known dye (photosensitizer), metal nanoparticles, carbon-based compound, or the like. As shown in fig. 4, for example, the ultrasonic wave generation source 20 has a structure in which the photoacoustic material 21 described above is fixed to an insertion-side end face of an optical fiber 22 disposed in a hollow portion of a flexible hollow device 2 (for example, a guide wire) and is covered with a resinous sealing member 23. A light generating unit (light generating means) 25 that generates laser light is connected to the other end (end on the opposite side to the end to which the ultrasonic wave generating source 20 is fixed) of the optical fiber 22, and the photoacoustic material 21 is irradiated with the laser light from the light generating unit 25 through the optical fiber 22, whereby a photoacoustic signal, i.e., an ultrasonic wave, is generated from the photoacoustic material 21.

The light generating unit 25 sends a trigger signal to the ultrasonic imaging apparatus 10 while emitting laser light. The ultrasonic imaging apparatus 10 (control unit 155) sets the operation of the transmission unit 11 to a standby state for a certain period of time by the trigger signal, and operates in the monitoring mode. The time for operating in the monitoring mode may be predetermined or may be set by the user via the input unit 13. The time may not be constant, and may be variable such that the time is gradually shortened according to the traveling state of the device.

Next, the operation of the operation support system 1 having such a configuration will be described with reference to the flow of fig. 5.

First, when an imaging region is set for an affected part as an imaging target and imaging is started, the control unit 155 controls transmission and reception in the imaging mode without receiving a trigger signal from the light generation unit 25. That is, the ultrasonic waves are transmitted and received to and from the imaging region, and an ultrasonic image of the imaging region is created.

Upon receiving the trigger signal from the light generation unit 25, the control unit 155 of the ultrasonic imaging apparatus 10 switches the operation mode to the monitoring mode, and sets the operation of the transmission unit 11 to the standby state (S50). On the other hand, the light generating section 25 generates laser light at a given frequency together with the emission of the trigger signal (S51). When the photoacoustic material is irradiated with the laser light through the optical fiber to generate the ultrasonic wave from the ultrasonic wave generating unit 20 (S52), the ultrasonic wave propagates as a spherical wave in the subject, and is detected by the transducer elements of the ultrasonic probe 3 (S53). The receiving unit 12 performs beamforming (phase addition) on the signals detected by the transducer elements, and passes the signals as frame data to the ultrasonic image creating unit 151. The ultrasonic image generator 151 forms an image (monitor image) in the same manner as the generation of the B-mode image in the imaging mode (S54), and passes the image to the position estimator 152.

The position estimating unit 152 estimates the position and direction of the ultrasound source 20 from the monitoring image received from the ultrasound image creating unit 151 (S55). Here, when the ultrasonic wave generation source 20 deviates from the imaging range determined by the transmission/reception conditions, the original image of the ultrasonic wave generation source 20 is not shown in the monitoring image. However, if the angle of the ultrasonic wave reaching the transducer elements of the ultrasonic probe 2 from the ultrasonic wave generation source 20 is the same as the angle at which the grating lobes are generated, the artifacts are generated in the same manner as the grating lobe artifacts. As a result, for example, a GL artifact 611 as shown in fig. 6 appears in the imaging region 600. The lower diagram of fig. 6 is an enlarged diagram of a part (artifact portion) of the upper diagram. Here, assuming that the arrangement direction of the transducer elements is the θ direction and the depth direction is the r direction, the GL artifact 611 is higher on the side closer to the ultrasonic wave generation source 20 in the θ direction and has a slope that becomes lower as it becomes farther away. This is because, as a result of the beamforming by the receiving unit 12, even if the depth is the same, the near portion is drawn shallowly and the far portion is drawn deeply. It is possible to estimate from the inclination of the monitor image which direction the ultrasonic wave generation source 20 approaches with respect to the imaging region, that is, the arrival direction.

Upon receiving the monitoring image from the ultrasound image generator 151, the position estimator 152 calculates the inclination of the artifact. The inclination need not be calculated as a strict angle as long as the arrival direction of the device is known, and for example, the inclination of the region can be determined by extracting a region where an artifact occurs (a region having a luminance value equal to or greater than a threshold value) from the luminance value of the image, and obtaining the coordinates (depth) of both end portions in the θ direction or the coordinates (position in the θ direction) of both end portions in the r direction. The gradient may be obtained as a feature amount by using a recognizer that performs feature extraction from the artifact image or a region from which the artifact image is extracted by machine learning.

Furthermore, the arrival direction and position may be estimated using a machine learning model such as CNN (convolutional neural network). The machine learning model is a model that uses a large amount of input data and a set of teaching data to learn a correct answer to a new input, and for example, a model that uses B-mode images having different positions of ultrasonic wave generation sources to learn a relationship between an artifact and a position may be used. In this case, the position information may include not only the arrival direction but also the distance to the imaging region and the depth information.

In the example shown in fig. 6, the GL artifact appears in a shape with a high right side and a low left side, and the ultrasonic wave generation source 20 is estimated to come from the right side in the θ direction with respect to the imaging region 500.

When the position estimating unit 152 estimates the arrival direction of the device 2 as described above, the result is sent to the display image generating unit 153.

On the other hand, when a predetermined time has elapsed since the trigger signal, the light generation unit 25 stops the generation of the laser light, and the control unit 155 switches the operation mode to the imaging mode. In the imaging mode, as in the normal imaging, an ultrasonic beam is irradiated to the subject via the ultrasonic probe 3, and an image signal as a reflected wave is received by the receiving unit 12. The ultrasonic image generator 151 receives data for a plurality of frames from the receiver 12, generates a B-mode image, and sends the B-mode image to the display image generator 153.

The display image generator 153 generates an image to be displayed on the display 14 using the information on the arrival direction of the device 2 received from the position estimator 152 and the ultrasonic image received from the ultrasonic image generator 151, and displays the image on the display 14 (S56, S57). The display mode can be various types, but it is preferable to display the positional information on a screen showing the ultrasonic image in a superimposed manner.

A display example is shown in fig. 7. Fig. 7(a) displays a mark 71 showing an arrow indicating the arrival direction of the ultrasonic wave generation source 20 on a screen 700 showing the imaging region 70 in a superimposed manner. Thus, the operator can recognize at a glance where the device has traveled from the mapped blood vessel. In fig. 7(B), regions 72A and 72B for indicating directions are provided on both sides of the screen on which the imaging region is projected, and for example, when the device is traveling from the left side of the region, the left region 72A is made bright to indicate the direction of arrival. In this case, the mark 71 shown in fig. 7(a) may be displayed together. Further, although not shown, when not only the arrival direction of the ultrasonic wave generation source 20 but also the position is estimated, the positions may be indicated by marks such as dots in the regions 72A and 72B.

In the above description, the operation when the ultrasonic wave generation source 20 is outside the imaging region is described, but when the device tip enters the imaging region while the imaging mode and the monitoring mode are alternately executed, the ultrasonic wave generation source 20 is drawn in the imaging region as a bright point, for example. Therefore, the position of the ultrasound wave generation source 20 can be calculated from the bright point, and a marker or the like indicating the distal end of the device is displayed at a corresponding position of the ultrasound image acquired in the imaging mode, whereby the positional relationship between the device and the blood vessel to be treated by the catheter can be visually shown.

According to the present embodiment, even when the device distal end is outside the imaging region, the arrival direction can be estimated from the GL artifact of the ultrasonic wave emitted from the ultrasonic wave generation source 20 and presented. Thus, the operator can confirm on the ultrasonic image which direction the distal end of the device under insertion is going to the affected part, or whether the distal end of the device under insertion is going to the affected part accurately and practically.

< embodiment 2>

In the present embodiment, a piezoelectric element is used as the ultrasonic wave generation source 20. The configuration of the ultrasonic imaging apparatus 10 is the same as that of embodiment 1, but in this embodiment, the light generating unit 25 for operating the ultrasonic wave generating source 20 is not necessary, and the piezoelectric element operates upon receiving an electric signal from the transmitting unit 11 of the ultrasonic imaging apparatus 10. The present embodiment will be described below focusing on differences from embodiment 1. Fig. 8 shows a configuration of the operation support system according to the present embodiment.

The piezoelectric element 20A vibrates by applying an electric signal (voltage) in the same manner as the transducer of the ultrasonic probe 3, and can generate an ultrasonic wave identical to the ultrasonic wave generated by the ultrasonic probe 3 by controlling the frequency of the signal. The piezoelectric element 20A can be fixed to the device distal end using a known material such as piezoelectric ceramics. Further, a catheter or the like in which a piezoelectric element is mounted as an actuator or a sensor on the tip of a device can be obtained, and the device can be mounted using such a piezoelectric element. The piezoelectric element 20A at the front end of the device is connected to a power supply by a wire passing through the catheter. A dedicated power supply may be used as the power supply, but in the example shown in fig. 8, the piezoelectric element is driven by a signal from the transmission unit 11 by being connected to the transmission unit 11 of the ultrasonic imaging apparatus 10.

Next, the operation of the ultrasonic imaging apparatus 10 having such a configuration will be described with reference to the flow of fig. 9. Here, a case where the control unit 155 switches the operation mode of the ultrasonic imaging apparatus 10 will be described as an example. In fig. 9, the same processes as those in fig. 5 are denoted by the same reference numerals, and redundant description is omitted.

In the monitoring mode, the control unit 155 operates the transmission unit 11 to apply a voltage to the piezoelectric element 20A in a pulse shape at a predetermined repetition frequency, for example (S501). Thereby, ultrasonic waves are generated in pulses from the piezoelectric element 20A (S502). The ultrasonic wave generated from the piezoelectric element 20A is set to a time for which data of at least 1 frame can be obtained. The ultrasound from the piezoelectric element 20A is processed by the receiving unit 12 in the same manner as the ultrasound received by the ultrasound probe 3, and a B-mode image is formed in the ultrasound image forming unit 151 (S53, S54). The position estimating unit 152 estimates the position and direction of the piezoelectric element 20A using the B-mode image (S55). The position estimating unit 152 estimates the arrival direction of the piezoelectric element 20A, as in embodiment 1, by determining from which side of the imaging region the device 2 is approaching, based on the angle of the GL artifact appearing in the image. When the piezoelectric element 20A enters the imaging region, the piezoelectric element 20A is drawn as a bright spot in the B-mode image, for example, and therefore the position thereof on the image can be determined.

On the other hand, when the reception of the ultrasonic wave emitted from the piezoelectric element 20A is completed in the reception unit 12, the control unit 155 switches the operation of the transmission unit 11, and transmits a signal for driving each transducer of the ultrasonic probe 3 to irradiate the ultrasonic beam formed into a beam to the subject. The receiving unit 12 receives the RF signal as the reflected wave of the ultrasonic beam, performs beam forming and addition, and sends the resultant to the ultrasonic image forming unit 151 (S50).

Then, the display image forming unit 153 superimposes the position information (the arrival direction when outside the imaging region and the position when inside the imaging region) from the position estimating unit 152 on the ultrasonic image formed by the ultrasonic image forming unit 151 to create a display image, and displays the display image on the display unit 14 (S56, S57).

According to the present embodiment, as in embodiment 1, when the device 2 is outside the imaging region, the arrival direction thereof can be estimated and presented to the operator with good visual recognition. In addition, according to the present embodiment, by using the piezoelectric element as the ultrasonic wave generation source, the light generation device (light generation unit 25) for emitting light from the photoacoustic material can be eliminated, and in particular, the system configuration can be simplified by driving it by the transmission unit of the ultrasonic imaging device.

< modification 1>

The above-described embodiment estimates the arrival direction and position of the ultrasonic wave generation source, and when the ultrasonic wave generation source is outside the imaging region, the arrival direction is displayed so as to be visually recognized as shown in fig. 7, but the reception condition may be changed so that the ultrasonic wave generation source outside the imaging region is the imaging region, and imaging may be performed.

Fig. 10 shows an example of a control flow of the control unit when the present modification is executed. In fig. 10, the processes (S21 to S28) having the same contents as those in fig. 2 are denoted by the same reference numerals, and redundant description thereof is omitted. In the present modification, after the superimposed image of the ultrasound image and the position information is displayed in step S28, the control unit 155 determines whether or not to perform imaging for confirming the position of the ultrasound generation source 20(20A) (S29), and if it is determined that it is necessary, performs the following control: the phase adjustment condition of the receiving unit 12 is changed so that the beam forming is performed at the estimated position of the ultrasonic wave generation source 20 (S30), and the monitoring mode is operated.

The determination at step S30 may be performed in response to a user instruction of "whether or not confirmation of the position of the ultrasound wave generation source is necessary" received via the input unit 13 after the superimposed image is displayed, or may be determined based on artifacts used for estimation and the reliability of the estimated positional information. For example, the GL artifact varies in inclination depending on the position of the ultrasound wave generation source as described above, but also varies when the depth of the ultrasound wave generation source increases. In such a case, the reliability of the position information estimated from the gradient of the GL artifact is low. In addition, since GL artifacts are unclear, reliability may be reduced. For such reliability, for example, in the case where CNN is used for position estimation, since the confidence levels of the output position information are combined and output, such reliability is obtained by using the confidence levels. The reliability may be calculated from the luminance value of the GL artifact, the distribution thereof, and the like. For example, if the luminance value (the maximum value or the addition value) does not satisfy a predetermined threshold, it is determined that the reliability is low.

The display image generation unit 153 displays the image obtained by changing the phase adjustment condition in step S30 on the display unit 14. For example, as shown in fig. 11, an image 601 of the imaging region used for position estimation and an image 602 obtained by phase adjustment at the estimated position may be displayed in parallel, or may be displayed together with the ultrasound image of the imaging region. This process S29, S30 may be repeated several times until an image of the ultrasonic wave generation source 20 is obtained. The operator can confirm the estimated position and the arrival direction by such display.

As described above, according to the present invention, when catheter treatment is performed, the catheter progress condition, which is the most important information for the operator, can be presented to the operator in real time with good visual recognition. Further, although the configuration and operation of the ultrasonic imaging apparatus and the operation support system according to the present invention have been described, the present invention includes elements described in the embodiments, in which elements having the same functions are substituted with elements having the same functions, and other elements are added.