阅读说明：本技术 系统控制器、网络系统以及网络系统中的方法 (System controller, network system and method in network system ) 是由 V·S·拉那图加 于 2018-12-20 设计创作，主要内容包括：【问题】为了高效地将数据分发到多个分发目的地。【解决方案】根据本公开,提供了一种系统控制器,该系统控制器控制将从发送侧的设备接收到的数据分发到接收侧的设备的IP交换机。系统控制器在IP交换机处构建多个虚拟网络,并且将由虚拟网络中的任何虚拟网络接收到的数据发送到与虚拟网络连接的每个分发目的地。根据这种配置,数据被高效地分发到多个分发目的地。(To efficiently distribute data to a plurality of distribution destinations. According to the present disclosure, there is provided a system controller that controls an IP switch that distributes data received from a device on a transmitting side to a device on a receiving side. The system controller constructs a plurality of virtual networks at the IP switch, and transmits data received by any of the virtual networks to each distribution destination connected to the virtual networks. According to this configuration, data is efficiently distributed to a plurality of distribution destinations.)

1. A system controller that controls an IP switch that distributes data received from a device on a transmission side to a device on a reception side,

the system controller constructs a plurality of virtual networks in the IP switch, and causes data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

2. The system controller of claim 1, wherein the IP switch comprises a plurality of switches layered and the virtual network is built among the plurality of switches.

3. The system controller of claim 2, wherein any two switches of the plurality of switches are connected to each other by a plurality of links.

4. The system controller according to claim 2, wherein each of the distribution destinations to which data received by any of the virtual networks is transmitted is a device on a reception side connected to one of switches in which the any of the virtual networks is built, or is another one of switches connected to the one of switches in which the any of the virtual networks is built.

5. The system controller according to claim 3, further comprising a load distribution unit that performs load distribution of data flowing between the plurality of switches based on a table specified from an application side.

6. The system controller according to claim 5, wherein the load distribution unit performs the load distribution for each of the data flows.

7. The system controller according to claim 2, further comprising a display control unit that controls display of a network topology based on information on data flows in each of the switches and information on data flows of devices connected to the IP switch.

8. The system controller of claim 1, wherein the data is audio data or video data.

9. A network system, comprising:

an IP switch that distributes data received from a device on a transmitting side to a device on a receiving side; and

a system controller controlling the IP switch, wherein,

the system controller constructs a plurality of virtual networks in the IP switch, and causes data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

10. The network system of claim 9, wherein the IP switch comprises a plurality of switches layered, and the virtual network is built among the plurality of switches.

11. The network system of claim 10, wherein any two switches of the plurality of switches are connected to each other by a plurality of links.

12. The network system according to claim 10, wherein each of the distribution destinations to which data received by any of the virtual networks is transmitted is a device on a reception side connected to one of switches in which the any of the virtual networks is built, or is another one of switches connected to the one of switches in which the any of the virtual networks is built.

13. The network system according to claim 11, wherein the system controller includes a load distribution unit that performs load distribution of data flowing between the plurality of switches based on a table specified from an application side.

14. The network system according to claim 13, wherein the load distribution unit performs the load distribution for each of the data flows.

15. The network system according to claim 10, wherein the system controller includes a display control unit that controls display of a network topology based on information on data flows in each of the switches and information on data flows of devices connected to the IP switch.

16. The network system according to claim 9, wherein the data is audio data or video data.

17. A method in a network system for controlling an IP switch that distributes data received from a device on a transmission side to a device on a reception side, the method comprising:

a plurality of virtual networks are built in the IP switch, and data received by any of the virtual networks is transmitted to respective distribution destinations connected to the virtual networks.

Technical Field

The present disclosure relates to a system controller, a network system and a method in a network system.

Background

The following patent document 1 conventionally describes a method and system for controlling a hybrid network including a software-defined network (SDN) switch and a legacy (legacy) switch.

Reference list

Patent document

Patent document 1: japanese patent application laid-open (laid-open) No. 2016-521529

Disclosure of Invention

Problems to be solved by the invention

However, as described in the above-mentioned patent document 1, it is difficult to design a conventional network switch with a high degree of freedom, and it is difficult to efficiently distribute data to a plurality of distribution destinations.

Therefore, it is desirable to efficiently distribute data to a plurality of distribution destinations.

Solution to the problem

According to the present disclosure, there is provided a system controller that controls an IP switch that distributes data received from a device on a transmission side to a device on a reception side, the system controller building a plurality of virtual networks in the IP switch and causing data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

Further, according to the present disclosure, there is provided a network system including an IP switch that distributes data received from a device on a transmission side to a device on a reception side; and a system controller that controls the IP switch, wherein the system controller constructs a plurality of virtual networks in the IP switch, and causes data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

Further, according to the present disclosure, there is provided a method in a network system for controlling an IP switch that distributes data received from a device on a transmission side to a device on a reception side, the method including constructing a plurality of virtual networks in the IP switch and causing data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

ADVANTAGEOUS EFFECTS OF INVENTION

As described above, according to the present disclosure, data can be efficiently distributed to a plurality of distribution destinations.

Note that the above-described effects are not necessarily restrictive, and any effect described in this specification or other effects that can be grasped from this specification may be exhibited in addition to or instead of the above-described effects.

Drawings

Fig. 1 is a schematic diagram illustrating a schematic configuration of a system according to an embodiment of the present disclosure.

Fig. 2 is a schematic diagram illustrating a specific application example of the present disclosure.

Fig. 3 is a schematic diagram illustrating a specific application example of the present disclosure.

Fig. 4 is a schematic diagram illustrating general unicast, broadcast, and multicast.

Fig. 5 is a schematic diagram illustrating a configuration of an SDN controller and its periphery.

Fig. 6 is a schematic diagram for explaining the problem of broadcast distribution in a multilink environment.

Fig. 7 is a schematic diagram for explaining the problem in the ridge configuration in detail.

Fig. 8 is a schematic diagram illustrating the configuration of an IP switch according to the present embodiment.

Fig. 9 is a schematic diagram illustrating broadcast communication in a multilink environment according to the present embodiment.

Fig. 10 is a schematic diagram illustrating a state in which an AV device is connected to a leaf switch of an IP switch.

Fig. 11 is a schematic diagram illustrating a state in which an AV device is connected to a leaf switch of an IP switch.

Fig. 12 is a diagram illustrating a specific example of AV stream load distribution.

Fig. 13 is a schematic diagram illustrating information created by an SDN controller to visualize a network topology.

Fig. 14 is a diagram schematically illustrating the overall configuration of the operating room system.

Fig. 15 is a diagram illustrating a display example of an operation screen on the centralized operation panel.

Fig. 16 is a diagram illustrating an example of a state of an operation to which the operating room system is applied.

Fig. 17 is a block diagram illustrating an example of functional configurations of the camera head and CCU shown in fig. 16.

Detailed Description

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that in this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and duplicate description will be omitted.

Note that the description will be given in the following order.

1. Summary of the disclosure

2. Application examples of the present disclosure

Configuration example of SDN controller

4. Flexible network design using SDN

4.1. Problem of broadcast distribution in a multilink environment

4.2. Broadcast distribution in a multilink environment according to the present embodiment

AV stream load distribution method

5. Visualization of network topology

6. Application example

1. Summary of the disclosure

First, a schematic configuration of a system 1000 according to an embodiment of the present disclosure will be described with reference to fig. 1. As shown in fig. 1, the system 1000 includes a plurality of AV apparatuses 100, an IP switch 200, a system manager 400, and an SDN controller (system controller) 500. In fig. 1, AV stream data and control data are transmitted and received between the AV apparatus 100 and the IP switch 200. Further, control data is transmitted and received between the IP switch 200 and the system manager 400 or the SDN controller 500. In fig. 1, AV traffic (traffic) is represented by thick lines and control data traffic is represented by thin lines between the AV device 100 and the IP switch 200 and between the IP switch 200 and the system manager 400 or the SDN controller 500. The system manager 400 is a device that controls and switches the AV device 100.

The plurality of AV apparatuses 100 are, for example, apparatuses such as IP converters. For example, a device such as a camera is connected as the AV source 180 on the transmission side to the AV device 100 on the transmission side (corresponding to the transmitter in fig. 1). Further, an apparatus such as a display is connected as an AV destination 190 on the reception side to the AV apparatus 100 on the reception side (corresponding to the receiver in fig. 1). In the case where the AV apparatus 100 is an IP converter, the AV apparatus 100 performs interface conversion between the AV source 180 or the AV destination 190 and the IP switch 200. In this example, video from multiple cameras is provided to multiple displays via the IP switch 200.

In the system 1000 of the present embodiment, the flexible network design is performed by using a Software Defined Network (SDN). SDN is an architecture in which the network is centrally managed by a network controller (SDN controller 500) and resources can be dynamically allocated. As a result, application-dependent flexible traffic load distribution can be implemented for existing networks without being limited by switch functionality or protocols.

Further, in the system 1000 of the present embodiment, the network topology is visualized, and an improvement in the operation management is achieved by the flow-based visualization.

2. Application examples of the present disclosure

As an example, the system 1000 of the present disclosure assumes that audio and video are provided for IP transport (i.e., for AV transport). In the system 1000, IP transmission is performed between the AV apparatuses 100. As the AV source 180, an image acquisition and/or acquisition image transmission apparatus, a monitoring camera, and the like, such as a camera, an AV streaming (streaming) server, an apparatus of a television startup system, and a camera of a video conference system, may be assumed. Further, as the AV destination 190, a PC display, a large-sized display in a video conference or the like, a projector, a recording server, or the like can be assumed.

In the system 1000, it is assumed that AV streaming is switched by using IP, and that an AV signal is converted into IP and transmitted over a network. Further, the AV signal may also be compressed before being converted into IP. The AV signal may be converted into IP and transmitted as RTP stream, or another transmission method may be used. It is also assumed that the transmission is performed as multicast to improve the efficiency of the AV signal.

Fig. 2 and 3 are schematic diagrams illustrating specific application examples of the present disclosure. Fig. 2 illustrates a system 1000 for a broadcast station. In the system 1000 of fig. 2, the AV source 180 is a device such as a camera, an AV production switcher (production switcher), and a streaming server. Further, the AV destination 190 is a device such as various displays and storage devices. In the example shown in fig. 2, a case is assumed where the AV apparatus 100 is incorporated into the AV source 180 or the AV destination 190.

Fig. 3 illustrates a system 1000 for an operator. In the system 1000 of fig. 3, the AV source 180 is a device such as a PC screen and a streaming server. Further, the AV destination 190 is a device such as various displays and storage devices. In the example shown in fig. 3, the AV apparatus 100 is configured separately from the AV source 180 or the AV destination 190.

Fig. 4 is a schematic diagram illustrating general unicast, broadcast, and multicast. In unicast, one AV source 180 sends to one AV destination 190. In a broadcast, one AV source 180 sends to all other AV destinations 190. In multicasting, one AV source 180 (or multiple AV sources 180) sends to multiple AV destinations 190.

Configuration example of SDN controller

Fig. 5 is a schematic diagram illustrating a configuration of the SDN controller 500 and its periphery. As shown in fig. 2, the SDN controller 500 includes a load distribution unit 502, a broadcast unit 504, and a flow display control unit 506.

Together with the SDN controller 500, a system controller application 550 is included. The application 550 operates on the SDN controller 500. The application 550 includes a load distribution setting unit 552 and a monitoring application 554 as its functional configuration. Further, the application 550 maintains device flow information 556. Each component of the SDN controller 500 and the application 550 can be constituted by a central processing unit such as a CPU included in the SDN controller 500 and a program (software) for realizing the function.

The above-described flexible network design is performed by setting a flow entry list (flow entry list) of the IP switch 200 so that the broadcasting unit 504 of the SDN controller 500 transmits a broadcast packet in a virtual network described later. Further, for example, the user performs setting from the load distribution setting unit 552 of the application 550, and the load distribution unit 502 of the SDN controller 500 sets the IP switch 200, thereby realizing multicast load distribution.

Further, the flow display control unit 506 of the SDN controller 500 controls display in the monitoring application 554 of the application 550 based on information obtained from the IP switch 200 and flow information obtained from the AV device 100 and the like, thereby performing visualization of the network topology described above. Note that the display for this display may be integrated with the SDN controller 500, or may be a separate body.

4. Flexible network design using SDN

4.1. Problem of broadcast distribution in a multilink environment

Fig. 6 is a schematic diagram for explaining a problem of broadcast distribution in a multi-link environment of a medium-scale to large-scale network. In the existing system, there is a case where broadcast communication is performed between the system controller 10 and the IP converter 20. Fig. 6 illustrates a configuration (left diagram) using a large-scale rack-type (chassis type) switch as the system controller 10 and a spine-leaf configuration (right diagram) using a plurality of IP switches 30, 32, and 34.

In the case of using a large-scale rack-type switch as shown in fig. 6, there is a problem that the network cost is expensive. Further, in the case of using a plurality of IP switches 30, 32, and 34, in the existing network such as IGMP, transmission data is transmitted to the IP switch 30 of the upper layer and then distributed to the IP switches 32 and 34 of the lower layer. However, since the IP switches 30, 32, and 34 are connected together by only one link and a plurality of links cannot be configured, there is a problem that multicast and broadcast cannot be compatible with each other in a plurality of links.

Fig. 7 is a schematic diagram for explaining the problem in the ridge configuration in detail. In fig. 7, the IP switches include a spine switch 30 and leaf switches 32 and 34. In the case of distributing data of video as described above, in the leaf switch 32, the VLAN is divided for load distribution of video, and VLAN1 and VLAN 2 are provided. In this case, for example, even if the system controller 40 connected to VLAN 2 attempts to communicate by broadcasting, the system controller 40 cannot communicate with the IP converter 20 of another VLAN1 (marked with x in fig. 7) because the VLAN is divided into VLAN1 and VLAN 2. Therefore, in the case where communication is performed by broadcasting, a ridge configuration used in a general medium-scale to large-scale configuration cannot be used.

As noted above, the spine configuration is impractical and the system is expensive when large-scale rack-mounted switches are used.

4.2. Broadcast distribution in a multilink environment according to the present embodiment

Fig. 8 is a schematic diagram illustrating the configuration of the IP switch 200 according to the present embodiment. In the present embodiment, the switch is made to have multiple stages, and cost reduction is achieved by load distribution by a plurality of links. SDN eliminates the limitations on switch functionality and protocols, enabling application-dependent flexible traffic load distribution. Furthermore, the unique configuration of the SDN controller 500 enables broadcast and multicast communications even over multiple links between switches.

When comparing the right diagram of fig. 6 with fig. 8, in the right diagram of fig. 6, in the case where the capacity of one uplink between the spine and the leaf is 40 giga (G), the number of IP converters 20 of each of the leaf switches 32 and 34 is four on the transmission side (Tx) and four on the reception side (Rx). However, the capacity of each IP converter 20 is 10G.

On the other hand, in the case of fig. 8, in the case where the capacity of each of three uplinks between the spine and the leaf is 40 giga (G), the number of AV devices 101 (corresponding to the IP converter 20 in fig. 6) of each leaf is 12 on the transmission side (Tx) and 12 on the reception side (Rx). Thus, the number of IP converters per leaf increases from 8 to 48. Further, since it can be realized by a multi-stage switch without using a large-scale rack switch as shown in the left diagram of fig. 6, the cost of hardware can be reduced.

Hereinafter, the present embodiment will be described specifically. In the present embodiment, the broadcast packet is transmitted in the IP switch 200 by constructing a virtual network in a network topology in which a plurality of links exist between switches.

Fig. 9 is a schematic diagram illustrating broadcast communication in a multilink environment according to the present embodiment. As shown in fig. 9, IP switch 200 includes SDN switches 210, 220, and 230 arranged in multiple stages (two stages in fig. 9). Note that in the case where the IP switch 200 includes SDN switches arranged in multiple stages, for example, two SDN switches are connected in the lower layer of the SDN switch 220 shown in fig. 9, and a plurality of AV devices are connected in the lower layer of each of the two SDN switches.

SDN switch 210 and SDN switch 220 are connected to each other by two uplinks (Up links) 240 and 242. Further, the SDN switch 210 and the SDN switch 230 are connected to each other through two uplink (Up Link)250 and 252.

In SDN switch 220, two virtual networks 222 and 224 corresponding to two uplinks 240 and 242 are constructed. Similarly, in SDN switch 230, two virtual networks 232 and 234 are constructed corresponding to two uplinks 250 and 252. In addition, a virtual network 212 is built in the SDN switch 210.

As described above, in the present embodiment, a virtual network is defined for each uplink, and endpoints are separated for each uplink. Then, flow control is performed so that flooding of the broadcast packet is performed in the virtual network.

In fig. 9, the port of the SDN switch 220 to which the AV device 120 is connected is set to port "1", and the port of the SDN switch 220 to which the AV device 122 is connected is set to port "2". Further, the port of the SDN switch 220 to which the AV device 124 is connected is set to port "3", and the port of the SDN switch 220 to which the AV device 126 is connected is set to port "4".

Further, in fig. 9, the port of SDN switch 220 to which uplink 240 is connected is set to port "5", and the port of SDN switch 220 to which uplink 242 is connected is set to port "6". At this time, the flow entry list of the SDN switch 220 is defined as follows.

IF Broadcast Packet AND In_port＝Port1 Output＝Port2,Port5

IF Broadcast Packet AND In_port＝Port2 Output＝Port1,Port5

IF Broadcast Packet AND In_port＝Port5 Output＝Port1,Port2

IF Broadcast Packet AND In_port＝Port3 Output＝Port4,Port6

IF Broadcast Packet AND In_port＝Port4 Output＝Port3,Port6

IF Broadcast Packet AND In_port＝Port6 Output＝Port3,Port4

Similar settings are made in SDN switch 210 and SDN switch 230. As described above, for example, according to the first line of the above-described flow entry list, a broadcast packet (represented by a solid arrow a1 in fig. 9) sent from AV device 120 to SDN switch 220 enters SDN switch 220 from port "1" of SDN switch 220 and is output from port "2" and port "5" and sent to SDN switch 210 and IP converter 122. In the SDN switch 210 and the SDN switch 230, similar flows are performed, whereby broadcast packets sent from the AV device 120 to the SDN switch 220 are distributed to the AV devices 122, 124, 126, 128, 130, 132, and 134 in flow broadcasts of arrows indicated by dotted lines. Note that in this example, AV device 120 may be, for example, an AV source such as a camera, and AV devices 122, 124, 126, 128, 130, 132, and 134 may be AV destinations such as displays. The data to be broadcast distributed may be control data or an AV stream. Further, in a case where the control data is broadcast-distributed in a case where the AV devices 122, 124, 126, 128, 130, 132, and 134 are devices such as displays, the AV devices 122, 124, 126, 128, 130, 132, and 134 may be devices on the transmission side. Further, the AV device 120 on the transmission side may be a device such as the system manager 400 or the SDN controller 500.

The broadcast distribution as described above can be implemented by setting the flow entry list of the IP switches (SDN switches 210, 220, and 230) based on the virtual network by the broadcast unit 504 of the SDN controller 500. The IP switches (SDN switches 210, 220, and 230) set up a path based on the flow entry list. As a result, free path design by the SDN controller 500 becomes possible.

AV stream load distribution method

Next, an AV stream load distribution method according to the present embodiment will be described. Fig. 10 and 11 are schematic diagrams for explaining an AV stream load distribution method according to the present embodiment. Fig. 10 and 11 are diagrams illustrating a state in which the AV devices 140, 142, 144, and 146 are connected to the leaf switch 260 of the IP switch 200.

Fig. 10 is a schematic diagram illustrating load distribution for each interface. In the example shown in fig. 10, data transmitted from the AV devices 140 and 142 flows through links connected to the virtual network 262, and is transmitted to the upper-level switch. Further, the data transmitted from the AV devices 144 and 146 flow through the link connected to the virtual network 264 and are transmitted to the upper-level switch. By dividing the virtual networks 262 and 264 for each of the AV devices 140, 142, 144, and 146, dynamic changes can be made as the situation stands, as compared to a conventional VLAN system. Further, fig. 11 is a schematic diagram illustrating load distribution for each flow. As shown in fig. 11, data transmitted from AV devices 140, 142, 144, and 146 is transmitted to different virtual networks 262 and 264 in the leaf switch 260 for each individual stream. As a result, load distribution can be performed for each flow. Note that reference numeral 262 shown in fig. 10 may be a switch, and a network including the switch and the AV devices 140 and 142 may be used as a virtual network. Similarly, reference numeral 264 shown in fig. 10 may be a switch, and a network including the switch and the AV devices 144 and 146 may be used as a virtual network. This also applies to fig. 11.

As shown in fig. 10 and 11, load distribution of AV streams is explicitly performed on the basis of a UI or a setting file. Load distribution is performed based on the size of the flow and the bandwidth of the uplink and relay (trunk) ports. Load distribution for the downlink is performed on a host basis.

Fig. 12 is a diagram illustrating a specific example of AV stream load distribution. The configuration of the IP switch 200 is similar to that of fig. 9. As described above, multicast load distribution is assigned to the SDN controller 500 by the application 550. The load distribution specified by the application 550 is represented in the table at the lower part of fig. 12. The table can also be specified by a user operating the UI of the application 550.

In fig. 12, the ID of the IP switch 220 is set to "1", the ID of the IP switch 230 is set to "2", and the ID of the IP switch 210 is set to "3". Further, the IP switches 210, 220, and 230 each include ports connected to the AV devices 150, 152, 154, and 156. Further, the IDs of the AV apparatuses 150, 152, 154, and 156 are set to "1", "2", "3", and "4", respectively. In the table shown in the lower part of fig. 12, in the network interface list (network interface list), for each device connected to the IP switch, such as AV devices 150, 152, 154, and 156, the IP address (ipAddress) of the AV device of each ID, the ID (peerswitch ID) and the port number (peerswitch port) of the connected IP switch, and the like are defined. Further, in the streamer (streamer) list (streamerList), an ID of the stream (streamer), a streamer type (Sender or Receiver), a bandwidth (bandwidth), a multicast destination address (mcastAddres), an ID of a connected IP switch (switchid), an output port (switchPort), and the like are defined.

In AV stream load distribution, the load distribution unit 502 of the SDN controller 500 performs path setting based on the table shown in fig. 12. For example, in the case of an AV stream whose ID is 1 transmitted from the AV apparatus 150, path setting is performed as follows.

If In_port＝Port1 AND Multicast_Address＝224.0.0.1Output＝Port49

If In_port＝Port2 AND Multicast_Address＝224.0.0.2Output＝Port50

Path setting is performed based on the size of the flow and the bandwidth of the uplink and trunk ports. Each of the IP switches 210, 220, and 230 identifies information in the table of fig. 12 and performs load distribution. Accordingly, the load distribution unit 502 of the SDN controller 500 performs path setting based on the table shown in fig. 12, thereby being able to distribute the load of the AV stream.

5. Visualization of network topology

Next, visualization of the network topology will be described. In existing networks, monitoring cannot be performed on a data flow basis (e.g., multicast-based). On the other hand, by using the SDN, the flow information can be managed by the IP switch 200, so that a data flow monitoring function can be realized. In the present embodiment, the network topology between AV devices is visualized by using the SDN and centrally managing the network with the SDN controller 400.

The SDN controller 500 can obtain data flow statistics from the IP switch 200. That is, the SDN controller 500 can acquire data flow information from leaf switches and spine switches constituting the IP switch 200.

Also, the SDN controller 500 can acquire connection information from the terminal device (each AV device) and flow information output by each terminal device. The flow display control unit 506 of the SDN controller 500 performs control for displaying flow information between terminal devices based on data flow information obtained from the IP switch and flow information output from each AV device obtained from the AV device.

Further, the flow display control unit 506 performs control for separating and displaying flow information for each network traffic. For example, a separate display is performed for each of the multicast traffic list, the unicast traffic list, and the broadcast traffic list.

Also, the UI information is displayed by referring to the path information and the transmission table of the IP switch 200, and the path is determined and displayed by viewing the traffic actually flowing through the IP switch 200.

Fig. 13 is a schematic diagram illustrating display information created by the flow display control unit 506 of the SDN controller 500 to visualize a network topology. The information shown in fig. 13 is displayed on a display or the like in accordance with the flow of data of the actual traffic based on the path information. The SDN controller 500 can create the information shown in fig. 13 by grasping the flow of data flows in the IP switch 200 and acquiring flow information output by the terminal device.

As shown in fig. 13, the system configuration is displayed on the left side. A state in which the AV devices 150, 152, and 154 are connected to each other via the IP switch 200 and the SDN controller 500 is connected to the IP switch 200 is illustrated.

In the upper right part of fig. 13, for example, the bit rate (8,560,557,792 bps) and the data amount (53911911992 bytes) of the multicast AV stream of the AV device 152(IPC _ TX1) are shown as a stream list. Also, the bit rate (384 bps) and the data amount (3006300) of the broadcast data of the control data from the AV device 154(IPC _ RX1) are shown. Note that the data flow information of the AV device may be directly acquired by the flow display control unit 506 of the SDN controller 500, or the flow display control unit 506 may acquire information acquired as the device flow information 556 by the application 550.

Further, flow information in each of the IP switches 200 is shown in the lower right part of fig. 13. Accordingly, since both the data stream information in the AV apparatuses 150, 152, and 154 and the data stream information in the IP switch 200 are visualized by using the UI, the user can recognize the data stream flowing from the AV apparatus on the transmitting side to the AV apparatus on the receiving side through the IP switch 200. Further, since the change of the network system is reflected in the information visualizing the network topology shown in fig. 13, even in the case where an AV device is further added, it is possible to recognize a data stream flowing from the AV device on the transmitting side to the AV device on the receiving side through the IP switch 200.

6. Application example

The technology according to the present disclosure can be applied to various products. For example, techniques according to the present disclosure may be applied to operating room systems.

Fig. 14 is a diagram schematically illustrating an overall configuration of an operating room system 5100 to which the technique according to the present disclosure can be applied. Referring to fig. 14, in an operating room system 5100, devices installed in an operating room are connected to each other via an audio visual controller (AV controller) 5107 and an operating room control device 5109 so as to be able to cooperate with each other.

Various devices can be installed in the operating room. As an example, fig. 14 illustrates various devices 5101 for endoscopic surgery, a ceiling camera 5187 that is provided on the ceiling of an operating room and images a hand-side area of a surgeon, an operating room camera 5189 that is provided on the ceiling of an operating room and images the state of the entire operating room, a plurality of display devices 5103A to 5103D, a recorder 5105, a patient bed 5183, and an illumination apparatus 5191.

Here, among these devices, the device 5101 belongs to an endoscopic surgery system 5113 described later, and includes an endoscope, a display device that displays an image captured by the endoscope, and the like. Each device belonging to the endoscopic surgery system 5113 is also referred to as a medical device. On the other hand, the display devices 5103A to 5103D, the recorder 5105, the patient bed 5183, and the illumination device 5191 are devices provided separately from the endoscopic surgery system 5113 in, for example, an operating room. Each device that does not belong to the endoscopic surgical system 5113 is also referred to as a non-medical device. The audiovisual controller 5107 and/or the operating room control device 5109 control the operation of these medical and non-medical devices in cooperation with each other.

The audiovisual controller 5107 comprehensively controls processing regarding image display in the medical device and the non-medical device. Specifically, among the devices included in the operating room system 5100, the device 5101, the ceiling camera 5187, and the operating room camera 5189 can each be a device (hereinafter also referred to as a transmission source device) having a function of transmitting information to be displayed during an operation (hereinafter also referred to as display information). Further, the display devices 5103A to 5103D can each be a device to which display information is output (hereinafter also referred to as an output destination device). Further, the recorder 5105 can be a device corresponding to both the transmission source device and the output destination device. The audiovisual controller 5107 has a function of controlling operations of the transmission source device and the output destination device to acquire display information from the transmission source device and transmit the display information to the output destination device for display or recording. Note that the display information is various images captured during an operation, various types of information on the operation (e.g., physical information of a patient, a result of a past examination, information on a method of the operation, and the like), and the like.

Specifically, information on an image of a surgical site in a body cavity of a patient captured by an endoscope is transmitted as display information from the device 5101 to the audiovisual controller 5107. Further, information about an image of the surgeon's hand area captured by the ceiling camera 5187 can be transmitted from the ceiling camera 5187 as display information. Further, information on an image representing the state of the entire operating room captured by the operating room camera 5189 can be transmitted from the operating room camera 5189 as display information. Note that in the case where another device having an imaging function exists in the operating room system 5100, the audiovisual controller 5107 may acquire information about an image captured by the other device from the other device as display information.

Alternatively, for example, information on these images captured in the past is recorded in the recorder 5105 by the audiovisual controller 5107. The audiovisual controller 5107 can acquire information on images captured in the past as display information from the recorder 5105. Note that various types of information about the operation may also be recorded in advance in the recorder 5105.

The audiovisual controller 5107 causes at least one of the display devices 5103A to 5103D as output destination devices to display the acquired display information (in other words, images captured during surgery and various types of information regarding the surgery). In the illustrated example, the display device 5103A is a display device mounted to hang from a ceiling of an operating room, the display device 5103B is a display device mounted on a wall of the operating room, the display device 5103C is a display device mounted on a desk in the operating room, and the display device 5103D is a mobile device having a display function (e.g., a tablet Personal Computer (PC)).

Further, although not illustrated in fig. 14, the operating room system 5100 may include devices external to the operating room. The devices outside the operating room may be, for example, a server connected to a network built inside and outside a hospital, a PC used by medical staff, a projector installed in a conference room of a hospital, and the like. When such an external device is outside the hospital, the viewing controller 5107 can also cause a display device of another hospital to display information via a video conference system or the like to perform remote medical treatment.

The operating room control device 5109 comprehensively controls processing other than processing related to image display in the non-medical device. For example, the operating room control device 5109 controls driving of the bed 5183, the ceiling camera 5187, the operating room camera 5189, and the lighting device 5191.

A centralized operation panel 5111 is provided in the operating room system 5100, and the user can give an instruction about image display to the viewing controller 5107 or an instruction about operation of a non-medical device to the operating room control device 5109 via the centralized operation panel 5111. The centralized operation panel 5111 is configured as a touch panel provided on the display surface of the display device.

Fig. 15 is a diagram illustrating a display example of an operation screen on the centralized operation panel 5111. In fig. 15, as an example, an operation screen corresponding to a case where the operating room system 5100 is provided with two display devices as output destination devices is illustrated. Referring to fig. 15, operation screen 5193 is provided with a transmission source selection area 5195, a preview area 5197, and a control area 5201.

In the transmission source selection area 5195, a transmission source device included in the operating room system 5100 and respective thumbnail screens showing display information of the transmission source device are displayed in association with each other. The user can select display information to be displayed on the display device from any transmission source device displayed in the transmission source selection area 5195.

In the preview area 5197, a preview of a screen displayed on the respective two display apparatuses (monitor 1 and monitor 2) as output destination apparatuses is displayed. In the illustrated example, four images are displayed in picture-in-picture (PinP) in one display device. These four images correspond to display information transmitted from the transmission source device selected in the transmission source selection area 5195. Of these four images, one image is displayed relatively large as a main image, and the remaining three images are displayed relatively small as sub images. The user can switch the main image and the sub image from each other by appropriately selecting one of the four areas where the corresponding images are displayed. Further, a state display area 5199 is provided below the area where the four images are displayed, and the state regarding the operation (for example, the elapsed time of the operation, the body information of the patient, and the like) is appropriately displayed in the area.

The control area 5201 is provided with a transmission source operation area 5203 which displays Graphical User Interface (GUI) components for performing operations on the transmission source device, and an output destination operation area 5205 which displays GUI components for performing operations on the transmission destination device. In the illustrated example, in the transmission source operation area 5203, GUI components for performing various operations (pan, tilt, and zoom) on the camera in the transmission source device having an imaging function are provided. The user can operate the operation of the camera in the transmission source device by appropriately selecting these GUI components. Note that although not illustrated, in a case where the transmission source device selected in the transmission source selection area 5195 is a recorder (in other words, in a case where images recorded in the recorder in the past are displayed on the preview area 5197), GUI components for performing operations such as reproduction of images, reproduction stop, rewind, and fast forward can be set in the transmission source operation area 5203.

Further, in the output destination operation area 5205, a GUI component for performing various operations (swapping, flipping, color adjustment, contrast adjustment, switching between 2D display and 3D display) on the display device as the output destination device is provided. The user can manipulate the display on the display device by appropriately selecting these GUI components.

Note that the operation screen displayed on the centralized operation panel 5111 is not limited to the illustrated example, and the user may be able to make an operation input via the centralized operation panel 5111 to each device included in the operating room system 5100 that can be controlled by the audiovisual controller 5107 and the operating room control device 5109.

Fig. 16 is a diagram illustrating an example of a state of an operation to which the above-described operating room system is applied. A ceiling camera 5187 and an operating room camera 5189 are provided on the ceiling of an operating room, and can image the hand area of a surgeon (sureon) 5181 who treats an affected part of a patient 5185 on a patient bed 5183 and the state of the entire operating room. The ceiling camera 5187 and the operating room camera 5189 can be provided with a magnification adjustment function, a focal length adjustment function, an imaging direction adjustment function, and the like. The illumination device 5191 is provided on the ceiling of the operating room and illuminates at least the area at hand of the surgeon 5181. The lighting device 5191 can appropriately adjust the amount of irradiation light, the wavelength (color) of irradiation light, the irradiation direction of light, and the like.

As shown in fig. 14, the endoscopic surgery system 5113, the patient bed 5183, the ceiling camera 5187, the operating room camera 5189, and the lighting device 5191 are connected to each other via the audio-visual controller 5107 and the operating room control apparatus 5109 (not shown in fig. 16) so as to be able to cooperate with each other. The centralized operation panel 5111 is provided in the operating room, and as described above, the user can appropriately operate these devices existing in the operating room via the centralized operation panel 5111.

Hereinafter, the configuration of the endoscopic surgery system 5113 will be described in detail. As shown, the endoscopic surgery system 5113 includes an endoscope 5115, other surgical tools 5131, a support arm device 5141 that supports the endoscope 5115, and a cart 5151 on which various devices for endoscopic surgery are mounted.

In endoscopic surgery, a laparotomy is not performed by incising the abdominal wall, but the abdominal wall is pierced by a plurality of cylindrical opening devices called trocars 5139a to 5139 d. Then, the lens barrel 5117 of the endoscope 5115 and other surgical tools 5131 are inserted into the body cavity of the patient 5185 from the trocars 5139a to 5139 d. In the illustrated example, as other surgical tools 5131, a pneumoperitoneum tube 5133, an energy treatment tool (energy treatment tool)5135, and forceps 5137 are inserted into a body cavity of a patient 5185. Further, the energy treatment tool 5135 is a treatment tool which performs incision and dissection of tissue, closure of blood vessels, or the like by high-frequency current or ultrasonic vibration. However, the illustrated surgical tool 5131 is only an example, and various surgical tools commonly used in endoscopic surgery, such as forceps, a retractor, etc., may be used as the surgical tool 5131.

An image of a surgical site in a body cavity of a patient 5185 imaged by the endoscope 5115 is displayed on the display device 5155. The surgeon 5181 performs treatment, such as excision of an affected part, etc., by using the energy treatment tool 5135 and the forceps 5137 while viewing an image of the surgical site displayed on the display device 5155 in real time. Note that, although not illustrated, during an operation, the pneumoperitoneum tube 5133, the energy treatment tool 5135, and the forceps 5137 are supported by the surgeon 5181, an assistant, or the like.

(supporting arm device)

The support arm device 5141 includes an arm 5145 extending from a base 5143. In the illustrated example, the arm 5145 includes joints 5147a, 5147b, and 5147c and links 5149a and 5149b, and is driven by the control of the arm control device 5159. The endoscope 5115 is supported by the arm 5145, and its position and posture are controlled. As a result, stable position fixation of the endoscope 5115 can be achieved.

(endoscope)

The endoscope 5115 includes a lens barrel 5117 into which a region of a predetermined length from a distal end is inserted into a body cavity of a patient 5185, and a camera head 5119 connected to a proximal end of the lens barrel 5117. In the illustrated example, the endoscope 5115 formed as a so-called rigid mirror including the rigid lens barrel 5117 is illustrated, but the endoscope 5115 may also be formed as a so-called flexible mirror including the flexible lens barrel 5117.

At the distal end of the lens barrel 5117, an opening in which an objective lens is embedded is provided. The light source device 5157 is connected to the endoscope 5115, light generated by the light source device 5157 is guided to the distal end of the lens barrel by a light guide extending inside the lens barrel 5117, and the light is emitted toward an observation object in the body cavity of the patient 5185 via the objective lens. Note that the endoscope 5115 may be a forward-looking endoscope, an oblique-looking endoscope, or a side-looking endoscope.

An optical system and an imaging element are provided inside the camera head 5119, and reflected light (observation light) from an observation target is focused on the imaging element by the optical system. The observation light is photoelectrically converted by the imaging element, and an electric signal corresponding to the observation light, that is, an image signal corresponding to an observation image is generated. The image signal is transmitted as RAW data to a Camera Control Unit (CCU) 5153. Note that in the camera head 5119, a function of adjusting the magnification and the focal length by appropriately driving the optical system is mounted.

Note that the camera head 5119 may be provided with a plurality of imaging elements in order to cope with stereoscopic vision (3D display) or the like, for example. In this case, a plurality of relay optical systems are provided inside the lens barrel 5117 to guide observation light to each of the plurality of imaging elements.

(various devices mounted on the cart)

The CCU5153 includes a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and the like, and comprehensively controls the operations of the endoscope 5115 and the display device 5155. Specifically, the CCU5153 performs various types of image processing for displaying an image based on an image signal, such as development processing (demosaicing processing) or the like, on the image signal received from the camera head 5119. The CCU5153 supplies the display device 5155 with an image signal on which image processing is performed. Further, the audiovisual controller 5107 shown in fig. 14 is connected to the CCU 5153. The CCU5153 also supplies the image signal on which the image processing is performed to the av controller 5107. In addition, the CCU5153 transmits a control signal to the camera head 5119 to control driving thereof. The control signal can include information about imaging conditions, such as magnification and focal length. Information on the imaging conditions may be input via the input device 5161, or may be input via the centralized operation panel 5111 described above.

The display device 5155 displays an image based on an image signal subjected to image processing by the CCU5153 by control from the CCU 5153. In the case where the endoscope 5115 is compatible with high resolution imaging (for example, 4K (the number of horizontal pixels 3840 × the number of vertical pixels 2160), 8K (the number of horizontal pixels 7680 × the number of vertical pixels 4320), or the like), and/or in the case where the endoscope 5115 is compatible with 3D display, as the display device 5155, a display device capable of high resolution display and/or 3D display can be used corresponding to each case. In the case where the display device 5155 is compatible with high-resolution imaging such as 4K or 8K, a more immersive sensation can be obtained by using a display device having a size of 55 inches or more. Further, a plurality of display devices 5155 having different resolutions and sizes may be provided according to applications.

The light source device 5157 includes a light source such as a Light Emitting Diode (LED), and supplies irradiation light for imaging a surgical site to the endoscope 5115.

The arm control device 5159 includes a processor (e.g., a CPU or the like), and controls the driving of the arm 5145 of the support arm device 5141 in accordance with a predetermined control method by operating in accordance with a predetermined program.

The input device 5161 is an input interface to the endoscopic surgical system 5113. A user can input various types of information and instructions to the endoscopic surgical system 5113 via the input device 5161. For example, the user inputs various types of information about the surgery, such as physical information of the patient and information about the surgery method, via the input device 5161. Further, for example, the user inputs an instruction to drive the arm 5145, an instruction to change the imaging condition (the type of irradiation light, magnification, focal length, and the like) of the endoscope 5115, an instruction to drive the energy therapy tool 5135, and the like via the input device 5161.

The type of the input device 5161 is not limited, and the input device 5161 may be any of various known input devices. As the input device 5161, for example, a mouse, a keyboard, a touch panel, a switch, a foot switch 5171, a joystick, or the like can be applied. In the case of using a touch panel as the input device 5161, the touch panel may be provided on the display surface of the display device 5155.

Alternatively, the input device 5161 is a device worn by the user, such as a glasses-type wearable device, a head-mounted display (HMD), or the like, and performs various inputs according to gestures and line of sight of the user detected by these devices. Further, the input device 5161 includes a camera capable of detecting movement of a user, and performs various inputs according to gestures and a line of sight of the user detected from a video captured by the camera. Also, the input device 5161 includes a microphone capable of picking up a voice of a user, and various inputs are performed by the voice via the microphone. As described above, the input device 5161 is enabled to input various information without contact, whereby particularly a user (e.g., surgeon 5181) belonging to a cleaning area can operate a device belonging to a non-cleaning area without contact. Further, since the user can operate the apparatus without releasing the user's hand from the surgical tool, the user's convenience is improved.

The treatment tool control apparatus 5163 controls driving of the energy treatment tool 5135 for cauterization, incision, closure of blood vessels, and the like of tissue. The pneumoperitoneum device 5165 injects gas into the body cavity of the patient 5185 via the pneumoperitoneum tube 5133 to inflate the body cavity for the purpose of securing the field of view of the endoscope 5115 and securing the working space of the surgeon. The recorder 5167 is a device capable of recording various types of information about a procedure. The printer 5169 is a device capable of printing various types of information about a procedure in various formats (such as text, images, graphics, and the like).

Hereinafter, a specific characteristic configuration in the endoscopic surgical system 5113 will be described in detail.

(supporting arm device)

The support arm device 5141 includes a base 5143 as a base and an arm 5145 extending from the base 5143. In the illustrated example, the arm 5145 includes a plurality of joints 5147a, 5147b, and 5147c and a plurality of links 5149a and 5149b coupled together by the joint 5147b, but in fig. 16, the configuration of the arm 5145 is simplified for the sake of simplicity. In fact, the shapes, the number, and the arrangement of the joints 5147a to 5147c and the links 5149a and 5149b, the directions of the rotational axes of the joints 5147a to 5147c, and the like can be appropriately set so that the arm 5145 has a desired degree of freedom. For example, the arm 5145 can suitably have six or more degrees of freedom. As a result, the endoscope 5115 can be freely moved within the movable range of the arm 5145, so that the lens barrel 5117 of the endoscope 5115 can be inserted into the body cavity of the patient 5185 from a desired direction.

Each of the joints 5147a to 5147c is provided with an actuator, and each of the joints 5147a to 5147c can be rotated about a predetermined rotation axis by the driving of the actuator. The arm control device 5159 controls the drive of the actuator, thereby controlling the rotation angle of each of the joints 5147a to 5147c, and controls the drive of the arm 5145. As a result, control of the position and posture of the endoscope 5115 can be achieved. At this time, the arm control device 5159 can control the driving of the arm 5145 by various known control methods such as force control or position control.

For example, the surgeon 5181 appropriately performs operation input via the input device 5161 (including the foot switch 5171), whereby the drive of the arm 5145 can be appropriately controlled by the arm control device 5159 in accordance with the operation input, and the position and posture of the endoscope 5115 can be controlled. By this control, the endoscope 5115 at the distal end of the arm 5145 can be moved from an arbitrary position to an arbitrary position and then fixedly supported at a position after the movement. Note that the arm 5145 can be operated by a so-called master-slave method. In this case, the user can remotely operate the arm 5145 via the input device 5161 installed at a location remote from the operating room.

Further, in the case where force control is applied, the arm control device 5159 may perform so-called power assist control in which an external force is received from a user and an actuator of each of the joints 5147a to 5147c is driven so that the arm 5145 moves smoothly with the external force. As a result, when the user moves the arm 5145 while directly contacting the arm 5145, the arm 5145 can be moved with a relatively weak force. Therefore, the endoscope 5115 can be moved more intuitively and with a simpler operation, and the convenience of the user can be improved.

Here, generally, in an endoscopic operation, the endoscope 5115 is supported by a surgeon called an assistant (scope). In contrast, by using the support arm device 5141, the position of the endoscope 5115 can be fixed more reliably without depending on the hand of the person, so that an image of the surgical site can be stably obtained, and the surgery can be smoothly performed.

Note that the arm control device 5159 is not necessarily provided in the cart 5151. Further, the arm control device 5159 does not necessarily have to be one device. For example, an arm control device 5159 may be provided at each of the joints 5147a to 5147c of the arm 5145 of the support arm device 5141, and a plurality of arm control devices 5159 cooperate with each other, whereby drive control of the arm 5145 can be realized.

(light Source device)

When imaging the surgical site, the light source device 5157 provides illumination light to the endoscope 5115. The light source device 5157 includes, for example, a white light source including an LED, a laser light source, or a combination thereof. At this time, in the case where the white light source includes a combination of R, G and a B laser light source, the output intensity and the output timing of each color (each wavelength) can be controlled with high accuracy, so that adjustment of the white balance of a captured image can be performed in the light source device 5157. Further, in this case, it is also possible to capture images corresponding to each of R, G and B in a time-division manner by emitting laser light from each of R, G and B laser light sources to an observation object in a time-division manner and controlling driving of an imaging element of the camera head 5119 in synchronization with emission timing. According to this method, a color image can be obtained without providing a color filter in the imaging element.

Further, the driving of the light source device 5157 may be controlled so that the intensity of light to be output is changed at predetermined time intervals. By controlling the driving of the imaging element of the camera head 5119 in synchronization with the change timing of the light intensity to acquire an image in a time-division manner and synthesize the image, a high dynamic range image can be generated without so-called occlusion shadow or popping highlight (brown out high light).

Further, the light source device 5157 may be capable of providing light of a predetermined wavelength band corresponding to special light observation. In special light observation, for example, by using the wavelength dependence of light absorption in human tissue, so-called narrow band imaging for imaging a predetermined tissue such as blood vessels in the mucosal surface layer with high contrast is performed by emitting narrow band light compared with irradiation light (in other words, white light) at the time of ordinary observation. Alternatively, in the special light observation, fluorescence observation in which an image is obtained by fluorescence generated by emitting excitation light may be performed. In fluorescence observation, fluorescence from human tissue may be observed by irradiating the human tissue with excitation light (autofluorescence observation), or a fluorescence image may be obtained by locally injecting an agent such as indocyanine green (ICG) into the human tissue and irradiating the human tissue with excitation light corresponding to the fluorescence wavelength of the agent, for example. The light source device 5157 may be capable of providing narrow band light and/or excitation light corresponding to such special light observations.

(Camera head and CCU)

The functions of the camera head 5119 and the CCU5153 of the endoscope 5115 will be described in more detail with reference to fig. 17. Fig. 17 is a block diagram illustrating an example of the functional configurations of the camera head 5119 and the CCU5153 illustrated in fig. 16.

Referring to fig. 17, the camera head 5119 includes a lens unit 5121, an imaging unit 5123, a driving unit 5125, a communication unit 5127, and a camera head control unit 5129 as its functions. Further, the CCU5153 includes as its functions a communication unit 5173, an image processing unit 5175, and a control unit 5177. The camera head 5119 and the CCU5153 are communicably connected to each other in both directions by a transmission cable 5179.

First, a functional configuration of the camera head 5119 will be described. The lens unit 5121 is an optical system provided at a connection portion with the lens barrel 5117. Observation light captured from the distal end of the lens barrel 5117 is guided to the camera head 5119 and is incident on the lens unit 5121. The lens unit 5121 includes a combination of a plurality of lenses including a zoom lens and a focus lens. The optical characteristics of the lens unit 5121 are adjusted so that observation light is focused on the light receiving surface of the imaging element of the imaging unit 5123. Further, positions on the optical axes of the zoom lens and the focus lens are movable to adjust the magnification and focus of a captured image.

The imaging unit 5123 includes an imaging element, and is disposed at the subsequent stage of the lens unit 5121. Observation light passing through the lens unit 5121 is focused on a light receiving surface of the imaging element, and an image signal corresponding to an observation image is generated by photoelectric conversion. The image signal generated by the imaging unit 5123 is supplied to the communication unit 5127.

As an imaging element constituting the imaging unit 5123, for example, an element which is a Complementary Metal Oxide Semiconductor (CMOS) type image sensor capable of color imaging with a bayer array is used. Note that as the imaging element, for example, an element compatible with imaging of a high-resolution image of 4K or more can be used. An image of the surgical site is obtained at high resolution, whereby the surgeon 5181 can grasp the state of the surgical site in more detail and can perform the operation more smoothly.

Further, the imaging elements constituting the imaging unit 5123 include a pair of imaging elements for acquiring image signals for the right and left eyes to cope with 3D display. By performing the 3D display, the surgeon 5181 can grasp the depth of the living tissue in the surgical site more accurately. Note that in the case where the imaging unit 5123 includes a multi-piece type imaging element, a plurality of systems of the lens unit 5121 are provided corresponding to the respective imaging elements.

Further, the imaging unit 5123 is not necessarily provided in the camera head 5119. For example, the imaging unit 5123 may be provided inside the lens barrel 5117 immediately after the objective lens.

The driving unit 5125 includes an actuator, and moves the zoom lens and the focus lens of the lens unit 5121 by a predetermined distance along the optical axis by the control of the camera head control unit 5129. As a result, the magnification and focus of the image captured by the imaging unit 5123 can be appropriately adjusted.

The communication unit 5127 includes a communication device for transmitting/receiving various types of information to/from the CCU 5153. The communication unit 5127 transmits the image signal obtained from the imaging unit 5123 to the CCU5153 as RAW data via the transmission cable 5179. At this time, in order to display the captured image of the surgical site with low time delay, the image signal is preferably transmitted by optical communication. This is because, during the operation, the surgeon 5181 performs the operation while observing the state of the affected part by capturing an image, and therefore, in order to perform the operation more safely and reliably, it is necessary to display the video image of the operation site in real time as much as possible. In the case of performing optical communication, the communication unit 5127 is provided with a photoelectric conversion module that converts an electric signal into an optical signal. The image signal is converted into an optical signal by the photoelectric conversion module and then transmitted to the CCU5153 via the transmission cable 5179.

Further, the communication unit 5127 receives a control signal for controlling driving of the camera head 5119 from the CCU 5153. The control signal includes information on imaging conditions, such as information specifying a frame rate of a captured image, information specifying an exposure value at the time of imaging, and/or information specifying a magnification and a focus of the captured image. The communication unit 5127 supplies the received control signal to the camera head control unit 5129. Note that the control signal from the CCU5153 may also be transmitted by optical communication. In this case, the communication unit 5127 is provided with a photoelectric conversion module that converts an optical signal into an electrical signal, and the control signal is converted into an electrical signal by the photoelectric conversion module and then supplied to the camera head control unit 5129.

Note that the above-described imaging conditions (such as the frame rate, the exposure value, the magnification, and the focus) are automatically set by the control unit 5177 of the CCU5153 based on the acquired image signal. That is, a so-called Auto Exposure (AE) function, Auto Focus (AF) function, and Auto White Balance (AWB) function are installed in the endoscope 5115.

The camera control unit 5129 controls driving of the camera 5119 based on a control signal from the CCU5153 received via the communication unit 5127. For example, the camera head control unit 5129 controls driving of the imaging element of the imaging unit 5123 based on information specifying the frame rate of a captured image and/or information specifying exposure at the time of imaging. Further, for example, the camera head control unit 5129 appropriately moves the zoom lens and the focus lens of the lens unit 5121 via the drive unit 5125 based on information specifying the magnification and focus of a captured image. The camera head control unit 5129 may also have a function of storing information for identifying the lens barrel 5117 and the camera head 5119.

Note that by disposing the lens unit 5121, the imaging unit 5123, and the like in a sealed structure having high air-tightness and water-tightness, the camera head 5119 can be made resistant to autoclaving (autoclave sterilization).

Next, the functional configuration of the CCU5153 will be described. The communication unit 5173 includes a communication device for transmitting/receiving various types of information to/from the camera head 5119. The communication unit 5173 receives an image signal transmitted from the camera head 5119 via the transmission cable 5179. At this time, as described above, the image signal can be appropriately transmitted by optical communication. In this case, in order to be suitable for optical communication, the communication unit 5173 is provided with a photoelectric conversion module that converts an optical signal into an electrical signal. The communication unit 5173 supplies the image signal converted into an electric signal to the image processing unit 5175.

Further, the communication unit 5173 transmits a control signal for controlling the driving of the camera head 5119 to the camera head 5119. The control signal may also be transmitted by optical communication.

The image processing unit 5175 performs various types of image processing on an image signal as RAW data transmitted from the camera head 5119. Examples of the image processing include various types of known signal processing, such as development processing, image quality enhancement processing (such as band enhancement processing, super-resolution processing, Noise Reduction (NR) processing, and/or camera shake correction processing), and/or enlargement processing (electronic zoom processing), and the like. Further, the image processing unit 5175 performs detection processing on the image signal to perform AE, AF, and AWB.

The image processing unit 5175 includes a processor such as a CPU or a GPU, and is capable of performing the above-described image processing and detection processing by the processor operating in accordance with a predetermined program. Note that in the case where the image processing unit 5175 includes a plurality of GPUs, the image processing unit 5175 appropriately divides information related to image signals and performs image processing in parallel by the plurality of GPUs.

The control unit 5177 performs various types of control regarding imaging of the surgical site by the endoscope 5115 and display of the captured image. For example, the control unit 5177 generates a control signal for controlling the driving of the camera head 5119. At this time, in the case where the imaging condition is input by the user, the control unit 5177 generates a control signal based on the input of the user. Alternatively, in the case where an AE function, an AF function, and an AWB function are installed in the endoscope 5115, the control unit 5177 generates control signals by appropriately calculating an optimal exposure value, a focal length, and a white balance from the result of the detection processing performed by the image processing unit 5175.

Further, the control unit 5177 causes the display device 5155 to display an image of the surgical site based on the image signal subjected to the image processing by the image processing unit 5175. At this time, the control unit 5177 recognizes various objects in the operation site image by using various image recognition techniques. For example, the control unit 5177 detects the color of an object, the shape of an edge, or the like included in the surgical site image, thereby being able to recognize a surgical tool such as forceps, a specific body part, bleeding, fog when using the energy therapy tool 5135, or the like. When causing the display device 5155 to display the image of the surgical site, the control unit 5177 causes the display device 5155 to display various types of surgical assistance information superimposed on the image of the surgical site by using the recognition result. The surgical assistance information is displayed superimposed and presented to the surgeon 5181, whereby the surgery can be performed more safely and reliably.

The transmission cable 5179 connecting the camera head 5119 and the CCU5153 together is an electric signal cable suitable for communication of electric signals, an optical fiber suitable for optical communication, or a composite cable thereof.

Here, in the illustrated example, the communication is performed by wire using the transmission cable 5179, but the communication between the camera head 5119 and the CCU5153 may also be performed wirelessly. In the case where communication between the two is performed wirelessly, there is no need to install the transmission cable 5179 in the operating room, so that it is possible to eliminate a case where movement of medical personnel in the operating room is obstructed by the transmission cable 5179.

In the above, an example of an operating room system 5100 to which techniques according to the present disclosure can be applied has been described. Note that here, as an example, a case where the medical system to which the operating room system 5100 is applied is the endoscopic surgery system 5113 has been described, but the configuration of the operating room system 5100 is not limited to such an example. For example, the operating room system 5100 can be applied to an examination flexible endoscopic system or a microsurgical system instead of the endoscopic surgical system 5113.

The technique according to the present disclosure can be suitably applied to the network constituting the operating room system 5100 in the above-described configuration. By applying the technique according to the present disclosure to the operating room system 5100, a network system that efficiently connects devices to each other at low cost can be constructed.

In the above, preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings; however, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can conceive various modifications or corrections within the scope of the technical idea described in the claims, and it is understood that these modifications or corrections also belong to the technical scope of the present disclosure.

Further, the effects described in the present specification are merely illustrative or exemplary and not restrictive. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification, together with or instead of the above-described effects.

Note that the following configuration also belongs to the technical scope of the present disclosure.

(1) A system controller that controls an IP switch that distributes data received from a device on a transmission side to a device on a reception side,

the system controller constructs a plurality of virtual networks in the IP switch, and causes data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

(2) The system controller according to (1), wherein the IP switch includes a plurality of switches layered, and the virtual network is built in the plurality of switches.

(3) The system controller according to (2), wherein any two switches of the plurality of switches are connected to each other through a plurality of links.

(4) The system controller according to (2) or (3), wherein each of the distribution destinations to which data received by any of the virtual networks is transmitted is a device on a reception side connected to one of switches in which the any of the virtual networks is built, or is another one of switches connected to the one of switches in which the any of the virtual networks is built.

(5) The system controller according to (3) or (4), further comprising a load distribution unit that performs load distribution based on a size of a data stream, a bandwidth of the link, or a bandwidth of a port of the switch, in a case where the received data is multicast.

(6) The system controller according to (5), wherein the load distribution unit performs the load distribution for each data flow.

(7) The system controller according to any one of (2) to (6), further comprising a display control unit that controls display of a network topology based on information on a data flow in each of the switches and information on a data flow of a device connected to the IP switch.

(8) The system controller according to any one of (1) to (7), wherein the data is audio data or video data.

(9) A network system, comprising:

an IP switch that distributes data received from a device on a transmitting side to a device on a receiving side; and

a system controller controlling the IP switch, wherein,

the system controller constructs a plurality of virtual networks in the IP switch, and causes data received by any of the virtual networks to be transmitted to respective distribution destinations connected to the virtual networks.

(10) The network system according to (9), wherein the IP switch includes a plurality of switches layered, and the virtual network is constructed in the plurality of switches.

(11) The network system according to (10), wherein any two switches of the plurality of switches are connected to each other through a plurality of links.

(12) The network system according to (10) or (11), wherein each of the distribution destinations to which data received by any of the virtual networks is transmitted is a device on a reception side connected to one of switches in which the any of the virtual networks is constructed, or is another one of switches connected to the one of switches in which the any of the virtual networks is constructed.

(13) The network system according to (11) or (12), further comprising a load distribution unit that performs load distribution based on a size of a data stream, a bandwidth of the link, or a bandwidth of a port of the switch, in a case where the received data is multicast.

(14) The network system according to (13), wherein the load distribution unit performs the load distribution for each data flow.

(15) The network system according to any one of (11) to (14), further comprising a display control unit that controls display of a network topology based on information on a data flow in each of the switches and information on a data flow of a device connected to the IP switch.

(16) The network system according to any one of (9) to (15), wherein the data is audio data or video data.

(17) A method in a network system for controlling an IP switch that distributes data received from a device on a transmission side to a device on a reception side, the method comprising:

a plurality of virtual networks are built in the IP switch, and data received by any of the virtual networks is transmitted to respective distribution destinations connected to the virtual networks.

List of labels

100 AV equipment

200 IP switch

222. 224 virtual network

500 SDN controller

502 load distribution unit

504 broadcast unit

506 flow display control unit