阅读说明：本技术 模拟装置、模拟方法以及模拟程序 (Simulation device, simulation method, and simulation program ) 是由 大川洋平 柴田义也 斋藤千智 林剣之介 傅健忠 山口雄纪 于 2019-10-11 设计创作，主要内容包括：本发明提供一种降低操作对象的举动模拟所需的运算负荷的模拟装置、模拟方法以及模拟程序。模拟装置包括：第一设定部,设定表示操作对象的模型的构成条件；第二设定部,设定对操作对象施加的外力的条件；第一模拟部,在构成条件及外力的条件下,模拟操作对象的举动；生成部,生成学习数据,所述学习数据包含构成条件、外力的条件以及表示第一模拟部所进行的模拟中的位于操作对象表面的多个代表点的举动的数据；以及学习部,通过使用学习数据的监督学习而生成学习模型,所述学习模型将构成条件、外力的条件及多个代表点的初始条件作为输入,而输出表示多个代表点的举动的数据。(The invention provides a simulation device, a simulation method and a simulation program for reducing the computational load required for behavior simulation of an operation object. The simulation apparatus includes: a first setting unit that sets a configuration condition of a model representing an operation target; a second setting unit that sets a condition for applying an external force to the operation object; a first simulation unit that simulates the behavior of an operation target under the configuration conditions and the external force conditions; a generation unit that generates learning data including configuration conditions, external force conditions, and data indicating behaviors of a plurality of representative points located on the surface of the operation object in the simulation performed by the first simulation unit; and a learning unit that generates a learning model by supervised learning using learning data, the learning model having as input a configuration condition, an external force condition, and initial conditions of the plurality of representative points, and outputs data indicating behaviors of the plurality of representative points.)

1. A simulation apparatus, comprising:

a first setting unit that sets a configuration condition of a model representing an operation target;

a second setting unit that sets a condition for applying an external force to the operation object;

a first simulation unit that simulates a behavior of the operation target under the configuration condition and the external force condition;

a generation unit configured to generate learning data including the configuration condition, the external force condition, and data indicating behaviors of a plurality of representative points on the surface of the operation object in the simulation performed by the first simulation unit; and

and a learning unit that generates a learning model by supervised learning using the learning data, the learning model having the configuration condition, the external force condition, and the initial conditions of the plurality of representative points as inputs, and outputs data indicating behaviors of the plurality of representative points.

2. Simulation device according to claim 1, wherein

The learning data generated by the generation unit includes, as input data, the configuration condition, the external force condition at a predetermined time of the simulation by the first simulation unit, and data indicating behaviors of the plurality of representative points at the predetermined time, and includes, as output data, data indicating behaviors of the plurality of representative points at a time when a predetermined time has elapsed from the predetermined time.

3. The simulation apparatus according to claim 1 or 2, further comprising:

a second simulation unit which simulates the behavior of the robot; and

a prediction unit configured to input, to the learning model, a condition of an external force applied to the operation object by the robot in the simulation performed by the second simulation unit and the constituent condition, and predict behaviors of the plurality of representative points,

the second simulation unit synthesizes the behavior of the representative points predicted by the prediction unit and the simulated behavior of the robot.

4. Simulation device according to any of the claims 1 to 3, wherein

The operation object includes any one of a flexible object, a liquid, and a gas.

5. Simulation device according to claim 4, wherein

When the operation target is a liquid, the surface of the operation target is a liquid surface of the liquid, and the behaviors of the plurality of representative points are expressed by behaviors of the liquid surface in a vertical direction at a plurality of positions.

6. Simulation device according to claim 5, wherein

The first simulation unit simulates behavior of a plurality of particles using the model that decomposes the liquid into the plurality of particles,

the data indicating the behaviors of the plurality of representative points is data indicating the behavior of a part of the particles located on the liquid surface of the liquid at a predetermined timing of the simulation performed by the first simulation unit among the plurality of particles.

7. Simulation device according to any of the claims 1 to 6, wherein

The configuration condition includes any one of a condition indicating flexibility and a condition indicating viscosity of the operation object.

8. Simulation device according to any of the claims 1 to 7, wherein

The first simulation unit simulates behavior of a node using the model in which the operation target is decomposed into a plurality of constituent elements connected by the node,

the number of the plurality of representative points is less than the number of the nodes.

9. A method of simulation, comprising:

setting a configuration condition of a model representing an operation object;

setting a condition of an external force applied to the operation object;

simulating the behavior of the operation object under the composition condition and the external force condition;

generating learning data including the configuration condition, the external force condition, and data indicating behaviors of a plurality of representative points on the surface of the operation object in the simulation; and

and a learning model that outputs data representing behaviors of the plurality of representative points by using the configuration condition, the external force condition, and the initial conditions of the plurality of representative points as inputs is generated by supervised learning using the learning data.

10. A simulation program for causing a processor provided in a simulation apparatus to function as:

a first setting unit that sets a configuration condition of a model representing an operation target;

a second setting unit that sets a condition for applying an external force to the operation object;

a first simulation unit that simulates a behavior of the operation target under the configuration condition and the external force condition;

a generation unit configured to generate learning data including the configuration condition, the external force condition, and data indicating behaviors of a plurality of representative points on the surface of the operation object in the simulation performed by the first simulation unit; and

and a learning unit that generates a learning model by supervised learning using the learning data, the learning model having the configuration condition, the external force condition, and the initial conditions of the plurality of representative points as inputs, and outputs data indicating behaviors of the plurality of representative points.

Technical Field

The present invention relates to a simulation (simulation) apparatus, a simulation method, and a simulation program.

Background

Conventionally, various processes have been performed on an operation target by a robot. The operation of the robot may be verified by simulation, and the robot may be designed while referring to the simulation result.

For example, patent document 1 below describes a method in which: the operation of the industrial machine is simulated and the simulation results and actual data from the operation of the industrial machine are stored or compared.

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2008-542888

Disclosure of Invention

Problems to be solved by the invention

When the object to be operated by the robot is a liquid or a flexible object having flexibility, the behavior of the object may be simulated in accordance with the behavior of the robot. The behavior of the operation target can be simulated by the conventional technique such as the finite element method or the particle method.

However, if the behavior of the operation target is simulated with high accuracy, the computational load may increase, and the time required for the simulation may become long.

Accordingly, the present invention provides a simulation apparatus, a simulation method, and a simulation program that reduce the computational load required for behavior simulation of an operation target.

Means for solving the problems

The simulation apparatus of an embodiment of the present disclosure includes: a first setting unit that sets a configuration condition of a model representing an operation target; a second setting unit that sets a condition for applying an external force to the operation object; a first simulation unit that simulates the behavior of an operation target under the configuration conditions and the external force conditions; a generation unit that generates learning data including configuration conditions, external force conditions, and data indicating behaviors of a plurality of representative points located on the surface of the operation object in the simulation performed by the first simulation unit; and a learning unit that generates a learning model by supervised learning using learning data, the learning model having as input a configuration condition, an external force condition, and initial conditions of the plurality of representative points, and outputs data indicating behaviors of the plurality of representative points.

According to this embodiment, by simulating the behavior of the model representing the operation target by an arbitrary method and generating the learning model by supervised learning using the simulation result, it is possible to replace the simulation with a large operation load with the prediction of the behavior of the representative point by the learning model with a relatively small operation load, and it is possible to reduce the operation load required for the behavior simulation of the operation target.

In the above-described embodiment, the learning data generated by the generation unit may include, as input data, the configuration conditions, the conditions of the external force at the predetermined time of the simulation performed by the first simulation unit, and data indicating the behaviors of the plurality of representative points at the predetermined time, and may include, as output data, data indicating the behaviors of the plurality of representative points at a time after a lapse of a predetermined time from the predetermined time.

According to this embodiment, the behavior of the representative point after the elapse of the predetermined time can be predicted by the learning model, and the behavior of the representative point at an arbitrary time can be predicted by repeating the prediction by the learning model.

In the embodiment, the method may further include: a second simulation unit which simulates the behavior of the robot; and a prediction unit that predicts behaviors of the plurality of representative points by inputting, to the learning model, conditions and constituent conditions of an external force applied to the operation object by the robot in the simulation performed by the second simulation unit, and the second simulation unit synthesizes the behaviors of the plurality of representative points predicted by the prediction unit and the simulated behavior of the robot.

According to this embodiment, by learning the model, the behavior of the representative point of the model representing the operation target is predicted with a relatively small computational load, and is synthesized with the result of simulating the behavior of the robot by an arbitrary method, it is possible to reduce the computational load required for the simulation of the entire robot including the robot and the operation target.

In the embodiment, the operation object may include any one of a flexible object, a liquid, and a gas.

According to this embodiment, even if the simulation calculation load is large, the behavior of the representative point of the operation target can be predicted by the calculation using the learning model having a relatively small calculation load, and the calculation load required for the behavior simulation of the operation target can be reduced.

In the above embodiment, when the operation target is a liquid, the surface of the operation target may be a liquid surface of the liquid, and the behaviors of the plurality of representative points may be expressed by behaviors of the liquid surface at a plurality of positions in a vertical direction.

According to this embodiment, by predicting the behavior of a plurality of representative points located on the liquid surface of the liquid using the learning model, the computational load of the learning model can be reduced as compared with the case of predicting the behavior of the entire liquid.

In the above-described embodiment, the first simulation unit may simulate the behavior of the plurality of particles using a model in which the liquid is decomposed into the plurality of particles, and the data indicating the behavior of the plurality of representative points may be data indicating the behavior of a part of the particles located on the liquid surface of the liquid at a predetermined time of the simulation performed by the first simulation unit among the plurality of particles.

According to this embodiment, a learning model that reproduces the behavior of the particles located on the liquid surface in the result of simulating the behavior of the entire liquid by the particle model can be generated, and the computational load of the learning model can be reduced compared to the case of predicting the behavior of the entire liquid.

In the above embodiment, the configuration condition may include any one of a condition indicating flexibility and a condition indicating viscosity of the operation target.

According to this embodiment, even if the simulation calculation load is large, the behavior of the representative point of the operation target can be predicted by the calculation using the learning model having a relatively small calculation load, and the calculation load required for the behavior simulation of the operation target can be reduced.

In the above embodiment, the first simulation unit may simulate the behavior of the node using a model in which the operation target is decomposed into a plurality of constituent elements connected by the node, and the number of the plurality of representative points may be smaller than the number of the nodes.

According to this embodiment, a learning model that reproduces behaviors of nodes located on the surface of the operation target among results of simulating the behaviors of the entire operation target by the finite element method can be generated, and the computational load on the learning model can be reduced compared to a case where the behaviors of the entire nodes representing the operation target are predicted.

The simulation method of another embodiment of the present disclosure includes: setting a configuration condition of a model representing an operation object; setting a condition of an external force applied to an operation object; simulating the behavior of the operation object under the composition condition and the external force condition; generating learning data including a composition condition, an external force condition, and data indicating behaviors of a plurality of representative points on a surface of an operation object in a simulation; and generating a learning model by supervised learning using learning data, the learning model having as input a composition condition, an external force condition, and initial conditions of the plurality of representative points, and outputting data representing behaviors of the plurality of representative points.

According to this embodiment, by simulating the behavior of the model representing the operation target by an arbitrary method and generating the learning model by supervised learning using the simulation result, it is possible to replace the simulation with a large operation load with the prediction of the behavior of the representative point by the learning model with a relatively small operation load, and it is possible to reduce the operation load required for the behavior simulation of the operation target.

A simulation program according to another embodiment of the present disclosure causes a processor provided in a simulation apparatus to function as: a first setting unit that sets a configuration condition of a model representing an operation target; a second setting unit that sets a condition for applying an external force to the operation object; a first simulation unit that simulates the behavior of an operation target under the configuration conditions and the external force conditions; a generation unit that generates learning data including configuration conditions, external force conditions, and data indicating behaviors of a plurality of representative points located on the surface of the operation object in the simulation performed by the first simulation unit; and a learning unit that generates a learning model by supervised learning using learning data, the learning model having as input a configuration condition, an external force condition, and initial conditions of the plurality of representative points, and outputs data indicating behaviors of the plurality of representative points.

According to this embodiment, by simulating the behavior of the model representing the operation target by an arbitrary method and generating the learning model by supervised learning using the simulation result, it is possible to replace the simulation with a large operation load with the prediction of the behavior of the representative point by the learning model with a relatively small operation load, and it is possible to reduce the operation load required for the behavior simulation of the operation target.

ADVANTAGEOUS EFFECTS OF INVENTION

According to the present invention, it is possible to provide a simulation apparatus, a simulation method, and a simulation program that reduce the computational load required for behavior simulation of an operation target.

Drawings

Fig. 1 is a schematic view of a behavior of a first operation target simulated by a simulation apparatus according to an embodiment of the present invention.

Fig. 2 is a diagram showing a hardware configuration of the simulation apparatus according to the present embodiment.

Fig. 3 is a diagram showing a functional configuration of the simulation apparatus according to the present embodiment.

Fig. 4 is a flowchart of a first process executed by the simulation apparatus of the present embodiment.

Fig. 5 is a flowchart of a second process executed by the simulation apparatus of the present embodiment.

Fig. 6 is a flowchart of a process executed by the conventional simulation apparatus.

Fig. 7 is a schematic diagram of the behavior of the second operation target simulated by the simulation apparatus according to the present embodiment.

Detailed Description

Hereinafter, an embodiment (hereinafter, referred to as "the present embodiment") according to one aspect of the present invention will be described with reference to the drawings. In the drawings, the same or similar structures are denoted by the same reference numerals.

Application example § 1

Fig. 1 is a schematic diagram of behavior of a first manipulation object T1 simulated by the simulation apparatus 10 according to the embodiment of the present invention. The first manipulation object T1 is a liquid put in the container object Ob. The container object Ob is held and moved by the robot hand H. When the robot hand H holds and moves the container object Ob, the simulation apparatus 10 simulates the operation of the robot hand H, that is, moves the first manipulation object T1 as a liquid for a short time so as not to spill the container object Ob. In fig. 1, the left side shows the first manipulation object T1 in a stationary state, and the right side shows a state in which the container object Ob is moved by the robot H while the liquid surface of the first manipulation object T1 is shaking.

The simulation apparatus 10 sets the configuration conditions of the model indicating the first manipulation object T1 and the conditions of the external force applied to the first manipulation object T1, and simulates the behavior of the first manipulation object T1 by a conventional simulation method under these conditions. The simulation apparatus 10 may simulate the behavior of a plurality of particles by a so-called particle method using a model in which the first manipulation object T1, which is a liquid, is decomposed into a plurality of particles, for example. In this case, the model may be configured under conditions that define the particle size and the interaction between the particles. The external force condition may include a boundary condition of the container object Ob and a condition of the force applied by the robot H.

The simulation apparatus 10 may use, as the learning data, data indicating the behavior of a part of the particles located on the liquid surface of the first manipulation object T1 among the behaviors of the plurality of particles simulated by the particle method. Here, the data indicating the behavior of the particle may include data indicating at least one of a position, a velocity, a motion amount, and an acceleration of the particle. When the operation target is a liquid like the first operation target T1, the surface of the operation target is the liquid surface of the liquid, and the behavior of the plurality of representative points can be expressed by the behavior in the vertical direction of the liquid surface at a plurality of positions. The simulation apparatus 10 may generate data indicating the behavior of a part of the particles on the liquid surface of the first manipulation object T1 by, for example, dividing the inside of the container object Ob into three-dimensional lattices, selecting particles having the largest value in the Z direction (vertical direction) for each lattice intersecting the XY plane of the container object Ob, setting XYZ coordinates of the particles as one representative point, and repeating this process.

In fig. 1, as a part of the particles located on the liquid surface of the first manipulation object T1, a first particle p1, a second particle p2, a third particle p3, a fourth particle p4, a fifth particle p5, a sixth particle p6, a seventh particle p7, an eighth particle p8, a ninth particle p9, a tenth particle p10, an eleventh particle p11, and a twelfth particle p12 are shown. The first particle p1, the second particle p2, the third particle p3, the fourth particle p4, the fifth particle p5, the sixth particle p6, the seventh particle p7, the eighth particle p8, the ninth particle p9, the tenth particle p10, the eleventh particle p11, and the twelfth particle p12 are representative points located at a certain time on the liquid surface of the first manipulation object T1. Here, the particles located on the liquid surface of the first manipulation object T1 may change at every moment, and may not necessarily be the same particles. In this example, when the container object Ob is moved by the robot hand H, the particles whose liquid is stirred and located on the liquid surface of the first manipulation object T1 change from the left side state where the heights of the first particle p1, the second particle p2, the third particle p3, the fourth particle p4, the fifth particle p5, and the sixth particle p6 are almost the same, and the particles transition to the right side state where the heights of the seventh particle p7, the eighth particle p8, the ninth particle p9, the tenth particle p10, the eleventh particle p11, and the twelfth particle are uneven.

The simulation apparatus 10 generates the learning data including the configuration conditions of the model of the first manipulation object T1, the conditions of the external force applied to the first manipulation object T1, and the data indicating the behavior of the plurality of representative points (the first to twelfth particles and the like) located on the liquid surface of the first manipulation object T1 in the simulation. The simulation apparatus 10 generates a learning model by supervised learning using learning data, which receives as input the configuration conditions of the model indicating the first manipulation object T1, the conditions of the external force applied to the first manipulation object T1, and the initial conditions of the plurality of representative points (the first to twelfth particles and the like), and outputs data indicating the behavior of the plurality of representative points (the first to twelfth particles and the like). The learning model predicts at least one of the position, the speed, the motion amount, and the acceleration of the plurality of representative points, instead of directly performing a simulation for reproducing the physical phenomenon.

In this way, by simulating the behavior of the model representing the first operation target T1 by an arbitrary method and generating a learning model by supervised learning using the simulation result, it is possible to replace a simulation with a large computation load with the prediction of the behavior of the representative point by a learning model with a relatively small computation load, and it is possible to reduce the computation load required for the behavior simulation of the operation target and to reduce the computation amount.

Construction example 2

[ hardware configuration ]

Fig. 2 is a diagram showing a hardware configuration of the simulation apparatus 10 according to the present embodiment. The simulation apparatus 10 includes a Central Processing Unit (CPU) 10a, a Random Access Memory (RAM) 10b, a Read Only Memory (ROM) 10c, a communication Unit 10d, an input Unit 10e, and a display Unit 10 f. The CPU10a, the RAM10b, the ROM10c, the communication section 10d, the input section 10e, and the display section 10f are connected to each other via a bus so as to be able to transmit and receive data to and from each other. In the present example, a case where the simulation apparatus 10 includes one computer is described, but the simulation apparatus 10 may be realized by combining a plurality of computers. The configuration shown in fig. 2 is an example, and the simulation apparatus 10 may have a configuration other than these, or may not have some of these configurations.

＜CPU＞

The CPU10a is a processor provided in the simulation apparatus 10, and performs control and data calculation and processing related to the execution of programs stored in the RAM10b and the ROM10 c. The CPU10a is an arithmetic unit that executes a program (simulation program) for generating a learning model that reproduces a simulation result of behavior of an operation target by a conventional simulation method. The CPU10a receives various data from the input unit 10e or the communication unit 10d, and displays the result of the data operation on the display unit 10f or stores the result in the RAM10b or the ROM10 c.

＜RAM＞

The RAM10b is a storage unit provided in the simulation apparatus 10, and can rewrite data. The RAM10b may include, for example, semiconductor memory elements. The RAM10b may store simulation programs executed by the CPU10a, data for constituting the robot or the surrounding environment in the simulation space, and the like. These are merely examples, and data other than these may be stored in the RAM10b, or some of these may not be stored.

＜ROM＞

The ROM10c is a storage unit provided in the simulation apparatus 10, and can read data. The ROM10c may include, for example, a semiconductor memory element. The ROM10c may store, for example, a simulation program or data not to be rewritten.

< communication section >

The communication unit 10d is an interface for connecting the simulation apparatus 10 to another device. The communication unit 10d can be connected to a communication Network such as a Local Area Network (LAN) or the internet.

< input part >

The input unit 10e receives data input from a user, and may include a pointing device such as a keyboard or a mouse, and a touch panel, for example.

< display part >

The Display unit 10f visually displays the operation result obtained by the CPU10a, and may include, for example, a Liquid Crystal Display (LCD). The display unit 10f may display a simulation result of the operation target or a simulation result of the entire robot and the operation target.

The simulation program may be stored in a computer-readable storage medium such as the RAM10b or the ROM10c, or may be provided via a communication network connected via the communication unit 10 d. In the simulation apparatus 10, the CPU10a executes a simulation program to realize the operations of the first processing unit 20 and the second processing unit 30 described with reference to the following drawings. These physical configurations are merely examples, and may not necessarily be independent configurations. For example, the simulation apparatus 10 may include a Large-Scale Integration (LSI) in which the CPU10a and the RAM10b or the ROM10c are integrated.

[ functional Structure ]

Fig. 3 is a diagram showing a functional configuration of the simulation apparatus 10 according to the present embodiment. The simulation apparatus 10 includes a first processing unit 20 and a second processing unit 30. The first processing unit 20 performs a process of generating a learning model for predicting the behavior of the operation target. The second processing unit 30 performs a simulation of the entire robot and the operation object using the generated learning model. The first processing unit 20 includes a first setting unit 11, a second setting unit 12, a first simulation unit 13, a generation unit 14, a learning unit 15, and a first storage unit 16. The second processing unit 30 includes a second simulation unit 17, a second storage unit 18, and a prediction unit 19.

< first setting part >

The first setting unit 11 sets a configuration condition of a model representing an operation target. Here, the operation object may be, for example, an object directly held by a robot hand, or an object indirectly held, or an object directly or indirectly operated by an end effector of a robot not limited to a hand, or an object directly or indirectly operated by a person. The configuration conditions of the model may include physical conditions for moving the model representing the operation object in the simulation space in the same manner as in reality, and may include any of conditions representing flexibility and conditions representing viscosity of the operation object. The operation object may include any one of a flexible object, a liquid, and a gas. The flexible material is a flexible material, and includes, for example, a cable, paper, or cloth made of rubber. When the object to be operated is a liquid or a gas, a container storing the liquid or the gas to be operated may be gripped by a robot. When the operation target is a gas, a robot may hold a nozzle for injecting the gas to be operated and blow the gas to another member.

According to the simulation apparatus 10 of the present embodiment, by generating the learning model for predicting the behavior of the representative point of the operation target, even if the operation target has a large computational load for simulation such as flexible objects, liquids, and gases, the behavior of the representative point of the operation target can be predicted by the computation of the learning model having a relatively small computational load, so that the computational load required for the behavior simulation of the operation target can be reduced, and the computational load can be reduced.

< second setting part >

The second setting unit 12 sets a condition for applying an external force to the operation object. The external force applied to the operation object may include an external force directly or indirectly applied to the operation object by the robot. The external force applied to the operation object may include an external force applied by a structure other than the robot, for example, an external force applied by the conveying device.

< first analog part >

The first simulation unit 13 simulates the behavior of the operation target under the configuration conditions of the model representing the operation target and the conditions of the external force applied to the operation target. The first simulation unit 13 may simulate the behavior of the operation target by using any method, for example, a particle method or a finite element method.

< generation part >

The generation unit 14 generates learning data including configuration conditions of a model of the operation target, conditions of an external force applied to the operation target, and data indicating behaviors of a plurality of representative points located on the surface of the operation target in the simulation performed by the first simulation unit 13. The learning data generated by the generation unit 14 may be stored as learning data 16a in the first storage unit 16. Here, the plurality of representative points may be located at any part of the operation object, but when located on the surface of the operation object, the behavior of the outline of the operation object can be represented by relatively few representative points. The generation unit 14 may generate not only learning data including data indicating behaviors of a plurality of representative points located on the surface of the operation object, but also learning data including configuration conditions of a model of the operation object, conditions of an external force applied to the operation object, and data indicating behaviors of a plurality of representative points located on a part of the operation object in the simulation performed by the first simulation unit 13.

The generation unit 14 may generate learning data including, as input data, configuration conditions of a model representing the operation target, conditions of an external force applied to the model representing the operation target at a predetermined time of the simulation performed by the first simulation unit 13, and data representing behaviors of a plurality of representative points at the predetermined time, and including, as output data, data representing behaviors of a plurality of representative points at a time after a predetermined time has elapsed from the predetermined time. Here, the predetermined time may be the minimum time in the simulation performed by the first simulation unit 13, or may be several times the minimum time. In this way, the behavior of the representative point after the elapse of the predetermined time can be predicted by the learning model, and the behavior of the representative point at an arbitrary time can be predicted by repeating the prediction by the learning model. The input data included in the learning data is data input to the learning model in supervised learning of the learning model. The output data included in the learning data is data to be compared with the output of the learning model in the supervised learning of the learning model, and is data indicating a positive solution.

When the operation target is a liquid, the surface of the operation target is a liquid surface of the liquid, and the behavior of the plurality of representative points can be expressed by the behavior of the liquid surface at a plurality of positions in the vertical direction. By predicting the behavior of a plurality of representative points located on the liquid surface using the learning model, the computational load of the learning model can be reduced compared to the case of predicting the behavior of the entire liquid, and the computational complexity can be reduced.

When the object to be operated is a liquid, the first simulation unit 13 may simulate the behavior of a plurality of particles using a model in which the liquid to be operated is decomposed into a plurality of particles. In this case, the configuration conditions of the model representing the operation target may include conditions relating to the number of particles, the size of the particles, and the interaction between the particles, and the first simulation unit 13 may simulate the behavior of a plurality of particles representing the operation target by a particle method. The data indicating the behaviors of the plurality of representative points located on the liquid surface may be data indicating the behaviors of a part of the particles located on the liquid surface at a predetermined time of the simulation performed by the first simulation unit 13 among the plurality of particles. In this way, a learning model that reproduces the behavior of the particles positioned on the liquid surface from the results of simulating the behavior of the entire liquid by the particle model can be generated, and the computational load on the learning model can be reduced compared to the case of predicting the behavior of the entire liquid, thereby reducing the amount of computation.

The first simulation unit 13 may simulate the behavior of the node using a model in which the operation target is decomposed into a plurality of constituent elements connected by the node. In this case, the configuration conditions of the model representing the operation object may include conditions relating to the number of the component elements and nodes, the sizes of the component elements, and the shapes of the component elements, and the first simulation unit 13 may simulate the behavior of the plurality of nodes representing the operation object by the finite element method. Also, a plurality of representative points may be located on the surface of the operation object. Also, the number of the plurality of representative points located on the surface of the operation object may be less than the number of the nodes. In this way, a learning model that reproduces behaviors of a part of nodes located on the surface of the operation target among the results of simulating the behaviors of the entire operation target by the finite element method can be generated, and the computation load of the learning model can be reduced compared to the case of predicting the behaviors of the entire nodes representing the operation target, thereby reducing the computation amount.

< department of learning >

The learning unit 15 generates a learning model by supervised learning using learning data, which receives as input a configuration condition of a model representing the operation target, a condition of an external force applied to the operation target, and initial conditions of a plurality of representative points located on the surface of the operation target, and outputs data representing behaviors of the plurality of representative points. The learning model generated by the learning unit 15 may be stored as the learning model 16b in the first storage unit 16. The learning model 16b generated by the learning unit 15 may be, for example, a model using a neural network, and the learning model 16b may be generated by optimizing weight parameters and the like of the neural network with respect to learning data by an error back propagation method.

< second analog part >

The second simulation unit 17 simulates the behavior of the robot. In the present embodiment, the second simulation unit 17 simulates the behavior of the robot. The second simulation unit 17 simulates the operation of the hand in the simulation space with reference to the hand data 18a stored in the second storage unit 18. The robot data 18a may include data relating to the trajectory of the robot taught by the teaching, data relating to the size of the arm constituting the robot, data relating to the torque of the servomotor constituting the robot, and the like. The second simulation unit 17 may also perform simulation of the surrounding environment in accordance with the operation of the robot by referring to the environment data 18b stored in the second storage unit 18. The environment data 18b may include data relating to the temperature, humidity, and the like of the environment in which the robot is installed, data relating to a conveying device, a processing device, and the like used together with the robot, data relating to an operator or an obstacle that may interfere with the robot, and the like. The second simulation unit 17 may simulate the operation of the robot in cooperation with the conveying device or the processing device while avoiding interference with the operator or an obstacle.

< part of prediction >

The prediction unit 19 inputs, to the learning model 16b, the condition of the external force applied to the operation target by the robot in the simulation performed by the second simulation unit 17 and the configuration condition of the model representing the operation target, and predicts the behaviors of the plurality of representative points. The prediction unit 19 predicts the behaviors of a plurality of representative points located on the surface of the operation target, based on the data output from the learning model 16b, instead of directly simulating the behavior of a model representing the operation target.

The second simulation unit 17 synthesizes the behavior of the plurality of representative points predicted by the prediction unit 19 and the simulated behavior of the robot. For example, when the object to be operated is a liquid contained in a container and the motion of holding and moving the container by a robot is simulated, the motion of the robot is simulated by the second simulation unit 17, the behavior of the liquid surface of the liquid contained in the container is predicted by the prediction unit 19, and the second simulation unit 17 synthesizes these motions to simulate the whole of the robot and the object to be operated. In this way, by learning the model, the behavior of the representative point of the model representing the operation target can be predicted with a relatively small computational load, and the behavior can be synthesized with the result of simulating the behavior of the manipulator by an arbitrary method.

Action example 3

Fig. 4 is a flowchart of a first process executed by the simulation apparatus 10 of the present embodiment. The first process is a process executed by the first processing unit 20 of the simulation apparatus 10, and is a process of generating a learning model for predicting the behavior of a plurality of representative points of the operation target.

First, the simulation apparatus 10 sets the configuration conditions of the model indicating the operation target (S10), sets the conditions of the external force applied to the operation target (S11), and sets the initial conditions of the operation target (S12).

Subsequently, the simulation apparatus 10 simulates the behavior of the operation target over a predetermined time period (S13). Here, the predetermined time may be a minimum time in the simulation, or may be a time several times the minimum time.

The simulation apparatus 10 acquires values indicating the behavior of a plurality of representative points located on the surface of the operation target (S14). The simulation device 10 generates learning data including, as input data, the model configuration condition of the operation target, the condition of the external force at the initial time of the simulation, and data indicating the behaviors of the plurality of representative points at the initial time, and including, as output data, data indicating the behaviors of the plurality of representative points at a time after a predetermined time has elapsed from the initial time (S15).

Subsequently, if the generation of the learning data is not finished (no in S16), the simulation apparatus 10 again performs the setting of the configuration conditions (S10), the setting of the external force conditions (S11), and the setting of the initial conditions (S12), simulates the behavior of the operation target (S13), and generates the learning data (S14, S15). Here, when the simulation is performed on the same operation target, the setting of the configuration condition (S10) may be omitted. In addition, when successive behavior simulations are performed on the same operation target, the initial condition of the operation target in the next simulation can be determined based on data indicating the behavior of the operation target obtained by the previous simulation.

On the other hand, when the generation of the learning data is finished (yes in S16), the simulation device 10 generates and stores a learning model that outputs data indicating the behavior of the plurality of representative points, using the learning data (S17). By the above operation, the first process is ended.

Fig. 5 is a flowchart of a second process executed by the simulation apparatus 10 of the present embodiment. The second process is a process executed by the second processing unit 30 of the simulation apparatus 10, and is a process of simulating the whole including the manipulator and the operation object using the learning model generated by the first process.

First, the simulation apparatus 10 sets the configuration conditions of the model indicating the operation target (S18), sets the initial conditions of the operation target (S19), and sets the conditions of the external force applied to the operation target (S20). Here, the condition of the external force applied to the operation object may be determined based on a predetermined operation of the manipulator.

Subsequently, the simulation apparatus 10 outputs data indicating the behavior of the operation target by learning the model (S21), and deforms the model indicating the operation target in the simulation space (S22). The simulation apparatus 10 simulates the behavior of the robot and the surrounding environment over a predetermined time period, and synthesizes the behavior with the deformation of the model representing the operation target (S23). Here, the predetermined time may be the same as the predetermined time in the first processing, and may be a minimum time in the simulation, or a time several times the minimum time.

If the simulation is not finished (no in S24), the simulation apparatus 10 sets a condition for applying an external force to the operation target based on the next operation of the manipulator (S20), predicts the behavior of the operation target by learning the model (S21), deforms the model representing the operation target (S22), and synthesizes the behavior of the manipulator and the surrounding environment and the deformation of the model representing the operation target (S23).

On the other hand, when the simulation is finished (yes in S24), the simulation device 10 writes the simulation result in the storage unit or displays the simulation result on the display unit, thereby ending the second process.

Fig. 6 is a flowchart of a process executed by the conventional simulation apparatus. The processing performed by the conventional simulation apparatus differs from the second processing performed by the simulation apparatus 10 of the present embodiment in that the processing for simulating the behavior of the operation target is executed over a predetermined time period (S103).

In the process performed by the conventional simulation apparatus, first, configuration conditions of a model representing an operation target are set (S100), initial conditions of the operation target are set (S101), and conditions of external force applied to the operation target are set (S102). Here, the condition of the external force applied to the operation object may be determined based on a predetermined operation of the manipulator.

Subsequently, the conventional simulation apparatus simulates the behavior of the operation target over a predetermined time period (S103). Here, the process (S103) of simulating the behavior of the operation target over a predetermined time is a process performed using a finite element method or a particle method, and generally, a calculation load is large and a relatively long time is required until a result is obtained. A conventional simulation apparatus deforms a model representing an operation target in a simulation space based on a simulation result of the behavior of the operation target (S104), simulates the behavior of a robot and a surrounding environment over a predetermined time, and synthesizes the simulated behavior with the deformation of the model representing the operation target (S105).

If the simulation is not finished (no in S106), the conventional simulation apparatus sets a condition for applying an external force to the operation object based on the next operation of the manipulator (S102), simulates the behavior of the operation object over a predetermined time (S103), deforms the model representing the operation object (S104), and synthesizes the behavior of the manipulator and the surrounding environment and the deformation of the model representing the operation object (S105).

In this way, the conventional simulation apparatus simulates the behavior of the operation target over a predetermined time period by the finite element method or the particle method every time the simulation is advanced for the predetermined time period. On the other hand, the simulation apparatus 10 according to the present embodiment calculates the behavior of the representative point of the operation target by the learning model every time the simulation is advanced for a predetermined time. Due to such a difference, the simulation apparatus 10 of the present embodiment can reduce the calculation load required for the behavior simulation of the operation target compared to the conventional simulation apparatus, and can reduce the calculation amount.

Fig. 7 is a schematic diagram of the behavior of the second manipulation object T2 simulated by the simulation apparatus 10 according to the present embodiment. The second operation object T2 is paper or cloth, and is a flexible object. The second manipulation object T2 is held by the robot and moved. When the second manipulation object T2 is gripped and moved by the robot, the simulation device 10 simulates the movement of the robot, that is, moves the second manipulation object T2, which is a flexible object, in a short time without being damaged or entangled. In fig. 7, the left side shows the second manipulation object T2 in a stationary state, and the right side shows a state in which the second manipulation object T2 is deformed while being held by the robot.

The simulation apparatus 10 sets the configuration conditions of the model indicating the second manipulation object T2 and the conditions of the external force applied to the second manipulation object T2, and simulates the behavior of the second manipulation object T2 by the conventional simulation method under these conditions. The simulation apparatus 10 simulates the behavior of a plurality of nodes by a so-called finite element method using, for example, a model in which the second manipulation object T2, which is a flexible object, is decomposed into a plurality of constituent elements connected by nodes. In this case, the configuration conditions of the model may include conditions relating to the number of the component elements and nodes, the sizes of the component elements, and the shapes of the component elements. Also, the external force condition may include a condition of a force applied by the robot.

The simulation apparatus 10 may use, as the learning data, data indicating the behavior of a part of the nodes located on the surface of the second manipulation object T2, among the behaviors of the plurality of nodes simulated by the finite element method. Here, the data indicating the behavior of the node may include data indicating at least one of a position, a velocity, a motion amount, and an acceleration of the node.

In fig. 7, a first node p11, a second node p12, a third node p13, a fourth node sub p14 and a fifth node p15 are shown as a part of nodes located on the surface of the second operand T2. These are just examples, and nodes having a plurality of representative points may include points other than these. In this example, when the robot holds the second manipulation object T2, the first node p11, the second node p12, the third node p13, the fourth node sub p14, and the fifth node p15 are shifted from a state in which they are positioned on the left side of a straight line to a state in which the positions of the first node p11, the second node p12, the third node p13, the fourth node sub p14, and the fifth node p15 are positioned on the right side of the straight line.

The simulation apparatus 10 generates the learning data including the configuration conditions of the model of the second manipulation object T2, the conditions of the external force applied to the second manipulation object T2, and the data indicating the behavior of the plurality of representative points (the first to fifth nodes) on the surface of the second manipulation object T2 in the simulation. The simulation apparatus 10 generates a learning model by supervised learning using learning data, which receives as input the configuration conditions of the model indicating the second manipulation object T2, the conditions of the external force applied to the second manipulation object T2, and the initial conditions of the plurality of representative points (first to fifth nodes), and outputs data indicating the behavior of the plurality of representative points (first to fifth nodes). The learning model predicts at least one of the position, the speed, the motion amount, and the acceleration of the plurality of representative points, instead of directly performing a simulation for reproducing the physical phenomenon.

In this way, by simulating the behavior of the model representing the second manipulation object T2 by an arbitrary method and generating a learning model by supervised learning using the simulation result, it is possible to replace a simulation with a large computation load with the prediction of the behavior of the representative point by a learning model with a relatively small computation load, and it is possible to reduce the computation load required for the simulation of the entire body including the robot and the manipulation object and to reduce the computation amount.

The embodiments described above are for the purpose of facilitating understanding of the present invention, and are not intended to be restrictive. The elements included in the embodiments and their arrangement, materials, conditions, shapes, sizes, and the like are not limited to those exemplified, and can be appropriately changed. Also, the structures shown in different embodiments can be partially replaced or combined with each other.

[ Note 1]

A simulation apparatus (10) comprising:

a first setting unit (11) that sets a configuration condition of a model representing an operation target;

a second setting unit (12) for setting a condition of an external force applied to the operation object;

a first simulation unit (13) that simulates the behavior of the operation target under the configuration conditions and the external force conditions;

a generation unit (14) that generates learning data including the configuration conditions, the external force conditions, and data indicating behaviors of a plurality of representative points on the surface of the operation object in the simulation performed by the first simulation unit (13); and

and a learning unit (15) that generates a learning model by supervised learning using the learning data, the learning model having the configuration conditions, the external force conditions, and the initial conditions of the plurality of representative points as inputs, and outputs data indicating behaviors of the plurality of representative points.

[ Note 2]

The simulation apparatus (10) of claim 1, wherein

The learning data generated by the generation unit (14) includes, as input data, the configuration condition, the external force condition at a predetermined time of the simulation performed by the first simulation unit (13), and data indicating behaviors of the plurality of representative points at the predetermined time, and includes, as output data, data indicating behaviors of the plurality of representative points at a time after a predetermined time has elapsed from the predetermined time.

[ Note 3]

The simulation apparatus (10) of claim 1 or 2, further comprising:

a second simulation unit (17) for simulating the behavior of the robot; and

a prediction unit (19) that predicts the behavior of the plurality of representative points by inputting, to the learning model, the condition of the external force applied to the operation object by the robot in the simulation performed by the second simulation unit (17) and the constituent condition,

the second simulation unit (17) synthesizes the behavior of the representative points predicted by the prediction unit (19) and the simulated behavior of the robot.

[ Note 4]

Simulation device (10) according to any of the claims 1 to 3, wherein

The operation object includes any one of a flexible object, a liquid, and a gas.

[ Note 5]

Simulation device (10) according to claim 4, wherein

When the operation target is a liquid, the surface of the operation target is a liquid surface of the liquid, and the behaviors of the plurality of representative points are expressed by behaviors of the liquid surface in a vertical direction at a plurality of positions.

[ Note 6]

Simulation device (10) according to claim 5, wherein

The first simulation unit (13) simulates the behavior of a plurality of particles using the model that decomposes the liquid into the plurality of particles,

the data indicating the behaviors of the plurality of representative points is data indicating the behavior of a part of the particles located on the liquid surface of the liquid at a predetermined timing of the simulation performed by the first simulation unit (13) among the plurality of particles.

[ Note 7]

Simulation device (10) according to any of the claims 1 to 6, wherein

The configuration condition includes any one of a condition indicating flexibility and a condition indicating viscosity of the operation object.

[ Note 8]

Simulation device (10) according to any of the claims 1 to 7, wherein

The first simulation unit (13) simulates the behavior of the node using the model in which the operation object is decomposed into a plurality of constituent elements connected by nodes,

the number of the plurality of representative points is less than the number of the nodes.

[ Note 9]

A method of simulation, comprising:

setting a configuration condition of a model representing an operation object;

setting a condition of an external force applied to the operation object;

simulating the behavior of the operation object under the composition condition and the external force condition;

generating learning data including the configuration condition, the external force condition, and data indicating behaviors of a plurality of representative points on the surface of the operation object in the simulation performed by the first simulation unit (13); and

and a learning model that outputs data representing behaviors of the plurality of representative points by using the configuration condition, the external force condition, and the initial conditions of the plurality of representative points as inputs is generated by supervised learning using the learning data.

[ Note 10]

A simulation program causes a processor provided in a simulation device (10) to function as:

a first setting unit (11) that sets a configuration condition of a model representing an operation target;

a second setting unit (12) for setting a condition of an external force applied to the operation object;

a first simulation unit (13) that simulates the behavior of the operation target under the configuration conditions and the external force conditions;

a generation unit (14) that generates learning data including the configuration conditions, the external force conditions, and data indicating behaviors of a plurality of representative points on the surface of the operation object in the simulation performed by the first simulation unit (13); and

and a learning unit (15) that generates a learning model by supervised learning using the learning data, the learning model having the configuration conditions, the external force conditions, and the initial conditions of the plurality of representative points as inputs, and outputs data indicating behaviors of the plurality of representative points.