阅读说明：本技术 信息处理方法、信息处理系统以及信息处理装置 (Information processing method, information processing system, and information processing apparatus ) 是由 穴吹元嗣 本田义雅 于 2020-12-23 设计创作，主要内容包括：一种信息处理方法,使计算机执行如下处理,从自主移动体获得作为自主移动体的自主移动处理中的行驶控制处理的第1预处理的结果的第1处理结果以及由自主移动体获得的传感数据(步骤S11),根据传感数据,执行作为比第1预处理高级的预处理的第2预处理,获得第2处理结果(步骤S12),对第1处理结果与第2处理结果的差异进行判断(步骤S15),按照判断出的差异,将使第1处理结果变更为第3处理结果的变更指示,输出给自主移动体(步骤S17),第3处理结果是根据第1处理结果以及第2处理结果的至少其中一方而得到的结果。(An information processing method for causing a computer to execute processing for obtaining a 1 st preprocessing result, which is a result of a 1 st preprocessing in an autonomous moving process of an autonomous moving body, and sensed data obtained by the autonomous moving body from the autonomous moving body (step S11), executing a 2 nd preprocessing, which is a preprocessing higher than the 1 st preprocessing, based on the sensed data to obtain a 2 nd processing result (step S12), determining a difference between the 1 st processing result and the 2 nd processing result (step S15), and outputting a change instruction for changing the 1 st processing result to a 3 rd processing result according to the determined difference to the autonomous moving body (step S17), wherein the 3 rd processing result is a result obtained based on at least one of the 1 st processing result and the 2 nd processing result.)

1. An information processing method for causing a computer to execute a process,

obtaining, from an autonomous mobile body, a 1 st processing result as a result of a 1 st preprocessing, which is a preprocessing of a travel control processing in an autonomous movement processing of the autonomous mobile body, and sensing data obtained by the autonomous mobile body,

performing a 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, obtaining a 2 nd processing result,

determining a difference between the 1 st processing result and the 2 nd processing result,

and outputting a change instruction to change the 1 st process result to a 3 rd process result to the autonomous moving object according to the determined difference, wherein the 3 rd process result is obtained based on at least one of the 1 st process result and the 2 nd process result.

2. The information processing method according to claim 1,

the 1 st pre-processing, performed using the 1 st resource,

the 2 nd preprocessing, performed using the 2 nd resource,

the 1 st resource is a different resource than the 2 nd resource.

3. The information processing method according to claim 1 or 2,

the 1 st pre-processing, performed using the 1 st algorithm,

the 2 nd preprocessing, performed using the 2 nd algorithm,

the 1 st algorithm is a different algorithm than the 2 nd algorithm.

4. The information processing method according to any one of claims 1 to 3,

the 1 st preprocessing and the 2 nd preprocessing include a recognition processing of recognizing an environment in which the autonomous moving body is located.

5. The information processing method according to any one of claims 1 to 4,

the 1 st and 2 nd pretreatments include a travel determination process of the autonomous moving body.

6. The information processing method according to any one of claims 1 to 5,

the 3 rd processing result is the 2 nd processing result.

7. The information processing method according to any one of claims 1 to 5,

the 3 rd processing result is a result of correcting the 1 st processing result based on the difference.

8. The information processing method according to any one of claims 1 to 7,

executing the 2 nd preprocessing to obtain a 2 nd processing result when the request for execution of the 2 nd preprocessing is accepted from the autonomous moving body,

outputting the change instruction to the autonomous mobile unit as a response to the request.

9. The information processing method according to claim 8,

the request includes information specifying a specific process among the 2 nd preprocessing,

and executing the specific processing to obtain the 2 nd processing result.

10. The information processing method according to claim 8 or 9,

in a case where the request is accepted, a judgment is made as to whether or not the 2 nd preprocessing is executed,

and rejecting or ignoring the request under the condition that the No. 2 preprocessing is judged not to be executed.

11. The information processing method according to claim 10,

the determination as to whether or not to execute the 2 nd preprocessing is made in accordance with at least one of a resource that the autonomous mobile body has, a movement state of the autonomous mobile body, an external environment of the autonomous mobile body, a time of day, and a response time to the request.

12. An information processing system capable of communicating with an autonomous mobile unit, wherein,

obtaining, from the autonomous mobile body, a 1 st processing result as a result of a 1 st preprocessing, which is a preprocessing of a travel control processing in autonomous movement of the autonomous mobile body, and the sensor data obtained by the autonomous mobile body,

performing a 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, obtaining a 2 nd processing result,

determining a difference between the 1 st processing result and the 2 nd processing result,

and outputting a change instruction to change the 1 st process result to a 3 rd process result to the autonomous moving object according to the determined difference, wherein the 3 rd process result is obtained based on at least one of the 1 st process result and the 2 nd process result.

13. An information processing apparatus mounted on an autonomous moving body, wherein the information processing apparatus,

executing 1 st preprocessing for obtaining a 1 st processing result, the 1 st preprocessing being preprocessing of a travel control processing in autonomous movement of the autonomous moving body,

outputting the sensor data obtained by the autonomous moving body to an external device,

obtaining a 2 nd processing result from the external device, the 2 nd processing result being a result of performing a 2 nd preprocessing which is a preprocessing higher than the 1 st preprocessing based on the sensing data,

determining a difference between the 1 st processing result and the 2 nd processing result,

and changing the 1 st processing result into a 3 rd processing result according to the determined difference, wherein the 3 rd processing result is obtained according to at least one of the 1 st processing result and the 2 nd processing result.

14. The information processing apparatus according to claim 13,

outputting a request for execution of the 2 nd preprocessing to the external device,

obtaining the 2 nd processing result as a response to the request.

15. The information processing apparatus according to claim 14,

outputting the request to the external device, based on at least one of a resource possessed by the autonomous mobile body, a movement state of the autonomous mobile body, an external environment of the autonomous mobile body, a time of day, and a response time to an inquiry to the external device.

16. The information processing apparatus according to claim 14 or 15,

the request includes information specifying a specific process among the 2 nd preprocessing,

the 2 nd processing result is a result of executing the specific processing.

17. The information processing apparatus according to any one of claims 14 to 16,

when the 2 nd processing result is obtained after a predetermined time or more has elapsed from the output of the sensing data or the output of the request,

(A) the determination of the difference is not performed, or

(B) A difference between a part of the 1 st processing result and a processing result corresponding to the part of the 2 nd processing result is determined, and the part of the 1 st processing result is changed to the 3 rd processing result according to the determined difference.

Technical Field

The present disclosure relates to an information processing method, an information processing system, and an information processing apparatus relating to autonomous movement of an autonomous moving body.

Background

In an automatic driving system of an automatic driving vehicle, it is desirable to overlap hardware or software for improving safety. For example, patent documents 1 and 2 propose a method for improving safety by superimposing systems such as a power supply, a detection function, and a control function on an autonomous vehicle.

(Prior art document)

(patent document)

Patent document 1: international publication No. 2018/154860

Patent document 2: japanese patent No. 3881197

Disclosure of Invention

Problems to be solved by the invention

However, from the viewpoints of cost, power consumption, space, and the like, there are limitations on a computer that can be mounted on an autonomous moving body such as an autonomous vehicle, and the performance of an autonomous moving system mounted on the autonomous moving body may be low. Therefore, in the methods disclosed in patent documents 1 and 2, although the safety of autonomous movement can be improved by overlapping hardware or software mounted on the autonomous moving body, there are cases where the performance of autonomous movement is insufficient.

Accordingly, the present disclosure provides an information processing method and the like capable of improving performance of autonomous movement.

Means for solving the problems

An information processing method according to the present disclosure causes a computer to execute processing for obtaining a 1 st processing result as a result of a 1 st preprocessing and sensing data obtained by an autonomous moving body from the autonomous moving body, the 1 st preprocessing is a preprocessing of a travel control processing in the autonomous moving process of the autonomous moving body, performing a 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, obtaining a 2 nd processing result, determining a difference between the 1 st processing result and the 2 nd processing result, and outputting a change instruction to change the 1 st processing result to the 3 rd processing result to the autonomous moving body according to the determined difference, the 3 rd processing result is obtained from at least one of the 1 st processing result and the 2 nd processing result.

These general or specific aspects can be realized by a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or any combination of the system, the method, the integrated circuit, the computer program, and the recording medium.

Effects of the invention

With the information processing method and the like according to one aspect of the present disclosure, the performance of autonomous movement can be improved.

Drawings

Fig. 1 is a block diagram showing an example of an autonomous vehicle and a remote autonomous server according to embodiment 1.

Fig. 2 is a flowchart showing an example of the information processing method according to embodiment 1.

Fig. 3 is a flowchart showing another example of the information processing method according to embodiment 1.

Fig. 4 is a block diagram showing an example of an autonomous vehicle, a remote autonomous driving server, and a remote processing management server according to a modification of embodiment 1.

Fig. 5 is a flowchart showing an example of an information processing method according to a modification of embodiment 1.

Fig. 6 is a block diagram showing an example of an autonomous vehicle and a remote autonomous server according to embodiment 2.

Fig. 7 is a flowchart showing an example of the operation of the autonomous vehicle according to embodiment 2.

Fig. 8 is a block diagram showing an example of an autonomous vehicle, a remote autonomous driving server, and a remote processing management server according to a modification of embodiment 2.

Fig. 9 is a flowchart showing an example of the operation of the autonomous vehicle according to the modification of embodiment 2.

Fig. 10 is a flowchart showing an example of an information processing method according to a modification common to the respective embodiments.

Detailed Description

An information processing method according to an aspect of the present disclosure causes a computer to execute processing for obtaining a 1 st processing result as a result of a 1 st preprocessing and sensing data obtained by an autonomous mobile body, the 1 st preprocessing is a preprocessing of a travel control processing in the autonomous moving process of the autonomous moving body, performing a 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, obtaining a 2 nd processing result, determining a difference between the 1 st processing result and the 2 nd processing result, and outputting a change instruction to change the 1 st processing result to the 3 rd processing result to the autonomous moving body according to the determined difference, the 3 rd processing result is obtained from at least one of the 1 st processing result and the 2 nd processing result.

Unlike the 1 st preprocessing executed by an autonomous moving body such as an autonomous vehicle with a limited computer that can be mounted in terms of cost, power consumption, space, and the like, the 2 nd preprocessing that is higher than the 1 st preprocessing is executed by a server or the like with less restrictions on cost, power consumption, space, and the like, and a difference between the 1 st processing result that is a result of the 1 st preprocessing and the 2 nd processing result that is a result of the 2 nd preprocessing is determined. In accordance with the difference, the 1 st process result, which is the result of the 1 st process that is the result of the travel control process of the autonomous moving object, is changed to a 3 rd process result change instruction based on the 2 nd process result that is the result of the 2 nd process of the higher order, and the changed instruction is output to the autonomous moving object. Or, in accordance with the difference, a change instruction to change the 1 st processing result to the 3 rd processing result, which is a result obtained by correcting or limiting the 1 st processing result, is output to the autonomous moving body. Accordingly, the performance of autonomous movement can be improved by using the 3 rd processing result which is higher than the 1 st processing result in the travel control processing of the autonomous moving body.

The 1 st preprocessing may be performed using a 1 st resource, and the 2 nd preprocessing may be performed using a 2 nd resource, where the 1 st resource and the 2 nd resource are different resources.

Thus, in the 2 nd preprocessing which is higher than the 1 st preprocessing, the 2 nd preprocessing which is different from the 1 st preprocessing can be executed. Therefore, the 2 nd preprocessing can be processed with higher accuracy, higher speed, or lower delay than the 1 st preprocessing.

The 1 st preprocessing may be performed using a 1 st algorithm, and the 2 nd preprocessing may be performed using a 2 nd algorithm, where the 1 st algorithm and the 2 nd algorithm are different algorithms.

Thus, in the 2 nd preprocessing which is more advanced than the 1 st preprocessing, it is possible to perform the 1 st preprocessing using a 2 nd algorithm different from the 1 st algorithm used in performing the 1 st preprocessing. Therefore, the 2 nd preprocessing can be a high-precision, high-speed, low-delay or multi-functional processing than the 1 st preprocessing.

The 1 st preprocessing and the 2 nd preprocessing may include a recognition processing of recognizing an environment in which the autonomous moving body is located.

Each of the preprocessing may include recognition processing as described above, or the result of the recognition processing may be used in the travel control processing of the autonomous moving body. Therefore, the safety or comfort of the autonomous moving body can be improved.

The 1 st and 2 nd preprocessing may include a travel determination process of the autonomous moving body.

The respective pre-processes may include a process of determining the travel of the autonomous mobile unit, or may use the result of the process of determining the travel of the autonomous mobile unit in the process of controlling the travel of the autonomous mobile unit. Therefore, the safety or comfort of the autonomous moving body can be improved.

Further, the 3 rd processing result may be the 2 nd processing result.

As described above, in the travel control process of the autonomous moving body, the 2 nd processing result, which is a result of the 2 nd preprocessing that is higher than the 1 st preprocessing, is used, so that the performance of autonomous movement can be improved.

Further, the 3 rd processing result may be a result of correcting the 1 st processing result based on the difference.

As described above, since the 3 rd processing result obtained by correcting the 1 st processing result based on the difference between the 2 nd processing result and the 1 st processing result, which is the result of the 2 nd preprocessing higher than the 1 st preprocessing, is used in the travel control of the autonomous moving body, the performance of autonomous movement can be improved.

When a request for execution of the 2 nd preprocessing is received from the autonomous mobile unit, the 2 nd preprocessing may be executed, the 2 nd processing result may be obtained, and the change instruction may be output to the autonomous mobile unit as a response to the request.

As described above, for example, the 2 nd preprocessing can be executed at a timing when the autonomous moving body desires to execute the 2 nd preprocessing at a high level.

Further, the request may include information specifying a specific process in the 2 nd preprocessing, and the specific process may be executed to obtain the 2 nd processing result.

As described above, it is possible to selectively execute, for example, a specific process desired by the autonomous moving body specified in the 2 nd preprocessing.

Further, it may be determined whether or not the 2 nd preprocessing is executed when the request is accepted, and the request may be rejected or ignored when it is determined that the 2 nd preprocessing is not executed. For example, the determination as to whether or not to execute the 2 nd preprocessing may be made in accordance with at least one of a resource that the autonomous mobile body has, a moving state of the autonomous mobile body, an external environment of the autonomous mobile body, a time of day, and a response time to the request.

When the request is received, depending on the situation, the performance of autonomous movement may not be improved even if the 2 nd preprocessing is performed. For example, even if the 2 nd preprocessing is executed to perform the travel control processing of the autonomous mobile body using the 3 rd processing result according to the resource possessed by the autonomous mobile body, the moving state of the autonomous mobile body, the external environment of the autonomous mobile body, the time, and the response time to the request, the performance of the autonomous movement may not be improved. In such a case, the request can be denied or ignored.

An information processing system according to an aspect of the present disclosure is an information processing system capable of communicating with an autonomous mobile unit, in the information processing system, a 1 st processing result as a result of the 1 st preprocessing and the sensed data obtained by the autonomous mobile body are obtained from the autonomous mobile body, the 1 st preprocessing is preprocessing of a travel control processing in autonomous movement of the autonomous moving body, performing a 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, obtaining a 2 nd processing result, determining a difference between the 1 st processing result and the 2 nd processing result, and outputting a change instruction to change the 1 st processing result to the 3 rd processing result to the autonomous moving body according to the determined difference, the 3 rd processing result is obtained from at least one of the 1 st processing result and the 2 nd processing result.

With the above, an information processing system capable of improving performance of autonomous movement can be provided.

An information processing apparatus according to an aspect of the present disclosure is an information processing apparatus mounted on an autonomous moving body, in the information processing apparatus, a 1 st preprocessing is executed to obtain a 1 st processing result, the 1 st preprocessing being a preprocessing of a travel control processing in an autonomous movement of the autonomous moving body, sensing data obtained by the autonomous moving body is output to an external apparatus, a 2 nd processing result is obtained from the external apparatus, the 2 nd processing result is a result of performing 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, determining a difference between the 1 st processing result and the 2 nd processing result, and changing the 1 st processing result to a 3 rd processing result according to the determined difference, the 3 rd processing result is obtained from at least one of the 1 st processing result and the 2 nd processing result.

For example, unlike the 1 st preprocessing executed by an autonomous mobile unit such as an autonomous vehicle with a limited computer that can be mounted in terms of cost, power consumption, space, and the like, the 2 nd preprocessing that is higher than the 1 st preprocessing is executed by an external device such as a server with less restrictions on cost, power consumption, space, and the like, the 2 nd processing result that is the result of the 2 nd preprocessing is transmitted to the autonomous vehicle, and the autonomous vehicle determines the difference between the 1 st processing result that is the result of the 1 st preprocessing and the 2 nd processing result that is the result of the 2 nd preprocessing. In accordance with the difference, the 1 st process result, which is a result of the 1 st process in the traveling control process of the autonomous moving body, is changed to a 3 rd process result based on the 2 nd process result, which is a result of the 2 nd process of the higher order. Or, according to the difference, the 1 st processing result is changed to the 3 rd processing result obtained by correcting or limiting the 1 st processing result. Accordingly, the performance of autonomous movement can be improved by using the 3 rd processing result which is higher than the 1 st processing result in the travel control processing of the autonomous moving body.

Further, a request for execution of the 2 nd preprocessing may be output to the external device, and the 2 nd processing result may be obtained as a response to the request.

As described above, for example, when the autonomous moving body desires to execute the 2 nd preprocessing at a high-level timing, the external device can be caused to execute the 2 nd preprocessing.

The request may be output to the external device based on at least one of a resource possessed by the autonomous mobile body, a movement state of the autonomous mobile body, an external environment of the autonomous mobile body, a time of day, and a response time to an inquiry to the external device.

Depending on the situation, the performance of autonomous movement may not be improved even if the 2 nd preprocessing is performed. For example, even if the 2 nd preprocessing is executed to perform the travel control processing of the autonomous mobile body using the 3 rd processing result according to the resource possessed by the autonomous mobile body, the moving state of the autonomous mobile body, the external environment of the autonomous mobile body, the time, and the response time to the request, the performance of the autonomous movement may not be improved. In such a case, the request may not be output. In other words, depending on the situation, the performance of autonomous movement can be improved by performing the 2 nd preprocessing in some cases. In a situation where the performance of autonomous movement can be improved in this way, the request can be output.

Further, the request may include information specifying a specific process in the 2 nd preprocessing, and the 2 nd processing result may be a result of executing the specific process.

As described above, by specifying a specific process desired by the autonomous moving body, for example, in the 2 nd preprocessing, it is possible to cause an external device to selectively execute the specific process.

When the 2 nd processing result is obtained after a predetermined time or more has elapsed from the output of the sensing data or the output of the request, (a) the determination of the difference is not performed, or (B) a difference between a part of the 1 st processing result and a processing result corresponding to the part of the 2 nd processing result is determined, and the part of the 1 st processing result may be changed to the 3 rd processing result in accordance with the determined difference.

As described above, when the 2 nd processing result is obtained after a predetermined time or more has elapsed from the output of the sensing data or the output of the request, a communication delay may occur with the external device. In this case, the processing load of the information processing apparatus can be reduced by not performing the determination of the difference or performing the determination of the difference only on a part of the processing results that are not affected by the delay.

The following description refers to embodiments and the like with reference to the accompanying drawings.

In addition, the embodiments to be described below are general or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of the steps, and the like shown in the following embodiments are merely examples, and the present disclosure is not limited thereto.

(embodiment mode 1)

Embodiment 1 will be described with reference to fig. 1 to 3.

Fig. 1 is a block diagram showing an example of an autonomous vehicle (specifically, an information processing device 20 mounted on the autonomous vehicle) and a remote autonomous server 10 according to embodiment 1.

The autonomous vehicle is, for example, a vehicle that can travel automatically without a driving operation by a person. The autonomous driving vehicle includes sensors such as a camera, a thermal imager, a radar, a Light Detection and Ranging (optical radar), a sonar, a GPS (Global Positioning System), and an IMU (Inertial Measurement Unit), and can autonomously travel by recognizing the surrounding environment and the like. An autonomous vehicle is an example of an autonomous moving body. The autonomous moving body may be a mobile robot, a flight body such as a crane, or a ship.

The remote autopilot server 10 is capable of performing wireless communication with an autopilot and remotely controlling the autopilot of the autopilot. In addition, the autonomous driving is an example of autonomous movement.

The autonomous vehicle is mounted with an information processing device 20. The information processing apparatus 20 is a computer including a processor, a memory, a communication interface, and the like. The memory is rom (read Only memory), ram (random Access memory), or the like, and can store a program executed by the processor. The information processing device 20 includes: a sensing data obtaining unit 21, a sensing data transmitting unit 22, a vehicle automatic driving system 23, a 1 st processing result transmitting unit 24, a processing result changing unit 25, a communication confirming unit 26, and a travel restricting unit 27. The sensor data obtaining unit 21, the sensor data transmitting unit 22, the vehicle automatic driving system 23, the 1 st processing result transmitting unit 24, the processing result changing unit 25, the communication confirming unit 26, and the travel limiting unit 27 are realized by a processor or the like that executes a program stored in a memory.

The sensing data obtaining unit 21 obtains sensing data of a sensor such as a camera, a thermal imager, a radar, a LiDAR, a sonar, a GPS, or an IMU provided in the autonomous vehicle. The sensed data obtaining unit 21 may obtain sensed data of a sensor provided in another vehicle, a traffic light, or the like.

The sensing data transmitting unit 22 transmits the sensing data obtained by the sensing data obtaining unit 21 to the remote autopilot server 10. The sensor data transmitting unit 22 transmits the sensor data to the remote autopilot server 10 via, for example, a communication interface provided in the information processing device 20. The sensor data transmitting unit 22 can transmit sensor data with low delay by using, for example, a data compression technique and a high-speed transmission technique such as 5G.

The vehicle automatic driving system 23 executes the 1 st preprocessing, which is preprocessing of the travel control processing in the automatic driving processing of the automatically driven vehicle, based on the sensing data obtained by the sensing data obtaining unit 21, and obtains the 1 st processing result. The 1 st processing result is the result of the 1 st preprocessing. The preprocessing includes, for example, recognition processing or running determination processing of an autonomous vehicle. The recognition processing includes processing for recognizing an environment in which the autonomous vehicle is located. The environment is the self-position, the surrounding objects, the road surface state, the weather, or the road condition, etc. Specifically, the preprocessing includes estimation processing of the self position of the autonomous vehicle, detection processing of an object around the autonomous vehicle, movement prediction processing of an object around the autonomous vehicle, travel determination processing of the autonomous vehicle, route planning processing of the autonomous vehicle, and the like. For example, the vehicle automatic driving system 23 may perform these processes using a learning model. The processing result includes, for example, a recognition result or a travel determination result of the autonomous vehicle. Specifically, the processing result includes an estimation result of the self position of the autonomous vehicle, a detection result of an object around the autonomous vehicle, a movement prediction result of an object around the autonomous vehicle, a travel determination result of the autonomous vehicle, a route planning result of the autonomous vehicle, or the like.

The 1 st processing result transmitting unit 24 transmits the 1 st processing result, which is the result of the 1 st preprocessing performed by the vehicle automatic driving system 23, to the remote automatic driving server 10. The 1 st processing result transmitting unit 24 transmits the 1 st processing result to the remote autopilot server 10, for example, via a communication interface or the like provided in the information processing device 20.

The sensed data transmitting unit 22 may transmit only the sensed data used when the vehicle automatic driving system 23 executes the 1 st preprocessing, or may transmit not only the sensed data used when the vehicle automatic driving system 23 executes the 1 st preprocessing but also sensed data not used when the 1 st preprocessing is executed (for example, sensed data with high resolution which cannot be processed by the vehicle automatic driving system 23).

The processing result changing unit 25 changes the 1 st processing result to the 3 rd processing result in accordance with the change instruction received from the remote autopilot server 10. In other words, the process result changing unit 25 changes the 1 st process result to the 3 rd process result, and uses the 3 rd process result instead of the 1 st process result originally used for the travel control of the autonomous vehicle.

The communication confirming unit 26 confirms a communication state of wireless communication between the autonomous vehicle and the remote autonomous driving server 10. For example, the communication confirming unit 26 transmits an inquiry to the remote autopilot server 10 via a communication interface or the like provided in the information processing device 20, and confirms the communication state in response to the inquiry. Specifically, the communication confirming unit 26 can determine that there is no communication connection when there is no response to the inquiry, and can determine that a communication delay has occurred when there is a delay in the response to the inquiry. When determining that there is no communication connection or when determining that a communication delay has occurred, the communication confirmation unit 26 notifies the travel restriction unit 27 of a travel restriction instruction for restricting travel.

The travel restriction unit 27 restricts the travel of the autonomous vehicle in accordance with the travel restriction instruction. Specifically, the travel limiting unit 27 decelerates the autonomous vehicle, stops the autonomous vehicle, or increases the avoidance margin. Accordingly, even when the communication between the autonomous vehicle and the remote autonomous server 10 is cut off, the safety of the autonomous vehicle can be ensured.

The remote autopilot server 10 is a computer that includes a processor, memory, and a communication interface, among other things. The remote autopilot server 10 is an example of an information processing system that can wirelessly communicate with an autopilot. The memory is ROM, RAM, or the like, and can store a program executed by the processor. The remote autopilot server 10 includes a sensing data acquisition unit 11, a simulation autopilot system 12, a 1 st processing result acquisition unit 13, a difference determination unit 14, and a change instruction output unit 15. The sensing data obtaining unit 11, the simulated automatic driving system 12, the 1 st processing result obtaining unit 13, the difference determining unit 14, and the change instruction output unit 15 are realized by a processor or the like that executes a program stored in a memory. The components constituting the remote autopilot server 10 may be distributed among a plurality of servers.

The sensed data obtaining unit 11 obtains sensed data obtained by an autonomous vehicle from the autonomous vehicle. For example, the sensed data obtaining unit 11 obtains sensed data transmitted from the autonomous vehicle and received via a communication interface or the like provided in the remote autonomous server 10.

The simulation autopilot system 12 executes the 2 nd preprocessing, which is preprocessing of the travel control processing in the autopilot of the autonomous vehicle, based on the sensed data obtained by the sensed data obtaining unit 11, and obtains the 2 nd processing result. The 2 nd preprocessing is a preprocessing superior to the 1 st preprocessing. For example, the simulated autopilot system 12 may use a learning model to perform pre-process 2. The 2 nd processing result is the result of the 2 nd preprocessing.

For example, there are limited computers that can be mounted on an autonomous vehicle from the viewpoints of cost, power consumption, and space. Therefore, the resources used by the vehicle automatic driving system 23 are reduced, or the calculation amount of the algorithm is reduced. Here, the resource refers to the processing amount or processing speed of the processor, the storage capacity, the power, or the like. On the other hand, the remote autopilot server 10 has fewer restrictions on cost, power consumption, space, and the like. Therefore, resources used in the simulation automatic driving system 12 can be increased, or the amount of calculation of the algorithm can be increased. Therefore, the 2 nd preprocessing executed by the remote autopilot server 10 can be a higher-level processing than the 1 st preprocessing executed by the information processing device 20 mounted on the autopilot vehicle. For example, the 1 st preprocessing is executed using the 1 st resource (for example, a small resource of an autonomous vehicle), and the 2 nd preprocessing is executed using the 2 nd resource (for example, a large resource of the remote autonomous server 10) different from the 1 st resource, so that the 2 nd preprocessing can be a higher-level preprocessing than the 1 st preprocessing. That is, the 2 nd resource is richer than the 1 st resource. For example, the 1 st preprocessing is executed using the 1 st algorithm (for example, an algorithm with a small amount of calculation that can be processed by the autonomous vehicle), and the 2 nd preprocessing is executed using a 2 nd algorithm different from the 1 st algorithm (for example, an algorithm with a large amount of calculation that can be processed by the remote autonomous server 10), so that the 2 nd preprocessing can be a higher-level preprocessing than the 1 st preprocessing. I.e. the 2 nd algorithm is superior to the 1 st algorithm. The 1 st preprocessing may be executed using both the 1 st resource and the 1 st algorithm, and the 2 nd preprocessing may be executed using both the 2 nd resource and the 2 nd algorithm.

The 1 st process result obtaining unit 13 obtains the 1 st process result from the autonomous vehicle. For example, the 1 st process result obtaining unit 13 obtains the 1 st process result, which is a process result transmitted from the autonomous vehicle and received via the communication interface or the like provided in the remote autonomous server 10.

The difference determination unit 14 determines a difference between the 1 st process result obtained by the 1 st process result obtaining unit 13 and the 2 nd process result obtained by the automated simulation system 12. The operation of the difference determination unit 14 will be described in detail later.

The change instruction output unit 15 outputs a change instruction for changing the 1 st processing result to the 3 rd processing result to the autonomous vehicle according to the difference determined by the difference determination unit 14. In other words, the change instruction output unit 15 instructs the autonomous vehicle to change the 1 st process result to the 3 rd process result, and the 3 rd process result is used for the travel control of the autonomous vehicle instead of the 1 st process result originally used for the travel control of the autonomous vehicle. Specific examples of the result of the 3 rd process will be described later.

Next, the operation of the remote autopilot server 10 will be described with reference to fig. 2.

Fig. 2 is a flowchart showing an example of the information processing method according to embodiment 1. For example, the information processing method according to embodiment 1 is a method executed by a computer (specifically, a processor) provided in the remote autopilot server 10. Fig. 2 is a flowchart showing the operation of the remote autopilot server 10.

First, the remote automated driving server 10 obtains the 1 st processing result, which is the result of the 1 st preprocessing of the travel control processing in the automated driving processing of the automated driving vehicle, and the sensed data obtained by the automated driving vehicle from the automated driving vehicle (step S11). For example, the remote autopilot server 10 obtains a result of detection processing by the autopilot, which is performed by the detection processing (for example, detection of the number of obstacles, detection of the position of an obstacle, or the like) of an obstacle around the autopilot based on the sensed data. Further, the remote autopilot server 10, for example, obtains the estimation processing result obtained by the autopilot performing estimation processing of the position of the autopilot from the sensed data. For example, the remote autopilot server 10 obtains a result of the travel determination processing (for example, determination of continuation of travel or stop) performed by the autopilot based on the sensed data.

Next, the remote autopilot server 10 executes the 2 nd preprocessing, which is a higher-level preprocessing than the 1 st preprocessing, based on the sensed data, and obtains the 2 nd processing result (step S12). For example, the remote autopilot server 10 performs detection processing (for example, detection of the number of obstacles or detection of the positions of obstacles, etc.) on the obstacles around the autopilot based on the sensing data, and obtains the result of the detection processing. For example, the remote autopilot server 10 performs estimation processing on the position of the autopilot based on the sensed data, and obtains the result of the estimation processing. For example, the remote autopilot server 10 performs a travel determination process (e.g., determination of continuation or stop of travel) of the autopilot based on the sensor data, and obtains the result of the travel determination process.

Next, the remote autopilot server 10 determines whether or not there is a processing delay in the 2 nd preprocessing (step S13). For example, in the remote autopilot server 10, the occupancy rate of the resource for the 2 nd preprocessing is high, and therefore, a processing delay of the 2 nd preprocessing may occur.

When the process delay occurs in the 2 nd preprocessing (yes in step S13), the remote autopilot server 10 outputs an instruction to restrict the travel of the autopilot (step S14). For example, it is preferable that the travel control of the autonomous vehicle is performed without a delay as much as possible from the timing of obtaining the sensing data. This is because, if it takes time to perform travel control using the result of preprocessing after the sensor data is obtained, the autonomous vehicle moves a lot from the point where the sensor data is obtained, and the current position of the autonomous vehicle that is deviated from the point where the sensor data is obtained loses its effectiveness as a result of preprocessing performed on the basis of the sensor data. Therefore, when the process delay occurs in the 2 nd preprocessing, the autonomous vehicle may be in a dangerous state because the result of the 2 nd preprocessing performed by the remote autonomous server 10 cannot be effectively used, and therefore the remote autonomous server 10 outputs an instruction to restrict the travel of the autonomous vehicle to the autonomous vehicle. The instruction to restrict the travel of the autonomous vehicle is, for example, an instruction to decelerate the autonomous vehicle, stop the autonomous vehicle, and enlarge the avoidance margin. In addition, the remote autopilot server 10 may notify the autopilot of an alarm when the processing delay occurs in the 2 nd preprocessing.

If the no processing delay occurs in the 2 nd preprocessing (no in step S13), the remote autopilot server 10 determines the difference between the 1 st processing result and the 2 nd processing result (step S15). For example, the remote autopilot server 10 determines the difference between the number of obstacles indicated by the 1 st processing result and the number of obstacles indicated by the 2 nd processing result. For example, the remote autopilot server 10 determines a difference between the position of the obstacle indicated by the 1 st processing result and the position of the obstacle indicated by the 2 nd processing result (for example, a Root Mean Square error value of the position of the obstacle indicated by the 1 st processing result when the position of the obstacle indicated by the 2 nd processing result is correct). For example, the remote autopilot server 10 determines a difference between the position of the autonomous vehicle indicated by the 1 st processing result and the position of the autonomous vehicle indicated by the 2 nd processing result (for example, an RMS error value of the position of the autonomous vehicle indicated by the 1 st processing result when the position of the autonomous vehicle indicated by the 2 nd processing result is correct). For example, the remote autopilot server 10 determines a difference between the travel determination processing result of the autonomous vehicle indicated by the 1 st processing result and the travel determination processing result of the autonomous vehicle indicated by the 2 nd processing result (for example, the number of times the travel determination processing result indicated by the 1 st processing result and the travel determination processing result indicated by the 2 nd processing result become different results for a certain period). These are merely examples of the difference determination, and are not limited thereto.

Next, the remote autopilot server 10 determines whether or not the determined difference satisfies a predetermined condition (step S16). The predetermined condition is, for example, a condition concerning the magnitude of the determined difference. For example, the remote autopilot server 10 determines whether or not the number of obstacles indicated by the 1 st processing result is different from the number of obstacles indicated by the 2 nd processing result. For example, the remote autopilot server 10 determines whether or not the RMS error value is equal to or greater than a predetermined threshold value. For example, the remote autopilot server 10 determines whether or not the number of times the travel determination processing result indicated by the 1 st processing result and the travel determination processing result indicated by the 2 nd processing result become different results within a predetermined period is equal to or greater than a predetermined threshold value. These are only examples of the predetermined conditions, and are not limited to these.

When the determined difference satisfies the predetermined condition (yes in step S16), the remote autopilot server 10 outputs a change instruction to change the 1 st processing result to the 3 rd processing result to the autopilot vehicle (step S17). The case where the determined difference satisfies the predetermined condition is, for example, a case where the number of obstacles indicated by the 1 st processing result is different from the number of obstacles indicated by the 2 nd processing result, a case where the RMS error value is equal to or greater than a predetermined threshold value, a case where the number of times the travel determination processing result indicated by the 1 st processing result and the travel determination processing result indicated by the 2 nd processing result become different within a certain period is equal to or greater than a predetermined threshold value, or the like.

The 3 rd processing result is obtained from at least one of the 1 st processing result and the 2 nd processing result. The 2 nd processing result is a result of the 2 nd preprocessing of an advanced level, so the 3 rd processing result, which is a result of the processing based on the 2 nd processing result, is a result of the advanced processing. For example, the 3 rd processing result may be the 2 nd processing result. In this way, in the travel control of the autonomous moving body, the 1 st processing result is not used, and the 2 nd processing result, which is a result of the 2 nd preprocessing, is used, so that the performance of the autonomous driving can be improved. Further, for example, the 3 rd processing result may be obtained by correcting the 1 st processing result according to the judged difference. For example, only a necessary range or a range that can be processed in the 1 st processing result is corrected in accordance with the difference between the 2 nd processing result and the 1 st processing result. Therefore, the result of correcting the 1 st processing result (in other words, the 1 st processing result after correction) based on the difference between the 2 nd processing result, which is the result of the 2 nd preprocessing, and the 1 st processing result, is used instead of the 1 st processing result in the travel control of the autonomous vehicle, and the performance of the autonomous vehicle can be improved.

Further, the remote autopilot server 10 may output an instruction to restrict the travel of the autonomous vehicle when the determined difference satisfies the predetermined condition (yes in step S16). Further, in this case, the remote autopilot server 10 may notify the remote monitor of the autopilot or the occupant or the like of the abnormality.

When the determined difference does not satisfy the predetermined condition (no in step S16), the remote autopilot server 10 does not output the change instruction to the autopilot (step S18). The case where the determined difference does not satisfy the predetermined condition is, for example, a case where the number of obstacles indicated by the 1 st processing result is the same as the number of obstacles indicated by the 2 nd processing result, a case where the RMS error value is smaller than a predetermined threshold value, a case where the number of times that the travel determination processing result indicated by the 1 st processing result and the travel determination processing result indicated by the 2 nd processing result become different within a certain period is smaller than a predetermined threshold value, or the like. In this case, it can be said that the 1 st processing result is not inferior to the 2 nd processing result, the autonomous vehicle does not receive the change instruction, and the travel control of the autonomous vehicle is performed using the 1 st processing result which is not inferior to the 2 nd processing result. In other words, when it is confirmed that the 1 st process result is not inferior to the 2 nd process result or that the degree to which the 1 st process result is inferior to the 2 nd process result is within the allowable range, the 1 st process result can be used in the travel control of the autonomous vehicle.

In addition, when the reception of the change instruction is delayed due to a communication delay, the autonomous vehicle may limit the travel of the autonomous vehicle by ignoring the received change instruction. This is because, as in the case of the above-described 2 nd preprocessing, even if the 1 st processing result is changed to the 3 rd processing result, the autonomous vehicle cannot effectively use the 3 rd processing result, and the autonomous vehicle may be in a dangerous state.

In addition, the sensed data obtained by the autonomous vehicle is susceptible to weather, and the vehicle autonomous system 23 may not obtain an accurate 1 st processing result by executing the 1 st preprocessing using the sensed data affected by weather. For example, when the view field around the autonomous vehicle is poor due to weather such as rain or cloudy days, the autonomous vehicle may not be able to accurately recognize obstacles present around the autonomous vehicle. It is then also possible to determine whether the weather is suitable for automatic driving by the vehicle automatic driving system 23. This will be described with reference to fig. 3.

Fig. 3 is a flowchart showing another example of the information processing method according to embodiment 1.

The flowchart shown in fig. 3 differs from the flowchart shown in fig. 2 in that step S19 is added. The rest (in other words, steps S11 to S18) are the same as those in fig. 2, and therefore, the description thereof is omitted.

If the process of step 2 is not delayed (no in step S13), the remote autopilot server 10 determines whether the weather is suitable for autopilot by the vehicle autopilot system 23 (step S19). For example, it is determined that the weather is suitable for automatic driving by the vehicle automatic driving system 23 when the weather is sunny, and it is determined that the weather is not suitable for automatic driving by the vehicle automatic driving system 23 when the weather is rainy or cloudy.

When the weather is not suitable for the automatic driving by the vehicle automatic driving system 23 (no in step S19), the remote automatic driving server 10 outputs a change instruction without performing the determination processing of the difference between the 1 st processing result and the 2 nd processing result in step S15 or the like (step S17). In the case where the weather is not suitable for the automated driving by the vehicle automated driving system 23, it is possible to estimate that the 1 st processing result is inferior to the 2 nd processing result without specially judging the difference.

When the weather is suitable for the automatic driving by the vehicle automatic driving system 23 (yes in step S19), the remote automatic driving server 10 performs the processing after step S15, as in the description of fig. 2. Since the 1 st processing result may be a result that is not inferior to the 2 nd processing result in the case where the weather is suitable for automatic driving by the vehicle automatic driving system 23, it is preferable to determine whether or not to output a change instruction by determining the difference.

This makes it possible to determine whether the weather is suitable for automatic driving by the vehicle automatic driving system 23.

As described above, unlike the 1 st preprocessing executed by an autonomous vehicle (specifically, the information processing device 20 mounted on the autonomous vehicle) in which a computer is limited and which can be mounted in terms of cost, power consumption, space, and the like, the 2 nd preprocessing which is higher than the 1 st preprocessing is executed by the remote autonomous server 10 in which restrictions on cost, power consumption, space, and the like are small, and the difference between the 1 st processing result which is the result of the 1 st preprocessing and the 2 nd processing result which is the result of the 2 nd preprocessing is determined. Then, according to the difference, the 1 st processing result is changed to the 1 st preprocessing result, which is a preprocessing result of the running control processing of the autonomous vehicle, and the 2 nd processing result is a result of the 2 nd preprocessing of the higher order, and a change instruction of the 3 rd processing result based on the 2 nd processing result is output to the autonomous vehicle. Or, in accordance with the difference, a change instruction to change the 1 st processing result to the 3 rd processing result obtained by correcting or limiting the 1 st processing result is output to the autonomous vehicle. Therefore, the 3 rd processing result, which is higher than the 1 st processing result, is used in the travel control processing of the autonomous vehicle, so that the performance of autonomous driving can be improved. For example, by improving the performance of autonomous driving, the area in which the autonomous vehicle can travel can be enlarged.

(modification of embodiment 1)

For example, execution of the 2 nd preprocessing or the like at the remote autopilot server may be started upon a request from the autopilot. Here, a modification of embodiment 1 will be described with reference to fig. 4 and 5.

Fig. 4 is a block diagram showing an example of an autonomous vehicle (specifically, an information processing device 20a mounted on the autonomous vehicle), a remote autonomous driving server 10a, and a remote processing management server 30 according to a modification of embodiment 1.

The information processing device 20a differs from the information processing device 20 of embodiment 1 in that it further includes a remote processing request unit 28, and a sensing data obtaining unit 21a, a sensing data transmitting unit 22a, and a 1 st processing result transmitting unit 24a instead of the sensing data obtaining unit 21, the sensing data transmitting unit 22, and the 1 st processing result transmitting unit 24. The rest is the same as the information processing device 20, and therefore, the description thereof is omitted. The remote processing request unit 28 is realized by a processor or the like that executes a program stored in a memory, as with other components.

The remote process requesting unit 28 outputs a request for execution of the 2 nd preprocessing to the remote autopilot server 10a via the remote process management server 30. For example, when there are a plurality of remote autopilot servers 10a having a function of executing the 2 nd preprocessing, the remote processing management unit 31 of the remote processing management server 30 inquires of the plurality of remote autopilot servers 10a whether the 2 nd preprocessing is executable upon receiving the request, and selects a remote autopilot server 10a capable of executing the 2 nd preprocessing from the plurality of remote autopilot servers 10a in accordance with the result of the inquiry. The remote processing management unit 31 notifies the selected remote automated driving server 10a to the automated driving vehicle. The remote processing request unit 28 instructs the sensed data obtaining unit 21a to obtain the sensed data, instructs the sensed data transmitting unit 22a to transmit the sensed data to the selected remote automatic driving server 10a, and instructs the 1 st processing result transmitting unit 24a to transmit the 1 st processing result to the selected remote automatic driving server 10 a.

The sensing data obtaining unit 21a receives an instruction from the remote processing request unit 28, and obtains sensing data obtained by a sensor provided in the autonomous vehicle. The other parts related to the sensing data obtaining unit 21a are the same as the sensing data obtaining unit 21, and therefore, the description thereof is omitted.

The sensing data transmitting unit 22a transmits the sensing data obtained by the sensing data obtaining unit 21a to the selected remote autopilot server 10 a. The other parts related to the sensing data transmitting unit 22a are the same as the sensing data transmitting unit 22, and therefore, the description thereof is omitted.

The 1 st processing result transmitting unit 24a transmits the 1 st processing result, which is the result of the 1 st preprocessing performed by the vehicle automatic driving system 23a, to the selected remote automatic driving server 10 a. The other parts related to the 1 st processing result transmitting unit 24a are the same as the 1 st processing result transmitting unit 24, and therefore, the description thereof is omitted.

The remote automated driving server 10a is different from the remote automated driving server 10 according to embodiment 1 in that it further includes a request acquisition unit 16 and a 2 nd preprocessing execution determination unit 17, and includes a sensed data acquisition unit 11a and a simulated automated driving system 12a instead of the sensed data acquisition unit 11 and the simulated automated driving system 12. The rest is the same as the remote autopilot server 10, and therefore, the description thereof is omitted. The request acquisition unit 16 and the 2 nd preprocessing execution determination unit 17 are realized by a processor or the like that executes a program stored in a memory, as with other components.

The request obtaining unit 16 obtains a request for execution of the 2 nd preprocessing from the autonomous vehicle. For example, the request obtaining unit 16 obtains a request for execution of the 2 nd preprocessing from the autonomous vehicle via the remote processing management server 30.

The 2 nd preprocessing execution determination unit 17 determines whether or not to execute the 2 nd preprocessing when the reception is requested. For example, the 2 nd preprocessing execution determination unit 17 may determine whether or not to execute the 2 nd preprocessing based on the task status or resource status of the remote autopilot server 10 a. For example, when the remote autopilot server 10a executes many tasks or resources are reduced, it is determined that the 2 nd preprocessing is not executed. For example, the 2 nd preprocessing execution determination unit 17 may determine whether or not to execute the 2 nd preprocessing based on at least 1 of the resource of the autonomous vehicle, the moving state of the autonomous vehicle, the external environment of the autonomous vehicle, the time, and the response time to the request. The movement state is a movement speed, an acceleration, a deceleration, a movement direction (steering angle), and the like of the autonomous vehicle. For example, when the resources of the autonomous vehicle are insufficient, the moving state of the autonomous vehicle is, for example, a state in which the vehicle speed is fast, a state in which the acceleration is large, or a state in which the steering angle is large, when the external environment of the autonomous vehicle is an environment in which the obstacle is located close to the vehicle, a large number of environments, or a type of moving body, when the environment of the autonomous vehicle is a place in which the amount of traffic is large (for example, an intersection or the like), an environment in which the brightness of the surroundings is dark, an environment in which the weather is rainy or cloudy, or the like, when the time is the time of night, or when the response time to the request is short, it is determined that the 2 nd preprocessing is executed. For example, when the autonomous vehicle has sufficient resources, the moving state of the autonomous vehicle is, for example, a state in which the vehicle speed is slow, a state in which the acceleration is small, or a state in which the steering angle is small, when the external environment of the autonomous vehicle is an environment in which the position of an obstacle is far, a small number of environments, or an environment in which the type of an object does not move, when the position of the autonomous vehicle is a place in which the traffic volume is small, an environment in which the brightness of the surroundings is bright, an environment in which the weather is fine, or the like, when the time is the time of day, when the response time to the request is long, or the like, it is determined that the pre-process 2 is not to be executed.

Thus, when the request is received, depending on the situation, the performance of the automatic driving may not be improved even if the 2 nd preprocessing is executed. For example, depending on the resources of the autonomous vehicle, the moving state of the autonomous vehicle, the external environment of the autonomous vehicle, the time of day, or the response time to the request, even if the 2 nd preprocessing is executed and the travel control of the autonomous vehicle is performed using the 3 rd processing result, the performance of autonomous driving may not be improved. In such a case, the remote autopilot server 10a can reject or ignore the request.

When the request is received from the autonomous vehicle, the sensing data obtaining unit 11a obtains the sensing data when determining that the 2 nd preprocessing is to be executed. The other parts related to the sensing data obtaining unit 11a are the same as the sensing data obtaining unit 11, and therefore, the description thereof is omitted.

When the simulated automatic driving system 12a determines that the 2 nd preprocessing is to be executed when receiving a request from the automatic driving vehicle, the simulated automatic driving system executes the 2 nd preprocessing based on the sensed data obtained by the sensed data obtaining unit 11a, and obtains the 2 nd processing result. The other portions related to the simulated automatic driving system 12a are the same as those of the simulated automatic driving system 12, and therefore, the description thereof is omitted.

Next, the operation of the remote autopilot server 10a will be described with reference to fig. 5.

Fig. 5 is a flowchart showing an example of an information processing method according to a modification of embodiment 1. For example, the information processing method according to the modification of embodiment 1 is a method executed by a computer (specifically, a processor) provided in the remote autopilot server 10 a. Therefore, fig. 5 is also a flowchart showing the operation of the remote autopilot server 10 a.

The flowchart shown in fig. 5 differs from the flowchart shown in fig. 2 in that step S21, step S22, and step S23 are added. The rest of the steps (in other words, steps S11 to S18) are the same as those shown in fig. 2, and therefore, the description thereof is omitted.

The remote autopilot server 10a determines whether or not the request for execution of the 2 nd preprocessing is received from the autopilot (step S21).

The remote autopilot server 10a repeats the processing of step S21 until the request is accepted, in a case where the request for execution of the 2 nd preprocessing is not accepted from the autopilot (no in step S21).

When the request for execution of the 2 nd preprocessing is received from the autonomous vehicle (yes in step S21), the remote autonomous driving server 10a determines whether or not the 2 nd preprocessing is executed (step S22).

If it is determined that the No. 2 preprocessing is not executed (no in step S22), the remote autopilot server 10a rejects or ignores the request (step S23). Thus, for example, the remote processing management server 30 does not select the remote autopilot server 10a that rejects or ignores the request as the server that performs the 2 nd preprocessing, and selects the server that performs the 2 nd preprocessing from the other remote autopilot servers 10 a. In addition, in the case where all the remote automated driving servers 10a reject or ignore the request, the remote processing management server 30 may instruct the automated driving vehicle to restrict the traveling.

When determining that the 2 nd preprocessing is to be executed (yes in step S22), the remote autopilot server 10a performs the processing after step S11, as described with reference to fig. 2.

In addition, the request from the autonomous vehicle may include information specifying a specific process in the 2 nd preprocessing. In this case, the remote autopilot server 10a performs a specific process at step S12 to obtain the 2 nd process result. This is because, depending on the moving state of the autonomous vehicle, the external environment, and the like, the difference may be determined only for a specific process (for example, only for the obstacle detection process, only for the travel determination process, and the like). As described above, the remote autopilot server 10a can selectively execute the specific process specified in the 2 nd preprocessing, for example, which is desired by the autopilot. In addition, when only the obstacle detection is performed, the area in which the obstacle detection is performed may be limited. For example, when the autonomous vehicle performs a lane change, obstacle detection may be performed on the lane after the lane change.

In addition, when receiving requests from a plurality of autonomous vehicles, the remote processing management server 30 may select an autonomous vehicle that preferentially receives the request, based on at least 1 of the resource of each autonomous vehicle, the moving state of each autonomous vehicle, the external environment of each autonomous vehicle, the time, and the response time to the request.

As described above, the execution of the 2 nd preprocessing performed by the remote autopilot server 10a may be started when a request is received from the autopilot. The remote autopilot server 10a is thus able to execute the 2 nd preprocessing at a timing when the autopilot wishes to execute the 2 nd preprocessing of the high level.

(embodiment mode 2)

In embodiment 1, an example in which the remote autopilot server 10 determines the difference between the 1 st processing result and the 2 nd processing result has been described, but this determination may be performed by an information processing device mounted on an autopilot. This will be described as embodiment 2 with reference to fig. 6 and 7.

Fig. 6 is a block diagram showing an example of an autonomous vehicle (specifically, an information processing device 200 mounted on the autonomous vehicle) and a remote autonomous server 100 according to embodiment 2.

The remote autopilot server 100 wirelessly communicates with the autopilot.

The autonomous vehicle is mounted with an information processing device 200. The information processing apparatus 200 is a computer including a processor, a memory, a communication interface, and the like. The memory is ROM, RAM, or the like, and can store a program executed by the processor. The information processing apparatus 200 includes: a sensory data obtaining unit 201, a sensory data transmitting unit 202, a vehicle automatic driving system 203, a 2 nd processing result obtaining unit 204, a difference determining unit 205, a processing result changing unit 206, a communication confirming unit 207, and a travel limiting unit 208. The sensor data obtaining unit 201, the sensor data transmitting unit 202, the vehicle automatic driving system 203, the 2 nd process result obtaining unit 204, the difference determining unit 205, the process result changing unit 206, the communication confirming unit 207, and the travel limiting unit 208 are realized by a processor or the like that executes a program stored in a memory.

The functions of the sensed data obtaining unit 201, the sensed data transmitting unit 202, the vehicle automatic driving system 203, the communication confirming unit 207, and the travel limiting unit 208 are basically the same as those of the sensed data obtaining unit 21, the sensed data transmitting unit 22, the vehicle automatic driving system 23, the communication confirming unit 26, and the travel limiting unit 27 of embodiment 1, and therefore, the description thereof is omitted.

The 2 nd processing result obtaining unit 204 obtains a 2 nd processing result, which is a result of executing the 2 nd preprocessing, which is a higher-order preprocessing than the 1 st preprocessing, based on the sensed data, from the remote autopilot server 100. For example, the 2 nd process result obtaining unit 204 obtains the 2 nd process result, and the 2 nd process result is transmitted from the remote automated driving server 100 and received via a communication interface or the like provided in the automated driving vehicle.

The difference determination unit 205 determines the difference between the 1 st process result obtained by the vehicle automatic driving system 203 and the 2 nd process result obtained by the 2 nd process result obtaining unit 204. The difference between the difference determination unit 205 of embodiment 2 and the difference determination unit 14 of embodiment 1 is that the information processing device 200 includes the difference determination unit, and the remote autopilot server 10 includes the difference determination unit, and the functions thereof are basically the same. Therefore, the detailed description of the difference determination unit 205 is omitted.

The processing result changing unit 206 changes the 1 st processing result to the 3 rd processing result according to the determined difference. While the example in which the processing result changing unit 25 changes the 1 st processing result to the 3 rd processing result in accordance with the change instruction from the remote autopilot server 10 has been described in embodiment 1, in embodiment 2, the processing result changing unit 206 changes the 1 st processing result to the 3 rd processing result in accordance with the difference determined by the autopilot itself.

The remote autopilot server 100 is a computer that includes a processor, memory, and a communication interface, among other things. The remote autopilot server 100 is an example of an apparatus external to the information processing apparatus 200. The memory is ROM, RAM, or the like, and can store a program executed by the processor. The remote autopilot server 100 includes a sensing data acquisition unit 101, a simulation autopilot system 102, and a 2 nd processing result transmission unit 103. The sensing data obtaining unit 101, the simulated automatic driving system 102, and the 2 nd processing result transmitting unit 103 are realized by a processor or the like that executes a program stored in a memory. The components constituting the remote autopilot server 100 may be distributed among a plurality of servers.

The functions of the sensed data obtaining unit 101 and the automated driving simulation system 102 are basically the same as those of the sensed data obtaining unit 11 and the automated driving simulation system 12 according to embodiment 1, and therefore, the description thereof is omitted.

The 2 nd processing result transmitting unit 103 transmits the 2 nd processing result, which is the result of the 2 nd preprocessing executed by the simulation autonomous driving system 102, to the autonomous vehicle. The 2 nd processing result transmitting unit 103 transmits the 2 nd processing result to the autonomous vehicle via a communication interface or the like provided in the remote autonomous server 100.

In embodiment 2, the 2 nd processing result is transmitted from the remote automatic driving server 100 to the automatic driving vehicle, and the determination of the difference between the 1 st processing result and the 2 nd processing result is performed not in the remote automatic driving server 100 but in the automatic driving vehicle.

Next, the operation of the autonomous vehicle will be described with reference to fig. 7.

Fig. 7 is a flowchart showing an example of the operation of the autonomous vehicle (specifically, the information processing device 200) according to embodiment 2.

First, the autonomous vehicle determines whether or not the autonomous vehicle is in communication with the remote autonomous server 100 (step S41).

And an autonomous vehicle which restricts traveling of the autonomous vehicle when the autonomous vehicle is not in communication connection with the remote autonomous server 100 (no in step S41) (step S42). In the case where the autonomous vehicle is not communicatively connected to the remote autonomous driving server 100, the autonomous vehicle cannot obtain the 2 nd processing result, that is, cannot determine a difference between the 1 st processing result and the 2 nd processing result, and cannot change the 1 st processing result to the 3 rd processing result. Therefore, when the autonomous vehicle is not in communication connection with the remote autonomous server 100, the autonomous vehicle may be in a dangerous state, and thus the travel of the autonomous vehicle is restricted. Therefore, even when the communication between the autonomous vehicle and the remote autonomous server 10 is cut off, the safety of the autonomous vehicle can be ensured.

When the autonomous vehicle determines that the autonomous vehicle is in communication with the remote autonomous server 100 (yes in step S41), the 1 st preprocessing, which is preprocessing of the travel control processing in autonomous driving of the autonomous vehicle, is executed based on the sensor data, and the 1 st processing result is obtained (step S43). For example, the autonomous vehicle performs a process of detecting an obstacle around the autonomous vehicle (for example, detection of the number of obstacles or detection of the position of the obstacle) based on the sensed data, and obtains the result of the detection process. Further, for example, the autonomous vehicle performs an estimation process of the position of the autonomous vehicle from the sensed data, and obtains a result of the estimation process. For example, the autonomous vehicle performs a travel determination process (for example, determination of whether to continue traveling or stop) of the autonomous vehicle on the basis of the sensor data, and obtains the result of the travel determination process.

The autonomous vehicle then outputs the sensor data obtained by the autonomous vehicle to the remote autonomous server 100 (step S44). The remote autopilot server 100 that has received the sensed data executes the 2 nd preprocessing, which is a higher-order preprocessing than the 1 st preprocessing, based on the sensed data, and obtains the 2 nd processing result. Then, the remote autopilot server 100 transmits the obtained 2 nd processing result to the autopilot vehicle.

Next, the autonomous vehicle determines whether or not the 2 nd processing result transmitted from the remote autonomous server 100 is obtained without delay (step S45). For example, when the second process result is obtained after a predetermined time or more has elapsed since the output of the sensor data or the output of the request, the autonomous vehicle determines that the second process result is obtained without delay. When a communication delay occurs between the autonomous vehicle and the remote autonomous driving server 100, the 2 nd processing result may not be obtained without a delay.

When the second process result is obtained without delay (no in step S45), the autonomous vehicle does not perform the difference determination and restricts the traveling of the autonomous vehicle (step S42). As in the case of the 2 nd preprocessing described in embodiment 1, when the processing result is changed to the 3 rd processing result, the autonomous vehicle cannot effectively use the 3 rd processing result, and the autonomous vehicle may be in a dangerous state.

When the second process result is obtained without delay (yes in step S45), the autonomous vehicle determines the difference between the first process result and the second process result (step S46), and determines whether or not the determined difference satisfies a predetermined condition (step S47). The processing of step S46 and step S47 is the same as the processing of step S15 and step S16 described in fig. 2 except that it is performed in the autonomous vehicle, and therefore, the description thereof is omitted.

When the determined difference satisfies the predetermined condition (yes in step S47), the autonomous vehicle executes a process of changing the 1 st processing result to the 3 rd processing result (step S48). Then, using the 3 rd processing result, the travel control processing is executed. In this way, the 1 st processing result is not used for the travel control processing of the autonomous vehicle, but is used for the travel control of the autonomous vehicle based on the 3 rd processing result of the 2 nd processing result which is the result of the 2 nd preprocessing, and the performance of the autonomous vehicle can be improved.

When the determined difference satisfies the predetermined condition (yes in step S47), the autonomous vehicle may restrict the travel of the autonomous vehicle. Further, in this case, the autonomous vehicle may notify a remote monitor of the autonomous vehicle or an occupant or the like of an abnormality.

When the determined difference does not satisfy the predetermined condition (no in step S47), the autonomous vehicle does not execute the change processing (step S49). In this case, for example, the 1 st processing result is not inferior to the 2 nd processing result, and the autonomous vehicle may perform travel control of the autonomous vehicle using the 1 st processing result which is not inferior to the 2 nd processing result without changing the 1 st processing result to the 3 rd processing result.

Then, the autonomous vehicle determines whether or not the vehicle has reached the destination (step S50), and if the vehicle has not reached the destination (no in step S50), the processing of steps S41 to S49 is repeated until the vehicle has reached the destination, and if the vehicle has reached the destination (yes in step S50), the autonomous vehicle is stopped and the processing is ended.

As described above, unlike the 1 st preprocessing executed by the autonomous vehicle in which a computer is limited in terms of cost, power consumption, space, and the like, the 2 nd preprocessing, which is a result of the 2 nd preprocessing, is executed by an external device (for example, the remote autonomous server 100) having less restrictions on cost, power consumption, space, and the like, the 2 nd preprocessing, which is a higher level than the 1 st preprocessing, is transmitted to the autonomous vehicle, and the autonomous vehicle determines a difference between the 1 st processing result, which is a result of the 1 st preprocessing, and the 2 nd processing result, which is a result of the 2 nd preprocessing. Then, according to the difference, the 1 st processing result is changed to the 3 rd processing result based on the 2 nd processing result, the 1 st processing result being the result of the 1 st preprocessing which is the preprocessing of the travel control processing of the autonomous vehicle, and the 2 nd processing result being the result of the 2 nd preprocessing which is the higher level. Or changing the 1 st processing result to a 3 rd processing result obtained by correcting or limiting the 1 st processing result according to the difference. Therefore, the 3 rd processing result, which is higher than the 1 st processing result, is used in the travel control processing of the autonomous vehicle, so that the performance of autonomous driving can be improved. For example, by improving the performance of autonomous driving, the area in which the autonomous vehicle can travel can be enlarged.

(modification of embodiment 2)

For example, execution of the 2 nd preprocessing in the remote automated driving server, or the like, may be started when there is a request from the automated driving vehicle. Here, a modification of embodiment 2 will be described with reference to fig. 8 and 9.

Fig. 8 is a block diagram showing an example of an autonomous vehicle (specifically, an information processing device 200a mounted on the autonomous vehicle), a remote autonomous server 100a, and a remote processing management server 30 according to a modification of embodiment 2.

The information processing apparatus 200a is different from the information processing apparatus 200 according to embodiment 2 in that it further includes a remote processing request unit 210, and a sensing data acquisition unit 201a and a sensing data transmission unit 202a instead of the sensing data acquisition unit 201 and the sensing data transmission unit 202. The rest is the same as the information processing apparatus 200, and therefore, the description thereof is omitted. The remote processing request unit 210 is realized by a processor or the like that executes a program stored in a memory, as with other components.

The remote process requesting unit 210 outputs a request for executing the 2 nd preprocessing to the remote autopilot server 100a via the remote process management server 30. The remote processing management server 30 has basically the same functions as the modification of embodiment 1, and therefore, description thereof is omitted. For example, the remote processing request unit 210 outputs a request to the remote automated driving server 100a based on at least 1 of the resource of the automated driving vehicle, the moving state of the automated driving vehicle, the external environment of the automated driving vehicle, the time, and the response time to the inquiry to the remote automated driving server 100 a. For example, when the resource of the autonomous vehicle is insufficient, the movement state of the autonomous vehicle, for example, a state in which the vehicle speed is high, a state in which the acceleration is high, or a state in which the steering angle is high, the external environment of the autonomous vehicle is an environment in which the obstacle is close to the position, a large number of environments, or the type is a moving body, the environment in which the position of the autonomous vehicle is a place in which the amount of traffic is large (for example, an intersection or the like), the environment in which the surrounding brightness is dark, the weather is an environment in which it is rainy or cloudy, or the like, and when the time is the time of night, the request is output when the response time to the request is short, or the like. For example, when the resources of the autonomous vehicle are sufficient, the movement state of the autonomous vehicle, for example, a state in which the vehicle speed is slow, a state in which the acceleration is small, or a state in which the steering angle is small, the external environment of the autonomous vehicle is an environment in which the position of an obstacle is far, a small number of environments, or an environment in which the type of an object does not move, the environment in which the position of the autonomous vehicle is a place in which the traffic volume is small, an environment in which the brightness of the surroundings is bright, an environment in which the weather is fine, or the like, and when the time is the time of day, or the time is long, the request is not output.

Thus, even if the 2 nd preprocessing is executed depending on the situation, the performance of the automatic driving may not be improved. For example, depending on the resources of the autonomous vehicle, the moving state of the autonomous vehicle, the external environment of the autonomous vehicle, the time of day, or the response time to the request, even if the 2 nd preprocessing is executed to perform the travel control of the autonomous vehicle using the 3 rd processing result, the performance of autonomous driving may not be improved. In this case, the autonomous vehicle may be configured not to output the request. In addition, in other words, the performance of the automatic driving can sometimes be improved by executing the 2 nd preprocessing according to the situation. The autonomous vehicle can output the request under the condition that the performance of the autonomous vehicle is improved.

The remote processing request unit 210 instructs the sensed data obtaining unit 201a to obtain the sensed data, and instructs the sensed data transmitting unit 202a to transmit the sensed data to the remote autopilot server 100a selected by the remote processing management unit 31.

The sensing data obtaining unit 201a obtains sensing data obtained by a sensor provided in the autonomous vehicle in accordance with an instruction from the remote processing requesting unit 210. The other parts related to the sensing data obtaining unit 201a are the same as the sensing data obtaining unit 201, and therefore, the description thereof is omitted.

The sensed data transmitting unit 202a transmits the sensed data obtained by the sensed data obtaining unit 201a to the selected remote autopilot server 100 a. The other parts related to the sensing data transmitting unit 202a are the same as the sensing data transmitting unit 202, and therefore, the description thereof is omitted.

The remote automated driving server 100a is different from the remote automated driving server 100 according to embodiment 2 in that it further includes a request acquisition unit 104 and a 2 nd preprocessing execution determination unit 105, and includes a sensing data acquisition unit 101a and a simulation automated driving system 102a instead of the sensing data acquisition unit 101 and the simulation automated driving system 102. The rest is the same as the remote autopilot server 100, and therefore, the description thereof is omitted. The request acquisition unit 104 and the 2 nd preprocessing execution determination unit 105 are realized by a processor or the like that executes a program stored in a memory, as with other components.

The request obtaining unit 104 is basically the same as the request obtaining unit 16 according to the modification of embodiment 1, and therefore, description thereof is omitted.

The 2 nd preprocessing execution determination unit 105 is basically the same as the 2 nd preprocessing execution determination unit 17 of the modification of embodiment 1, and therefore, description thereof is omitted.

When the request is received from the autonomous vehicle, the sensing data obtaining unit 101a obtains the sensing data when it is determined that the 2 nd preprocessing is executed. The other parts related to the sensing data obtaining unit 101a are the same as the sensing data obtaining unit 101, and therefore, the description thereof is omitted.

The simulation autopilot system 102a, upon receiving a request from an autopilot, executes the 2 nd preprocessing based on the sensed data obtained by the sensed data obtaining unit 101a when it is determined that the 2 nd preprocessing is executed, and obtains the 2 nd processing result. The other portions related to the simulated automatic driving system 102a are the same as those of the simulated automatic driving system 102, and therefore, the description thereof is omitted.

Next, the operation of the autonomous vehicle will be described with reference to fig. 9.

The difference between the flowchart shown in fig. 9 and the flowchart shown in fig. 7 is that step S51 is added instead of step S41. The other portions (in other words, step S42 to step S50) are the same as those shown in fig. 7, and therefore, the description thereof is omitted.

The autonomous vehicle outputs the request for execution of the 2 nd preprocessing to the remote autonomous server 100a (step S51). For example, after the output request, the autonomous vehicle receives a notification from the remote processing management server 30 indicating to which remote autonomous server 100a the sensed data should be output as a server capable of executing the 2 nd preprocessing. Also, as a response to the request, the 2 nd processing result is obtained at step S45. In addition, when the autonomous vehicle outputs the request and does not respond to the request, it is determined that the autonomous vehicle is not in communication with the remote processing management server 30 or the like, and the travel of the autonomous vehicle may be restricted.

In addition, the request for execution of the 2 nd preprocessing may include information specifying a specific process in the 2 nd preprocessing. Depending on the moving state of the autonomous vehicle, the external environment, or the like, the difference may be determined only for a specific process (for example, only for an obstacle detection process, only for a travel determination process, or the like). Thus, the autonomous vehicle can specify, for example, a specific process desired by the autonomous vehicle from the 2 nd preprocessing, and selectively cause the remote autonomous server 100a to execute the specific process. In this case, the autonomous vehicle obtains the result of the 2 nd processing obtained by executing the specific processing in step S45 and step S46, and determines the difference only for the specific processing.

In addition, even when the second process result is obtained after a predetermined time or more has elapsed since the output of the sensor data or the output of the request (in other words, even when no is performed in step S45), the process proceeds to step S46, and a difference between a partial process result of the first process result and a process result corresponding to the partial process result of the second process result is determined. For example, the partial processing results are portions that are less susceptible to the delay. For example, the partial processing result is a result of the surrounding recognition processing, and the result of the surrounding recognition processing is less likely to be affected by a delay. For example, since the result of the self-position estimation process is susceptible to delay, the determination of the difference may not be performed with respect to the result of the process. The autonomous vehicle may change the partial processing result of the 1 st processing result to the 3 rd processing result according to the determined difference. In this way, when the 2 nd processing result is obtained after a predetermined time or more has elapsed from the output of the sensing data or the output of the request, there is a possibility that a communication delay may occur with the remote autopilot server 100 a. At this time, the processing load of the autonomous vehicle (specifically, the information processing apparatus 200a) is reduced by not performing the determination of the difference or performing the determination of the difference only on a part of the processing results that are not affected by the delay.

As explained above, execution of the 2 nd preprocessing or the like at the remote automated driving server can be started by outputting a request from the automated driving vehicle. Accordingly, the 2 nd preprocessing can be executed at the timing when the autonomous vehicle desires to execute the 2 nd preprocessing at a higher level.

(common modification of each embodiment)

In the above embodiments, the instruction to change the preprocessing result or the change control is performed according to whether or not there is a delay related to the 2 nd preprocessing, but in the case where there is a delay, it may be performed according to whether or not the correction processing can be performed with respect to the delay.

The instruction or control to change the preprocessing result and the instruction or control to drive the vehicle can be performed according to an ODD (Operational Design Domain) corresponding to the delay and the difference, in addition to the difference between the 1 st processing result and the 2 nd processing result. The ODD is set, for example, with a time zone, a region, a driving state (speed, acceleration, steering angle, etc.), and an environment (weather, illuminance, etc.) as elements.

The above processing is explained with reference to fig. 10. Fig. 10 is a flowchart showing an example of an information processing method according to a modification common to the respective embodiments. Note that, substantially the same processing as in the above embodiments is omitted.

If the server (e.g., remote autopilot server) has a delay in the 2 nd preprocessing (yes in step S13), it is determined whether or not the delay can be corrected (step S60). Specifically, when there is a communication delay between the server and the autonomous vehicle or when there is a processing delay as described above, the server determines whether or not it is possible to correct (suppress or reduce) the time lag in the processing result due to the delay. For example, it is determined whether or not the delay amount is equal to or less than a threshold value.

In a case where the correction processing is possible for the delay (yes in step S60), the server executes the correction processing (step S61). Specifically, the server executes the correction processing for the 2 nd preprocessing result when the delay amount is equal to or less than the threshold value.

In the case where the 2 nd preprocessing has no delay (no in step S60), or after the correction processing is executed, the processing proceeds to steps S15, S16.

If the correction processing cannot be performed for the delay (no in step S60), the server determines whether the 1 st ODD is satisfied (step S62). Specifically, the 1 st ODD is an ODD set for autonomous driving of the autonomous vehicle. For example, the 1 st ODD is that the area is outside the intersection, the speed is 20km/h or less, and the weather is clear.

If the 1 st ODD is satisfied, the server does not output a change instruction (step S18). In this case, the running control process in the automated driving is performed using the result of the 1 st preprocessing performed in the automated driving vehicle.

When the difference between the 1 st and 2 nd preprocessing satisfies the predetermined condition (yes in step S16), the server determines whether or not the 2 nd ODD is satisfied (step S63). Specifically, the 2 nd ODD is an ODD different from the 1 st ODD set for the automatic driving of the autonomous vehicle. For example, the 2 nd ODD indicates that the area is the whole area, the speed is 15km/h or less, and the weather is sunny or rainy. Thus, the 2 nd ODD is at least partially relaxed compared to the 1 st ODD. On the other hand, communication is used when the 2 nd processing result is used, so items affecting delay in the 2 nd ODD are the same as or stricter than the 1 st ODD.

When the difference satisfies the predetermined condition and the 2 nd ODD is satisfied (yes in step S63), the server outputs a change instruction (step S17). In this case, the running control process in the automatic driving is performed using the result of the 2 nd preprocessing executed at the server or the result of the 1 st preprocessing after being corrected.

When the difference between the 1 st and 2 nd preprocessing does not satisfy the predetermined condition (no in step S16), the server determines whether or not the 2 nd ODD is satisfied (step S64). This process is substantially the same as the process of step S63.

If the difference does not satisfy the predetermined condition but satisfies the 2 nd ODD (yes in step S64), the server does not output a change instruction (step S18). The reason why the 2 nd ODD is used in the case where the difference does not satisfy the prescribed condition in this way is because the difference between the result of the preprocessing in the server and the result of the preprocessing of the autonomous vehicle is not sufficient to satisfy the prescribed condition. In other words, the result of the preprocessing of the autonomous vehicle can be used as is the result of the preprocessing of the server.

If the 1 st ODD is not satisfied (no in step S62) or if the 2 nd ODD is not satisfied (no in step S63 or S64), the server outputs a travel restriction instruction (step S14).

Fig. 10 shows an example in which the server determines whether or not the correction process for the delay is possible, and determines the instruction to change the preprocessing result and the instruction to limit the travel in accordance with the ODDs corresponding to the delay and the difference. However, the autonomous vehicle may determine whether or not the delay correction process is possible, and may execute the change control of the preprocessing result and the travel restriction according to the ODD corresponding to the delay and the difference.

The instruction to change the preprocessing result or the change control may be performed according to whether or not there is a communication failure related to the 2 nd preprocessing. Specifically, the communication failure is a communication data loss. For example, the instruction or control of changing the preprocessing result may be performed according to whether or not the packet loss rate is equal to or greater than a threshold value. In addition, the communication failure may include the communication delay described above.

In the case where there is a communication failure, the instruction to change the preprocessing result or the change control may be performed according to whether or not the correction processing can be performed with respect to the communication failure. Specifically, the instruction or control to change the preprocessing result is performed according to whether or not data missing due to communication data loss can be supplemented. For example, the instruction or control of the change of the preprocessing result is performed to such an extent that the lost packet can be replenished.

(other embodiments)

The information processing method, the information processing system (for example, a remote automatic driving server) and the information processing apparatus according to one or more aspects of the present disclosure have been described above based on the embodiments, but the present disclosure is not limited to these embodiments. The embodiment obtained by implementing various modifications that can be conceived by a person skilled in the art to each embodiment and the embodiment configured by combining the constituent elements in different embodiments are also included in the scope of one or more embodiments of the present disclosure within a scope not departing from the gist of the present disclosure.

For example, the present disclosure realizes the steps included in the information processing method as a program for causing a processor to execute. The present disclosure can be realized as a non-transitory computer-readable recording medium such as a CD-ROM on which the program is recorded.

For example, when the present disclosure is implemented by a program (software), each step is executed by executing the program using hardware resources such as a CPU, a memory, and an input/output circuit of a computer. In other words, the CPU obtains data from the memory, the input-output circuit, or the like and performs an operation, or outputs an operation result to the memory, the input-output circuit, or the like, thereby executing each step.

In the above embodiments, each of the components included in the information processing system and the information processing apparatus may be implemented by dedicated hardware or by executing a software program suitable for each of the components. Each of the components can be realized by a program execution unit such as a CPU or a processor reading out and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

A part or all of the functions of the information processing system and the information processing apparatus according to the above embodiments are typically realized by an LSI which is an integrated circuit. These may be singulated separately or may be singulated in a manner including a part or all of them. Further, the integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that is Programmable after LSI manufacturing or a reconfigurable processor that can reconfigure connection and setting of circuit cells within an LSI may be used.

Furthermore, various modifications within the scope that can be conceived by those skilled in the art are also included in the present disclosure, as far as the scope of the present disclosure is not exceeded.

Industrial applicability

The present disclosure can be applied to a remote control system of an autonomous vehicle.

Description of the symbols

10, 10a, 100, 100a remote autopilot server

11, 11a, 21, 21a, 101, 101a, 201, 201a sensing data obtaining part

12, 12a, 102, 102a simulation automatic driving system

13 1 st processing result obtaining part

14, 205 difference judging section

15 change instruction output unit

16, 104 request acquisition part

17, 105 2 nd preprocessing execution judging part

20, 20a, 200, 200a information processing apparatus

22, 22a, 202, 202a sensing data transmitting part

23, 203 automatic driving system for vehicle

24, 24a 1 st processing result transmitting part

25, 206 processing result changing part

26, 207 communication confirming section

27, 208 travel restriction unit

28, 210 remote processing request section

30 remote processing management server

31 remote processing management part

103 2 nd processing result transmitting part

204 2 nd processing result obtaining part.

The claims (modification according to treaty clause 19)

(modified) an information processing method of causing a computer to execute a process of,

obtaining, from an autonomous mobile body, a 1 st processing result as a result of 1 st preprocessing, which is preprocessing of a travel control process in autonomous movement processing of the autonomous mobile body, and sensing data obtained by the autonomous mobile body, wherein the 1 st preprocessing is executed by a 1 st computer provided in the autonomous mobile body,

performing 2 nd preprocessing as preprocessing superior to the 1 st preprocessing based on the sensing data, the 2 nd preprocessing being performed by a 2 nd computer connected to the autonomous moving body via a network, the 2 nd computer having a higher processing capability than the 1 st computer, and obtaining a 2 nd processing result,

determining a difference between the 1 st processing result and the 2 nd processing result,

and outputting a change instruction to change the 1 st process result to a 3 rd process result to the autonomous moving object according to the determined difference, wherein the 3 rd process result is obtained based on at least one of the 1 st process result and the 2 nd process result.

2. The information processing method according to claim 1,

the 1 st pre-processing, performed using the 1 st resource,

the 2 nd preprocessing, performed using the 2 nd resource,

the 1 st resource is a different resource than the 2 nd resource.

3. The information processing method according to claim 1 or 2,

the 1 st pre-processing, performed using the 1 st algorithm,

the 2 nd preprocessing, performed using the 2 nd algorithm,

the 1 st algorithm is a different algorithm than the 2 nd algorithm.

4. The information processing method according to any one of claims 1 to 3,

the 1 st preprocessing and the 2 nd preprocessing include a recognition processing of recognizing an environment in which the autonomous moving body is located.

5. The information processing method according to any one of claims 1 to 4,

the 1 st and 2 nd pretreatments include a travel determination process of the autonomous moving body.

6. The information processing method according to any one of claims 1 to 5,

the 3 rd processing result is the 2 nd processing result.

7. The information processing method according to any one of claims 1 to 5,

the 3 rd processing result is a result of correcting the 1 st processing result based on the difference.

8. The information processing method according to any one of claims 1 to 7,

executing the 2 nd preprocessing to obtain a 2 nd processing result when the request for execution of the 2 nd preprocessing is accepted from the autonomous moving body,

outputting the change instruction to the autonomous mobile unit as a response to the request.

9. The information processing method according to claim 8,

the request includes information specifying a specific process among the 2 nd preprocessing,

and executing the specific processing to obtain the 2 nd processing result.

10. The information processing method according to claim 8 or 9,

in a case where the request is accepted, a judgment is made as to whether or not the 2 nd preprocessing is executed,

and rejecting or ignoring the request under the condition that the No. 2 preprocessing is judged not to be executed.

11. The information processing method according to claim 10,

the determination as to whether or not to execute the 2 nd preprocessing is made in accordance with at least one of a resource that the autonomous mobile body has, a movement state of the autonomous mobile body, an external environment of the autonomous mobile body, a time of day, and a response time to the request.

(modified) an information processing system that is connected to an autonomous mobile body via a network and is capable of communicating with the autonomous mobile body, wherein,

obtaining, from the autonomous mobile body, a 1 st process result as a result of a 1 st process that is a process of a travel control process in autonomous movement of the autonomous mobile body, and the sensor data obtained by the autonomous mobile body, wherein the 1 st process is executed by a 1 st computer provided in the autonomous mobile body, and a processing capability of the 1 st computer is lower than that of the information processing system,

performing a 2 nd preprocessing which is a higher-order preprocessing than the 1 st preprocessing based on the sensing data, obtaining a 2 nd processing result,

determining a difference between the 1 st processing result and the 2 nd processing result,

and outputting a change instruction to change the 1 st process result to a 3 rd process result to the autonomous moving object according to the determined difference, wherein the 3 rd process result is obtained based on at least one of the 1 st process result and the 2 nd process result.

(modified) an information processing apparatus mounted on an autonomous moving body, wherein the information processing apparatus,

executing 1 st preprocessing for obtaining a 1 st processing result, the 1 st preprocessing being preprocessing of a travel control processing in autonomous movement of the autonomous moving body,

outputting the sensor data obtained by the autonomous moving body to an external device,

obtaining a 2 nd processing result from the external device, the 2 nd processing result being a result of performing a 2 nd preprocessing which is a preprocessing higher than the 1 st preprocessing, based on the sensed data, wherein the 2 nd preprocessing is performed by the external device connected to the autonomous moving body via a network, a processing capability of the external device is higher than that of the information processing device,

determining a difference between the 1 st processing result and the 2 nd processing result,

and changing the 1 st processing result into a 3 rd processing result according to the determined difference, wherein the 3 rd processing result is obtained according to at least one of the 1 st processing result and the 2 nd processing result.

14. The information processing apparatus according to claim 13,

outputting a request for execution of the 2 nd preprocessing to the external device,

obtaining the 2 nd processing result as a response to the request.

15. The information processing apparatus according to claim 14,

outputting the request to the external device, based on at least one of a resource possessed by the autonomous mobile body, a movement state of the autonomous mobile body, an external environment of the autonomous mobile body, a time of day, and a response time to an inquiry to the external device.

16. The information processing apparatus according to claim 14 or 15,

the request includes information specifying a specific process among the 2 nd preprocessing,

the 2 nd processing result is a result of executing the specific processing.

17. The information processing apparatus according to any one of claims 14 to 16,

when the 2 nd processing result is obtained after a predetermined time or more has elapsed from the output of the sensing data or the output of the request,

(A) the determination of the difference is not performed, or

(B) A difference between a part of the 1 st processing result and a processing result corresponding to the part of the 2 nd processing result is determined, and the part of the 1 st processing result is changed to the 3 rd processing result according to the determined difference.