阅读说明：本技术 收割机、障碍物判定程序、记录有障碍物判定程序的记录介质、障碍物判定方法、农业机械、控制程序、记录有控制程序的记录介质、控制方法 (Harvester, obstacle determination program, recording medium having obstacle determination program recorded thereon, obstacle determination method, agricultural machine, control program, recording medi) 是由 江户俊介 石见宪一 宫下隼辅 于 2020-04-24 设计创作，主要内容包括：收割机具备：机体主体(1)；收获部(15),所述收获部设置于机体主体(1)的前方,并且能够相对于机体主体(1)上下摆动；高度检测部,所述高度检测部能够检测收获部(15)所在的高度位置(H)；以及障碍物检测部,所述障碍物检测部能够检测行进方向前方的障碍物。障碍物检测部具备：第一传感器(21)及第二传感器(22),所述第一传感器及第二传感器设置在上下方向上的不同位置,并且将行进方向前方作为检测范围而输出检测信息；选择部,所述选择部根据收获部(15)的高度位置(H)来选择第一传感器(21)和第二传感器(22)各自的检测信息中的至少一方；以及判定部,所述判定部基于由选择部选择的检测信息,对障碍物进行判定。(The harvester is provided with: a body main body (1); a harvesting part (15) which is arranged in front of the machine body (1) and can swing up and down relative to the machine body (1); a height detection unit capable of detecting a height position (H) at which the harvesting unit (15) is located; and an obstacle detection unit capable of detecting an obstacle ahead in the direction of travel. The obstacle detection unit includes: a first sensor (21) and a second sensor (22) which are provided at different positions in the vertical direction and which output detection information with the forward direction of the traveling direction as a detection range; a selection unit that selects at least one of the detection information of the first sensor (21) and the detection information of the second sensor (22) according to the height position (H) of the harvesting unit (15); and a determination unit that determines the obstacle based on the detection information selected by the selection unit.)

1. A harvester, wherein the harvester is provided with:

a main body of the machine body;

a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body;

a height detection unit capable of detecting a height position of the harvesting unit; and

an obstacle detection unit capable of detecting an obstacle ahead in a traveling direction,

the obstacle detection unit includes:

a first sensor and a second sensor that are provided at different positions in the vertical direction and output detection information with the front in the traveling direction as a detection range;

a selection unit that selects at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and

a determination unit that determines the obstacle based on the detection information selected by the selection unit.

2. A harvester according to claim 1,

the first sensor and the second sensor are respectively arranged on the machine body,

the selection unit selects the detection information of the lower one of the first sensor and the second sensor when the height position of the harvesting unit is higher than a preset height position, and selects the detection information of the higher one of the first sensor and the second sensor when the height position of the harvesting unit is lower than the preset height position.

3. A harvester according to claim 1,

the first sensor and the second sensor are respectively arranged on the machine body,

the harvesting part has: a harvesting head, the harvesting head being capable of swinging up and down relative to the body main body; and a gathering wheel which can swing up and down relative to the harvesting head,

the selection unit selects at least one of the detection information of the first sensor and the detection information of the second sensor based on a height position of the harvesting head and a height position of the tucking wheel.

4. A harvester according to claim 1,

the first sensor and the second sensor are respectively arranged on the machine body,

the harvesting part has: a harvesting head, the harvesting head being capable of swinging up and down relative to the body main body; and a tucking wheel, the position of which can be changed back and forth relative to the harvesting head,

the selection unit selects at least one of the detection information of the first sensor and the detection information of the second sensor based on a height position of the harvesting head and a front-rear position of the tucking wheel.

5. A harvester according to claim 1,

the first sensor and the second sensor are respectively arranged on the harvesting part,

at least one of the first sensor and the second sensor is configured to be able to change a sensing direction according to a height position of the harvesting portion.

6. An obstacle determination program for a harvester, the harvester comprising: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and a first sensor and a second sensor which are provided at different positions in the vertical direction and output detection information with the forward direction of the traveling direction as a detection range, wherein,

the obstacle determination program is configured to cause a computer to realize a height detection function of detecting a height position where the harvesting unit is located and an obstacle detection function of detecting an obstacle ahead in a traveling direction,

the obstacle detection function includes:

a selection function of selecting at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and

a determination function that determines the obstacle based on the detection information selected by the selection function.

7. A recording medium having an obstacle determination program recorded thereon, the obstacle determination program being for a harvester, the harvester comprising: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and a first sensor and a second sensor which are provided at different positions in the vertical direction and output detection information with the forward direction of the traveling direction as a detection range, wherein,

the recording medium has recorded thereon an obstacle determination program for causing a computer to realize a height detection function for detecting a height position at which the harvesting unit is located and an obstacle detection function for detecting an obstacle ahead in a traveling direction,

the obstacle detection function includes:

a selection function of selecting at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and

a determination function that determines the obstacle based on the detection information selected by the selection function.

8. An obstacle determination method for a harvester, the harvester including: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and a first sensor and a second sensor that are provided at different positions in the vertical direction and that output detection information with the forward direction of the traveling direction as a detection range, wherein the obstacle determination method includes:

a height detection step of detecting a height position at which the harvesting unit is located; and

an obstacle detecting step of detecting an obstacle ahead in a traveling direction,

the obstacle detecting step includes:

a selection step of selecting at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and

a determination step of determining the obstacle based on the detection information selected by the selection step.

9. An agricultural machine capable of automatically traveling, comprising:

a camera device facing a field;

a detection device that is a sensor of a different kind from the image pickup device and that faces a field;

an obstacle detection unit that detects an obstacle around the body based on at least one of the captured image of the imaging device and a detection result of the detection device; and

a device control unit that controls a predetermined device,

when the obstacle detection unit detects an obstacle, the device control unit performs a detection-time control corresponding to the detection of the obstacle.

10. The agricultural machine of claim 9, wherein,

the obstacle detection unit includes: a first detection unit that detects an obstacle around the body based on a captured image of the imaging device; and a second detection unit that detects an obstacle around the body based on a detection result of the detection device,

the obstacle detection unit does not output a signal indicating that an obstacle is detected when the first detection unit does not detect the obstacle and the second detection unit does not detect the obstacle,

the obstacle detection unit outputs a signal indicating that an obstacle is detected when the obstacle is detected only by one of the first detection unit and the second detection unit,

the obstacle detection unit outputs a signal indicating that an obstacle is detected when both the first detection unit and the second detection unit detect an obstacle.

11. The agricultural machine of claim 9 or 10, wherein,

the agricultural machine is provided with a state determination unit for determining whether the imaging state of the imaging device is normal,

when the state determination unit determines that the imaging state of the imaging device is abnormal, the obstacle detection unit detects an obstacle around the body based on a detection result of the detection device.

12. The agricultural machine of claim 11,

the state determination unit determines whether or not the imaging state of the imaging device is normal based on at least one of a value indicating the imaging state of the imaging device and a captured image of the imaging device.

13. An agricultural machine according to any one of claims 9 to 12,

the detection device is a temperature distribution sensor that detects a temperature distribution in a field of view.

14. An agricultural machine according to any one of claims 9 to 12,

the detection device is a short-wave infrared sensor for detecting short-wave infrared rays.

15. An agricultural machine according to any one of claims 9 to 14, wherein,

the obstacle detection unit has a first detection unit that detects an obstacle around the body based on the image captured by the image capture device,

the first detection unit detects an obstacle around the body by using a neural network learned using deep learning.

16. An agricultural machine according to any one of claims 9 to 15, wherein,

the obstacle detection unit has a second detection unit that detects an obstacle around the body based on a detection result of the detection device,

the second detection unit detects an obstacle around the body by using a neural network learned using deep learning.

17. A control program for an agricultural machine capable of automatic travel, the agricultural machine comprising: a camera device facing a field; and a detection device which is a sensor of a different kind from the image pickup device and which faces a field, wherein,

the control program is configured to cause a computer to realize an obstacle detection function of detecting an obstacle around a body of the agricultural machine based on at least one of a captured image of the imaging device and a detection result of the detection device, and a device control function of controlling a predetermined device in the agricultural machine,

the device control function performs a detection-time control corresponding to the detection of the obstacle when the obstacle is detected by the obstacle detection function.

18. A recording medium having a control program recorded thereon, the control program being for an agricultural machine capable of automatic travel, the agricultural machine comprising: a camera device facing a field; and a detection device which is a sensor of a different kind from the image pickup device and which faces a field, wherein,

the recording medium has recorded thereon a control program for causing a computer to realize an obstacle detection function of detecting an obstacle around a body of the agricultural machine based on at least one of a captured image of the imaging device and a detection result of the detection device, and a device control function of controlling a predetermined device in the agricultural machine,

the device control function performs a detection-time control corresponding to the detection of the obstacle when the obstacle is detected by the obstacle detection function.

19. A control method for an agricultural machine capable of automatic travel, the agricultural machine comprising: a camera device facing a field; and a detection device that is a sensor of a different type from the imaging device and that faces a field, wherein the control method includes:

an obstacle detection step of detecting an obstacle around the body of the agricultural machine based on at least one of a captured image of the imaging device and a detection result of the detection device; and

a device control step of controlling a predetermined device in the agricultural machine,

when an obstacle is detected in the obstacle detecting step, the device controlling step performs a detection-time control corresponding to the detection of the obstacle.

Technical Field

The present invention relates to a harvester, which comprises: a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and an obstacle detection unit capable of detecting an obstacle ahead in the direction of travel.

The present invention also relates to an agricultural machine including an imaging device directed toward a field.

Background

[1] For example, a harvester disclosed in patent document 1 (in this document, "combine harvester") includes an obstacle detection unit (in this document, "ultrasonic sensor") that is a sensor that outputs detection information with a detection range at the front in the traveling direction.

[2] Patent document 2 describes an invention of an agricultural machine (a "combine" in patent document 1) capable of automatic travel.

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2017-169539

Patent document 2: japanese Kokai publication Hei-2-107911

Disclosure of Invention

Problems to be solved by the invention

[1] The problems associated with the background art [1] are as follows.

However, in the configuration in which the harvesting unit is provided in front of the body main body so as to be vertically swingable, it is considered that there is a possibility that the detection of the obstacle in front may not be appropriately performed by the sensor depending on the height position of the harvesting unit. Accordingly, the present invention provides a harvester capable of detecting an obstacle ahead in a traveling direction with high accuracy regardless of a height position of a harvesting unit.

[2] The problems associated with the background art [2] are as follows.

Patent document 2 does not describe detection of an obstacle around the body. Here, in the agricultural machine described in patent document 2, it is conceivable to adopt the following configuration: the obstacle detection device is provided with an imaging device that faces a field, and detects an obstacle around the body based on an image captured by the imaging device.

However, in this configuration, when fog or dust is generated, it is assumed that a captured image is not clear. Further, when the captured image is unclear, it is assumed that the detection accuracy of the obstacle is lowered.

The invention aims to provide an agricultural machine with good obstacle detection precision.

Means for solving the problems

[1] The solution corresponding to the problem [1] is as follows.

The harvester of the present invention is characterized by comprising: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; a height detection unit capable of detecting a height position of the harvesting unit; and an obstacle detection unit capable of detecting an obstacle ahead in a traveling direction, the obstacle detection unit including: a first sensor and a second sensor that are provided at different positions in the vertical direction and output detection information with the front in the traveling direction as a detection range; a selection unit that selects at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and a determination unit that determines the obstacle based on the detection information selected by the selection unit.

According to the present invention, the plurality of sensors, which have a detection range in front of the traveling direction, are vertically separated from each other. Therefore, even when the obstacle in front is not appropriately detected by one of the first sensor and the second sensor depending on the height position of the harvesting unit, the obstacle can be appropriately detected by the other of the first sensor and the second sensor. In addition, even when the height position of the harvesting unit is determined by the height position of the sensors at different heights, the detection information of the sensors is selected by the selection unit according to the present invention. That is, a sensor capable of accurately detecting an obstacle ahead in the traveling direction is selected from the plurality of sensors based on the height position of the harvesting unit. Thus, a harvester capable of detecting an obstacle ahead in the traveling direction with high accuracy regardless of the height position of the harvesting unit can be realized.

In the present invention, it is preferable that the first sensor and the second sensor are provided in the body main body, respectively, and the selection unit selects the lower detection information of the first sensor and the second sensor when the height position of the harvesting unit is higher than a preset height position, and selects the higher detection information of the first sensor and the second sensor when the height position of the harvesting unit is lower than the preset height position.

According to this structure, the first sensor and the second sensor are separated vertically in the body main body, respectively, and therefore, the first sensor and the second sensor are firmly supported by the body main body, respectively. Since the harvesting portion is disposed in front of the main body, in a structure in which the first sensor and the second sensor are respectively disposed in the main body, the harvesting portion may block a detection range of the first sensor or a detection range of the second sensor. According to the present invention, based on the set height position, when the height position of the harvesting unit is low, the detection information of the sensor located on the higher side is selected, and when the height position of the harvesting unit is high, the detection information of the sensor located on the lower side is selected. Thus, the determination unit can appropriately determine the obstacle ahead. In the present invention, the term "the height position of the harvesting unit is higher than a preset height position" also includes the term "the height position of the harvesting unit is equal to or higher than the preset height position". In the present invention, the phrase "in the case where the height position of the harvesting unit is lower than the set height position" also includes the meaning of "in the case where the height position of the harvesting unit is equal to or lower than the set height position".

In the present invention, it is preferable that the first sensor and the second sensor are provided in the body main body, respectively, and the harvesting unit includes: a harvesting head, the harvesting head being capable of swinging up and down relative to the body main body; and the gathering wheel can swing up and down relative to the harvesting head, and the selection part selects at least one of the detection information of the first sensor and the detection information of the second sensor based on the height position of the harvesting head and the height position of the gathering wheel.

If the selection unit selects the respective detection information of the first sensor and the second sensor based on only the height position of the harvesting head without considering the height position of the tucking wheel, the detection range of the first sensor or the detection range of the second sensor is blocked by the tucking wheel, and the determination unit may not be able to appropriately determine the obstacle ahead. In this configuration, the selection unit selects at least one of the detection information of the first sensor and the detection information of the second sensor in consideration of the height position of the harvesting head and the height position of the tucking wheel. For example, even when the height position of the harvesting head is high, if the height position of the tucking wheels is low relative to the harvesting head, there is a possibility that an obstacle ahead in the traveling direction can be detected with high accuracy based on the detection information on the higher side of the first sensor and the second sensor. That is, with this configuration, the selection unit can select at least one of the first sensor and the second sensor so as to avoid the leading harvesting head and the tucking wheel as much as possible. Therefore, even when the tucking wheels swing up and down with respect to the harvesting head, the plurality of sensors can be used separately according to the height position of the harvesting unit, and the determination unit can appropriately determine the obstacle ahead.

In the present invention, it is preferable that the first sensor and the second sensor are provided in the body main body, respectively, and the harvesting unit includes: a harvesting head, the harvesting head being capable of swinging up and down relative to the body main body; and a tucking wheel, the position of which can be changed forward and backward relative to the harvesting head, the selection part selecting at least one of the detection information of the first sensor and the detection information of the second sensor based on the height position of the harvesting head and the forward and backward position of the tucking wheel.

In the structure that the gathering wheel can change the position relative to the harvesting head, if the gathering wheel changes the position, the separation distance between the gathering wheel and the first sensor and the second sensor arranged on the machine body changes. In this case, by bringing the tucking wheel close to the first sensor or the second sensor, there is a possibility that the detection range of the first sensor or the detection range of the second sensor is blocked by the tucking wheel. In this configuration, the selection unit selects at least one of the detection information of the first sensor and the detection information of the second sensor in consideration of the height position of the harvesting head and the front-rear position of the tucking wheel. For example, it is considered that the closer the tucking wheel is located to the front side, the smaller the proportion of the detection range blocked by the tucking wheel in the detection range of the first sensor or the detection range of the second sensor. That is, with this configuration, the selection unit can select the first sensor or the second sensor so as to avoid the harvesting head and the tucking wheel in front as much as possible. Therefore, even when the tucking wheels are shifted forward and backward with respect to the harvesting head, the selection unit can use a plurality of sensors separately according to the forward and backward positions of the tucking wheels, and the determination unit can appropriately determine the obstacle ahead.

In the present invention, it is preferable that the first sensor and the second sensor are provided in the harvesting unit, respectively, and at least one of the first sensor and the second sensor is configured to be capable of changing a sensing direction according to a height position of the harvesting unit.

In the configuration in which the first sensor and the second sensor are provided in the harvesting unit, respectively, the first sensor and the second sensor are inclined up and down along with the vertical swing of the harvesting unit. Therefore, the detection ranges of the first sensor and the second sensor are shifted upward from the front side in the traveling direction or shifted downward from the front side in the traveling direction. In this configuration, the direction of sensing by at least one of the first sensor and the second sensor can be adjusted. Thus, at least one of the first sensor and the second sensor can keep the detection range forward in the traveling direction regardless of the vertical swing of the harvesting unit. Thus, the obstacle detection unit can accurately detect an obstacle ahead in the traveling direction regardless of the height position of the harvesting unit.

Another aspect of the present invention is an obstacle determination program for a harvester including: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and a first sensor and a second sensor that are provided at different positions in a vertical direction and that output detection information with a forward direction of travel as a detection range, wherein the obstacle determination program is configured to cause a computer to realize a height detection function of detecting a height position at which the harvesting unit is located and an obstacle detection function of detecting an obstacle in the forward direction of travel, the obstacle detection function including: a selection function of selecting at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and a determination function that determines the obstacle based on the detection information selected by the selection function.

Another aspect of the present invention is summarized as a recording medium having an obstacle determination program recorded thereon, the recording medium having the obstacle determination program recorded thereon, the harvester including: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and a first sensor and a second sensor that are provided at different positions in a vertical direction and that output detection information with a forward direction of travel as a detection range, wherein the recording medium has recorded thereon an obstacle determination program that causes a computer to implement a height detection function of detecting a height position at which the harvesting unit is located and an obstacle detection function of detecting an obstacle in the forward direction of travel, the obstacle detection function including: a selection function of selecting at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and a determination function that determines the obstacle based on the detection information selected by the selection function.

Another aspect of the present invention is summarized as an obstacle determination method for a harvester including: a main body of the machine body; a harvesting part which is arranged in front of the machine body and can swing up and down relative to the machine body; and a first sensor and a second sensor that are provided at different positions in the vertical direction and that output detection information with the forward direction of the traveling direction as a detection range, wherein the obstacle determination method includes: a height detection step of detecting a height position at which the harvesting unit is located; and an obstacle detecting step of detecting an obstacle ahead in the traveling direction, the obstacle detecting step including: a selection step of selecting at least one of the detection information of each of the first sensor and the second sensor according to a height position of the harvesting unit; and a determination step of determining the obstacle based on the detection information selected by the selection step.

[2] The solution corresponding to the problem [2] is as follows.

An agricultural machine according to the present invention is an agricultural machine capable of automatically traveling, the agricultural machine including: a camera device facing a field; a detection device that is a sensor of a different kind from the image pickup device and that faces a field; an obstacle detection unit that detects an obstacle around the body based on at least one of the captured image of the imaging device and a detection result of the detection device; and a device control unit that controls a predetermined device, and when the obstacle detection unit detects an obstacle, the device control unit performs detection-time control that is control in accordance with the detection of the obstacle.

According to the present invention, for example, in a case where a captured image of an imaging device is unclear, an obstacle around a body can be detected based on a detection result of a detection device in a situation where the accuracy of obstacle detection based only on the captured image of the imaging device is likely to decrease.

Therefore, according to the present invention, it is possible to realize an agricultural machine in which the accuracy of obstacle detection is less likely to decrease in a situation where the accuracy of obstacle detection based only on captured images is likely to decrease.

That is, according to the present invention, an agricultural machine with excellent obstacle detection accuracy can be realized.

Further, according to the present invention, an agricultural machine that performs appropriate control when an obstacle is detected can be realized.

In the present invention, it is preferable that the obstacle detecting unit includes: a first detection unit that detects an obstacle around the body based on a captured image of the imaging device; and a second detection unit that detects an obstacle around the body based on a detection result of the detection device, wherein the obstacle detection unit does not output a signal indicating that the obstacle is detected when the obstacle is not detected by the first detection unit and is not detected by the second detection unit, wherein the obstacle detection unit outputs a signal indicating that the obstacle is detected when the obstacle is detected only by one of the first detection unit and the second detection unit, and wherein the obstacle detection unit outputs a signal indicating that the obstacle is detected when the obstacle is detected by both the first detection unit and the second detection unit.

According to this configuration, the obstacle detecting unit outputs a signal indicating that the obstacle is detected, when the obstacle is detected only by one of the first detecting unit and the second detecting unit. Therefore, detection omission of the obstacle detecting unit can be reduced as compared with a configuration in which a signal indicating that an obstacle is detected is not output when an obstacle is detected only by one of the first detecting unit and the second detecting unit.

In the present invention, it is preferable that the imaging apparatus further includes a state determination unit that determines whether or not the imaging state of the imaging apparatus is normal, and the obstacle detection unit detects an obstacle around the body based on a detection result of the detection device when the state determination unit determines that the imaging state of the imaging apparatus is not normal.

According to this configuration, when the imaging state of the imaging device is abnormal, the obstacle detection is performed based on the detection result of the detection device. Therefore, according to this configuration, even when the imaging state of the imaging device is abnormal, it is possible to realize an agricultural machine in which the obstacle detection accuracy is not easily lowered.

In the present invention, it is preferable that the state determination unit determines whether or not the imaging state of the imaging device is normal based on at least one of a value indicating the imaging state of the imaging device and a captured image of the imaging device.

With this configuration, the state determination unit can easily and accurately determine whether or not the imaging state of the imaging device is normal. Thus, when the imaging state of the imaging device is abnormal, an agricultural machine which can easily and reliably detect an obstacle based on the detection result of the detection device can be realized.

In the present invention, it is preferable that the detection device is a temperature distribution sensor that detects a temperature distribution in a field of view.

According to this configuration, for example, when the captured image of the imaging device is unclear, the obstacle around the body can be detected with high accuracy based on the detection result of the temperature distribution sensor in a situation where the accuracy of obstacle detection based on the captured image alone is likely to decrease.

In the present invention, it is preferable that the detection device is a short-wave infrared sensor that detects short-wave infrared rays.

According to this configuration, for example, when the captured image of the imaging device is unclear, the obstacle around the body can be detected with high accuracy based on the detection result of the short-wave infrared sensor under the situation where the accuracy of the obstacle detection based on the captured image alone is likely to decrease.

In the present invention, it is preferable that the obstacle detecting unit includes a first detecting unit that detects an obstacle around the body based on a captured image of the imaging device, and the first detecting unit detects the obstacle around the body by using a neural network learned by deep learning.

According to this configuration, it is possible to detect an obstacle with high accuracy using a captured image of the imaging device by using the neural network learned by the deep learning.

In the present invention, it is preferable that the obstacle detecting unit includes a second detecting unit that detects an obstacle around the body based on a detection result of the detecting device, and the second detecting unit detects the obstacle around the body by using a neural network learned by deep learning.

According to this configuration, it is possible to detect an obstacle with high accuracy based on the detection result of the detection device by using the neural network learned by the deep learning.

Another aspect of the present invention is summarized as a control program for an agricultural machine capable of automatic travel, including: a camera device facing a field; and a detection device that is a sensor of a type different from that of the imaging device and faces a field, wherein the control program is configured to cause a computer to realize an obstacle detection function of detecting an obstacle around a body of the agricultural machine based on at least one of a captured image of the imaging device and a detection result of the detection device, and a device control function of controlling a predetermined device in the agricultural machine, and the device control function performs a detection-time control corresponding to detection of the obstacle when the obstacle is detected by the obstacle detection function.

Another aspect of the present invention is summarized as a recording medium having a control program recorded thereon, the control program being for an agricultural machine capable of automatic traveling, the agricultural machine including: a camera device facing a field; and a detection device that is a sensor of a type different from that of the image pickup device and faces a field, wherein the recording medium has recorded thereon a control program that causes a computer to realize an obstacle detection function of detecting an obstacle around a body of the agricultural machine based on at least one of a captured image of the image pickup device and a detection result of the detection device, and a device control function of controlling a predetermined device in the agricultural machine, and the device control function performs control corresponding to detection of the obstacle when the obstacle is detected by the obstacle detection function.

Another aspect of the present invention is summarized as a control method for an agricultural machine capable of automatic travel, including: a camera device facing a field; and a detection device that is a sensor of a different type from the imaging device and that faces a field, wherein the control method includes: an obstacle detection step of detecting an obstacle around the body of the agricultural machine based on at least one of a captured image of the imaging device and a detection result of the detection device; and a device control step of controlling a predetermined device in the agricultural machine, wherein when the obstacle is detected in the obstacle detection step, the device control step performs a detection-time control corresponding to the detection of the obstacle.

Drawings

Fig. 1 is a view showing a first embodiment (hereinafter, the same applies to fig. 10), and is an overall side view of a harvester.

Fig. 2 is a functional block diagram showing a control system of the harvester.

Fig. 3 is a flowchart showing the processing of the selection unit.

Fig. 4 is a side view showing selection of detection information of each of the first sensor and the second sensor according to the height position of the harvesting unit.

Fig. 5 is a side view showing selection of detection information of each of the first sensor and the second sensor according to the height position of the harvesting unit.

Fig. 6 is a side view showing selection of detection information of each of the first sensor and the second sensor according to the height position of the harvesting unit.

Fig. 7 is a flowchart showing another embodiment of the process of the selection unit.

Fig. 8 is a side view showing another embodiment of selection of detection information of each of the first sensor and the second sensor according to the height position of the harvesting unit.

Fig. 9 is a side view showing another embodiment of selection of detection information of each of the first sensor and the second sensor according to the height position of the harvesting unit.

Fig. 10 is a side view showing a state in which a first sensor and a second sensor are provided in a harvesting unit as another embodiment.

Fig. 11 is a view showing a second embodiment (hereinafter, the same applies to fig. 15), and is a left side view of the combine harvester.

Fig. 12 is a plan view showing the structure and the visual field area of the combine harvester.

Fig. 13 is a block diagram showing a configuration related to the control unit.

Fig. 14 is a diagram illustrating a flow of obstacle detection by the first detection unit.

Fig. 15 is a diagram illustrating a flow of obstacle detection by the second detection unit.

Detailed Description

[ first embodiment ]

The first embodiment will be described below with reference to fig. 1 to 10.

Hereinafter, an embodiment of a combine harvester as an example of the harvester of the present invention will be described based on the drawings. In this embodiment, when the front-rear direction of the machine body 1 is defined, the definition is made along the machine body traveling direction in the working state. In fig. 1, the direction indicated by reference numeral (F) is the body front side, and the direction indicated by reference numeral (B) is the body rear side. The direction shown by reference numeral (U) is the upper side of the body, and the direction shown by reference numeral (D) is the lower side of the body. The direction on the front side of the drawing of fig. 1 is the left side of the body, and the direction on the back side of the drawing of fig. 1 is the right side of the body. When defining the left-right direction of the body main body 1, the left-right direction is defined in a state viewed from the body traveling direction.

[ basic structure of harvester ]

As shown in fig. 1, a whole-feed combine harvester as one embodiment of the harvester includes a main body 1, a pair of left and right crawler-type traveling devices 11, a boarding unit 12, a threshing device 13, a grain tank 14, a harvesting unit 15, a conveying device 16, and a grain discharge device 17.

The running gear 11 is arranged at the lower part of the combine harvester. The combine harvester can travel by itself through the travel device 11. The boarding unit 12, the threshing device 13, and the grain tank 14 are provided above the traveling device 11, and constitute an upper portion of the machine body 1. A rider of the combine harvester and a monitor monitoring the operation of the combine harvester can ride on the riding part 12. A satellite positioning module 80 is provided on the top of the boarding portion 12. The Satellite positioning module 80 receives signals (including GPS signals) from a GNSS (Global Navigation Satellite System) of the artificial Satellite GS, and acquires the vehicle position. An engine (not shown) for driving is provided below the boarding portion 12. The grain discharging device 17 is connected to the lower rear portion of the grain box 14.

The harvesting unit 15 is supported by the front part of the machine body 1. The conveyor 16 is disposed adjacent to the rear side of the harvesting unit 15. The harvesting section 15 harvests crops in a field. The crop to be harvested is, for example, standing straw such as rice, but may be soybean, corn, or the like. The harvesting unit 15 and the transport device 16 are configured to be vertically swingable about a transverse axis of the machine body by telescopic operation of a hydraulic cylinder, not shown, which is stretched across the machine body 1 and the transport device 16. With this configuration, the harvesting unit 15 can adjust the height of the ground when harvesting the crop in the field. The combine harvester can travel by the traveling device 11 while harvesting the crop in the field by the harvesting unit 15.

The harvesting unit 15 includes a harvesting head 15A and a tucking wheel 15B. When harvesting a crop from the field, the raking wheel 15B rakes the top portion of the crop rearward. The harvesting head 15A collects crops (e.g., harvested straws) harvested from a field at a portion of the harvesting portion 15 that is connected to the inside of the conveyor 16 in communication therewith. The harvesting head 15A is configured to be vertically swingable with respect to the machine body 1, and is configured to swing integrally with the conveyor 16. The tucking wheels 15B are configured to be vertically swingable with respect to the harvesting head 15A, and are configured to be changeable in position forward and backward with respect to the harvesting head 15A.

The crop (e.g., harvested straw) harvested by the harvesting portion 15 is transported to the threshing device 13 by a transport device 16. The harvested crops are threshed by a threshing device 13. Grains obtained by the threshing process as a harvest are stored in the grain tank 14. The grains stored in the grain tank 14 are discharged to the outside of the machine through the grain discharging device 17 as needed. The grain discharging device 17 is configured to be capable of swinging around a longitudinal axis core at the rear part of the machine body.

A first sensor 21 and a second sensor 22 are provided at the left end of the front wall portion of the threshing device 13. The first sensor 21 and the second sensor 22 are each sonar, for example, and the forward direction in the traveling direction is set as the detection range. The first sensor 21 and the second sensor 22 are disposed at vertically different positions, and the first sensor 21 is disposed above the second sensor 22.

Various objects exist in the field as detection targets of the first sensor 21 and the second sensor 22. As such detection targets, fig. 1 schematically shows a normal standing grain stalk group shown by reference numeral Z0, a weed group shown by reference numeral Z1, a lodging grain stalk group shown by reference numeral Z2, a person shown by reference numeral Z3, and a stone shown by reference numeral Z4.

[ Structure of control Unit ]

The control unit 30 shown in fig. 2 is a core element of a control system of a combine harvester, and is shown as an aggregate of a plurality of ECUs. The control unit 30 includes a selection unit 31, a determination unit 32, a control unit 34, a travel control unit 35, a warning control unit 36, and a notification control unit 37. The selection unit 31 and the determination unit 32 are also part of the obstacle detection unit 33.

The obstacle detector 33 is configured to detect an obstacle in front of the combine in the traveling direction. The obstacle detecting unit 33 includes the first sensor 21, the second sensor 22, the selecting unit 31, and the determining unit 32. Positioning information output from the satellite positioning module 80, detection information output from each of the first sensor 21 and the second sensor 22, and height position information output from the harvesting height detection unit 23, which is a "height detection unit" in the present invention, are input to the control unit 30 through a wiring network. As described above, the harvesting unit 15 and the conveyor 16 (see fig. 1, 4, and the like) are configured to be vertically swingable. The harvesting height detector 23 is provided at a swing axis of the conveyor 16. The harvesting height detector 23 is configured to detect height position information of the harvesting unit 15 by detecting a swing angle of the conveyor 16. The height position information of the harvesting unit 15 is information related to the height position H (see fig. 4 and the like) of the harvesting unit 15. In the present embodiment, the height position H is the height of the harvesting portion 15 to the ground. In the present embodiment, since the first sensor 21 and the second sensor 22 are sonars, the detection information output from the first sensor 21 and the second sensor 22 is distance measurement information.

The determination unit 32 has the following functions: the specific information is determined as the obstacle based on the detection information transmitted from at least one of the first sensor 21 and the second sensor 22, for example, by using a neural network of machine learning (deep learning).

As will be described in detail later, the height position H of the harvesting unit 15 is detected by the harvesting height detecting unit 23 (see fig. 4 and the like), and the selecting unit 31 selects at least one of the detection information of the first sensor 21 and the detection information of the second sensor 22 based on the height position H. In other words, the selection unit 31 is configured to be able to select which of the detection information of each of the first sensor 21 and the second sensor 22 is used based on the harvesting height of the harvesting unit 15. Of the detection information of the first sensor 21 and the second sensor 22, the detection information selected by the selection unit 31 is transmitted to the determination unit 32. In this way, the obstacle detecting unit 33 is configured to be able to detect an obstacle ahead in the traveling direction.

The control unit 34 determines a control mode based on the type of the obstacle detected by the obstacle detection unit 33. The control pattern is stored in, for example, a ROM as a look-up table corresponding to the type of the obstacle, and the control pattern corresponding to the type of the obstacle is selected by the control unit 34. Then, the output control is output from the control unit 34 to the travel control unit 35, the warning control unit 36, and the notification control unit 37, respectively, in accordance with the selected control mode.

The travel control unit 35 has an engine control function, a steering control function, a vehicle speed control function, and the like, and supplies a travel control signal to the travel device 11. When the combine is driven manually, the travel control unit 35 generates a control signal based on an operation by the rider and controls the travel device 11. In the case of automatic driving of the combine harvester, the travel control unit 35 controls the travel device 11 with respect to steering and vehicle speed based on an automatic travel command provided from an automatic travel control module of the control unit 30 and positioning information from the satellite positioning module 80. The travel control unit 35 is configured to be able to output a deceleration command and a stop command to the travel device 11 based on output control from the control unit 34.

The warning control unit 36 is a module for notifying animals and humans, such as a travel route located in front of the machine body 1 shown in fig. 1, of the state of work travel of the machine body 1 and various warnings, and is configured to be able to control the output of the horn 41. The horn 41 is provided at an arbitrary position of the main body 1. The notification control unit 37 is configured to output the control mode determined by the control unit 34 to the terminal CT such as a smartphone or a tablet computer, for example, and display the control mode on the terminal CT. The terminal CT is carried by a rider of the combine, a supervisor or manager of the field. The notification control unit 37 is configured so that a person carrying the terminal CT can confirm the state and history of the control mode by the terminal CT.

[ selection of first and second sensors ]

As described above, the selection unit 31 shown in fig. 2 is configured to select at least one of the detection information of the first sensor 21 and the detection information of the second sensor 22 in accordance with the height position H of the harvesting unit 15 shown in fig. 4 and the like. Fig. 3 shows a process of selecting the detection information of each of the first sensor 21 and the second sensor 22 by the selection unit 31. The selection process from the start to the end shown in fig. 3 is repeated at a constant cycle.

The selection unit 31 obtains the height position H, which is the position of the harvesting unit 15 in the vertical direction (step # 01). The height position H is height position information output from the harvesting height detecting unit 23 to the selecting unit 31. After obtaining the height position H, the selector 31 determines whether or not the height position H is equal to or less than a preset height position H1 (or less than a preset height position H1) (step # 02). If the height position H is equal to or lower than the set height position H1 (YES in step #02), the selection unit 31 selects the detection information of the first sensor 21 (step # 03). The height position of the first sensor 21 is higher than the height position of the second sensor 22. If the height position H is higher than the set height position H1 (NO in step #02), the routine proceeds to step # 04. The value of the set height position H1 can be changed as appropriate.

After the process of step #02 or step #03 is completed, the selector 31 determines whether or not the height position H is equal to or higher than a preset height position H2 (or higher than a preset height position H2) (step # 04). If the height position H is equal to or higher than the set height position H2 (YES in step #04), the selection unit 31 selects the detection information of the second sensor 22 (step # 05). The height position of the second sensor 22 is lower than the height position of the first sensor 21. If the height position H is lower than the set height position H2 (no in step #04), the process of the selection unit 31 ends. The value of the set height position H2 can be changed as appropriate.

Fig. 4 to 6 show a state in which at least one of the first sensor 21 and the second sensor 22 is selected according to the height position H of the harvesting portion 15. As described above, the height position H is acquired by the height-of-height detector 23 (see fig. 2), and the set height positions H1 and H2 are threshold values set in advance for the height position H. The set height positions H1 and H2 shown in fig. 4 to 6 are set to different values, respectively, and the set height position H1 is set to be higher than the set height position H2. The height position H of the harvesting portion 15 shown in fig. 4 to 6 is based on the position of the lower end of the harvesting portion 15. In the up-down direction, a region below the set height position H1 (or less than the set height position H1) shows a height region S1. When the height position H of the harvesting unit 15 is within the range of the height area S1, a determination of yes is made in step #02 shown in fig. 3, and the selection unit 31 (see fig. 2) selects the detection information of the first sensor 21. The height region S2 is shown in a region above the set height position H2 (or higher than the set height position H2). When the height position H of the harvesting unit 15 is within the range of the height area S2, yes is determined in step #04 shown in fig. 3, and the selection unit 31 selects the detection information of the second sensor 22.

In fig. 4, the height position H of the harvesting portion 15 is lower than the set height position H1 and lower than the set height position H2. That is, the height position H of the harvesting portion 15 is located within the range of the height section S1, but is located outside the range of the height section S2. In this case, a determination of yes is made in step #02 shown in fig. 3, and a determination of no is made in step #04, and the selection unit 31 selects only the detection information of the first sensor 21.

In fig. 5, the height position H of the harvesting portion 15 is higher than the set height position H1 and higher than the set height position H2. That is, the height position H of the harvesting portion 15 is located within the range of the height section S2, but is located outside the range of the height section S1. In this case, the determination of "no" is performed in step #02 shown in fig. 3, and the determination of "yes" is performed in step #04, and the selection unit 31 selects only the detection information of the second sensor 22.

In fig. 6, the height position H of the harvesting section 15 is located between the set height position H1 and the set height position H2. That is, the height position H of the harvesting portion 15 is located within the range of the height region S1 and within the range of the height region S2. In this case, a yes determination is made in step #02 shown in fig. 3, and a yes determination is also made in step #04, and the selection unit 31 selects both the detection information of the first sensor 21 and the detection information of the second sensor 22.

As described above, in the present invention, in the determination of the obstacle by the determination unit 32, there are a case where either one of the first sensor 21 and the second sensor 22 is used and a case where both the first sensor 21 and the second sensor 22 are used simultaneously.

[ other embodiments of the first embodiment ]

Another embodiment obtained by modifying the above embodiment will be described below. The present invention is similar to the embodiments described above except for the matters described in the following other embodiments. The above-described embodiments and the following other embodiments may be appropriately combined within a range where no contradiction occurs. The scope of the present invention is not limited to the above-described embodiments and the following other embodiments.

(1) In the above embodiment, the harvesting-height detecting unit 23 is configured to detect the height position H at which the harvesting unit 15 is located by detecting the swing angle of the conveyor 16, but the present invention is not limited to this embodiment. As described above, the tucking wheels 15B are configured to be vertically swingable with respect to the harvesting head 15A. Therefore, for example, the harvesting height detecting unit 23 may be configured to be able to detect the swing angle of the conveyor 16 and the swing angle of the tucking wheel 15B.

Preferably, the set height positions H1 and H2 are changed according to the relative positional relationship between the tucking wheels 15B and the harvesting head 15A. Specifically, as described with reference to fig. 4 to 6, when the tucking wheel 15B approaches the harvesting head 15A in the vertical direction, the visibility above the tucking wheel 15B is improved. Therefore, even in the case where the height position H of the harvesting portion 15 is higher than the set height position H1, the tucking wheels 15B may not interfere with the sensing of the first sensor 21. According to the above-described case, for example, the following configuration is also possible: the set height position H1 is adjusted upward as the tucking wheels 15B approach the harvesting head 15A, and the range of the height region S1 is expanded upward. When the tucking wheel 15B is separated from the harvesting head 15A and lifted, a visual confirmation space is secured between the harvesting head 15A and the tucking wheel 15B. Therefore, even when the height position H of the harvesting unit 15 is lower than the set height position H2, the second sensor 22 may sense the height position from the gap between the harvesting head 15A and the tucking wheels 15B. According to the above-described case, for example, the following configuration is also possible: the set height position H2 is adjusted downward as the tucking wheel 15B is farther from the harvesting head 15A, and the range of the height region S2 is expanded downward. In this way, the selection unit 31 may be configured to select at least one of the detection information of the first sensor 21 and the detection information of the second sensor 22 based on the height position of the harvesting head 15A and the height position of the tucking wheel 15B.

(2) In the above embodiment, the two set height positions H1, H2 having different set values are provided, but one of the pieces of detection information of the first sensor 21 and the second sensor 22 may be selected based on one set height position H3, as shown in fig. 7 to 9, for example. Fig. 7 shows another embodiment of the selection process of the detection information of each of the first sensor 21 and the second sensor 22 by the selection unit 31. The selection process from the start to the end shown in fig. 7 is repeated at a constant cycle. Similarly to step #01 of fig. 3, the selection unit 31 obtains the height position H of the harvesting unit 15 (step # 11). After obtaining the height position H, the selector 31 determines whether or not the height position H is equal to or less than a preset height position H3 (or less than a preset height position H3) (step # 12). If the height position H is equal to or lower than the set height position H3 (YES in step #12), the selection unit 31 selects the detection information of the first sensor 21 (step # 13). When the determination of "no" is made at step #12, the selection unit 31 selects the detection information of the second sensor 22 (step # 14). The value of the set height position H3 can be changed as appropriate.

In the vertical direction in fig. 8 and 9, the height region S1 is shown in a region below the set height position H3 (or below the set height position H3), and the height region S2 is shown in a region above the set height position H3 (or above the set height position H3). In fig. 8, the height position H of the harvesting part 15 is lower than the set height position H3 and is located only within the range of the height region S1. In this case, a yes determination is made in step #12 shown in fig. 7, and the selection unit 31 selects only the detection information of the first sensor 21. In fig. 9, the height position H of the harvesting unit 15 is higher than the set height position H3 and is located only within the range of the height region S2. In this case, no is determined in step #12 shown in fig. 7, and the selection unit 31 selects only the detection information of the second sensor 22. That is, when the height position H of the harvesting unit 15 is equal to or less than the set height position H3, the selection unit 31 selects the detection information of the higher one of the first sensor 21 and the second sensor 22. When the height position H of the harvesting unit 15 is equal to or greater than the set height position H3, the selection unit 31 selects the lower detection information of the first sensor 21 and the second sensor 22.

Further, the following structure is also possible: in connection with the above-described other embodiment (1), the range of the height region S2 is expanded downward as the tucking wheel 15B is further away from the harvesting head 15A and the set height position H3 is adjusted downward. Further, the following structure is also possible: the set height position H3 is adjusted upward as the tucking wheels 15B approach the harvesting head 15A, and the range of the height region S1 is expanded upward.

(3) In the above embodiment, the harvesting-height detecting unit 23 is configured to detect the height position H at which the harvesting unit 15 is located by detecting the swing angle of the conveyor 16, but the present invention is not limited to this embodiment. As described above, the tucking wheels 15B are configured to be able to change positions in the front-rear direction with respect to the harvesting head 15A. Therefore, for example, the harvesting height detecting unit 23 may be configured to be able to detect the swing angle of the conveyor 16 and the front-rear position of the tucking wheel 15B. That is, the selection unit 31 may be configured to select at least one of the detection information of the first sensor 21 and the detection information of the second sensor 22 based on the height position of the harvesting head 15A and the front-rear position of the tucking wheel 15B. For example, it is considered that the closer the tucking wheel 15B is located to the front side, the smaller the proportion of the detection range blocked by the tucking wheel 15B in the detection range of the first sensor 21 or the detection range of the second sensor 22. In this case, the set height position H1 shown in fig. 4 to 6 may be adjusted up and down according to the front-rear position of the tucking wheel 15B. For example, the following structure is also possible: the reach wheel 15B is adjusted to the set height position H1 to the upper side as it is located at the front side, and the range of the height region S1 is expanded to the upper side. The set height position H2 shown in fig. 4 to 6 may be adjusted up and down according to the front-rear position of the tucking wheels 15B. For example, the following structure is also possible: the reach wheel 15B is adjusted to the set height position H2 downward toward the front side, and the range of the height region S2 is expanded downward.

(4) The first sensor 21 and the second sensor 22 may be provided in the harvesting unit 15, respectively. For example, as shown in fig. 10, the following structure is also possible: second sensors 22, 22 are provided in respective regions of a pair of left and right dividers in the harvesting unit 15, and first sensors 21, 21 are provided at respective positions on the rear side of the tucking wheel 15B and on the right and left sides of the tucking wheel 15B. As shown in fig. 10, the swing members 15C are provided at both left and right end portions of the harvesting head 15A so as to be vertically swingable, and the tucking wheels 15B are supported by free end portions of the swing members 15C. The first sensors 21 and 21 may be provided on the rocking base end sides of the left and right rocking members 15C and 15C, respectively. Further, the following structure is also possible: when the height position H of the harvesting unit 15 is lower than the set height position H1, the selection unit 31 selects the first sensor 21. When the harvesting operation is performed with the harvesting unit 15 lowered, the second sensor 22 is positioned near the root of the crop in the field, and therefore, it is difficult to detect the obstacle ahead, but with this configuration, the first sensor 21 can easily detect the obstacle ahead from a position higher than the second sensor 22. Further, the following structure is also possible: when the height position H of the harvesting unit 15 is higher than the set height position H2, the selection unit 31 selects the second sensor 22. In the state where the harvesting unit 15 is raised, it is considered that the first sensor 21 is raised too high from the ground and it is difficult to detect the obstacle in front, but with this configuration, the second sensor 22 is easy to detect the obstacle in front from a position lower than the first sensor 21.

In the example shown in fig. 10, it is considered that when the harvesting unit 15 swings upward, the first sensor 21 tilts upward, and the direction sensed by the first sensor 21 may be located above the front in the traveling direction. Therefore, for example, the following structure is also possible: in the case where the height position H of the harvesting portion 15 is higher than the set height position H1, the direction of sensing by the first sensor 21 is adjusted downward. Further, the following structure is also possible: in the case where the height position H of the harvesting portion 15 is lower than the set height position H2, the direction of sensing by the second sensor 22 is adjusted upward. In short, at least one of the first sensor 21 and the second sensor 22 may be configured to be able to change the sensing direction according to the height position H of the harvesting portion 15.

(5) In the above embodiment, the first sensor 21 and the second sensor 22 are provided at the left end portion in the front wall portion of the threshing device 13, but the present invention is not limited to this embodiment. For example, the first sensor 21 and the second sensor 22 may be provided at the front right end of the boarding portion 12. In this case, the respective height positions of the first sensor 21 and the second sensor 22 at the front right end portion of the riding section 12 may be the same or substantially the same as the respective height positions of the first sensor 21 and the second sensor 22 at the front wall portion of the threshing device 13. The pair of first sensors 21 and the pair of second sensors 22 and 22 may be provided on the front wall portion of the threshing device 13 and the front right end portion of the riding portion 12, respectively. The first sensor 21 and the second sensor 22 may be provided at left and right center portions of the front end portion of the main body 1 (for example, at a left front end corner portion of the riding portion 12).

(6) In the above embodiment, the first sensor 21 and the second sensor 22 are sonars, but the present invention is not limited to this embodiment. For example, the first sensor 21 and the second sensor 22 may be imaging devices (e.g., CCD cameras, CMOS cameras, infrared cameras). If the first sensor 21 and the second sensor 22 are imaging devices, detection objects shown by reference numerals Z0, Z1, Z2, Z3, and Z4 in fig. 1 are recognized with high accuracy. The first sensor 21 and the second sensor 22 may be radars (millimeter waves) or LIDAR (laser scanners, laser radars, for example). If the first sensor 21 and the second sensor 22 are millimeter wave radars, detection that is not easily affected by weather can be performed. If the millimeter wave radar is configured to be capable of scanning in three dimensions in the vertical direction in addition to the front direction, the left and right directions, the detection range can be made wider than that of a millimeter wave radar of a type that scans in two dimensions. If the first sensor 21 and the second sensor 22 are LIDAR, the separation distance to the detection target can be measured with high accuracy. In addition, if the LIDAR is configured to be capable of scanning in three dimensions in the vertical direction in addition to the front direction, the left and right directions, the detection range can be made wider than in a LIDAR of a type that scans in two dimensions. The first sensor 21 and the second sensor 22 may be formed by a combination of different sensors.

For example, when the first sensor 21 and the second sensor 22 are imaging devices, the detection information output from the imaging devices is captured image data. When the first sensor 21 and the second sensor 22 are sonar, millimeter wave radar, and LIDAR, the detection information output from the sonar, millimeter wave radar, and LIDAR is distance measurement information.

(7) In the above embodiment, the harvesting-height detecting unit 23 is configured to be able to detect the height position H as the height position information of the harvesting unit 15 by detecting the swing angle of the conveyor 16. The selection unit 31 is configured to select at least one of the detection information of the first sensor 21 and the detection information of the second sensor 22 according to the height position H of the harvesting unit 15, but is not limited to this embodiment. For example, the harvesting height detecting unit 23 may be configured to detect a swing angle of the conveyor 16, and the selecting unit 31 may select at least one of the detection information of the first sensor 21 and the detection information of the second sensor 22 according to the swing angle of the conveyor 16. That is, the "height position" of the harvesting unit 15 is not limited to the height position H, and may be a swing angle of the conveyor 16. The height position H of the harvesting unit 15 shown in fig. 4 to 6 and 8 to 10 is based on the position of the lower end of the harvesting unit 15, but may be based on the position of a harvesting knife (not shown) of the harvesting unit 15, for example.

(8) A control program may be configured to cause a computer to realize the functions of the respective components in the above-described embodiments. Further, the present invention may be configured as a recording medium on which a control program for causing a computer to realize the functions of the respective members in the above-described embodiments is recorded. Further, the present invention may be configured as a control method for performing the operations performed by the respective members in the above-described embodiments in one or more steps.

[ second embodiment ]

A second embodiment of the present invention will be described below with reference to fig. 11 to 15. In the following description, unless otherwise specified, the direction of arrow F shown in fig. 11 and 12 is referred to as "front" and the direction of arrow B is referred to as "rear". The direction of arrow L shown in fig. 12 is referred to as "left", and the direction of arrow R is referred to as "right". The direction of arrow U shown in fig. 11 is referred to as "up", and the direction of arrow D is referred to as "down".

[ integral structure of combine harvester ]

As shown in fig. 11 and 12, a whole-feed combine harvester 101 (corresponding to an "agricultural machine" of the present invention) includes a crawler-type traveling device 111 (corresponding to a "predetermined device" of the present invention), a steering unit 112, a threshing device 113, a grain tank 114, a harvesting device 115, a conveying device 116, a grain discharging device 118, and a satellite positioning module 180.

The traveling device 111 is provided at a lower portion of the combine harvester 101. The traveling device 111 is driven by power from an engine (not shown). The combine harvester 101 can travel by itself through the travel device 111.

The driving unit 112, the threshing device 113, and the grain tank 114 are provided above the traveling device 111. The operator can board the operator monitoring the work of the combine harvester 101 in the driver section 112. Further, the operator may monitor the operation of the combine harvester 101 from outside the combine harvester 101.

The grain discharge device 118 is provided above the threshing device 113 and the grain tank 114. The satellite positioning module 180 is attached to the upper surface of the driver unit 112.

The harvesting device 115 is disposed at the front of the combine harvester 101. The conveyor 116 is provided so as to extend across the rear end of the harvester 115 and the front end of the thresher 113.

The harvesting device 115 gathers the standing grain stalks of the harvested objects and harvests the standing grain stalks of the field. Thus, the harvesting device 115 harvests the grain of the field. The combine harvester 101 can perform harvesting travel in which the traveling device 111 travels while harvesting standing grain stalks of the field by the harvesting device 115.

The harvested straw harvested by the harvesting device 115 is transported to the threshing device 113 by the transporting device 116. The threshing device 113 threshes the harvested grain stalks. The grains obtained by the threshing process are stored in a grain tank 114. The grains stored in the grain tank 114 are discharged outside the machine through the grain discharging device 118 as needed.

[ Structure relating to automatic traveling ]

As shown in fig. 13, the combine harvester 101 includes a control unit 130. The control unit 130 is an aggregate of a plurality of ECUs. The control unit 130 includes a travel control unit 135 (corresponding to the "device control unit" of the present invention).

In addition, as shown in fig. 11, the satellite positioning module 180 receives a GPS signal from a satellite GS used in GPS (global positioning system). Next, as shown in fig. 13, the satellite positioning module 180 transmits, to the travel control unit 135, positioning data indicating the position of the vehicle of the combine harvester 101 based on the received GPS signal.

The travel control unit 135 calculates the position coordinates of the combine harvester 101 as time passes based on the positioning data output from the satellite positioning module 180. Next, the travel control unit 135 performs automatic travel based on the calculated position coordinates of the combine harvester 101 with the passage of time and a travel route set in advance.

More specifically, the travel control unit 135 controls the travel device 111 to perform the harvesting travel by the automatic travel along the travel path.

In this way, the combine harvester 101 can travel automatically.

In addition, various operation members (not shown) are disposed in the driver section 112. When the operator gets on the driver unit 112, the operator can control the travel of the combine harvester 101 through these operation members. That is, the combine harvester 101 can travel not only automatically but also manually.

[ Structure relating to obstacle detection ]

As shown in fig. 11 and 12, the combine harvester 101 includes a visible light camera 121 (corresponding to the "imaging device" of the present invention) and a temperature distribution sensor 122 (corresponding to the "detection device" of the present invention).

The visible-light camera 121 can acquire a captured image of visible light by detecting the visible light. In addition, the temperature distribution sensor 122 detects a temperature distribution in the field of view. That is, the temperature distribution sensor 122 is a different kind of sensor from the visible light camera 121.

The visible light camera 121 and the temperature distribution sensor 122 are attached to the tip of the grain discharger 118 in a state adjacent to each other. Also, the visible light camera 121 and the temperature distribution sensor 122 are directed toward the field.

The visual field of the visible light camera 121 and the temperature distribution sensor 122 in the top view is the visual field region S shown in fig. 12. The visual field region S is a circular region centered on the position in the left front of the combine harvester 101 in the non-harvesting region CA of the field.

The present invention is not limited to this, and the visual field region S may be a region having any shape other than a circle.

In fig. 12, the harvested area SA is shown. The harvested area SA is located on the right side of the combine harvester 101. In addition, the uncurved area CA is located on the front side and the left side of the combine harvester 101.

As shown in fig. 13, the control unit 130 includes an obstacle detection unit 134. The obstacle detector 134 includes a first detector 131 and a second detector 132.

As shown in fig. 13, the captured image of the visible light camera 121 is transmitted to the first detection unit 131. The first detection unit 131 detects an obstacle around the body based on the captured image of the visible light camera 121. At this time, the first detection part 131 detects an obstacle around the body by using a neural network learned using deep learning.

Fig. 14 shows a flow of obstacle detection by the first detection unit 131. Hereinafter, the obstacle detection by the first detection unit 131 will be described assuming that the object to be detected in the first detection unit 131 is a human object.

As shown in fig. 14, the first detection unit 131 receives a pixel value of each pixel included in the captured image captured by the visible light camera 121. Next, data indicating the estimation result (detection result) is output from the first detection unit 131. The data includes the existence region of the person and the estimated probability thereof.

In the estimation result in fig. 14, a human figure region F1 as an existing region of a human figure is represented by a rectangular frame. The estimated probability is linked to the human figure region F1. In addition, the human figure region F1 is defined by four corner points. The coordinate positions of these four corner points in the captured image are also included in the estimation result. When the detection target object is not estimated in the captured image, the human figure region F1 is not output, and the estimation probability is zero.

With the above-described configuration, the first detection unit 131 detects an obstacle located in the visual field S.

As shown in fig. 13, the detection result of the temperature distribution sensor 122 is transmitted to the second detection unit 132. The second detection unit 132 detects an obstacle around the body based on the detection result of the temperature distribution sensor 122. At this time, the second detection unit 132 detects an obstacle around the body by using a neural network learned using deep learning.

Fig. 15 shows a flow of obstacle detection by the second detection unit 132. Hereinafter, the obstacle detection by the second detection unit 132 will be described assuming that the object to be detected in the second detection unit 132 is a human object.

As shown in fig. 15, the detection result of the temperature distribution sensor 122 is input to the second detection section 132 as a captured image representing the temperature distribution. At this time, the pixel value of each pixel included in the captured image showing the temperature distribution is input to the second detection unit 132. Next, data indicating the estimation result (detection result) is output from the second detection unit 132. The data includes the existence region of the person and the estimated probability thereof.

In the estimation result in fig. 15, a human figure region F2 as an existing region of a human figure is represented by a rectangular frame. The estimated probability is linked to the human figure region F2. In addition, the human figure region F2 is defined by four corner points. The coordinate positions of these four corner points in the captured image are also included in the estimation result. When the detection target object is not estimated in the captured image, the human figure region F2 is not output, and the estimation probability is zero.

With the above-described configuration, the second detection unit 132 detects an obstacle located in the visual field region S.

When the first detector 131 does not detect an obstacle and the second detector 132 does not detect an obstacle, the obstacle detector 134 does not output a signal indicating that an obstacle is detected.

When an obstacle is detected only by one of the first detector 131 and the second detector 132, the obstacle detector 134 outputs a signal indicating that the obstacle is detected.

When both the first detection unit 131 and the second detection unit 132 detect an obstacle, the obstacle detection unit 134 outputs a signal indicating that the obstacle is detected.

With the above-described configuration, the obstacle detector 134 can detect an obstacle around the body based on the captured image of the visible light camera 121 and the detection result of the temperature distribution sensor 122.

[ control in the case where an obstacle is detected ]

As shown in fig. 13, the combine harvester 101 includes a horn 141 (corresponding to the "predetermined device" of the present invention). The control unit 130 includes a warning control unit 136 (corresponding to the "device control unit" of the present invention) and a notification control unit 137 (corresponding to the "device control unit" of the present invention).

A signal indicating that an obstacle is detected is transmitted from the obstacle detection unit 134 to the travel control unit 135, the warning control unit 136, and the notification control unit 137. Hereinafter, the signal indicating that an obstacle is detected will be referred to as a detection signal, and control by the travel control unit 135, the warning control unit 136, and the notification control unit 137 will be described.

Upon receiving the detection signal, the travel control unit 135 executes detection-time stop control (corresponding to "detection-time control" in the present invention). The detection-time stop control is control in accordance with the detection of the obstacle.

Specifically, the detection-time stop control is control for stopping the driving of the traveling device 111. Therefore, when the check-time stop control is executed, the combine 101 stops traveling.

In this way, when the obstacle is detected by the obstacle detection unit 134, the travel control unit 135 performs control corresponding to the detection of the obstacle, that is, detection-time stop control.

The warning controller 136 controls a horn 141 (corresponding to a "predetermined device" of the present invention). When receiving the detection signal, the warning control unit 136 executes a detection-time warning control (corresponding to the "detection-time control" of the present invention). The warning control at the time of detection is control in accordance with the detection of an obstacle.

Specifically, the warning control at the time of detection is a control for emitting a warning sound from the horn 141. Therefore, when the detection-time warning control is performed, the horn 141 emits a warning sound.

Thus, when the detected obstacle is a human or an animal, the obstacle can be warned.

In this way, when the obstacle is detected by the obstacle detector 134, the warning controller 136 performs the detection-time warning control corresponding to the detection of the obstacle.

The notification control unit 137 controls the mobile communication terminal CT1 (corresponding to the "predetermined device" of the present invention). In addition, the portable communication terminal CT1 is located outside the combine harvester 101. When receiving the detection signal, the notification control unit 137 executes a detection-time notification control (corresponding to the "detection-time control" of the present invention). The detection-time notification control is a control corresponding to the detection of the obstacle.

Specifically, the detection-time notification control is a control for displaying a notification screen on the mobile communication terminal CT 1. Therefore, when the detection-time notification control is executed, a notification screen is displayed on the portable communication terminal CT 1. The notification screen includes a message notifying that an obstacle is detected.

This notifies the holder of the portable communication terminal CT1 of the detection of the obstacle. For example, the holder of the portable communication terminal CT1 may be an operator who monitors the work of the combine harvester 101 from outside the combine harvester 101.

In this way, when the obstacle detection unit 134 detects an obstacle, the notification control unit 137 performs a detection-time notification control corresponding to the detection of the obstacle.

[ Structure relating to State determination section ]

As shown in fig. 13, the combine harvester 101 includes a temperature sensor 123 and an illuminance sensor 124. The control unit 130 further includes a state determination unit 133.

The temperature sensor 123 detects the temperature near the visible-light camera 121. Further, the illuminance sensor 124 detects illuminance outside the combine harvester 101.

The state determination unit 133 determines whether or not the imaging state of the visible light camera 121 is normal. The following describes the structure of the state determination unit 133.

As shown in fig. 13, the captured image of the visible light camera 121 is transmitted to the state determination unit 133. The state determination unit 133 is configured to be able to determine whether or not the imaging state of the visible light camera 121 is normal based on the image captured by the visible light camera 121.

Specifically, the visible-light camera 121 acquires captured images at predetermined time intervals. Then, the state determination unit 133 compares these captured images to detect a region in which the pixel value has not substantially changed even after the elapse of time. Hereinafter, this region is referred to as an invariant region.

When the constant region in the captured image is equal to or larger than the predetermined area, the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal.

For example, when relatively large dirt adheres to the visible light camera 121, the constant region having a predetermined area or more is detected. As a result, the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal.

In addition, as described above, the temperature sensor 123 detects the temperature near the visible-light camera 121. As shown in fig. 13, the detection result of the temperature sensor 123 is sent to the state determination unit 133.

Here, when the temperature in the vicinity of the visible light camera 121 is a relatively high temperature, a large amount of noise is likely to be included in the captured image of the visible light camera 121. Therefore, the temperature in the vicinity of the visible light camera 121 is a value representing the shooting state of the visible light camera 121.

The state determination unit 133 is configured to be able to determine whether the imaging state of the visible light camera 121 is normal or not based on the detection result of the temperature sensor 123. When the detection result of the temperature sensor 123 is equal to or higher than the predetermined temperature, the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal.

As described above, the illuminance sensor 124 detects the illuminance outside the combine harvester 101. As shown in fig. 13, the detection result of the illuminance sensor 124 is sent to the state determination unit 133.

Here, when the illuminance outside the combine harvester 101 is relatively high or relatively low, the image captured by the visible light camera 121 is likely to be unclear. Therefore, the illuminance outside the combine harvester 101 is a value indicating the imaging state of the visible-light camera 121.

The state determination unit 133 is configured to be able to determine whether or not the imaging state of the visible light camera 121 is normal based on the detection result of the illuminance sensor 124. When the detection result of the illuminance sensor 124 is higher than the predetermined upper limit illuminance, the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal. Further, when the detection result of the illuminance sensor 124 is smaller than the predetermined lower limit illuminance, the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal.

In each of the above determinations based on the captured image of the visible light camera 121, the detection result of the temperature sensor 123, and the detection result of the illuminance sensor 124, the state determination unit 133 determines that the imaging state of the visible light camera 121 is normal if it is not determined that the imaging state of the visible light camera 121 is not normal.

The present invention is not limited to the above-described configuration. For example, the state determination unit 133 may be configured to determine whether or not the imaging state of the visible light camera 121 is normal based on any one or both of the captured image of the visible light camera 121, the detection result of the temperature sensor 123, and the detection result of the illuminance sensor 124.

In this way, the state determination unit 133 determines whether or not the imaging state of the visible light camera 121 is normal based on at least one of the value indicating the imaging state of the visible light camera 121 and the image captured by the visible light camera 121.

As shown in fig. 13, the determination result of the state determination unit 133 is transmitted to the obstacle detection unit 134.

The obstacle detection unit 134 performs obstacle detection in the first detection unit 131 and the second detection unit 132 while the state determination unit 133 determines that the imaging state of the visible light camera 121 is normal.

When the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal, the obstacle detection unit 134 stops the obstacle detection in the first detection unit 131. In this case, the obstacle detecting unit 134 detects an obstacle only in the second detecting unit 132 of the first detecting unit 131 and the second detecting unit 132.

That is, in this case, the obstacle detecting unit 134 detects an obstacle around the body based on only the captured image of the visible light camera 121 and the detection result of the temperature distribution sensor 122 among the detection results of the temperature distribution sensor 122.

In this way, when the state determination unit 133 determines that the imaging state of the visible light camera 121 is not normal, the obstacle detection unit 134 detects an obstacle around the body based on the detection result of the temperature distribution sensor 122.

With the above-described configuration, the obstacle detecting unit 134 can detect an obstacle around the body based on only the captured image of the visible light camera 121 and the detection result of the temperature distribution sensor 122 among the detection results of the temperature distribution sensor 122. As described above, the obstacle detector 134 can detect an obstacle around the body based on the captured image of the visible light camera 121 and the detection result of the temperature distribution sensor 122.

That is, the obstacle detecting unit 134 is configured to detect an obstacle around the body based on at least one of the captured image of the visible light camera 121 and the detection result of the temperature distribution sensor 122.

With the above-described configuration, for example, when the captured image of the visible light camera 121 is unclear, and the accuracy of obstacle detection based on the captured image of the visible light camera 121 alone is likely to decrease, it is possible to detect an obstacle around the body based on the detection result of the temperature distribution sensor 122.

Therefore, with the configuration described above, it is possible to realize the combine harvester 101 in which the obstacle detection accuracy is less likely to decrease in a situation where the obstacle detection accuracy based on only the captured image is likely to decrease.

In other words, with the above-described configuration, the combine harvester 101 having excellent obstacle detection accuracy can be realized.

In addition, with the above-described configuration, it is possible to realize the combine harvester 101 that performs appropriate control when an obstacle is detected.

The above-described embodiments are merely examples, and the present invention is not limited thereto and can be modified as appropriate.

[ other embodiments of the second embodiment ]

Another embodiment obtained by modifying the above embodiment will be described below. The present invention is similar to the embodiments described above except for the matters described in the following other embodiments. The above-described embodiments and the following other embodiments may be appropriately combined within a range where no contradiction occurs. The scope of the present invention is not limited to the above-described embodiments and the following other embodiments.

(1) Instead of the temperature distribution sensor 122, a short-wave infrared sensor that detects short-wave infrared rays may be provided. In this case, the short-wave infrared sensor corresponds to the "detection device" of the present invention.

(2) The traveling device 111 may be a wheel type or a semi-crawler type.

(3) In the above-described embodiment, the visible light camera 121 is provided as a component corresponding to the "imaging device" of the present invention. However, the present invention is not limited to this, and may include another sensor instead of the visible-light camera 121 as long as the sensor can acquire a captured image.

For example, an infrared camera may be provided instead of the visible-light camera 121. In this case, any type of sensor may be provided as long as it is a sensor different from the infrared camera as a component corresponding to the "detection device" of the present invention.

For example, an ultraviolet camera may be provided instead of the visible light camera 121. In this case, any type of sensor may be provided as long as it is a sensor different from the ultraviolet camera as a component corresponding to the "detection device" of the present invention.

(4) The obstacle detecting unit 134 may be configured to perform obstacle detection only by the first detecting unit 131 of the first detecting unit 131 and the second detecting unit 132 while the state determining unit 133 determines that the imaging state of the visible light camera 121 is normal.

(5) The first detector 131 and the second detector 132 may be combined into one detector. In this case, the detection unit may be configured to detect an obstacle around the body by using a neural network learned using deep learning, or may be configured to detect an obstacle around the body by a method not using a neural network.

(6) The first detection unit 131 may be configured to detect an obstacle around the body by a method not using a neural network.

(7) The second detection unit 132 may be configured to detect an obstacle around the body by a method not using a neural network.

(8) The temperature sensor 123 may not be provided.

(9) The illuminance sensor 124 may not be provided.

(10) The state determination unit 133 may not be provided.

(11) The obstacle detector 134 may be configured not to output a signal indicating that an obstacle is detected when the obstacle is detected by only one of the first detector 131 and the second detector 132.

(12) The travel control unit 135 may be configured not to perform the detection-time stop control when the obstacle detection unit 134 detects an obstacle.

(13) The warning control unit 136 may be configured not to perform the warning control when the obstacle detection unit 134 detects an obstacle.

(14) The warning controller 136 may not be provided.

(15) The notification control unit 137 may be configured not to perform the notification control when the obstacle detection unit 134 detects an obstacle.

(16) The notification control unit 137 may not be provided.

(17) The object to be detected in the first detecting unit 131 may be any object other than a human being, or may be a plurality of objects.

(18) The object to be detected in the second detection unit 132 may be any object other than a human being, or may be a plurality of objects.

(19) The notification control unit 137 may be configured to cause a display device provided in the driver unit 112 to display a notification screen during the detection-time notification control. In this case, the display device corresponds to the "predetermined device" of the present invention. This notifies the operator riding on the driver unit 112 of the detection of the obstacle.

(20) The notification control unit 137 may be configured to display a notification screen on a management terminal provided in a management center located outside the combine harvester 101 during the detection-time notification control. In this case, the management terminal corresponds to "a predetermined device" of the present invention. This informs the user of the management terminal of the detection of the obstacle.

(21) A control program may be configured to cause a computer to realize the functions of the respective components in the above-described embodiments. Further, the present invention may be configured as a recording medium on which a control program for causing a computer to realize the functions of the respective members in the above-described embodiments is recorded. Further, the present invention may be configured as a control method for performing the operations performed by the respective members in the above-described embodiments in one or more steps.

Note that the configuration disclosed in the above embodiment (including other embodiments, the same applies hereinafter) can be applied in combination with the configuration disclosed in the other embodiments as long as no contradiction occurs. The embodiments disclosed in the present specification are illustrative, and the embodiments of the present invention are not limited thereto, and may be appropriately modified within a range not departing from the object of the present invention.

Industrial applicability

The invention can be applied to a semi-feeding combine harvester besides a full-feeding combine harvester. The present invention can also be applied to a grain-picking type corn harvester, a bean harvester, and the like.

In addition, the invention can be used for various agricultural machines such as a full-feeding combine harvester, a half-feeding combine harvester, a corn harvester, a rice transplanter, a tractor and the like.

Description of the reference numerals

(first embodiment)

1: machine body

15: harvesting part

15A: harvesting head

15B: gathering wheel

23: reaping height detecting part (height detecting part)

31: selection part

32: determination unit

33: obstacle detection unit

H: height position

H1: setting the height position

H2: setting the height position

(second embodiment)

101: combine harvester (agricultural machinery)

111: running gear (specified device)

121: visible light camera (Camera equipment)

122: temperature distribution sensor (detection device)

131: first detecting part

132: second detecting part

133: state determination unit

134: obstacle detection unit

135: running control unit (device control unit)

136: alarm control unit (device control unit)

137: notification control section (device control section)

141: horn (specified device)

CT 1: portable communication terminal (specified device)