阅读说明：本技术 车辆周边监视装置、控制方法及存储介质 (Vehicle periphery monitoring device, control method, and storage medium ) 是由 坂口敦俊 日隈友仁 里见洋平 于 2021-05-13 设计创作，主要内容包括：一种车辆周边监视装置、控制方法及存储介质。车辆周边监视装置具有：拍摄车辆的侧后方的侧后方拍摄部；拍摄车辆的后方的后方拍摄部；检测部,检测在利用侧后方拍摄部拍摄的侧后方图像中的与利用后方拍摄部拍摄到的后方图像重叠的侧后方重叠区域、及后方图像中的与侧后方图像重叠的后方重叠区域存在的物标；判定部,判定在利用检测部检测出的物标中,是否包括仅在侧后方图像中的侧后方重叠区域和后方图像中的后方重叠区域的任一方中视觉确认出的特定物标；以及显示控制部,在由判定部判定为包括特定物标时,在没有视觉确认出特定物标的侧后方图像或后方图像中重叠与特定物标对应的特定物标显示像,并显示于设置于车室内的至少一个显示部。(A vehicle periphery monitoring device, a control method, and a storage medium. A vehicle periphery monitoring device is provided with: a lateral rear imaging unit that images a lateral rear of the vehicle; a rear imaging unit that images the rear of the vehicle; a detection unit that detects a target object that exists in a lateral overlapping region that overlaps with the rearward image captured by the rearward image capturing unit in the lateral rearward image captured by the lateral rearward image capturing unit and a rearward overlapping region that overlaps with the lateral rearward image in the rearward image; a determination unit that determines whether or not a specific object that is visually recognized only in one of the lateral overlapping area in the lateral image and the posterior overlapping area in the posterior image is included in the objects detected by the detection unit; and a display control unit that, when the determination unit determines that the specific object is included, superimposes a specific object display image corresponding to the specific object on the lateral rearward image or the rearward image in which the specific object is not visually recognized, and displays the specific object display image on at least one display unit provided in the vehicle interior.)

1. A vehicle surroundings monitoring device includes:

a lateral rear imaging unit that images a lateral rear of the vehicle;

a rear imaging unit that images the rear of the vehicle;

a detection unit that detects a target object that exists in a lateral-posterior overlapping region that overlaps with the posterior image captured by the posterior image capturing unit in the lateral-posterior image captured by the lateral-posterior image capturing unit and a posterior overlapping region that overlaps with the lateral-posterior image in the posterior image;

a determination unit configured to determine whether or not the target objects detected by the detection unit include a specific target object that is visually recognized only in one of the lateral overlapping area in the lateral overlapping area and the posterior overlapping area in the posterior image; and

and a display control unit that, when the determination unit determines that the specific object is included, displays a specific object display image corresponding to the specific object on at least one display unit provided in the vehicle interior, while superimposing the specific object display image on the lateral rearward image or the rearward image in which the specific object is not visually recognized.

2. The vehicle surroundings monitoring apparatus according to claim 1,

in the case where the specific object is present in the 1 st blind area in the side rear image generated by the approaching vehicle approaching to the side rear of the vehicle,

the display control unit displays the specific object display image corresponding to the specific object visually recognized in the rearward image, on the display unit while superimposing the specific object display image on the rearward image in which the specific object is not visually recognized due to the approaching vehicle in the lateral rearward direction.

3. The vehicle surroundings monitoring apparatus according to claim 1 or 2,

in the case where the specific object is present in the 2 nd blind area in the rear image generated by the approaching vehicle approaching to the rear of the vehicle,

the display control unit displays the specific object display image corresponding to the specific object visually recognized in the lateral rearward image on the display unit while superimposing the specific object display image on the rearward image in which the specific object is not visually recognized due to the rearward approaching vehicle.

4. The vehicle surroundings monitoring device according to any one of claims 1 to 3,

the display control unit deletes the lateral and posterior overlapping regions of the lateral and posterior images or the posterior overlapping region of the posterior image on a virtual projection plane, synthesizes and processes the lateral and posterior overlapping regions into a single image, and displays the single image on one display unit.

5. The vehicle surroundings monitoring device according to any one of claims 1 to 3,

the display control unit displays the lateral rearward image including the lateral rearward overlapping region and the rearward image including the rearward overlapping region on a virtual projection plane, on a plurality of display units, respectively.

6. The vehicle surroundings monitoring apparatus according to claim 5,

the display control unit displays the specific object display image on both of an image in which the specific object can be visually confirmed and an image in which the specific object is not visually confirmed, when the lateral rearward image and the rearward image are displayed on the plurality of display units individually.

7. The vehicle surroundings monitoring device according to any one of claims 1 to 6,

the display control unit causes the specific object display image to blink and causes the blinking cycle to change in accordance with a distance between the vehicle and the specific object, when the specific object display image is displayed on the display unit.

8. The vehicle surroundings monitoring device according to any one of claims 1 to 7,

the display control unit displays the specific object display image as a see-through image so that the specific object display image has lower visual recognition than other images displayed on the display unit when the specific object display image is displayed on the display unit.

9. The vehicle surroundings monitoring device according to any one of claims 1 to 8,

the display control unit displays the specific object display image on the display unit while superimposing the specific object display image on the lateral rearward image or the rearward image when the specific object is approaching the vehicle.

10. The vehicle surroundings monitoring device according to any one of claims 1 to 9,

the vehicle periphery monitoring device is an electronic mirror device mounted on the vehicle.

11. A control method, in which method,

the side rear of the vehicle is photographed,

the rear of the vehicle is photographed,

detecting a target object existing in a lateral and posterior overlapping region where a captured range in the captured lateral and posterior images overlaps with the captured posterior image and a posterior overlapping region where the captured range in the posterior image overlaps with the lateral and posterior images,

determining whether or not a specific target object visually recognized only in one of the lateral overlapping area in the lateral rearward image and the rearward overlapping area in the rearward image is included in the detected target objects,

when it is determined that the specific object is included, a specific object display image corresponding to the specific object is superimposed on the lateral rearward image or the rearward image in which the specific object is not visually recognized, and is displayed on at least one display unit provided in the vehicle interior.

12. A storage medium which stores a program and can be read by a computer,

when the program is executed by a processor, the following processing is performed:

the side rear of the vehicle is photographed,

the rear of the vehicle is photographed,

detecting a target object existing in a lateral and posterior overlapping region where a captured range in the captured lateral and posterior images overlaps with the captured posterior image and a posterior overlapping region where the captured range in the posterior image overlaps with the lateral and posterior images,

determining whether or not a specific target object visually recognized only in one of the lateral overlapping area in the lateral rearward image and the rearward overlapping area in the rearward image is included in the detected target objects,

when it is determined that the specific object is included, a specific object display image corresponding to the specific object is superimposed on the lateral rearward image or the rearward image in which the specific object is not visually recognized, and is displayed on at least one display unit provided in the vehicle interior.

Technical Field

The present disclosure relates to a vehicle periphery monitoring device.

Background

Japanese patent laid-open No. 2020-028027 discloses an electronic mirror device comprising: a1 st composite image is generated by combining a rear image captured by a rear camera that captures the rear of a vehicle and a side rear image captured by a side rear camera that captures the side rear of the vehicle, and is displayed on a display unit provided in a vehicle cabin.

The electronic mirror device determines whether or not an object is present in a blind area generated by a following vehicle approaching from the rear of the vehicle, and generates a2 nd synthetic image in which a lateral rear image including an image area corresponding to the object is displayed so as to be superimposed on a rear image when the object is present. The image displayed on the display unit is switched to the 2 nd synthetic image.

In the technique described in japanese patent application laid-open No. 2020 and 028027, the 1 st synthetic image is visually recognized as an image in which the rearward image and the side rearward image are continuous, whereas in the 2 nd synthetic image, in a portion in which the side rearward image is displayed superimposed on the rearward image, the rearward image and the side rearward image are different in view angle, and therefore, the images are displayed discontinuously. Therefore, even if the same object is present, the same object looks different between the case of visual confirmation in the 1 st composite screen and the case of visual confirmation in the 2 nd composite screen, and therefore there is a problem that the passenger visually confirming the display portion feels annoyed.

Disclosure of Invention

In view of the above circumstances, an object of the present disclosure is to provide a vehicle periphery monitoring device that can improve the visibility of a display unit and can effectively recognize the peripheral situation of a vehicle occupant.

A vehicle periphery monitoring device according to claim 1 of the present disclosure includes: a lateral rear imaging unit that images a lateral rear of the vehicle; a rear imaging unit that images the rear of the vehicle; a detection unit that detects a target object (object target) that is present in a lateral-rearward overlapping region that overlaps with a rearward image captured by the rearward image capturing unit in the lateral-rearward image captured by the lateral-rearward image capturing unit and a rearward overlapping region that overlaps with the lateral-rearward image in the rearward image; a determination unit configured to determine whether or not the target objects detected by the detection unit include a specific target object that is visually recognized only in one of the lateral overlapping area in the lateral rearward image and the rearward overlapping area in the rearward image; and a display control unit that, when the determination unit determines that the specific target object is included, displays a specific target object display image corresponding to the specific target object on at least one display unit provided in the vehicle interior, while superimposing the specific target object display image on the lateral rearward image or the rearward image in which the specific target object is not visually recognized.

In claim 1, the side rear imaging unit images the side rear of the vehicle, and the rear imaging unit images the rear of the vehicle. The display control unit causes at least one display unit provided in the vehicle interior to display the side rearward image captured by the side rearward image capturing unit and the rearward image captured by the rearward image capturing unit. Here, regarding a side-rear overlapping region overlapping with a rear image in a side-rear image and a rear overlapping region overlapping with the side-rear image in the rear image, there is a case where a blind spot (blind spot) in the side-rear image or in the rear image is caused by another vehicle or the like approaching from the rear side of the vehicle. In contrast, the detection unit detects the target object existing in the lateral overlapping area and the rear overlapping area. The determination unit determines whether or not the target objects detected by the detection unit include a specific target object recognized only in one of the lateral overlapping area and the rear overlapping area. When the determination unit determines that the specific object is included, the display control unit superimposes a specific object display image corresponding to the specific object on the lateral and posterior images or the posterior image in which the specific object is not visually recognized, and displays the superimposed specific object display image on the display unit. That is, among the target objects detected by the detection unit, a specific target object recognized only in either one of the lateral overlapping area and the rear overlapping area exists in an area blocked by another vehicle or the like approaching to the rear side of the vehicle and becoming a blind spot in the lateral image or the rear image. Therefore, by displaying the specific object display image corresponding to the specific object in a superimposed manner on the lateral rearward image or the rearward image in which the specific object is not visually recognized, it is possible to make the occupant recognize that the specific object exists behind another vehicle or the like approaching the vehicle rearward. Further, the display control unit displays the specific object display image superimposed on the lateral rearward image or the rearward image in which the specific object is not visually recognized, and therefore displays an image maintaining a constant angle of view in a region other than the specific object display image in the image displayed on the display unit. Therefore, the specific object display image in the image can be easily recognized, and the occurrence of overlooking or misrecognizing of the specific object display image before and after the specific object display image is displayed can be suppressed.

In this way, the passenger's visibility of the specific object, which is present in the lateral-rearward image or the blind spot in the rearward image due to another vehicle or the like approaching from the rearward side of the vehicle, is improved. As a result, the visibility of the target object displayed on the display unit can be improved, and the occupant of the vehicle can effectively recognize the surrounding situation.

A vehicle periphery monitoring device according to claim 2 of the present disclosure is the vehicle periphery monitoring device according to claim 1, wherein when the specific object is present in a1 st blind spot in the lateral rear image generated by a lateral rear approaching vehicle approaching the lateral rear of the vehicle, the display controller displays a specific object display image corresponding to the specific object visually recognized in the rear image, on the display unit while superimposing the specific object display image on the lateral rear image in which the specific object is not visually recognized by the lateral rear approaching vehicle.

Thus, even when there is a target object in the 1 st blind area in the lateral-rearward image generated by the vehicle approaching to the lateral rearward side of the vehicle, the target object can be displayed in the form of a specific object display image in the lateral-rearward image by determining the target object as the specific object. Thus, the occupant can recognize visually that the specific object is present in the vicinity of the rear of the vehicle in the lateral rear direction, and can clearly recognize the surrounding situation in the lateral rear direction.

A vehicle periphery monitoring device according to claim 3 of the present disclosure is the vehicle periphery monitoring device according to claim 1 or 2, wherein the display controller displays a specific object display image corresponding to the specific object visually recognized in the lateral-rearward image on the display unit while superimposing the specific object display image on the rearward image in which the specific object is not visually recognized by the rearward approaching vehicle, when the specific object is present in a2 nd blind area in the rearward image generated by the rearward approaching vehicle approaching rearward of the vehicle.

Thus, even when there is a target object in the 2 nd blind area in the rear image generated by the vehicle approaching in the rear direction, the target object can be displayed as a specific object display image in the rear image by determining the target object as the specific object. Thus, the occupant can recognize visually the specific object display image that the specific object is present in the vicinity of the rear of the vehicle in the lateral rear direction, and can clearly recognize the surrounding situation of the rear side of the vehicle in the rear direction.

A vehicle periphery monitoring device according to claim 4 of the present disclosure is the vehicle periphery monitoring device according to any one of claims 1 to 3, wherein the display control unit deletes the side-rear overlap region or the rear overlap region from either one of the side-rear image and the rear image on the virtual projection plane, and combines the side-rear image and the rear image from which the side-rear overlap region or the rear overlap region is deleted into a single image to be displayed on one of the display units.

In this way, the lateral rearward processed image and the rearward processed image displayed on the display unit are continuous images, and therefore, an image close to the situation where the vehicle is observed rearward can be displayed on the display unit. As a result, the burden on the occupant when recognizing the specific object is reduced, and the visibility of the display unit is improved.

A vehicle periphery monitoring device according to claim 5 of the present disclosure is the vehicle periphery monitoring device according to any one of claims 1 to 3, wherein the display control unit individually displays the lateral rear image including the lateral rear overlap region and the rear image including the rear overlap region on a virtual projection plane on a plurality of display units.

Thus, the occupant can observe the lateral rearward image including the lateral rearward overlapping region and the rearward image including the rearward overlapping region on the virtual projection plane, and can recognize the surrounding situation behind the vehicle in a wide range. Further, since both the specific target object and the corresponding specific target object display image are displayed on the plurality of display portions, the position of the specific target object can be clearly recognized from the plurality of display portions. As a result, the visibility of the display unit is improved.

A vehicle periphery monitoring device according to claim 6 of the present disclosure is the vehicle periphery monitoring device according to any one of claims 1 to 5, wherein the display control unit displays the specific object display image on the display unit while superimposing the specific object display image on the lateral rearward image or the rearward image when the specific object is approaching the vehicle.

Thus, when the risk level is higher, that is, when the specific target object is approaching the vehicle, the display unit displays the specific target object display image, thereby effectively attracting the attention of the occupant.

A vehicle periphery monitoring device according to claim 7 of the present disclosure is the electronic mirror device mounted on the vehicle, in addition to the configuration described in any one of claims 1 to 6.

Effects of the invention

The vehicle periphery monitoring device according to claim 1 of the present disclosure has an excellent effect of improving the visibility of the display unit and enabling the occupant of the vehicle to effectively recognize the peripheral situation.

The vehicle periphery monitoring device according to claim 2 of the present disclosure has an excellent effect of enabling an occupant to clearly recognize a peripheral situation in which the lateral rear portion approaches the rear side of the vehicle.

The vehicle periphery monitoring device according to claim 3 of the present disclosure has an excellent effect of enabling an occupant to clearly recognize a peripheral situation in which the rear approaches the rear side of the vehicle.

The vehicle periphery monitoring device according to claim 4 of the present disclosure has an excellent effect of reducing the burden on the occupant when recognizing a specific object and improving the visibility of the display unit.

The vehicle periphery monitoring device according to claim 5 of the present disclosure has an excellent effect of improving the visibility of the display unit by displaying the peripheral situation behind the vehicle in a wide range and displaying the specific object display image in a form in which the position of the specific object can be clearly recognized.

The vehicle periphery monitoring device according to claim 6 of the present disclosure has an excellent effect of being able to effectively attract attention of an occupant in relation to a specific target object behind a vehicle.

The vehicle periphery monitoring device according to claim 7 of the present disclosure has an excellent effect of improving the visibility of the display portion of the electronic mirror device and enabling the occupant of the vehicle to effectively recognize the peripheral situation.

Drawings

Exemplary embodiments of the present application are described in detail below based on the following figures, wherein:

fig. 1 is a block diagram showing a hardware configuration of an electronic mirror device according to embodiment 1;

fig. 2 is a block diagram showing a functional configuration of the electronic mirror device of embodiment 1;

fig. 3 is a view showing the arrangement positions of the side rear camera and the display unit of the electronic mirror device according to embodiment 1, and is a perspective view of the vehicle interior of the vehicle having the electronic mirror device as viewed from the rear side of the vehicle;

fig. 4 is a plan view showing the shooting ranges of the lateral and rearward cameras and the lateral and rearward overlapping regions of the lateral and rearward images and the rearward overlapping region of the rearward image;

fig. 5 is a schematic view showing an example of a state in which a lateral rearward processed image and a rearward processed image are subjected to composite processing into a single image and displayed on a display unit;

fig. 6 is a flowchart showing an example of the overlay processing executed by the electronic mirror ECU;

fig. 7 is a plan view showing an example of a1 st blind area caused by a vehicle approaching from the lateral rear side and a situation where a specific object exists in the 1 st blind area;

fig. 8 is a plan view showing an example of a2 nd blind area caused by a rear approaching vehicle and a situation where a specific target object is present in the 2 nd blind area;

fig. 9 is a schematic view showing an example of a specific object display image corresponding to a specific object existing in the 1 st blind area of fig. 7;

fig. 10 is a schematic view showing an example of a specific object display image corresponding to a specific object existing in the 2 nd blind area of fig. 8;

FIG. 11 is a schematic view corresponding to FIG. 9 showing an example of a specific object display image;

fig. 12 is a block diagram showing a hardware configuration of the electronic mirror device of embodiment 2;

fig. 13 is a block diagram showing a functional configuration of the electronic mirror device of embodiment 2;

FIG. 14 is a schematic view corresponding to FIG. 9 showing an example of a display image of a specific object corresponding to a specific object existing in the 1 st blind area; and is

Fig. 15 is a schematic view corresponding to fig. 14 showing an example of the specific object display image.

Detailed Description

(hardware constitution)

Hereinafter, embodiment 1 of the present invention will be described with reference to fig. 1 to 11. The in-vehicle system 40 shown in fig. 1 includes a bus 42, and a plurality of electronic control units and a plurality of sensor units that perform different controls are connected to the bus 42. Further, fig. 1 shows only a part of the in-vehicle system 40 related to the present invention. Each Electronic Control Unit is a Control Unit including a CPU, a memory, and a nonvolatile storage Unit, and hereinafter referred to as an ECU (Electronic Control Unit). Among the plurality of ECUs connected to the bus 42 is an electronic mirror ECU 22.

The left rear camera 12, the right rear camera 14, the rear camera 16, the electronic mirror display 18, and the camera housing ACT (actuator) 20 are connected to the electronic mirror ECU22, respectively. The electronic mirror ECU22, the left rear camera 12, the right rear camera 14, the rear camera 16, the electronic mirror display 18, and the camera housing ACT20 constitute the electronic mirror device 10, and the electronic mirror device 10 is an example of a vehicle periphery monitoring device. The electronic mirror display 18 in the electronic mirror device 10 is an example of a display unit.

As shown in fig. 3, a base portion of a camera support body 32L having a substantially rectangular parallelepiped shape and an arc-shaped distal end portion is attached to a vehicle front end portion of a vehicle vertical direction intermediate portion of a left side door (front side door, not shown) of a vehicle such that the distal end portion of the camera support body 32L protrudes outward of the vehicle. The left rear camera 12 is attached near the distal end portion of the camera support body 32L, and the imaging optical axis (lens) of the left rear camera 12 faces the left rear of the vehicle to image the left rear of the vehicle. The camera support body 32L is rotatable in the vehicle front-rear direction, and is rotated by the driving force of the camera storage ACT20 to a storage position where the camera support body 32L is substantially along the outer side surface of the vehicle in the longitudinal direction or a return position where the left rear camera 12 captures the left rear of the vehicle.

The left rear camera 12 is also provided with a mechanism for changing the direction of the shooting optical axis with its lens set at a fixed focus, and shoots a certain shooting range 62 shown in fig. 4 with the camera support body 32L at the return position. In the present embodiment, the vehicle left side rear image 62A projected on the virtual projection plane 66 behind the vehicle 50 is assumed to be the side rear image captured by the left side rear camera 12. Fig. 4 shows that the imaging range 62 of the left rear camera 12 partially overlaps the imaging range 60 of the rear camera 16, and that a laterally overlapping region of the laterally overlapping image 62A that overlaps the rear image 60A of the rear camera 16 projected on the virtual projection plane 66 is 62A 1.

A base portion of a camera support body 32R having a shape bilaterally symmetrical to the camera support body 32L is attached to a vehicle front side end portion of a vehicle vertical direction intermediate portion of a right side door (front side door, not shown) of the vehicle. The right rear camera 14 is attached near the distal end portion of the camera support body 32R, and the right rear camera 14 captures an image with its imaging optical axis (lens) directed to the right rear of the vehicle. The camera support body 32R is also rotatable in the vehicle front-rear direction, and is rotated by the driving force of the camera storage ACT20 to a storage position where the longitudinal direction of the camera support body 32R is substantially along the outer side surface of the vehicle, or a return position where the right side rear camera 14 captures the right side rear of the vehicle.

The right rear camera 14 is also provided with a mechanism for changing the direction of the shooting optical axis with the lens thereof set to a fixed focus, and shoots a certain shooting range 64 shown in fig. 4 with the camera support body 32R in the return position. In the present embodiment, the vehicle right side rearward image 64A projected on the virtual projection plane 66 behind the vehicle 50 is assumed to be a rearward side image captured by the right side rearward camera 14. Fig. 4 shows that the imaging range 64 of the right rear camera 14 partially overlaps the imaging range 60 of the rear camera 16, and that a side-rear overlapping region of the side-rear image 64A that overlaps the rear image 60A of the rear camera 16 projected on the virtual projection plane 66 is 64A 1.

The rear camera 16 is disposed at the rear of the vehicle 50 (see fig. 4), and the imaging optical axis (lens) faces the rear of the vehicle to image the rear of the vehicle 50. The position at which rear camera 16 is disposed may be any position as long as it can capture an image behind vehicle 50, and may be disposed at a rear end portion of vehicle 50 (for example, near a rear bumper) or may be disposed so as to capture an image behind vehicle 50 through a rear window glass. The rear camera 16 is also provided with a mechanism for changing the direction of the shooting optical axis with the lens thereof set to a fixed focus, and shoots a certain shooting range 60 shown in fig. 4. In the present embodiment, the rear image 60A projected to the center on the virtual projection plane 66 behind the vehicle 50 is set as the rear image captured by the rear camera 16. Fig. 4 shows a region of the rearward image 60A that overlaps the vehicle left side rearward image 62A and the vehicle right side rearward image 64A projected on the virtual projection plane 66 as a rearward overlapping region 60A 1.

A center monitor 34 is provided at a center portion of an instrument panel of the vehicle, and the electronic mirror display 18 is provided at a position spaced upward of the vehicle from the center monitor 34. The electronic mirror display 18 displays an image synthesized by applying a synthesizing process, which will be described later, to the left rear image (moving image) captured by the left rear camera 12, the right rear image (moving image) captured by the right rear camera 14, and the rear image (moving image) captured by the rear camera 16, by the electronic mirror ECU 22.

As shown in fig. 1, the electronic mirror ECU22 includes a CPU (Central Processing Unit) 24, a ROM (Read Only Memory) 26, a RAM (Random Access Memory) 28, and a storage device 30.

The CPU24 is a central processing unit that executes various programs to control the respective units. That is, the CPU24 reads out a program from the ROM26 or the storage device 30, and executes the program with the RAM28 as a work area.

(function constitution)

Fig. 2 is a block diagram showing a functional configuration of the electronic mirror ECU 22. As shown in the figure, the electronic mirror ECU22 includes a side rear imaging unit 220, a rear imaging unit 230, a detection unit 240, a determination unit 250, and a display control unit 260. Each functional configuration is realized by reading an execution program stored in the ROM26 or the storage device 30 and executing the execution program.

The lateral rearward imaging unit 220 images the lateral rearward sides of the vehicle 50 as moving images by the left and right rearward cameras 12 and 14, and outputs the captured lateral rearward images to the detection unit 240 and the display control unit 260.

The rear imaging unit 230 images the rear of the vehicle 50 as a moving image by the rear camera 16, and outputs the imaged rear image to the detection unit 240 and the display control unit 260.

The detector 240 analyzes the lateral rearward images 62A and 64A and the rearward image 60A, and detects the presence of the target object in the lateral rearward overlapping regions 62A1 and 64A1 overlapping the rearward image 60A in the lateral rearward images 62A and 64A and the left and right rearward overlapping regions 60A1 overlapping the lateral rearward images 62A and 64A in the rearward image 60A. Then, information on the detected target object is output to the determination unit 250. The detection unit 240 may be configured to detect the target object existing in the regions corresponding to the side-rear overlap regions 62a1 and 64a1 and the rear overlap region 60a1 based on the detection result of a radar or the like having the rear of the vehicle 50 as the detection range.

Based on the information on the target object output from the detection unit 240, the determination unit 250 determines whether or not the target object existing in the lateral overlap region 62a1, 64a1 and the rear overlap region 60a1 includes a specific target object recognized only in one of the lateral overlap region 62a1, 64a1 and the rear overlap region 60a 1. The determination unit 250 analyzes the lateral rearward images 62A and 64A and the rearward image 60A, for example, and determines the target object by comparing the target object detected in the lateral rearward images 62A and 64A with the target object detected in the rearward image 60A.

The display control unit 260 has a function of generating an image obtained by combining the side rearward images 62A and 64A and the rearward image 60A output from the side rearward image capturing unit 220 and the rearward image capturing unit 230, and displaying the image on the scope display 18. The routine of the synthesis processing includes a superimposition processing (see fig. 6) as a subroutine, in which when a vehicle 52 approaching to the rear side of the vehicle 50 in the lateral direction (see fig. 7) or a vehicle 54 approaching to the rear side of the vehicle 50 in the rear direction (see fig. 8) is detected, the specific object display image is superimposed on the rear image and the rear image.

(action and Effect)

Next, the operation of embodiment 1 and the combining process in which the mirror ECU22 causes the mirror display 18 to display the lateral rearward images captured by the left and right lateral rearward cameras 12 and 14 and the rearward image captured by the rearward camera 16 will be described with reference to fig. 4 and 5. This synthesis process is specifically described below: the mirror ECU22 extracts the side rear images 62A and 64A on the virtual projection plane 66 captured by the left side rear camera 12 and the right side rear camera 14.

In the next step, the electronic mirror ECU22 extracts an image corresponding to the extraction region 60A2 in which the right and left rear overlap regions 60A1 are deleted from the rear image 60A on the virtual projection plane 66 captured by the rear camera 16.

In the next step, the electronic mirror ECU22 synthesizes the lateral rear image 62A to the left side of the extraction region 60a2, and synthesizes the lateral rear image 64A to the right side of the extraction region 60a2 to generate a single image.

In the next step, the electronic mirror ECU22 causes the electronic mirror display 18 to display the lateral-rear images 62A, 64A and the extraction area 60a2 synthesized as a single image. Fig. 5 shows an example of an image displayed on the electron mirror display 18 by the above-described combining process.

As shown in fig. 5, the image displayed on the mirror display 18 is generated by deleting the right and left rear overlapping regions 60A1 from the rear image 60A on the virtual projection plane 66 to generate an image of the extraction region 60A2, and combining the side rear image and the rear image into a single image. As a result, the lateral rearward image and the rearward image displayed on the mirror display 18 are smoothly continuous images, and the mirror display 18 can display an image close to the rearward of the observation vehicle 50. As a result, the burden on the occupant when recognizing a specific object described later is reduced, and the visibility of the electronic mirror display 18 is improved.

In the next step, the mirror ECU22 determines whether there is a laterally rear approaching vehicle 52 (see fig. 7) approaching laterally rearward of the vehicle 50 or a rearward approaching vehicle 54 (see fig. 8) approaching rearward of the vehicle 50. The side rear approaching vehicle 52 and the rear approaching vehicle 54 can be detected by analyzing the side rear images 62A, 64A and the rear image 60A, for example, and whether or not the side rear approaching vehicle 52 and the rear approaching vehicle 54 are present within a relatively close distance can be determined based on whether or not the size of the image area corresponding to the side rear approaching vehicle 52 or the rear approaching vehicle 54 included in the side rear images 62A, 64A and the rear image 60A is equal to or larger than a predetermined value. The determination of the presence or absence of the laterally approaching vehicle 52 or the rearwardly approaching vehicle 54 may be based on, for example, a detection result of a radar or the like having the rear of the vehicle 50 as a detection range. In this case, the distance between the vehicle 50 and the side-rear approaching vehicle 52 or the rear approaching vehicle 54 can be determined more accurately.

If the determination is no, that is, if it is determined that neither the laterally rear approaching vehicle 52 nor the rear approaching vehicle 54 is present, the process returns to the start of the synthesis processing step, and the above steps are repeated. During this time, the generation and display of a single image in which the lateral rearward image and the rearward image are combined are continued.

On the other hand, as shown in fig. 7, for example, when there is a rear-side approaching vehicle 52 on the rear side of the right side of the vehicle 50, the determination in the above step is that the superimposition processing included as a subroutine in the above-described synthesis processing is executed (see fig. 6). As shown in fig. 7, in the case where the rear-side approaching vehicle 52 is traveling to the right rear-side of the vehicle 50, the 1 st blind area 70 is generated in the rear-side image 64A of the right rear camera 14 due to the rear-side approaching vehicle 52. The 1 st blind area 70 includes a side rear overlapping area 64a1 of the right side rear camera 14. Therefore, when the two-wheeled vehicle 56 is present in the rear-side overlap region 64a1 of the right rear-side camera 14, the two-wheeled vehicle 56 is hidden by the rear-side approaching vehicle 52 and does not appear in the rear-side image of the mirror display 18 (see fig. 9). Thus, since the presence of the two-wheeled vehicle 56 may not be recognized by the occupant visually checking the electronic mirror display 18, there is a high risk of contact with the two-wheeled vehicle 56, for example, when the vehicle 50 intends to make a lane change in a rearward direction approaching the vehicle 52 in a lateral rearward direction. Therefore, in the subsequent superimposing process, an image in which the specific target object display image M corresponding to the two-wheeled vehicle 56 is superimposed and displayed on the lateral and rearward image is generated and displayed as described later.

As an example, as shown in fig. 8, when there is a rear approaching vehicle 54 behind the vehicle 50, the determination in the above step is yes, and the superimposition processing is similarly executed. As shown in fig. 8, when the rear approaching vehicle 54 is traveling behind the vehicle 50, a2 nd blind area 72 is generated in the rear image 60A of the rear camera 16 due to the rear approaching vehicle 54. The 2 nd blind area 72 includes the rear overlap area 60a1 of the rear camera 16. Therefore, when the two-wheeled vehicle 56 is present in the rear overlap area 60a1 on the right side of the vehicle 50, the two-wheeled vehicle 56 is not visually recognized in the rear image of the mirror display 18 due to the rear approaching vehicle 54, and may be visually recognized in a state where a part of the rear approaching vehicle 54 is hidden in the side-rear image (see fig. 10). Thus, the presence of the two-wheeled vehicle 56 may not be recognized by the occupant visually checking the electronic mirror display 18, and therefore, when the vehicle 50 intends to make a lane change to the right lane, the risk of contact with the two-wheeled vehicle 56 is increased. Therefore, an image in which the specific target object display image M corresponding to the two-wheeled vehicle 56 is displayed superimposed on the rear image is generated and displayed by the superimposition processing.

Specifically, as shown in fig. 6, in step S100, the electronic mirror ECU22 detects the presence of a target object in the lateral rear overlap regions 62A1 and 64A1 of the left and right lateral rear images 62A and 64A and the lateral rear overlap region 60A1 of the rear image 60A.

In next step S101, the electronic mirror ECU22 determines whether or not the specific objects visually recognized only in either one of the side rear overlap region 62a1 and the rear overlap region 60a1 on the left side of the vehicle 50 and the specific objects visually recognized only in either one of the side rear overlap region 64a1 and the rear overlap region 60a1 on the right side of the vehicle 50 are included in the objects detected in the side rear overlap regions 62a1, 64a1 and the rear overlap region 60a 1. If the determination at step S101 is yes, the process proceeds to step S102. When the determination at step S101 is no, the superimposition processing is ended, and the process returns to the synthesis processing.

For example, as shown in fig. 7, when the side rear approaching vehicle 52 is traveling to the right side of the vehicle 50 and to the rear side, the two-wheeled vehicle 56 is present in the 1 st blind area 70 caused by the side rear approaching vehicle 52, and therefore, the side rear image 64A captured by the right side rear camera 14 is not visually recognized. Therefore, in step S100, the presence of the target object (the two-wheeled vehicle 56) is not detected (visually confirmed) from the lateral rear overlap region 64A1 corresponding to the right-side rear camera 14 of the vehicle 50 by the image recognition of the lateral rear image 64A and the rear image 60A, and the two-wheeled vehicle 56 is detected (visually confirmed) as the target object from the right-side rear overlap region 60A1 corresponding to the rear camera 16. Then, in the next step S101, the two-wheeled vehicle 56 detected in step S100 is set as a specific target object visually recognized only in the rear overlap region 60a 1. Therefore, in step S101, the mirror ECU22 determines that the target object (the two-wheeled vehicle 56 in the example of fig. 7) detected in step S100 includes the specific target object (the two-wheeled vehicle 56).

Further, as shown in fig. 8, when the rear approaching vehicle 54 is traveling behind the vehicle 50, the two-wheeled vehicle 56 is present in the 2 nd blind area 72 generated by the rear approaching vehicle 54, and therefore is not visually recognized in the rear image 60A of the rear camera 16. Therefore, in step S100, the presence of the target object (the two-wheeled vehicle 56) is detected (visually confirmed) from the side-rear overlap region 64A1 by the image recognition of the side-rear image 64A and the rear image 60A corresponding to the right side rear camera 14, while the presence of the target object is not detected (visually confirmed) from the rear overlap region 60A 1. Then, in the next step S101, the two-wheeled vehicle 56 detected in step S100 is set as a specific target object visually recognized only in one of the side-to-rear overlapping areas 64a 1. Therefore, in step S101, the mirror ECU22 determines that the target object (the two-wheeled vehicle 56 in the example of fig. 9) detected in step S100 includes the specific target object (the two-wheeled vehicle 56).

In step S102, the electronic mirror ECU22 determines whether or not the specific object target is approaching the vehicle 50. The determination in step S102 can be achieved by determining whether or not a region corresponding to the specific target object (the two-wheeled vehicle 56 in fig. 7 and 8) exists in the lateral-rearward images 62A and 64A or the rearward image 60A in which the specific target object is recognized, and the size of the region increases with the passage of time. Further, the determination may be made based on a detection result of a radar or the like having a range including the lateral rear of the vehicle 50 as a detection range.

If the determination at step S102 is yes, the process proceeds to step S103. If the determination at step S102 is no, the superimposition processing is ended, and the process returns to the synthesis processing. In step S103, the electronic mirror ECU22 superimposes a specific object display image corresponding to the specific object on the lateral/posterior image or the posterior image in which the specific object is not visually recognized, and displays the superimposed image on the electronic mirror display 18.

For example, in the situation shown in fig. 7, the electronic mirror ECU22 generates an image in which the side-rear image 64A corresponding to the right side rear camera 14 of the two-wheeled vehicle 56 in which the specific target object is not visually recognized is superimposed on the specific target object display image M corresponding to the two-wheeled vehicle 56 visually recognized in the right side rear superimposed region 60a 1. Then, as shown in fig. 9, the electronic mirror ECU22 displays the generated lateral rearward image on the electronic mirror display 18. The specific target object display image M is displayed as a frame line surrounding the specific target object (the two-wheeled vehicle 56), for example.

The specific object display image M is displayed in a superimposed manner on the lateral rearward approaching vehicle 52 appearing in the lateral rearward image 64A. This allows the occupant visually checking the electronic mirror display 18 to recognize that the two-wheeled vehicle 56 is present behind the side-rear approach vehicle 52.

On the other hand, in the case of the situation shown in fig. 8, the electronic mirror ECU22 generates an image in which the rear image 60A corresponding to the rear camera 16 of the two-wheeled vehicle 56 for which no specific target object is visually recognized is superimposed on the specific target object display image M corresponding to the two-wheeled vehicle 56 visually recognized in the lateral rear superimposed region 64a1 of the right lateral rear camera 14. Then, as shown in fig. 10, the electronic mirror ECU22 displays the generated rearward image on the electronic mirror display 18.

The specific object display image M is displayed in a superimposed manner on the rear approaching vehicle 54 appearing in the rear processed image. This allows the occupant visually checking the mirror display 18 to recognize the presence of the two-wheeled vehicle 56 approaching to the rear side of the rearward approaching vehicle 54.

The specific object display image M may be a predetermined icon. Further, the displayed specific object display image may be caused to blink, and the number of times of blinking may be increased as the distance between the specific object and the vehicle 50 is closer. As shown in fig. 11, an image of a portion corresponding to a specific object (the two-wheeled vehicle 56) recognized in the lateral and rearward images 62A and 64A or the rearward image 60A may be extracted as the specific object display image M. The image constituting the specific object display image M may be generated as a see-through image so that the visibility is lower than the visibility of the other image displayed on the electronic mirror display 18.

[ 2 nd embodiment ]

Hereinafter, embodiment 2 of the present invention will be described with reference to fig. 12 to 15. Note that the same components as those in embodiment 1 are denoted by the same reference numerals, and description thereof is omitted. The basic configuration of the in-vehicle system 80 according to embodiment 2 is the same as that of embodiment 1, and is characterized in that "the side rear images and the rear images captured by the left side rear camera 12, the right side rear camera 14, and the rear camera 16 are individually displayed on a plurality of display units" (see fig. 14).

(hardware constitution)

That is, the in-vehicle system 80 includes the electronic mirror device 90 including the electronic mirror ECU92, the left rear camera 12, the right rear camera 14, the rear camera 16, the electronic mirror display 94, and the camera storage ACT 20. The electron mirror display 94 as a display unit includes a left display 94A, a right display 94B, and a center display 94C. The left display 94A displays an image captured with the left rear camera 12. The right display 94B displays an image captured with the right rear camera 14. The center display 94C displays an image captured by the rear camera 16.

(function constitution)

Fig. 13 is a block diagram showing a functional configuration of the electronic mirror ECU 92. The display control unit 960 extracts the left and right lateral rear images 62A and 64A on the virtual projection plane 66 shown in fig. 4, and displays them on the left display 94A and the right display 94B, respectively. That is, the display controller 960 displays the lateral rearward images 62A and 64A including the lateral rearward overlapping regions 62A1 and 64A1 on the mirror display 94 as lateral rearward images.

The display control unit 960 extracts the rear image 60A on the virtual projection plane 66 shown in fig. 4 and displays the rear image on the center display 94C. That is, the display control unit 960 displays the rearward image 60A including the rearward overlapping area 60A1 on the mirror display 94 as a rearward image.

The display control unit 960 executes the overlay processing shown in fig. 6, similarly to the display control unit 260 according to embodiment 1. Note that the side rear imaging unit 920, the rear imaging unit 930, the detection unit 940, and the determination unit 950 have the same functions as the side rear imaging unit 220, the rear imaging unit 230, the detection unit 240, and the determination unit 250 of embodiment 1, and therefore, the description thereof is omitted.

(action and Effect)

The electronic mirror device 90 having the above configuration is configured in the same manner as the electronic mirror device 10 according to embodiment 1, except that the lateral rearward processed image and the rearward image are displayed separately, and therefore, the same effects as those of embodiment 1 can be obtained.

As shown in fig. 14, the occupant in the vehicle compartment can observe the lateral rear images 62A and 64A including the left and right lateral rear overlap regions 62A1 and 64A1 on the virtual projection plane 66 and the rear image 60A including the left and right rear overlap region 60A1 on the virtual projection plane 66. Thus, the occupant observing the electronic mirror display 94 can recognize the surrounding situation behind the vehicle over a wide range. In addition, as an example, the schematic diagram of fig. 14 is a side rear image and a rear image displayed on the electronic mirror display 94 in the state shown in fig. 7.

In the present embodiment, both the specific target object (the motorcycle 56) and the specific target object display image M corresponding thereto are displayed on the electronic mirror display 94 including the left display 94A, the right display 94B, and the center display 94C. Therefore, the position of the specific object can be clearly recognized by visually confirming the image of the specific object and the specific object display image M on the plurality of displays. As a result, the visibility of the electron mirror display 94 is improved.

In addition, in the case where both the specific target object (the two-wheeled vehicle 56) and the corresponding specific target object display image M are displayed on the electronic mirror display 94 as in the present embodiment, the same display image as the specific target object display image M may be displayed in a manner superimposed on the image of the specific target object (the two-wheeled vehicle 56) as in the example shown in fig. 15. Thus, even when the specific target object and the specific target object display image are displayed on different displays, the occupant can quickly recognize that the specific target object and the specific target object display image are information related to the same target object by observing the same display.

In the above-described embodiment 1, the lateral rearward image 62A of the left rearward camera 12 and the lateral rearward image 64A of the right rearward camera 14 on the virtual projection plane 66 are displayed on the mirror display 18 as the lateral rearward images of the present invention, and the extraction area 60A2 in which the left and right rearward overlapping areas 60A1 are deleted from the rearward image 60A of the rearward camera 16 is displayed on the mirror display 18 as the rearward image of the present invention. However, the present invention is not limited to this, and an image extracted by deleting the lateral overlapping regions 62A1, 64A1 from the lateral images 62A, 64A may be displayed on the electronic mirror display 18 as a lateral image of the present invention, and the rear image 60A on the virtual projection plane 66 may be displayed on the electronic mirror display 18 as a rear image of the present invention.