阅读说明：本技术 操作辅助系统和操作辅助方法 (Operation assistance system and operation assistance method ) 是由 齐藤航太 松仪良广 松浦侑纪 崔胜镐 松尾祥和 佐藤大海 于 2019-07-24 设计创作，主要内容包括：本发明提供一种操作辅助系统和操作辅助方法。操作辅助系统(200)具有：动作识别部(27),其识别乘员的手部的规定动作；位置识别部(28),其识别乘员的手(H)相对于第1车辆(10)的相对位置即第1相对位置(90)；确定部(29),其确定使用第2车辆(110)的操作部(38)操作的第2车辆(110)的第2车载装置,其中所述第2车辆的操作部被设置于相对于第2车辆(110)的相对位置、即相当于第1相对位置(90)的第2相对位置(92)；和控制部(30),其辅助对具有与第2车载装置共同功能的第1车辆(10)的第1车载装置进行的操作。根据本发明,当乘员对开关操作感到迷惑时能够适当地辅助对车载装置进行的操作。(The invention provides an operation assisting system and an operation assisting method. An operation assistance system (200) is provided with: a motion recognition unit (27) that recognizes a predetermined motion of the hand of the occupant; a position recognition unit (28) that recognizes a 1 st relative position (90), which is the relative position of the hand (H) of the occupant with respect to the 1 st vehicle (10); a determination unit (29) that determines a 2 nd in-vehicle device of a 2 nd vehicle (110) that is operated using an operation unit (38) of the 2 nd vehicle (110), the 2 nd vehicle operation unit being provided at a 2 nd relative position (92) that is a relative position with respect to the 2 nd vehicle (110), that is, a 1 st relative position (90); and a control unit (30) that assists an operation performed on a 1 st in-vehicle device of a 1 st vehicle (10) having a function common to the 2 nd in-vehicle device. According to the present invention, it is possible to appropriately assist the operation of the in-vehicle device when the occupant is confused about the switch operation.)

1. An operation assist system that assists an operation performed on a 1 st vehicle-mounted device of a 1 st vehicle, characterized in that,

comprises a motion recognition unit, a position recognition unit, a storage unit, a determination unit, and a control unit,

the motion recognition unit recognizes a predetermined motion of a hand of an occupant seated in the 1 st vehicle;

the position recognition portion recognizes a 1 st relative position that is a relative position of the occupant's hand with respect to the 1 st vehicle;

the storage unit stores a relative position of an operating unit of a 2 nd vehicle with respect to the 2 nd vehicle in association with a 2 nd in-vehicle device of the 2 nd vehicle operated using the operating unit;

the determination portion determines the 2 nd in-vehicle device of the 2 nd vehicle that is operated using the operation portion of the 2 nd vehicle, which is provided at a 2 nd relative position equivalent to the 1 st relative position with respect to the 2 nd vehicle, from the storage of the storage portion;

the control portion assists an operation performed on the 1 st in-vehicle device of the 1 st vehicle, wherein the 1 st in-vehicle device of the 1 st vehicle has a function common to the 2 nd in-vehicle device.

2. The operation assistance system according to claim 1,

the motion recognition unit recognizes a motion recognized as a state in which the occupant is confused about an operation as the predetermined motion,

the control unit assists the operation of the 1 st vehicle-mounted device when the motion recognition unit recognizes the motion recognized as a state in which the occupant is confused about the operation.

3. The operation assist system according to claim 1 or 2,

the storage unit stores a relative position of the operation unit of the 2 nd vehicle with respect to the 2 nd vehicle in which the occupant has been seated in the past, in association with a 2 nd in-vehicle device of the 2 nd vehicle operated by the operation unit.

4. The operation assist system according to claim 1 or 2,

the storage portion stores vehicle model information of the 2 nd vehicle taken in the past with respect to the occupant in correspondence with the occupant, wherein the vehicle model information is at least one of producer information indicating a producer of the 2 nd vehicle, producer information indicating a producer country of the 2 nd vehicle, and vehicle model information indicating a vehicle model of the 2 nd vehicle,

the control portion assists an operation performed on the 1 st vehicle-mounted device in a case where at least a part of the vehicle model information on the 1 st vehicle does not coincide with the vehicle model information on the 2 nd vehicle on which the occupant has been taken in the past.

5. The operation assist system according to claim 1 or 2,

the 1 st vehicle-mounted device and the 2 nd vehicle-mounted device are at least one of an air conditioning device, a hazard warning lamp device, and a wiper device.

6. An operation assisting method that assists an operation performed on a 1 st in-vehicle device of a 1 st vehicle, characterized in that,

executing a storage step of storing, using a storage unit, a relative position of an operating unit of a 2 nd vehicle with respect to the 2 nd vehicle in correspondence with a 2 nd in-vehicle device of the 2 nd vehicle operated using the operating unit;

the operation recognition step, the position recognition step, the determination step and the control step are executed by using an arithmetic section,

recognizing a predetermined motion of a hand of a passenger seated in the 1 st vehicle in the motion recognizing step;

in the position recognition step, a 1 st relative position that is a relative position of the hand of the occupant with respect to the 1 st vehicle is recognized;

in the specifying step, the 2 nd in-vehicle device of the 2 nd vehicle that is operated using the operating portion of the 2 nd vehicle provided at the 2 nd relative position that is a relative position with respect to the 2 nd vehicle, that is, the 1 st relative position, is specified based on the storage in the storage unit;

in the control step, an operation of the 1 st in-vehicle device of the 1 st vehicle having a function common to the 2 nd in-vehicle device is assisted.

Technical Field

The present invention relates to an operation assisting system and an operation assisting method for assisting an operation performed on an in-vehicle device of a 1 st vehicle.

Background

Japanese patent laid-open publication No. 2008-257363 discloses an operation assisting device for assisting an operation of an in-vehicle device by a vehicle occupant (operator). When it is determined that the occupant is confused, the operation support device detects an in-vehicle device that is an operation target from the line-of-sight direction of the occupant or the position of the finger, and outputs operation support information of the in-vehicle device through a voice output device or a display device.

Disclosure of Invention

When the occupant who is riding the vehicle mistakenly thinks that the occupant is riding another vehicle, the occupant may want to operate the switch by extending his or her hand to the position where the switch is present in the other vehicle when operating the in-vehicle device. When there is no switch at this position in the own vehicle, the occupant may be confused about the switching operation. In the device disclosed in japanese patent application laid-open No. 2008-257363, the device cannot be an in-vehicle device for detecting an operation target when there is no switch in the direction of the line of sight of the occupant or in the position of the finger.

The present invention has been made in view of the above-described problems, and an object thereof is to provide an operation assisting system and an operation assisting method for appropriately assisting an operation performed on an in-vehicle device when an occupant is confused about a switch operation.

The 1 st aspect of the present invention is an operation support system for supporting an operation of a 1 st vehicle-mounted device of a 1 st vehicle, characterized in that,

comprises a motion recognition unit, a position recognition unit, a storage unit, a determination unit, and a control unit,

the motion recognition unit recognizes a predetermined motion of a hand of a passenger seated in the 1 st vehicle (a predetermined motion on the hand);

the position recognition portion recognizes a 1 st relative position that is a relative position of the occupant's hand with respect to the 1 st vehicle;

the storage unit stores a relative position of an operating unit of a 2 nd vehicle with respect to the 2 nd vehicle in association with a 2 nd in-vehicle device of the 2 nd vehicle operated using the operating unit;

the determination portion determines the 2 nd in-vehicle device of the 2 nd vehicle that is operated using the operation portion of the 2 nd vehicle, which is provided at a 2 nd relative position equivalent to the 1 st relative position with respect to the 2 nd vehicle, from the storage of the storage portion;

the control portion assists an operation performed on the 1 st in-vehicle device of the 1 st vehicle, wherein the 1 st in-vehicle device of the 1 st vehicle has a function common to the 2 nd in-vehicle device.

A 2 nd aspect of the present invention is an operation assisting method for assisting an operation of a 1 st onboard device of a 1 st vehicle,

executing a storage step of storing, using a storage unit, a relative position of an operating unit of a 2 nd vehicle with respect to the 2 nd vehicle and a 2 nd in-vehicle device of the 2 nd vehicle operated using the operating unit in a correspondence relationship;

the operation recognition step, the position recognition step, the determination step and the control step are executed by using an arithmetic section,

recognizing a predetermined motion of a hand of a passenger seated in the 1 st vehicle in the motion recognizing step;

in the position recognition step, a 1 st relative position that is a relative position of the hand of the occupant with respect to the 1 st vehicle is recognized;

in the specifying step, the 2 nd in-vehicle device of the 2 nd vehicle that is operated using the operating portion of the 2 nd vehicle provided at the 2 nd relative position that is a relative position with respect to the 2 nd vehicle, that is, the 1 st relative position, is specified based on the storage in the storage unit;

in the control step, an operation of the 1 st in-vehicle device of the 1 st vehicle having a function common to the 2 nd in-vehicle device is assisted.

According to the present invention, it is possible to appropriately assist the operation of the in-vehicle device when the occupant is confused about the switch operation.

The above objects, features and advantages should be readily understood from the following description of the embodiments with reference to the accompanying drawings.

Drawings

Fig. 1 is a configuration diagram of an operation support system according to the present embodiment.

Fig. 2 is a diagram for explaining the relative position.

Fig. 3 is a diagram showing personal information stored in the personal identification DB.

Fig. 4 is a diagram showing occupant-vehicle information stored in the occupant-vehicle apparatus DB.

Fig. 5 is a flowchart of the main process performed by the operation assisting apparatus.

Fig. 6 is a flowchart of the operation assisting process performed by the operation assisting device.

Fig. 7A is a diagram showing a relative position in the 1 st vehicle, and fig. 7B is a diagram showing a relative position in the 2 nd vehicle.

Detailed Description

Hereinafter, the operation support system and the operation support method according to the present invention will be described in detail with reference to the drawings by referring to preferred embodiments.

[1. Structure of operation assistance System 200 ]

As shown in fig. 1, the operation assistance system 200 includes the 1 st vehicle 10 and the server 102.

[1.1. the 1 st vehicle 10]

The structure of the 1 st vehicle 10 will be described with reference to fig. 1. In the present specification, a host vehicle on which an occupant who is the subject of the description is seated is referred to as a 1 st vehicle 10, and vehicles other than the host vehicle are referred to as 2 nd vehicles 110 (fig. 7B). In the following description, the virtual 1 st vehicle 10 is a vehicle used on the premise that a plurality of occupants and vehicles are present, for example, a vehicle used for vehicle rental, vehicle sharing, or the like, a company vehicle, a utility vehicle, or the like.

The 1 st vehicle 10 has a camera 12, an identification information sensor 14, a communication device 16, an operation assist device 18, and various in-vehicle devices 20. As shown in fig. 2, various operation portions 38 for controlling the in-vehicle device 20 are provided on an instrument panel (dashboard)58 and a steering wheel 60 of the 1 st vehicle 10. The operation unit 38 is a switch such as a button or an operation lever.

The camera 12 is provided at a position, for example, a top plate, capable of imaging a predetermined range including 1 or more operation portions 38 and the periphery of each operation portion 38. When a plurality of operation units 38 are present in the cabin of the 1 st vehicle 10, the plurality of operation units 38 may be imaged by the 1 camera 12, or the operation units 38 may be individually imaged by the plurality of cameras 12. The camera 12 captures an image of the operation unit 38 and the periphery of the operation unit 38 to acquire image information, and outputs the acquired image information to the operation assisting device 18.

The identification information sensor 14 detects information for identifying an occupant (hereinafter referred to as identification information 70 (fig. 3)). The identification information sensor 14 is a biological information sensor that detects biological information 70a (fig. 3) of the occupant, an input device for a personal identification number 70b (fig. 3) input by the occupant, and the like. As the biological information sensor, at least one sensor for detecting biological information 70a of face, eye, blood vessel, fingerprint, palm, and voice of the passenger is provided. The identification information sensor 14 outputs the identification information 70 to the operation assisting device 18.

The communication device 16 includes a communication circuit, an antenna, and the like, and performs data communication with the server 102 via the communication line 100, and also performs data communication with the operation assisting device 18 via a communication line in the 1 st vehicle 10.

The operation assisting device 18 is an Electronic Control Unit (ECU) having an arithmetic unit 24, a storage unit 25, an input/output interface (not shown), and the like. The arithmetic unit 24 includes a processor including a CPU or the like, and executes a program stored in the storage unit 25 by the processor to realize various functions. In the present embodiment, the calculation unit 24 functions as an occupant recognition unit 26, a motion recognition unit 27, a position recognition unit 28, a specification unit 29, and a control unit 30. Various functions are described in the following [2 ].

The storage unit 25 includes memories such as ROM and RAM. The storage unit 25 stores therein the coordinate information 32, a personal identification database 34 (hereinafter referred to as a personal identification DB 34), and a passenger-in-vehicle device database 36 (hereinafter referred to as a passenger-in-vehicle device DB 36).

As shown in fig. 2, the coordinate information 32 defines relative positions in a case where the front surfaces of the instrument panel 58 and the steering wheel 60 are set as a coordinate plane 64. In the present embodiment, the coordinate plane 64 has the center position of the steering wheel 60 as the origin, and has a straight line extending in the vertical direction of the vehicle and a straight line extending in the width direction of the vehicle as coordinate axes. In the present embodiment, the coordinate plane 64 is divided at predetermined intervals in the vertical direction and the width direction, and the position of each cell is represented by a vertical coordinate position Vn and a width coordinate position Hn. The coordinate positions Vn to Hn of the squares are relative positions with the center position of the steering wheel 60 as a reference position. The coordinate plane 64 is commonly used in the 1 st vehicle 10 and the 2 nd vehicle 110.

The individual identification DB34 holds the individual information 66 shown in fig. 3. The personal information 66 associates an occupant identification number 68 with occupant identification information 70. As described above, the identification information 70 is information for identifying the occupant, and is, for example, information indicating at least one of the biometric information 70a and the personal identification code 70 b.

The occupant-vehicle apparatus DB36 holds the occupant-vehicle information 72 shown in fig. 4. The occupant-vehicle information 72 associates the occupant identification number 68, the vehicle model information 74, the riding information 76, and the in-vehicle device information 78. The vehicle model information 74 is information relating to vehicles (including the 1 st vehicle 10, the 2 nd vehicle 110.), and includes producer information 80 indicating a producer (production company) of the vehicle, producer information 82 indicating a production country of the vehicle, and vehicle type information 84 indicating a vehicle type of the vehicle. The ride information 76 indicates whether the occupant is riding in the vehicle. The riding information 76 includes information indicating the latest riding date and time, riding number, and the like, in addition to information indicating that riding is performed. The in-vehicle device information 78 is information related to the in-vehicle device 20, and includes device type information 86 indicating the type of the in-vehicle device 20 and position information 88 indicating the relative position of the operation unit 38 where the in-vehicle device 20 is disposed in the vehicle.

The storage unit 25 stores vehicle model information 74 of the vehicle itself (here, the 1 st vehicle 10), unlike the occupant-in-vehicle device DB 36.

The description is continued with reference to fig. 1. The in-vehicle device 20 includes an air conditioning device 40, a hazard lamp device 42, a wiper device 44, a notification device 46, and the like. The air conditioner 40 includes a main body for sending cool air or warm air to the inside of the vehicle, an air conditioner ECU for controlling the main body, and an air conditioner switch 48 (operation unit 38). The air conditioner 40 performs operations such as starting, stopping, and changing the set temperature in response to an operation instruction output from the operation assisting device 18 or an operation of the air conditioner switch 48 by the occupant. The hazard lamp device 42 includes a lamp body, a hazard lamp circuit, a hazard lamp controller that controls energization of the hazard lamp circuit, and a hazard lamp switch 50 (operation unit 38). The hazard lamp 42 is activated or deactivated in response to an operation instruction from the occupant to operate the hazard lamp switch 50 or the operation support device 18. The wiper device 44 includes a wiper arm, a wiper circuit including a motor that drives the wiper arm, and a wiper controller and a wiper switch 52 (operation unit 38) that perform energization control of the wiper circuit. The wiper device 44 starts and stops in response to an operation instruction from the occupant to operate the wiper switch 52 or from the operation assisting device 18. The notification device 46 has a display device 54 that provides visual information to the occupant and an audio device 56 that provides voice information to the occupant.

[1.2. Server 102]

The server 102 is installed in a facility of a system administrator such as a rental car, and includes a server-side storage unit 104. The server-side storage unit 104 stores the personal information 66 (fig. 3) of each occupant and the occupant-vehicle information 72 (fig. 4). The personal information 66 and the occupant-vehicle information 72 are registered in the server-side storage unit 104 by an operation input device such as an operator of a system administrator.

[2. operation of the operation assisting device 18 ]

[2.1. Main treatment ]

The main processing performed by the operation assisting device 18 will be described with reference to fig. 5.

In step S1, the control unit 30 requests the server 102 to transmit the personal information 66 and the occupant-vehicle information 72 via the communication device 16 at a predetermined timing. For example, the control unit 30 outputs a request instruction for each piece of information (the personal information 66 and the occupant-vehicle information 72) when the ignition switch or the start switch (both not shown) of the 1 st vehicle 10 is operated and the power supply is switched from off to on. The communication device 16 transmits a request instruction to the server 102, and downloads the personal information 66 and the occupant-vehicle information 72 from the server 102 and stores them in the storage unit 25.

In step S2, the occupant recognition unit 26 acquires the identification information 70 detected by the identification information sensor 14.

In step S3, the occupant recognition unit 26 collates the acquired identification information 70 with the identification information 70 of the individual information 66 stored in the individual recognition DB 34. When the identification information 70 matches or is similar to the identification information 70 in the personal identification DB34, the occupant identification unit 26 identifies the occupant seated in the 1 st vehicle 10 as the occupant who is the identification number 68 associated with the identification information 70.

When the occupant can be identified in step S3 (step S4: YES), the process proceeds to step S5. On the other hand, when the occupant cannot be identified in step S3 (step S4: NO), the main process ends.

When the flow advances from step S4 to step S5, the operation assisting process shown in fig. 6 is performed. The operation assisting process of step S5 is repeated when the power supply of the 1 st vehicle 10 is turned on (yes in step S6), and ends when the power supply of the 1 st vehicle 10 is switched off (no in step S6). The main process ends together with the end of the operation assisting process.

[2.2. operation aid treatment ]

The operation assisting process performed in step S5 in fig. 5 will be described with reference to fig. 6.

In step S11, the motion recognition unit 27 and the position recognition unit 28 acquire image information output by the camera 12.

In step S12, the motion recognition unit 27 performs a known image recognition process using the image information to recognize a predetermined motion of the hand of the occupant. The motion recognition unit 27 recognizes a motion recognized as a state in which the occupant is confused about the operation, for example, a motion in which the hand H (fig. 7A) of the occupant repeatedly moves a little in a short time as a predetermined motion. In the present specification, the repeated small movements of the hand H in a short time are also referred to as confusing movements of the hand H. The motion recognition unit 27 can recognize that the hand H of the occupant is performing the confusion motion by using a known gesture recognition technique (for example, the technique of japanese patent laid-open publication No. 2014-174776). Alternatively, the motion recognition unit 27 recognizes that the hand H of the passenger is performing the confusing motion when recognizing that the hand H repeats the same motion within a certain range for a predetermined time. In the present embodiment, this state is assumed to be: an occupant who is riding in the 1 st vehicle 10 erroneously attempts to perform a switching operation in view of riding in the 2 nd vehicle 110.

When the motion recognition unit 27 recognizes the confusion motion of the hand H (yes in step S13), the process proceeds to step S14. On the other hand, when the motion recognition unit 27 does not recognize the confusion motion of the hand H (no in step S13), the operation support processing is ended.

When proceeding from step S13 to step S14, the position recognition portion 28 recognizes the relative position of the hand H with respect to the 1 st vehicle 10 from the image information and the coordinate information 32. This relative position is referred to as the 1 st relative position 90. For example, in the situation shown in FIG. 7A, the position recognition section 28 recognizes the 1 st relative position 90 of the hand H as the coordinate positions V2-H2, V3-H2.

In step S15, the determination unit 29 determines the 2 nd vehicle 110 estimated to be in which the occupant is mistaken for the ride. Here, the specifying unit 29 searches the occupant-vehicle device DB36 using the identification number 68 of the occupant identified in step S2, and extracts the occupant-vehicle information 72 whose identification number 68 matches. The determination unit 29 determines the 2 nd vehicle 110 from the vehicle model information 74 included in the extracted occupant-vehicle information 72.

At this time, the determination unit 29 uses the extracted occupant-vehicle information 72 when there are only 1 piece of the occupant-vehicle information 72. On the other hand, when there are a plurality of pieces of extracted occupant-vehicle information 72, the determination unit 29 selects 1 piece of occupant-vehicle information 72. At this time, the determination section 29 selects the occupant-vehicle information 72, of which at least 1 piece of information included in the vehicle model information 74 is different from the vehicle model information 74 of the 1 st vehicle 10. Preferably, the occupant-vehicle information 72 having the riding information 76 "present" is selected. However, when a plurality of occupant-vehicle information 72 are selected, the control unit 30 selects 1 occupant-vehicle information 72 according to a predetermined rule. For example, priority orders of the producer information (manufacturer information) 80, the country of production information (country of manufacture information) 82, the vehicle type information 84, and the riding information 76 included in the vehicle model information 74 are set in advance. The result of comparison of information having a high priority order is preferentially used. Alternatively, the occupant-vehicle information 72 with the latest riding date and time of the riding information 76 or the occupant-vehicle information 72 with the largest number of riding times is selected. Of course, when the vehicle model information 74 included in the occupant-vehicle information 72 and the vehicle model information 74 of the 1 st vehicle 10 match, the determination unit 29 does not use the occupant-vehicle information 72. This is because the occupant-vehicle information 72 is information relating to the 1 st vehicle 10 on which the occupant is seated.

In step S16, the determination unit 29 determines the in-vehicle device 20 of the 1 st vehicle 10 (the 1 st in-vehicle device) that is estimated to be the occupant' S attempt to operate. Here, the determination portion 29 determines the in-vehicle device 20 of the 2 nd vehicle 110 (the 2 nd in-vehicle device) that is operated using the operation portion 38 of the 2 nd vehicle 110, which is provided at the 2 nd relative position 92 corresponding to the 1 st relative position 90, which is the relative position with respect to the 2 nd vehicle 110, from the occupant-vehicle information 72 stored in the storage portion 25. Specifically, the specifying unit 29 compares the 1 st relative position 90 identified by the position identifying unit 28 in step S14 with the position information 88 included in the occupant-vehicle information 72 of the 2 nd vehicle 110 specified in step S15. When there is the matching position information 88, the specifying unit 29 specifies the in-vehicle device 20 (1 st in-vehicle device) of the 1 st vehicle 10 having a function common to the in-vehicle device 20 (2 nd in-vehicle device) associated with the position information 88. For example, as shown in fig. 7B, the air-conditioning switch 48 (operation unit 38) of the air-conditioning apparatus 40 is disposed at the 2 nd relative position 92 (coordinate positions V2-H2, V3-H2) corresponding to the 1 st relative position 90 (coordinate positions V2-H2, V3-H2). In this case, the determination unit 29 determines that the in-vehicle device 20 of the 2 nd vehicle 110 operated using the air-conditioning switch 48 is the air-conditioning device 40. The determination unit 29 determines that the air conditioner 40, which is the in-vehicle device 20 of the 1 st vehicle 10 having the function common to the in-vehicle device 20 of the 2 nd vehicle 110, is the in-vehicle device 20 that the occupant intends to operate.

In step S17, the control unit 30 assists the operation of the in-vehicle device 20. For example, the control unit 30 outputs a notification instruction to the notification device 46. The notification device 46 guides the 1 st relative position 90 of the operation unit 38 of the in-vehicle device 20 (the 1 st in-vehicle device) by an image or voice. Alternatively, the control unit 30 makes an inquiry to the occupant about the start or stop of the in-vehicle device 20, and performs control such as the start or stop of the in-vehicle device 20 in accordance with a response such as the voice of the occupant.

[3. modification ]

In addition to the above-described embodiments, various modifications of the present invention are also conceivable.

In the above-described embodiment, the 1 st vehicle 10 is assumed to be a vehicle, a company vehicle, a utility vehicle, or the like used for vehicle rental, vehicle sharing, or the like, but the 1 st vehicle 10 may be a vehicle owned by an individual. In this case, the storage unit 25 may store the personal information 66 of the specified occupant in advance, and omit the processing (determination of possibility or impossibility of recognition) performed in step S3 (recognition of the occupant) and step S4 shown in fig. 5. The storage unit 25 may store the occupant-vehicle information 72 corresponding to the identified occupant in advance, and omit the processing (request for transmission of each information) performed at step S1 and the processing (acquisition of the identification information 70) performed at step S2 shown in fig. 5.

In the above embodiment, the arithmetic unit 24 of the operation assisting device 18 included in the 1 st vehicle 10 functions as the specifying unit 29. Instead, a processor outside the 1 st vehicle 10, for example, a processor provided in the server 102 may function as the determination unit 29. The coordinate information 32 and the occupant-vehicle device DB36 may be stored in the server-side storage unit 104.

In this case, in the operation assisting process, when the position recognition portion 28 recognizes the 1 st relative position 90, the information of the 1 st relative position 90 is transmitted from the 1 st vehicle 10 to the server 102. In addition, when the determination portion 29 of the server 102 determines the in-vehicle device 20 (1 st in-vehicle device) that the occupant attempts to operate, information of the in-vehicle device 20 is transmitted from the server 102 to the 1 st vehicle 10.

If the in-vehicle device 20 cannot be identified in step S16 shown in fig. 6, the process may return to step S15 to identify the other 2 nd vehicle 110.

[4 ] invention that can be obtained according to the embodiment ]

The following describes an invention that can be grasped from the above-described embodiment and modifications.

The present invention is an operation assisting system 200 that assists an operation performed on a 1 st in-vehicle device 20 of a 1 st vehicle 10,

has a motion recognition unit 27, a position recognition unit 28, a storage unit 25 (occupant-vehicle device DB36), a determination unit 29, and a control unit 30, wherein,

the motion recognition unit 27 recognizes a predetermined motion of a hand of an occupant seated in the 1 st vehicle 10;

the position recognition portion 28 recognizes the 1 st relative position 90, which is the relative position of the hand H of the occupant with respect to the 1 st vehicle 10;

the storage unit 25 stores the relative position of the operation unit 38 of the 2 nd vehicle 110 with respect to the 2 nd vehicle 110 in correspondence with the 2 nd in-vehicle device 20 of the 2 nd vehicle 110 operated using the operation unit 38;

the determination section 29 determines the 2 nd in-vehicle device (the in-vehicle device 20) of the 2 nd vehicle 110 operated using the operation section 38 of the 2 nd vehicle 110 from the storage of the storage section 25, wherein the operation section 38 of the 2 nd vehicle 110 is disposed at the 2 nd relative position 92 corresponding to the 1 st relative position 90, which is a relative position with respect to the 2 nd vehicle 110;

the control portion 30 assists an operation performed on a 1 st in-vehicle device (in-vehicle device 20) of the 1 st vehicle 10, wherein the 1 st in-vehicle device of the 1 st vehicle 10 has a function common to a 2 nd in-vehicle device.

According to the above configuration, since the operation of the hand H performed when the occupant is confused about the operation of the operation unit 38 is recognized, the in-vehicle device 20 to be controlled is specified from the relative position of the hand H, and the operation of the in-vehicle device 20 is assisted, it is possible to appropriately assist the operation of the in-vehicle device 20 when the occupant is confused about the switching operation.

In the present invention, the following may be used:

the motion recognition unit 27 recognizes the motion recognized as a state in which the occupant is confused about the operation as a predetermined motion,

the control unit 30 assists the operation of the 1 st in-vehicle device (in-vehicle device 20) when the operation recognition unit 27 recognizes the operation that is recognized as a state in which the occupant is confused about the operation.

In the present invention, the following may be used:

the storage unit 25 stores the relative position of the operation unit 38 of the 2 nd vehicle 110 with respect to the 2 nd vehicle 110 on which the occupant has been seated in the past, in association with the 2 nd in-vehicle device 20 of the 2 nd vehicle 110 operated using the operation unit 38.

The occupant may misunderstand that the 1 st vehicle 10 currently being seated is the 2 nd vehicle 110 that has been seated in the past. According to the above configuration, since the in-vehicle device 20 is specified based on the device type information 86 associated with the riding information 76 indicating that the occupant has ridden, it is possible to specify an appropriate in-vehicle device 20 and assist the operation.

In the present invention, the following may be used:

the storage section 25 stores vehicle model information 74 of a 2 nd vehicle 110 on which an occupant has been taken in the past, the vehicle model information 74 being at least one of producer information 80 indicating a producer of the 2 nd vehicle 110, producer country information 82 indicating a producer country of the 2 nd vehicle 110, and vehicle type information 84 indicating a vehicle type of the 2 nd vehicle 110, in correspondence with the occupant,

the control portion 30 assists the operation of the 1 st in-vehicle device (in-vehicle device 20) when at least a part of the vehicle model information 74 regarding the 1 st vehicle 10 does not coincide with the vehicle model information 74 regarding the 2 nd vehicle 110 on which the occupant has been seated in the past.

When at least one of the producers, producing countries, and models of the 1 st vehicle 10 and the 2 nd vehicle 110 is different, the relative position of the operation portion 38 is different. According to the above configuration, since the in-vehicle device 20 is specified based on the device type information 86 associated with the producer information 80, the country of production information 82, and the vehicle type information 84, it is possible to specify an appropriate in-vehicle device 20 and assist the operation.

In the present invention, in the case of the present invention,

the 1 st in-vehicle device (in-vehicle device 20) and the 2 nd in-vehicle device (in-vehicle device 20) are at least one of an air-conditioning device 40, a hazard lamp device 42, and a wiper device 44.

With the above configuration, the operation of the air conditioner 40, the hazard lamp device 42, and the wiper device 44 can be assisted.

Further, the present invention is an operation assisting method of assisting an operation performed on the 1 st in-vehicle device 20 of the 1 st vehicle 10

Performing a storage step of storing, using the storage unit 25, a relative position of the operation unit 38 of the 2 nd vehicle 110 with respect to the 2 nd vehicle 110 in correspondence with the 2 nd in-vehicle device 20 of the 2 nd vehicle 110 operated using the operation unit 38;

the operation recognition step (step S12), the position recognition step (step S14), the determination step (step S15, step S16), and the control step (step S17) are executed by using the arithmetic unit 24,

in the motion recognition step, recognizing a predetermined motion of a hand of a passenger riding in the 1 st vehicle 10;

in the position recognition step, a 1 st relative position 90, which is a relative position of the hand H of the occupant with respect to the 1 st vehicle 10, is recognized;

in the specifying step, the 2 nd in-vehicle device (the in-vehicle device 20) of the 2 nd vehicle 110 that is operated using the operating portion 38 of the 2 nd vehicle 110 provided at the 2 nd relative position 92 corresponding to the 1 st relative position 90, which is a relative position with respect to the 2 nd vehicle 110, is specified based on the storage in the storage unit 25;

in the control process, an operation of a 1 st in-vehicle device (in-vehicle device 20) of the 1 st vehicle 10 is assisted, wherein the 1 st in-vehicle device of the 1 st vehicle 10 has a function common to a 2 nd in-vehicle device (in-vehicle device 20).

According to the above configuration, since the operation of the hand H performed when the occupant is confused about the operation of the operation unit 38 is recognized, the in-vehicle device 20 to be controlled is specified from the relative position of the hand H, and the operation of the in-vehicle device 20 is assisted, it is possible to appropriately assist the operation of the in-vehicle device 20 when the occupant is confused about the switching operation.

The operation support system and the operation support method according to the present invention are not limited to the above embodiments, and it is needless to say that various configurations can be adopted within a range not departing from the gist of the present invention.