阅读说明：本技术 语音对话装置、输入装置以及输出装置 (Voice conversation device, input device, and output device ) 是由 岩田沙织 于 2020-02-25 设计创作，主要内容包括：语音对话装置(1)具备：判定部(13),基于识别部(10)的识别结果来判定是否具有使语音对话的动作暂停的意思,识别部(10)对用户通过使身体活动而示出的动作的意图进行识别；和停止部(14),在判定部(13)的判定结果是判定为具有使语音对话的动作暂停的意思的情况下,使语音对话的动作以规定时间的期间停止。(A voice conversation device (1) is provided with: a determination unit (13) that determines whether or not the user has the intention to pause the operation of the voice conversation on the basis of the recognition result of the recognition unit (10), the recognition unit (10) recognizing the intention of the user to move the body; and a stopping unit (14) that stops the operation of the voice conversation for a predetermined time period when the determination result of the determination unit (13) is that the operation of the voice conversation is to be suspended.)

1. A voice conversation apparatus includes:

a determination unit that determines whether or not the user has a meaning to pause the operation of the voice conversation based on a recognition result of a recognition unit that recognizes an intention of the user to move the body; and

and a stopping unit that stops the operation of the voice conversation for a predetermined time period when the determination result of the determination unit determines that the voice conversation is to be paused.

2. The voice dialog apparatus of claim 1, wherein,

the action of the voice dialog is the input of a voice,

the stop unit stops the input of the voice for a predetermined time period when it is determined by the determination unit that the input of the voice has been stopped.

3. The voice dialog apparatus of claim 1 or 2, wherein,

the action of the voice dialog is the output of a voice,

the stop unit stops the output of the voice for a predetermined time period when it is determined by the determination unit that the output of the voice has been stopped.

4. The voice conversation apparatus according to any one of claims 1 to 3,

the stopping unit may be configured to cancel the stop if a cancellation condition of the stop is satisfied while the operation of the voice conversation is suspended.

5. The voice dialog apparatus of claim 4, wherein,

the stopping unit regards that the stopping cancellation condition is satisfied when the determination result of the determination unit is that the stop in the operation of the voice conversation is canceled.

6. An input device is provided with:

a determination unit that determines whether or not the input of the voice conversation is to be suspended based on a recognition result of a recognition unit that recognizes an intention of a motion indicated by a user moving a body; and

and a stop unit that stops the input of the voice conversation for a predetermined time period when the determination result of the determination unit determines that the input of the voice conversation is to be suspended.

7. An output device is provided with:

a determination unit that determines whether or not the output of the voice conversation is to be suspended based on a recognition result of a recognition unit that recognizes an intention of a motion indicated by a user moving a body; and

and a stop unit that stops the output of the voice conversation for a predetermined time period when the determination result of the determination unit determines that the output of the voice conversation is to be suspended.

Technical Field

The present invention relates to a voice conversation apparatus, an input apparatus, and an output apparatus that perform a conversation by voice.

Background

Conventionally, a voice conversation device in which a person and a computer exchange intention through a voice conversation is known (see patent document 1 and the like).

Documents of the prior art

Patent document

Patent document 1: japanese patent laid-open publication No. 2018-189984

Disclosure of Invention

Problems to be solved by the invention

However, input and output in a voice conversation may not be stopped in the middle of the conversation once started. Therefore, when a user of a voice conversation performs the voice conversation and another task at the same time, the user may not be able to concentrate on the voice conversation and may not be able to make the voice conversation smooth.

The invention aims to provide a voice conversation device, an input device and an output device which can make voice conversation smooth.

Means for solving the problems

A voice conversation apparatus according to an embodiment includes: a determination unit that determines whether or not the user has a meaning to pause the operation of the voice conversation based on a recognition result of a recognition unit that recognizes an intention of the user to move the body; and a stopping unit configured to stop the operation of the voice conversation for a predetermined time period when it is determined that the determination result of the determination unit indicates that the operation of the voice conversation is to be suspended.

An input device according to an embodiment includes: a determination unit that determines whether or not the input of the voice conversation is to be suspended based on a recognition result of a recognition unit that recognizes an intention of a motion indicated by a user moving a body; and a stopping unit that stops the input of the voice conversation for a predetermined time period when the determination result of the determination unit determines that the input of the voice conversation is to be suspended.

An output device according to an embodiment includes: a determination unit that determines whether or not the output of the voice conversation is to be suspended based on a recognition result of a recognition unit that recognizes an intention of a motion indicated by a user moving a body; and a stopping unit that stops the output of the voice conversation for a predetermined time period when it is determined by the determination unit that the output of the voice conversation is to be suspended.

Drawings

Fig. 1 is a configuration diagram of a voice conversation apparatus according to an embodiment.

Fig. 2 is an explanatory view showing a configuration example of the camera.

Fig. 3 is a flowchart executed in a case where the voice input is suspended.

FIG. 4 is an illustration showing a gesture to pause speech input.

Fig. 5 is a flowchart executed in a case where the voice output is suspended.

FIG. 6 is an illustration showing a gesture to pause speech output.

Fig. 7 is an exemplary view showing a detection unit of another example.

Fig. 8 is a configuration diagram showing another example of the input device.

Fig. 9 is a configuration diagram showing another example of the output device.

Detailed Description

Hereinafter, an embodiment of a voice conversation apparatus, an input apparatus, and an output apparatus will be described with reference to fig. 1 to 6.

As shown in fig. 1, the voice dialogue device 1 assists driving of the vehicle by exchanging meanings with a crew member such as a driver in an dialogue. The voice dialogue device 1 performs voice dialogue with a crew member such as a driver to provide various information about driving by voice, or controls an in-vehicle device or the like to support driving of a vehicle.

The voice dialogue apparatus 1 includes a controller 2 that controls the operation of the voice dialogue apparatus 1, a sound collection unit 3 that collects sound, and a sound output unit 4 that outputs sound. The sound collecting unit 3 is constituted by a microphone, for example. The sound output unit 4 is constituted by, for example, a speaker mounted on the vehicle. The controller 2 understands the content of the speech of the other party by speech recognition based on the speech data Da inputted from the sound collector 3, and executes speech output such as speech guidance from the audio output unit 4 as a response thereto.

The controller 2 includes: a voice recognition unit 5 for performing voice recognition on the voice data Da inputted from the sound collection unit 3; a search database 6 in which search data used for determining speech recognition is stored; and a guidance output unit 7 for outputting a voice guidance corresponding to the result of the voice recognition. When the speech data Da is input from the sound collecting unit 3, the speech recognizing unit 5 compares the speech data Da with the search database 6, analyzes the speech data Da, and recognizes the input speech. After the voice recognition, the voice recognition unit 5 outputs the voice recognition result to the guidance output unit 7. The guidance output unit 7 outputs the corresponding output data Db to the audio output unit 4 based on the input speech recognition result. The voice output unit 4 outputs a voice guidance corresponding to the voice recognition result based on the input output data Db.

The voice dialogue apparatus 1 has a function of stopping a voice dialogue in the middle, a so-called action pause function. This is because: in a voice conversation during driving, for example, voice guidance may be missed or voice input may be difficult to perform smoothly whenever the driver focuses on driving.

The voice conversation device 1 includes a gesture recognition unit 10, and the gesture recognition unit 10 recognizes a gesture performed by a crew member such as a driver using a part of a body. The gesture recognition unit 10 is provided in the controller 2. The gesture recognition unit 10 receives a detection signal St from a detection unit 11 that can detect a body motion of a crew member such as a driver, and recognizes a gesture based on the detection signal St.

As shown in fig. 2, the detection section 11 includes a camera 12 that takes an image. The camera 12 is preferably disposed at a position where the driver can easily take an image, for example, on the upper surface of the dashboard of the vehicle. The camera 12 outputs captured surrounding image data St1 to the controller 2. The gesture recognition unit 10 recognizes a gesture taken by the camera 12 by performing image analysis on the image data St1 input from the camera 12, for example.

Returning to fig. 1, the voice conversation apparatus 1 includes the determination unit 13, and the determination unit 13 determines whether or not the user has the meaning of suspending the operation of the voice conversation. The determination unit 13 is provided in the controller 2. The determination unit 13 of this example determines whether or not the action of the voice conversation is to be suspended based on the recognition result of the recognition unit 15, and the recognition unit 15 recognizes the intention of the action indicated by the user moving the body. The recognition unit 15 is preferably a gesture recognition unit 10 that recognizes a body motion as a meaning of the body of the user. The determination unit 13 determines whether or not there is a gesture for pausing the operation of both the input and output of the voice conversation.

The speech dialogue device 1 includes a stop unit 14, and the stop unit 14 stops the operation of the speech dialogue based on the determination result of the determination unit 13. The stop unit 14 is provided in the controller 2. When the determination result of the determination unit 13 is that the action of the voice conversation is to be suspended, the stop unit 14 stops the action of the voice conversation for a predetermined time period. The stop unit 14 of this example can stop these operations in both input and output of the voice conversation.

While the voice conversation is suspended, the stop unit 14 cancels the stop if the cancellation condition for the stop is satisfied. In the case of voice input, the cancellation condition of the stop is preferably restart of the utterance under the stop. In the case of voice output, the cancellation condition for stopping is preferably, for example, execution of the same gesture as a gesture performed when the voice guidance is suspended.

Next, an operation of the speech dialogue apparatus 1 according to the present embodiment will be described with reference to fig. 3 to 6.

As shown in fig. 3, the voice dialogue apparatus 1 starts execution of the flowchart shown in the figure when performing an operation of system startup. The system activation includes, for example, a start instruction by voice input, an operation of a switch provided around the driver's seat, a start operation in the car navigation system, and the like.

In step 101, when detecting the utterance, the determination unit 13 starts determining whether or not the input of the voice dialogue (hereinafter, referred to as voice input) should be suspended. Further, the determination unit 13 of the present example detects utterances based on the voice data Da input from the sound collection unit 3.

In step 102, the determination unit 13 determines whether or not the gesture recognition unit 10 recognizes a gesture of suspension of the voice input. In this example, the gesture recognition unit 10 recognizes the gesture of the driver by monitoring the image data St1 of the camera 12 as the detection unit 11, for example. The determination unit 13 monitors whether or not there is a gesture of a pause of the voice input based on the recognition result of the image recognition.

As shown in fig. 4, the gestures for pausing of the preferred voice input are, for example, the actions of "waiting" and cutting off the conversation. The preferred action to cut off the session is, for example, an action to spread out the hand.

Returning to fig. 3, in step 102, when a gesture of pause of the voice input is recognized, the flow shifts to step 103. On the other hand, in step 102, when the gesture of the pause of the voice input is not recognized, the process proceeds to step 105.

When the determination unit 13 determines that there is a gesture for pausing the voice input, the pause unit 14 pauses the voice input in step 103. Therefore, when the user wants to concentrate on driving, the input of the voice conversation can be suspended in advance. At this time, the stopping unit 14 keeps the microphone as the sound collecting unit 3 activated, and suspends the voice input. In this way, speech input can be redetected.

In step 104, the stopping unit 14 cancels the suspension of the voice input when the utterance is detected while the voice input is suspended. That is, when the driver wants to restart the voice input and make a voice while the voice input is suspended, the suspension of the voice input is released. Thereby, the voice input can be restarted. Note that the utterance in this case may be any content as long as it can recognize the voice input.

In step 105, the speech dialogue apparatus 1 determines whether or not the utterance is completed, that is, whether or not the speech input is completed. When it is determined that the utterance is completed, the speech dialogue apparatus 1 completes the speech input, and when it is determined that the utterance has not yet been completed, the process returns to step 102, and the above-described process is repeated.

The voice recognition unit 5 analyzes the voice input by referring to the search database 6 based on the voice data Da of the voice input. Then, the voice recognition unit 5 outputs the analyzed voice recognition result to the guidance output unit 7. The guidance output unit 7 operates the voice output unit 4 based on the voice recognition result input from the voice recognition unit 5, and executes the voice guidance.

As shown in fig. 5, when the voice guidance output (hereinafter, referred to as voice output) is executed from the voice output unit 4, the voice dialogue apparatus 1 starts execution of the flow shown in the drawing.

In step 201, the determination unit 13 determines whether or not the gesture recognition unit 10 recognizes a gesture for suspending the voice output. The gesture recognition unit 10 recognizes the gesture of the driver by monitoring the image data St1 of the camera 12 as the detection unit 11, for example. Then, the determination unit 13 monitors whether or not there is a gesture for suspending the voice output based on the recognition result of the image recognition.

As shown in fig. 6, the gesture of the pause of the preferred speech output is, for example, an action that promotes "quiet" silence. The silencing-urging action is preferably an action in which the index finger stands at the mouth, for example.

Returning to fig. 5, in step 201, when a gesture of pause of voice output is recognized, the flow shifts to step 202. On the other hand, in step 201, when the gesture of pause of the voice output is not recognized, the process proceeds to step 205.

When the determination unit 13 determines that there is a gesture for pausing the voice output, the pause unit 14 pauses the voice guidance in step 202. Therefore, when the driver wants to drive intensively, the voice guidance can be suspended in advance.

In step 203, the determination unit 13 determines whether or not the gesture recognition unit 10 recognizes a gesture for releasing the pause of the voice output. The gesture for releasing the pause is preferably the same as the gesture for pausing the voice output. When the determination unit 13 recognizes the gesture for pause release, the process proceeds to step 204. On the other hand, when the determination unit 13 does not recognize the gesture for suspension cancellation, the process returns to step 202 to maintain the state in which the voice output is suspended.

When the determination unit 13 recognizes that there is a gesture for releasing the pause of the voice output, the stop unit 14 restarts the voice guidance in step 204. Therefore, continuation of the voice guidance stopped in the middle can be heard again.

In step 205, the voice dialogue apparatus 1 determines whether or not the voice guidance has ended, that is, whether or not the voice output has ended. When it is determined that the voice guidance is ended, the voice dialogue apparatus 1 ends the voice output, and when it is determined that the voice guidance is not ended, the process returns to step 102, and the above-described process is repeated.

According to the voice dialogue apparatus 1 of the above embodiment, the following effects can be obtained.

The determination unit 13 of the speech dialogue apparatus 1 determines whether or not the speech dialogue operation is to be suspended based on the recognition result of the gesture recognition unit 10 when the speech dialogue operation is performed. The stop unit 14 of the speech dialogue device 1 stops the operation of the speech dialogue for a predetermined time period when the determination unit 13 determines that the operation has been suspended. Therefore, when a user such as a driver cannot concentrate on a voice conversation, the voice conversation is suspended by a gesture, and then when an environment capable of concentrating on a voice conversation is available, the voice conversation can be continued. Therefore, the voice conversation can be made smooth. In addition, the driver can perform a smooth voice conversation according to the driving situation. The driver can input a voice conversation or listen to an output in accordance with his or her own pace.

The action of the voice conversation as the pause object is the input of voice. When the determination result of the determination unit 13 is that the input of the voice has been determined to have stopped, the stop unit 14 stops the input of the voice for a predetermined time period. Therefore, the voice input can be appropriately shifted to the suspended state as needed according to the intention of the user, and therefore, the voice input can be focused on other jobs.

The action of the voice conversation to be paused is voice output. The stop unit 14 stops the output of the voice for a predetermined time period when the determination result of the determination unit 13 is that the voice output has been determined to have stopped. Therefore, the voice output can be appropriately shifted to the suspended state as needed according to the intention of the user, and therefore, the voice output can be focused on other jobs.

A gesture is an action using a part of the user's body. Therefore, the operation of the voice conversation can be shifted to the paused state so as to be understandable by the physical movement of the user.

While the voice conversation is suspended, the stop unit 14 cancels the stop if the stop cancellation condition is satisfied. Therefore, even if the voice conversation is stopped by a predetermined gesture, the stopped state can be released and the original normal state can be restored.

The stop unit 14 regards that the stop release condition is satisfied when the determination result of the determination unit 13 indicates that the stop in the operation of the voice conversation is released. Therefore, the user can return the suspended voice conversation to the original state by a simple operation of physically moving the user.

The present embodiment can be modified as follows. The present embodiment and the following modifications can be combined and implemented within a range not technically contradictory to each other.

[ Speech dialogue apparatus 1]

As shown in fig. 7, the detection unit 11 may be a touch panel 32 provided on a steering wheel 31 mounted on the vehicle. The touch panel 32 is configured by various sensors such as a capacitance type sensor and a resistance film type sensor. The gesture using the touch pad 32 includes, for example, the number of times of touch operations of the touch pad 32, the timing of the touch operations, the direction in which the touch operations are performed on the surface of the touch pad 32, and a combination thereof. In this way, the voice conversation can be paused by a simple technique of operating the touch panel 32 with a finger.

As shown in fig. 8, the action pause function may be applied to, for example, the input device 35 that performs only voice input. The input device 35 includes, for example, a voice recognition unit 5, a search database 6, a gesture recognition unit 10, a determination unit 13, and a stop unit 14. The input device 35 is not limited to an input device for voice conversation, and may be used in other devices and input devices. In the case of such an input device 35, it is possible to provide the input device 35 capable of appropriately pausing the voice input by the gesture of the user.

As shown in fig. 9, the action pause function may be applied to, for example, the output device 36 that performs only voice output. The output device 36 includes, for example, the guidance output unit 7, the gesture recognition unit 10, the determination unit 13, and the stop unit 14. The output device 36 is not limited to an output device for voice conversation, and may be used as an output device for other devices or apparatuses. In the case of such an output device 36, it is possible to provide an output device 36 that can appropriately pause speech output by a gesture of a user.

[ concerning speech dialogue ]

The voice conversation is not limited to communication in which only voice is exchanged, and may include a gesture performed through a part of the body, for example.

The voice dialogue may be performed by using an existing in-vehicle device or by using a different component from the in-vehicle device.

The speech dialogue may be in the form of dialogue with a robot, for example. The robot can be applied to various designs such as an anthropomorphic robot and an animal-imitating robot.

[ meaning of action of pausing of voice conversation ]

The gesture for pausing the operation of the voice conversation may be changed to various modes such as changing the expression of the face and shaking the head left and right.

The gesture for pausing the operation of the voice conversation is not limited to the form described in the embodiments, and may be any form as long as the pause can be transmitted to the vehicle side.

The gesture for pausing the operation of the voice conversation may be a method of combining voices.

The gesture for suspending the operation of the voice conversation may be configured to be capable of freely registering and changing.

[ pause on speech dialogue ]

The predetermined time for pausing the voice conversation is not limited to a certain time, but various times according to the mode of resuming from the pause are used.

The pause of the voice conversation is not particularly limited as long as it is a method of pausing the input and output of the voice conversation, and the operation states of various devices constituting the voice conversation apparatus 1 are not particularly limited.

[ Release Condition for suspension ]

The pause release condition may be a mode in which the resume is instructed by voice, for example.

The pause release condition may be, for example, an operation of a switch provided in the vehicle or an operation of touching a sensor mounted in the vehicle.

The pause of the voice conversation may be automatically restored to the original normal state after the pause for a predetermined time. In this case, the voice conversation can be prevented from being temporarily suspended.

When the voice conversation is suspended, the user may check whether or not the original normal state can be restored by voice or the like, and may restore the original normal state when the user has permitted the restoration.

[ meaning expression with respect to passing through the user's body ]

The meaning of the user's body is not limited to a gesture in which the user draws a foot with a finger of the body, and may be, for example, an instruction by voice.

The meaning of the user's body includes, for example, a movement of the line of sight, and a mode in which the body itself does not have a movement and a part of the body moves.

[ concerning the identification part 15]

The recognition unit 15 is not limited to the gesture recognition unit 10, and may include the voice recognition unit 5, for example, when the voice is monitored as a pause condition.

Instead of the gesture recognition unit 10, the recognition unit 15 may be the voice recognition unit 5.

The recognition unit 15 is not limited to the voice recognition unit 5 and the gesture recognition unit 10, and may be a recognition unit capable of recognizing another body motion.

[ others ]

The detection unit 11 is not limited to the device or the component mounted in the vehicle, and may be a terminal such as a high-performance mobile phone. In this case, for example, gestures and voice of the user may be collected using a camera function and a microphone function provided in the terminal.

The voice conversation apparatus 1 is not limited to use in a vehicle, and may be used in other systems and devices.

The controller 2 (the voice recognition unit 5, the guidance output unit 7, the gesture recognition unit 10, the determination unit 13, and/or the stop unit 14) shown in fig. 1 constituting the voice dialogue apparatus 1 can be constructed as a computer system as follows: comprising one or more processors and non-transitory memory storing commands executable by the processors and used to implement voice dialog processing according to any of the above embodiments and other examples. Similarly, the input device 35 of fig. 8 and the output device 36 of fig. 9 can be constructed as such a computer system. Alternatively, the controller 2, the input device 35, and the output device 36 may be configured by dedicated hardware such as an Application Specific Integrated Circuit (ASIC).

The present disclosure includes the following embodiments.

(embodiment mode 1)

A computer system is provided with:

one or more processors; and

a non-transitory memory storing commands executable by the processor and used for implementing voice dialog processing,

the voice dialog processing includes:

recognizing speech for a speech dialogue;

recognizing a gesture associated with a motion of the voice conversation based on a detection signal from a detection unit that detects a user motion;

determining whether or not a gesture indicating that an action of the voice conversation is suspended has been performed after the voice conversation starts; and

when a gesture indicating that the operation of the voice conversation is suspended is performed, the operation of the voice conversation is suspended temporarily.

(embodiment mode 2)

In the computer system of embodiment 1, the act of voice dialog comprises voice recognition based on a voice input,

causing the action of the voice conversation to pause comprises: responsive to recognition of the 1 st gesture, speech recognition based on the speech input is suspended.

(embodiment mode 3)

In the computer system according to embodiment 2, the voice dialogue processing further includes: pausing speech recognition based on the speech input is deactivated in response to a user utterance.

(embodiment mode 4)

The computer system according to any one of embodiments 1 to 3, wherein the operation of the voice conversation includes voice output based on a result of voice recognition,

causing the action of the voice conversation to pause comprises: in response to the recognition of the 2 nd gesture, the voice output based on the voice recognition result is suspended.

(embodiment 5)

In the computer system of embodiment 4, the voice dialog processing further comprises: in response to the recognition of the 3 rd gesture, the suspension of the voice output based on the voice recognition result is released.

(embodiment mode 6)

In the computer system according to embodiment 5, the 3 rd gesture is the same as the 2 nd gesture.