阅读说明：本技术 远程呈现系统 (Telepresence system ) 是由 R·图利 于 2017-11-15 设计创作，主要内容包括：用于远程呈现的系统提供有远程用户位置处的用户站和位于本地对象位置处的远程呈现装置。感测本地对象周围的物体的距离,使得保持远程呈现装置与本地对象之间的选定距离。感测本地对象与远程用户的眼睛特征和注视。自动调整远程呈现装置的显示器的高度,使得本地对象的视线水平与显示器中用户图像的视线水平匹配。当感测到预期的目光接触时,调整远程呈现装置、其显示器和容纳摄像机和其他传感器的传感器中心的位置,并且可以以不同的视角重新创建面部图像以类似于本地对象与远程用户之间的目光接触,同时保留所有其他面部表情。(A system for telepresence is provided with a user station at a remote user location and a telepresence apparatus at a local object location. The distance of objects around the local object is sensed such that a selected distance between the telepresence apparatus and the local object is maintained. Eye features and gaze of the local object and the remote user are sensed. The height of the display of the telepresence apparatus is automatically adjusted so that the level of gaze of the local object matches the level of gaze of the user's image in the display. When the intended eye contact is sensed, the location of the telepresence apparatus, its display, and the center of the sensor that houses the camera and other sensors is adjusted, and the facial image may be recreated at a different perspective to resemble the eye contact between the local object and the remote user, while retaining all other facial expressions.)

1. A telepresence system, comprising telepresence apparatus and user stations, for a video conference between a remote user and a local object, wherein the telepresence apparatus is located at the location of the local object, comprising:

A display to display the remote user's face to the local object;

at least one sensor for sensing a gaze level of a local object; and

motorized means for a telepresence apparatus for automatically adjusting the height of its display to match the remote user's gaze level in the display to the gaze level of the local object.

2. The telepresence system of claim 1, configured to match a gaze level of a remote user in a display of a telepresence apparatus to a gaze level of a local object while the gaze level of the local object changes.

3. The telepresence system of claim 1, wherein in response to physical movement of the local object, a display of the telepresence apparatus is automatically panned, tilted, and rotated such that the display faces the local object.

4. The telepresence system of claim 1, wherein the motorized device coupled to a display of the telepresence apparatus is adapted to cover a wide range of heights from a seated short child to a standing tall child.

5. The telepresence system of claim 1, wherein the motorized device coupled to a display of the telepresence apparatus is electronically controlled such that a height and a position of the display are automatically adjusted in response to the eye position of the subject.

6. The telepresence system of claim 1, wherein the sensor is coupled to a motorized directional control device that is electronically controlled such that a position of the sensor is responsive to an eye position of the local object or a physical movement of the local object.

7. The telepresence system of claim 1, wherein the user station is a location of the remote user, comprising a plurality of sensors to assist in 3D image mapping, eye tracking, and gaze recognition of the remote user.

8. The telepresence system of claim 1, wherein the telepresence apparatus further comprises:

At least one sensor for acquiring geographic information of objects surrounding the local object or surrounding the telepresence apparatus; and

A computing unit to control a speed, displacement, and direction of movement of a motorized device disposed at a bottom of the telepresence apparatus such that a selected distance is automatically maintained between the telepresence apparatus and the local object based on the geographic information.

9. The telepresence system of claim 8, configured to maintain a selected distance between the telepresence apparatus and the local object while the location of the local object changes.

10. The telepresence system of claim 8, further comprising:

A method of avoiding potential obstacles when a remote user commands the telepresence apparatus to maneuver around the locality of a local object.

11. The telepresence system of claim 8, wherein motorized devices disposed at a bottom of the telepresence apparatus are electronically controlled such that the telepresence apparatus responds to a command from a remote user or a location of the local object.

12. The telepresence system of claim 8, wherein the motorized device disposed at a bottom of the telepresence apparatus is manually operable such that the telepresence apparatus is manually moved by a person to a desired location at the location of the object.

13. The telepresence system of claim 8, wherein the sensor is coupled to a motorized directional control device that is electronically controlled such that a position of the sensor is responsive to an eye position of the local object or a physical movement of the local object.

14. The telepresence system of claim 1, wherein the telepresence apparatus further comprises: at least one sensor for sensing eye movement;

At least one sensor for sensing audio gain, gaze location, human gestures, or a combination thereof;

A method of processing information from the sensor to identify a point of gaze;

A method of processing information from the sensor to identify the presence of an expected eye contact between a remote user and a local object; and

a computing unit to execute a gaze matching system upon detecting an expected eye contact between a remote user and a local object based on information from the sensor.

15. The telepresence system of claim 14, wherein the sensor is coupled to a motorized directional control device that is electronically controlled such that a position of the sensor is responsive to an eye position of the local object or a physical movement of the local object.

16. the telepresence system of claim 14, further comprising:

A method of providing a facial image substantially centered in a display while preserving gaze intent and all other facial expressions when there is no intent to indicate eye contact between a remote user and a local object.

17. The telepresence system of claim 14, further comprising:

A method for providing a facial image substantially centered in a display upon detecting an intent of eye contact between a remote user and a local object, while resembling eye contact and retaining all other facial expressions.

18. the telepresence system of claim 14, wherein the gaze matching system matches the gaze point of the remote user with the local object to resemble eye contact while preserving all other facial expressions.

19. the telepresence system of claim 14, wherein the gaze matching system executed by the computing unit when the expected eye contact between the remote user and the local object is detected comprises:

A motorized device for adjusting a position of the telepresence apparatus;

Motorized means for translating, tilting and rotating the display of the telepresence apparatus;

Motorized means for translating, tilting and rotating the sensor in the remote presentation means;

Methods of translating off-center images vertically and horizontally; and

A method of recreating a face image substantially centered in a display.

20. The gaze matching system of claim 19, wherein all motorized devices are electronically controlled based on information from the sensors.

Technical Field

The present invention relates generally to video conferencing, and more particularly to telepresence systems having particular features, such as automatically matching the elevation of a display of a telepresence apparatus to a local object, maintaining a selected distance from the local object, identifying a gaze direction, and matching eye contact between a remote user and the local object.

background

Due to globalization of business organizations and decentralization of labor, the demand for video conference services and devices is rapidly expanding in multinational corporate enterprises. In recent years, video conferencing has been widely adopted by public and healthcare departments as an efficient form of communication between remote users. However, the information conveyed through a video conference is not as effective as in a face-to-face conversation, particularly in the sense that subtle non-verbal communication cues such as eye contact, eye gaze, and interpersonal distance tend to be overlooked or impossible to reproduce in a video conference. In contrast, people who are engaged in a face-to-face conversation tend to stay in eye contact with others within an interpersonal distance.

In most video conferencing systems and telepresence systems, cameras or image sensors that capture images of the interacting objects are typically located above or around a monitor that displays the image of the remote user. It is obvious that even if the user wants to see the eyes of the image of the interactive object in the monitor, it will appear to look down in the viewing angle of the interactive object when the camera is placed on top of the monitor. Also, the user appears to be looking to one side when the camera is placed on the side of the monitor, or to be looking up when the camera is placed under the monitor. Due to this camera orientation problem, the user's gaze direction displayed in the subject's monitor will be different from its actual intent. However, adjustments to the image are often ignored in these systems. Therefore, these systems are not sufficient to maintain gaze direction or to maintain eye contact between users.

Telepresence robotic systems remain one of the most realistic two-way interactive video conferences available to date on the market because it overcomes physical distance while helping to create a sense of presence for remote users to enable social communications. Improvements have been made to improve the interaction between users, in particular to allow eye contact between a user and an interactive object via a telepresence apparatus. The proposed method in these improvements includes manipulating the head of the telepresence apparatus to face the direction of the user's voice and using a half-silvered mirror or beam splitter panel that allows the camera to be mounted directly behind to provide apparent eye contact between the user's image and the interactive object. Another teaching discloses a telepresence system that relies on a projected life size image of the proxy location to maintain the user's perspective and vertical gaze. Another teaches using 3D re-creation techniques to reposition the head and eye positions of a user's image in a telepresence apparatus display.

however, these telepresence systems have their limitations in ease of use, affordability, and overall user experience. For example, extensive computational and network resources are required to render and transmit high-resolution real-size background images, the user's intended gaze direction is not really preserved, and there are inherent problems with beam splitters, such as reduced image quality of ambient light and bulkiness of settings, limiting the user's mobility to the specific physical location where the beam splitter is pre-installed. A

Recently, telepresence robotics (E-Travel) systems have been developed that replicate the height of the remote user in the proxy robot and preserve the perspective and vertical gaze of the remote user. In another approach, a telepresence apparatus equipped with haptic feedback is configured to move in response to a force applied by a local object. However, none of these systems have been shown to be able to automatically match the height of the robot to the local object, or to continuously maintain a selected distance between the robot and the local object in response to real-time movement of the object.

Behavioral scientists have shown that having generally accepted interpersonal distance and maintaining sustained eye contact according to various situations has a significant impact on social interaction. The lack of these interactions may create a sense of disinterest or concentration, may distract the conversation, or may reduce the attention between the participants, and may make it difficult for the remote user to maintain a sense of presence in the same physical location as the interactive object. Furthermore, studies have shown that people who are participating in video conferences and who are presented with eye contact avoidance will be considered negative due to camera parallax errors. The results also show that people prefer to interact with their devices of similar height, and that they tend to be closer to the telepresence device when they are about the same height.

Therefore, there is a need in the art for a method or system for better personal contact and enhancing the presence of remote users: the expected eye gaze of both users can be specifically retained while the physical movement of the object can be responded to in real time so that the telepresence apparatus automatically maintains a selected distance from the local object and matches the height of the object.

summary of The Invention

The present invention relates to a system that improves the quality of interaction and the effectiveness of communications between one person and another person separated by a distance. Specifically, the present invention discloses a telepresence system that has the following features: automatically matching, with minimal human intervention, a gaze level of a remote user image displayed in a telepresence apparatus to a gaze level of a local object, maintaining a selected distance between the telepresence apparatus and the local object, identifying a gaze direction, and maintaining gaze contact between an interacting local object and the remote user when an intent to maintain gaze contact is detected.

The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent to one skilled in the art from a reading of the following description and drawings, and from the claims.