阅读说明：本技术 信息处理系统 (Information processing system ) 是由 韩联羿 于 2018-06-07 设计创作，主要内容包括：提供一种信息处理系统,能够构成更真实地反映用户的状态的化身。信息处理系统具备：终端装置,其获取包括用户的心率变动率的生命数据,并将获取的生命数据经由通信网络实时地发送；以及信息处理服务器,其对从终端装置发送的生命数据进行处理。信息处理服务器存储生命数据,并基于心率变动率,推断用户的状态,生成用于显示反映推断结果的化身的显示用数据,在经由通信网络接收到显示用数据的发送请求的情况下,将显示用数据向该请求的请求源发送。(Provided is an information processing system capable of configuring an avatar that more truly reflects the state of a user. The information processing system includes: a terminal device that acquires vital data including a heart rate variation rate of a user and transmits the acquired vital data in real time via a communication network; and an information processing server that processes the vital data transmitted from the terminal device. The information processing server stores the vital data, estimates the state of the user based on the heart rate fluctuation rate, generates display data for displaying an avatar reflecting the estimation result, and transmits the display data to a request source of the request when receiving a transmission request of the display data via a communication network.)

1. An information processing system, comprising:

a vital data acquisition unit that acquires vital data of a user;

a terminal device that transmits the vital data acquired by the vital data acquisition unit via a communication network; and

an information processing server that processes the vital data transmitted from the terminal device,

the information processing server has:

a storage unit that stores the vital data;

a user state estimation unit that analyzes a correlation between 1 st vital data, which is a heart rate fluctuation rate including a heart rate and a fluctuation rate of a heart rate interval, among the vital data, and 2 nd vital data different from the 1 st vital data, and estimates a state of the user based on an analysis result of the analysis;

an avatar data generation unit that generates display data for displaying an avatar reflecting the state of the user estimated by the user state estimation unit; and

a communication interface that transmits the display data to a request source of the request when receiving a transmission request of the display data via a communication network,

the information processing system is capable of displaying an avatar reflecting the user's status in the request source.

2. An information processing server is characterized by comprising:

a storage unit that stores 1 st vital data including a heart rate of a user and a heart rate fluctuation rate that is a fluctuation rate of a heart rate interval received from a terminal device via a communication network;

a user state estimation unit that analyzes a correlation between the 1 st vital data including the heart rate and the heart rate fluctuation rate and the 2 nd vital data of a different kind from the 1 st vital data, and estimates a state of the user based on an analysis result of the analysis;

an avatar data generation unit that generates display data for displaying an avatar reflecting the state of the user estimated by the user state estimation unit; and

a communication interface that transmits the display data to a request source of the request when receiving a transmission request of the display data via a communication network,

the information processing server is capable of displaying an avatar reflecting the state of the user in the request source.

3. The information processing server according to claim 2,

the avatar data generation unit generates data for displaying the avatar reflecting past vital data corresponding to the information input from the request source,

the communication interface transmits the display data of the avatar reflecting the past vital data to the request source, thereby being able to confirm the time-series change of the state of the user in the request source.

4. The information processing server according to claim 2 or 3,

the user state inference section infers a mental state of the user,

the avatar data generation unit generates display data of the avatar reflecting the mental state of the user.

5. The information processing server according to claim 2 or 3,

the user state inference section infers a health state of the user,

the avatar data generation unit generates display data of the avatar reflecting the health state of the user.

6. The information processing server according to claim 2 or 3,

the user state inference section infers an activity state of the user,

the avatar data generation unit generates display data of the avatar reflecting the activity state of the user.

7. The information processing server according to any one of claims 2 to 6,

the data for display is three-dimensional data including information related to the interior of the avatar,

and generating display data for displaying the interior of the avatar in response to a transmission request for the display data from the request source, and transmitting the display data to the request source.

8. An information processing method for processing information in an information processing server having a processor and a storage unit,

the processor has the following functions:

receiving 1 st vital data including a heart rate of a user and a variation rate of a heart rate interval, that is, a heart rate variation rate, from a terminal apparatus via a communication network;

storing the received heart rate and heart rate variability in the storage unit;

analyzing a correlation between 1 st life data including the heart rate and the heart rate variation rate and 2 nd life data of a different kind from the 1 st life data, and inferring a state of the user based on an analysis result of the analysis;

generating display data for displaying an avatar reflecting the inferred state of the user;

transmitting the display data to a request source of the request when receiving a transmission request of the display data via a communication network,

the information processing method can display an avatar reflecting the state of the user in the request source.

9. A program for causing a computer to execute the steps of:

receiving 1 st vital data including a heart rate of a user and a variation rate of a heart rate interval, that is, a heart rate variation rate, from a terminal apparatus via a communication network;

storing the received heart rate and heart rate variability in the storage unit;

analyzing a correlation between 1 st life data including the heart rate and the heart rate variation rate and 2 nd life data of a different kind from the 1 st life data, and inferring a state of the user based on an analysis result of the analysis;

generating display data for displaying an avatar reflecting the inferred state of the user; and

when a request for transmission of the display data is received via a communication network, the program transmits the display data to a request source of the request, thereby displaying an avatar reflecting the state of the user in the request source.

Technical Field

The present invention relates to an information processing system that collects and visualizes information related to a living body.

Background

In recent years, a technique has been known in which information on a user as a living body is collected using various sensors, and the information is reflected on an avatar and displayed, thereby visualizing the information. Here, the avatar means "a character appearing on the web in place of the user" ("modern term basic knowledge 2017", free national society, page 1231).

For example, patent document 1 discloses a technique for measuring emotional data of an application having an internet access function. Specifically, the mental state of the user when talking to a rendering of a website, a video, or the like is inferred based on physiological data such as electrodermal activity (EDA), accelerometer readings, skin temperature, or facial expressions and head gestures observed by a network camera, and the mental state information is associated with the rendering. In patent document 1, the mental state information is displayed using a visual expression such as an avatar.

Patent document 2 discloses a technique of detecting electric potentials at a plurality of positions of the head of a user, or acceleration or angular velocity of the head, estimating the motion of the head and/or the expression of the face based on the detection result, and giving the estimated expression to an avatar together with the motion of the head to be displayed on a display.

Disclosure of Invention

In either of patent documents 1 and 2, the avatar is used as a means for communication by displaying the mental state and expression of the user on the avatar. However, with the recent development of telecommunication and the advent of various applications, it is expected that the methods of using avatars will be further diversified and advanced. Therefore, it is desired to construct an avatar that more truly reflects the state of the user, in addition to a simple mental state and expression such as joy and sadness.

In view of the above circumstances, it is an object of the present invention to provide an information processing system capable of configuring an avatar that more truly reflects the state of a user.

In order to solve the above problem, an information processing system according to an aspect of the present invention includes: a vital data acquisition unit that acquires vital data including at least a heart rate of a user; a terminal device that transmits the vital data acquired by the vital data acquisition unit in real time via a communication network; and an information processing server that processes the vital data transmitted from the terminal device, the information processing server including: a storage unit for storing the vital data; a user state estimation unit that estimates a state of the user in real time based on at least the heart rate and a heart rate fluctuation rate calculated from the heart rate; an avatar data generation unit that generates display data for displaying an avatar reflecting at least an estimation result by the user state estimation unit; and a communication interface that transmits the display data to the request source when receiving a transmission request of the display data via the communication network.

In the information processing system, the user state estimating unit may estimate a mental state of the user, and the avatar data generating unit may generate display data of the avatar reflecting the mental state of the user.

In the information processing system, the avatar data generation unit may generate display data for changing the expression or posture of the avatar in accordance with the mental state of the user.

In the information processing system, the avatar data generation unit may generate display data for changing a color or a display range of the avatar's halo in accordance with a mental state of the user.

In the information processing system, the user state estimating unit may estimate a health state of the user, and the avatar data generating unit may generate display data of the avatar reflecting the health state of the user.

In the information processing system, the avatar data generation unit may generate display data for locally changing the color of the avatar according to the health status of the user.

In the information processing system, the avatar data generation unit may generate display data for changing a color or a display range of the avatar's halo in accordance with the health status of the user.

In the information processing system, the user state estimating unit may estimate an activity state of the user, and the avatar data generating unit may generate display data of the avatar reflecting the activity state of the user.

In the information processing system, the avatar data generation unit may generate display data for changing the shape of the avatar in accordance with the activity state of the user.

In the information processing system, the display data may be three-dimensional data including information relating to the interior of the avatar, and the display data may be generated to display the interior of the avatar in response to a transmission request of the display data from the request source and transmitted to the request source.

In the information processing system, the vital data acquiring unit may further acquire vital data of a type different from the heart rate, the information processing server may further include a correlation analyzing unit that analyzes correlation between a plurality of types of vital data different from each other, and the user state estimating unit may further estimate the state of the user based on an analysis result by the correlation analyzing unit.

According to the present disclosure, since life data including at least the heart rate of the user is acquired, the state of the user is estimated in real time based on the life data, and display data of an avatar reflecting the estimation result is generated, an avatar more truly reflecting the state of the user can be configured.

Drawings

Fig. 1 is a system block diagram schematically showing an example of an information processing system according to an embodiment of the present invention.

Fig. 2 is a system block diagram schematically showing an example of the configuration of the user terminal shown in fig. 1.

Fig. 3 is a system block diagram schematically showing an example of the configuration of the information processing server shown in fig. 1.

Fig. 4 is a schematic diagram showing an example of information stored in the user management database stored in the storage unit shown in fig. 3.

Fig. 5 is a schematic diagram showing an example of information stored in the user information database stored in the storage unit shown in fig. 3.

Fig. 6 is a schematic diagram showing an example of information stored in the correlation information database stored in the storage unit shown in fig. 3.

Fig. 7 is a flowchart showing a process of constructing the correlation information database executed by the correlation analysis unit shown in fig. 3.

Fig. 8 is a sequence diagram of information collection processing executed in the information processing system shown in fig. 1.

Fig. 9 is a sequence diagram of display processing of an avatar executed in the information processing system shown in fig. 1.

FIG. 10 is a diagram showing a display example of an avatar.

FIG. 11 is a schematic diagram showing another display example of an avatar.

Fig. 12 is a diagram for explaining an application example of the SNS in the information processing system shown in fig. 1.

Fig. 13 is a diagram for explaining an application example of the SNS in the information processing system shown in fig. 1.

Detailed Description

Hereinafter, embodiments of the present invention will be described in detail. Furthermore, the following examples are examples for explaining the present invention, and are not intended to limit the present invention only to these embodiments. The present invention can be variously modified without departing from the gist thereof. Further, those skilled in the art may adopt an embodiment in which elements described below are replaced with equivalent elements, and these embodiments are also included in the scope of the present invention.

(1) Structure of the embodiment

Fig. 1 is a system block diagram schematically showing an example of an information processing system according to an embodiment of the present invention. As shown in fig. 1, the information processing system 1 includes: a vital data collecting unit 10 for collecting vital data, which is the biological information of a user, a user terminal 20, and an information processing server 30. The user terminal 20 and the information processing server 30 are connected via a communication network N (but not limited thereto).

The network N is a communication network including the internet, LAN, private line, telephone line, enterprise network, mobile communication network, Bluetooth (registered trademark), wifi (wireless fidelity), other communication line, or a combination thereof, and may be wired or wireless.

The vital data collecting unit 10 includes a plurality of devices that are attached to or provided around the body of the user, monitor the body of the user, and collect vital data. Specifically, the vital data collecting unit 10 includes a pulse meter 12, a blood pressure meter 13, a thermometer 14, a network camera 15 that captures the movement of the face and body of the user, a surface myoelectric potential sensor 16 that measures the movement of the muscle of the user, and the like, in addition to a heart rate meter 11 that measures the heart rate of the user. Each apparatus may be provided with 1 apparatus, or may be provided with a plurality of apparatuses. For example, by attaching the plurality of pulse meters 12 to a plurality of positions of the body of the user, the measurement accuracy can be improved. Further, a microphone, a pedometer, or the like that collects the voice of the user may also be provided as the vital data collecting unit 10.

Fig. 2 is a system block diagram schematically showing an example of the configuration of the user terminal 20 of the information processing system according to the embodiment of the present invention. The user terminal 20 can be any terminal device capable of exchanging data with another communication apparatus via a communication network, such as a tablet terminal, a Personal Computer (PC), a notebook PC, a smartphone, a mobile phone, and a Personal Digital Assistant (PDA). In the present embodiment, a dedicated application is installed in the tablet terminal and the application is executed, whereby the tablet terminal can be used as the user terminal 20.

The user terminal 20 includes a communication interface 21, an input unit 22, a display unit 23, an imaging unit 24, a signal input/output unit 25, a storage unit 26, and a processor 27.

The communication interface 21 is a hardware module for connecting the user terminal 20 to the communication network N and communicating with other terminals on the communication network N. The communication interface 21 is a modem device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, and a soft modem.

The input unit 22 is an input device such as various operation buttons and a touch panel. The display unit 23 is, for example, a liquid crystal display or an organic EL display. The imaging unit 24 is a camera built in the tablet terminal.

The signal input/output unit 25 is an interface for connecting an external device to the user terminal 20 by wireless communication based on standards such as wired (cable) or Bluetooth (registered trademark), and transmitting and receiving signals to and from the external device. In the present embodiment, each device included in the vital data collecting unit 10 is connected to the user terminal 20 via the signal input/output unit 25.

The storage unit 26 is a logical device provided in a storage area of the physical device, and stores an operating system program, a driver program, various data, and the like used for processing of the user terminal 20. Here, the physical device is a recording medium that can be read by a computer, such as a semiconductor memory. Examples of the driver include a communication interface driver for controlling the communication interface 21, an input device driver for controlling the input unit 22, a display device driver for controlling the display unit 23, an imaging device driver for controlling the imaging unit 24, and various drivers for controlling external devices connected to the signal input/output unit 25.

In addition to the various programs and data described above, the storage unit 26 stores a dedicated application 261 that is executed by the processor 27 to execute a predetermined operation in cooperation with the information processing server 30. Examples of the application 261 include an application (vital information processing application) for processing vital data collected by the vital data collection unit 10, an application (SNS application) for an SNS (social network service), and an application (health management application) for managing the health of a user.

The processor 27 is constituted by an arithmetic logic operation unit (CPU, etc.) that handles arithmetic operations, logical operations, bit operations, etc., and various registers, and centrally controls each part of the user terminal 20 by executing various programs stored in the storage section 26. Such as program counters, data registers, command registers, general purpose registers, etc. The processor 27 reads the application 261 and functions as an application execution unit 271 for vital information processing, SNS, health management, and the like.

Such a user terminal 20 preferably receives various vital data outputted from the vital data collecting unit 10 and transmits the data to the information processing server 30 via the communication network N all the time and in real time.

In the present embodiment, the vital data collecting means 10 is connected to the user terminal 20, and the vital data is transmitted to the information processing server 30 via the user terminal 20. However, the vital data collection units 10 may be provided with communication functions, and the identification codes (IDs) of the vital data collection units 10 may be registered in advance in the information processing server 30, and the vital data may be directly transmitted from the vital data collection units 10 to the information processing server 30.

In fig. 1, each figure shows one vital data collection unit 10 and one user terminal 20, but the present invention is not limited to this. That is, it is also possible to connect 2 or more user terminals 20 to which the vital data collecting means 10 are connected, respectively, to the communication network N, and to simultaneously access the information processing server 30 from each user terminal 20.

Fig. 3 is a system block diagram schematically showing an example of the configuration of an information processing server of an information processing system according to an embodiment of the present invention. The information processing server 30 is a server device that accumulates vital data transmitted from the user terminal 20 (or the vital data collection unit 10), estimates the state of the user in real time based on the accumulated vital data, visualizes the state of the user in response to a request from the user terminal 20, and provides the user terminal 20 with the user state. The information processing server 30 is configured by, for example, a host computer having high arithmetic processing capability, and functions as a server by operating a predetermined server program in the host computer. Note that the computer constituting the information processing server 30 does not necessarily need to be 1 computer, and may be configured by a plurality of computers distributed over the communication network N.

The information processing server 30 includes a communication interface 31, a storage unit 32, and a processor 33.

The communication interface 31 is a hardware module connected to the communication network N and used to communicate with other terminals on the communication network N. Specifically, the communication interface 31 is a modem device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

The storage unit 32 is a logical device provided in a storage area of a physical device constituted by a computer-readable recording medium such as a magnetic disk drive or a semiconductor memory (ROM, RAM, etc.). The storage unit 32 may be constructed by mapping a plurality of physical devices to one logical device, or may be constructed by mapping one physical device to a plurality of logical devices. The storage unit 32 stores various programs including an operating system program and a driver program, and various data used for executing these programs. Specifically, the storage unit 32 stores an information processing program 321, a user management database 322, a user information database 323, and a correlation information database 324, which are executed by the processor 33.

The information processing program 321 is a program executed by the processor 33 to realize a function of accumulating life data of the user, visualizing and providing a state (mental state, health state, activity state, etc.) of the user based on the accumulated life data.

FIG. 4 is a diagram illustrating information stored in the user management database 322. The user management database 322 stores account information of the user including a user ID, a user name, a password, and the like, and information for managing access restrictions. The access restriction is used to restrict the range of information disclosed to another user when another user requests viewing of information related to the user. The access restriction can be set by the user in a hierarchical manner in a range from "disclosure in full (no access restriction)" to "disclosure not disclosed except for the user" according to the relationship between the user and another user.

Fig. 5 is a diagram illustrating information stored in the user information database 323. The user information database 323 stores, for each user ID, user basic information D1 such as the year, month, day, height, weight, and blood type of the user, life data D2, and user state information D3 indicating the state of the user estimated based on the life data D2.

The vital data D2 includes primary data directly acquired by the vital data collecting unit 10 and secondary data acquired from the primary data. The raw data includes heart rate, pulse rate, blood pressure, body temperature, movement of face, scalp, or body muscles, movement of eyeball or pupil, and voice. In addition, the secondary data includes: a heart rate fluctuation rate calculated from a heart rate, an expression of a face or a posture of a body calculated from a movement of a muscle of a face or a scalp or a body, a movement of a diaphragm or a stretch of a spine calculated from a movement of a muscle of an abdomen or a back, a fluctuation rate of a movement of an eyeball calculated from a movement of an eyeball, a change in tone (size, height, speed, etc.) of a sound, and the like. These secondary data may be calculated by the user terminal 20 and transmitted to the information processing server 30, or may be calculated by the information processing server 30.

Further, since the vital data D2 is transmitted from the user terminal 20 in real time and accumulated in the information processing server 30, the information processing server 30 may sequentially delete the vital data D2 after a predetermined period (for example, several years) has elapsed since the reception. In this case, the user status information acquired from the vital data to be deleted may be stored.

The user state information D3 includes information indicating the mental states such as feelings (joy, anger, sadness), pressure levels, health states such as health levels and uncomfortable parts, and the active states such as "asleep", "awake", "eating" and "exercising". These pieces of information may be expressed by digitizing the levels, by text (or symbol) information, or by combining text (or symbol) information with numeric values.

FIG. 6 is a diagram illustrating information stored in the correlation information database 324. The correlation information database 324 stores information (correlation information) that correlates the vital data with the state (mental state, health state, activity state) of the user. As an example of the correlation information, a table showing a relationship between a plurality of different types of vital data (two types of data a and B in fig. 6) and the state of the user at that time is listed. Fig. 6 shows that when the level of a certain vital data (data a) is "5" and the level of the other vital data (data B) is "4", the state of the user is "X1".

Here, the heart rate, which is one item of life data, varies depending on physical conditions (for example, at normal body temperature or during fever), mental states (for example, when the mood is calm, or when there is tension or excitement), active states (for example, when there is calm or exercise), and the like. On the other hand, a state in which the heart rate interval fluctuates to some extent is normal, and if there is stress in the body and mind or the autonomic nerve function is reduced, the fluctuation of the heart rate interval becomes small. In addition, in the traditional Chinese medicine, mental states (emotional and stress levels) and health states (function levels of organs and the like) are also determined based on the heart rate and the heart rate fluctuation rate.

Therefore, by measuring the heart rate and the rate of change of the heart rate interval (heart rate fluctuation rate), the mental state, the health state, and the activity state of the user can be inferred to some extent. Further, by using other vital data of the user (blood pressure, body temperature, movement of the eyes of the user captured by the camera, movement of muscles of the face (expression), change in tone of voice, movement of muscles of the body (movement), movement of the diaphragm, stretching of the spine, and the like) in combination, items that can be inferred with respect to the state of the user can be added, and the accuracy of the inference can be improved. For example, in addition to mental information and health information, an activity state such as sleep, awake, or the like can be inferred.

Here, the information used when estimating the state of the user need not be only the life data, and the state of the user (estimation result) estimated based on the life data may be used. That is, the state of another type of user may be estimated based on the estimation result based on the life data and the life data. For example, from the pressure level of the user and the movement of the muscles of the face, which are inferred based on the heart rate and/or the heart rate fluctuation rate, the mental state of the user can be inferred in more detail.

The relevance information database 324 stores one or more relevance information that is used in inferring the status of the user as such. The correlation information does not necessarily have to be in the form of a table, and a function having a plurality of types of vital data as variables, and a function having an estimation result based on the vital data and the vital data as variables may be stored as the correlation information.

The correlation information stored in the correlation information database 324 may be generated in advance based on external information, or may be generated based on vital data accumulated in the information processing server 30. Furthermore, the correlation information generated in advance based on the external information may be updated based on the vital data accumulated in the information processing server 30.

The processor 33 is constituted by an arithmetic logic operation unit (CPU, etc.) that processes arithmetic operations, logical operations, bit operations, and the like, and various registers, and centrally controls each part of the information processing server 30 by executing various programs stored in the storage unit 32. Examples of the various registers are a program counter, a data register, a command register, a general purpose register, etc. The processor 33 executes the information processing program 321 to realize a predetermined information processing function in cooperation with the user terminal 20.

The functional units realized by the processor 33 executing the information processing program 321 include an authentication management unit 331, a user information management unit 332, a user state estimation unit 333, an avatar data generation unit 334, and a correlation analysis unit 335.

The authentication management unit 331 performs authentication when the user terminal 20 accesses the information processing server 30. Specifically, when the user terminal 20 requests access, the authentication management unit 331 requests the user terminal 20 to input a user ID and a password, and refers to the user management database 322 to perform authentication as to whether or not the access of the user terminal 20 is permitted.

The user information management unit 332 manages the user information database 323 based on the information transmitted from the user terminal 20.

The user state estimation unit 333 estimates the state of the user based on the vital data accumulated in the vital data D2 and the correlation information database 324.

The avatar data generator 334 generates an avatar, which is a character displayed in the internet space, as the user's own body, and generates display data (hereinafter referred to as avatar data) for displaying the user's life data and the estimation result (the state of the user) by the user state estimator 333 in the avatar. The type of the life data and the state of the user reflected on the avatar, and the display method of the avatar are not particularly limited. Examples of the display of the avatar will be described later.

Here, since the vital data is transmitted from the user terminal 20 in real time and the estimated state of the user changes constantly, it is preferable to display the avatar as an animation. The avatar data generator 334 may generate three-dimensional data including information indicating the inside of the avatar as avatar data, and may construct display data for observing the state of the avatar from the inside (e.g., inside the digestive tract) and display data for a cross-section at a time in response to a request from the user terminal 20.

The correlation analysis unit 335 analyzes the correlation between the vital data (input data) transmitted from the user terminal 20 and the correlation between the vital data (input data) and the estimation result (output data) by the user state estimation unit 333, thereby constructing a database of correlation information in which the vital data and the state of the user are correlated with each other.

Fig. 7 is a flowchart showing a process of constructing the correlation information database executed by the correlation analysis unit 335.

First, in step S10, the correlation analysis unit 335 acquires the 1 st information and one or more types of 2 nd information associated with the state of the user in advance. Here, since it is known that the heart rate variability has a correlation with the stress level of the user as described above, the heart rate variability and the stress level can be associated in advance, and the heart rate variability can be used as the 1 st information in step S10. In addition, as the 2 nd information, data other than the heart rate fluctuation rate, such as the movement of a specific part such as an eyeball, a change in tone of voice, the way of raising a diaphragm, and the extension of a spine, is acquired. The 2 nd information may be two or more different kinds of information.

In the next step S11, the correlation analysis unit 335 analyzes the correlation between the 1 st information and the 2 nd information. In the above examples, the correlation between the heart rate fluctuation rate and the movement of the eyeball, the correlation between the heart rate fluctuation rate and the change in tone of the voice, and the correlation between the heart rate fluctuation rate and the movement of the diaphragm were analyzed.

In the next step S12, the correlation analysis unit 335 determines whether or not the correlation between the 1 st information and the 2 nd information is strong. For example, the correlation is determined to be strong when the correlation coefficient between the two is equal to or greater than a predetermined value, and is determined to be weak when the correlation coefficient is less than the predetermined value.

When the correlation between the 1 st information and the 2 nd information is weak (yes in step S12), the correlation analysis unit 335 ends the process.

On the other hand, when the correlation between the 1 st information and the 2 nd information is strong (YES in step S12), the correlation analysis section 335 associates the 2 nd information with the state of the user associated with the 1 st information in advance based on the analysis result of the correlation (step S13). Specifically, a table is generated in which the 2 nd information is associated with the state of the user. Alternatively, a function may be generated in which the 2 nd information is used as an input variable and the state of the user is used as an output value. This enables the state of the user to be directly estimated based on the 2 nd information. In the above example, the stress level of the user can be estimated from the data of the movement of the eyeball of the user without the heart rate fluctuation rate. Alternatively, the joy, anger, sadness and sadness of the user can be inferred from the change of the tone of the voice. In addition, the user's tension can be inferred from the convex pattern of the diaphragm, the extension of the spine. The correlation analysis unit 335 accumulates the correlation information between the 2 nd information and the user state acquired in this manner in the correlation information database 324 (see fig. 3). Thereafter, the correlation analysis unit 335 ends the process.

As the 1 st information, information other than the heart rate fluctuation rate may be used as long as it is information associated with the state of the user. For example, if the movement of the eyeball and the pressure level are temporarily associated in step S13, the movement of the eyeball may be used as the new 1 st information in step S10. In this case, by analyzing the correlation with the new 1 st information with other vital data as the 2 nd information in step S11, the pressure level by the movement of the eyeball can be associated with the other vital data.

The object of analyzing the correlation is not limited to the correlation between the vital data, and may be the correlation between data arbitrarily input by the user and the vital data, the correlation between data estimated from the vital data and the vital data, or the like. Specifically, the user inputs data such as the birth year, month, day, birth time, location, blood type, DNA type, divinatory (for example, eight-character learning) result, and his/her own evaluation of vital data, and correlates these input data with vital data (such as heart rate fluctuation rate).

By accumulating the correlations between the life data and the data other than the life data, it is possible to estimate a wider variety of items with respect to the state (mental state, health state, activity state) of the user and to improve the estimation accuracy. By reflecting the state of the user estimated in this way on the avatar, it is possible to display the avatar closer to the current state of the user. Further, by accumulating the analysis results of such correlations, there is a possibility that a disease or the like which the user himself does not perceive can be estimated.

Further, by accumulating the analysis results of a large number (for example, several hundreds to several tens of thousands of people) of users for a certain period (for example, 1 year time to several years), it is possible to grasp a general trend relating to the state of the user. For example, it is also possible to extract a tendency that a user who is born in B month from the a region is likely to get X disease.

The analysis result by the correlation analysis unit 335 is accumulated in the correlation information database 324. The correlation information accumulated by the analysis by the correlation analysis unit 335 may be used only for estimating the state of the user used for the analysis. Alternatively, the generalized relevance information may be used in order to infer the status of other users.

(2) Operation of the embodiment

Fig. 8 is a sequence diagram of the information collection process executed in the information processing system 1 according to the embodiment of the present invention.

When the user terminal 20 requests access to the information processing server 30 (step S101), the information processing server 30 requests the user terminal 20 for a user ID and a password, or new user registration (step S201).

If the user terminal 20 transmits user registration information for performing new user registration (step S102), the information processing server 30 issues a user ID and a password, and regenerates user information (step S202).

When the user ID and the password are transmitted from the user terminal 20 (step S103) and authentication is successful in the information processing server 30 (step S203), the user terminal 20 is in the login state, and the vital data can be accumulated in the information processing server 30 from the user terminal 20.

When the vital data collected by the vital data collection unit 10 (see fig. 1) is transmitted from the user terminal 20 (step S104), the information processing server 30 receives the vital data and stores the same in the user information database 323 (step S204). Next, the information processing server 30 infers the state (mental state, health state, activity state) of the user based on the accumulated vital data (step S205), and generates avatar data for displaying an avatar reflecting the state of the user (step S206).

Fig. 9 is a sequence diagram of the display processing of the avatar executed in the information processing system 1 according to the embodiment of the present invention.

When the user terminal 20 requests access to the information processing server 30 (step S111), the information processing server 30 requests the user terminal 20 for a user ID and a password (step S211).

When the user ID and the password are transmitted from the user terminal 20 (step S112) and the authentication is successful in the information processing server 30 (step S212), the user terminal 20 enters the login state. When the login state of the user terminal 20 is maintained, steps S112, S211, and S212 are omitted.

When the user terminal 20 requests avatar data of a specific user from the information processing server 30 (step S113), the information processing server 30 refers to the user management database 322 and confirms access restriction of the user who has requested the avatar data (step S213). Also, the requested avatar data is transmitted to the user terminal 20 within the range of the access limit (step S214). For example, in the case where the access restriction is set to "not disclosed except for the principal", the information processing server 30 does not transmit avatar data to a user other than the user of the avatar data.

The user terminal 20 displays an avatar on the screen based on the received avatar data (step S114).

FIG. 10 is a schematic diagram showing a display example of an avatar, showing avatar A1 simulating the whole body of a human. Alternatively, for example, the model a11 of the heart may be displayed superimposed on the avatar a1, so that the model a11 of the heart beats in accordance with the heart rate of the user. Further, the color of the avatar a1 may be changed as a whole in accordance with the body temperature of the user. Alternatively, the estimated mental state of the user (for example, joy, anger, sadness, or pressure level) may be reflected on the expression or color (face color) of the face a12 of the avatar a 1. The estimated health status (for example, a level of health to discomfort, or a pressure level) of the user may be reflected on the halo (backlight) a13 of the avatar a 1. For example, the display range of the halo a13 may be expanded as the health is improved, or the color of the halo a13 may be changed according to the pressure level. In addition, the color of the part of the avatar a1 corresponding to the uncomfortable position of the user's body may be changed according to the degree of discomfort. As an example, in the case where the shoulder stiffness of the user is severe, the brightness of the portion a14 of the shoulder of the avatar a1 is lowered, showing poor blood circulation. In addition, the shape of the avatar a1 may be changed in accordance with the activity state of the user. As an example, the posture of the avatar a1 is changed according to the movement of the muscle of the user acquired by the surface electromyography sensor 16 (see fig. 1).

The avatar data generator 334 may change the form of the avatar a1 in response to a request transmitted from the user terminal 20. For example, the display unit 23 of the user terminal 20 is caused to display the slider a15 that can be moved by the operation of the input unit 22, and when the slider a15 is moved, information indicating the position of the slider a15 is transmitted to the information processing server 30. The avatar data generator 334 generates avatar data reflecting past vital data based on the position of the slider a15, and transmits the avatar data to the user terminal 20. Thus, the avatar a1 reflecting the vital data of the user in the desired period is displayed on the user terminal 20. In this way, the avatar a1 in the past can also be displayed, whereby the user can confirm the time-series change in the health status or the like.

Alternatively, by performing a predetermined operation (for example, a click operation) on the avatar a1 displayed on the display unit 23 of the user terminal 20, information indicating that the operated region is selected is transmitted to the information processing server 30. The avatar data generator 334 generates avatar data indicating the inside (for example, an organ) of the selected region, and transmits the avatar data to the user terminal 20. Thereby, the avatar a1 with the internal area desired by the user exposed is displayed on the user terminal 20. The display method of the internal area may be a method of displaying the cross section of the avatar a1, or a method of inserting a small camera into the avatar a1 and displaying the video captured by the camera.

FIG. 11 is a schematic diagram representing other display examples of avatars, showing avatar A2 simulating the user's head. In this avatar a2, an area (emotional area a21) indicating the emotion of the user, an area (right brain area a22) indicating the activity state of the right brain, and an area (left brain area a23) indicating the activity state of the left brain are provided, and the size, color, and the like of each area are changed based on the vital data and the estimated state of the user.

As described above, according to the present embodiment, since life data including at least the heart rate of the user is acquired in real time and avatar data is generated based on the state of the user estimated in real time based on the life data, an avatar that reflects the state of the user can be configured.

Further, according to the present embodiment, since the correlation between a plurality of life data and the correlation between the estimation result of the state of the user and the life data are analyzed and the state of the user is further estimated based on the correlation, it is possible to increase items that can be estimated for the state of the user and to improve the estimation accuracy.

The information processing system 1 that displays an avatar based on the vital data of the user can be used in various applications. As an example, by combining the information processing system 1 with an SNS (social network system) such as "facebook (registered trademark)", "LinkedIn (registered trademark)", it is possible to use an avatar as a user profile.

Fig. 12 and 13 are diagrams for explaining an application example of the SNS in the information processing system 1. For example, as shown in fig. 12, in a certain SNS, a user a is associated with a user B, C, D as a "friend". In addition, user B is associated with user E, F as a "friend" in addition to user A. The mental state, health state, activity state, or halo of each avatar of these users are quantified, and statistics are calculated based on the user's dating relationships. For example, the halo values for digitizing the halos of the avatars of the users a to F are 2, 3, 8, 5, 6, and 4, respectively. In this case, as shown in fig. 13, the sum of the light ring values of the friend of the user a (the user B, C, D) is 3+8+5 — 16, and the average value is about 5.3. On the other hand, the sum of the light ring values of the friend of the user B (the user A, E, F) is 2+6+4 — 12, and the average value is about 4. Therefore, it is known that the user a has more friends than the user B with a higher halo value.

Further, by changing the value used for the analysis, for example, the user a can perform the analysis of many unhappy friends and many unhealthy friends. Alternatively, a comparison between the working network of user a and the private network can also be made. As the numerical value used for the analysis, in addition to numerical values such as a halo, a mental state, a health state, and an activity state, the vital data itself (heart rate and the like) may be used. In the above description, the average value is used as the statistical value, and the median value, the mode value, and the like may be used.

In addition to the friend-making relationship related to the network as illustrated in fig. 12, the above-described light ring value, vital data, and the like may be acquired for a person who is interested in a published message or a person who is evaluated, and may be displayed in the form of an integral or ranking. This makes it possible to analyze the trend of the user a, such as a high user a being high in the stress level of the user a, a high user B being high in the health level of the user B, but low in the activity level.

As another example, the information processing system 1 may be used in a recruitment site or a fellow site. In this case, the avatar may be presented to the business or the fellow person as the material of the member (job applicant, fellow person applicant).

As another example, the information processing system 1 may be used in a game site. For example, avatars may be confronted with each other in a game of competition.

As still another example, the information processing system 1 may be used in a health management application. In this case, even when the user is at a remote location, the index indicating the health state of the user, such as pulse or body temperature, can be acquired from the avatar displayed on the user terminal 20. In addition, the organ may be displayed with the abdomen of the avatar open, or the digestive tract of the avatar may be displayed from the inside. Further, there is a use method in which the user observes the posture of the whole body of the avatar to correct the posture of the user.

In the above-described embodiment and modification, a person is used as the user of the information processing system 1, but animals such as pets and livestock may be used as the user. That is, a life data collecting unit is installed in a dog, a cat, or the like, and an avatar of an animal is generated based on the collected life data. In this case, the veterinarian can also observe the avatar for medical attention.

The embodiment described above is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art may substitute the resources (hardware resources or software resources) described in the embodiments with equivalents, and such a substitution is also included in the scope of the present invention.

Description of the reference numerals

An information processing system; a vital data collection unit; a cardiotachometer; a sphygmomanometer; a thermometer; a web camera; a surface electromyography sensor; a user terminal; a communication interface; an input portion; a display portion; a photographing part; a signal input output portion; a storage portion; a processor; an information processing server; a communication interface; a storage portion; a processor; an application program; an application program execution section; an information processing program; a user management database; a user information database; a database of relevance information; 331.. an authentication management part; a user information management section; a user state inference section; an avatar data generating section; a correlation analysis section.