阅读说明：本技术 事务存储器同步 (Transactional memory synchronization ) 是由 E.佩雷拉 于 2020-02-12 设计创作，主要内容包括：提供一种具有两个或更多个服务器的在线游戏服务,所述两个或更多个服务器位于两个或更多个数据中心中,所述在线游戏服务包括：跨所述两个或更多个服务器实现的事务层,所述事务层用于处理视频游戏的分别由所述两个或更多个服务器执行的不同会话之间的同步；其中所述视频游戏的所述不同会话分别被配置为通过相应的客户端装置提供对虚拟空间的查看；其中所述事务层被配置为标识提供对所述虚拟空间的近似区域的查看的会话,并且实现所标识的会话的事务存储器同步,使得所述虚拟空间中由所标识的会话生成的事件跨所标识的会话中的每个会话被同步。(Providing an online gaming service having two or more servers located in two or more data centers, the online gaming service comprising: a transaction layer implemented across the two or more servers, the transaction layer to handle synchronization between different sessions of a video game respectively performed by the two or more servers; wherein the different sessions of the video game are each configured to provide for viewing of a virtual space by a respective client device; wherein the transaction layer is configured to identify sessions that provide views of approximate regions of the virtual space and to implement transactional memory synchronization of the identified sessions such that events in the virtual space generated by the identified sessions are synchronized across each of the identified sessions.)

1. A method, the method comprising:

executing a first session of a video game in a first data center;

executing a second session of the video game in a second data center concurrently with the first session;

wherein the first session and the second session independently generate a first game state and a second game state, respectively, the first game state and the second game state simultaneously representing a shared virtual space;

wherein an event implemented via a first session in the shared virtual space is implemented by transactions with respect to a first set of memory objects stored at the first data center, the first session accessing a first repository configured to generate a first set of update packages based on the transactions with respect to the first set of memory objects and transmit the first set of update packages to the second data center;

wherein an event implemented via the second session in the shared virtual space is implemented by transactions with respect to a second set of memory objects stored at the second data center, the second session accessing a second repository configured to generate a second set of update packages based on the transactions with respect to the second set of memory objects and transmit the second set of update packages to the first data center;

applying the first set of update packages to the second set of memory objects at the second data center;

applying the second set of update packages to the first set of memory objects at the first data center;

wherein the generation, transmission, and application of the first and second sets of update packages enables shared real-time interactivity in the shared virtual space via the first and second sessions.

2. The method of claim 1, wherein the first session and the second session are not in communication with each other such that the first session is performed using input received at the first data center and the first set of memory objects to continuously update the first game state and the second session is performed using input received at the second data center and the second set of memory objects to continuously update the second game state.

3. The method of claim 2, wherein the input received at the first data center is received from a first client device, and wherein the input received at the second data center is received from a second client device.

4. The method of claim 1, wherein the generating, the transmitting, and the applying of the first set of update packages and the second set of update packages enable synchronization of at least some memory objects in the first set of memory objects with corresponding memory objects in the second set of memory objects.

5. The method of claim 4, wherein applying the first set of update packages changes one or more values of the corresponding memory objects in the second set of memory objects, and wherein applying the second set of update packages changes one or more values of the at least some memory objects in the first set of memory objects.

6. The method of claim 1, wherein each of the update packages comprises a timestamp, a memory identifier, and a value for a memory object identified by the memory identifier.

7. The method of claim 6, wherein applying each of the update packages comprises setting a value of the memory object identified by the memory identifier to the value of the update package.

8. The method of claim 6, wherein applying each of the update packets comprises determining whether the timestamp of the update packet is a most recent timestamp for the memory object identified by the memory identifier, and if so, setting a value of the memory object identified by the memory identifier to the value of the update packet.

9. The method of claim 1, wherein the event implemented via the first session in the shared virtual space comprises a change in location of a first virtual character controlled by the first session, and wherein the event implemented via the second session in the shared virtual space comprises a change in location of a second virtual character controlled by the second session.

10. The method of claim 1, wherein execution of the first session renders a first view of the shared virtual space, and wherein execution of the second session renders a second view of the shared virtual space, such that the events in the shared virtual space that are implemented by the first session and the second session are viewable through the first view and the second view of the shared virtual space simultaneously.

11. An online gaming service having two or more servers located in two or more data centers, the online gaming service comprising:

a transaction layer implemented across the two or more servers, the transaction layer to handle synchronization between different sessions of a video game respectively performed by the two or more servers;

wherein the different sessions of the video game are each configured to provide for viewing of a virtual space by a respective client device;

wherein the transaction layer is configured to identify sessions that provide views of approximate regions of the virtual space and to implement transactional memory synchronization of the identified sessions such that events in the virtual space generated by the identified sessions are synchronized across each of the identified sessions.

12. The online gaming service of claim 11, wherein each session of the video game is executed independently of other sessions of the video game, such that each session of the video game does not manage communication with the other sessions of the video game, and such that each session of the video game is unaware of the other sessions of the video game.

13. An online gaming service as recited in claim 12, wherein implementing transactional memory synchronization of the identified sessions comprises identifying memory transactions that result in events in the virtual space generated by one of the identified sessions, and propagating the memory transactions to other ones of the identified sessions.

14. The online gaming service of claim 13, wherein propagating the memory transaction to other ones of the identified sessions comprises generating an update package that includes a timestamp of the memory transaction, an identifier of a memory object to which the memory transaction applies, and a value set by the memory transaction.

15. The online gaming service of claim 11, wherein the transaction layer is configured to identify sessions that provide a view of an approximate area of the virtual space by analyzing virtual locations in the virtual space respectively associated with the different sessions of the video game.

16. An online gaming service having two or more servers located in two or more data centers, the online gaming service comprising:

a transaction layer implemented across the two or more servers, the transaction layer to handle synchronization between different sessions of a video game respectively performed by the two or more servers;

wherein the different sessions of the video game are each configured to provide for viewing of a virtual space by a respective client device;

wherein the transaction layer is configured to implement transactional memory synchronization of the different sessions such that events in the virtual space generated by the different sessions are synchronized across each of the different sessions;

wherein each session of the video game is performed independently of other sessions of the video game such that each session of the video game does not manage communications with the other sessions of the video game and such that each session of the video game is unaware of the other sessions of the video game.

17. The online gaming service of claim 16, wherein implementing transactional memory synchronization for the identified sessions comprises identifying memory transactions that result in events in the virtual space generated by one of the sessions, and propagating the memory transactions to other ones of the sessions.

18. The online gaming service of claim 16, wherein propagating memory transactions to other ones of the sessions comprises generating an update package that includes a timestamp of the memory transaction, an identifier of a memory object to which the memory transaction is applied, and a value set by the memory transaction.

Technical Field

The present disclosure relates to systems and methods for transactional memory synchronization between different sessions of an interactive application, such as a video game.

Background

Description of the Related Art

The current rapidly developing technology area is the video game area, now covering numerous game platforms, including dedicated game consoles, Personal Computers (PCs), and more recently cloud games and mobile devices. One example of a network gaming service/system isNetwork, which includes various gaming services that support console-based and cloud-based gaming.

In a cloud gaming setting, a user can access many games on a cloud gaming site over a network (such as the internet) and begin interacting/playing the games. To select a playable game, the user accesses his/her account on the cloud gaming site and initiates one of a plurality of games available for play with the user account. Video generated from the cloud video game is transmitted to the client device. An example of a cloud gaming system isA Now cloud gaming service.

Currently, simulations, such as video games, may be performed by cloud server computers and represent virtual worlds. To facilitate interaction between a large virtual world and many players in such a virtual world, the virtual world may be processed by several different server computers. There may be multiple sessions of the video game, each session controlling a different portion of the virtual world, and such sessions being performed by different server computers that may be located in different data centers. Thus, a given player may connect to different server computers (and different sessions) as the player moves throughout the virtual world. Thus, while there may be multiple sessions, they do not represent portions of the same real or virtual space or virtual world at the same time. If two players share the same virtual space, they are typically connected to the same server machine.

However, this may lead to several problems. For example, when a player switches from one server to another as they transition from one part of the virtual world to another, they may experience a lag or delay in the transition. Furthermore, all users wishing to interact in a particular region of the virtual world need to connect to the same server machine or data center. This can be problematic because there may be too many users wishing to connect to the same server machine in order to access the same virtual space, and the server machine may not have enough resources or bandwidth to handle all such connections without degrading game quality. Another problem is that the location of a particular server machine may be in a data center that is remote from a given player, making it difficult to establish a strong network connection to support high quality game streaming for the given player. Thus, switching from one server to another may cause a change in the quality of the game streaming due to different network conditions resulting from different locations of the servers. In some cases, game streaming quality may degrade in the form of increased latency, decreased video quality, and the like.

In summary, currently, different server machines (and by extension, different data centers) do not represent the same (virtual) reality at the same time in the context of a given simulated or video game. Instead, different server machines handle different parts of the virtual world, and this can create problems as described above.

It is in this context that embodiments of the present disclosure are presented.

Disclosure of Invention

Implementations of the present disclosure provide methods and systems for transactional memory synchronization between different sessions of an interactive application, such as a video game or simulation.

In some implementations, a method is provided that includes the operations of: executing a first session of a video game in a first data center; executing a second session of the video game in a second data center concurrently with the first session; wherein the first session and the second session independently generate a first game state and a second game state, respectively, the first game state and the second game state simultaneously representing a shared virtual space; wherein the event implemented via the first session in the shared virtual space is implemented by transactions with respect to a first set of memory objects stored at a first data center, the first session accessing a first repository configured to generate a first set of update packages based on the transactions with respect to the first set of memory objects and to transmit the first set of update packages to a second data center; wherein the event implemented via the second session in the shared virtual space is implemented by transactions with respect to a second set of memory objects stored at a second data center, the second session accessing a second repository configured to generate a second set of update packages based on the transactions with respect to the second set of memory objects and to transmit the second set of update packages to the first data center; applying the first set of update packages to a second set of memory objects at a second data center; applying the second set of update packages to the first set of memory objects at the first data center; wherein the generation, transmission and application of the first and second sets of update packages enables shared real-time interactivity in the shared virtual space via the first and second sessions.

In some implementations, the first session and the second session are not in communication with each other such that the first session is performed using the input received at the first data center and the first set of memory objects to continuously update the first game state and the second session is performed using the input received at the second data center and the second set of memory objects to continuously update the second game state.

In some implementations, the input received at the first data center is received from a first client device, and wherein the input received at the second data center is received from a second client device.

In some implementations, the generation, transmission, and application of the first and second sets of update packages enables synchronization of at least some memory objects in the first set of memory objects with corresponding memory objects in the second set of memory objects.

In some implementations, applying the first set of update packages changes one or more values of corresponding memory objects in the second set of memory objects, and wherein applying the second set of update packages changes one or more values of at least some memory objects in the first set of memory objects.

In some implementations, each of the update packages includes a timestamp, a memory identifier, and a value for the memory object identified by the memory identifier.

In some implementations, applying each of the update packages includes setting a value of the memory object identified by the memory identifier to the value of the update package.

In some implementations, applying each of the update packets includes determining whether a timestamp of the update packet is a most recent timestamp for the memory object identified by the memory identifier, and if so, setting a value of the memory object identified by the memory identifier to the value of the update packet.

In some implementations, the event implemented via the first session in the shared virtual space includes a change in a location of a first avatar controlled by the first session, and wherein the event implemented via the second session in the shared virtual space includes a change in a location of a second avatar controlled by the second session.

In some implementations, execution of the first session renders a first view of the shared virtual space, and wherein execution of the second session renders a second view of the shared virtual space, such that events in the shared virtual space that are implemented by the first session and the second session are simultaneously viewable through the first view and the second view of the shared virtual space.

In some implementations, an online gaming service is provided having two or more servers located in two or more data centers, the online gaming service comprising: a transaction layer implemented across two or more servers, the transaction layer to handle synchronization between different sessions of a video game respectively performed by the two or more servers; wherein the different sessions of the video game are each configured to provide for viewing of the virtual space by a respective client device; wherein the transaction layer is configured to identify sessions that provide views of the approximate region of the virtual space and to implement transactional memory synchronization of the identified sessions such that events in the virtual space generated by the identified sessions are synchronized across each of the identified sessions.

In some implementations, each session of the video game is performed independently of other sessions of the video game, such that each session of the video game does not manage communications with other sessions of the video game, and such that each session of the video game is unaware of the other sessions of the video game.

In some implementations, implementing transactional memory synchronization of the identified sessions includes identifying a memory transaction that caused an event in virtual space generated by one of the identified sessions, and propagating the memory transaction to the other of the identified sessions.

In some implementations, propagating the memory transaction to other ones of the identified sessions includes generating an update package that includes a timestamp of the memory transaction, an identifier of a memory object to which the memory transaction applies, and a value set by the memory transaction.

In some implementations, the transaction layer is configured to identify sessions that provide a view of an approximate area of the virtual space by analyzing virtual locations in the virtual space that are respectively associated with different sessions of the video game.

In some implementations, an online gaming service is provided having two or more servers located in two or more data centers, the online gaming service comprising: a transaction layer implemented across two or more servers, the transaction layer to handle synchronization between different sessions of a video game respectively performed by the two or more servers; wherein the different sessions of the video game are each configured to provide for viewing of the virtual space by a respective client device; wherein the transaction layer is configured to implement transactional memory synchronization of the different sessions such that events in the virtual space generated by the different sessions are synchronized across each of the different sessions; wherein each session of the video game is executed independently of the other sessions of the video game such that each session of the video game does not manage communications with the other sessions of the video game and such that each session of the video game is unaware of the other sessions of the video game.

In some implementations, implementing transactional memory synchronization for the identified sessions includes identifying a memory transaction that caused an event in virtual space generated by one of the sessions, and propagating the memory transaction to the other ones of the sessions.

In some implementations, propagating the memory transaction to other ones of the sessions includes generating an update package that includes a timestamp of the memory transaction, an identifier of a memory object to which the memory transaction is applied, and a value set by the memory transaction.

Other aspects and advantages of the disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the disclosure.

Drawings

The disclosure, together with further advantages thereof, may best be understood by reference to the following description taken in conjunction with the accompanying drawings.

Figure 1 conceptually illustrates a system for cloud gaming with transactional memory synchronization across data centers, according to implementations of the present disclosure.

FIG. 2 illustrates a system for synchronization of memory objects between application sessions according to an implementation of the present disclosure.

Figure 3 conceptually illustrates a system in which updates are filtered based on virtual geographic proximity, according to an implementation of the present disclosure.

Fig. 4A illustrates an exemplary system for loading game files for games available through a cloud gaming site according to an implementation of the present disclosure.

Figure 4B is a flow diagram conceptually illustrating various operations performed to stream a cloud video game to a client device, according to implementations of the present disclosure.

FIG. 5 illustrates an embodiment of an information service provider architecture according to an implementation of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art, that the present disclosure may be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order not to obscure the present disclosure.

Currently, multiple server computers/machines, which may be in multiple different data centers, do not represent the same (virtual) reality in a given video game or simulated scenario, but instead each represent a different portion of the virtual world. This can be problematic because all users who wish to access a particular portion of the virtual world need to connect to the same server/data center, and because resources in the data center are limited, which can lead to resource problems as previously described.

To address such issues, implementations according to the present disclosure provide transactional memory synchronization across multiple data centers so that they may represent the same reality. To share reality among data centers, a transaction layer is created that tracks changes and replicates those changes across data centers. Broadly, a given transaction includes a timestamp indicating the time of the transaction, a memory location, and a value or change in value (e.g., a variable to be updated). By enabling transactional memory synchronization across data centers, multiple data centers may represent the same reality, enabling resource utilization to be spread across data centers, and also providing users with better network connectivity by allowing users to connect to data centers that are closer to their location.

Moreover, implementations of the present disclosure aid developers by providing a system that automates the creation, transmission, reception, and application of memory transactions. Thus, an application developer may develop without knowing how many data centers represent a shared reality of the video game. Further, the transaction mechanism can work out the consolidated state of the shared reality so that developers do not need to worry about managing states among multiple data centers.

Broadly speaking, implementations of the present disclosure apply to cloud gaming systems. An example of a cloud gaming system isA Now cloud gaming system. In such a system, the client device may be a game console, such as4 game console or may be another device such as a personal computer, laptop computer, tablet computer, mobile phone, mobile device, etc. Such asNetwork game services/systems such as Network can provide various game services that support cloud-based games.

For ease of description, throughout this disclosure, it should be understood that reference to a "user" or "player" will generally be synonymous with the user/player (client) device associated with or operated by the user, as the user typically interfaces with the system through use or operation of the user device in accordance with this disclosure. Thus, while the term "user" is used in this disclosure for ease of description, it should be understood that the term "user" may encompass both a user and a user device operated by or otherwise associated with the user, and further that the terms "user" and "user device" are often used interchangeably in the present description of implementations, as will be apparent to those skilled in the art.

Figure 1 conceptually illustrates a system for cloud gaming with transactional memory synchronization across data centers, according to implementations of the present disclosure. As shown, there are multiple data centers, each executing a copy of an application representing the same reality. For example, the data center 100 includes server resources 102. The server resources 102 can include any computing resources in the data center 100 that can be used to execute and stream applications, including by way of example and not limitation various server computers/machines, blade computing resources, CPU/GPU/APU, volatile and non-volatile memory or storage devices, local networking devices, and the like. In some implementations, the server resources 102 can be virtualized computing resources that are abstracted over the underlying computing hardware.

The application session 104 executes on a server resource 102 in the data center 100. In some implementations, the application session 104 is a session of a video game, simulation, or other interactive program. Broadly, the application session 104 generates and updates a program state (or game state) that defines the virtual space/reality of the application session 104. Generally, for purposes of this disclosure, the "reality" of an application/video game is a virtual interactive environment generated and maintained by the application/video game in which multiple users can interact through respective client devices and affect the state of objects.

In representing the virtual space, the application session 104 performs transactions with respect to the memory objects 106. The memory objects 106 may include any variable, data, or data structure that defines the state of and interaction between objects or attributes in a virtual space. By way of example only, and not limitation, examples of memory objects or items defined by memory objects may include the following and/or attributes of the following: a character, avatar, person, animal, clothing, movement, weapon, shield, armor, vehicle, landscape, virtual location, virtual time, inventory, experience, virtual currency, health, life, injury, virtual object, virtual action, and the like.

According to implementations of the present disclosure, multiple data centers may represent the same reality. For example, in the illustrated implementation, additional data centers 110 and 120 are shown that may execute the same application. In the case of data center 110, server resources 112 are used to execute application sessions 114 that perform transactions with respect to memory objects 116. Likewise, data center 120 uses server resources 122 to execute an application session 124 that performs transactions with respect to memory objects 126. Application sessions 114 and 124 are different sessions of the same application as application session 104 and memory objects 116 and 126 may include many or all of the same memory objects as memory object 106. While data centers 100, 110, and 120 are shown in the illustrated implementation, it should be understood that additional data centers may be present and that the principles of the present disclosure apply similarly regardless of the number of data centers.

Each application session receives input from one or more client devices operated by a respective user. Further, each application session renders video (including image data and audio data) that provides a view of the virtual space to the client device for presentation on a display that may be integrated into the client device (e.g., laptop, tablet, mobile phone, etc.) or connected to the client device (e.g., television, display, or head mounted display connected to a game console, personal computer, etc.). For example, in the illustrated implementation, a client device 132 operated by a user 134 may communicate with the data center 100 over the network 130 to provide input to the application session 104 and to stream video from the application session 104. The input may be generated based on controller input from a controller device that may be connected to or integrated into the client device 132. Examples of the controller device include the following: a keyboard, mouse, trackball, touch pad, touch screen, joystick, motion controller, game controller, image capture device or camera, microphone, depth camera, and the like. The transmission of input and video as described to enable interactivity/gameplay with an application or video game is referred to as streaming the application or streaming the video game (gameplay of the video game).

Thus, further in accordance with the illustrated implementation, user 138 may operate client device 136 to stream application session 114; and user 142 may operate client device 140 to stream application session 124. It should be understood that there may be many more client devices and users, and in some implementations there may be hundreds or thousands of client devices and users connected to the data center to interact with the application session. Various users may stream from different application sessions, but users are able to simultaneously participate in the same virtual space/reality, such that their activities are synchronized across sessions.

For example, in the illustrated implementation, interactivity is streamed from data centers 100, 110, and 120 using application sessions 104, 114, and 124 to stream to client devices 132, 136, and 140, respectively. Each of the application sessions represents the same reality, and thus to facilitate synchronization of activities across application sessions, a transactional system is provided to handle data changes. Broadly, the transaction system is configured such that when a piece of data is changed, then the underlying transaction layer creates and sends an update package to other servers, and the other servers may apply the update package to process such updates according to the configuration of the application.

This is made easy for application developers by providing a transactional layer to facilitate the way data changes across different sessions of the same reality, so that application developers do not have to consider how to transfer the data and maintain synchronization between different sessions of the application. Instead, developers can develop applications in a single sense and do not have to develop mechanisms to support synchronization between objects that exist simultaneously on multiple servers or in multiple data centers.

With continued reference to FIG. 1, each of the data centers implements an instance of a transaction layer. As shown, the server source 102 in the data center 100 implements a transaction layer 108; the server sources 112 in the data center 110 implement a transaction layer 118; and server sources 122 in data center 120 implement transaction layer 128. Each of the transaction layers is configured to synchronize transactions with respect to the memory objects through application sessions across the data center.

By way of example and not limitation, a user 134 operating a client device 132 may interact with a virtual space generated by an application session 104. That is, the application session 104 streams video that provides a view of the virtual space to the client device 132 and updates the application state of the application session 104, in part, by performing transactions with respect to the memory objects 106. Execution of the application session 104 may be driven in part by input received from the client device 132, for example, based on controller input provided by the user 134 using a controller device operatively connected to or integrated into the client device 132.

When a transaction is performed on a given memory object that is one of the memory objects 106, the transaction layer 108 is configured to enable synchronization across the data centers. To accomplish this, the transaction layer 108 creates an update package that includes an identifier/location of the memory object, a timestamp of the transaction, and a value used to update the memory object (e.g., a new value of the memory object, an offset from an existing value, etc.). Transaction layer 108 creates and transmits update packages to data centers 110 and 120, and corresponding transaction layers 118 and 128 at data centers 110 and 120 process and apply the update packages, respectively. For example, the transaction layer 118 may apply an update package to update the value of a corresponding one of the memory objects 116; and the transaction layer 128 may apply the update package to update the value of the corresponding one of the memory objects 126.

By way of example and not limitation, a memory object may represent a location of a character controlled by user 134 via client device 132. As the user 134 moves the character, the application session 104 applies input from the client device 132 that effects the movement of the character such that its position changes from a first position to a second position in the virtual space. This may be implemented as a transaction with respect to a memory object, changing its value from a value identifying a first location to a value identifying a second location. When this occurs, the transaction layer 108 generates an update package that includes a timestamp of the transaction, an identifier of the memory object, and a value that identifies the second location. The transaction layer 108 transmits the update package to the other transaction layers 118 and 128 that apply the update package to update the corresponding ones of the memory objects 116 and 126 with a value that identifies the second location. In this manner, the location of roles that have changed in application session 104 is updated across application sessions 114 and 124. Thus, for example, a user 138 streaming from the application session 114 using the client device 136 may see the character moving to the second location while viewing the virtual space in which the character resides. Similarly, the user 132, using the client device 140 to stream from the application session 124, can see the character moving to the second location while viewing the virtual space in which the character is located.

It should be understood that users 134, 138, and 142 are in a shared space, but each of them is streaming interactivity from a unique and different session of the application. In a sense, the different application sessions 104, 114, and 124 each independently generate a virtual space being shared, but are synchronized with each other through synchronization of the transaction layer by virtue of the memory objects (for the purpose of shared interactivity). The transaction layer propagates changes from each data center to each of the other data centers, maintaining synchronization between different application sessions. This enables real-time interactivity between users in a shared space that is consistent across data centers.

Continuing with the scenario described above, if the character of user 134 kicks a stone in the shared virtual space and the stone is about to trip the character of user 138, information indicating the action should be sent from data center 100 to data center 110 so that it can be rendered by application session 114 in data center 110. Then, if the character of the user 138 skips the stone, information of the action is sent back so that the application session 104 in the data center 100 can render the jump. In accordance with the above principles, the act of user 134 kicking a stone is reflected in one or more transactions with one or more of memory objects 106 (e.g., memory objects defining the state/movement/location of the character of user 134, memory objects defining the state/movement/location of a stone, etc.). These transactions are packed by transaction layer 108 and transferred to transaction layer 118 where they are applied to memory objects 116 and rendered by application session 114. Further in accordance with the principles of the present disclosure, the action of the user 138 skipping over the stone is reflected in one or more transactions with respect to one or more of the memory objects 116 (e.g., a memory object defining the state/movement/location of the character of the user 138, a memory object defining the state/movement/location of the stone, etc.). These transactions are packed by the transaction layer 118 and transferred to the transaction layer 108 where they are applied to the memory objects 106 and rendered by the application session 104. In addition, the user's 142 character in the virtual space may be seeing kicks to and skips over a stone, and thus the transaction layer 128 also receives packed transactions from the transaction layers 108 and 118 that are applied by the transaction layer 128 and rendered by the application session 124 so that the user 142 can see the activity occurring in the virtual space consistent with the reality represented by the other application sessions 104 and 114.

According to implementations of the present disclosure, data centers may be geographically remote from each other. For example, data center 100 (and client devices 132 and users 134) may be in the united states, while data center 110 (and client devices 136 and users 138) may be in japan, and data center 120 (and client devices 140 and users 142) may be in the united kingdom. Each player connects to a data center executing an application program proximate to its respective client device. However, each data center may run the application without knowing the fact that the application is running in multiple data centers. Each instance of the application in each data center executes independently such that at each frame, each session is processed with the available data it has, generating a frame with inputs reflected in the available memory objects. And because the memory objects are being synchronized, the session can therefore represent a synchronized reality. This simplifies the development of the code for the application, since synchronization is handled by the underlying transaction layer rather than at the application level, and thus the application may not be aware of the fact that it may be running simultaneous sessions in different datacenters. Each session need not be specifically configured to interact directly with another data center. For example, each session does not need to know the data center to which the various users are connected, as each session does not involve sending messages to a particular data center based on the knowledge that a particular player is connected to that data center. In this sense, each session is unaware of the other sessions and does not manage communications with other sessions in other data centers. Instead, the underlying transaction layer handles updates to each of the data centers as transactions occur with respect to the memory objects.

In some implementations, the application of the update package by each data center may be according to a predefined configuration for processing incoming updates. For example, in some implementations, incoming updates are always applied. In some implementations, the update is applied if the timestamp of the update is immediately (later) than the timestamp of the last transaction applied to the associated memory object (e.g., by a local application session application or from a previous update application received from another data center). In some implementations, the update is first processed by an application session at the receiving data center and then applied, possibly after being modified in some way by the application session.

FIG. 2 illustrates a system for synchronization of memory objects between application sessions according to an implementation of the present disclosure. Broadly, in some implementations, the transaction layer is implemented by a library that identifies data sets that should be synchronized between data centers, and therefore updates should be transmitted against the library when changes are detected. Thus, when a transaction occurs with respect to such data, the change is effected through the library. Typically, the library is configured to package such memory transactions (e.g., including memory locations/identifiers, timestamps, changed values/new values) and broadcast the transactions to other data centers. When received by each data center, the application specifies what happens when a new data update is received. Different applications may have different configurations for the way in which the changed data is managed. In some implementations, whenever new data is received, if it is the most recent update (e.g., based on a timestamp), the new data is applied; while in other implementations, updates may be evaluated and possibly modified and applied differently for any new data, depending on the current application state. Thus, each server may run an instance of a library module that identifies transactions that should be updated across the servers. When such a transaction occurs, it is identified by the library module and transmitted to other data centers.

With continued reference to the implementation of fig. 2, a library module 200 is shown instantiated on the server resource 102, which also runs the application session 104 and hosts the memory object 106, as previously described. The library module 200 includes synchronization data 202 configured to identify memory objects/data that should be synchronized across data centers. In some implementations, the synchronization data 202 includes logical memory locations or identifiers for some or all of the memory objects 106 that are consistent across all instances of the library modules so that when the library modules communicate updates to each other, they may reference equivalent memory objects in different data centers. In some implementations, the synchronization data 202 includes a mapping of local memory objects to logical memory locations/identifiers, such that a given logical memory location/identifier maps to an equivalent memory object in each data center.

As described above, the synchronization data 202 may be configured to identify which memory objects are to be synchronized across data centers. In some implementations, a particular memory object or memory identifier is marked or identified (or pre-marked/pre-identified) in the synchronization data 202 as the data to be synchronized. In some implementations, the synchronization data 202 identifies the type or class of data/variables to be synchronized. In some implementations, the synchronization data 202 identifies transactions or transaction types to be synchronized. Regardless of the particular implementation, the synchronization data 202 is configured to enable identification of which memory objects 106 are to be synchronized across the datacenter.

In some implementations, the application session 104 is configured to reference the library module 200 to determine which memory objects or memory transactions need to be synchronized. Then, if a memory transaction occurs that requires synchronization, application session 104 may invoke library module 200 to process the transaction and/or synchronization. In some implementations, the library module 200 is configured to process memory transactions from the application sessions 104 and to determine which transactions are to be synchronized across the various application sessions or datacenters. It should be understood that for memory transactions that do not require synchronization, then only the affected local memory objects will change (one or more of the memory objects 106) as a result of the transaction; for memory transactions that do require synchronization, the local memory objects and the corresponding memory objects in the other sessions/data centers will all change as a result of the transaction.

It should be understood that the library module 200 may be configurable to allow a developer to determine which memory objects and/or transactions are to be synchronized across the data center. In some implementations, the library module 200 (e.g., when included as part of a game engine or application library collection) may include a standard or default configuration that pre-identifies certain data or transaction types as data or transaction types to be synchronized by the library module 200. However, in some implementations, the configuration may be adjusted by the developer. In some implementations, the application may be configured by a developer to specify which data or transactions are to be synchronized or which default configurations are to be adjusted. In the case of static libraries, this may occur during application build, while in the case of dynamic libraries, this may occur when the libraries are loaded at runtime.

It should be appreciated that when a developer marks which transactions or objects are important for synchronization, then the library is configured to handle the process of updating the state of such transactions or such objects across the data center. It should be understood that not all features of the virtual space need be synchronized. For example, while it may be desirable to synchronize a player's character or avatar across sessions, it may not be necessary to synchronize other elements in the virtual space, such as foliage or randomly generated movements of birds flying overhead, or elements that do not generally affect the interactivity between users in the shared virtual space. It should be appreciated that since not all virtual objects or states are synchronized, the amount of data that needs to be transferred between data centers for synchronization is reduced and network bandwidth is conserved. This helps to achieve real-time or substantially real-time synchronization across sessions by reducing the likelihood that latency due to insufficient bandwidth will be a problem.

With continued reference to FIG. 2, when it is determined that transactions on memory objects need to be synchronized across sessions/datacenters, then the packaging logic 204 of the library module 200 is configured to package relevant information for updating other datacenters. In some implementations, the packing logic 204 constructs an update package 208 that contains information for updating and synchronizing related memory objects at other data centers. In some implementations, the update package 208 includes: a timestamp 210 indicating a time of a transaction or change to a memory object as determined or performed by the application session 104; a memory identifier/location 212 that identifies or locates a memory object to be updated at the receiving data center; and a value 214 that may define a new/current/updated value of the memory object. In some implementations, the value 214 defines an offset or change from a previous value or other reference value. The packing logic 208 packs the information to generate an update package 208 and transmits the update package 208 to other data centers.

In the illustrated implementation, update package 208 is shown as being transmitted to data center 110, where it is processed by a corresponding library module 216 similar to library module 200 but configured for application sessions 114 and memory objects 116. Update package 208 is processed by an update handler 222, which in some implementations is included as part of library module 216. In other implementations, the update handler 222 is separate from the library module 216.

The update handler 222 is configured to receive the update package 208 and apply the information in the update package 208 according to predefined settings. In some implementations, the update handler implements rules for the manner and/or time in which the update is applied. It should be appreciated that updates may be handled differently depending on the transaction or memory object affected. For some transactions or memory objects, the update handler 222 may be configured to always apply the latest updates. That is, if the timestamp 210 is later than the time of the last modification to the memory object, the incoming update package 208 will be applied. For some transaction or memory objects, the update handler 222 may be configured to evaluate the update package 208 and apply it if it meets certain criteria, or apply it with some modification. It should be appreciated that on a per transaction basis, there may be different modes of processing update data as it is received.

While the transfer of updates from data center 100 to data center 110 has been described above, it should be understood that memory transactions generated by application session 114 at data center 110 may also trigger updates to be transferred from data center 110 to data center 100. Each data center includes substantially the same library modules as described above. In the illustrated implementation, the library module 216 also includes synchronization data 218, similar to the synchronization data 202, and packing logic 220, similar to the packing logic 204. Likewise, the library module 200 at the data center 100 also includes an update handler 206, similar to the update handler 222, for processing incoming update packages received from other data centers. Thus, in accordance with the above, the application session 114 can generate memory transactions that trigger updates to be transmitted from the data center 110 to other data centers (as determined based on the synchronization data 218). Thus, in the illustrated implementation, the packing logic 220 will generate an update package that is transmitted to the data center 100 and applied to the memory object 106 by the update handler 206.

While synchronization has been described above with respect to data centers 100 and 110, it should be understood that the concepts may be extended to any number of data centers, such as data center 120 and other data centers, that also run concurrent sessions of applications and library modules. Thus, each server/data center has library modules, and the library modules communicate with each other to propagate changes between the various sessions of the application to ensure synchronization of the shared virtual space/environment. In this way, the developer does not have to specify the manner in which transactions should be updated across data centers, but may encode as if running only on a local data center. The library module is configured to identify which transactions need to be propagated to other sessions/data centers, thereby causing such transactions or related memory objects to be pre-tagged or pre-identified in the library module to trigger updates when such transactions occur.

By way of example only, and not limitation, some examples of transactions that may be pre-identified in the library module as triggering an update include the following: virtual objects (e.g., doors) are damaged to the extent that items are removed from the virtual world (e.g., picked up by a player), player inventory (e.g., of virtual items or property) is increased/decreased; a virtual object (e.g., a vehicle) is destroyed, etc. It should be appreciated that such transactions may include any information important or necessary for another player connected to another data center to know whether to interact in the same shared virtual space/environment.

In some implementations, updates may or may not be sent based on other factors. For example, updates triggered for transmission may be filtered at the sending data center based on the virtual geographic proximity of the virtual objects of the receiving data center.

Figure 3 conceptually illustrates a system in which updates are filtered based on virtual geographic proximity, according to an implementation of the present disclosure. In the illustrated implementation, a virtual space 300 is conceptually illustrated, wherein a player P₁、P₂And P₃Represented in virtual space 300. It should be understood that the player may be represented in the virtual space by a character, avatar, vehicle, or other virtual object that may represent the player. Thus, the position of a player in virtual space is actually the position of the player's representation, which is synonymously referenced for ease of description. As shown in the illustrated implementation, player P₁And P₂In the virtual space 300 compared to the player P₃Closer to each other.

In addition, player P₁Controlled by the data center 100; player P₂Controlled by the data center 110; and player P₃Controlled by the data center 120. That is, the player P is controlled from the client device connected to the data center 100₁(ii) a Controlling player P from a client device connected to data center 110₂(ii) a Controlling player P from client devices connected to data center 120₃。

As described above, in some implementations, the transmission or based in part on the geographic location in the virtual space 300No updates are sent. To facilitate this functionality, the library modules may be configured to share with each other location data identifying the location of players respectively associated with their data centers. Thus, in the illustrated implementation, library module 200 may share identified player P with other library modules 216 and 230 (of data centers 110 and 120, respectively)₁Position data of the location of (a). Likewise, library module 216 may share identified player P with other library modules 200 and 230₂Location data of the location of (a); and library module 230 may share identified player P with other library modules 200 and 216₃Position data of the location of (a). In this manner, each of the data centers receives data indicating geographic locations of interest of the other data centers. In this case, the geographic location of interest for a given data center is defined by the location of the player associated with that data center.

For example, when a game occurs with player P₁When a relevant change (e.g., a change in location, inventory, health, injury, appearance, associated weapons (shots), etc.), this may trigger the library module 200 to publish update packages to the data centers relevant to such updates. In some implementations, this may be based on distance from player P₁The distance of (c). For example, in some implementations, if a player associated with another data center is at player P₁Within a predefined distance or radius, the library module 200 will send an update package to the data center. In the illustrated implementation, this is represented by a radius R₁Shown. Because of player P₂At a radius R₁Within, the library module 200 will send the update package to library module 216 because player P is now playing₂Can be considered close enough to be close enough to player P₁The associated changes may affect player P₂Or otherwise associated therewith. In contrast, because of player P₃Not at the radius R₁Library module 200 will not send update packages to library module 230 because player P₃Can be considered far enough to communicate with player P₁The associated change may not affect player P₃Or not otherwise associated therewith.

In a similar manner, player P may be identified₂Related toThe change is propagated to player P₂Is predefined radius R₂The players in the house. Thus, data center 110 (using library module 216) will send information about player P to data center 100 (via library module 200)₂Will not send such updates to the data center 120 because player P₃At a radius R₂And (c) out. And similarly, data center 120 (using library module 230) will not send information about player P to either of data centers 100 or 110₃Is updated (however, as the player enters radius R₃When such updates will not be sent to the corresponding data center).

Although the foregoing has been described with respect to players, it should be understood that the principles may be applied to any type of virtual object in the virtual space 300. That is, updates may be sent based on virtual geographic proximity in the virtual space.

In some implementations, communicating updates based on geographic proximity may be implemented by the library module as a configurable mechanism that may be turned on/off or activated/deactivated by a developer/application.

By filtering update transmissions based on proximity, resources may be conserved, which may be useful in a large game world in which many players may be present, some of which may be remote from each other. In this way, the sending server can decide whether the data is important for the recipient. Thus, another layer of transfer is provided in which servers transfer to each other locations that are important to their respective players, or affected locations thereof, so that each server can determine whether to send updates to another server.

In some implementations, a transaction or memory object may be marked with a location and/or a radius at setup. Typically, the location will be updated periodically, while the proximity to which another entity will be affected by a given transaction or memory object is configured at setup.

When a piece of data is changed, the library module may evaluate the list of servers and check each server to determine if an update should be sent to that server. Thus, by filtering on the sending side, resources are saved, since not all updates are sent to all servers. Each server periodically learns of locations or transactions or objects that are important to each other.

In some other implementations, the transaction itself may communicate the spatial relationship. For example, the update package may also include location or proximity information. The update handler may determine whether to apply the update based in part on such location/proximity information.

In some implementations, different servers/data centers may have rights to certain transaction or memory objects. For example, a given virtual object, such as a character or vehicle, may be controlled by only one data center, and no other data center will create a transaction for that virtual object. Instead, the other data centers will only receive data from the authorized data centers and apply it to update their corresponding virtual objects. In some implementations, a data center may advertise its rights to other data centers through transactions, and from this point on, the other data centers will not create such transactions. Or if another data center attempts to create a transaction, the library may generate a fault state that may later be used for debugging purposes.

Broadly speaking, implementations of the present disclosure can be included as part of a game engine.

Broadly speaking, a game engine is a software development framework that provides features that enable the efficient development of video games. The game engine may include a software library with reusable modules to handle various aspects of game functionality, including, by way of example and not limitation, graphics rendering (e.g., including vertex processing, polygon processing, shading, lighting, texturing, etc.), sound, physics (including collision processing), animation, scripting, artificial intelligence, networking, streaming, memory management, threading, localization support, scene graphs, cinematography, and the like.

The game engine may be optimized for different hardware platforms, such as game consoles, mobile devices, personal computers, and the like. By way of example and not limitation, the game engine may optimize memory usage (e.g., how to prioritize various tasks in a graphics pipeline, etc.) depending on the platform. In some implementations, the hardware may be a bladed version of some specific processing entity, such as a game console. Thus, users may be assigned to particular blades, which gives the same hardware that the console game has been optimized.

It should be understood that game server logic may also be present to provide streaming and/or other services (packetization, encoding, quality of service (QOS) monitoring, bandwidth testing, access to social networks/friends, etc.).

In some implementations, the cloud infrastructure can run a hypervisor that abstracts the hardware and provides a virtual machine framework on which an Operating System (OS) can be loaded. Thus, the stack may include an application/video game running on an OS that is loaded on a Virtual Machine (VM) instantiated by a hypervisor, the VM being loaded on the underlying hardware. In this manner, the execution of the application program need not be coupled to specific hardware.

In some implementations, the application/video game may be executed through a container that is abstracted at the application layer, packaging code and dependencies together, thereby enabling software development without knowledge of the OS or hardware platform, and facilitating cross-platform software portability.

In some implementations, a distributed game engine is employed, where different portions of the game engine may be processed by different computing entities. For example, the functionality of a game engine, such as a physics engine, rendering engine (2D/3D graphics), sound, script, animation, AI, networking, streaming (coding), memory management, threads, etc., may be divided into different functional processing blocks and/or services that are distributed among many different computations. It should be appreciated that for distributed game engines, low latency communication is required to avoid latency issues. To maintain a desired frame rate, the total time of computation and communication should satisfy certain constraints. Therefore, it may or may not be effective to divide certain tasks according to whether the process can be completed in a short time.

An advantage of using distributed game engines is that elastic computing can be utilized, where computing resources can be scaled up or down as needed. For example, in a massive multiplayer game that is traditionally executed on a single hardware server, after, for example, about 100 players, the hardware resources become limited so that no more players can be added. The game may queue additional players, which means that players must wait to join the game. However, with distributed game engines, more computing nodes can be added to meet the demand by using elastic cloud computing resources, thereby enabling, for example, thousands of players. The game is no longer constrained by the limitations of the particular hardware server.

Thus, the cloud gaming engine may have functionality that is distributed to different processing entities. It should be understood that different functions may be performed in different frameworks. For example, some functions (e.g., social) may run more easily in a container, while graphics may run better using a VM connected to a GPU.

To facilitate distribution of the functionality of the cloud gaming engine, the distribution/synchronization layer may manage distribution of jobs, e.g., sending out jobs, reclaiming data, identifying tasks to perform, and processing queuing times, e.g., if a job completes faster than necessary. In some implementations, a given task may be dynamically subdivided, if desired. For example, an animation may have lighting, and if the lighting is particularly complex, the lighting may be subdivided into three lighting jobs that are emitted for computation and recombined upon return. Thus, if the game engine functions require more work, they can be subdivided.

Cloud service providers provide computing at a specified performance level, for example, in terms of number of input/output operations per second ("IOPS"). Thus, the game provider may specify VMs, dedicated processing power, amount of memory, etc. from the cloud service provider and instantiate a distributed cloud game engine using the cloud service provider's system.

In some implementations, the library module and the update handler may be one or more components or modules of the game engine. In some implementations, the library module and the update handler may be separate components or integrated. In some implementations, the library module and the update handler may operate as a supplement to the game engine. In some implementations, the game engine may be a distributed game engine, as described above.

As described above, implementations of the present disclosure may be applied to a cloud gaming system. An example of a cloud gaming system isA Now cloud gaming system. In such a system, the client device may be a gaming console, such as4 game console or may be another device such as a personal computer, laptop computer, tablet computer, mobile phone, mobile device, etc.

Broadly, to implement a cloud game, when a user request for a game name is received, several operations are performed by one or more servers within a data center associated with the cloud game site. When the cloud gaming site receives the user request, a data center hosting the game associated with the selected game name is identified, and the request is sent to the identified data center to instantiate the game for the selected game name. In response to the request, a server at the data center identifies game code, loads the identified game code and initializes a file associated with the game code in preparation for presenting the game content to the user. The game data associated with the game may include general game data and user-specific game data. Thus, initializing the file may include identifying, loading, and initializing generic game data and user-specific game data. Initializing generic game data may include initializing a graphics engine, installing graphics data, initializing sound files, installing artwork, and the like. Initializing user-specific data may include locating, transmitting, and installing user data, user history, game history, and the like.

Upon loading and initializing generic game data, a "start up" screen may be provided for rendering at the client device. A start-up screen may be designed to provide a representative image of the game being loaded to allow the user to preview the type of game being loaded. Once the generic game data is loaded, certain initial content may be rendered and selection/navigation screens may be presented for selection and customization by the user. User selection inputs provided at the selection/navigation screen may include game level selections, one or more game icon selections, game mode selections, game rewards, and other user-related data that may require additional game content to be uploaded. In some implementations, game content is made available for viewing and interaction by streaming the game content from the game cloud system to a user's computing device. In some implementations, after the user-specific data is loaded, the game content is available for playing the game.

FIG. 4A illustrates an exemplary system for loading game files for a game available through a cloud gaming site. The system includes a plurality of client devices 400 communicatively connected to a cloud gaming site 404 over a network 402, such as the internet. Upon receiving a request from a client device 400 to access the cloud gaming site 404, the cloud gaming site 404 accesses user account information 406 stored in a user data store 408 to identify a user associated with the client device through which the request was initiated. In some embodiments, the cloud gaming site may also authenticate the identified user to determine all games that the user is authorized to view/play. After user account identification/authentication, the cloud gaming site accesses the game name data store 410 to identify the game names available at the game cloud site for the user account initiating the request. The game name data store 410, in turn, interacts with the game database 412 to obtain the game names for all games available for the cloud gaming site. When a new game is launched, the game database 412 will be updated with the game code and the game name data storage area 410 will be provided with the game name information of the newly launched game. When a request is initiated, the requesting client device may or may not register with the cloud gaming site. If the user of the requesting client device is not a registered user, the cloud gaming site may identify the user as a new user and select a game name (e.g., a default set of game names) that is appropriate for the new user. The identified game name is returned to the client device for presentation on the display screen 400-a, as shown in fig. 4A.

User interaction at one of the game names rendered on the client device is detected and a signal is sent to the cloud gaming site. The signal includes game name information where the user interaction is detected and the user interaction registered at the game name. In response to a signal received from the client device, the cloud gaming site actively determines a data center hosting the game, and sends a signal to the identified data center to load the game associated with the game name for which the user interaction was detected. In some embodiments, there may be more than one data center hosting the game. In such embodiments, the cloud gaming site may determine the geographic location of the client device that initiated the request and identify a data center that is geographically close to the client device and signal the data center to preload the game. The geographic location of the user may be determined using Global Positioning System (GPS) mechanisms within the client device, the IP address of the client, ping information of the client, and the like. Of course, the foregoing manner of detecting the geographic location of the user may be exemplary, and other types of mechanisms or tools may be used to determine the geographic location of the user. Identifying a data center that is close to the client device can minimize latency during user interaction with the game. In some embodiments, the identified data center may not have the bandwidth/capacity needed to host the game or may be over-utilized. In these embodiments, the cloud gaming site may identify a second data center that is geographically close to the client device. The loading of the game includes loading game code and executing an instance of the game.

In response to receiving the signal from the cloud gaming site, the identified data center may select a server at the data center to instantiate the game on the server. The server is selected based on available hardware/software capabilities and game requirements. The server may include a plurality of game consoles, and the server may determine which of the plurality of game consoles to use to load the game. The game console may resemble a stand-alone game console or may be a rack server or a blade server. The blade server, in turn, may include multiple server blades, with each blade having the circuitry required to instantiate a single dedicated application, such as a game. Of course, the above-described game consoles are exemplary, and should not be considered limiting. Other types of game consoles (including gaming stations, etc.) and other forms of blade servers may also be used to host the identified game.

Once the game console is identified, a generic game related code for the game is loaded onto the game console, and a signal identifying the game console instantiating the game is returned to the client device over the network via the cloud game site. Thus, the loaded game is available to the user.

Figure 4B is a flow diagram conceptually illustrating various operations performed to stream a cloud video game to a client device, according to implementations of the present disclosure. The game system 418 executes a video game and generates raw (uncompressed) video 420 and audio 422. Video 420 and audio 422 are captured and encoded for streaming purposes, as indicated at reference numeral 424 in the illustrated figure. The encoding may provide compression of the video and audio streams to reduce bandwidth usage and optimize the gaming experience. Examples of encoding formats include H.265/MPEG-H, H.264/MPEG-4, H.263/MPEG-4, H.262/MPEG-2, WMV, VP6/7/8/9, and so forth.

The encoded audio 426 and the encoded video 428 are further packetized into network packets, as indicated at reference numeral 432, for transmission purposes over a network, such as the internet. The network packet encoding process may also employ a data encryption process, thereby providing enhanced data security. In the illustrated implementation, audio data packets 434 and video data packets 436 are generated for transmission over a network, as indicated at reference numeral 440.

Gaming system 418 additionally generates haptic feedback data 430, which is also packetized into network data packets for network transmission. In the illustrated implementation, a haptic feedback data packet 438 is generated for transmission over a network, as further indicated at reference numeral 440.

The aforementioned operations of generating raw video and audio and haptic feedback data, encoding the video and audio, and packetizing the encoded audio/video and haptic feedback data for transmission are performed on one or more servers that collectively define a cloud gaming service/system. As indicated at reference numeral 440, the audio, video, and haptic feedback data packets are transmitted over a network (such as and/or including the internet). As indicated at reference numeral 442, audio data packets 434, video data packets 436, and haptic feedback data packets 438 are decoded/reassembled by the client device to define encoded audio 446, encoded video 448, and haptic feedback data 450 at the client device. If the data is already encrypted, the network packet will also be decrypted. The encoded audio 446 and encoded video 448 are then decoded by the client device, as indicated at reference numeral 444, to generate client-side raw audio and video data for rendering on a display device 452. Haptic feedback data 450 may be processed/transmitted to generate haptic feedback effects at controller device 456 or other interface device through which haptic effects may be rendered. One example of a haptic effect is a vibration or rumble of the controller device 456.

It will be appreciated that the video game is responsive to user input and may therefore perform a program flow similar to that described above for the transmission and processing of user input, but in the reverse direction from the client device to the server. As shown, user manipulation of the controller device 456 may generate input data 458. The input data 458 is packetized at the client device for transmission over a network to the cloud gaming system. The input data packets 460 are unpacked and reassembled by the cloud game server to define the input data 462 on the server side. The input data 462 is fed to the gaming system 418, which processes the input data 462 to update the game state of the video game.

During transmission of the audio data packets 434, video data packets 436, and haptic feedback data packets 438 (reference numeral 440), data transmission over the network may be monitored to ensure quality of service for the cloud game stream. For example, network conditions, including both upstream and downstream network bandwidth, may be monitored as indicated by reference numeral 464, and game streaming may be adjusted in response to changes in available bandwidth. That is, encoding and decoding of network packets may be controlled based on current network conditions, as indicated by reference numeral 466.

Fig. 5 illustrates an embodiment of an information service provider architecture. An Information Service Provider (ISP)570 provides a vast number of information services to geographically dispersed users 582 connected via a network 586. An ISP may provide only one type of service, such as stock price updates, or may provide multiple services, such as broadcast media, news, sports, games, and the like. In addition, the services provided by each ISP are dynamic, i.e., services can be added or removed at any point in time. Thus, an ISP that provides a particular type of service to a particular individual may change over time. For example, a user may be served by an ISP near the user when the user is in her home and by a different ISP when the user travels to a different city. The home ISP will transmit the required information and data to the new ISP so that the user information "follows" the user to a new city, making the data closer to the user and easier to access. In another implementation, a master-server relationship may be established between a master ISP that manages information for a user and a server ISP that directly interfaces with the user under the control of the master ISP. In another embodiment, data is transmitted from one ISP to another ISP as the client moves around the world, so that the ISP that is better positioned to provide services to the user becomes the ISP that provides those services.

The ISP 570 includes an Application Service Provider (ASP)572 that provides computer-based services to customers over a network. Software provided using the ASP model is sometimes also referred to as on-demand software or software as a service (SaaS). A simple form of providing access to a particular application (such as customer relationship management) is to use a standard protocol (such as HTTP). The application software resides on the vendor's system and is accessed by the user through a web browser using HTML, through dedicated client software provided by the vendor, or through other remote interfaces, such as a thin client.

Services provided over a wide geographic area typically use cloud computing. Cloud computing is a computing approach in which dynamically extensible and often virtualized resources are provided as services over the internet. Users need not become experts in the technical infrastructure that support their "cloud". Cloud computing can be divided into different services, such as infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Cloud computing services typically provide common business applications that are accessed online from a web browser, while software and data are stored on servers. Based on the way the internet is depicted in a computer network diagram, the term cloud is used as a metaphor for the internet (e.g., using servers, storage, and logic), and is an abstraction of the complex infrastructure that it hides.

In addition, the ISP 570 includes a Game Processing Server (GPS)574 that is used by game clients to play single-player and multi-player video games. Most video games played over the internet are run through a connection to a game server. Typically, games use a dedicated server application that collects data from players and distributes it to other players. This is more efficient and effective than a peer-to-peer arrangement, but it requires a separate server to host the server application. In another embodiment, the GPS establishes communication between players, and the players' respective game playing devices exchange information without relying on a centralized GPS.

The dedicated GPS is a server that runs independently of the client. Such servers typically run on dedicated hardware located within the data center, providing more bandwidth and dedicated processing power. For most PC-based multiplayer games, a dedicated server is the preferred method of hosting the game server. Massively multiplayer online games run on dedicated servers, usually hosted by software companies that own game names, allowing them to control and update content.

Broadcast Processing Server (BPS)576 distributes audio or video signals to viewers. Broadcasting to a very small range of viewers is sometimes referred to as narrowcasting. The last station of a broadcast distribution is how the signal reaches the listener or observer and it may travel from the air to the antenna and receiver as a radio or television station or may travel through a workstation or directly from the network via cable television or cable broadcast (or "wireless cable"). The internet can also bring broadcasts or television to recipients, allowing sharing of signals and bandwidth, particularly by multicasting. Historically, broadcasts have been defined by geographic area, such as national broadcasts or regional broadcasts. However, with the popularity of the fast internet, broadcasts are not defined geographically, as content can reach almost any country in the world.

The Storage Service Provider (SSP)578 provides computer storage space and related management services. The SSP also provides periodic backup and archiving. By providing storage as a service, users can order more storage as needed. Another major advantage is that SSPs include backup services and if a hard drive of a computer fails, a user will not lose all of their data. Furthermore, multiple SSPs may have full or partial copies of user data, allowing users to access data in an efficient manner regardless of where the user is located or the means used to access the data. For example, a user may access personal files in a home computer as well as a mobile phone (as the user moves).

The communications provider 580 provides connectivity to the user. One type of communication provider is an Internet Service Provider (ISP), which provides access to the internet. The ISP connects its customers using data transmission technologies suitable for delivering internet protocol datagrams, such as dial-up, DSL, cable modem, fiber optic, wireless, or dedicated high-speed interconnects. Communication providers may also provide messaging services such as email, instant messaging, and SMS messaging. Another type of communication provider is a Network Service Provider (NSP), which sells bandwidth or network access by providing direct backbone access to the internet. Network service providers may include telecommunications companies, data carriers, wireless communication providers, internet service providers, cable operators providing high speed internet access, and the like.

A data exchange 588 interconnects several modules within ISP 570 and connects those modules to users 582 via a network 586. Data exchange 588 may cover a small area where all modules of ISP 570 are very close together, or may cover a large geographic area when different modules are geographically dispersed. For example, the data exchange 588 may comprise a fast gigabit ethernet (or faster gigabit ethernet) within a cabinet of a data center, or an intercontinental virtual area network (VLAN).

User 582 accesses the remote service using a client device 584 that includes at least a CPU, memory, display, and I/O. The client device may be a PC, mobile phone, notebook computer, tablet computer, gaming system, PDA, or the like. In one embodiment, the ISP 570 identifies the type of device used by the client and adjusts the communication method employed. In other cases, the client device accesses ISP 570 using standard communication methods (such as html).

Embodiments of the present disclosure may be practiced with various computer system configurations, including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like. The present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wired or wireless network.

In view of the above-described embodiments, it should be appreciated that the present disclosure may employ various computer-implemented operations involving data stored in computer systems. The operations are those requiring physical manipulations of physical quantities. Any of the operations described herein that form part of the present disclosure are useful machine operations. The present disclosure also relates to an apparatus or device for performing these operations. The apparatus may be specially constructed for the required purposes, or it may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.

The present disclosure may also be embodied as computer readable code on a computer readable medium. Alternatively, the computer readable code may be downloaded from a server using the data exchange interconnect described above. The computer readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of the computer readable medium include hard disk drives, Network Attached Storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-R, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer readable medium may include a computer readable tangible medium distributed over a network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the method operations are described in a particular order, it should be understood that other housekeeping operations may be performed between the operations, or the operations may be adjusted so that they occur at slightly different times, or the operations may be distributed in a system that allows processing operations to occur at various intervals associated with the processing, as long as the processing of the overlay operation is performed in a desired manner.

Although the foregoing disclosure has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the disclosure is not to be limited to the details given herein, but may be modified within the scope and equivalents of the described embodiments.