U.S. Pat. No. 11,219,824

DETAILED DESCRIPTION

Cloud gaming is a service wherein a client initiates a gaming session with a game server and a connection is negotiated between the client and the game server. The game server executes the appropriate game application and renders the graphics. Rendered graphics are streamed to the client in real-time over the connection, and user inputs are streamed from the client to the game server in real-time.

A gaming cloud gaming service provider typically employs many servers capable of providing gaming cloud gaming streams to many clients. Each server typically requires a unique public Internet protocol (IP) address, which introduces significant cost and security implications. Each server is essentially the same and can host any gaming cloud gaming session that any other can host. The gaming cloud gaming system typically exposes a single end-point with statically configured routes to a server. The statically configured routes employ the same port values among each of the servers. As gaming session requests are received from clients, the end-point directs the requests to the appropriate port for that particular service and to a server with available bandwidth.

It is realized herein that gaming cloud gaming systems can use dynamically configured reverse proxies to create and break routes dynamically as demand for gaming sessions and connections fluctuates. A gaming session request arrives at a provision manager, which then instructs a reverse proxy to set up connections to a game server. The provision manager is the entry point that is always available to clients through the Internet. The provision manager monitors the capacity of the gaming cloud gaming system and, as realized herein, can bring reverse proxy nodes online and offline to meet demand. An instructed reverse proxy node then randomly assigns as many ports as are necessary for the gaming session. (“Random” is defined broadly to include true random, pseudorandom and quasirandom.)

Dynamic port allocations allow various games having different demands to be efficiently allocated to game servers of varying capability. “Dynamic” refers to configuring processes on-demand for a particular gaming session between a particular client and a particular server. When the gaming session ends, the processes drop the configuration, thereby closing any associated connections. The processes then either terminate or are reconfigured for another gaming session. The dynamically configurable proxies effectively form a software defined network (SDN) such that, under zero load, no game servers are reachable through the reverse proxy. Conversely, static routes can introduce latency issues because of the varying performance of the game servers running various game applications. Static routes require large numbers of open ports on the game servers to support the variety of bandwidth demands. Each of the game servers uses the same port values for certain games, certain processes, or certain connections. It is realized herein that static routes result in unbalanced loads on the game servers. By dynamically adding and removing routes, the load on the game servers can be more intelligently distributed among the game servers.

It is further realized herein that the ports should only remain open for the duration of the gaming session. Once the gaming session ends, the used ports are closed through the reverse proxy. Each new session gets one or more randomly selected ports and potentially a different server. This makes it difficult to attack the game servers, because no connection is persistent. Additionally, it is realized herein that, the reverse proxy should examine packets and only pass those that originate from the appropriate source IP address, which would be the IP address for the game server or the client.

Once the provision manager assigns the gaming session to a particular reverse proxy node and a game server, and the routes are established, the provision manager then responds to the initial gaming session request from the client, instructing it to use the connections, which are identified by an IP address for the reverse proxy and port numbers. To establish a connection, the reverse proxy evaluates rules to route the initial packets. The rules are essentially a configuration of a network address translation (NAT) process, such as IPTables. A (NAT) process routes packets. The reverse proxy uses the rules dynamically to instruct the NAT process how to route packets based on its instructions from the provision manager.

It is realized herein that the NAT process can be configured such that packet processing can be completed in logarithmic time. Initial packets that create new connections to game servers through the reverse proxy are routed by the NAT process. Certain NAT processes utilize a binary tree, or “B-Tree,” rule structure wherein the leaf nodes contain the actual NAT rules that determine the NAT fate of a packet. It is realized herein that branches can be made based on whether a packet's associated destination port lies within a configurable defined range. Once packets have been initially routed, the NAT process tracks the connection internally and routes subsequent packets over that connection automatically in constant time.

It is realized herein that the time needed to route a packet tends to be constant among all sessions, all servers, and all ports. The constant allows a non-degraded real-time gaming cloud gaming service.

It is also realized herein that such a gaming cloud gaming system can gain fault tolerance by having multiple dynamically configurable reverse proxy processes. If a connection is disrupted or a reverse proxy process faults, the provision manager can bring another reverse proxy process online, new ports are assigned, and the connection between the game server and the client is reestablished through the new reverse proxy.

FIG. 1is a block diagram of one embodiment of a gaming cloud gaming system, system100. System100includes clients110, reverse proxy nodes120, game servers130, a provision manager150. Provision manager150, reverse proxy nodes120, and game servers130constitute “the gaming cloud” as far as clients110are concerned. Clients110gain access to the gaming cloud through the Internet140. Clients110can include a variety of computing devices, including: desktop personal computers (PCs), laptop PCs, handheld game devices, mobile phones, Smartphones, tablet computers, and others. Clients110generally need the capability to transmit and receive game data from the gaming cloud through the Internet140. Clients110are also capable of displaying graphics rendered on the gaming cloud and streamed to them.

Game servers130are responsible for executing game applications and rendering game graphics. Game servers130typically include many server computers to provide the game streams to connecting clients. Reverse proxy nodes120proxy for game servers130. Reverse proxy nodes120also allow the distribution of gaming sessions among game servers130according to available processing bandwidth. Reverse proxy nodes120are typically individual proxy processes executing on one or more computers on the gaming cloud network. Reverse proxy nodes120are brought online and offline according to demand from clients110. Reverse proxy nodes120have their own IP addresses and prevent direct access to game servers130from the Internet140. Packetized game data originating at clients110are received by reverse proxy nodes120over the Internet140. The packets use the IP addresses of reverse proxy nodes120as the destination address. Reverse proxy nodes120then route the game data to the appropriate server for each gaming session of clients110. Similarly, game data originating at game servers130are transmitted to reverse proxy nodes120, where the packets are then routed toward the appropriate clients.

Provision manager150is a persistent process executing on a computer on the gaming cloud network. Clients110generate gaming session requests to initiate gaming sessions. Provision manager150is configured to receive the gaming session requests via reverse proxy nodes120and coordinate the various elements of the gaming cloud to establish the necessary connections for each gaming session. Provision manager150determines which of game servers130should host a given gaming session and assigns a node of reverse proxy nodes120. The hosting game server determines what connections are needed for the gaming session. For example, one game session may require three user datagram protocol (UDP) connections and four transmission control protocol (TCP) connections. Each different game application may use different combinations of connections. Game servers130pass the connection needs back to provision manager150. Provision manager150instructs the assigned reverse proxy node to add the necessary routes for those connections. Reverse proxy nodes120are configured to randomly select port values to use for their respective gaming sessions. Those ports are then opened on the assigned reverse proxy node. The selected ports are then communicated back to provision manager150and to game servers130. Provision manager150then communicates the connection instructions through reverse proxy nodes120to clients110. The connection instructions for a given client include the IP address for the assigned reverse proxy node and the randomly selected port values. Additionally, connection instructions can also include the quantity and type of connections for a given gaming session. Clients110then use the connection information to establish connections to game servers130through the Internet140and reverse proxy nodes120.

FIG. 2is a block diagram of another embodiment of a gaming cloud gaming system, system200. System200includes an entry point220, a client240, a reverse proxy250, and a game server260. Client240initiates a game session by submitting a gaming session request210to entry point220. Gaming session request210includes connection details for client240and identifies the particular game for which the session is request.

Entry point220receives gaming session request210and generates connection instructions230. Entry point220is a persistent process that manages the allocation of gaming cloud resources, particularly the number of reverse proxy nodes and the distribution of gaming sessions among all game servers.

Game server260is one server, typically of many, responsible for executing game applications and rendering graphics to be streamed to one or more clients for their respective gaming sessions. Game server260uses connection instructions230to determine what connections are necessary for the requested gaming session. The quantity and types of connections are added to connection instructions230.

Reverse proxy250is a dynamically configurable proxy for game server260. Reverse proxy250uses connection instructions230randomly to select ports for the necessary connections to game server260. The randomly selected ports are also added to connection instructions230.

Client240is a computing device capable of at least displaying remotely rendered graphics and transmitting and receiving game data to and from game server260. Client240uses connection instructions230, including the IP address of reverse proxy250, the randomly selected ports, and the quantity and types of connections, to establish the necessary connections to game server260through reverse proxy250. Client240transmits initial packets for the necessary connections to reverse proxy250over proxy connections270. Proxy connections270include the quantity and types of connections deemed necessary by game server260in response to connection instructions and gaming session request210. The initial packets from client240are processed by reverse proxy250to establish routes280to game server260. Routes280include a route for each connection to its designated port on game server260. The initial packets for each connection are identifiable by their originating IP address, which is that of client240, the IP address of reverse proxy250, and the destination port on game server260.

Reverse proxy250employs a binary-tree rule structure to carry out NAT. NAT is generally an independent process configured by reverse proxy250. In certain embodiments, reverse proxy250employs a separate binary tree for TCP and UDP connections. A packet begins processing at the root of the binary tree and moves down the branches toward leaf nodes. In the binary tree, the leaf nodes contain NAT rules for determining the NAT fate of a packet. The branches are made based on whether the destination port of a packet falls within a configurable range for the branch, eventually arriving at a leaf node for a particular port on game server260. Once the route is established by processing the initial packets through the binary tree, the route is tracked internal to the NAT process and subsequent packets associated with the connection are routed automatically. This proxy routing provides for a determinable worst-case routing time, which can then be used to determine whether additional reverse proxy nodes should be brought online or if existing reverse proxy nodes can be taken offline. Additionally, the routing time is held constant among all clients, servers, and reverse proxy nodes.

When the gaming session terminates, whether voluntarily by client240or by broken connections in proxy connection270or routes280, the ports randomly selected by reverse proxy250are closed and the routes removed between reverse proxy250and game server260. If client240were to initiate a new gaming session or if entry point220were to initiate a recovery of an involuntary termination of proxy connections270or routes280, entry point220would generate new connection instructions to a different reverse proxy node and possibly a different game server. The new reverse proxy node would randomly select new ports to be used for the gaming session. Client240would then use the new connection instructions to reestablish its connections to the game server.

FIG. 3is a flow diagram of a method for initiating a gaming session. The method begins in a start step310. A gaming session is initiated by a client request. In a receiving step320, the gaming session request is received and a dynamically configurable reverse proxy is assigned to the gaming session. The reverse proxy has an IP address and proxies for at least one game server. A particular game server and at least one port on that game server are selected in a selection step330. The assigned reverse proxy forms a route from the reverse proxy node to the selected port on the game server. Additionally, given the selected port value, NAT rules can be formed for routing the initial packets of a particular connection.

Continuing the embodiment ofFIG. 3, in a communication step340, the at least one port value selected in selection step330and the IP address of the dynamically configurable reverse proxy are transmitted back to the originating client. The client uses this information to establish at least one connection with the game server through the dynamically configurable reverse proxy. The client transmits the initial packets for the connections to the reverse proxy, which uses a NAT process to carry out the initial routing. There are a variety of NAT schemes available, e.g., IPTables, for routing the initial packets. In certain embodiments the NAT process employs a binary-tree rule structure based on the destination port for each of the initial packets. A binary-tree structure based on the port value allows for packet routing in logarithmic time.

Ultimately, the gaming session will terminate. A gaming session can end in a variety of ways, the most basic of which is the client simply voluntarily ending the session. Alternatively, a gaming session may be involuntarily ended in the event of a connection interruption or a process fault or failure in the game server or one of its proxies. Upon termination of the gaming session, any routes added for that gaming session are removed and the associated ports closed on the reverse proxy node. The method then ends in an end step350.

Those skilled in the art to which this application relates will appreciate that other and further additions, deletions, substitutions and modifications may be made to the described embodiments.