U.S. Pat. No. 11,071,909

DETAILED DESCRIPTION

Example embodiments of this disclosure describes methods, apparatuses, computer-readable media, and systems for matching players for online gaming. In example embodiments of the disclosure, players who play an online game may be matched based at least in part on optimizing game quality factors (GQFs) of virtual games, in accordance with the disclosure herein. Virtual games may be games that are iteratively being optimized for predicted quality if the virtual game is instantiated as an online game. The optimization may be performed by adding and/or subtracting players thereto.

It should be noted that players that are to be placed in an online game, when matched according to balancing a skill level between all of the player allocated to a single game, may not result in the most enjoyable game for the players. Indeed, there may be various factors associated with the game itself, rather than the individual matches that may result in an enjoyable game. Embodiments of the disclosure, as described herein, optimizes factors related to a quality of a virtual game, before that virtual game is instantiated into an online game played by the players allocated to the game. In one sense, a player is added to a virtual game only if the addition of that player improves the virtual game to which he or she is added. Therefore, the mechanism for matchmaking, as disclosed herein, may improve the game playing experience of the players.

When comparing players in a one-to-one fashion, there may be a large number of comparisons when deciding which players to allocate to a particular online game. In fact, if a particular player is compared across each group of the other players that can be allocated to a particular online game, the analysis may approach an n-factorial (n!) order of complexity. The various embodiments for forming online games, as discussed herein, may allocated players to an online game in an iterative evaluation that may use comparisons on the order of n-squared (n2), or less. Thus, the mechanism of player matchmaking, as disclosed herein, may be more efficient than other mechanisms of player matchmaking, resulting in more efficient usage of computer resources.

According to example embodiments, players who wish to play an online game may be filtered into various bins and then within the bins may be iteratively packed into virtual games according to a set of game quality factors (GQFs) evaluated for the virtual game. After iterating through the game packing process, a game may be instantiated as an online game to be hosted by one or more game system(s) for players to play via their client devices.

The filtering mechanism may be implemented to minimize constraints, since all players who wish to play a particular game may not be matched with all other players who wish to play that same game. For example, different players may wish to play in different modes of the game that are not permitted to play together. As another example, in games where a player may choose a particular role, players may be filtered according to those roles. For example, in an online soccer game, a particular player may wish to play goalie, and a particular game may only have two goalies. Thus, in some cases, players may be filtered such that role preferences of the players may be catered to, without causing constraints in the online game. Still further examples of filtering players may include allocating players to different bins according to their geographic location or according to which geographically distributed data center houses the game system(s) they may connect to for enabling the online game.

The game packing process, according to example embodiments, may entail creating one or more virtual games, and iteratively adding a player to one of the virtual games and determining, based at least in part on the GQFs, whether the added player is to remain in the virtual game. For example, for a particular virtual game (e.g., a game being evaluated prior to actualization of the game on one or more game systems), a set of GQF values may be evaluated based at least in part on the players that are currently allocated to the virtual game. Next, a new player who wishes to play the game may be added to the virtual game, and a new set of GQF values may be determined with the new player added to the virtual game. The old set of GQF values without the new player added may be compared to the new set of GQF values with the new player added. If the addition of the new player improves the GQF of the virtual game (e.g., the new GQF values are an improvement over the old GQF values) then the new player may remain in the virtual game. However, if the addition of the new player does not result in an improvement according to the new GQF values, then the new player may be removed from the virtual game and evaluated in a similar manner for a different virtual game.

Thus, when a player is added to a virtual game, according to example embodiments, the player remains in that virtual game if there is an improvement in the virtual game according to GQF considerations. If there is not an improvement, then the player is evaluated in another virtual game, and again assessed according to a change in the GQF values of that virtual game by addition of the new player. In this way, the player may be added to the virtual game where he/she effects a maximal absolute improvement in the GQF values of that virtual game.

The GQFs may include any variety of metrics including, but not limited to, fullness of the game, skill balance of players, skill balance of teams, network connectivity speed balance, combinations thereof, or the like. In example embodiments, the GQFs may have a hierarchy or a ranked level of importance. For example, fullness of the game may be a more important factor to optimize than skill balance of players.

In example embodiments, the GQF values of virtual games may be determined based at least in part on parameters and/or metrics associated with the players that are included in the virtual games. The parameters associated with the players may be associated with the respective players in a player datastore accessible by the matchmaking system(s). For example, based at least in part on historical game play, each player may have a corresponding skill score. A player who exhibits a relatively higher level of skill compared to another player may have a higher skill score than the other player. The skill score of each of the players may be associated with the respective players by way of a user account for each of the players. The user account for each of the players may associated various information about the respective players, including his/her skill score, and may be stored and accessed by one or more matchmaking system(s).

The matchmaking system(s), according to example embodiments, may further access skill scores of the players who wish to play an online game. The skill score for each player, in some example embodiments, may be for the particular online game and/or game mode of play of the online game. Once the skill score is determined, such as from a player datastore, the one or more matchmaking system(s) may use the skill scores, or any other parameters associated with a player to be packed into a virtual game, to determine GQFs associated with virtual games being evaluated. In other words, parameters associated with a current roster of players of a virtual game may be used to determine the GQF values of that virtual game.

As a new player is evaluated for addition in a particular virtual game, the matchmaking system(s) may use parameters associated with a roster of players of a virtual game, including the new player, to determine the GQF values of that virtual game. Thus, the matchmaking system(s) in this way may compare the GQF values with and without the new player to determine if the new player improves the GQFs for the particular virtual game.

In some cases, addition of a new player may push the number of players allocated to a particular virtual game over a maximum number of allowed players. In this case, when a new player is evaluated in the particular virtual game, a pre-existing player may need to be removed from the particular virtual game. In some example embodiments, the matchmaking system(s) may determine a player to remove from the virtual game, the removal of whom would have no adverse impact on the GQF values of the particular virtual game. Thus, the pre-existing players to be replaced may be based at least in part on which player's replacement will improve the game the most.

According to example embodiments, virtual games may be instantiated as online games when a timer associated with the virtual game expires. This timer may be a timer associated with a player waiting to play the online game. For example, if a player wait timer expires at 30 seconds, and a player is currently in a virtual game and has been waiting for 30 seconds, then that virtual game in which he or she belongs may be instantiated by the matchmaking system(s)150. In example embodiments, a virtual game may be instantiated to an online game if the game has been iteratively evaluated for a threshold number of iterations without an improvement in the GQF values of the virtual game. The matchmaking system(s)150may instantiate a virtual game by sending a roster of players of the virtual game to be instantiated to a game system(s) for hosting the online game.

Certain implementations and embodiments of the disclosure will now be described more fully below with reference to the accompanying figures, in which various aspects are shown. However, the various aspects may be implemented in many different forms and should not be construed as limited to the implementations set forth herein. It will be appreciated that the disclosure encompasses variations of the embodiments, as described herein. Like numbers refer to like elements throughout.

FIG. 1illustrates a schematic diagram of an example environment100with game system(s)110and steaming system(s)120to enable online gaming via game client device(s)130and streaming client device(s)140, in accordance with example embodiments of the disclosure.

The example environment100may include one or more non-streaming player(s)134(1),134(2),134(3), . . .134(N), hereinafter referred to individually or collectively as non-streaming player(s)134, who may interact with respective game client device(s)130(1),130(2),130(3), . . .130(N), hereinafter referred to individually or collectively as game client device(s)130via respective input device(s)132(1),132(2),132(3), . . . ,132(N), hereinafter referred to individually or collectively as input devices132.

The example environment100may further include one or more streaming player(s)144(1),144(2), . . .144(N), hereinafter referred to individually or collectively as streaming player(s)144, who may interact with respective streaming client device(s)140(1),140(2), . . .140(N), hereinafter referred to individually or collectively as streaming client device(s)140via respective input device(s)142(1),142(2), . . . ,142(N), hereinafter referred to individually or collectively as input devices142.

The game client devices130may be configured to render content associated with the online game to respective non-streaming players134. This content may include video, audio, haptic, combinations thereof, or the like content components. The game client device(s)130may receive game state information from the one or more game system(s)110that may host the online game played by the player(s)134,144of environment100. The game client device(s)130may use the game state information to render current events of the online game as content. The game state information may be received repeatedly and/or continuously and/or as events of the online game transpire. The game state information may be based at least in part on the interactions that each of the player(s)134,144have via their input device(s)132,142responsive to events of the online game hosted by the game system(s)110.

As events transpire in the online game, the game system(s)110may update game state information and send that game state information to the game client device(s)130. For example, if the players134,144are playing an online soccer game, and the player134,144playing one of the goalies move in a particular direction, then that movement and/or goalie location may be represented in the game state information that may be sent to each of the game client device(s)130for rendering the event of the goalie moving in the particular direction. In this way, the content of the online game is repeatedly updated throughout game play. When the game client device(s)130receive the game state information from the game system(s)110, the game client device(s)130may render updated content associated with the online game to its respective player134. This updated content may embody events that may have transpired since the previous state of the game (e.g., the movement of the goalie).

The game client device(s)130may accept input from respective non-streaming players134via respective input device(s)132. The input from the non-streaming players134may be responsive to events in the online game. For example, in an online basketball game, if a non-streaming player134sees an event in the rendered content, such as an opposing team's guard blocking the paint, the non-streaming player134may use his/her input device132to try to shoot a three-pointer. Intended action by the non-streaming player134, as captured via his/her input device132may be received by the game client device130and sent to the game system(s)110.

The game client device(s)130may be any suitable device, including, but not limited to a Sony Playstation® line of systems, a Nintendo Wii® line of systems, a Microsoft Xbox® line of systems, any gaming device manufactured by Sony, Microsoft, Nintendo, or Sega, an Intel-Architecture (IA)® based system, an Apple Macintosh® system, a netbook computer, a notebook computer, a desktop computer system, a set-top box system, a handheld system, a smartphone, a personal digital assistant, combinations thereof, or the like. In general, the game client device(s)130may execute programs thereon to interact with the game system(s)110and render game content based at least in part on game state information received from the game system(s)110. Additionally, the game client device(s)130may send indications of player input to the game system(s)110. Game state information and player input information may be shared between the game client device(s)130and the game system(s)110using any suitable mechanism, such as application program interfaces (APIs).

The game system(s)110may receive inputs from various players134,144and update the state of the online game based thereon. As the state of the online game is updated, the state may be sent the various game client system(s)130for rendering online game content to players134. In this way, the game system(s)110host the online game.

Additionally, the game state information may also be sent by the game system(s)110, and on behalf of the streaming player(s)144, to the streaming system(s)120. In example embodiments, the streaming system(s)120may receive online game state information from the game system(s)110, render the content of the online game for the streaming players144, and stream the rendered online content to individual ones of the streaming client devices140. The streaming system(s)120, therefore, may be able to use the game state data from the gaming system(s)110to render the online game for the streaming players144, without the streaming client device(s)140having to render the content associated with the online game. The streaming players144may experience (e.g., view, hear, feel, etc.) the content associated with the online game on their streaming client devices140based at least in part on streaming content as received from the streaming system(s)120.

The streaming client device(s)140may be any of the devices as discussed with respect to the game client device(s)130, except the streaming client devices operate in a manner where game content is streamed from the streaming system(s)120, rather than locally rendering at the streaming client device(s)140. As a result, the streaming client device(s)140may include functionality and/or software to present streaming content to its respective streaming player(s)144.

The example environment100may further include matchmaking system(s)150to match players134,144who wish to play the same game and/or game mode with each other. Players134,144may be matched according to game quality factors (GQFs) associated with virtual games that the matchmaking system(s)150evaluate and/or optimize. The matchmaking system(s)150may add and/or remove players134,144from virtual games being optimized and evaluating those games according to changes in GQF values as players are added and/or removed.

The matchmaking system(s)150may receive an indication from the game system(s)110and/or the streaming system(s)120of players134,144who wish to play an online game. The matchmaking system(s)150may access information about the players134,144who wish to play a particular game, such as from a player datastore152. The information about the players134,144may include skill scores and/or other information that may be used to determine GQF values of virtual games being optimized. A player's skill score in a particular game may be an estimate of a player's expected performance in that game based at least in part on historic game performance data.

The matchmaking system(s)150in example embodiments, may continue to create, optimize, and/or instantiate virtual games as new players134,144wish to play join the optimization process. When a timer measuring how long a player134,144has been waiting to play an online game expires, the virtual game in which that player is at that time assigned may be instantiated by the matchmaking system(s)150. In some example embodiments, if a virtual game has been iterated a threshold number of times without improvements in the virtual game's GQF values, then that game may be instantiated.

FIG. 2illustrates a flow diagram of an example method200to form one or more online games by selecting players for each of the games, in accordance with example embodiments of the disclosure. The method200may be performed by the matchmaking system(s)150, individually or in cooperation with one or more other elements of the environment100ofFIG. 1.

At block202a plurality of players who wish to play an online game may be filtered into separate bins. This filtering process may be performed by the matchmaking system(s), in example embodiments, to sort players according to groups of players who can play with each other. In other words, this filtering process may reduce constraints in matchmaking online game players.

The filtering process of block202may be according to any suitable set of criteria, such as individual players' choices of game modes, individual choices of players' roles, the player's physical location, and/or the quality of players' network connection. When filtered, in example embodiments, players in a particular bin may be compatible for grouping with each other in an online game. In other words, the filtering process of block202may separate players134,144from each other into separate bins, if those players are not compatible for grouping together in the same online game.

At block204, for each of the bins, virtual games may be iteratively packed with players from the bin to form final game rosters based at least in part on game quality factors (GQFs) corresponding to each of the games. This process may involve placing a new player134,144added to a bin into a virtual game and determining if the addition of that player results in an improvement in the GQF values of that virtual game. If there is an improvement, then the player may remain in that virtual game. However, if there is no improvement in the GQF values of the virtual game, then the new player may be removed from that virtual game and evaluated within another virtual game. In this way, the new player may be placed in the first virtual game within which his or her placement results in an improvement in the GQF values of that virtual game.

In example embodiments, the iterative packing of players into games may be stopped according to any suitable stopping criteria. For example, the iterative packing and evaluation of a particular virtual game may be stopped if a threshold number of iterations have been performed without an improvement and/or appreciable improvement in the GQF values associated with a particular game. Additionally, or alternatively, the iterative packing and evaluation of a particular virtual game may be stopped if a timer associated with a player134,144assigned to the virtual game expires. The expiration of a player's timer may indicate that the player has been waiting for a maximum allowed period of time to be placed into an online game. As a result, when the player's timer expires, the player is to be placed in a final game roster and, as a result, his or her virtual game is to be instantiated.

At block206, online games may be instantiated, based at least in part on the final game rosters. Instantiating the virtual game may entail the matchmaking system(s)150sending a final game roster of players134,144to the game system(s)110.

In some embodiments, the operations of method200may be performed out of the order presented, with additional elements, and/or without some elements. Some of the operations of method200may further take place substantially concurrently and, therefore, may conclude in an order different from the order of operations shown above.

FIG. 3illustrates a flow diagram of an example method300by which a new player who wishes to play an online game is matched to a virtual game prior to instantiating the online game, in accordance with example embodiments of the disclosure. The method300may be performed by the matchmaking system(s)150, individually or in cooperation with one or more other elements of the environment100ofFIG. 1.

At block302, one or more virtual games may be generated by assigning players from a bin to the one or more virtual games. Since all of the players are from the same bin, the players are compatible with each other when matched into the one or more virtual games. This process may be at the start-up of the matchmaking process when there are no pre-existing virtual games yet.

At block304, a new player to assign to one of the one or more virtual games may be identified. The identification of this new player may be based upon an indication from one or both of the game system(s)110and/or the streaming system(s)120indicating that the new player wishes to join an online game. This new player may also be filtered into the same bin from which the one or more virtual games of block302are generated by a process such as that described in conjunction with block202of method200ofFIG. 2.

At block306, a virtual game from the one or more virtual games may be selected to which the new player may be added. In example embodiments, the selection of the virtual game may be by selecting the virtual game that has a player timer that is closest to expiry. In other example embodiments, the virtual game may be selected by any other suitable criteria, such as by selecting the virtual game that was first created or selecting the virtual game randomly.

At block308, a set of game quality factor (GQF) values associated with the selected virtual game may be determined. These GQF values may be determined using one or more functions or algorithms that score the virtual game according to each of the factors included in the GQFs. There any be any number of GQFs that may be used for evaluating the virtual games. For example, GQFs may include fullness of the game, skill balance of players, skill balance of teams, network connectivity speed balance, or the like. A value may be determined for each of the GQFs.

Fullness of the game factor value may be based at least in part on the number of players134,144assigned to a virtual game relative to the maximum number of allowed player in the game. In general, it may be desirable to completely fill an online game. In some alternative cases, it may be desirable to have an online game at a target fullness number of players other than the maximum allowed number of players. Thus, in example embodiments, a relatively high value may be generated for the fullness factor when a virtual game is at or near a target and/or optimal number of players.

The skill balance of players factor value may be based at least in part on any suitable measure of spread in the skill score of players in a virtual game. For example, the two highest skill scores may be compared and the tighter the spread, the greater the skill balance of players factor value for a virtual game may be. Other measures of spread that the matchmaking system(s)150may determine may include mean, median, mode, mean average deviation, interquartile range, standard deviation, variance, or the like of the skill scores of the players assigned to a virtual game. In example embodiments, a relatively high value may be generated for the skill balance of players factor value when there is relatively low spread in the skill score for players assigned to a virtual game.

The skill balance of teams factor value may be based at least in part on any suitable measure of difference in the aggregate skill scores of players on two different teams in a virtual game. This GQF factor may be relevant in online games that are team-centric, such as, for example, FIFA 18® by Electronic Arts, of Redwood City, Calif. In some cases, the skill scores of all the players on each team may be summed and compared to each other. In other cases, the one or two best players' skill scores on each team may be compared to get a measure of team differences in skill. In still other cases, an average skill score of one team may be compared to an average skill score of other team(s) for the virtual game. Indeed, any suitable measure of team skill balance may be used to generate the skill balance of teams factor value. In example embodiments, a relatively high value may be generated for the skill balance of teams factor value when there is relatively low difference in the skill levels of teams of a virtual game.

The network connectivity speed balance factor value may be based at least in part on any suitable measure of network speed that players experience during online gameplay, particularly the streaming player144of a virtual game. Since network latency and/or network bandwidth constraints may cause variations in times when players of an online game may experience events of the online game, and may influence their reaction times to those events, it may generally be desirable to have minimal variations in network speed for players grouped together in an online game. The network speeds for online players may be received by the matchmaking system(s)150from the streaming system(s)120and/or the game system(s)110. In some cases, the matchmaking system(s)150may request network latency and/or network bandwidth information from the streaming system(s)120and/or the game system(s)110. The streaming system(s)120and/or game system(s)110may use any suitable mechanism to measure network speeds between themselves and either a streaming client device140or a game client device130, such as a ping test. The matchmaking system(s)150may use any variety of metrics that measure spread in network speeds experienced by players134,144, such as a variance, a standard deviation, an interquartile range, or the like. In example embodiments, a relatively high value may be generated for the network connectivity speed balance factor value when there is relatively low spread in the network speeds experienced by players of a virtual game.

At block310, the new player may be added to the selected virtual game with player replacement if needed. At block312, a new set of GQF values for the virtual game with the new player may be determined. This determination may be similar to the determination of the GQF values at block308, except parameters for the new player is used in determining the new set of GQF values for the virtual game.

At block314, it may be determined if the GQF values improved. In other words, the matchmaking system150may determine if the new set of GQF values after adding the new player to the selected virtual game is an improvement over the GQF values prior to adding the new player to the selected virtual game. Individual factor values of the set of GQF values may be compared to corresponding factor values of the new set of GQF values to determine if the new set of GQF values are an improvement from prior to adding the new player to the selected virtual game. In some example embodiments, a hierarchy of the GQFs, where some GQFs are more important than others, may be considered when determining if the new set of GQF values are an improvement over the previous set of GQF values.

At block314, if it is determined that the GQF values improved by adding the new player to the selected virtual game, then the method300may proceed to block316where the new player is kept in the selected virtual game. On the other hand, if it is determined at block316that the GQF values did not improve by including the new player in the selected virtual game, then the method300may return to block306to select a new virtual game from the one or more virtual games in which to place the new player for evaluation of the newly selected virtual game with inclusion of the new player. In this way, the new player may be evaluated in a selected one of the one or more virtual games until the new player fits into a virtual game that improves due to the new player's inclusion in that virtual game.

It should be noted that some of the operations of method300may be performed out of the order presented, with additional elements, and/or without some elements. Some of the operations of method300may further take place substantially concurrently and, therefore, may conclude in an order different from the order of operations shown above.

FIGS. 4A and 4Billustrate flow diagrams of an example method400to pack players into a virtual game using game quality factors (GQFs) and then instantiate the virtual game, in accordance with example embodiments of the disclosure. The method400may be performed by the matchmaking system(s)150, individually or in cooperation with one or more other elements of the environment100ofFIG. 1. In example embodiments, the method400may be an example implementation of blocks204and206of method200ofFIG. 2.

At block402, a new player who wishes to play an online game may be determined. At block404, one or more metrics corresponding with the new player may be determined. These one or more metrics may include any variety of descriptors of the new player such as his or her skills score for the online game that he or she wishes to play. In example embodiments, the matchmaking system(s)150may access a player datastore152to retrieve the one or more metrics corresponding with the new player. These one or more metrics for the new player may be accessed from the player datastore152using the new player's identifier.

At block406, a virtual game may be selected from among one or more existing virtual games to be packed for the online game. In some cases, the selection of the virtual game may be based upon a game that may have players with timers that are relatively close to expiry. In other cases, other mechanisms may be used to select the virtual game from among the one or more existing virtual games.

At block408, an initial set of GQF values associated with the virtual game may be determined based at least in part on one or more metrics associated with the current players associated with the virtual game. This determination of GQF values may be based at least in part on the one or more metrics corresponding to the new player. At block410, the new player may be added to the virtual game. At block412, it may be determined if there is more than a maximum allowed number of players in the virtual game. The matchmaking system may compare the number of players in a virtual game to data about the maximum allowed number of players in that online game.

If it is determined that there is not more than a maximum number of players in the virtual game at block412, then the method400may proceed to block422. On the other hand, if it is determined that there is more than a maximum number of players in the virtual game, then the method400may proceed to block414where an existing player is removed from the virtual game and the removal of whom is least likely to reduce a value of the one or more game quality factors. In example embodiments, a player whose removal would have no adverse impact on the GQF values of the virtual game may be removed from the virtual game. At block416, it may be determined if the removed existing player can be added to another existing virtual game.

If at block416, it is determined that the removed existing player can be added to another existing virtual game, then at block418the removed existing player may be added to another one of the one or more existing virtual games. If, on the other hand, it is determined that block416that the removed existing player cannot be added to another existing virtual game, then at block420a new virtual game may be created and the removed existing player may be added to the new virtual game.

From blocks418or420, the method400may proceed to block422. At block422, a new set of GQF values associated with the virtual game may be determined based at least in part on one or more metrics corresponding with the first player. At block424, it may be determined if the new set of GQF values are an improvement over the initial set of GQF values. If it is determined at block424that the new set of GQF values are not an improvement over the initial set of GQF values, then at block426the existing player may be returned back to the virtual game. This operation may be optional and may only occur if a player was removed at block414. At block428, the new player may be removed from the virtual game. In other words, the processes of block426and428revert the virtual game back to the original set of players prior to the addition of the new player to the virtual game.

If a block424it is determined that the new set of GQF values are an improvement over the initial set of GQF values, then the method400may proceed to block430. From block428, the method400may bifurcate and proceed to block406, where the new player may be added to another virtual game selected from among the one or more existing virtual games. At the same time, the method400may proceed to block430where it may be determined if a timer associated with the virtual game has expired.

If at block430it is determined that a timer associated with the virtual game has expired, then the method400may proceed to block434where the virtual game is instantiated as an online game. If, on the other hand, it is determined that a timer associated with the virtual game has not expired, then the method400may proceed to block432to where it may be determined if the virtual game has been iterated a threshold number of times without improvement in GQF values.

If at block432it is determined that the existing virtual game has been iterated a threshold number of times, then the method400may proceed to block434where the virtual game is instantiated as an online game. If, on the other hand, it is determined at block432that the virtual game has not been iterated a threshold number of times without improvement, then the method400may return to block402where a new player who wishes to play an online game may be identified and the procedures of method400may be repeated.

According to some embodiments, the operations of method400may be performed out of the order presented, with additional elements, and/or without some elements. Some of the operations of method400may further take place substantially concurrently and, therefore, may conclude in an order different from the order of operations shown above.

FIG. 5illustrates a flow diagram of an example method500to determine whether a first set of game quality factors (GQFs) is better than a second set of GQFs for a virtual game, in accordance with example embodiments of the disclosure. The method500may be performed by the matchmaking system(s)150, individually or in cooperation with one or more other elements of the environment100ofFIG. 1. In example embodiments, the method500may be an implementation of the operations of block424of method400ofFIG. 4.

At block502, a rank order of GQFs may be determined. This rank order may determine which of the game quality factors are given more importance in optimizing game packing relative to the other factors of the GQFs. In other words, a lower-ranking factor may not be optimized at the cost of a higher-ranking factor. In this way, a hierarchy of the game quality factors may be considered during the game packing optimization process.

At block504, a first set of GQF values associated with a first virtual game player roster may be identified. At block506, a second set of GQF values associated with a second virtual game player roster may be identified. At block508, a GQF value from the first set of GQF values may be compared with a GQF value from the second set of GQF values, where the compared values correspond to the highest-ranked GQF.

At block510, it may be determined if the GQF value from the second set is greater than the GQF value from the first set. If at block510it is determined that the GQF value from the second set is greater than the GQF value from the first set then the method500may proceed to block520, where it may be determined that the second set of GQF values is an improvement over the first set of GQF values. If, on the other hand, at block510it is determined that the GQF value from the second set is not greater than the GQF value from the first set, then the method500may proceed to block512, where it may be determined if the GQF value from the second set is equal to the GQF value from the first set.

If at block512, it is determined that the GQF value from the second set is not equal to the GQF value from the first set, then the method500may proceed to block518, where it is determined that the second set of GQF values is not an improvement over the first set of GQF values. If, on the other hand, it is determined at block512that the GQF value from the second set is equal to the GQF value from the first set, then the method500may proceed to block514, where it may be determined if there are more GQF values to evaluate.

If it is determined at block514that there are no more GQFs to evaluate, then the method500may proceed to block518, where it is determined that the second set of GQF values is not an improvement over the first set of GQF values. If, on the other hand, it is determined at block514that there are more GQFs to evaluate, then the method500may proceed to block516, where a GQF value from the first set of GQF values is compared with a GQF value from a second set of GQF values, where the compared values correspond to the next highest ranked GQF. The method500may then return to block510to determine if the net highest ranked GQF can determine if the second set of GQF values are an improvement over the first set of GQF values.

According to some embodiments, the operations of method500may be performed out of the order presented, with additional elements, and/or without some elements. Some of the operations of method500may further take place substantially concurrently and, therefore, may conclude in an order different from the order of operations shown above.

FIG. 6illustrates a chart600of an example packing of a virtual game, in accordance with example embodiments of the disclosure.

The chart600demonstrates an example player packing progression in time from T0to T1to T2to T3. At time T0, there may be four different virtual games being optimized prior to instantiation as indicated as virtual game1, virtual game2, virtual game3, and virtual game4. The players assigned to each of the virtual games are shown under the virtual game labels. Additionally, game quality factor (GQF) values for each of the virtual games are shown underneath the listing of players that are currently in each of the virtual games. For example, virtual game2at time T0includes player E, player F, player G, and player H, and has GQF values for fullness at100, skill balance at82, and network quality at77. In this example, the GQF values may be on a scale of 0-100. However, it should be understood that the GQF values may be on any suitable scale (e.g., 0-1, 1-10, 0-1000, etc.).

In this example, GQFs have a hierarchy, where fullness is ranked above skill balance, and skill balance is ranked above network quality. In other words, when evaluating a virtual game, fullness may not be sacrificed in favor of skill balance or network quality. Similarly, skill balance may not be sacrificed in favor of network quality.

At time T1, the matchmaking system(s)150may determine that another player, player M, is to be matched into an online game. At time T2, the player M may be added to virtual game1. In this example, the maximum number of players in any game can be 4. Therefore, by adding player M to virtual game1, one of the existing players from player A, player B, player C, or player D must be removed from virtual game1to accommodate the addition of player M virtual game1. In this case, player B may be removed from virtual game1and placed in virtual game3.

At this point the GQF values for each of the virtual games may be recalculated. This recalculation takes into account the shuffling of players, specifically the addition of player M into virtual game1and player B into virtual game3. Player B may be moved, in example embodiments, based at least in part on a determination that the removal of player B may have the least detrimental effect on virtual game1's GQF values. In this case, the GQF values for virtual game2and virtual game4do not change since there is no change to the player roster for either one of those virtual games. However, virtual game1's GQF values change—while the fullness score does not change, the skill balance score decreases and the network quality score increases.

In this case, since there is a decrease in the skill balance score, without any improvement in the fullness score, player M may not remain in virtual game1. In other words, the addition of player M into virtual game1in favor of player B, does not provide an improvement in the set of GQF values for virtual game1, and therefore the addition of player M to virtual game1is to be unwound. It should be noted that the network quality factor value of virtual game1improved with the addition of player M into virtual game1. However, the network quality factor is ranked less than the skill balance factor, and therefore, in aggregate the set of GQF values for virtual game1diminished due to the addition of player M.

At time equals T3, player M is removed from virtual game1and instead added to virtual game2. As expected, the GQF values for virtual game1revert back to what they originally were at time T0, prior to the addition of player M. When player M is added to virtual game2, a player from virtual game2, in this case player G, is removed and placed in another virtual game, specifically virtual game3. Again, since there is no change to the player roster for virtual game4, the GQF values for virtual game4remain the same as they were at time T2and time T0. However, the GQF values for both virtual game2and virtual game3change relative to what they were at time T2.

For virtual game2, the fullness factor value does not change. However, the skill balance factor value improves, while the network quality factor value decreases slightly. Since the skill balance factor has a higher ranking than the network quality factor, player M is to remain in virtual game2, since the addition of player M improves the set of GQF values for virtual game2. Therefore, player G also remains in virtual game3. This example in chart600is a time slice of the game packing process, and it should be understood that this type of iterative optimization of the GQF values may continue until one or more of the virtual games are instantiated as an online game.

FIG. 7illustrates a flow diagram of an example method700for determining thresholds iterations when packing a virtual game, in accordance with example embodiments of the disclosure. The method700may be performed by the matchmaking system(s)150, individually or in cooperation with one or more other elements of the environment100ofFIG. 1.

At block702, game quality factor (GQF) values may be collected after iterations of virtual games. At block704, it may be determined if the iteration thresholds demonstrate expected improvements. This determination may provide an indication of whether optimizations completed by reaching an iteration limit provide as high GQFs as an optimization that did not end according to the iteration threshold. Alternatively, this determination may provide an indication of a probability that a set of GQFs can improve after a certain number of iterations without improvement.

If at block704it is determined that the iteration thresholds demonstrate the expected improvements, then the method700may continue to block710where the iteration thresholds are continued to be used for the purposes of game packing. However, if it is determined at block704that the iteration thresholds do not demonstrate the expected improvements in GQF values, then the method may continue to block706where new iteration thresholds may be determined. After determining the new iteration thresholds, at block708the new iteration thresholds may be deployed, after which, at block710the iteration thresholds may be used for the purposes of game packing.

It should be noted that some of the operations of method700may be performed out of the order presented, with additional elements, and/or without some elements. Some of the operations of method700may further take place substantially concurrently and, therefore, may conclude in an order different from the order of operations shown above.

FIG. 8illustrates a block diagram of example matchmaking system(s)150that may provide matchmaking for online games based on game quality factors (GQFs), in accordance with example embodiments of the disclosure.

The matchmaking system(s)150may include one or more processor(s)800, one or more input/output (I/O) interface(s)802, one or more network interface(s)804, one or more storage interface(s)806, and computer-readable media810.

In some implementations, the processors(s)800may include a central processing unit (CPU), a graphics processing unit (GPU), both CPU and GPU, a microprocessor, a digital signal processor or other processing units or components known in the art. Alternatively, or in addition, the functionally described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc. Additionally, each of the processor(s)800may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems. The one or more processor(s)800may include one or more cores.

The one or more input/output (I/O) interface(s)802may enable the matchmaking system(s)150to detect interaction with a user and/or other systems, such as one or more game system(s)110or one or more streaming system(s)120. The I/O interface(s)802may include a combination of hardware, software, and/or firmware and may include software drivers for enabling the operation of any variety of I/O device(s) integrated on the matchmaking system(s)150or with which the matchmaking system(s)150interacts, such as displays, microphones, speakers, cameras, switches, and any other variety of sensors, or the like.

The network interface(s)804may enable the matchmaking system(s)150to communicate via the one or more network(s). The network interface(s)804may include a combination of hardware, software, and/or firmware and may include software drivers for enabling any variety of protocol-based communications, and any variety of wireline and/or wireless ports/antennas. For example, the network interface(s)804may comprise one or more of a cellular radio, a wireless (e.g., IEEE 802.1x-based) interface, a Bluetooth® interface, and the like. In some embodiments, the network interface(s)804may include radio frequency (RF) circuitry that allows the matchmaking system(s)150to transition between various standards. The network interface(s)804may further enable the matchmaking system(s)150to communicate over circuit-switch domains and/or packet-switch domains.

The storage interface(s)806may enable the processor(s)800to interface and exchange data with the computer-readable medium810, as well as any storage device(s) external to the matchmaking system(s)150, such as the player datastore152.

The computer-readable media810may include volatile and/or nonvolatile memory, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Such memory includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, RAID storage systems, or any other medium which can be used to store the desired information and which can be accessed by a computing device. The computer-readable media810may be implemented as computer-readable storage media (CRSM), which may be any available physical media accessible by the processor(s)800to execute instructions stored on the memory810. In one basic implementation, CRSM may include random access memory (RAM) and Flash memory. In other implementations, CRSM may include, but is not limited to, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or any other tangible medium which can be used to store the desired information and which can be accessed by the processor(s)300. The computer-readable media810may have an operating system (OS) and/or a variety of suitable applications stored thereon. The OS, when executed by the processor(s)800may enable management of hardware and/or software resources of the matchmaking system(s)150.

Several functional blocks having instruction, data stores, and so forth may be stored within the computer-readable media810and configured to execute on the processor(s)800. The computer readable media810may have stored thereon a player identifier block812, a filter block84, a GQF evaluator block816, a virtual game block818, a game comparer block820, and a game finalizer block822. It will be appreciated that each of the functional blocks812,814,816,818,820,822, may have instructions stored thereon that when executed by the processor(s)800may enable various functions pertaining to the operations of the matchmaking system(s)150.

The instructions stored in the player identifier block812, when executed by the processor(s)800, may configure the matchmaking system(s)150to identify player(s), such as new player(s) who wish to join an online game. In some cases, the processor(s)800may request player identification information, such as from the game system(s)110and/or streaming system(s)120. The processor(s)800may further access a player datastore152to obtain parameters and other data, such as skill scores for players134,144.

The instructions stored in the filter block814, when executed by the processor(s)800, may configure the matchmaking system(s)150to filter player(s)134,144who wish to join an online game into separate bins according to one or more parameters. These parameters may include game modes that a player wishes to play, player roles, the player's physical location, and/or the quality of the player's network connection. The quality of the player's network connection may be tested by the game system(s)110and/or the streaming system(s)120by any suitable mechanism, such as a ping test. In some cases, the processor(s)800may provision network speed measurement, such as by messaging the game system(s)110and/or streaming system(s)120.

The instructions stored in the GQF evaluator block816, when executed by the processor(s)800, may configure the matchmaking system(s)150to determine GQF values for virtual games that are being optimized. The processor(s)800may determine any variety of GQF values, such as fullness of the game, skill balance of players, skill balance of teams, network connectivity speed balance, or the like. The processor(s) may generate GQF values on any suitable scale, such as for example, a 0-100 range or a 0-1 range.

The processor(s)800may generate the fullness of the game factor value based at least in part on the number of players assigned to a virtual game relative to the maximum number of allowed player in the game. In general, it may be desirable to completely fill an online game. In some alternative cases, it may be desirable to have an online game at a target fullness number of players other than the maximum allowed number of players. Thus, in example embodiments, a value of 100 (on a 0-100 range) may be generated for the fullness factor when a game is at a target and/or optimal number of players in a virtual game.

The processor(s)800may generate the skill balance of players factor value based at least in part on any suitable measure of spread in the skill score of players in a virtual game. For example, the two highest skill scores may be compared to the average skill score in the game, and the tighter the spread, the greater the skill balance of players factor value for a virtual game may be. Other measures of spread that the one or more processor(s)800may use to generate the skill balance of players factor value may include mean, median, mode, mean average deviation, interquartile range, standard deviation, variance, or the like of the skill scores of the players assigned to a virtual game. Thus, in example embodiments, a relatively high value may be generated for the skill balance of players factor value when there is relatively low spread in the skill score for players assigned to a virtual game.

The processor(s)800may generate the skill balance of teams factor value based at least in part on any suitable measure of difference in the skill scores of players on two different teams in a virtual game. This GQF factor may be relevant in online games that are team-centric, such as, for example, Madden NFL® by Electronic Arts, of Redwood City, Calif. In some cases, the one or two best players' skill scores on each team may be compared to get a measure of team differences in skill. In other cases, an average skill score of one team may be compared to an average skill score of other team(s) for the virtual game. Indeed, any suitable measure of team skill balance may be used by the processor(s)800to generate the skill balance of teams factor value. In example embodiments, a relatively high value may be generated for the skill balance of teams factor value when there is relatively low difference in the skill levels of teams of a virtual game.

The processor(s)800may generate the network connectivity speed balance factor value based at least in part on any suitable measure of network speed that players experience during online gameplay, particularly the streaming player144, for a virtual game. Since network latency and/or network bandwidth constraints may cause variations in times when players of an online game may experience events of the online game, and may influence their reaction times to those events, it may generally be desirable to have minimal variations in network speed for players grouped together in an online game. The network speeds for online players may be received from the streaming system(s)120and/or the game system(s)110. In some cases, the processor(s)800may request network latency and/or network bandwidth information from the streaming system(s)120and/or the game system(s)110. The streaming system(s)120and/or game system(s)110may use any suitable mechanism to measure network speeds between themselves and either a streaming client device140or a game client device130, such as a ping test. The processor(s)800may use any variety of metrics that measure spread in network speeds experienced by players134,144, such as a variance, a standard deviation, an interquartile range, or the like. In example embodiments, a relatively high value may be generated for the network connectivity speed balance factor value when there is relatively low spread in the network speeds experienced by players of a virtual game.

The instructions stored in the virtual game block818, when executed by the processor(s)800, may configure the matchmaking system(s)150to manage and track rosters associated with various virtual games being optimized. The processor(s)800may also create new virtual games, such as at the start-up of the matchmaking system(s)150and/or when all pre-existing virtual games in a particular bin are already full and a new player is added to the bin to be placed in a virtual game.

The instructions stored in the game comparer block820, when executed by the processor(s)800, may configure the matchmaking system(s)150to assess whether one set of GQF values are better than another set of GQF values. In one sense the processor(s)800may compare GQF values before and after adding a new player to a virtual game. This determination of whether there is an improvement in GQF values may take into account any hierarchy and/or ranking of the various factors of the GQFs.

The instructions stored in the game finalizer block822, when executed by the processor(s)800, may configure the matchmaking system(s)150to finalize and/or instantiate a virtual game. The processor(s)800may maintain timers associated with each of the players134,144being matched into an online game, where the timer may indicate how long the corresponding player has been waiting to get placed in an online game. Individual player timers may expire, the expiry of which may indicate that the player should be in an online game. When a player's timer expires, the processor(s)800may instantiate the virtual game in which that player is at that time. The processor(s) may also track the number of times a virtual game has been iterated without an improvement in its GQF values. In some example embodiments, if a virtual game has been iterated a threshold number of times without improvements in the virtual game's GQF values, then the processor(s)800may be instantiate that virtual game as an online game. Instantiating a game may entail sending an online game roster of players to the game system(s)110to host the online game.

The illustrated aspects of the claimed subject matter may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as illustrative forms of implementing the claims.

The disclosure is described above with reference to block and flow diagrams of systems, methods, apparatuses, and/or computer program products according to example embodiments of the disclosure. It will be understood that one or more blocks of the block diagrams and flow diagrams, and combinations of blocks in the block diagrams and flow diagrams, respectively, can be implemented by computer-executable program instructions. Likewise, some blocks of the block diagrams and flow diagrams may not necessarily need to be performed in the order presented, or may not necessarily need to be performed at all, according to some embodiments of the disclosure.

Computer-executable program instructions may be loaded onto a general purpose computer, a special-purpose computer, a processor, or other programmable data processing apparatus to produce a particular machine, such that the instructions that execute on the computer, processor, or other programmable data processing apparatus for implementing one or more functions specified in the flowchart block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction that implement one or more functions specified in the flow diagram block or blocks. As an example, embodiments of the disclosure may provide for a computer program product, comprising a computer usable medium having a computer readable program code or program instructions embodied therein, said computer readable program code adapted to be executed to implement one or more functions specified in the flow diagram block or blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions that execute on the computer or other programmable apparatus provide elements or steps for implementing the functions specified in the flow diagram block or blocks.

It will be appreciated that each of the memories and data storage devices described herein can store data and information for subsequent retrieval. The memories and databases can be in communication with each other and/or other databases, such as a centralized database, or other types of data storage devices. When needed, data or information stored in a memory or database may be transmitted to a centralized database capable of receiving data, information, or data records from more than one database or other data storage devices. In other embodiments, the databases shown can be integrated or distributed into any number of databases or other data storage devices.

Many modifications and other embodiments of the disclosure set forth herein will be apparent having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the disclosure is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.