U.S. Pat. No. 11,679,330

DETAILED DESCRIPTION

The present specification is directed towards multiple embodiments. The following disclosure is provided in order to enable a person having ordinary skill in the art to practice the invention. Language used in this specification should not be interpreted as a general disavowal of any one specific embodiment or used to limit the claims beyond the meaning of the terms used therein. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Also, the terminology and phraseology used is for the purpose of describing exemplary embodiments and should not be considered limiting. Thus, the present invention is to be accorded the widest scope encompassing numerous alternatives, modifications and equivalents consistent with the principles and features disclosed. For purpose of clarity, details relating to technical material that is known in the technical fields related to the invention have not been described in detail so as not to unnecessarily obscure the present invention.

In the description and claims of the application, each of the words “comprise” “include” and “have”, and forms thereof, are not necessarily limited to members in a list with which the words may be associated. It should be noted herein that any feature or component described in association with a specific embodiment may be used and implemented with any other embodiment unless clearly indicated otherwise.

As used herein, the indefinite articles “a” and “an” mean “at least one” or “one or more” unless the context clearly dictates otherwise.

FIG.1illustrates an exemplary system100architecture that includes at least one computer system110, and may include one or more servers150and one or more databases160, among other components, in accordance with some embodiments of the present specification.

Computer System110

Computer system110may be configured as a gaming console, a handheld gaming device, a personal computer (e.g., a desktop computer, a laptop computer, etc.), a smartphone, a tablet computing device, and/or other device that can be used to interact with an instance of a video game.

Computer system110may include one or more processors112(also interchangeably referred to herein as processors112, processor(s)112, or processor112for convenience), one or more storage devices114, one or more peripherals140, and/or other components. Processors112may be programmed by one or more computer program instructions. For example, processors112may be programmed by an NPC application120and/or other instructions (such as gaming instructions used to instantiate the game).

NPC Application120

Depending on the system configuration, NPC application120(or portions thereof) may be part of a game application, which creates a game instance to facilitate gameplay. Alternatively or additionally, NPC application120may run on a device such as a server150or a peripheral140to determine one or more NPC configurations for users in an “online” game hosted by server150.

NPC application120may include instructions that program computer system110. The instructions may include without limitation, an NPC modification engine122, an NPC management engine124, an NPC profile engine126, an Artificial Intelligence (“AI”) engine128, and/or other instructions130that program computer system110and/or servers150to perform various operations, each of which are described in greater detail herein. As used herein, for convenience, the various instructions will be described as performing an operation, when, in fact, the various instructions program the processors112(and therefore computer system110) and/or the server(s)150to perform the operation.

In one embodiment the NPC Application120is implemented on a server and comprises one or more application interfaces that are configured to receive, and transmit, data to at least one of a gaming server150hosting a multiplayer video game or a computing device110executing a video game. The multiple application interfaces in the NPC Application120may be configured to enable multiple different servers150, each hosting a different multi-player video game, and/or multiple different computer systems110, each executing the same or different video game clients, to directly and concurrently access the NPC Application. Accordingly, the NPC Application, which may be hosted on one or more servers, can provide the functionality described herein to multiple different video games, each of which is being hosted concurrently, thereby enabling the NPC Application to provide improved NPC generation to various multi-player gaming environments at the same time.

Peripherals140

Peripherals140may be used to obtain an input (e.g., direct input, measured input, etc.) from a player. Peripherals140may include, without limitation, a game controller, a gamepad, a keyboard, a mouse, an imaging device such as a camera, a motion sensing device, a light sensor, a biometric sensor, and/or other peripheral device that can obtain an input from a player. Peripherals140may be coupled to a corresponding computer system110via a wired and/or wireless connection.

Server150

Server150may include one or more computing devices and may include one or more physical processors programmed by computer program instructions. For example, server150may include all or a portion of a multiplayer video game host.

Although illustrated inFIG.1as a single component, computer system110and server150may each include a plurality of individual components (e.g., computer devices) each programmed with at least some of the functions described herein. In this manner, some components of computer system110and/or server150may perform some functions while other components may perform other functions, as would be appreciated. The one or more processors112may each include one or more physical processors that are programmed by computer program instructions. The various instructions described herein are exemplary only. Other configurations and numbers of instructions may be used, so long as the processor(s)112are programmed to perform the functions described herein.

Furthermore, it should be appreciated that although the various instructions are illustrated inFIG.1as being co-located within a single processing unit, in implementations in which processor(s)112includes multiple processing units, one or more instructions may be executed remotely from the other instructions.

The description of the functionality provided by the different instructions described herein is for illustrative purposes, and is not intended to be limiting, as any of instructions may provide more or less functionality than is described. For example, one or more of the instructions may be eliminated, and some or all of its functionality may be provided by other ones of the instructions. As another example, processor(s)112may be programmed by one or more additional instructions that may perform some or all of the functionality attributed herein to one of the instructions.

The various instructions described herein and processed by processor112may be stored in a storage device114, which may comprise random access memory (RAM), read only memory (ROM), and/or other memory. The storage device may store the computer program instructions (e.g., the aforementioned instructions) to be executed by processor112as well as data that may be manipulated by processor112. The storage device may comprise floppy disks, hard disks, optical disks, tapes, or other storage media for storing computer-executable instructions and/or data.

The various components illustrated inFIG.1may be coupled to at least one other component via a network102, which may include any one or more of, for instance, the Internet, an intranet, a PAN (Personal Area Network), a LAN (Local Area Network), a WAN (Wide Area Network), a SAN (Storage Area Network), a MAN (Metropolitan Area Network), a wireless network, a cellular communications network, a Public Switched Telephone Network, and/or other network. InFIG.1, as well as in other figures, different numbers of entities than those depicted may be used. Furthermore, according to various implementations, the components described herein may be implemented in hardware and/or software that configure hardware.

In embodiments, various databases160are configured to store multiple NPCs created and modified by NPC application120. The various databases160described herein may be, include, or interface to, for example, an Oracle™ relational database sold commercially by Oracle Corporation. Other databases, such as Informix™, DB2 (Database 2) or other data storage, including file-based, or query formats, platforms, or resources such as OLAP (On Line Analytical Processing), SQL (Structured Query Language), a SAN (storage area network), Microsoft Access™ or others may also be used, incorporated, or accessed. The database may comprise one or more such databases that reside in one or more physical devices and in one or more physical locations. The database may store a plurality of types of data and/or files and associated data or file descriptions, administrative information, or any other data.

During operation, NPC application120may track a player's play profile and continuously monitor the profile during the playing of a game by the player. Examples of player's play profile tracked and monitored is based on the game being played, and may include average speed, weapon usage, preference given for weapons and ammo currently in inventory, how often the player engaged given enemy distance, how often the player fired the first shot in an engagement, how much time a player spent in specific areas of the map, time to kill for each weapon type at each distance range, movement attributes (strafe, jump, run, walk, and crouch usage), proximity to teammates, cosmetic factors like favorite skin and emotes, among other attributes that distinguish the manner of playing by the player from other players.

An NPC modification engine122may modify an existing NPC corresponding to the player, according to the tracked and monitored data. In some embodiments, the data is map specific. In some embodiments, monitoring the data comprises monitoring the inter-relationship between the different attributes of the player's profile during game play. In one embodiment, monitoring the inter-relationship involves observing how the player's play profile changed based on game score, time left in match, and game type, among other factors. In some implementations, specific characteristics being analyzed could be across all levels/maps or specific to a level or map or even a portion of a level/map. In some implementations, the data could relate to how players interact with each other, use weapons/items, navigate through levels, react to attacks, or any other information that relates the player's play profile to various factors in the game.

In some embodiments, NPC application120tracks a single human player over time and NPC modification engine122, independently, or in association with AI engine128, generates an NPC to mimic the tracked human player. In some embodiments, the tracked human player is a high performing human player, or ‘star’ player. In other embodiments, NPC application tracks a plurality of human players in order to generate NPCs which share attributes with a large population of human players to provide for more evenly balanced gaming. In some embodiments, the plurality of human players being tracked comprise all human players in a game or a subset of human players in a game, wherein the subset may be based on attributes such as, but not limited to, level, skill, or playstyle. A larger subset of human players provides a greater data set to allow the system to discern general play patterns in the player groups being tracked. The NPC application120tracks the human players over a predetermined time period and game data is logged in the system. In some embodiments, the time period comprises since the beginning of the current game session, since the inception of the game (including all played game sessions), the past X months, wherein X ranges from 0 to 60 months and all increments within, and since the last time an update was performed to modify NPCs based on logged data.

NPC modification engine122, in association with AI engine128or independently, uses these factors or logged data to modify the baseline artificial intelligence (AI) of the existing NPC and generate new NPCs, thus continuously improving the performance of the NPC(s) or changing a performance level to a level comparable with a performance level of one or more human players competing with or against the NPC(s). Accordingly, a reaction time of the NPCs, how the NPC reacts to one or more specific events, the choice the NPC may make in response to one or more game options, and certain other NPC traits may evolve or change based on human player profile data or game data. In embodiments, changing a performance level of an NPC comprises decreasing a performance level of an NPC to match or be in range with the performance level of at least one lower skilled human player or increasing a performance level of an NPC to match or be in range with the performance level of at least one higher skilled human player. In some embodiments, the existing NPC is a generic or a default NPC generated by NPC application120and available from database(s)160, and which may be deployed in any game engine executed on system110. An original or default NPC may also be referred to as a baseline NPC that is continuously developed based on the player's play profile to improve its performance in tune with the player's performance.

NPC modification engine122and AI engine128may also correlate player's play profile and game profile information in order to evolve the NPC(s). The game profile information for all the types of games played by the player, such as games provided by different game engines, may be used by NPC modification engine122for its purpose. In one implementation, a game profile may be generated for a gameplay session based on gameplay information. Gameplay information may describe various game characteristics of a gameplay session that may influence the quality of gameplay. For example, gameplay information may include, without limitation, a number of players, types of roles (e.g., snipers), types of in-game items used or purchased (e.g., weapons, vehicles, armor, custom suits, custom paint, tires, engine modifications, etc.), composition of teams (e.g., number and/or types of roles in each team), maps or game levels played (e.g., battle zones, racetracks, sporting arenas, etc.), duration of gameplay (e.g., how duration of a given gameplay session), player skill levels, player styles (e.g., aggressive, prefers to be a sniper, etc.), types of matches (e.g., team death match, capture the flag, etc.), and/or other information related to a gameplay session.

Accordingly, game outcomes may be used to modify one or more NPC behavioral or decision traits. In particular, AI engine128may be configured to associate, in real-time, specific game outcome data, such as an amount of time a game is played, an amount of virtual resources earned, lost, or exchanged, a score, a time of completion, a number of levels achieved, or other metrics indicative of engagement, with a reaction time of the NPCs, how the NPC reacts to one or more specific events, or the choice the NPC may make in response to one or more game options. To do so, AI engine128may comprise one or more nodes, virtually interlinked in a form of one or more layers, that is trained by providing one or more datasets having the game data or human profile data described above, wherein, as a result of the training process, one or more of the aforementioned nodes is associated with a coefficient, weight, or other value indicator used to weight the importance of the node and to associate inputted data, such as a game event, with a preferred behavior, such as how the NPC should react. In one embodiment, a training of the AI engine128occurs substantially concurrently with an execution of the AI engine128to identify, predict or otherwise select preferred NPC behavior based upon game events occurring within the game. Referring toFIG.3, the AI engine322receives game data316and human player data326as described above. The AI engine322subjects the game data316and human player data326to one or more artificial engine or machine learning processes354, including one or more of linear regression, logistic regression, decision trees, naïve Bayes, k-means, random forest, dimensionality reduction algorithms, gradient boosting algorithms, supervised learning, unsupervised learning, or reinforcement learning. In one embodiment the receipt of game data316and human player data326occurs at a different time or substantially concurrently with the generation of NPC data346, which is indicative of how the NPC should behave, react, choose, or otherwise act, and/or the supply of the NPC data346to more than one server executing different games.

A game profile may be specific for a given gameplay session (e.g., different game profiles may be associated with different gameplay sessions) and/or may be used to generate a broader game profile for a particular game (e.g., different games may be associated with different game profiles). In this manner, a given game or gameplay session may be characterized using a game profile.

According to an aspect of the invention, the play profile may be generated for the player based on player information as well as play information from the game played by the player. Player information may describe various characteristics of the player, which may be used to assess how the player plays in a given gameplay session, a match, and/or a game.

For example, player information may comprise a variety of player attributes including, without limitation, screen name (or gamer tag), style of gameplay (e.g., aggressive), a role preference (e.g., an explicit indication by the player of such preference), a role actually played, a duration of gameplay sessions, a number of gameplay sessions played by in a given login session, in-game items used or purchased by the player, membership in a clan or team, preference to play with clan mates or friends, demographic information of the player (e.g., geographic location, gender, income level, etc.), win/loss records, scores, and/or other attributes or information without limitation that may be used to determine the play profile of the player in a given gameplay session, a match, and/or a game. Further, the play style of the player may be gathered from information about an average speed, use of weapons, preference given to certain weapons and ammo made available to the player, how often the player engaged given enemy distance, how often the player fired the first shot in an engagement, how much time the player spent in specific areas of a map of the game, time taken by the player to kill for each weapon type at each distance range, movement attributes of the player, proximity maintained by the player with the teammates, cosmetic attributes selected by the player for its playing character in the game, among other attributes or information that may be used to determine the play style of the player. While examples provided in the present specification are in context of FPS games, the attributes/information specific to any type of game may be determined by the various embodiments.

Information from a player play profile may be indexed by time. For example, the foregoing player information may include all player information known about a player, a subset of all information (e.g., information related to the last day, week, month, previous “N” number of game sessions, login sessions, etc.). In this manner, a player profile may relate to all-time gameplay of the player, recent gameplay of the player, time of day (e.g., a player may be associated with different player profiles at different times of the day, such as having an aggressive play style during evening hours and a more relaxed play style during morning hours), and/or other subset.

According to an aspect of the invention, a player play profile may include a numerical value or other metric representative of the player's overall player skill. A player skill value may, for example, be determined according to historical player performance data represented (or conveyed) by one or more player play profile attributes. For example, player play profile attributes such as number of games played, winning percentage, highest score, lowest score, and the like may be used to determine a player skill value. It should be appreciated that the number and type of player profile attributes used to determine a player skill value may vary depending on the type of video game. As an example, in a first-person-shooter game, numerical values associated with attributes such as Score Per Minute (“SPM”), Kill/Death Ratio (“KDR”), Win/Loss Ratio (“WLR”), or other attributes may be used to generate a player skill value for the player. The player skill value may be continually updated and stored over time.

In embodiments, the modified and improved NPC(s) are stored in database160. NPC management engine124may select and retrieve one or more NPCs from database160.

NPC management engine124may generate one or more modified NPCs in real-time (“on the fly”) when a demand for the NPCs is identified. Alternatively, NPC management engine may generate the modified or improved NPC corresponding to the player in order to play with/against NPCs of other players. In some embodiments, NPC management engine124may license NPCs corresponding to popular players in order to generate value. In some embodiments, NPC management engine124uses improved NPCs to create simulated opposing teams/players for professional teams to practice with/against. In some embodiments, NPC management engine124replaces human players who left a game with NPCs that mimic the human player's play profile. In some embodiments, NPC management engine124uses modified and improved NPCs to develop new baseline NPCs in order to improve a quality of AI bots overall.

In some embodiments, NPCs created in NPC application120are styled by NPC profile engine126to resemble their corresponding human players in terms of both player profile attributes and gameplay actions such that players may not recognize NPCs as non-human, computer-controlled players. A variety of known computational and/or statistical methods may be used to ensure that an NPC is generated (for current gameplay or later selection) having attributes and attribute values typical of other human players in a gameplay session. Details about such methods are described in U.S. Pat. No. 10,118,099, entitled ‘System and Method for Transparently Styling Non-Player Characters in a Multiplayer Vide Game’, and incorporated herein by reference in its entirety.

During gameplay, AI engine128may control an NPC's behavior (including gameplay actions) such that the NPC's gameplay more closely mimics the gameplay of the corresponding player. In one implementation, AI engine128may analyze an NPC player play profile (as selected or generated in the manner described above) to determine an appropriate skill level of play of the NPC.

In one implementation, an NPC may be directed (or trained) by AI engine128to modify or improve its gameplay behavior corresponding to the changes in the player's gameplay behavior. AI engine128may monitor player and NPC gameplay performance in an effort to continually improve NPC performance, logic, strategy, and/or other NPC characteristics. In some implementations, the player's play profile is monitored over and over in multiple gameplay sessions, and AI engine128may fine tune the performance of the NPC each time so that it behaves in a manner more and more consistent with that of the player in its gameplay sessions. In this regard, NPC management engine124may, over time, have access to a plurality of NPCs (e.g., stored in database160) that have been fine-tuned according to the play profile of each player. In implementations wherein a NPC is not saved or stored for later gameplay sessions, AI engine128may nonetheless fine tune one or more of the NPC's player profile attributes and save them in a template or model for later use by NPC management engine124and or NPC profile engine126.

In embodiments of the present specification the NPC application120is configured as a standalone system component that may interface with multiple game engines. Referring toFIG.4, the AI engine422receives game data416and human player data426as described above. The AI engine422subjects the game data416and human player data426to one or more artificial engine or machine learning processes454, including one or more of linear regression, logistic regression, decision trees, naïve Bayes, k-means, random forest, dimensionality reduction algorithms, gradient boosting algorithms, supervised learning, unsupervised learning, or reinforcement learning. In one embodiment the receipt of game data416and human player data426occurs at a different time or substantially concurrently with the generation of NPC data446, which is indicative of how the NPC should behave, react, choose, or otherwise act, and/or the supply of the NPC data346to more than one server executing different games. The NPC data446is then transmitted to one or more game servers466a,446b,446c,466dusing application programming interfaces specific to game servers466a,446b,446c,466d. The NPC data446, thereby enabling different game servers executing different games to share the resources of a common NPC generation system.

FIG.2illustrates an exemplary flow chart of processing operations for modifying existing NPCs or generating new NPCs in multiplayer video games, in accordance with some embodiments of the present specification. The various processing operations and/or data flows depicted inFIG.2are described in greater detail herein. The described operations may be accomplished using some or all of the system components (enabling all of the features and functionality) described in detail above and, in some implementations, various operations may be performed in different sequences and various operations may be omitted. Additional operations may be performed along with some or all of the operations shown in the depicted flow diagrams.

One or more operations may be performed simultaneously. Accordingly, the operations as illustrated (and described in greater detail below) are exemplary by nature and, as such, should not be viewed as limiting.

At202, at least one player's play profile is tracked. The play profile may track user's play style, performance, skill levels, appearances, or any other information that determines the manner of play by the player. In some embodiments, multiple player's play profiles are tracked simultaneously. In embodiments, each play profile is tracked for the player across all the games played by the player.

At204, the play profile is monitored for changes to the tracked play profile. The play profile may be monitored on a continuous basis to determine how various game-related factors such as and not limited to game score, time remaining in a match, type of game, and the like, affect the player's play profile. In embodiments, the play profile is monitored continuously from the time of inception of the first game played by the player. In some embodiments, the monitoring is initiated based on a trigger by the player or by NPC application120, or any other external factor such as since last balance update, from a specific day or time. In some embodiments, the monitoring is performed for a specified period of time, for example a few months.

At206, one or more NPCs corresponding to the player's play profile are modified in real time or generated by AI engine128. Initially, in some embodiments, a baseline NPC assigned to the player is developed to mimic the play profile of the user. In some embodiments, the initial attributes of the NPC are not based on any tracked human player attributes and are rather default attributes provided by the game. In other embodiments, the initial attributes of the NPC are based on tracked attributes of a plurality of human players. In other embodiments, the initial attributes of the NPC are based on the attributes of a single tracked human player. Over the course of time and with the player's experience in gaming, the assigned NPC is further modified and improved based on the continuous monitoring performed at204. According to some implementations, modifying comprises revising scripts that control NPCs. In an example, if player's play profile data indicates that the player generally reacts a certain way given a certain set of conditions, then the system can script the corresponding NPC to react the same way if those conditions arise.

Among multiple application of the embodiments of the present specification, an NPC customized according to a player is available to play in a video game in place of the player's character when the player exits the game, or is in some way not presently engaged in the video game. Therefore, embodiments of the present specification enable player characters to be used as NPCs, for a part of the game. Sometimes, embodiments enables player characters to be used as NPCs for all of the game. This is implemented in cases where the total number of players required are less than those available in the game.

While embodiments of the present specification are described in context of improving NPC performance based on and corresponding to a player's play profile, they may also be applicable to tracking and monitoring multiple players play profiles in order to improve the performance of at least one or more NPCs. In embodiments, a baseline set of NPCs are modified according to a larger data set of players in a game. So over some time period, the system can collect data about how human players actually play, and adjust NPC behavior to reflect the way humans actually play the game.

The various implementations may track and monitor all human players, a subset of human players based on game level, gaming skill, playstyle, or a particular human player such as a star player, or any other type of player specified within NPC application120.

The above examples are merely illustrative of the many applications of the methods and systems of present specification. Although only a few embodiments of the present invention have been described herein, it should be understood that the present invention might be embodied in many other specific forms without departing from the spirit or scope of the invention. Therefore, the present examples and embodiments are to be considered as illustrative and not restrictive, and the invention may be modified within the scope of the appended claims.