U.S. Pat. No. 12,138,541

DETAILED DESCRIPTION

FIGS.1-8disclose engine-agnostic and game-agnostic techniques for influencing player experience based on game context using one or more machine learning (ML) algorithms for natural language processing (referred to herein as “an NLP ML algorithm”) to select labeled actions or content based on a text-based log that represents previous game events and one or more curves representative of one or more target player experiences.

FIG.1is a block diagram of a video game processing system100that supports coordinating game decisions using a semantic natural language processing (NLP) machine learning (ML) algorithm according to some embodiments. The processing system100includes or has access to a system memory105or other storage element that is implemented using a non-transitory computer readable medium such as a dynamic random-access memory (DRAM). However, some embodiments of the memory105are implemented using other types of memory including static RAM (SRAM), nonvolatile RAM, and the like. The processing system100also includes a bus110to support communication between entities implemented in the processing system100, such as the memory105. Some embodiments of the processing system100include other buses, bridges, switches, routers, and the like, which are not shown inFIG.1in the interest of clarity.

The processing system100includes a central processing unit (CPU)115. Some embodiments of the CPU115include multiple processing elements (not shown inFIG.1in the interest of clarity) that execute instructions concurrently or in parallel. The processing elements are referred to as processor cores, compute units, or using other terms. The CPU115is connected to the bus110and the CPU115communicates with the memory105via the bus110. The CPU115executes instructions such as program code120stored in the memory105and the CPU115stores information in the memory105such as the results of the executed instructions. The CPU115is also able to initiate graphics processing by issuing draw calls.

An input/output (I/O) engine125handles input or output operations associated with a display130that presents images or video on a screen135. In the illustrated embodiment, the I/O engine125is connected to a game controller140which provides control signals to the I/O engine125in response to a user pressing one or more buttons on the game controller140or interacting with the game controller140in other ways, e.g., using motions that are detected by an accelerometer. The I/O engine125also provides signals to the game controller140to trigger responses in the game controller140such as vibrations, illuminating lights, and the like. In the illustrated embodiment, the I/O engine125reads information stored on an external storage element145, which is implemented using a non-transitory computer readable medium such as a compact disk (CD), a digital video disc (DVD), and the like. The I/O engine125also writes information to the external storage element145, such as the results of processing by the CPU115. Some embodiments of the I/O engine125are coupled to other elements of the processing system100such as keyboards, mice, printers, external disks, and the like. The I/O engine125is coupled to the bus110so that the I/O engine125communicates with the memory105, the CPU115, or other entities that are connected to the bus110.

The processing system100includes a graphics processing unit (GPU)150that renders images for presentation on the screen135of the display130, e.g., by controlling pixels that make up the screen135. For example, the GPU150renders objects to produce values of pixels that are provided to the display130, which uses the pixel values to display an image that represents the rendered objects. The GPU150includes one or more processing elements such as an array155of compute units that execute instructions concurrently or in parallel. Some embodiments of the GPU150are used for general purpose computing. In the illustrated embodiment, the GPU150communicates with the memory105(and other entities that are connected to the bus110) over the bus110. However, some embodiments of the GPU150communicate with the memory105over a direct connection or via other buses, bridges, switches, routers, and the like. The GPU150executes instructions stored in the memory105and the GPU150stores information in the memory105such as the results of the executed instructions. For example, the memory105stores instructions that represent a program code160that is to be executed by the GPU150.

The CPU115, the GPU150, or a combination thereof execute machine learning algorithms such as a semantic NLP ML algorithm. In the illustrated embodiment, the system memory105stores program code165that represents a semantic NLP ML algorithm that has been trained using a corpus of natural language data. Many text corpuses are available for training machine learning algorithms including corpuses related to media/product reviews, news articles, email/spam/newsgroup messages, tweets, dialogues, and the like. The CPU115, the GPU150, or one or more of the compute units in the array155execute the program code165that represents the trained semantic NLP ML algorithm in either input/response modality or a semantic similarity modality.

In the illustrated embodiment, the CPU115and the GPU150execute corresponding program code120,160to implement a video game application. For example, user input received via the game controller140is processed by the CPU115to modify a state of the video game application. The CPU115then transmits draw calls to instruct the GPU150to render images representative of a state of the video game application for display on the screen135of the display130. As discussed herein, the GPU150can also perform general-purpose computing related to the video game application such as executing the program code165that represents a semantic NLP ML algorithm implemented in the game. In the illustrated embodiment, the system memory105includes a portion170that is reserved for storing game-related information, although this information can be stored in other locations in some embodiments.

The CPU115and the GPU150use instances of the semantic NLP ML algorithm to coordinate game decisions to influence the player experience by selecting actions or content to be served to the player based on a current state of the game and a target player experience. The state of the game changes substantially continuously as game events transpire and the GPU150keeps track of the events using a text log that is stored in the portion170of the memory105. One or more text strings that represent the game event are recorded in the text log in response to each event that occurs in the game. The text log therefore includes a sequence of text strings that represent game events that have transpired during a portion (or the entirety) of the game. A set of actions or events that are potentially served to the player of the game are labeled with text phrases and stored in the portion170of the system memory105. Some embodiments of the semantic NLP ML algorithm generate scores for the labeled actions or content based on the text log and a curve that represents a target player experience as a function of progress through the game. One or more of the labeled actions or content is then selected based on their scores and the selected actions or content are served to the player.

FIG.2is a block diagram of a cloud-based system200that supports coordinating game decisions using a semantic NLP ML algorithm according to some embodiments. The cloud-based system200includes a server205that is interconnected with a network210. Although a single server205shown inFIG.2, some embodiments of the cloud-based system200include more than one server connected to the network210. In the illustrated embodiment, the server205includes a transceiver215that transmits signals towards the network210and receives signals from the network210. The transceiver215can be implemented using one or more separate transmitters and receivers. The server205also includes one or more processors220and one or more memories225. The processor220executes instructions such as program code stored in the memory225and the processor220stores information in the memory225such as the results of the executed instructions.

The cloud-based system200includes one or more processing devices230such as a computer, set-top box, gaming console, and the like that are connected to the server205via the network210. In the illustrated embodiment, the processing device230includes a transceiver235that transmits signals towards the network210and receives signals from the network210. The transceiver235can be implemented using one or more separate transmitters and receivers. The processing device230also includes one or more processors240and one or more memories245. The processor240executes instructions such as program code stored in the memory245and the processor240stores information in the memory245such as the results of the executed instructions. The transceiver235is connected to a display250that displays images or video on a screen255and a game controller260. Some embodiments of the cloud-based system200are therefore used by cloud-based game streaming applications.

The processor220, the processor240, or a combination thereof execute program code representative of game that implements a semantic NLP ML algorithm in either input/response modality or a semantic similarity modality. As discussed herein, the semantic NLP ML algorithm is pre-trained using one or more text corpuses. The game time decisions that affect the player experience are coordinated using the semantic NLP ML algorithm, as discussed herein.

FIG.3is a block diagram that illustrates a text log300made up of a sequence of text strings that represent events in the game according to some embodiments. The text log300is generated by some embodiments of the GPU150shown inFIG.1and the processors220,240shown inFIG.2. Some embodiments of the text log are stored in a memory such as the system memory105shown inFIG.1and the memories225,245shown inFIG.2.

The text strings in the text log300represent a portion (or the entirety) of the game and indicate the player experience up to the current point in the game. As illustrated inFIG.3, the events recorded in the text log300include information such as the information in Table 1, which includes a identifying a text string associated with an event, a time at which the event occurs, and the text string representing the event.

TABLE 1NumberTimeText String10:00.00Player walking in rain20:00.00Ominous music30:30.00Lightning40:30.50Thunderclap50:32.00Zombie Spawns Near Player60:33.00Player Shoots Zombie, Misses70:33.10Zombie Moves Towards Player80:35:00Player Shoots Zombie, Hits90:35.10Zombie Moves Towards Player100:36.00Player Makes Headshot110:36.10Zombie Dies120:40.00Player Examines Zombie

Additional text strings are added to the text log300in response to events occurring in the game. In the illustrated embodiment, an action305occurs that is represented by the text string “Zombie revives,” which indicates that the zombie that appeared to be dead has returned from the grave (again). The text string representative of the action305is then added to the text log300, as indicated by the arrow310.

FIG.4is a plot400of a curve405that represents a target player experience as a function of game progress according to some embodiments. The vertical axis of the plot400indicates a target player experience (in arbitrary units) and the horizontal axis of the plot400indicates game progress from the beginning of the game (zero) to the end of the game (one). In the illustrated embodiment, the curve405represents a target level of calm/intensity and points on the curve405correspond to the events represented by the text strings in the text log300shown inFIG.3. However, in other embodiments, the curve405represents different target player experiences such as positive/negative sentiments, tracking of a semantic concept such as “victorious” or “sense of belonging,” and the like. The curve405can be bespoke, provided by a game developer, provided by a player, or created at runtime.

Initially, at time T0, the target player experience is relatively calm as indicated by the relatively low value of the curve405. The player begins to progress through the game and the environment experienced by the player is reflected in the text strings in the text log. For example, the text log indicates that the player is walking in the rain and ominous music is playing, which is intended to gradually raise the sense of intensity, as indicated by the increasing value of the curve405from T0to T1.

At the time T1, the text log includes a text string indicating that lightning struck and there was a thunderclap. A small spike in the curve405around the time T1indicates an increase in the intensity of the player experience corresponding to the lightning and thunder.

At the time T2, the text log includes a text string indicating that a zombie has spawned nearly player. Adding a threatening character near the player is intended to increase the intensity of the player experience, as indicated by the curve405rising from time T2to time T3.

At the time T3, the text log includes a text string indicating that the player attempted a shot at the zombie. However, the text string also indicates that the player missed the shot and so the intensity of the player experience continues to increase, as indicated by the curve405rising from time T3to time T4.

At the time T4, the text log includes a text string indicating that the zombie continued to move towards the player. Reducing the distance between the zombie and the player increases the intensity of the player experience, as indicated by the curve rising from time T4to time T5.

At the time T5, the text log includes a text string indicating that the player attempted a shot at the zombie. The text string also indicates that the player's shot hit the zombie. Successfully shooting the zombie reduces the intensity of the player experience, as indicated by the curve405falling from time T4to time T5.

At the time T6, the text log includes a text string indicating that the zombie is not dead and continued to move towards the player. Reducing the distance between the zombie and the player increases the intensity of the player experience, as indicated by the curve rising from time T6to time T7.

At the time T7, the text log includes a text string indicating that the player attempted a shot at the zombie. The text string also indicates that the player's shot hit the zombie in the head and the zombie died. The successful headshot and death of the zombie reduce the intensity of the player experience, as indicated by the curve405falling from time T7to time T8.

At the time T8, the text log includes a text string indicating that the player examined the dead zombie. The zombie appeared to be dead and the intensity of the player experience continued to decrease, as indicated by the curve falling from time T8to time T9.

At the time T9, the player is examining the “dead” zombie and the zombie revives, as shown in the action305inFIG.3. The unexpected revival of the zombie will cause the intensity of the player experience to spike, as indicated by the rising dashed arrow410after the time T9.

Some embodiments of the curve405are used to coordinate game decisions to influence player experience. For example, a semantic NLP ML algorithm uses the text log300shown inFIG.3and the curve405to score or rank labeled actions or content in a set (or a subset) of labeled actions or content. The semantic NLP ML algorithm returns a ranked list of the labeled actions or content, which is then used to select the action or content that is served to the player. Although a single curve405is shown inFIG.4, multiple curves are used in some embodiments to coordinate game decisions. The aspects of the player experience represented by the different curves are indicated by a curve type415. For example, a first curve type indicates that the corresponding curve represents calm/intensity, a second curve type indicates that the corresponding curve indicates positive/negative experiences, and a third curve type indicates a semantic type such as “victorious” or “sense of belonging.”

FIG.5is a block diagram of an event map500that maps or associates text strings to events or context that occur within a game according to some embodiments. The text strings are written or appended to a text log (such as the text log300shown inFIG.3) in response to the corresponding event occurring in the game. The event map500is created by game developers based on the events or context that are used to influence player experience within the game. For example, as indicated in the event map500, a sense of “foreboding” is associated with playing “ominous music” and a sense of “shock, surprise” is associated with a “thunderclap.” Associating the events with text strings that are added or appended to a text log allows the text log to be used to determine a current player experience, as discussed herein.

FIG.6is a block diagram of a table600of labeled actions or context for a game according to some embodiments. The labels for the actions or context are used by a semantic NLP ML algorithm to choose one or more of the actions or context to serve to a player to influence player experience based on a curve (such as the curve405shown inFIG.4) that represents a target player experience as a function of progress through the game. Some embodiments of the table600are created by a game developer prior to runtime to associate text labels with actions or context that are available for presentation to the player during the game. For example, a “music loop 1” is labeled with the text string “ominous” to indicate that “music loop 1” influences player experience by increasing a sense of worry or foreboding. For another example, the “sound effect 1” is labeled with the texturing “shocking” to indicate that playing “sound effect 1” influences player experience by surprising or shocking the player.

The example shown in table600is not meant to be exhaustive. To the contrary, the actions or context that are labeled with text strings and available for provision to the player during progress of the game can include anything developed for the game. The actions or contexts can include assets or data describing how new assets could be generated. For example, the labeled content can include all the characters created for the game or set up to be generated by the game, modular world tiles associated with features and rules that govern how they connect to other tiles, data tables describing enemy's spawns (e.g., how many enemies, level of challenge, spawn location), data tables describing loot (e.g., range of items, value of items, each of which can be tagged with a text string), weather system settings, animation sets for characters, world status settings, music, and the like. In some embodiments, game developers create multiple tables and associate different tables with different curves indicating different aspects of the player experience.

FIG.7is a block diagram including a game director700that makes coordinated game decisions using one or more instances of a semantic NLP ML algorithm701according to some embodiments. The game director700is implemented in some embodiments of the GPU150shown inFIG.1and the processors220,240shown inFIG.2. In the illustrated embodiment, a text log705such as the text log300shown inFIG.3, a curve710such as the curve405shown inFIG.4, and a set715of labels actions/contexts such as the set of labeled actions/context listed in table600inFIG.6are provided to the semantic NLP ML algorithm701. The set715includes labels720,721,722,723, which are collectively referred to herein as “the labels720-723.”

The game director700uses the instances of the semantic NLP ML algorithm701to generate scores for labeled actions/contexts based on a text log and a curve of target player experience. Generating the scores includes: (1) assessing the current player experience based on the text log and (2) generating the scores for the labeled actions/contexts based on the current player experience, the target player experience indicated by the curve, and the labeled actions/contexts. In the illustrated embodiment, the game director700assesses the current player experience using the semantic NLP ML algorithm701and sentiment analysis. For example, the game director700determines whether there is a calming ambient audio of wind or ominous music creating tension, are there enemies spawning all around the player or is the player in a serene, cozy space? The sentiment that is evaluated based on the curve corresponds to the type of the curve. For example, the relative calm or intensity of the situation is evaluated if the curve indicates calm/intensity, the fortune of the player experience is evaluated if the curve indicates good/bad fortune, and other semantic concepts such as “victorious” or “sense of belonging” are evaluated for corresponding types of curves.

The game director700then uses the semantic NLP ML algorithm701to generate scores725,726,727,728(collectively referred to herein as “the scores725-728”) that rank the labels720-723based on the current player experience and the target player experience indicated by the curve. For example, if the curve is used to track good/bad fortune for the player and if the text log indicates that the current player experience has been an unending stream of bad fortune, the semantic NLP ML algorithm701can return a score for opening a treasure chest to find a rare, valuable item that is higher than the score for opening the treasure chest to reveal a devastating trap. Conversely, the semantic NLP ML algorithm701can return a lower score for the rare, valuable item and a higher score for the devastating trap if the current player experience has been exceptionally good. For another example, if the curve tracks a semantic concept such as “victorious” or “sense of belonging,” the semantic NLP ML algorithm701ranks the labels720-723based on the current player experience such as whether the player quests dealt with themes regarding “belonging to a tribe” or how “victorious” the player has been in previous encounters with enemies.

As discussed herein, the semantic NLP ML algorithm701is pre-trained using one or more text corpuses. Pre-training the semantic NLP ML algorithm701on conventional text corpuses causes the semantic NLP ML algorithm701to generate higher scores725-728for responses that are consistent with conventional usage or interpretation of the terms in the text log705and the labels720-723. However, some embodiments of the semantic NLP ML algorithm701are implemented in game worlds that purposely redefine concepts to contrast with their real-world interpretations. Post-processing of the results provided by the semantic NLP ML algorithm701is therefore used to modify the scores of725-728based on one or more rules that redefine the associations between the terms or phrases in the text log705and the labels720-723.

In response to a request to serve dynamic content, the game director700uses the text log705, the curve710, and the set715of labeled actions/contexts to choose one or more actions or context to serve to the player. The game director700chooses the actions or context based on the scores725-728. In some embodiments, the actions or context that are served by the game director700are expressed in natural language to enable subsequent NLP-based analysis. For example, the game director700can choose an action730that results in an apparently dead zombie being revived, which is represented by the text string “Zombie revives.” Information that is used to select the action includes, but is not limited to:The player choice expressed in the game, e.g., where did the player choose to go, are they role-playing as a paragon or an immoral character, etc.The (possibly emergent) events that have transpired in the game so far, e.g., if there was procedurally generated content, what is it, what form to take, and how has it affected the game world and state.The analysis of the player's experience so far, e.g., have they struggled or are they having too easy a time, is there enough variety to their experience?The decisions made by other players that have affected the game world.

FIG.8is a flow diagram of a method800of coordinating game decisions using semantic NLP ML algorithms according to some embodiments. The game director700is implemented in some embodiments of the GPU150shown inFIG.1, the processors220,240shown inFIG.2, and the game director700shown inFIG.7.

At block805, the game director accesses a text log including text strings that are mapped to events that have occurred in the game or a portion of the game. At block810, the game director accesses the curve representing a target player experience as a function of progress through the game. At block815, the game director accesses a set of labeled actions/contexts that are available for providing to the player. Although the blocks805,810,815are presented sequentially inFIG.8, some embodiments of the method800perform the blocks805,810,815in a different order or concurrently. In some embodiments, an API call is used to perform the actions in blocks805,810,815. The parameters of the API call include a context indicated by a text log, a curve representing the target player experience, an indicator of the type of curve, an indicator of a current place in the curve, and metadata indicating labels of the available actions or content.

At block820, an instance of the semantic NLP ML algorithm generates scores for the labeled actions/contexts, as discussed herein. In some embodiments, the API call returns a ranked list of the metadata, which is used to select the action or content that is served to the player. Some embodiments of the API call also return one or more of the scores for the content metadata generated by the NLP ML algorithm, an actual curve representing the actual player experience indicated by the text log, and the like. For example, if the API call includes a type that indicates that the curve is a calm/intensity curve, the semantic NLP ML algorithm uses the text log to rank the labeled actions/content based on how closely they match the target player experience indicated by the calm/intensity curve as a function of the player's progress through the game.

At block825, the game director selects a subset of the labeled actions/contexts based on the scores. In some embodiments, the game director selects labeled action/contexts to be in the subset in response to the corresponding scores being above a threshold. At block830, the game director (or other processor) serves the subset of labeled actions/contexts to the player.

In some embodiments, certain aspects of the techniques described above may be implemented by one or more processors of a processing system executing software. The software comprises one or more sets of executable instructions stored or otherwise tangibly embodied on a storage element implemented using a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

A computer readable storage medium may include any storage medium, or combination of storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer readable storage medium may be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.