U.S. Pat. No. 12,257,505

It is to be expressly understood that the description and drawings are only for purposes of illustrating certain embodiments and are an aid of understanding. They are not intended to and should not be limiting.

DETAILED DESCRIPTION

I. Video Game Apparatus

With reference toFIG.1A, there is shown a video game apparatus10. In some embodiments, the video game apparatus10is a dedicated gaming console similar to an Xbox™, Playstation™ or Nintendo™ gaming console. In other embodiments, the video game apparatus10is a computer such as a personal computer, a laptop computer etc. In yet other embodiments, the video game apparatus10is a mobile device such as a smartphone, a tablet. In yet other embodiments, the video game apparatus10is a hand-held gaming console. The video game apparatus10may be any other computing device suitable for carrying out the embodiments of the disclosure.

The video game apparatus10comprises a non-transitory memory12, at least one processor11, an input/output module14and a power supply27, among other components.

In some embodiments, the video game apparatus10has a game control interface configured to receive input from a player1(i.e., a user of the video game apparatus10). The game control interface may comprise an internal game controller13(e.g., a touchscreen) and/or an external game controller17(e.g., a joystick). The video game apparatus10also has a display apparatus configured to display images for the player1to see. The display apparatus may comprise an internal display device15(e.g., a touchscreen) and/or an external display device16(e.g. a computer monitor, a television set).

The video game apparatus10may be connected to a data network30via a network input/output interface25. Depending on the implementation, the data network30may be the internet, a local area network, a wireless network, a combination of such networks or still other forms of data networks.

The various components of the video game apparatus10may communicate with each other over one or more buses21, which can be data buses, control buses, power buses and the like.

Interactions between the player1and the video game apparatus10are known as “gameplay”. For example, the player1may “play” a video game by viewing game images displayed on a screen80of the internal display device15or the external display device16and controlling aspects of the video game via the internal game controller13or the external game controller17. Accordingly, the video game apparatus10receives inputs from the internal game controller13or the external game controller17via the input/output module14. The video game apparatus10also supplies outputs to the internal display device15or the external display device16and/or an auditory device (e.g., a speaker, not shown) via the input/output module14. In other implementations, there may be more than one internal or external game controller and/or more than one internal or external display device connected to the input/output module14.

The at least one processor11may include one or more central processing units (CPUs) having one or more cores. The at least one processor11may also include at least one graphics processing unit (GPU) in communication with a video encoder/video codec (coder/decoder, not shown) for causing output data to be supplied to the input/output module14for display on the internal display device15or the external display device16. The at least one processor11may also include at least one audio processing unit in communication with an audio encoder/audio codec (coder/decoder, not shown) for causing output data to be supplied to the input/output module14to the auditory device.

The memory12may include RAM (Random Access Memory), ROM (Read Only Memory), flash memory, hard disk drive(s), Flash Memory, EEPROM, Optical Disk and/or any other suitable memory device, technology or configuration. The memory12stores a variety of information including computer-readable instructions18(game code), game data20and an operating system22.

When the video game apparatus10is powered on, the at least one processor11is configured to run a booting process which includes causing the at least one processor11to communicate with the memory12. In particular, the booting process causes execution of the operating system22. The operating system22may be any commercial or proprietary operating system suitable for the video game apparatus10. Execution of the operating system22causes the at least one processor11to generate images displayed on the internal display device15or the external display device16, including various options that are selectable by the player1via the internal game controller13or the external game controller17, including the option for the player1to start and/or select a video game to be played. The video game selected/started by the player1is encoded by the game code18.

The at least one processor11is configured to execute the game code18such that the at least one processor11is able to perform various kinds of information processing functions related to the video game that it encodes. In particular, and with reference toFIG.1B, execution of the game code18causes the at least one processor11to execute a game data processing function23and game rendering function24, which are now described.

The game rendering function24includes generation of a game image to be displayed on the internal display device15or the external display device16. For its part, the game data processing function23includes processing of information representing progress of the game or a current state of the game (e.g., processing of information relating to the game that is not necessarily displayed on the internal display device15or the external display device16). The game data processing function23may sometimes be referred to as “game logic”.

The game data processing function23and the game rendering function24are illustrated inFIG.1Bas forming part of a single set of computer-readable instructions18. However, in other embodiments, the game data processing function23and the game rendering function24may be separate programs or sets of computer-readable instructions stored in separate memories and executed by separate, possibly distant, processors. For example, the game data processing function23may be performed on a CPU and the game rendering function24may be performed on a GPU.

In the course of executing the game code18, the at least one processor11manipulates constructs such as game objects and/or levels according to certain game rules and/or applying certain artificial intelligence algorithms. In the course of executing the game code18, the at least one processor11creates, loads, stores, reads and generally accesses the game data20, which includes data related to the game objects and/or levels.FIG.1Cshows an example illustrating examples of game data20according to a present example embodiment. The game data20may include data related to the aforementioned constructs and therefore may include object data42and/or level data44.

A game object may refer to any element or portion of an element in a gaming environment that can be displayed graphically in a game image frame. Various types of game objects include a player character (PC), a non-player character (NPC), a player-controlled game object (such as a weapon) and an environment object, to name a few non-limiting possibilities. One game object may be associated with one or more other game objects, such as weapons held/controlled by a PC, or clothes donned by an NPC, or a formation of NPCs associated with a PC. A game object may have other non-visual representations such as numeric, geometric or mathematical representations. The object data42may store data relating to the current representation of a game object such as the graphical representation in a game image frame or a numeric, geometric or mathematical representation.

The object data42may store attributes of various game objects. In some cases, certain attributes of a game object may be controlled by the player1. In other cases, certain attributes of a game object may be controlled by the game code18. Non-limiting examples of attributes may include imaging data, position data, orientation data, material/texture data, color data, physical state data, visibility data, lighting data (e.g., direction, position, color and/or intensity), sound data, motion data, collision data, environment data, timer data and/or other data associated with the game object.

The game data20may also include data relating to the current view or camera angle of the video game (e.g., first-person view, third-person view, etc.) as displayed on the screen80of the internal display device15or the external display device16which may be part of the representations and/or attributes of the object data42and/or the level data44.

In executing the game code18, the at least one processor11may cause an initialization phase to occur after the player1has selected/started the game, causing initialization of the game. The initialization phase is used to carry out any necessary game setup and prepare the game data20for the start of the game. The game data20changes during the processing of the game code18(i.e., during the playing of the game) and the terminology “game state” is used herein to define the current state or properties of the game data20and hence the various object data42and/or the level data44and their corresponding representations and/or attributes.

After the initialization phase, the at least one processor11, in the course of execution of the game code18, may implement a “game loop”. The game loop may include (i) the game data processing function23being performed to process the player's input via the internal game controller13or the external game controller17and to update the game state (e.g., change data stored and maintained in the memory12regarding the gaming environment) and (ii) the game rendering processing function24being performed to cause the game image to be rendered based on the updated game state for display on the internal display device15or the external display device16. The game loop may also track the passage of time to control the rate of gameplay.

The game loop runs continuously during gameplay causing the game data processing function23and the game rendering function24to be performed over and over again, but with a constantly changing game state.

It should be appreciated that parameters other than player inputs can influence the game state. For example, various timers (e.g., elapsed time, time since a particular event, virtual time of day, etc.) can have an effect on the game state. In other words, the game keeps progressing even when the player1does not provide input, and as such, the game state may be updated in the absence of the player's input.

In general, the number of times that the game data processing function23is performed per second specifies the game state update rate (hereinafter “updates per second”) and the number of times that the game rendering function24is performed per second specifies game image rendering rate (hereinafter “frames per second”). It should be appreciated that the target number of frames per second may be more or less than 25 frames per second (e.g., 30 or 60 frames per second); however, it may be desired that the game data processing function23and the game rendering processing function24be performed not less than 20 to 25 times per second lest the human eye notice any lag in the rendering of the game image frames. Naturally, the higher the frame rate, the less time between images and the more power is required of the processor(s) to execute the game loop.

The game data processing function23and the game rendering function24may be called the same number of times per second, although this is not a requirement. By way of a specific and non-limiting example, it may be desirable to perform the game data processing function23and the game rendering function24every 40 milliseconds (ms) (i.e., 40 ms=1 second/25 frames per second—FPS). In the case where the game data processing function23is followed by the game rendering function24, it should be appreciated that, for a given cycle or period, both the game data processing function23and the game rendering function24would need to be performed in the allotted time for that period. Depending on the current game state, it should be appreciated that the time required to perform the game data processing function23and/or the game rendering function24may vary. If both the game data processing function23and the game rendering function24together take less than the allotted time, a sleep timer may be used before performing the next cycle of the game data processing function23and the game rendering function24. However, if the game data processing function23and the game rendering function24together take more than the allotted time to perform, this may cause jitter. One technique to avoid jitter is to occasionally omit performing the game data processing function23or the game rendering function24.

In other embodiments, the game data processing function23and the game rendering function24may be independently executed processes, i.e., not part of the same game loop. In such cases, the game data processing function23may be routinely performed at a specific rate (i.e., a specific number of updates per second) regardless of when the game rendering function24is performed and the game rendering function24may be routinely performed at a specific rate (i.e., a specific number of frames per second) regardless of when the game data processing function23is performed.

It should be appreciated that the process of routinely performing the game data processing function23and the game rendering function24may be implemented according to various techniques within the purview of the person skilled in the art. The techniques described in this document are simply non-limiting examples of how the game data processing function23and the game rendering processing function24may be performed.

As part of performing the game data processing function23, the player input received via the internal game controller13or the external game controller17(if any) and the game data20are processed. More specifically, as the player1plays the video game, the player1inputs various commands via the internal game controller13or the external game controller17such as move left, move right, move front, move back, jump, shoot, to name a few examples. In response to the received player input, the game data processing function23may update the game data20. In other words, the object data42and/or the level data44may be updated in response to player input received via the internal game controller13or the external game controller17.

It should be appreciated that on some occasions when the game data processing function23is performed, there may not be any player input received via the internal game controller13or the external game controller17.

Regardless of whether or not player input is received, the game data20is processed and may be updated. Such updating of the game data20may be in response to representations and/or attributes of the object data42and/or the level data44as the representations and/or attributes may specify updates to the game data20. For example, timer data may specify one or more timers (e.g., elapsed time, time since a particular event, virtual time of day, etc.), which may cause the game data20(e.g., the object data42and/or the level data44) to be updated. By way of another example, objects not controlled by the player1may collide (bounce off, merge, shatter, etc.), which may cause the game data20e.g., the object data42and/or the level data44to be updated in response to a collision.

In general, the game data20(e.g., the representations and/or attributes of the objects and/or levels) represents data that specifies a three-dimensional (3D) graphics scene of the game. The process of converting a three-dimensional (3D) graphics scene, which may include one or more 3D graphics objects, into two-dimensional (2D) rasterized game image for display on the internal display device15or the external display device16is generally referred to as rendering and is carried out by the game rendering function24.

FIG.1Dillustrates an example of a process of converting a 3D graphics scene to a game image for display on the internal display device15or the external display device16via the screen80. At step52, the game rendering function24processes the data that represents the three-dimensional (3D) graphics scene of the game and converts this data into a set of vertex data (also known as a vertex specification). The vertex data is suitable for processing by a rendering pipeline (also known as a graphics pipeline). At step55, the game rendering function24processes the vertex data according to the rendering pipeline. The output of the rendering pipeline is typically pixels for display on the internal display device15or the external display device16via the screen80(step60).

More specifically, at step52, the 3D graphics objects in the graphics scene may be subdivided into one or more 3D graphics primitives. A primitive may refer to a group of one or more vertices that are grouped together and/or connected to define a geometric entity (e.g., point, line, polygon, surface, object, patch, etc.) for rendering. For each of the 3D graphics primitives, vertex data is generated at this step. The vertex data of each primitive may include one or more attributes (e.g., position, the color, normal or texture coordinate information, etc.). In deriving the vertex data, a camera transformation (e.g., rotational transformations) may occur to transform the 3D graphics objects in the 3D graphics scene to the current view or camera angle. Also, in deriving the vertex data, light source data (e.g., direction, position, color and/or intensity) may be taken into consideration. The vertex data derived at this step is typically an ordered list of vertices to be sent to the rendering pipeline. The format of the ordered list typically depends on the specific implementation of the rendering pipeline.

At step55, the game rendering function24processes the vertex data according to the rendering pipeline. Non-limiting examples of known rendering pipelines include OpenGL and DirectX. Regardless of the specific rendering pipeline used to implement the rendering pipeline, the general process of the rendering pipeline is to create a 2D raster representation (e.g., pixels) of a 3D scene. The rendering pipeline in general calculates the projected position of the vertex data in to 2D screen space and performs various processing which may take into consideration lighting, colour, position information, texture coordinates and/or any other suitable process to derive the game image (e.g., pixels) for output on the internal display device15or the external display device16(step60).

In other embodiments, the video game apparatus10may be distributed across multiple geographical locations, for example, across multiple devices including configurations where the multiple devices communicate over a network (e.g., the internet), such that the game process and the rendering process may be split over multiple processors across the multiple devices. For example, in some cases, the video game apparatus10may be distributed between a server on the internet and one or more internet appliances or may comprise multiple cloud-based servers. Multiple players may therefore participate in the same video game, and the functionality of the game program (the game rendering processing function and/or the game data processing function) may be executed at least in part by the server. Therefore, in summary, when executing the game code18, the video game apparatus10is caused to (i) maintain a gaming environment8stored in then non-transitory memory12, the gaming environment8comprising various game objects; (ii) track attributes of the game objects in the gaming environment8; (iii) render images for display based on the game objects in the gaming environment8and one or more camera perspectives; (iv) be attentive to input from the player; and (v) respond to the input by making changes to the attributes of the game objects in the gaming environment8.

II. Overview

In accordance with embodiments of the present disclosure, by executing the game code18, the video game apparatus10implements two distinct phases, which can be referred to as a “coaching session” and a “gaming session”.

During the gaming session, rather than being controlled by inputs directly received from the player1(e.g., via the internal game controller13and/or the external game controller17), an autonomous agent (e.g., a virtual entity such as a non-player character (NPC), a virtual vehicle, etc.; an autonomous entity or an intelligent entity such as an intelligent agent, an artificial agent, etc.; or an artificial intelligence (AI) director, to list a few non-limiting examples) carries out actions in accordance with a “behavioral model”100associated with the agent. The agent and the behavioral model100are part of the gaming environment8maintained by the video game apparatus10.

For its part, the coaching session is used to adapt the behavioral model100used in the subsequent gaming session. In this regard, during the coaching session, the agent is “coached” by the player1, which includes the video game apparatus10receiving recommendations from the player1, as shown inFIG.2A. The nature of the interaction between the video game apparatus10and the player1gives the player1the illusion that the player1is making recommendations to the agent.

The greater the level of sophistication of the coaching session, the more consistent the “behavior” exhibited by the agent in the gaming session will be with the player's own decision-making process (as expressed by the aforesaid recommendations), which could lead to increased player satisfaction.

In some embodiments, the video game apparatus10may implement a single coaching session before implementing a gaming session. In other embodiments, multiple coaching sessions may be implemented before implementing a gaming session. In yet other embodiments, a coaching session may be followed by multiple gaming sessions.

For the purposes of the present description, aspects of this disclosure will be described below with respect to a non-limiting example in which the video game is of the “fighting video game” genre and the agent is a virtual fighter. Accordingly, the gaming environment8defines the virtual fighter, a behavioral model100associated with the virtual fighter, and an opponent character against whom the virtual fighter is to fight. It is understood, however, that various aspects to be described below can be applied to other video game genres and non-gaming virtual environments, or any other suitable applications.

The gaming environment8also defines current game data50. In this case, the current game data50is a subset of the game data20at a time T in the fighting session. The current game data50can include game data related to character states, character actions, environmental states, positional information about objects and/or characters in the video game, non-positional information about characters and/or objects in the video game, speeds, directions, health and other attributes, game modes, levels, and/or other information associated with the video game. For example, the current game data50may include, for each of the potentially many virtual characters interacting in the gaming environment8(such as PCs and NPCs), character position, character orientation, current character action, attributes, and/or other information with the video game. The current game data50can include dynamic state information that continually changes, such as character movement positions, and static state information, such as the identification of a game level or map within the game. The current game data50may include other information that depends on the type of game and other factors.

In some embodiments, at least a portion of the current game data50is determined or collected in a continuous fashion. In other embodiments, at least a portion of the current game data50is determined or collected on a periodic basis, such as multiple times per second. In yet other embodiments, the current game data50is determined or collected on an event-based basis.

In the context of the presently described fighting video game, the gaming session is a “fighting session” during which the virtual fighter carries out actions, within the gaming environment8, against the opponent character. These actions are carried out in accordance with the behavioral model100associated with the virtual fighter. The behavioral model100will have been adapted by the player1during one or more prior coaching sessions (to be described hereinbelow). During the coaching session, the opponent character may be a computer-controlled virtual entity or a PC controlled by another player. Alternatively, during the coaching session, the opponent character may be an agent associated with another player, in which case the opponent character may in fact be coached by another player in a similar fashion as is being described with respect to the virtual fighter associated with the player1. During the fighting session, the opponent character may be a PC controlled by the player or by another player, an agent coached by another player or a computer-controlled virtual entity. These example opponent characters are not intended to be limiting. It should also be understood that there is no particular limit as to the number of opponent characters against which the virtual fighter must fight.

During the fighting session, the virtual fighter may autonomously carry out a performable gameplay action A of a plurality of performable gameplay actions72. Examples of performable gameplay actions A in a fighting video game may include “flee”, “punch”, “heal”, “jump”, “shoot”, “use special power”, “stay close”, to name a few non-limiting possibilities. Each of performable gameplay action A of the plurality of performable gameplay actions72may be associated with an animation routine which can be initiated by the at least one processor11of the video game apparatus10when it is determined that such particular performable gameplay action A is to be carried out by the virtual fighter.

As shown inFIGS.3and4, the behavioral model100has inputs and outputs. The inputs of the behavioral model100are taken from the current game data50, which was described above. The outputs of the behavioral model100comprise an indicator I for each performable gameplay action A of the plurality of performable gameplay actions72. The indicator I for each performable gameplay action A of the plurality of performable gameplay actions72may be stored in the non-transitory memory12.

In this example of implementation, as shown inFIG.4, the behavioral model100is configured to compute the indicator I for each performable gameplay action A of the plurality of performable gameplay actions72based on the current game data50. The indicator I for a particular performable gameplay action A may be referred to as a “preference score”76associated with that particular performable gameplay action A. The preference score76may be understood as a representation of the player's preference for a particular performable gameplay action A according to a given context, the preference being characterized by the behavioral model100adapted to reflect a player's decision-making process as learned from the prior coaching session. The preference score76may further be understood as a measure of the utility of a particular performable gameplay action A, the utility being determined for a given context as defined by the current game data50; in this way, the preference score76may also be referred to as a “utility score”.

The preference score76may be configured as any metric that lends itself to numerical comparison. For example, depending on the embodiment, the preference score76may be configured as an integer value, a fractional value, a percentual value, etc.

During the fighting session, the game code18is configured to identify the performable gameplay action A having the highest preference score76(according to a performable gameplay action determination algorithm1150A) and the virtual fighter is caused to carry out the performable gameplay action A so identified. Reference is now made to the flowchart inFIG.9showing steps of the algorithm1150A executed by the at least one processor11according to which the performable gameplay action A having the highest preference score76is determined.

Specifically, at step1151A, the highest preference score76as output by the behavioral model100is determined. The highest preference score76may be associated with one or more performable gameplay actions A of the plurality of performable gameplay actions72(e.g., two or more performable gameplay actions A may be tied for the highest preference score76). Thus, at step1152A, one or more performable gameplay actions A associated with the highest preference score76are identified. If only one performable gameplay action A has been identified, the algorithm1150A proceeds to step1153A in which the performable gameplay action A with the highest preference score76is selected. If, on the other hand, two or more actions are tied for the highest preference score76, any one of these two or more performable gameplay actions A may be selected (e.g., at random) at step1154A.

In some embodiments, the preference score76for each performable gameplay action A of the plurality of performable gameplay actions72is computed continuously throughout the fighting session. In other embodiments, the preference score76for each performable gameplay action A of the plurality of performable gameplay actions72is computed on a periodic basis, such as multiple times per second. In yet other embodiments, the preference score76for each performable gameplay action A of the plurality of performable gameplay actions72is computed on an event-based basis.

In some cases, during the fighting session, the virtual fighter may be in the midst of carrying out a performable gameplay action A having the highest preference score76only to change course, mid-action, to a different performable gameplay action A having the highest preference score76. Accordingly, the at least one processor11need not wait until a previous performable gameplay action A is completed to recompute the preference score76for each performable gameplay action A of the plurality of performable gameplay actions72and determine a new highest preference score76.

As mentioned above, each performable gameplay action A may involve the game code18running an associated animation routine. This animation routine has a certain respective duration. Thus, it is possible that during the time that the animation routine associated with a first performable gameplay action A is running (because the first performable gameplay action A was found to have the highest preference score76), a second performable gameplay action A may be identified as having the highest preference score76. At this point, depending on the embodiment, either the animation routine associated with the first performable gameplay action A is interrupted and the animation routine associated with the second performable gameplay action A is commenced, or the game code18is configured to wait until the animation routine associated with the first performable gameplay action A ends, at which point the game code18may recompute the preference score76for each performable gameplay action A and determine which performable gameplay action A has the highest preference score76.

III. Computation of the Preference Score76

The preference score76associated with a particular performable gameplay action A is computed based on a set of “contextual parameters”84associated with that particular performable gameplay action A. As such, the behavioral model100is configured to output a preference score76for each performable gameplay action A of the plurality of performable gameplay actions72based on the set of contextual parameters84associated with that particular performable gameplay action A. The set of contextual parameters84is stored in the memory12of the video game apparatus10as part of the behavioral model100.

FIG.5Aillustrates the association between each performable gameplay action A of the plurality of performable gameplay actions72and its associated set of contextual parameters84. In accordance with the non-limiting example shown inFIG.5B, the performable gameplay action flee is associated with a set of contextual parameters84comprising the following four contextual parameters: [not] cornered, (<=25%) remaining health points, [not] favorable position and opponent tactical advantage. It should be appreciated that there is no particular limit as to the number or nature of contextual parameters that may be part of the set of contextual parameters84associated with a particular performable gameplay action A.

The set of contextual parameters84associated with the performable gameplay action flee reflects game state considerations and/or prerequisites that the virtual fighter must ‘evaluate’ during a fighting session in order to assess if this performable gameplay action flee should be carried out in a given context of the fighting session, the given context being defined by the current game data50. In other words, the set of contextual parameters84associated with the performable gameplay action flee reflects game state considerations and/or prerequisites that the virtual fighter must ‘evaluate’ during a fighting session in order to assess the utility of fleeing in a given context of the fighting session, the given context being defined by the current game data50.

For example, the contextual parameter [not]cornered reflects a prerequisite that the virtual fighter not be cornered (e.g., by the opponent character and/or an object or objects in the gaming environment8) in order to flee. Said differently, flee may be considered a useful gameplay action in a context of the fighting session only if the current game data50is reflective of the virtual fighter not being cornered in the gaming environment8.

In yet another example, the contextual parameter (T) can be tracked using a statistical toolbox34, as shown inFIG.13B. The statistical toolbox34may be stored as a table in the memory12as part of the behavioral model100.

In yet another example, assume that during a given coaching session, the player1indicates that the virtual fighter is not in a favorable position (i.e., the personalizable contextual parameter [not] favorable position) because there is not a lot of time remaining in the coaching session (i.e., the contextual factor [not] long remaining time). Now, it is possible that in a subsequent coaching session, the player1contradicts themselves and indicates that the virtual fighter is in fact in a favorable position (i.e., the personalizable contextual parameter favorable position) because there is not a lot of time remaining in the coaching session (i.e., the contextual factor [not] long remaining time). The apparent inconsistency between the contextual parameters ([not] favorable position and favorable position) with respect to the same contextual factor CF (e.g., [not] long remaining time) can be tracked using the statistical toolbox34, as shown inFIG.13B. As can be appreciated fromFIG.13B, the statistical toolbox34may track information concerning more than one personalizable contextual parameter86(in this case, [not] favorable position and favorable position).

In particular, the game code18continuously assesses all the contextual factors CF in the set of contextual factors90tied to a particular personalizable parameter86and cross-correlates them in order to determine if any contextual factors CF in the set of contextual factors90appear to be redundant, contradictory or unimportant. Accordingly, a statistical analysis of past player recommendations (e.g., an analysis of a given set of contextual factors90) may be conducted such that the game code18may be configured to generate one or more markers of inconsistencies46in a set of contextual factors90(i.e., markers of independence, conflict, importance in accordance with a few non-limiting examples). The statistical analysis may be performed at the end of every coaching session, any time a change is made to a set of contextual factors90or any other suitable time during the coaching session.

These markers of inconsistencies46may be represented as a a numerical value (e.g., ranging between 0 and 1) which characterizes the degree to which a contextual factor CF is independent from another contextual factor CF, the degree to which a contextual factor CF conflicts with and therefore contradicts another contextual factor CF, the degree of consistency of the player's recommendations with respect to associating a contextual factor CF with multiple personalizable contextual parameters86or the degree to which a contextual factor CF is important in evaluating a personalizable contextual parameter, in accordance with a few non-limiting examples. Once generated, these markers of inconsistences46may be presented to the player1as summarized in a statistical toolbox34associated with a particular personalizable contextual parameter86, as will be discussed below.

For example, results of a statistical analysis of past player recommendations from the player1, may suggest that the contextual factors pinned down and slowed down associated with the personalizable contextual parameter [not] favorable position indicates are not statistically independent and therefore may be considered redundant. At the time of game design, the game designer may encode any suitable algorithm for determining the statistical independence of contextual factors CF in a set of contextual factors90.

An output of this algorithm is the marker of independence which reflects the independence of a contextual factor CF with respect to another, in this case the independence of the contextual factors pinned down and slowed down. For example, in this case, the marker of independence having a value below a threshold value set by a game designer at the time of game design signifies a lack of independence between a pair of contextual factors CF. In this example, the marker of independence for both contextual factors pinned down and slowed down is considered below the threshold value and accordingly, these contextual factors CF may be considered as lacking independence (i.e., they are redundant). This marker of independence is included in the statistical toolbox34as shown inFIGS.13A and13B.

In yet another example, results of a statistical analysis of past player recommendations from the player1, may suggest that the contextual factor [not] long remaining time associated with the personalizable contextual parameter [not] favorable position is not statistically important when considering whether the virtual fighter is in an unfavorable position in a fighting and/or coaching session. At the time of game design, the game designer may encode any suitable algorithm for determining the statistical importance of contextual factors CF in a set of contextual factors90.

An output of this algorithm is the marker of importance which reflects the importance of the contextual factor [not] long remaining time when evaluating the personalizable contextual parameter [not] favorable position. For example, in this case, the marker of importance having a value below a threshold value set by a game designer at the time of game design signifies a lack of importance of a contextual factor CF. In this case, the marker of independence for the contextual factor [not] long remaining time is considered below the threshold value and accordingly, this contextual factor CF may be considered unimportant. This marker of importance is included in the statistical toolbox34as shown in inFIGS.13A and13B.

In yet another example, results of a statistical analysis of past player recommendations from the player1may suggest that the contextual factors long remaining time and low life associated with the personalizable contextual parameter favorable position conflict and therefore may be considered contradictory.

In this example, the statistical analysis may further rely upon the one or more game state variables92associated with the contextual factors long remaining time and low life to assess a conflict between these contextual factors CF. In this case, the game state variable92that may be relied upon are the game state variables92which tracks an amount of time remaining in a session, the game state variable92which tracks the number of health points or the number of health points with respect to a threshold amount of health points below which the virtual fighter is characterized as having low remaining life.

At the time of game design, the game designer may encode any suitable algorithm for determining conflicts between contextual factors CF in a set of contextual factors90. This algorithm may rely upon the values associated with the game state variables92described above. An output of this algorithm is the marker of conflict which reflects the level to which the contextual factor long remaining time conflicts with the contextual factor low life. For example, in this case, a marker of conflict having a value below above a particular threshold value set by a game designer at the time of game design signifies a conflict between a pair of contextual factors CF. In this case, the marker of conflict for each of the contextual factors long remaining time and low life is considered above the threshold value and accordingly, the contextual factors long remaining time and low life may be considered as conflicting. This marker of conflict is included in the statistical toolbox34as shown in inFIGS.13A and13B.

A marker of conflict may also be generated in the case of the previously discussed example wherein the player1indicates that the virtual fighter is not in a favorable position because there is not a lot of time remaining in the coaching session (i.e., the contextual factor [not] long remaining time) and yet there actually is a significant amount of time remaining in the coaching session (e.g., more than T seconds, T seconds being a threshold value encoded by the game designer at the time of game design for representing long remaining time).

A marker of conflict may also be generated in the case of another previously discussed example wherein the player1indicates that the virtual fighter is in a favorable position (i.e., the personalizable contextual parameter favorable position) because there is not a lot of time remaining (i.e., the contextual factor [not] long remaining time) and yet the player1previously indicated the virtual fighter was not in a favorable position (i.e., the personalizable contextual parameter [not] favorable position) because there was not a lot of time remaining (i.e., the contextual factor [not] long remaining time).

During a coaching session, the player1may be made aware of these inconsistencies by being provided the statistical toolbox34. The behavioral model100may be further adapted by the player1in order to address these inconsistencies. For example, the player1may change the composition of the set of contextual factors90associated with a particular personalizable parameter86in order to reconcile one or more of these inconsistences. Such modifications may include removing or adding one or more contextual parameters90from the set of contextual factors90associated with that particular personalizable parameter86.

Reference is now made to the flowchart inFIG.12showing steps of an algorithm1200executed by the at least one processor11according to which one or more markers of inconsistencies46in the set of contextual factors90are generated and signaled to the player1.

At step1210, the game code18consults the memory12to identify the set of contextual factors90associated with a given personalizable contextual parameter86.

At step1220, the game code18is configured to generate one or more markers of inconsistencies46(i.e., markers of independence, conflict or importance) for each contextual factor CF in the set of contextual factors90based at least on a statistical analysis of past player recommendations. The past player recommendations are defined as recommendations provided in one or more coaching sessions previous to a current coaching session during which previous instructions indicative of a previous gameplay action A to be performed by the agent in the gaming environment8and previous contextual feedback from the player regarding the previous gameplay action A have been received.

At step1230, the one or more markers of inconsistencies46are stored in the memory12as part of the statistical toolbox34.

At step1240, the game code18may be configured to request player input related to the detected inconsistencies and, in response to received player input, the game code18is configured to modify the set of contextual factors90associated with the given personalizable contextual parameter86in order to reconcile these inconsistencies.

Accordingly, the player1may be inconsistent in their recommendations, yet the behavioral model100remains robust to inconsistent recommendations. Additionally, given that the player1can only modify a portion of the behavioral model100, namely the personalizable contextual parameters86and that the player1cannot modify the non-personalizable parameters88of the behavioral model100, the stability of the behavioral model100may be retained since only a portion of the overall decision logic associated with the agent may be changed by the player1. Further, if the player1provides uninsightful recommendations, the default set of contextual factors90dprovided by the game designer and associated with a particular personalizable contextual parameter86may be relied upon until more insightful recommendations are provided by the player1(the default set of contextual factors90dpossibly including all potential contextual factors CF).

VI. Coaching Module32

In order to prompt the player1to identify the relationships between contextual factors CF and personalizable contextual parameters86, the coaching session may involve a structured dialogue in which the player's recommendations may be obtained by way of any suitable conversational agent as will be described further below with respect to a coaching module32of the video game.

As shown inFIG.14A, the coaching module32interacts with the player1to process the player's recommendations in order to adapt the behavioral model100in accordance with the player's recommendations. During the coaching session, the coaching module32interacts with the player1by ‘asking’ the player1for a recommendation as to how the virtual fighter should respond in each specific situation. In addition, the coaching module32further interacts with the player1by ‘asking’ the player1to answer questions that justify each recommendation provided. The coaching module32processes the player's recommendations to identify the portions of the behavioral model100to be personalized and then personalizes said portions in accordance with the player's recommendations. The coaching module32is thus provided for adapting the behavioral model100associated with the virtual fighter based on the player's recommendations including, if applicable, by interfacing with the player1when inconsistencies are detected in the player's recommendations.

The coaching module32may be any interactive artificial or virtual intelligence component, agent, or other functionality with which the player1can interact; in this way, the coaching module32is configured to simulate how a human would behave as a conversational partner. The coaching module32may be a program, module, utility, agent, or any other software component which manages and conducts conversations via auditory or textual methods to request and process player recommendations as well as adapt the behavioral model100.

During a coaching session, the coaching module32interacts with the player1by way of a structured dialogue in which focused dialogue topics are selected by the coaching module32based on the given situation. The purpose of the structured dialogue is to request and process player recommendations as well as adapt the behavioral model100. The coaching module32interacts with the player by asking the player to provide their recommendations including their instructions indicative of performable gameplay action A and the contextual feedback regarding the gameplay action A. The coaching module32processes the player's recommendations in view of identifying the set of contextual parameters84associated with said gameplay action A. The coaching module32also processes the player's recommendations in view of identifying a particular contextual parameter from the set of contextual parameters84associated with said gameplay action that is a personalizable contextual parameter86. The coaching module32further processes the player's recommendations in view of identifying a particular contextual factor CF from the set of contextual factors90associated with the personalizable contextual parameter86. The coaching module32also personalizes the personalizable contextual parameter86by creating an association between the particular contextual factor CF and the personalizable contextual parameter86, such association being stored in the memory12.

With reference toFIG.14B, the coaching module32is shown to include one more functionalities including natural language processing (NLP) functionalities82, text processing functionalities98, input processing functionalities99, natural language generation (NLG) functionalities74, text generation functionalities68, to name a few non-limiting examples.

The coaching module32may extract a performable gameplay action A, a particular personalizable contextual parameter86and a particular contextual factor CF from the player's recommendations using its NLP functionalities82, text processing functionalities98and/or input processing functionalities99. The NLP functionalities82may be provided by any suitable natural language processing module such as provided by a plurality of web services and backend applications, including IBM's Watson, Google Cloud Natural Language API, Amazon Lex, Microsoft Cognitive Services, as well as any proprietary solution, application, or service.

Alternatively, the coaching module32may be configured to read player control commands for selecting an item from the list or menu of performable gameplay actions A, of personalizable contextual parameters86and/or of contextual factors CF, using the input processing functionalities99of the coaching module32. The player commands may be provided by the player1via a joystick or pressing a button or clicking a mouse or touching a screen to input data, in accordance with a few non-limiting examples.

The NLG functionalities92of the coaching module32can prepare a natural language response to the received player recommendations in order to engage in dialogue with the player1. The NLG functionalities92can be provided by any suitable NLG engine capable of generating natural language responses, such as Arria NLG, AX Semantics, Yseop, Simplenglg and NaturalOWL, in accordance with a few non-limiting examples.

The coaching module32can communicate with components of the video game apparatus10to adapt the behavioral model100, including the memory12, the processor11, the game control interface including the internal game controller13(e.g., a touchscreen) and/or the external game controller17(e.g., a joystick), the display apparatus including the internal display device15(e.g., a touchscreen) and/or the external display device16(e.g. a computer monitor, a television set).

These example systems or components are not intended to be limiting, and the coaching module32may include fewer or more functionalities or components than illustrated or described.

An exemplary interaction between the coaching module32and the player1is shown inFIGS.15A to15H. InFIG.15A, a scene1500is shown in the screen80of the video game apparatus10, in this case, a computer. The scene1500is a scene of a coaching session comprising a situation including the virtual fighter and the opponent character in the gaming environment8.

As shown in a scene1501including the structured dialogue represented inFIG.15B, interceding ‘on behalf of’ the virtual fighter, the coaching module32‘asks’ the player for a recommendation as to how the virtual fighter should respond to the situation by asking the question “[w]hat should I do?”

In essence, the coaching module32is asking for instructions from the player1, the instructions indicative of a gameplay action A performable by the virtual fighter in the gaming environment8. In this regard, the coaching module32accesses the memory12to retrieve a list or menu of performable gameplay actions A that the virtual fighter may perform and the coaching module32provides this list or menu to the player1via the input/output module14of the video game apparatus10.

As shown in a scene1502including the structured dialogue represented inFIG.15C, the player1responds to the virtual fighter, via the coaching module32, and provides the recommendation “[i]nstead of staying close to your opponent, you should flee” via an input device of the video game apparatus10(e.g., keyboard, display, microphone etc.). Thus, in this case, the player1has effectively selected the performable gameplay action flee from the list or menu of performable gameplay actions A. As such, the player's recommendation comprises instructions indicative of a gameplay action A to be performed by the virtual fighter in the gaming environment8, namely the gameplay action flee.

The coaching module32, which is in communication with the input devices of the video game apparatus10receives the player's recommendation and processes it using its NLP functionalities82, input processing functionalities99and/or text processing functionalities98. The coaching module32is suitably configured to process the response received from the player1in view of extracting the appropriate recommendation (e.g., ‘flee’ and not ‘staying close’). Accordingly, the coaching module32extracts the instructions indicative of a performable gameplay action A from the player's response.

In this case, the coaching module32will extract the instructions indicative of the performable gameplay action flee from the player's response. The coaching module32may then access the portion of the behavioral model100of the virtual fighter associated with the performable gameplay action flee. At this stage, the virtual fighter, via the coaching module32, ‘seeks to understand’ why the player1has recommended that the virtual fighter flee in this current situation. In other words, the coaching module32seeks to add context to the performable gameplay action flee.

As shown a scene1503including the structured dialogue represented inFIG.15D, interceding ‘on behalf of’ the virtual fighter, the coaching module32‘asks’ the player1to justify the recommendation that the virtual fighter flee. In essence, the coaching module32is asking the player1to provide contextual feedback related to the performable gameplay action flee, the contextual feedback namely player input related to a contextual parameter that is personalizable. In this regard, the virtual fighter, via the coaching module32, ‘asks’ the player1why he/she/they should flee by asking the question “[w]hy should I do that?”

In this coaching session, the player1is provided with an opportunity to select the contextual feedback from a set of contextual feedback possibilities. Accordingly, the coaching module32accesses the memory12to retrieve a list or menu of personalizable contextual parameters86associated with the performable gameplay action flee and provides this list or menu to the player1in a similar fashion as described above with respect toFIG.15B.

As shown in a scene1504including the structured dialogue represented inFIG.15E, in response to the question “why should I do that” (from the virtual fighter, via the coaching module32), the player1responds “[b]ecause you're not in a favorable position’ via an input device of the video game apparatus10(e.g., keyboard, display, microphone etc.). Thus, in this case, the player1has effectively selected the personalizable contextual parameter [not] favorable position from the list or menu of personalizable contextual parameters86. As such, the player's recommendation comprises contextual feedback related to the performable gameplay action flee, the contextual feedback being player input related to a contextual parameter that is personalizable.

The coaching module32, which is in communication with the input devices of the video game apparatus10receives the player's recommendation and processes it using its NLP functionalities82, input processing functionalities99and/or text processing functionalities98. The coaching module32is suitably configured to process the response received from the player1in view of extracting the appropriate recommendation (e.g., ‘favorable position’). Accordingly, the coaching module32extracts contextual feedback related to the performable gameplay action flee from the player's response, the contextual feedback being player input related to a contextual parameter that is personalizable (i.e., [not] favorable position).

In this case, the coaching module32will extract from the player's response contextual feedback, namely player input related to a contextual parameter that is personalizable, specifically the personalizable contextual parameter [not] favorable position which is related to the gameplay action flee. The coaching module32may then access the portion of the behavioral model100related to the personalizable contextual parameter [not] favorable position. At this stage, the virtual fighter via the coaching module32‘seeks to further understand’ why the player1has assessed that the virtual fighter is not in a favorable position in this current situation. In other words, the coaching module32seeks to add context to the personalizable contextual parameter [not] favorable position and thus further context to the performable gameplay action flee.

As shown in a scene1505including the structured dialogue represented inFIG.15F, interceding ‘on behalf of’ the virtual fighter, the coaching module32‘asks’ the player for justify the assessment that the virtual fighter is not in a favorable position. In essence, the coaching module32is asking the player1to provide additional contextual feedback, more specifically, to provide contextual feedback related to the personalizable contextual parameter [not] favorable position. In this regard, the virtual fighter, via the coaching module32, ‘asks’ why he/she is (or they are) not in a favorable position by asking the question “[c]an you explain that?”

In this coaching session, the player1is provided with an opportunity to select the additional contextual feedback from a set of contextual feedback possibilities. Accordingly, the coaching module32accesses the memory12to retrieve a list or menu of potential contextual factors CF and provides this list or menu to the player1in a similar fashion as described above with respect toFIG.15B.

As shown in a scene1506including the structured dialogue represented inFIG.15G, the player1responds to the virtual fighter, via the coaching module32, and provides the response “[b]ecause you are pinned down’ via an input device of the video game apparatus10(e.g., keyboard, display, microphone etc.). Thus, in this case, the player1has effectively selected the contextual factor pinned down from the list or menu of contextual factors CF. As such, the player's recommendation comprises additional contextual feedback, specifically contextual feedback related to the personalizable contextual parameter [not] favorable position.

The coaching module32, which is in communication with the input devices of the video game apparatus10receives the player's recommendation and processes it using its NLP processing functionalities82, input processing functionalities99and/or text processing functionalities98. The coaching module32is suitably configured to process the response received from the player1in view of extracting the appropriate recommendation (e.g., ‘pinned down’). Accordingly, the coaching module32extracts contextual feedback related to the personalizable contextual parameter [not] favorable position from the player's response.

In this case, the coaching module32will extract from the player's response contextual feedback, namely player input related a particular contextual factor CF to be associated with a contextual parameter that is personalizable, specifically the contextual factor pinned down personalizable contextual parameter [not] favorable position which is related to the gameplay action flee. At this stage, the coaching module32adds the contextual factor pinned down to the set of contextual factors90associated with the personalizable contextual parameter [not] favorable position.

Accordingly, through the structured dialogue represented inFIGS.15A to15F, the coaching module32interprets the player's recommendations to adapt the behavioral model100associated with the virtual fighter by tying the contextual factor pinned down to the personalizable contextual parameter [not] favorable position.

Through this structured dialogue which provides a list of performable gameplay actions A, and an indication of a set of contextual feedback possibilities including a list of personalizable contextual parameters86and a list of contextual factors CF to select from, a common vocabulary may be enforced such that the player1may express himself/herself/themselves using common ontology shared by all game systems which improves the ability of the coaching module32to interpret the player's recommendations and explanations. This does not preclude the player from making mistakes in these recommendations and the coaching module32is suited to helping the player identify these mistakes, as is now described.

Specifically, in this non-limiting example, let it be assumed that in a previous coaching session (or sessions), the player1has associated four contextual factors90with the personalizable contextual parameter [not] favorable position, namely: low life, [not] long remaining time, pinned down, slowed down.

As previously mentioned, each contextual factor CF in the set of contextual factors90is associated with one or more of the game state variables92and the values associated with the one or more of the game state variables92is tracked by the game code18as part of the current game data50.

The statistical analysis described above may reveal one or more markers of inconsistencies46as previously described. In this case, the coaching module32is also configured to validate detected inconsistencies with the player1and may requests confirmation to make changes to the set of contextual factors90further to inconsistencies being detected.

In this case, the coaching module32may access the memory12in order to retrieve the statistical toolbox34shown inFIGS.13A and13Band the game code18may be configured to display the statistical toolbox34on the screen80of the video game apparatus10. The statistical toolbox34is displayed to the player1to provide the player1with information about the impact of each recommendation that has been provided on the behavioral model100by translating detected inconsistencies into questions that the player1must answer.

As shown in a scene1507inFIG.15H, the coaching module32may ask the player1one or more questions based on the markers of inconsistences46associated with the contextual factors CF associated with the personalizable contextual parameters favorable position and [not] favorable position. These questions alert the player1to the inconsistencies in the set of contextual factors90so that they may be resolved.

The set of contextual factors90associated with the personalizable contextual parameter favorable position may be modified further the player's response to these questions.

For example, the player1may choose to remove the contextual factor low life and [not] long remaining time from the set of contextual factors90associated with the personalizable contextual parameter favorable position.

Additionally, the player1may choose not to address the detected inconsistencies. For example, the player1may choose not to take any action with respect to perceived redundancy of the contextual factors slowed down and pinned down associated with the personalizable contextual parameter [not] favorable position as the player1may assess that these contextual factors are not redundant and are indeed independent.

VII. Video Game Process

It should be appreciated that the foregoing has disclosed, inter alia, a method of operating a computer for carrying out a video game, the method being for execution by at least one processor. With reference to the flowchart inFIG.16, the method1900can be described as having a plurality of steps, which do not have to be carried out in the illustrated order. At step1910, the method includes maintaining a gaming environment in a non-transitory memory, the gaming environment defining an autonomous agent and a behavioral model associated with the agent. At step1920, the method includes implementing a gaming session of the video game during which the agent carries out actions in accordance with the behavioral model. At step1930, the method includes implementing a coaching session of the video game. Implementing a coaching session of the video game includes substep1930A for receiving instructions from a player of the video game, the instructions indicative of a gameplay action to be performed by the agent in the gaming environment. Implementing a coaching session of the video game further includes substep1930B which comprises receiving contextual feedback from the player regarding said gameplay action. Implementing a coaching session of the video game further includes substep1930C which comprises adapting the behavioral model based on the instructions and the contextual feedback.

VIII. Additional Remarks

As a result of the above-described methods, systems and applications, a new video game paradigm has been described according to which an agent is coached during a coaching session to adopt a certain behavior; this behavior is then exhibited by the agent during a subsequent gaming session.

As previously mentioned, aspects of the present disclosure relate to adapting (i.e., training) the machine learning based behavioral model100to ‘control the behavior’ of an agent based on data obtained during interactions with the player1. Typically, training such a machine learning based model requires a large amount of data to cover, for example, the potentially very large number of unique game scenarios that may be encountered within a given video game. In typical cases, a large number of player interactions would be required to yield this large amount of data. However, in this case, the need for large amounts of interactions with the player1is reduced by efficiently structuring the interactions with the player1.

As described above, player interactions may include providing the player1with a list of performable gameplay actions A, a list of personalizable contextual parameters86and a list contextual factors CF to select from. Providing these lists to the player1effectively limits what is asked from the player1in order to simplify the process of extracting data from the player's recommendations. This therefore simplifies the adapting of the behavioral model100and therefore reduces the data requirements for training the behavioral model100which in turns reduces the number of player interactions required to train the behavioral model100.

Providing these lists has the added benefit of enforcing a common vocabulary across all game systems such that the player1may express himself/herself/themselves using common ontology shared by all game systems. This consistency improves the interpretability of the player's recommendations and therefore reduces the data requirements for training the behavioral model100which in turns reduces the number of player interactions required to train the behavioral model100. Furthermore, the behavioral model100may use a logistic regression model that is known to be fast to train and is known to be data efficient. Of course, any suitable model may be implemented.

Furthermore, aspects of the present disclosure combine machine learning with game artificial intelligence (AI) by using a scoring-based decision system to interface game AI with machine learning. Such an interface allows game AI to be interoperable with machine learning in order to yield a stable, interpretable, generalizable and evolvable behavioral model100.

Aspects of the present disclosure may be applicable to any genre of video game. For example, the video game may be one of a shooter game (such as a first person shooter game (FPS) or third person shooter game (TPS)), an action game, an action-adventure game, a simulation game, a puzzle game, a strategy game, a sports game, a competitive game, a role-playing game, or any other type of video game. Aspects of the present disclosure may also be applicable to non-gaming virtual environments such as simulation-based training environments (e.g., aircraft simulation, racing simulation etc.).

The description and drawings merely illustrate various principles. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody such principles and are included within its scope, as defined in the appended claims.

Furthermore, all examples recited herein are intended to aid the reader in understanding such principles and are to be construed as being without limitation to the recited examples.

Those skilled in the art will appreciate that when a processor is described as being “configured” to carry out an action or process, this can mean that the processor carries out the action or process by virtue of executing computer-readable instructions that are read from device memory where these computer-readable instructions are stored.

Those skilled in the art should appreciate that any feature of any embodiment disclosed herein may combined with (e.g., used instead of or in addition to) any feature of any other embodiment disclosed herein in some examples of implementation. Certain additional elements that may be needed for operation of some embodiments have not been described or illustrated as they are assumed to be within a purview of those ordinarily skilled in the art. Moreover, certain embodiments may be free of, may lack and/or may function without any element that is not specifically disclosed herein.

Although various embodiments and examples have been presented, this was for purposes of description, but should not be limiting. Various modifications and enhancements will become apparent to those ordinarily skill in the art.