U.S. Pat. No. 10,427,047

DETAILED DESCRIPTION

The present disclosure describes methods and systems directed towards identifying player engagement to generate contextual game play assistance. In particular, user gameplay information is monitored so that the user can be provided hints, advice, or walk-throughs regarding different obstacles found within the video game that the user may have problems with. The methods and systems of the present invention would monitor user gameplay information in order to identify what type(s) of hints, advice, or walk-through information could be provided to the user. The information can be based on the current level of frustration of the user with the obstacle caused, for example, by the repeated failure of the user to overcome the obstacle.

The present disclosure also describes methods and systems directed at proactively monitoring user gameplay information in order to identify what type(s) of hints, advice, or walk-through information could be provided to the user and provide the identified information prior to the user potentially being too frustrated to the point of giving up. To do so, the methods and systems would process the gameplay information about the user performance within the game along with information obtained from different sources (e.g. sensors, microphones, cameras) in order to make a determination as to the current temperament of the user regarding the game. Using information associated with the user and/or information related to other users, different thresholds of frustration can be identified whereby different types of information can be offered to the user to assist in overcoming the obstacle in order to prevent the user from becoming too frustrated with the video game.

As described herein, failure relates to a user performance (e.g. user attempts) with respect to a particular objective within the game where the user fails to complete the objective (i.e. is unsuccessful). The objective may, for example, constitute a challenge such as solving a puzzle within a pre-determined time limit or defeating an enemy. The user is unsuccessful in the attempt in completing the objective when the conditions for completing the objective are not satisfied. In the above example, this may involve the user not solving the puzzle within the pre-determined time limit or taking too much damage or repeatedly dying to the same enemy before the enemy is defeated. Failure can also be measured based on monitoring a number of successive attempts by the user on the same objective over a period of time where little or no progress has been made.

As referenced herein, the types of assistance provided to the user via the present invention can range from in the type and amount of information that is provided to the user with respect to a particular challenge within the video game. For example, a challenge may include solving a puzzle, completing a quest, or defeating an enemy within the game in order to advance. Hints may generally refer to information that points the user in a general direction regarding how to resolve the puzzle. For example, hints may be directed at identifying an important feature or mechanic about the puzzle. Advice may generally identify what the user has previously done in the past (via gameplay data) and provide information directing the user to try a different strategy. For example, advice may be directed at providing a suggested alternative choice at a particular point of the puzzle (where the user may have consistently chosen the wrong choice) that could direct the user in resolving the puzzle. Lastly, walk-through information may generally refer to a step-by-step guide instructing the user how to overcome the challenge. This information may include, for example, providing text-based instructions or an accompanying video describing how to overcome the challenge. In some embodiments, the walk-through information may include an in-game avatar (i.e. ghost) that the user can mimic in order to overcome the challenge. These are just examples of the types of assistance information characterizations that can be used. Different categories and labels can be associated with the assistance information that can be provided to the users that need assistance within the video games.

FIG. 1illustrates a system100for identifying player engagement for the purpose of generating contextual game play assistance. The system100aims to characterize the status (e.g. mood/frustration level) of the user alongside the progression of the user within the video game (e.g. location, time spent, experience) based on gameplay information from the video game in order to provide appropriate assistance within the video game. Different thresholds can also be assigned for the user that provides varying degrees of assistance based on the current status of the user so that different types of information can be catered to the user.

The system ofFIG. 1includes a first party platform110that is associated with the video games115being played by the user. As described herein, the first party platform110may coincide with a network that supports the play of video games115on various gaming consoles120from which users would participate. Some of these video games115may be played off of each user console120via a single player experience. In situations where the video games115involve multiple users (e.g. multi-player), the first party platform110can assist in the execution of the networking features that allows multiple users to participate within the same instance of the video game115.

The first party platform110may include various application programming interfaces (APIs) and related software that facilitates the present invention. For example, the first party platform110may include applications that track/monitor the gameplay information associated video games115associated with the first party platform110. The gameplay information would include user location, progress, and statistics that can be used to identify what types of assistance (e.g. hints, advice, walk-through) the user would be interested in.

The tracking features of the first party platform110can also be used to collect information from other users. The collected information from other users may be used to compile types of assistance information that can be used (e.g. hints, advice, walk-throughs) for others. For example, other users may provide commentary or video recordings regarding various challenges within the video games that can be provided to the first party platform110. Some users may be characterized as being ‘experienced’ or ‘professional’ based on past achievements and the associated gameplay information and/or commentary offered by these users may be emphasized as being possible assistance information that can be provided to users who need assistance overcoming challenges within the same game.

The assistance information provided from other users playing the same video game as the user may be processed and stored in an assistance server130associated with the video game115. The type of information can be characterized based on how detailed the information is. Short commentary can be labeled as a “hint” or “advice” while recorded gameplay or longer commentary can be labeled as “walk-through.” Based on the type of assistance being requested by the user, corresponding assistance information can be retrieved from the assistance server130. This assistance server130may be associated with the first party platform110or can be overseen and managed by a third party. Further details regarding the assistance server130will be provided below with reference toFIG. 4.

The first party platform110will be associated with a plurality of different video games115. Information related to these video games115may be stored with the first party platform110. In some embodiments the video games115may also be stored in a separate database that can be accessed by the first party platform110. In any case, the first party platform110would be capable of accessing the necessary information about the video games115in order to facilitate user participation in the video games with other users each using their user computing device120(e.g. console). For the times where the user is only participating by themselves (and thus do not need the capabilities of the first party platform110to connect with other users), the first party platform110can retrieve the necessary information and transmit it to the appropriate user computing device120that the user is using. Such video game-related information being transmitted by the first party platform110to the user computing device120may include actual information needed to run the game such as updates/patches that provides the user with additional content or addresses issues within the video game.

The user computing devices120as illustrated in the figure facilitate user participation within the video games115associated with the first party platform110. Exemplary computing devices120as described herein include gaming consoles. Other computing devices such as laptops, desktops, smart phones, tablets, and various other mobile/handheld computing devices are also useable in different embodiments. To facilitate identifying user engagement and generate contextual game assistance, the user computing device120would generate video game-based information that would characterize the user progress within the game. For example, such gameplay information may include where the user is located within the game, what event/challenge/obstacle the user is currently participating in, how long the user has been currently engaged in the event/challenge/obstacle, and current statistics of the user character/avatar within the game.

The gameplay information obtained from the user computing device120could be used to characterize performance of the user. In some embodiments, the characterization of the user performance can be performed by the first party platform110, by the video game itself115, or by a third party. The gameplay information would provide assistance in identifying the types of hints, advice, or walk-through information the user would be interested in as well as the specific information that would assist the user to overcome the current event/challenge/obstacle. For example, if the user has been stuck on the same event/challenge/obstacle for 5 minutes, the type of information being offered can be different than if the same user is still stuck after 30 minutes.

The threshold used to identify when the user should be offered/provided assistance information and/or what types of assistance information should be offered/provided to the user can be established in many different ways. The user may establish thresholds for the different types of assistance information, the thresholds may be preset, or thresholds may be customized over time based on user reaction to events/challenges/obstacles within the video games. Although there are exemplary three different types of assistance information described herein (e.g. hint, advice, walk-through), different characterizations and types of assistance information can be possible and therefore corresponding thresholds for offering/providing these assistance information could also be established.

The type of assistance information (e.g. hint, advice, walk-through information) provided to the user could be customized by the user. For example, the user could request only a particular type of assistance information (e.g. hints only) or indicate that a certain type of assistance information should not be provided (e.g. no walk-through information). Users may also be able to identify the source of assistance information that is preferred. For example, if particular third parties or other users provide assistance information, the user may be able to identify which sources the assistance information should come from preferably if available. In some instances, the video game may include assistance information associated with the video game. In this case, the user can indicate that this set of information is preferred if related information for providing assistance for the user to overcoming the particular event/challenge/obstacle is available.

In further embodiments, additional information can be used to characterize the performance of the user within the game. As illustrated inFIG. 1, the user may utilize input devices (e.g. controller122, sensors125, and/or a camera127) to provide user input to the console120. The input devices could be used to facilitate user control within the video game. However, the input devices can also be used to track/monitor the user performance with respect to the event/challenge/obstacle. For example, the controller122and/or sensors125may include features that can be used to detect the biometric characteristic of the user (e.g. heartbeat, temperature) as the user is participating within the game. Microphones and/or cameras may be used to capture gestures, facial features, and/or voice inputs from the users.

There may be key identifiers (e.g. specific words, gestures, facial expressions) that can be associated with increasing frustration at the event/challenge/obstacle. For example, during initially participating in an event, the user may provide little to no reaction upon failing. However, continued failure of the same event may detect, for example, increased frequency of certain words (e.g. expletives), increased heart rate, and certain facial expressions/gestures that can indicate increasing frustration levels for the user. When these identifies are spotted, this may be an indication that some assistance information should be provided to the user (or at the very least the assistance information should be made available/offered) so that the user can obtain assistance in overcoming the event. The assistance information can be retrieved from the assistance server130at the time the user frustration level reaches certain thresholds or when the user makes a request for information via a user input (e.g. a button on the controller122that specifically requests assistance). However, it may also be possible to proactively download associated assistance information to the user gaming console120as well so that the information is immediately available upon request from the user. What assistance information to provide to the gaming console120can be based on the current event the user is participating in as well as the current frustration level of the user. The proactive download of assistance information will allow the information to be available immediately upon request thereby eliminating or at least minimizing delay for the assistance information to be provided to the user. Since some players may quit playing the game upon receiving a certain frustration level, the aim is to allow the user to overcome the event with some form of assistance information, lower the frustration level and allow the user to continue participating and enjoying in the video game.

The information that can be provided to the users to assist the user in overcoming the event may be requested on an as needed basis. The actual information from the assistance server130can be requested at the time the frustration level reaches a certain level. The first party platform110

There may be many different ways to identify the key indicators of when assistance should be offered/provided to the user, as each user is capable of exhibiting frustration and dealing with frustration differently. For example, common facial expressions, gestures, and vocal expressions can be monitored for purposes of predicting when frustration is setting in for a user. Furthermore, tracking of a user's behavior over time can also provide adaptive indications of when a particular user is becoming frustrated. The system may first establish baseline measurements for identifiers of frustration (i.e. biometrics, facial expressions, vocal expressions) in non-frustrating parts of the game, such as during opening credits, cut-scenes, or menus. These measurements indicate the user's physical and behavioral state when not frustrated. Once baseline measurements have been established, deviations from that baseline can be correlated with frustrating situations in the game to establish a pattern by which to identify how a particular user expresses frustration. Cameras and/or other sensors may capture these reactions, associating them with the particular user as custom indicators of that user's frustration. For example, when a user fails to accomplish an objective in the game, his heart rate may spike and his facial expression may indicate anger. A different user might express frustration in the same situation through a shouted explicative. The system may learn the indicators of frustration for each individual user, and provide assistance content based to a particular user based on the known indicators of frustration for that user. In addition, users could also input their own identifiers as well as confirm and/or modify existing associations regarding indication of frustration for the user as appropriate so that the system can better identify when to offer assistance.

Additional details regarding how the system can configure the key identifiers and thresholds for identifying when to offer/provide assistance for the user will be provided inFIG. 5with respect to the performance server140. In particular, configuration for the user can be based on information obtained about the user that can be processed, analyzed, and stored within the performance server140. Each user can have a user profile that stores information over a period of time that could be usable to configure the key identifiers and thresholds for the user. Furthermore, configuration for the user can also be based on information obtained from other users playing the same video game as well. Information regarding various user performances within the video game and, for example, frustration levels with various events/challenges/obstacles could be collected, processed, analyzed, and stored within the performance server140. By comparing, for example, overall performance of other users facing the same event/challenge/obstacle, an average performance or frustration that other users experience can be obtained and used to compare with against the user. Thresholds for when the system should provide assistance to the user can also be adjusted based on the experiences of the user and other users.

FIG. 2is an example gaming console200used in context of the system for identifying player engagement for the purpose of generating contextual game play assistance ofFIG. 1. The gaming console200(which in some cases may also include other computing devices such as desktops, laptops, tablets, and mobile devices) is a device that the user can utilize to play various video games associated with the first party platform ofFIG. 1. Furthermore, the gaming console200allows the user to connect with the first party platform associated with the video games so that user gameplay can be monitored and recorded in order to identify the current player engagement and what types of assistance may be offered/provided to the player.

The gaming console200may include various elements as illustrated inFIG. 2, however, it should be noted that the elements are exemplary and that other embodiments may incorporate more or less than the elements illustrated. With reference toFIG. 2, the gaming console200includes a main memory202, a central processing unit (CPU)204, at least one vector unit206, a graphics processing unit208, an input/output (I/O) processor210, an I/O processor memory212, a controller interface214, a memory card216, a Universal Serial Bus (USB) interface218, and an IEEE 1394 interface220, an auxiliary (AUX) interface222for connecting a tracking device224, although other bus standards and interfaces may be utilized. The gaming console200further includes an operating system read-only memory (OS ROM)226, a sound processing unit228, an optical disc control unit230, and a hard disc drive232, which are connected via a bus234to the I/O processor210. The gaming console200includes at least one tracking device224.

The tracking device224may be a camera, which includes eye-tracking capabilities. The camera may be integrated into or attached as a peripheral device to the gaming console200. In typical eye-tracking devices, infrared non-collimated light is reflected from the eye and sensed by a camera or optical sensor. The information is then analyzed to extract eye rotation from changes in reflections. Camera-based trackers focus on one or both eyes and record their movement as the viewer looks at some type of stimulus. Camera-based eye trackers use the center of the pupil and light to create corneal reflections (CRs). The vector between the pupil center and the CR can be used to compute the point of regard on surface or the gaze direction. A simple calibration procedure of the viewer is usually needed before using the eye tracker.

Alternatively, more sensitive trackers use reflections from the front of the cornea and that back of the lens of the eye as features to track over time. Even more sensitive trackers image features from inside the eye, including retinal blood vessels, and follow these features as the eye rotates.

Most eye tracking devices use a sampling rate of at least 30 Hz, although 50/60 Hz is most common. Some tracking devises run as high as 1250 Hz, which is needed to capture detail of very rapid eye movement.

A range camera may instead be used with the present invention to capture gestures made by the user and is capable of facial recognition. A range camera is typically used to capture and interpret specific gestures, which allows a hands-free control of an entertainment system. This technology may use an infrared projector, a camera, a depth sensor, and a microchip to track the movement of objects and individuals in three dimensions. This user device may also employ a variant of image-based three-dimensional reconstruction.

The tracking device224may include a microphone integrated into or attached as a peripheral device to gaming console200that captures voice data. The microphone may conduct acoustic source localization and/or ambient noise suppression. The microphones may be usable to receive verbal instructions from the user to schedule, retrieve and display content on the gaming console200.

Alternatively, tracking device224may be the controller of the gaming console200. The controller may use a combination of built-in accelerometers and infrared detection to sense its position in 3D space when pointed at the LEDs in a sensor nearby, attached to, or integrated into the console of the entertainment system. This design allows users to control functionalities of the gaming console200with physical gestures as well as button-presses. The controller connects to the gaming console200using wireless technology that allows data exchange over short distances (e.g., 30 feet). The controller may additionally include a “rumble” feature (i.e., a shaking of the controller during certain points in the game) and/or an internal speaker.

The controller may additionally or alternatively be designed to capture biometric readings using sensors in the remote to record data including, for example, skin moisture, heart rhythm, and muscle movement.

As noted above, the gaming console200instead be implemented as a general-purpose computer, a set-top box, or a hand-held gaming device. Further, similar user devices may contain more or less operating components.

The CPU204, the vector unit206, the graphics processing unit208, and the I/O processor210communicate via a system bus236. Further, the CPU204communicates with the main memory202via a dedicated bus238, while the vector unit206and the graphics processing unit208may communicate through a dedicated bus240. The CPU204executes programs stored in the OS ROM226and the main memory202. The main memory202may contain pre-stored programs and programs transferred through the I/O Processor210from a CD-ROM, DVD-ROM, or other optical disc (not shown) using the optical disc control unit232. The I/O processor210primarily controls data exchanges between the various devices of the user device200including the CPU204, the vector unit206, the graphics processing unit208, and the controller interface214.

The graphics processing unit208executes graphics instructions received from the CPU204and the vector unit206to produce images for display on a display device (not shown). For example, the vector unit206may transform objects from three-dimensional coordinates to two-dimensional coordinates, and send the two-dimensional coordinates to the graphics processing unit208. Furthermore, the sound processing unit230executes instructions to produce sound signals that are outputted to an audio device such as speakers (not shown).

A user of the gaming console200provides instructions via the controller interface214to the CPU204. For example, the user may instruct the CPU204to store certain information on the memory card216or instruct the user device200to perform some specified action. Example controllers associated with the controller interface214may include a touch-screen, keyboards and game controllers.

Other devices may be connected to the gaming console200via the USB interface218, the IEEE 1394 interface220, and the AUX interface222. Specifically, a tracking device224, including a camera or a sensor may be connected to the user device having the first party portal200via the AUX interface222, while a controller may be connected via the USB interface218.

FIG. 3illustrates an example first party platform300ofFIG. 1. As described herein, the first party platform300is a network used to integrate and support video games associated with a particular platform (e.g. gaming console). The first party platform300may be implemented via a plurality of computing devices (e.g. servers) and/or within the cloud.

The first party platform300includes a number of different features that can be used to identify player engagement for the purpose of generating contextual game play assistance. As illustrated inFIG. 3, the first party platform300includes a database310of all video games that are currently supported by the first party platform300. The game database310may include information used by the gaming consoles to execute instances of the game in the cloud using the first party platform300. This may allow users to participate in multiplayer video games with other users on a shared gaming network. In some embodiments, users may download necessary information from the game database310to execute the particular video game on the user gaming console. This may be beneficial in allowing users to download updates (e.g. downloadable content, patches) for executing a single player game directly on their gaming console.

Since new video games are constantly being published and released for user consumption, an application programming interface (API)320is provided within the first party platform300to allow publishers to update the game database310with new information used for new video games as needed. The API320includes an interface that allows for publishers to upload new video game information to be stored within the game database310.

Users may also have their own user profiles330stored within the first party platform300. The user profiles330may include information such as what video games the user has played, gameplay details regarding progress within each of those video games, as well as information (e.g. achievements, trophies) regarding how well the user has performed within each of those video games. The user profile330may also include numerous other statistics that can be used to further characterize the user (e.g. gameplay behavior) associated with the video games with the first party platform as well as preferences regarding the types of assistance information the user would like to receive and when (e.g. thresholds) the assistance information should be offered/provided.

The gameplay monitoring module340receives information from the various video games being played by the user. In situations where the video games have ways for tracking user progress during gameplay, such information can be provided for direct use by the gameplay monitoring module340. For example, such information may include user performance such as time played log, accuracy, kill-death ratio, and character statistics within game. This information can be used to characterize a user performance within the video game and identify whether the user should be offered/provided assistance. This information can be transmitted to a performance server (illustrated inFIG. 5) using the communication interface350.

The communication interface350facilitates the transmission of information between the first party platform300and various other servers (e.g. assistance server, performance server) regarding the types of assistance information being offered/provided to the users or identifying when the assistance information should be offered/provided. The communication interface350also allows the users to communicate with the first party platform300via their consoles.

FIG. 4illustrates an example assistance server400ofFIG. 1. The assistance server includes a database410used to store and organize the various assistance information for various video games associated with the first party platform. The assistance server also includes communication interfaces420and/or application programming interfaces (APIs)430that allows association information to be uploaded and/or modified within the database. Lastly, the assistance server may have an assistance information processing application440that allows for the characterization, labeling, and rating of the assistance information associated with the assistance server400.

The assistance database410stores all the various assistance information that may be offered/provided to the user. The assistance information may be organized, for example, based on the applicable video game. Further organization can include type of assistance information, source/provider of the assistance information, when the assistance information was uploaded or last modified, rating/accuracy of the assistance information, and label regarding the event/challenge/obstacle the assistance information pertains to.

To facilitate building up the assistance database410that includes the assistance information for the various video games associated with the first party platform, the assistance server400may include communication interfaces420and/or application programming interfaces430that allow various users to upload new assistance information and/or modify existing assistance information. Such assistance information can be labeled by the source regarding the type of information being uploaded (e.g. hint, advice, walk-through) as well as referencing the identity of the source for the assistance information.

As assistance information is provided to users playing the video game, the recipients of the assistance information can provide ratings regarding how useful the assistance information is. These ratings can be collected via user input provided via their consoles and aggregated to be associated with the particular assistance information stored within the assistance server. The ratings can be received, for example, via in-game comments provided by the users that are transmitted to the assistance server using the communications interface420. Ratings from the users who were provided the assistance information could be beneficial in identifying what assistance information is accurate and beneficial. Various rating scales can be used to identify those information that is inaccurate or unhelpful and distinguishable from the information that was beneficial, clear, and accurate. Users may also provide commentary regarding why such assistance was unhelpful and inaccurate. The commentary may be beneficial in identifying those comments that could be flagged or removed. Furthermore, the assistance information of each type with higher ratings may be emphasized. Furthermore, contributors of consistently higher rating assistance information can be featured/emphasized.

Users can also provide requests for specific types of assistance information as well as assistance information from various sources (e.g. contributors). The assistance server, in organizing the assistance information stored within the database, allows for this customization of the types of assistance information that is being received by the user. The assistance information processing application440can ensure that the appropriate assistance information (e.g. pertaining to the corresponding video game, the preferred type/source) is selected for the user. The assistance information processing application440can also process the ratings for the assistance information stored within the assistance database410and provide/update an overall rating that users can reference. The overall rating can be used to rank or highlight more accurate or beneficial assistance information.

FIG. 5illustrates an example performance server500ofFIG. 1. The performance server500is used to identify when assistance information should be provided to users on a per-user basis. This is done by analyzing and comparing user performance within the video game and other user-related information (e.g. vocal, visual, biometric information). The performance server includes user profiles510that store information about each user. For example, the user profiles510may generally have information regarding overall performance thresholds indicating when to offer/provide assistance to users. Since different users may experience frustrations or have other issues in video games differently than other players, each user profile can stored the identified behavior of the user experiences frustrations (e.g. if the user becomes easily frustrated). In this way, assistance information can be provided accordingly.

The user profiles510may initially begin with standard indicators of when frustration may occur. The standard indicators can be based on calculations performed on various other users playing the same video game at the same obstacles within the video game. For example detection of an expletive can be used as an indicator. Other examples can include increased heart rate and detection of certain gestures or facial expression can also be used. As the performance server500receives gameplay data and/or user-related data from the first party platform, the processor540can characterize and predict when the user actually is becoming frustrated and modify accordingly the profile of the user. Furthermore, the user may also be capable of modifying and/or adding to the user profile the indicators regarding what the performance server may look for to detect increasing frustration.

The user profile can also be modified to identify the types of assistance information to suggest for the user and the associated thresholds the information should be offered/provided. For example, the performance server may have three different thresholds where different levels of frustration can be identified (e.g. level A, B, and C with A being the least and C being the most). The user could have the different levels configured over a period of gameplay data obtained and identify the types of assistance information that may be beneficial. For example, users may want to attempt an event 5 times before getting a hint from the assistance server. If the user is still unable to overcome the event after 5 more times with the first hint, some advice may be provided. After additional 5 times with the advice, further advice or hints could be provided. Finally after an additional 10 times, the user may request a walkthrough. However, if the user exhibits signs of increased frustration (e.g. detection of an expletive once every few seconds) that increases in frequency, this may indicate an increased need to provide the next level of assistance. Furthermore, this may induce a modification to the threshold (e.g. every 3-4 times instead of previous 5 times).

As stated, the user can customize the types of assistance they receive within the game. Some users may never wish to receive walk-through information no matter how frustrated they are while others may wish to receive walk-through information immediately. The user is capable of customizing and modifying their user profiles as needed.

The performance server500also includes an application programming interface520that allows for the uploading and/or updating of user profiles510stored on the performance server500. For example, the user can utilize the application programming interface520to modify existing information stored in the user profile510that may include thresholds regarding when or what assistance information would be offered/provided to the user within game. The application programming interface520can also be used to modify or add other functionalities to the monitoring540or processing540features of the performance server500as appropriate.

The communication interface530of the performance server500facilitates communication with other servers (e.g. assistance server) and the first party platform. By using the communication interface, the performance server can provide each current performance data and/or user profile of the user to the assistance server so that the assistance server can select the appropriate assistance information (via the assistance information processing application) that can then be sent to the first party platform. The communication interface530also allows the performance server500to receive gameplay data and/or user-related data (e.g. sensor, camera, microphone) from the first party platform. The information coming from the first party platform can be used to characterize the current performance of the user and identify, for example, the current frustration level of the user. Based on various thresholds, for example, compared with pre-defined parameters set by the system or the user, a particular type of assistance information may be recommended.

The performance processor540analyzes the data about the user (e.g. gameplay, sensor, biometrics) coming from the first party platform that can be used to characterize the current performance of the user within the game. By comparing the performance of the user, the performance processor540can identify (based on thresholds) when a user may begin feeling frustrated with a particular aspect of the video game. For example, if the user keeps failing to complete an objective, the rate of failure of the user may be compared with a pre-determined threshold (e.g. 5 times) or compared with others rate of failure (e.g. average 7 times). For example, the performance processor540can use the user profile510(which may have pre-determined thresholds or thresholds calculated based on past performance) or use calculations pertaining to a plurality of similar players who have played the same video game and experiences similar issues at the same objective to determine if the user could potentially be experiencing frustration. Furthermore, other indicators such as increased heart rate, detection of moisture at the controller from the hands of the user, or frequency of expletives could also be indicators that the user is becoming increasingly frustrated. All this information can be weighted and aggregated to determine a level of frustration that corresponds to a recommendation regarding what type of assistance should be provided.

FIG. 6illustrates a method600for identifying player engagement for the purpose of generating contextual game play assistance. As described above, user gameplay and user-related information regarding a video game is obtained and used to identify user performance with respect to the video game. In situations where the user is repeatedly failing a particular event/obstacle/challenge, the method aims to identify when would be an appropriate time and what the appropriate assistance information that should be provided so that the user can advance within the video game.

In step610, conditions regarding any user preferences regarding performance thresholds and/or assistance information can be provided by the user. If no such preferences are received, predetermined criteria can be used. The predetermined criteria can later be modified by the user and/or other entities based on user performance data obtained over time. For example, if the user is detected as being easily frustrated, the thresholds regarding when assistance information should be provided may be decreased.

In step620, the user gameplay of a video game and any user-related data obtained via sensors or other devices can be monitored. These sets of data identify user performance within the game. For example, gameplay information may pertain to user statistics within the game such as time played, current event/location, character details, attempts, and current equipment. Some of the gameplay information would be used to identify what type of assistance information would be beneficial for the user. Other types of gameplay information could be used to identify if the user is becoming frustrated at having to repeat the same event or being stuck on the same event for an extended period of time.

Furthermore, user-related data obtained via sensors or other devices (e.g. controller, microphone, cameras) could be used to detect user mood over the same period of time. For example, detection of expletives, increased heart rate, or certain gestures or facial expressions could be indicative of increasing frustrations.

In step630, a current user performance may be identified. The user performance would identify, for example, a user frustration level regarding the current event/obstacle/challenge that is preventing the user from advancing within the video game. This user performance may be based on various thresholds that correspond to different levels of frustration as well as associated with different types of assistance information that may be beneficial in alleviating the level of frustration and allowing the user to advance within the video game.

In step640, the assistance information is provided to the user based on the identified current performance of the user. Based on the current performance of the user identified in step630, different types of assistance information can be provided. With the aim of reducing frustration levels, less detailed hints and advice may be provided initially that are aimed at directing the user in the right direction with regards to overcoming the current event/obstacle/challenge within the game. However, as the frustration level increases and surprises one or more thresholds, more detailed assistance may be necessary so that the user can finally overcome the event/obstacle/challenge and move on in the game.

In step650, the user profile can then be updated. After the assistance has been provided in step640, user gameplay and user-related data can then be monitored again to determine if the performance of the user improves and is able to progress within the video game. Generally, if the user overcomes the event with the assistance, the user will no longer exhibit indications of frustration or exhibit reduced signs of frustration. The processor of the performance server can correlate a degree of frustration for the user, the type of assistance that was provided and that it was beneficial for this particular instance. This information can be used to modify the user profile. For example, if the user frustration level was higher than the threshold that the user profile has when providing the corresponding assistance, the user profile may be updated to lower the threshold so that assistance can be provided earlier. An aim in lowering the threshold is so to prevent the user from reaching that high level of frustration in the future.

The frustration level of the user can also be compared with the general frustration level of other players within the same game facing the same event/obstacle/challenge. Based on whether the frustration level of the user is higher or lower than the general frustration level of the other players, the personal threshold level of the user can be adjusted accordingly since this may be indicative that the user is more or less frustrated than other players.

Steps620-650may be repeated for as long as the user is playing within the game so as to allow for an adaptive identification of player engagement within the video game. Based on the characterization of the user, the appropriate assistance directed at addressing the current event/obstacle/challenge that the user is stuck on could be provided.

The present system may not only be concerned with the type of assistance information that is provided based on the user performance. Other embodiments may also store different types of ‘success’ messages that can be provided to the user after detecting that the user successfully overcame the event/obstacle/challenge after receiving the assistance information. These ‘success’ messages may be stored within the performance server or other servers uniquely dedicated at providing the ‘success’ messages. The types of ‘success’ messages may vary from simple ‘congratulations’ to more extensive messages that include music, graphics and/or videos. Similar to how the assistance information attempts to reduce the frustration level of the user playing the video game, the ‘success’ messages also provide this support to the user as the user is now able to progress within the video game.

The foregoing detailed description of the technology herein has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the technology and its practical application to thereby enable others skilled in the art to best utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the technology be defined by the claim.