U.S. Pat. No. 12,350,588

DETAILED DESCRIPTION

As previously discussed, a user that encounters a difficult portion of a video game may seek assistance by searching for a tutorial or gameplay video that may provide an example solution or strategy. Doing so may be a disjointed and clumsy process that may involve exiting from the gaming experience, launching a video service or website, searching for a relevant video, observing the video as many times as desired, exiting from the video service or website, and beginning the game again. Even after performing these steps, the user may forget portions of the previously-observed gameplay technique, which may cause the user to repeat the process.

The present subject matter discloses systems and techniques for providing an in-game guide that may aid a user in progressing through a video game. The guide may be generated based on an analysis of one or more videos that exhibit a player playing the video game. Based on the analysis, the guide may determine a sequence of actions that may succeed in advancing a user through all or a portion of a game. The guide may be provided in a semi-transparent “ghost layer” over the user's in-game display to provide tips and examples of actions based on the analysis that may aid the player in progressing through the game. A ghost layer as disclosed herein may display textual, image, and/or video information, which may be displayed as an overlay of a game such that the game and the ghost layer are concurrently visible to the player. As the user plays the game, the ghost layer may update and adjust what is displayed to provide information relevant to the user based on the current game status, the user's past actions, and any prospective actions in the sequence of actions that may help the user succeed in advancing.

The term “game controller” as used herein refers to any device that may be used by a user to provide input to a game. A game controller may include, for example, a mouse, keyboard, keypad, joystick, stylus, wand, stylus, trackball, trackpad, touchpad, touchscreen, camera, radar transmitter and/or receiver, gamepad, smartphone, and microphone.

FIG.1illustrates an example embodiment of the present subject matter. A game screen100may be played from the view of a third person. The user may control a single stick-figure-like character105. To advance through this portion of the game100, the user may need to control the character105to perform three actions. First, the character may walk to the right until reaching the edge of a chasm115, then the character may jump over the chasm115, and third, the character may deploy a ladder item130. Upon receiving a user's request for assistance using a game interface, a ghost layer200may display a ghost character110. The ghost character110may walk to the right until reaching the chasm115, jump120over the chasm115, and deploy the ladder item130near the right edge of game screen100, where the ghost character110may wait125for the user-controlled player105to catch up. The ghost character110may be displayed identically as character105or may be distinguishable from character105. For example, ghost character110may be displayed in a transparent or semi-transparent manner, in a different color scheme, shade, or tint, in a flashing state, with a halo or highlight, as a silhouette, marked with a label or icon, or having a combination of any one or more of these features, so long as the ghost character110may be distinguished from the player character105.

Alternatively, or in addition to performing two or more tasks collectively and consecutively the ghost character110may wait for the user-controlled character105to perform an individual task before proceeding to a next task. For example, the ghost character110may perform a first task, such as jumping120over the chasm115, and then waiting for the user-controlled character105to do the same before proceeding to the next task. Upon character105jumping over the chasm115, the ghost character110may perform the second task of walking to the right edge of the game screen100and waiting125for the player's character105to do the same before proceeding to the next task, and so on. Whether the ghost character110performs a single task before waiting for the user-controlled character105or performs two or more tasks collectively before waiting for the user-controlled character105, the ghost character110may be displayed performing the individual or collective tasks repeatedly for a fixed or infinite number of times. In this way, the user may have an opportunity to see the task performed as many times as desired prior to attempting to do so. It should be appreciated that “ghost” objects may also be provided by the ghost layer200where the user-controlled character105is being guided to build an object or to use an item. For example, if the user-controlled character105is being guided to deploy the ladder item130or to push a crate (not shown), a ghost ladder item130or a ghost crate may appear to help the user to understand how to proceed. Once user-controlled character105deploys the ladder item130or otherwise interacts with the suggested object, the ghost object may disappear. Ghost objects may be distinguishable from the “real” game objects in any manner previously described with respect to distinguishing the ghost character110from the user-controlled character105.

FIG.2illustrates a ghost layer200that may be overlaid on a game environment201of an game currently being played by a user. The ghost layer200, including ghost player105and/or ghost objects, such as ladder item130, may be generated based on an analysis of one or more previously-recorded gameplay videos305. Gameplay videos305may be sourced from, for example, video-sharing websites, live-streamed video, and other video streaming sources, conventional non-transitory computer memory storage devices, or as transmitted across a network via e-mail, instant messaging, SMS, text message, and the like. A game may provide varying degrees of support for performing the analysis used to generate the ghost layer200. In an example, a game may include little or no support. An analysis of one or more sourced gameplay videos305for a non-supportive game may be performed by a computer-implemented process based on one or more of computer vision techniques, machine learning techniques, geometric properties, images, sounds, and the like. The source videos305for which the analysis may be based may be identified as tutorials, guides, gameplay sessions, contests, speed runs, and other videos where the actions of the player may be generally more successful in advancing through a game.

FIG.3Ais a flow diagram illustrating an example of a method300in accordance with the present subject matter. Method300may begin by analyzing one or more source videos305. The source videos305may be tutorials, guides, walkthroughs, or any other videos exhibiting gameplay of a game. The videos may or may not support embedded metadata that may identify each action of the player at each point in the game. Where the source video may not support embedded metadata, an analyzer350may segment a source video305into individual player actions360-362using an analyzer359, as shown in block diagram355ofFIG.3B. Method300may segment one or more selected source video(s)305into individual player actions at stage310. Each individual player action360-362may correspond to a user input received via a game controller device. Method300may identify each player action360-362by using a unique identifier363, a semantic description364, and a time value365reflecting the temporal duration of the associated player action360-362in315. Where a game video305supports embedded metadata376identifying each action (371,381,391) of the player, the analysis may be streamlined or eliminated altogether in that the player actions (371,381391) may be extracted directly without the additional processing that may be associated with segmenting the video305to extract each player action (371,381,391). Based on identifying the actions (371,381,391) of the player for a video, a virtual walkthrough405may be created in325that includes video segments (410,430,440,450,460) and an associated ordered list of player actions (411-413,431-433,451-453,454-457), as shown inFIG.4. The virtual walkthrough405may identify the game to which it is applicable using a game identifier470and may be considered a digital roadmap to successfully progressing through a game from beginning to end. The virtual walkthrough405may be incomplete, based on the availability of source videos305that succeed in progressing through a game470, but may nevertheless provide one example approach to completing, a game470. The virtual walkthrough405may comprise video segments (410,430,440,450,460) extracted from a variety of selected source videos305, where each video segment (410,430,440,450,460) may be relied upon for suggesting one or more player actions (411-413,431-433,451-453,454-457). As previously discussed, the game currently being played by the User may be known as the instant game. Method300may capture one or more frames from the instant game330to compare with the video segments (410,430,440,450,460) compiled by the virtual walkthrough405. A match algorithm may determine, based on the capture frames, whether a scenario or moment occurring in the instant game suitable corresponds to a scenario or moment in either a video segment (410,430,440,450,460) stored in the virtual walkthrough405or another separately available video segment in335. Where the match algorithm determines a suitable match336, the corresponding video segment (410,430,440,450,460) may be normalized by translating, rotating, and/or scaling the video, adjusting the image parameters, varying the playback speed, adding highlights, labels, or other indicators and elements, and removing irrelevant content to prepare the video segment for overlaying as the ghost layer200in340. More specifically, the video segments (410,430,440,450,460) to be normalized may include one or more player actions (411-413,431-433,451-453,454-457) to be suggested to the user based on the currently identified scenario or moment in the instant game. In345, one or more suggested player actions (411-413,431-433,451-453,454-457) may be displayed in the ghost layer200overlaying onto the game environment201of the instant game, such that both the current gameplay201and the ghost layer overlay200are visible to the player. For example, the ghost layer overlay200may be semitransparent to allow the underlying gameplay201to be concurrently visible. The suggested player actions (411-413,431-433,451-453,454-457) may be repeated a single time, a fixed number of times, or indefinitely until the user performs the action. Method300may monitor for and detect completion of a suggested player action (411-413,431-433,451-453,454-457) based on the user's game controller inputs, changes to the user's character, and other changes occurring to the environment201of the instant game. As previously discussed, the model5may be configured to emulate the user's game controller inputs to produce the suggested player action in addition to merely displaying a suggested player action (411-413,431-433,451-453,454-457) via the ghost layer200. Therefore, method300may detect completion of a suggested player action not only in response to receiving actual game controller inputs from a user, but also in response to receiving emulated game controller inputs from the model. Upon detecting completion of a suggested player action in350, the ghost layer200may be updated to display one or more subsequent actions (411-413,431-433,451-453,454-457), which may be normalized and rendered in the ghost layer200as previously described with respect to345. It should be appreciated that any stage of method300may be merged with other stages, performed in an order different from that which is shown inFIG.3A, repeated, removed altogether, or otherwise rearranged as needed without departing from the scope of the present subject matter.

In analyzing the one or more selected source videos305using analyzer359, the present subject matter may segment, identify, and distinguish the actions taken by a successful player to generate a virtual, action-by-notion walkthrough405for a game470, as shown inFIG.4. It should be understood fromFIG.4that each action (411-413,431-433,451-453,454-457) may include the associated identifier (414-416), semantic description (417-419), and duration (420-422), though this additional information is only shown for player actions411-413. The virtual walkthrough405may include screenshots, video segments (410,430,440,450,460), and an ordered list of each player action (411-413,431-433,451-453,454-457) determined to successively advance a player through a game470from beginning to end. Each identified action may share a one-to-one correspondence with a game controller input performed by the user. A game controller input that causes a user-controlled character to perform two or more “chained” actions automatically without entering an additional game controller input to cause the second and subsequent actions may be identified as a single action rather than two or more separate actions. The actions (411-413,431-433,451-453,454-457) may be suggested to the user on a per-game controller-input-basis to avoid confusing the user when suggesting a sequence of actions using the ghost layer200. For example, in the case where depressing a first button on the game controller causes the user-controlled game character to jump and flip, the analysis may identify the jump and flip in a video as a single jump/flip action rather than a first jump action followed by a second flip action. Actions that involve the player actuating a single game controller input for a period of time, such as to effect movement in a single direction, for example, may be identified as a single action of a longer duration or a series of two or more identical actions having shorter durations.

The virtual walkthrough405may include a set of one or more video segments (410,430,440,450,460) to form a comprehensive master video480exhibiting every action (411-413,431-433,451-453,454-457) taken to complete a game in order. The master video480may include gameplay video segments (410,430,440,450,460) performed by different players or the same player and may be retrieved from a variety of video sources. Portions of the master video480determined relevant to the scenario that the user is confronting, as subsequently described, may be retrieved from the master video480for playback in the ghost layer200. Each action in the virtual walkthrough may be identified by a unique identifier tag, a semantic description, and a time value reflecting the temporal duration of the associated action. For example, a portion of the virtual walkthrough may be tagged in a manner representative of, “Video Segment 1, Action 1, ID #000001,” with a description of “Move Right,” along with a time duration of 4.0 seconds.” In addition to the foregoing description where a game includes little or no support, a game may also provide stronger support for the ghost layer analysis. For example, a game may support identifying a player's actions from a video by implementing an application programming interface (API) that allows for emitting metadata376. The metadata376may identify each action (371,381,391) of the player at each point in the game, among other types of information that may be useful to the analysis. For example, the emitted metadata376may also contain information that identifies each time the player lost a life, received damage, or otherwise made a mistake. These metrics may be used to assess whether one gameplay video may exhibit a more successful gameplay approach when compared with another, and therefore may be a more suitable basis for the ghost layer200assistance. The gameplay videos305sourced and utilized to perform the analysis may be selected based on metadata376metrics reflecting the skill level of the player in each video. The API may be configured to allow embedding the metadata376within a source video305, thereby facilitating subsequent analysis of the source video305to create the virtual walkthrough405of the game. In this way, accurately identifying a player's action based on one or more source videos305may be improved and the complexity of and time needed to perform the analysis may be reduced or eliminated altogether.

It should be appreciated that where source videos305are unavailable for every point of progression through a game, a complete and comprehensive virtual walkthrough405may not be possible to produce without additional efforts. In this situation, the analyzer359may attempt to extrapolate from portions of the game, for which source videos305exist, that are deemed similar to other portions of the game, for which source videos305do not exist, to predict successful actions for the player to take. A source video305may be similar to a portion of a game where one or more of the quantity and/or type of enemies are the same, the map or level is the same, one or more obstacles are the same, or a threshold number of other game entities are the same. For example, if source videos305exist for level 3 but not for level 1, and level 3 is deemed similar enough to level 1, then analyzer359may determine that the source videos305for level 3 may be appropriate to inform the suggested actions of a player while playing level 1. A machine learning model5may be constructed to provide a ghost layer200to a user who is currently playing a game. The model5may, based on an input of one or more captured frames of gameplay by the user who is currently playing a game, determine whether a same or similar scenario may be found within the same game based on one or more previously-recorded videos. The videos or video segments that may be used to determine the scenario that the current user may be confronting may or may not be the same as the video segments (410,430,440,450,460) upon which the virtual walkthrough405is based. Said another way, a first video may be used to determine where, in terms of progression, a user is at within a game, while a second video may be used to determine one or more suggested actions (411-413,431-433,451-453,454-457) that a user may take.

Referring hack toFIG.2, the model5may overlay a screenshot or segment of video, known as the ghost layer200, retrieved from the virtual walkthrough405on to the game environment201to suggest to the user one or more next actions (411-413,431-433,451-453,454-457) that may facilitate advancement in the game470. In some instances, only a single action may be displayed to the user using the ghost layer200until the user performs that action. In other instances, two or more actions may be displayed sequentially. The actions displayed via the ghost layer200may occur once, may be repeated a fixed plurality number of times, or may be repeated indefinitely until the user performs the actions. The model5may detect that the user has performed the suggested actions displayed in the guide based on one or more of the user's game controller inputs, changes to the user's character, and other changes occurring to the game environment201.

The model5may provide a variety of enhancements and transformations to the screenshot or video segment (410,430,440,450,460) provided from the virtual walkthrough405. These processes may be referred to as “normalizing”340the previously-recorded video segments. The goal of normalizing340may be to process the video segments (410,430,440,450,460) so that they may be layered over the existing game environment201in a manner that aesthetically matches the point of view, scale, and motion, without obstructing the user's view. For example, the model5may translate, rotate, scale the screenshot or video segment to more closely match the perspective of the user in the current game environment201. The model5may add labels, highlights, and other indicators and elements to the screenshot or video segment. The model5may add or remove sounds present in the video segment from the virtual walkthrough405. The model5may remove portions of the screenshot or video segment, such as portions that may be irrelevant to the current player or the current game environment201scenario. For example, the model5may detect and identify entities shown in a previously-recorded gameplay video based on whether they entities are moving or non-moving. A non-moving entity may be determined to be a dead or otherwise inactive enemy, for example, and may not displayed to the user in the ghost layer200of the game currently being played. The model5may adjust the gamma/brightness, contrast, tint, and transparency of the screenshot or video segment. The model5may vary the playback speed of the video. For example, the model5may be configured to playback the ghost layer200video at a reduced speed to ease the user's ability to more easily recognize the action(s) being performed.

The model5may be implemented using a machine learning model. As previously discussed, the model5may, based on an input of one or more frames of gameplay330by a user who is currently playing a game, determine a same or similar scenario335for which one or more previously-recorded game videos exist. The one or more frames of the game currently being played330may form a video and as used herein, will be referred to as the instant game video. The previously-recorded game videos may be compiled into a comprehensive master video480to simplify retrieval from a single source, as previously discussed, or may remain separate. The model5may match335the instant game video330with a segment of previously-recorded game video305, either from the master video480or a separately sourced video. A match336may be determined based on a prediction score that reflects the model's certainty about whether the situation shown in the previously-recorded game video is the same or a similar situation in the instant game video. The matching algorithm implemented by the model5may be based on a scale-invariant feature transform (SIFT) algorithm that may identify and match key points between the previously-recorded game video305and the instant game video330. The matching algorithm may also be based on deep-learning where a neural network may be trained to learn an embedding space for the previously-recorded game video and the instant game video, and the matching may be performed based on the distance between the embeddings of the previously-recorded game video and the instant game video. Where the matching algorithm determines, based on meeting a threshold prediction score or meeting a threshold distance between the embeddings, that a previously-recorded video segment suitably matches with the instant game video330, the model5may determine that providing the previously-recorded video segment in the ghost layer200may be helpful to the user and provide it where enabled by the user to do so.

The ghost layer200may be invoked to provide a variable degree of guidance to the user. As previously discussed, the ghost layer200may be displayed to the user in the form of a video that may be transformed, adjusted, and presented to the user within the instant game environment201. Therefore, the user may learn by observing each action shown in the ghost layer200video. Where desired, the model5may also take the control from the user by emulating the corresponding game controller inputs to perform the actions suggested in the previously-recorded game video(s) and/or the virtual walkthrough405. This feature may be implemented using reinforcement learning algorithms, for example. In an embodiment, an action space may be defined that includes all possible inputs that may be received from a game controller. A reinforcement learning algorithm may train using sourced gameplay videos305to attempt to mimic the gameplay shown in the source videos305. In an example, each time an action is taken, a vector including the current frame, action, and next frame may generated. The next frame may be based on the gameplay image shown after emulating a game controller input to perform the action. A pixel-distance loss calculation may be used to identify differences between the next frame obtained after taking the action and the next frame following the action in a source video305. The user may be offered the option to configure the degree of guidance from the ghost layer200. For example, the user may prefer the model5take control of game controller inputs during more difficult portions of the instant game and to allow the user to merely observe the ghost layer200video and provide his or her own game controller inputs during easier portions. Model5may also vary the degree to which it takes control. For example, where a user may have difficulty due to a disability or injury, the model5may reduce the complexity and/or the number of controller inputs needed in order to advance the player through a portion of the game by supplementing. Model5may achieve this by supplementing the user's game controller inputs with additional game controller inputs, thereby reducing the total number of game controller inputs to be provided by the user and/or the number of simultaneous game controller inputs. During other portions of the game, the user may prefer to not display the ghost layer200at all and disable it during those times.

In an embodiment, the ghost layer200provided to the user in the instant game may be based on the performance of a player who is currently live-streaming the same game, rather than based on the performance of a player in one or more pre-recorded source videos305. That is, the actions of the live streaming, player may be provided in the ghost layer200that may be transformed as previously-discussed and that may overlay the user's game environment201of the instant game. In this way, the live-streaming, player may provide “coaching” in real-time to the ghost layer-assisted user.

In an embodiment, a process may be employed to determine whether a match exists between a sourced gameplay video305and one or more frames of a game currently being played by a user. Based on a sourced gameplay video305, a three-dimensional or two-dimensional game environment including one or more entities may be reconstructed. An entity may be, for example, an enemy, an enemy pattern, an object, a doorway, a building, a level, a map, a track, an obstacle, or other feature that defines the gameplay viewed by the user. Each entity within game environment201may be extracted and coarsely defined using a set of coordinates and/or polygon shapes that may be stored in metadata. The metadata may include additional information for each game entity, such as texture or image information, entity type, description, and the like. A gaming platform associated with the game currently being played by the user may describe and/or define the entities present at each game state. In this context, a game state may refer to a set of one or more frames occurring between actions taken by the user. A current game state including one or more defined entities presented to the user may be compared with the reconstructed game environment to determine if a match is likely. A match may indicate, for example, that the scenario presented to the player in the source video305is the same or similar as the game environment201presented to the user in the game currently being played. A match336may be determined using an extraction algorithm to extract and compare the entities shown in the constructed came environment with the entities shown in the game environment201currently displayed to the user during gameplay. In an example, the matching between the game environment201and the constructed game environment may be carried out based on an entity-by-entity comparison or similar. A greater number of same game entities between the game environment201and the constructed game environment may indicate a greater similarity, while a fewer number of same game entities may indicate less similarity. A threshold may be configured to establish a minimum quantity of game entities that may be deemed the same for a match336to be positively identified. Weights may be assigned to each game entity based on its type and incorporated into the matching algorithm. For example, in a racing game, a same racetrack between a source video305and the track on which the player is racing in the game currently being played may be weighted to provide a greater indicator of similarity than the number or type of other cars in which the player is racing against. The ghost layer200interface may overlay one or more frames from the source video305, in addition to labels, annotations, or other elements, to the user's current game environment201based both on the matching and/or based on the entities extracted from the source video305, as previously described.

Embodiments of the presently disclosed subject matter may be implemented in and used with a variety of component and network architectures.FIG.5is an example computing device20suitable for implementing embodiments of the presently disclosed subject matter. The device20may be, for example, a desktop or laptop computer, or a mobile computing device such as a smart phone, tablet, or the like. The device20may include a bus21which interconnects major components of the computer20, such as a central processor24, a memory27such as Random Access Memory (RAM), Read Only Memory (ROM), flash RAM, or the like, a user display22such as a display screen, a user input interface26, which may include one or more controllers and associated user input devices such as a keyboard, mouse, touch screen, and the like, a fixed storage23such as a hard drive, flash storage, and the like, a removable media component25operative to control and receive an optical disk, flash drive, and the like, and a network interface29operable to communicate with one or more remote devices via a suitable network connection.

The bus21allows data communication between the central processor24and one or more memory components, which may include RAM, ROM, and other memory, as previously noted. Typically, RAM is the main memory into which an operating system and application programs are loaded. A ROM or flash memory component can contain, among other code, the Basic Input-Output system (BIOS) which controls basic hardware operation such as the interaction with peripheral components. Applications resident with the computer20are generally stored on and accessed via a computer readable medium, such as a hard disk drive (e.g., fixed storage23), an optical drive, floppy, disk, or other storage medium.

The fixed storage23may be integral with the computer20or may be separate and accessed through other interfaces. The network interface29may provide a direct connection to a remote server via a wired or wireless connection. The network interface29may provide such connection using any suitable technique and protocol as will be readily understood by one of skill in the art, including digital cellular telephone, WiFi, Bluetooth®, near-field, and the like. For example, the network interface29may allow the computer to communicate with other computers via one or more local, wide-area, or other communication networks, as described in further detail below.

Many other devices or components (not shown) may be connected in a similar manner (e.g., document scanners, digital cameras and so on). Conversely, all of the components shown inFIG.5need not be present to practice the present disclosure. The components can be interconnected in different ways from that shown. The operation of a computer such as that shown inFIG.5is readily known in the art and is not discussed in detail in this application. Code to implement the present disclosure can be stored in computer-readable storage media such as one or more of the memory27, fixed storage23, removable media25, or on a remote storage location.

FIG.6shows an example network arrangement according to an embodiment of the disclosed subject matter. One or more devices10,11, such as local computers, smart phones, tablet computing devices, and the like may connect to other devices via one or more networks7. Each device may be a computing device as previously described. The network may be a local network, wide-area network, the Internet, or any other suitable communication network or networks, and may be implemented on any suitable platform including wired and/or wireless networks. The devices may communicate with one or more remote devices, such as servers13and/or databases15. The remote devices may be directly accessible by the devices10,11, or one or more other devices may provide intermediary access such as where a server13provides access to resources stored in a database15. The devices10,11also may access remote platforms17or services provided by remote platforms17such as cloud computing arrangements and services. The remote platform17may include one or more servers13and/or databases15.

FIG.7shows an example arrangement according to an embodiment of the disclosed subject matter. One or more devices or systems10,11, such as remote services or service providers11, user devices10such as local computers, smart phones, tablet computing devices, and the like, may connect to other devices via one or more networks7. The network may be a local network, wide-area network, the Internet, or any other suitable communication network or networks, and may be implemented on any suitable platform including wired and/or wireless networks. The devices10,11may communicate with one or more remote computer systems, such as processing units14, databases15, and user interface systems13. In some cases, the devices10,11may communicate with a user-facing interface system13, which may provide access to one or more other systems such as a database15, a processing unit14, or the like. For example, the user interface13may be a user-accessible web page that provides data from one or more other computer systems. The user interface13may provide different interfaces to different clients, such as where a human-readable web page is provided to a web browser client on a user device10, and a computer-readable API or other interface is provided to a remote service client11.

The user interface13, database15, and/or processing units14may be part of an integral system or may include multiple computer systems communicating via a private network, the Internet, or any other suitable network. One or more processing units14may be, for example, part of a distributed system such as a cloud-based computing system, search engine, content delivery system, or the like, which may also include or communicate with a database15and/or user interface13. In some arrangements, a, a machine learning model5may provide various prediction models, data analysis, or the like to one or more other systems13,14,15.

More generally, various embodiments of the presently disclosed subject matter may include or be embodied in the form of computer-implemented processes and apparatuses for practicing those processes. Embodiments also may be embodied in the form of a computer program product having computer program code containing instructions embodied in non-transitory and/or tangible media, such as floppy diskettes, CD-ROMs, hard drives, USB (universal serial bus) drives, or any other machine readable storage medium, such that when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing embodiments of the disclosed subject matter. Embodiments also may be embodied in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such is over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, such that when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing embodiments of the disclosed subject matter. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits.

In some configurations, a set of computer-readable instructions stored on a computer-readable storage medium may be implemented by a general-purpose processor, which may transform the general-purpose processor or a device containing the general-purpose processor into a special-purpose device configured to implement or carry out the instructions. Embodiments may be implemented using hardware that may include a processor, such as a general purpose microprocessor and/or an Application Specific Integrated Circuit (ASIC) that embodies all or part of the techniques according to embodiments of the disclosed subject matter in hardware and/or firmware. The processor may be coupled to memory, such as RAM, ROM, flash memory, a hard disk or any other device capable of storing electronic information. The memory may store instructions adapted to be executed by the processor to perform the techniques according to embodiments of the disclosed subject matter.

The present subject matter may improve a user's enjoyment of a game by reducing the friction associated with finding and obtaining gameplay assistance. The embodiments disclosed herein may allow a user to receive in-game assistance while playing a game without exiting the game. The embodiments disclosed herein may relieve the user from searching for relevant gameplay videos on video sharing websites to observe a successful player advance through a portion of a game with which the user is having trouble. The embodiments disclosed herein may leverage the massive, pre-existing library of gameplay videos available to provide in-game assistance to a user in a manner that is convenient and saves time when compared with the conventional methods of finding assistance. The embodiments disclosed herein may assume varying degrees of control over the user's game controller inputs to reduce the difficulty of a game.

The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit embodiments of the disclosed subject matter to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to explain the principles of embodiments of the disclosed subject matter and their practical applications, to thereby enable others skilled in the art to utilize those embodiments as well as various embodiments with various modifications as may be suited to the particular use contemplated.

implementations disclosed herein can include systems, devices, arrangements, techniques, and compositions such as the following:1. A computer-implemented method comprising:receiving a source video;identifying a plurality of player actions that occur in gameplay captured in the source video;receiving a second video, the second video based on a currently executing game environment;determining a portion of the source video exhibits a first gameplay situation that is similar to a gameplay situation in the second video;adjusting a property of the determined portion c the source video to produce a guide video; andoverlaying the guide video on the currently executing game environment.2. The method of implementation 1, further comprising segmenting the source video portion into a plurality of player actions.3. The method of implementation 1 or 2, further comprising extracting a list of player actions from metadata embedded within the source video.4. The method of implementation 3, wherein the metadata further includes one or more metrics reflecting a skill level of a player and wherein the method further comprises selecting the source video based on the one or more metrics.5. The method of any one of the preceding implementations, further comprising:detecting a completed player action in the currently executing game environment; andoverlaying an updated guide video responsive to the detected completed player action.6. The method of any one of the preceding implementations, wherein each identified player action shares a one-to-one correspondence with a game controller input received from a user.7. The method of any one of the preceding implementations, further comprising compiling a virtual walkthrough based on:at least one of the source video and one or more secondary source videos; anda list of player actions corresponding to one or more segments of at least one of the source video and the one or more secondary source videos.8. The method of any one of the preceding implementations, wherein the determining is based on computing the distance between a first embedding of the segment of the source video with a second embedding of the second video across an embedding space.9. The method of any one of the preceding implementations, wherein adjusting the property includes one or more selected from the group consisting of:rotating the determined portion;translating the determined portion;scaling the determined portion;adjusting the playback speed of the determined portion;adding an element to the determined portion;removing an element from the determined portion; andadjusting the transparency of the determined portion.10. The method of any one of the preceding implementations, further comprising:identifying one or more suggested player actions of the plurality of player actions, whereinthe overlaying the guide video further comprises overlaying the one or more suggested player actions in the currently executing game environment; andthe detecting a completed player action further comprises detecting that a player performed one or more of the suggested player actions.11. The method of any one of the preceding implementations, wherein the completed player action is based on receiving emulated game controller inputs not originating from a game controller.12. The method of any one of the preceding implementations, wherein the determining the portion of the source video exhibits the first gameplay situation comprises selecting the source video from among a plurality of gameplay videos.13. The method of any one of the preceding implementations, wherein the receiving the source video and the identifying the plurality of player actions are performed subsequent to the receiving the second video.14. The method of any one of the preceding implementations, further comprising receiving a request from a user of the currently executing game environment to generate the guide video,wherein the overlaying the guide video is performed in response the request.15. A non-transitory computer-readable medium comprising instructions operable, when executed by one or more computing systems, to:receive a source video;identify a plurality of player actions that occur in gameplay captured in the source video;receive a second video, the second video based on a currently executing game environment;determine a portion of the source video exhibits a gameplay situation that is similar to a gameplay situation in the second video;adjust a property of the determined portion of the source video to produce a guide video; andoverlay the guide video on the currently executing game environment.16. The medium of implementation 15, further comprising instructions operable to segment the source video portion into a plurality of player actions.17. The medium of implementation 15 or 16, further comprising instructions operable to extract a list of player actions from metadata embedded within the source video.18. The medium of implementation wherein the metadata further includes one or more metrics reflecting a skill level of a player, andwherein the medium further comprises instructions operable to select the source video based on the one or more metrics.19. The medium of any one of the preceding implementations, further comprising instructions operable to:detect a completed player action in the currently executing game environment; andoverlay an updated guide video responsive to the detected completed player action.20. The medium of any one of the preceding implementations, wherein each identified player action shares a one-to-one correspondence with a game controller input received from a user.21. The medium of any one of the preceding implementations, further comprising instructions operable to compile a virtual walkthrough based on:at least one of the source video and one or more secondary source videos; anda list of player actions corresponding to one or more segments of at least one of the source video and the one or more secondary source videos.22. The medium of any one of the preceding implementations, wherein the determining is based on computing the distance between a first embedding of the segment of the source video with a second embedding of the second video across an embedding space.23. The medium of any one of the preceding implementations, wherein the instructions operable to adjust the property further comprises instructions operable to perform one or more selected from the group consisting of:rotate the determined portion;translate the determined portion;scale the determined portion;adjust the playback speed of the determined portion;add an element to the determined portion;remove an element from the determined portion; andadjust the transparency of the determined portion.24. The medium of any one of the preceding implementations, further comprising instructions operable to:identify one or more suggested player actions of the plurality of player actions, wherein:the instructions operable to overlay the guide video further comprises instructions operable to overlay the one or more suggested player actions in the currently executing game environment; andthe instructions operable to detect a completed player action further comprises instructions operable to detect that a player performed one or more of the suggested player actions.25. The medium of any one of the preceding implementations, wherein the completed player action is based on instructions operable to receive emulated game controller inputs not originating from a game controller.26. The medium of any one of the preceding implementations, wherein the instructions operable to determine the portion of the source video exhibits the first gameplay situation further comprises instructions operable to select the source video from among a plurality of gameplay videos.27. The medium of any one of the preceding implementations, wherein the instructions operable to receive the source video and the identify the plurality of player actions are performed subsequent to the instructions operable to receive the second video.28. The medium of claim15, further comprising instructions operable to receive a request from a user of the currently executing game environment to generate the guide video,wherein the instructions operable to overlay the guide video are performed in response the request.