U.S. Pat. No. 10,905,963

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1illustrates a system100configured to facilitate streaming of media streams, in accordance with one or more implementations. In some implementations, system100may comprise one or more of electronic storage102, processor104, external resources106, client computing platforms108, and/or other components. The various components of system100may be configured for electronic communication among one another. Such communication may be accomplished via one or more networks, one or more direct connections, one or more wireless connections, one or more wired connections, and/or other electronic communication media. One or more of the components of system100may be implemented in a server (not shown). The server may be in communication with client computing platforms108via a client/server architecture to enable users access to the server via client computing platforms108.

Processor104may be configured to provide information processing capabilities within system100. For example, processor104may configured to execute computer program modules that may include one or more of a game module110, a communication module112, an encoding module114, a vector processing module116, a streaming module118, an augmentation module120, a game play storage module122, a number-type conversion module124, and/or other modules. In some implementations, a given client computing platform108(e.g., a game console) may include one or more processors and/or computer program modules that are the same as or similar to processor104and/or the depicted computer program modules to facilitate operations and/or functions described herein. In other implementations, a server may include one or more processors and/or computer program modules that are the same as or similar to processor104and/or the depicted computer program modules to facilitate operations and/or functions described herein.

In some implementations, electronic storage102may comprise non-transitory electronic storage media that electronically stores information. Electronic storage102may include one or both of system storage that is provided integrally (e.g., substantially non-removable) with system100and/or removable storage that is removably connectable to system100via, for example, a port (e.g., a USB port, a firewire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage102may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. Electronic storage102may include virtual storage resources, such as storage resources provided via a cloud and/or a virtual private network. Electronic storage102may store software algorithms, information determined by processor104, information received via client computing platforms108, and/or other information that enables system100to function properly. Electronic storage102may be a separate component within system100, or electronic storage102may be provided integrally with one or more other components of system12(e.g., processor104, client computing platforms108, servers of system100, etc.). For example, in certain implementations, the non-transitory electronic storage media of electronic storage102may include a cache (e.g., L1 cache, L2 cache, etc.) of processor104.

In some implementations, electronic storage102, processor104, external resources106, client computing platforms108, and/or other components (e.g., servers of system100) may be operatively linked via one or more electronic communication links. For example, such electronic communication links may be established, at least in part, via a network such as the Internet and/or other networks. It will be appreciated that this is not intended to be limiting, and that the scope of this disclosure includes implementations in which electronic storage102, processor104, external resources106, client computing platforms108, and/or other components may be operatively linked via some other communication media.

In some implementations, the external resources106may include sources of information, hosts and/or providers of virtual spaces outside of system100, external entities participating with system100, and/or other resources. In some implementations, some or all of the functionality attributed herein to external resources106may be provided by resources included in system100.

In some implementations, processor104may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information. Although processor104is shown inFIG. 1as a single entity, this is for illustrative purposes only. In some implementations, processor104may include a plurality of processing units. These processing units may be physically located within the same device, or processor104may represent processing functionality of a plurality of devices operating in coordination. Processor104may be configured to execute modules110,112,114,116,118,120,122,124, and/or other modules. Processor104may be configured to execute modules110,112,114,116,118,120,122,124, and/or other modules by software; hardware; firmware; some combination of software, hardware, and/or firmware; and/or other mechanisms for configuring processing capabilities on processor104.

It should be appreciated that although modules110,112,114,116,118,120,122, and124are illustrated inFIG. 1as being co-located within a single processing unit, in implementations in which processor104includes multiple processing units, one or more of modules110,112,114,116,118,120,122, and/or124may be located remotely from the other modules. The description of the functionality provided by the different modules110,112,114,116,118,120,122, and/or124described below is for illustrative purposes, and is not intended to be limiting, as any of modules110,112,114,116,118,120,122, and/or124may provide more or less functionality than is described. For example, one or more of modules110,112,114,116,118,120,122, and/or124may be eliminated, and some or all of its functionality may be provided by other ones of modules110,112,114,116,118,120,122, and/or124. As another example, processor104may be configured to execute one or more additional modules that may perform some or all of the functionality attributed below to one of modules110,112,114,116,118,120,122, and/or124.

Game module110may be configured to execute an instance of a game world, for instance, at a game console. Game module110may be configured to generate, based on the game world, a primary view of the game world for local viewing. By way of example, the primary view may be a view rendered to a player of the instance of the game world on a display locally connected to his/her game console. The primary view may be generated based on interactions among various players of the game world, interactions of the players with simulated elements of the game world, simulated events of the game world, augmented elements, and/or other features. The primary view may be rendered based on one or more media formats (e.g., image format, video format, etc.) to depict the various interactions, simulated events, augmented elements, and/or other features. Communication module112may, for instance, be configured to obtain action requests specifying in-game actions. Game module110may be configured to implement requested actions in the instance of the game world. The requested actions may cause one or more simulated events in the instance of the game world, resulting in the generation of a primary view depicting the simulated events.

For example,FIG. 2illustrates a primary view200of a game world, in accordance with one or more implementations. As shown, primary view200depicts user characters representing players202a,202b, and202c, simulated events204(e.g., gunshots being fired), and simulated elements206(e.g., cliffs, rocks, chains, buildings, etc.) associated with the environment of the game world, among other features. Players202may, for instance, interact with one another through their user characters of the game world to complete missions, to engage in combat, and/or perform other tasks. Players202may input action requests specifying in-game actions, such as moving around in the game world, changing viewpoints of their user characters, discharging bullets from their guns, etc. These in-game actions may, for instance, cause one or more simulated events (e.g., simulated events204), such as gunshots being fired, explosions, death of a user character of a player202, etc., to be rendered as part of primary view200along with the user characters, simulated elements206, and/or other features.

The game world may, for instance, be a virtual space. An instance of the virtual space may be executed by computer modules to determine views of the virtual space. The views may then be communicated (e.g., via streaming, via object/position data, and/or other information) from server(s) and/or other sources to client computing platform(s)108for presentation to users. The view determined and transmitted to a given client computing platform108may correspond to a user character being controlled by a user via given client computing platform108. The view determined and transmitted to a given client computing platform108may correspond to a location in the virtual space (e.g., the location from which the view is taken, the location the view depicts, and/or other locations), a zoom ratio, a dimensionality of objects, a point-of-view, and/or view parameters. One or more of the view parameters may be selectable by the user.

The instance of the virtual space may comprise a simulated space that is accessible by users via clients (e.g., client computing platforms108) that present the views of the virtual space to a user. The simulated space may have a topography, express ongoing real-time interaction by one or more users, and/or include one or more objects positioned within the topography that are capable of locomotion within the topography. In some instances, the topography may comprise a 2-dimensional topography. In other instances, the topography may comprise a 3-dimensional topography. The topography may include dimensions of the space, and/or surface features of a surface or objects that are “native” to the space. In some instances, the topography may describe a surface (e.g., a ground surface) that runs through at least a substantial portion of the space. In some instances, the topography may describe a volume with one or more bodies positioned therein (e.g., a simulation of gravity-deprived space with one or more celestial bodies positioned therein). The instance executed by the computer modules may be synchronous, asynchronous, and/or semi-synchronous.

The above description of the manner in which views of the virtual space are provided is not intended to be limiting. The virtual space may be expressed in a more limited, or more rich, manner. For example, views determined for the virtual space may be selected from a limited set of graphics depicting an event in a given place within the virtual space. The views may include additional content (e.g., text, audio, pre-stored video content, and/or other content) that describes particulars of the current state of the place, beyond the relatively generic graphics. For example, a view may include a generic battle graphic with a textual description of the opponents to be confronted. Other expressions of individual places within the virtual space are contemplated.

Within the instance(s) of the virtual space, users may control characters, objects, simulated physical phenomena (e.g., wind, rain, earthquakes, and/or other phenomena), and/or other elements within the virtual space to interact with the virtual space and/or one another. The user characters may include avatars. As used herein, the term “user character” may refer to an object (or group of objects) present in the virtual space that represents an individual user. The user character may be controlled by the user with which it is associated. The user controlled element(s) may move through and interact with the virtual space (e.g., non-user characters in the virtual space, other objects in the virtual space). The user controlled elements controlled by and/or associated with a given user may be created and/or customized by the given user. The user may have an “inventory” of virtual goods and/or currency that the user can use (e.g., by manipulation of a user character or other user controlled element, and/or other items) within the virtual space.

The users may participate in the instance of the virtual space by controlling one or more of the available user controlled elements in the virtual space. Control may be exercised through control inputs and/or commands input by the users through client computing platforms108. The users may interact with one another through communications exchanged within the virtual space. Such communications may include one or more of textual chat, instant messages, private messages, voice communications, and/or other communications. Communications may be received and entered by the users via their respective client computing platforms108. Communications may be routed to and from the appropriate users through server(s).

Encoding module114may be configured to generate, at a game console, a media stream including a secondary view of the game world. The secondary view may be a lower-quality version of the primary view. By way of example, compared with the primary view, the secondary view may be associated with one or more of a lower resolution, frame rate, bit rate, and/or other quality factors. In one scenario, the secondary view may be associated with a higher frame rate than the primary view, but may be associated with a lower resolution and a lower bit rate. In another scenario, requirements for secondary views of the game world in the media stream may be between 15-30 frames per second, 320-640 pixels wide, and a bit rate of 1024 kilobits per second or less. On the other hand, requirements for primary views of the game world may be 15-30 frames per second, high-definition resolutions (e.g., 720p, 1080p, etc.), and 2048 kilobits per second or more. It should be noted that, although the secondary view may be a lower-quality version of the primary view, the quality of the primary and secondary views may not necessarily be distinguishable to viewers. For instance, with respect toFIG. 2, while the secondary view may be a lower-quality version of primary view200, the lower quality may not necessarily be noticed by viewers of the media stream containing the secondary view.

Encoding module114may be configured such that the generation of a media stream includes: prefetching, to the cache of the game console, one or more of portions of the primary view or instructions for encoding the portions of the primary view; and encoding, based on the prefetching, the portions of the primary view to produce the secondary view. For example, due to lower memory and processing capabilities of game consoles (e.g., compared with conventional desktops), traditional encoding approaches may not allow a game console to effectively encode and stream a current game session during the game session. As such, by prefetching data to the cache of the game console (e.g., L1 cache, L2 cache, etc.), and by encoding primary views of the game session using the prefetched data, a game console may avoid large latencies associated with directly loading the data from main memory to encode the primary views. As a result, the game console may be configured to more effectively encode and stream a current game session in real-time. In this way, users may effectively stream their current game sessions from their respective game consoles as they play in the game sessions.

In certain implementations, encoding module114may comprise code that indicates the prefetching to the cache and the encoding of the portions of the primary view to produce the secondary view. By way of example, the code may be written such that it indicates prefetching of the primary view portions and/or the encoding instructions to L1 cache, L2 cache, etc., of the gaming console in a way that reduces delays associated with cache misses. In one scenario, prefetching may be performed based on cache miss predictions such that data predicted to incur a cache miss may be prefetched to cache before that data is needed. In another scenario, certain lines of code may be grouped together to optimize the prefetching/encoding process (e.g., based on calculated frequency of use associated with the primary view portions and/or the encoding instructions).

In some implementations, the one or more processors may include a first processor of a first processor type and a second processor of a second processor type different from the first processor type, and the code of encoding module114may indicate execution of a first part of the code using the first processor and execution of a second part of the code using the second processor. For example, a PS3™ (PlayStation® 3) game console may multiple processing units of various types, such as a VMX (Vector Multimedia Extension), a SPU (Synergistic Processor Unit), a PPU (Physics Processing Unit), and/or other processing units. Although certain parts of the code may generally be more suitable for processing by a processing unit of a first processing unit type (e.g., PPU), the code may indicate processing of those certain parts of the code by a processing unit of second processing unit type (e.g., SPU), for instance, to balance the utilization of the processing units so that the processing unit of the first processing type is not significantly overutilized (e.g., to avoid waiting delays) and the processing unit of the second processing type is not significantly underutilized (e.g., to avoid waste of available processing power).

In certain implementations, the code may indicate direct memory access (DMA) transfers of the primary view portions and/or the encoding instructions to local memory (e.g., the cache, designated area of memory for threads, etc.). For example, game consoles such as the PS3 may include a specialized processing unit (e.g., SPU) that requires that data be stored in local memory prior to processing of the data by the specialized processing unit.

In various implementations, vector processing module116may include encoding module114such that vector processing module116may be configured to initiate the prefetching to the cache and the encoding of the portions of the primary view to produce the secondary view. In one use case, vector processing module116may interact with the VMX of the gaming console to perform vector operations to optimize routines associated with the encoding process. In some implementations, vector processing module116may be configured to load, in parallel, one or more of a first portion of the portions or the instructions for encoding the first portion from the cache to a first register associated with vector processing module116, and one or more of a second portion of the portions or the instructions for encoding the second portion from the cache to a second register associated with vector processing module116, such that the encoding of the portions of the primary view to produce the secondary view may be based on the loading from the cache to the first and second registers. In this way, by loading the first and second registers in parallel, vector processing module116may overcome slower processing speeds associated with game consoles by reducing loading delays (e.g., waiting for the loading of a register to be completed).

In certain implementations, portions of the primary view may be represented by a first set of numbers of a first number type. Number-type conversion module124may be configured to convert the first set of numbers into a second set of numbers of a second number type, and the encoding of the portions of the primary view to produce the secondary view may be based on the conversion. For example, the VMX of the Xbox 360® game console is configured to perform direct multiplication of floats, but is not configured to perform direct multiplication of integers due to a lack of integer multiplies. Thus, in such a scenario, encoding module114may work with the number-type conversion module to convert integers into floats when multiplication is needed for encoding. In some implementations, encoding module114may be configured to initiate lookup table function calls to get around the lack of integer multiplies (and/or other missing mathematical operations). For example, encoding module114may be configured to pre-compute one or more multiplications and store the pre-computed results in an array. At runtime, encoding module114may be configured with the pre-computed results such that the array may be “looked up” via the lookup table function calls (e.g., of encoding module114) to retrieve the pre-computed results for certain multiplications during the encoding process. In this way, computational load associated with conversion of the integers to floats for multiplications (and/or other mathematical operations) may be mitigated.

Streaming module118may be configured to effectuate streaming of the media stream (e.g., from the game console to a web server). For example, as a result of the optimizations described herein (e.g., prefetching of data to the cache of the game console, encoding of the prefetched data at the game console, explicit resource utilization balancing, etc.), the game console may be configured to effectively encode and stream a current game session while facilitating operations necessary to effectuate game play for the game session.

In some implementations, game module110may be configured to obtain game play information associated with a previous game session of a game. The game play information may define previous game play of the previous game session. The previous game play may, for instance, include actions initiated by players of the previous game session, interactions among user characters associated with the players, attributes of the user characters, environments of the previous game session (e.g., in which the actions were initiated, in which the user characters interacted, etc.), and/or other game play. For example, with respect toFIG. 2, assuming that primary view200represents a captured view of previous game play, the game play information may specify the movement of user characters of players202, the triggering of the weapons of user characters of players202, etc., along with the environment in which the actions took place, the sequence of the actions, and/or other information relating to the previous game play.

Game module110may be configured to execute the game play information to reproduce the previous game play during a reproduction of the previous game session. For example, the previous game play may be reproduced by reinitiating the actions defined in the game play information in their respective sequence order within the context of the environment in which the actions took place. In some implementations, game module110may be configured such that the reproduction of the previous game session may include generating one or more views of the previous game session for presentation to the user. By way of example, the views of the previous game session may include views that may have been presented to the players in the previous game session based on the previous game play associated with those players (e.g., the actions initiated by those players). In one use case, the one or more views of the previous game session may be generated and presented to a user facilitating augmentation of the reproduced game session to enable the user to see the previous game play as the user is providing augmentations to the reproduced game session.

In another scenario, the views of the previous game session that are presented to an augmenting user may be based on selection of the views by the user. The user may, for instance, be initially presented with a default viewpoint of one of the players, and the initial views of previous game session may be based on the default viewpoint. However, the user may select to switch to a viewpoint of another player to be presented with views based on the selected viewpoint.

In various implementations, game module110may be configured such that one or more properties of the reproduction of the previous game session are controllable by an augmenting user during the reproduction of the previous game session. The one or more properties may include viewpoint, magnification field, inclusion of screen objects, and/or other properties. As an example, an augmenting user may select the viewpoint from which the view presented to the user is based (e.g., viewpoints of players in the previous game session, new viewpoints unavailable to players in the previous game session, etc.) and the magnification field of the selected viewpoint such that the view presented to the user is a zoomed-in/zoomed-out view (e.g., a zoomed-in view generated based on a first-person viewpoint that is zoomed-in on a particular view area, a zoomed-out view generated based on a third-person viewpoint that is zoomed-out, etc.). As another example, the views of the previous game session may include screen objects (e.g., compass, maps, alerts, etc.) that were accessible to players in the previous game session. The user may opt to leave out such screen objects from the views presented to the user such that media streams created from those views will not render those screen objects. In this way, the control of the properties may enable the user to better analyze the previous game play, for instance, to provide better commentary for the previous game play since the user may be able to adjust the viewpoints along with the focus of the viewpoints to better suit his/her needs.

Augmentation module120may be configured to effectuate adding of augmentation content to the reproduced game session in response to user input received from an augmenting user during the reproduction of the previous game session. The addition of the augmentation content to the reproduced game session may create an augmented game session. In one use case, an augmenting user may be a commentator and may activate the reproduction of the previous game session to provide commentary of the previous game play in an augmented game session. The user may, for instance, be able to input requests to jump to any viewpoint (e.g., first-person, third-person, etc.) of any of the players of the previous game session to provide commentary based on what is seen through those viewpoints (e.g., the viewpoints of various players). In another scenario, the user may add screen objects (e.g., a map displaying the various locations of the players, a leaderboard showing the scores of the players, a clock/timer, etc.) to the reproduced game session such that a media stream of the augmented game session will include a rendering of the screen objects.

By way of example,FIGS. 3A and 3Billustrate a media stream of an augmented game session300created during reproduction of a previous game session, in accordance with one or more implementations.FIGS. 3A and 3Bmay, for instance, illustrate streamed views of the augmented game session generated based on viewpoints of players on opposing teams (e.g., Teams1and2). In one scenario, with respect toFIG. 3A, a commentating user may facilitate augmentation of the reproduced game session during the reproduction of the previous game session. As shown, the user may have been presented with a first view that is generated based on the viewpoint of player202aas the user augments content to the reproduced game session to create the augmented game session. The user may, for instance, add audio commentary302during the reproduction of the previous game session to point out an important aspect of the previous game play associated with Team1such that the media stream of the augmented game session will include the audio commentary. To provide viewers with an uncluttered view, the user may opt to remove screen objects (e.g., compass, maps, alerts, etc.) that may have been presented to player202ain the previous game session, as indicated inFIG. 3A.

In another scenario, as shown inFIG. 3B, an augmenting user may switch to a viewpoint of a player304on Team2(e.g., the team opposing Team1). As such, the user may be presented with a view generated based on the viewpoint of player304as the user adds augmentation content to the reproduced game session. For example, when an augmenting user realizes the strategy being implemented by player304, the user may provide his/her take on that strategy. In some implementations, the user may add audio and/or visual commentary (e.g., audio commentary306, visual commentary in the form of video window308, etc.) during the reproduction of the previous game session using a microphone and a camera connected to client computing platform108(e.g., a game console). The user may opt to include video window308depicting the user in real-time as he/she facilitates augmenting content to the reproduced game session to create the augmented game session. Video window308may, for instance, enable viewers of the augmented game session to see the facial expressions of the user as he/she facilitates the augmentation. To add more excitement to the augmented game session, an augmenting user may opt to include a map310depicting the current locations of the various players and a leaderboard312depicting the current scores of the players (e.g., players202and304), such that a media stream of the augmented game session will include the audio commentary306, the video window308, the map310, and the leaderboard312.

In certain implementations, game module110may be configured to generate a new view of the previous game session during the reproduction of the previous game session. Augmentation module120may be configured such that the augmentation content includes the new view. For example, an augmenting user may input a selection for a new viewpoint unavailable to the players in the previous game session, causing generation of a new view based on the new viewpoint selected by the user. Thus, the user may provide commentary for the new view in creating the augmented game session (e.g., such that a media stream of the augmented game session may include the new view and the commentary for the new view).

In a first scenario, a user may opt for an aerial viewpoint that causes the generation of a new view based on the new outlook from the sky. In a second scenario, the user may opt for a viewpoint with x-ray vision that enables the user to see through solid objects (e.g., walls, rocks, etc.) with perfect clarity. The user may, for instance, utilize a two-finger touch gesture on a touch display rendering the current view to “open-up” a vision hole on a solid object that enables the user to see through the solid object via the vision hole. In a third scenario, the user may opt to act as a player (e.g., a reporter that was not in the previous game session, a moving spy robot that was not accessible to the players in the previous game session, etc.) who may move around in the simulated space to have access to viewpoints that the players in the previous game session did not have. In this way, a user is able to provide viewers of the augmented game session with additional insight into the previous game session along with intriguing new views that would otherwise have been missed by simply watching typical recorded media streams having commentary.

In some implementations, augmentation module120may be configured to facilitate selection by an augmenting user of one or more views of the augmented game session that are streamed from among a plurality of views of the augmented game session. The plurality of views of the augmented game session may, for instance, include one or more views of the previous game session generated during the reproduction, new views generated during the reproduction, and/or other views associated with the augmented game session. In one use case, augmentation module120may facilitate the selection by an augmenting user of one or more views of the augmented game session by working with game module110to enable the user to control the one or more properties of the reproduction of the previous game session, such as being able to switch viewpoints (e.g., viewpoints available to the players in the previous game session, viewpoints unavailable to the players in the previous game session, etc.), provide magnifications of the viewpoints, and manage inclusion/omission of screen objects that were accessible to the players in the previous game session for views (e.g., resulting in new views).

In various implementations, augmentation module120may be configured such that one of the views of the augmented game session may include multiple views associated with the previous game session. For example, during the reproduction of the previous game session, the user may initially be providing commentary on a first view that is generated based on a viewpoint of a player in the previous game session. However, to make streaming of the augmented game session more interesting, the user may add a second view that is generated based on a viewpoint of an opposing player in the previous game session such that a single view associated with the augmented game session may include both the first and second views depicting viewpoints of opposing players. The first and second views may, for instance, be cropped, resized, and/or reshaped to fit within the single view associated with the augmented game session.

Streaming module118may be configured to effectuate streaming of one or more views of the augmented game session. In some implementation, the streaming may include real-time streaming of the one or more views of the augmented game session to one or more viewers as the augmentation content is added to the reproduced game session. By way of example, although the previous game play initially occurred during the previous game session, the game play information associated with the previous game session may be obtained (e.g., downloaded from a web service, loaded from storage at a game console, etc.) and utilized to reproduce the previous game play during a reproduction/augmentation session (e.g., the reproduced and augmented game sessions). Thus, the streaming of the one or more views of the augmented game session may include live, real-time streaming with respect to the augmented game session.

In various implementations, the augmented game session may include a plurality of players. The plurality of players may include a first player and an augmenting user. The game may include a plurality of player types and a plurality of in-game actions. The plurality of player types may include first and second player types. The first player may be of the first player type, and the user may be of the second player type. By way of example, the first player may be a player in the previous game session and the augmented game session, while the user may only be a player in the augmented game session. In one scenario, with respect to the augmented game session, the first player may be a simulated player whose actions may be predefined (e.g., the actions may be recorded from the previous game session in the form of the game play information), while the second player may be a live player whose actions are initiated during the augmented game session to facilitate augmenting content to the reproduced game session.

In another scenario, players of the first player type may be limited to their respective viewpoints. For example, in the previous game session, the players of the first player type may only see what their respective user character sees through a first-person viewpoint and/or the immediate area around their respective user character through a third-person viewpoint. On the other hand, players of the second player type may have access to their own viewpoints as well as the viewpoints of all of the players in the previous game session. For instance, during the reproduction of the previous game session, the user (e.g., being of the second player type) may have the ability to see all interactions taking place among the players in the game world (e.g., aerial viewpoint of the game world, god-like view of the game world, etc.), along with having access to the individual viewpoints of each of the players in the game world (e.g., first-person viewpoint of the players, third-person viewpoint of the players, etc.).

In certain implementations, the plurality of in-game actions may include first actions of the first player type and second actions of the second player type. The first actions of the first player type may be actions that are available to players of the first player type (e.g., the first player), while actions of the second type may be actions that are available to players of the second player type (e.g., the user). In one use case, the second actions may include activating x-ray vision to see through solid objects (e.g., walls, rocks, etc.), transitioning into the bodies of various user characters associated with other players to see what is happening from their viewpoints, instantly accessing viewpoints from any location in the simulated space to obtain “looks” that may have been unavailable to players in the previous game session, and/or other actions of the second player type. Such actions may, for instance, be unavailable to the first player and/or other players of the first player type.

As indicated, in various implementations, the previous game play may include the first actions, and the first actions may have been implemented on behalf of the first player in the previous game session. For example, the first actions may have been implemented during the previous game session responsive to action requests for the first actions by the first player, and recorded during the previous game session in the form of the game play information. The first actions may be reinitiated during the reproduction of the previous game session using the game play information. On the other hand, the second actions may be initiated for the first time during the reproduction of the previous game session. Thus, in some implementations, game module110may be configured to implement the second actions for the user during the reproduction of the previous game session responsive to action requests for the second actions by the user.

Game play storage module122may be configured to generate the game play information responsive to implementation of the first actions during the previous game session. For example, the game play information may be generated during the previous game session based on actions implemented in the previous game session. The game play information may be stored at a game console, at a web service, etc., to enable future reproduction of the previous game session for augmentation purposes (e.g., creating an augmented game session by adding augmentation content to the reproduced game session during the reproduction of the previous game session). The game play information may, for instance, be generated and stored using one or more languages suitable for recording game play information, such as Extensible Markup Language (XML) and/or other suitable languages.

FIG. 4illustrates a method for facilitating generation of media streams at a game console, in accordance with one or more implementations. The operations of method400presented below are intended to be illustrative. In some implementations, method400may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of method400are illustrated inFIG. 4and described below is not intended to be limiting.

In some implementations, method400may be implemented in one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information). The one or more processing devices may include one or more devices executing some or all of the operations of method400in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured through hardware, firmware, and/or software to be specifically designed for execution of one or more of the operations of method400.

In an operation402, an instance of a game world may be executed. For example, the instance of the game world may be executed at a game console. Operation402may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation404, a primary view of the game world may be generated based on the game world for local viewing. For example, the primary view may be the view rendered to a player of the instance of the game world on his/her display locally connected to his/her game consoles. The primary view may be generated based on interactions among various players of the game world, interactions of the players with simulated elements of the game world, simulated events of the game world, augmented elements, and/or other features. The primary view may rendered based on one or more media formats (e.g., image format, video format, etc.) to depict the various interactions, simulated events, augmented elements, and/or other features. Operation404may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation406, action requests specifying in-game actions may be obtained. Both players and augmenting users (e.g., who may also be players) may, for instance, input the action requests via their respective game consoles to interact with one another, to travel around the game world having one or more simulated elements, to complete objectives in the game world, and/or to perform other activities associated with the game world. Operation406may be performed by a communication module that is the same as or similar to communication module112, in accordance with one or more implementations.

In an operation408, the requested actions may be implemented in the instance of the game world. For example, when implemented, the requested actions may cause one or more simulated events in the instance of the game world, resulting in the generation of the primary view depicting the simulated events. Operation408may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation410, a media stream including a secondary view of the game world may be generated at the game console. The secondary view of the game world may be a lower-quality version of the primary view. By way of example, the generation of the secondary view may include: a prefetching, to the cache (e.g., L1 cache, L2 cache, etc.) of the game console, one or more of portions of the primary view or instructions for encoding the portions of the primary view; an encoding, based on the prefetching, of the portions of the primary view to produce the secondary view. For example, the portions of the primary view may be encoded using prefetched-cached portions of the primary view and prefetched-cached encoding instructions to produce the secondary view. Operation410may be performed by an encoding module that is the same as or similar to encoding module114, in accordance with one or more implementations.

It should be noted that, in some implementations, the encoding module (which is the same as or similar to encoding module114) may be configured to include code that indicates the prefetching to the cache and the encoding of the portions of the primary view to produce the secondary view. In other implementations, processors104may include a first processor of a first type and a second processor of a second type. The code may indicate execution of a first part of the code using the first processor and execution of a second part of the code using the second processor.

In certain implementations, operation410may be performed by a vector processing module that is the same as or similar to vector processing module116such that the vector processing module may initiate the prefetching to the cache and the encoding of the portions of the primary view to produce the secondary view. In some implementations, the vector processing module may be configured to load, in parallel: (1) one or more of a first portion of the portions or the instructions for encoding the first portion; and (2) one or more of a second portion of the portions or the instructions for encoding the second portion, from the cache to a register associated with the vector processing module such that the encoding of the portions of the primary view to produce the secondary view is based on the loading from the cache to the register.

In an operation412, streaming of the media stream may be effectuated. The media stream may, for instance, be streamed from the game console to one or more viewers via a web service. Operation412may be performed by a streaming module that is the same as or similar to streaming module118, in accordance with one or more implementations.

FIG. 5illustrates a method for facilitating streaming of augmented game sessions created via augmentation during reproduction of previous game sessions, in accordance with one or more implementations. The operations of method500presented below are intended to be illustrative. In some implementations, method500may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of method500are illustrated inFIG. 5and described below is not intended to be limiting.

In some implementations, method500may be implemented in one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information). The one or more processing devices may include one or more devices executing some or all of the operations of method500in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured through hardware, firmware, and/or software to be specifically designed for execution of one or more of the operations of method500.

In an operation502, game play information associated with a previous game session of a game may be obtained. The game play information may define previous game play of the previous game session. As indicated, the previous game play may include actions initiated by players of the previous game session, interactions among user characters associated with the players, attributes of the user characters, environments of the previous game session (e.g., in which the actions were initiated, the user characters interacted with, etc.), and/or other game play. Operation502may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation504, the game play information may be executed to reproduce the previous game play in the previous game session during a reproduction of the previous game session. For example, the previous game play may be reproduced by reinitiating the actions defined in the game play information in their respective sequence order within the context of the environment in which the actions took place. Operation504may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation506, a determination of whether user input is received from an augmenting user during the reproduction of the previous game session may be effectuated. The user input may, for instance, relate to adding of augmentation content to the reproduced game session, such as inputting audio/visual commentary, adding one or more screen objects (e.g., maps, leaderboards, clocks/timers, etc.), adding new views that may not have been available for presentation to the players in the previous game session, and/or other user inputs. Operation506may be performed by a communication module that is the same as or similar to communication module112, in accordance with one or more implementations. Responsive to a determination that user input is received from the user during the reproduction of the previous game session, method500may proceed to an operation508.

In operation508, augmentation content may be added to the reproduced game session based on the user input to create augmented game session. In some implementations, selection by the user of one or more views of the augmented game session that are to be streamed may be facilitated. The selection by the user may be from among a plurality of views of the augmented game session. The plurality of views of the augmented game session may, for instance, include generated views of the previous game session, new views generated during the reproduction of the previous game session, and/or other views associated with the augmented game session. Operation508may be performed by an augmentation module that is the same as or similar to augmentation module120, in accordance with one or more implementations.

In an operation510, streaming of the one or more views of the augmented game session (e.g., the views selected by the user) may be effectuated. For example, if the reproduction of the game session is performed on a game console, the streaming of the one or more views of the augmented games session may be from the game console to a web server (e.g., for distribution to one or more viewers from the web server). In some implementations, the streaming may include real-time streaming of the one or more views of the augmented game session to one or more viewers as the augmented content is added to the reproduced game session. Operation510may be performed by a streaming module that is the same as or similar to streaming module118, in accordance with one or more implementations.

FIG. 6illustrates a method for facilitating streaming of augmented game sessions created via augmentation based on player actions during reproduction of previous game sessions, in accordance with one or more implementations. The operations of method600presented below are intended to be illustrative. In some implementations, method600may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of method600are illustrated inFIG. 6and described below is not intended to be limiting.

In some implementations, method600may be implemented in one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information). The one or more processing devices may include one or more devices executing some or all of the operations of method600in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured through hardware, firmware, and/or software to be specifically designed for execution of one or more of the operations of method600.

By way of example, the augmented game session may include a plurality of players. The plurality of players may include a first player and an augmenting user. The first player may, for instance, be a player in the previous game session and the augmented game session, while the user may only be a player in the augmented game session. The first player may be of a first player type, and the user may be of a second player type. In one scenario, with respect to the augmented game session, the first player may be a simulated player whose actions may be predefined (e.g., the actions may be recorded from the previous game session in the form of the game play information), while the second player may be a live player whose actions are initiated during the augmented game session to facilitate augmenting content to the reproduced game session.

In an operation602, first actions of the first player type may be implemented during the previous game session on behalf of the first player. For example, the first actions may have been implemented during the previous game session responsive to action requests for the first actions by the first player. The first actions of the first player type may be actions that are available to players of the first player type (e.g., viewpoint-related actions of such players may be limited to their respective first-person and/or third-person viewpoints). Operation602may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation604, game play information associated with the previous game session may be generated. For example, the game play information may be generated during the previous game session based on actions implemented in the previous game session (e.g., the first actions of the first player type). The game play information may be stored at a game console, at a web service, etc., to enable future reproduction of the previous game session for augmentation purposes. Operation604may be performed by a game play storage module that is the same as or similar to game play storage module122, in accordance with one or more implementations.

In an operation606, one or more views of the previous game session may be generated for presentation to the user during the reproduction of the previous game session (e.g., using the game play information). For example, the views of the previous game session may be based on the respective viewpoints of the players in the previous game session along with the respective first actions of the players that were implemented in the previous game session (e.g., changing of the viewpoints, engaging in combat, moving around in the game world, etc.). The views of the previous game session may be generated during the reproduction of the previous game session to enable the user to select and switch to any of the views of the previous game session that the user wishes to include in a live media stream (e.g., real-time streaming with respect to the augmented game session), while the user adds his/her own audio and/or visual commentary to the selected views during the reproduction of the previous game session. The augmentation of the audio/visual commentary and the selected views may, for instance, result in generating of one or more views of the augmented game session. Operation606may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation608, a determination of whether second actions of a second player type is obtained from the user during the reproduction of the previous game session. The second actions of the second player type may be actions that are available to players of the second player type (e.g., viewpoint-related actions of such players may be more expansive than viewpoint-related actions of players of the first player type). Operation608may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations. Responsive to a determination that second actions is obtained from the user during the reproduction of the previous game session, method600may proceed to an operation610.

In operation610, the second actions may be implemented during the reproduction of the previous game session. For example, the user may request actions to switch among viewpoints of the various players of the previous game session (e.g., selecting generated views of the previous game session). As such, the user may be presented with the viewpoints of the various players indicated by the requested actions. Operation610may be performed by a game module that is the same as or similar to game module110, in accordance with one or more implementations.

In an operation612, the augmentation content may be added to the reproduced game session based on the second actions to create the augmented game session. The augmentation content may include new views of the previous game session (e.g., views generated based on viewpoints not available to players in the previous game session during the reproduction based on their predefined actions), audio/visual commentary, screen objects, and/or other content. Operation612may be performed by an augmentation module that is the same as or similar to augmentation module120, in accordance with one or more implementations.

In an operation614, streaming of one or more views of the augmented game session may be effectuated. As indicated, in some scenarios, if the reproduction of the game session is performed on a game console, the streaming of the one or more views of the augmented games session may be from the game console to a web server (e.g., for distribution to one or more viewers from the web server). In some implementations, the streaming may include real-time streaming of the one or more views of the augmented game session to one or more viewers as the augmented content is added to the reproduced game session. Operation614may be performed by a streaming module that is the same as or similar to streaming module118, in accordance with one or more implementations.

Although the present technology has been described in detail for the purpose of illustration based on what is currently considered to be the most practical and preferred implementations, it is to be understood that such detail is solely for that purpose and that the technology is not limited to the disclosed implementations, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present technology contemplates that, to the extent possible, one or more features of any implementation can be combined with one or more features of any other implementation.