U.S. Pat. No. 12,330,070

DETAILED DESCRIPTION

In the same way that many sports fans enjoy watching sports games involving professional or amateur sports players both to learn sports techniques and to share a sense of community, many video game fans enjoy watching professional or amateur video game players both to learn video game techniques and to share a similar sense of community. Competitive gaming, sometimes referred to as electronic sports or “esports,” involves the play of video games in a competitive environment comparable to traditional sports.

During competitive gaming involving single-player games, players sometimes compete to achieve a high score, or to achieve a particular goal within the video game in a record time, sometimes referred to as a “speed run.” During competitive gaming involving multi-player games, players sometimes likewise compete for best scores or times, but are also sometimes ranked against each other based on numbers of wins and/or losses against one another, or other statistics, often based on multiple game “matches” spread out over time, sometimes in a tournament style.

Competitive gaming includes localized video game competitions that take place in a particular physical location, in which case spectators may be in the physical vicinity of the players. Competitive gaming also includes virtual video game competitions with a more virtual organization structure conducted online, where gameplay footage may be distributed to spectators via a network connection via video stream. Due to increasing popularity of the competitive gaming community, competition providers such as Major League Gaming (MLG) have more recently endeavored to provide an increased level of organization and structure to video game competitions for a number of video games, especially at the professional level.

The subject technology described herein enhances and expands the functionality of server(s) that provide gameplay information to video game spectators and of user devices corresponding to spectators. The servers may be associated with a video game, a video game platform/console, a video game league, or a combination thereof. The server(s) monitor timers, spectator actions, and/or in-game events to identify that a trigger event has occurred, and in response, the server(s) can modify the virtual environment of the video game based on the trigger event. For example, the server can monitor spectator actions (e.g., spectator cheering via a microphone), identify that the actions are associated with a player (e.g., spectator is cheering for a specific player) and modify stats of the player in the virtual environment.

FIG.1illustrates an example of network environment in which some aspects of the technology can be implemented. Network environment100includes public network102, which can include one or more private networks, such as a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), or a network of public/private networks, such as the Internet. Public network102is communicatively coupled to gaming network server104, which may for example represent multiple servers, associated routers, and other network hardware220supporting online gameplay for a particular type of client computing device, such as a particular brand of video game console. The gaming network server104may be inclusive of a variety of servers and other network hardware/resources configured for implementing an esports management system112. The esports management system112may also optionally operate on other gaming network servers104for interoperability of gaming leagues between different gaming network servers104and therefore different types of client computing devices210. The esports management system112may include a number of servers and/or routers associated with operation and maintenance of network-based gameplay for particular video games, a particular video game platform/console, a video game league, or a combination thereof. For example, the esports management system112can generate a virtual esports environment for a particular video game, a particular video game platform/console, a video game league, or a combination thereof. The servers, routers, and other hardware used in esports management system112, gaming network server104, and public network102may be computer systems500as illustrated in and discussed with respect toFIG.5, or may include components of the computer systems500.

Public network102, and gaming network server104provide client and developer access to esports management system112. As illustrated, clients108,109, and110are permitted access to esports management system112using respective client devices (e.g.,108A,109A, and110A). Clients, and their respective devices, may take on a player role or a spectator role at any given time in a given video game match. Although client devices108A,109A, and110A, are depicted as personal computing devices, it is understood that the client devices can include various types of processor-based systems, including but not limited to: game consoles, smart phones, tablet computer systems, and the like. Example of hardware systems that can be used to implement the client device are illustrated in and discussed in further detail below with respect toFIGS.2and5. Similarly, developers (e.g.,105,106and107), are permitted access to league management system via respective their computing systems (e.g.,105A,106A, and107A). Developer computing systems may likewise be computer systems500as illustrated in and discussed with respect toFIG.5, or may include components of the computer systems500.

In some embodiments, client devices108A,109A, and110A can include various sensors that can observe spectator actions. For example, client devices108A,109A, and110A can include one or more microphones to record 3D audio feedback, which can be used to determine when a spectator is cheering or jeering. As another example, client devices108A,109A, and110A can include motion sensors, cameras, and/or haptics to determine when a user performs an action for cheering or jeering (e.g., jumping up and down, clapping hands, etc.). As yet another example, client devices108A,109A, and110A can utilize a controller or other input device to receive inputs to facilitate spectator participation (e.g., virtual clapping, cheers, throw virtual glitter/confetti, trigger virtual fireworks, related audiovisual effects, etc.). Additionally, the input device can also be used to provide feedback to the user. For example, the input device can shake in response to in-game thunder noises. In some embodiments, the data can include complexity, volume, and other characteristics of the spectator actions. For example, the data can include an intensity of how loud a cheer is, an intensity of how animated a dance is, etc.

It is understood that a greater or smaller number of clients and/or developers can be engaged with environment100than are illustrated inFIG.1and described herein, without departing from the scope of the technology.

Some video games, especially those with online multiplayer capabilities, allow spectators to watch gameplay without actually playing or influencing the outcome of the game in any way. The spectator functionality is typically developed by developers (e.g.,105,106and107) with an integrated “spectator” functionality. The integrated “spectator” functionality may be integrated into code of the particular video game that runs during gameplay and/or into online servers that help operate the particular video game's online multiplayer functionality. A spectator client using such an integrated spectator function generally has a copy of the video game whose gameplay he/she is spectating on his/her corresponding client device, allowing the spectator client device to render the gameplay in the same way or in a similar way that client computing devices of player clients render the gameplay, only without giving the spectator client any opportunity to control the gameplay via the spectator's client computing device.

This type of game-integrated spectating technology has the benefit of being more efficient in terms of network bandwidth usage than spectating via streaming of video content, as online games are typically optimized to use as little bandwidth as possible to maintain an acceptable gameplay speed for all players and spectators. The online games accomplish this by transmitting instructions as to how to render a shared virtual space, and then how to render modifications/updates to that shared virtual space over the course of gameplay over a time period between a beginning and an end of a game match. While the initial rendering of the shared virtual space may be complex—for instance, the game and/or game's servers may instruct player and spectator client devices to render numerous character avatars, pieces of scenery, landscapes, buildings, and so forth—later updates/modifications to the shared virtual space are generally less complex. For instance, for an update/modification during gameplay in the example of the preceding sentence, the game and/or game's servers may instruct player and spectator client devices to render that one of the character avatars has moved forward slightly while everything else previously rendered has remained static.

Spectating can occur via online video streaming, which allows spectators (via their spectator devices) to stream video content of recorded or live gameplay, either in real-time or with a slight delay. In reference toFIG.1, the gameplay video may be captured or recorded at player client devices (108A/109A/110A) and sent to spectator client devices (108A/109A/110A), optionally via networks102/104/112or similar video-streaming-focused networks not illustrated inFIG.1. Video streaming allows, for example, video games that do not have integrated spectator functionality to be spectated, for instance allowing spectators to watch a player play an older or legacy game title, an emulated game, or a game that otherwise has no online connectivity, as well as allowing a wide variety of computing devices to be used to spectate, regardless of processing capability or compatibility with the game title. That is, a spectator can spectate a player playing a graphically intensive game from their phone or another device that would not otherwise have the processing capabilities to render the gameplay of the graphically intensive game, and/or that is running an operating system that is not compatible with the graphically intensive game.

Integrated spectating technology may be provided by the game title itself, as well as associated platforms, service providers, and/or devices, including client device (such as a particular brand of video game console/platform), the gaming network server104, and/or an esports management system112. That is, client computing devices108A/109A/110A of a particular type—such as video game consoles of a particular brand—may include this integrated spectating technology, as well as the gaming network server104associated with client computing devices210of that particular type. Alternately or additionally, client computing devices210that are optionally of different types may all be compatible with an esports management system112that includes servers112and/or software running on the client computing devices210.

The particular type of client device (such as a particular brand of video game console/platform), the gaming network server104, and/or an esports management system112may include application programming interfaces (APIs) associated with spectating video games. InFIG.1, for instance, the esports management system112is illustrated as having an API114, though the APIs discussed herein may alternately or additionally be implemented in the gaming network server104and/or the client computing devices210. An individual video game and/or servers associated with the individual video game can then call spectator functions coded into such an API during gameplay to invoke spectator functionality. The spectator function may be standardized across that type of client device210, that gaming network server104, and/or that esports management system112. In this way, a spectator client device having access to the gaming network server104may further access the esports management system112, or a combination thereof, to access esports-related streams and related content and services.

Because the video game data may be in a standardized format using the API114or a gaming network server104API (or other type of data file), as long as the spectator client device can operate with the esports management system112or gaming network server104, spectator client device can spectate the game. For example, a client computing device running an operating system that is incompatible with a particular video game but that is compatible with the esports management system112or gaming network server104may still be able to spectate a game match of that otherwise incompatible video game. Game-integrated, console/platform-integrated, network-integrated, and/or league-integrated spectating technologies may provide extended levels of control for spectators because information about the entire shared virtual space is sent to the spectator, not just what a particular player sees, before the game content is rendered at the spectator's computing device. The spectator can be given an increased control over their view of the gameplay, allowing the spectator to pan, zoom, or rotate their perspective. A spectator may also emulate different viewing perspectives of different players currently playing the game, in series or in parallel. In this way, in a game with multiple players in the same shared virtual environment, a spectator may be able to view more than any individual player can, for example by repeatedly changing their view of the shared virtual space.

Game-integrated, console/platform-integrated, network-integrate, and/or league-integrated spectating technologies also allow additional ways for players and spectators to interact with the shared virtual environment and with each other. For example, integrated spectating technologies allow two client devices to interact as illustrated in and discussed with respect toFIG.3. In some games, spectators may form a virtual audience that may be visible and/or audible within the game by players and/or by other spectators, for example causing the game to generate different levels of cheering, chanting, booing, or jeering based on positive or negative inputs from spectators, the positive or negative inputs optionally concerning a particular human player, a particular computer-controlled simulated player, or a particular team of players, as illustrated in and discussed with respect toFIG.3.

Some games may generate a spectator-controllable avatar that a spectator that can move throughout the shared virtual environment of the game during gameplay responsive to inputs to the gaming network server104and/or esports management system112from spectator client devices210, where the spectator-controllable avatar has limited or no ability to interact with players in a manner that impacts the outcome of the game. Such a spectator avatar might be visible to players or might only be visible to other spectators. In some games, spectators might be able to alter the shared game environment in ways that affect gameplay for players, for instance by changing colors of certain areas or by setting traps that can hinder players.

Furthermore, esports management system112can modify the shared virtual environment to identify or spotlight individual spectators (e.g., by providing identifying effects on an avatar associated with the spectator(s) in the virtual environment) that are being particularly loud, have the best cheer, commentary, dances, costumes, length of being a fan, etc. Additionally, various contests can be hosted by esports management system112, which can then determine a winner (e.g., by user votes) and provide various prizes by modifying the virtual environment. For example, esports management system112can modify the virtual environment by generating a virtual asset (e.g., a unique virtual cosmetic item, a virtual special audiovisual effect, etc.) for the winning user and/or providing a virtual ability that is not typically available (e.g., flying) to the user. Additionally or alternatively, esports management system112can modify the virtual environment for groups of spectators associated with one or more players or teams. For example, fans of a specific player may be given special virtual abilities (e.g., flying) or assets (e.g., virtual t-shirts, penants, etc.) when the specific player breaks a record, hits a milestone, wins a battle, etc. Similarly, esports management system112can modify the virtual environment to penalize groups of spectators when the groups are not cheering enough. For example, the esports management system112can penalize spectators by removing their ability to fly.

Some games may also have periods of downtime. For example, some games may have a half-time break or times of lower action. To fill the downtime, esports management system112can receive third-party content (e.g., live content providers, streaming services, TikTok influencers, curated content, commentary, etc.) and integrate the third-party content into the virtual environment. It is also considered that the esports management system112can generate virtual sub-environments in the virtual environment, such that each virtual sub-environment can be directed to related games, contests, and/or entertainment.

FIG.2illustrates a network-based video game environment involving different types of client computing devices. The network environment100ofFIG.1may be implemented using different types of client computing devices210, each of which represent computer systems500or at least include components as being part of the computer system500ofFIG.5.

For instance,FIG.2illustrates a first client computing device210A that is a laptop portable computer, a second client computing device210B that is a home video game console connected to a television, a third client computing device210C that is a desktop home computer, and a fourth client computing device210D that is a smartphone/tablet portable computing device and/or a portable video game console. Any of the client computing devices210ofFIG.2may be assigned player roles or spectator roles for a particular game. In the context of the environment100ofFIG.1, each of the client computing devices108A,109A, and110A ofFIG.1are client computing devices210.

All four of the client computing devices210ofFIG.2are communicatively coupled together via network hardware220, which in the context of the environment100ofFIG.1, may include various servers, routers, and other network hardware within one or more esports management system(s)112, one or more gaming network(s)104, the public network102, or a combination thereof.

FIG.3illustrates a flow diagram illustrating server network operations for modifying virtual esports environments based on spectator actions. The method300ofFIG.3may be embodied as executable instructions in a non-transitory computer readable storage medium including but not limited to a CD, DVD, or non-volatile memory such as a hard drive. The instructions of the storage medium may be executed by a processor (or processors) to cause various hardware components of a computing device hosting or otherwise accessing the storage medium to effectuate the method. The steps identified inFIG.3(and the order thereof) are exemplary and may include various alternatives, equivalents, or derivations thereof including but not limited to the order of execution of the same.

At step310, a game server (e.g., esports management system112) establishes an interactive session associated with a virtual esports environment. The interaction session can include a plurality of user devices. A subset of the user devices can be associated with designated players, while another subset of the user devices can be associated with designated spectators.

At step320, the game league management server or associated esports management system112receives data from one of the user devices over a communication network (e.g., public network102and/or gaming network server104). The data can indicate a spectator action performed by a spectator associated with the user device.

At step330, the game league management server or associated esports management system112) identifies that the spectator action is associated with one of the players in the virtual esports environment. For example, the spectator may be a fan of a specific player and the spectator action may be the spectator cheering for the specific player.

At step340, the game league management server or associated esports management system112) modifies the virtual esports environment based on the spectator action and the association with the identified player. In some embodiments, modifying the virtual esports environment includes adjusting a parameter associated with the player within virtual esports environment of the interactive session. For example, when a spectator cheers for the player, esports management system112can adjust a speed parameter of the player, such that the player is faster for each spectator cheering. In some embodiments, modifying the virtual esports environment is further based on one of the players in the virtual esports environment meeting a conditional requirement. For example, esports management system112can perform modifications (e.g., audiovisual effects, virtual avatar capabilities, informative notifications, etc.) to the virtual environment when a player breaks a record, hits a milestone, wins a battle, performs a specific action (e.g., an advanced or difficult maneuver). It is further contemplated that modifying the virtual esports environment can include providing a virtual action that is otherwise unavailable to an avatar associated with the user device. For example, esports management system112can provide a spectator avatar the ability to fly. In some embodiments, modifying the virtual esports environment can include limiting a virtual action or ability of an avatar associated with the user device. For example, esports management system112can remove the ability to fly from a spectator avatar.

In some embodiments, step340can include steps342,344, and346.

At step342, the game server (e.g., esports management system112) receives data from the plurality of the user devices indicative of spectator actions associated with the identified player.

At step344, the game server (e.g., esports management system112) determines that a threshold number of spectator actions is met by the received data. For example, esports management system112may determine that a threshold number of spectators have applauded, cheered, or otherwise celebrated a player action. Thus, esports management system112can modify the virtual environment upon determining that the threshold number of spectator actions is met by the received data.

At step346, the game server (e.g., esports management system112) determines an intensity for each of the spectator actions. For example, esports management system112may determine how loudly each spectator is cheering and/or how aggressively each spectator is clapping. Thus, esports management system112can modify the virtual environment based on at least one of the determined intensities of each of the spectator actions. For example, a spectator can be penalized when the spectator is not cheering for their team or player by losing the ability to perform actions (e.g. cheering, dancing, flying, etc.) in the virtual environment.

FIG.4illustrates a flow diagram illustrating server network operations for modifying virtual esports environments based on interactions with third party content.

At step410, the game server (e.g., esports management system112) establishes an interactive session with a virtual esports environment. The interactive session can include a plurality of user devices, such that at least a subset of the user devices is associated with designated spectators.

At step420, the game server (e.g., esports management system112) determines a respective affiliation for each of the user devices. For example, a user may indicate that they are a fan of a team, such that esports management system112can determine an affiliation between the user device associated with the user and the team. As another example, the user may indicate that they are interested in a certain genre of games or entertainment, such that esports management system112can determine an affiliation between the user device with the genre of interest.

At step430, the game server (e.g., esports management system112) provides third party content to each of the user devices based on the respective affiliation for each of the user devices. For example, esports management system112can provide third-party content associated with a player or team of interest and/or games or media of interest.

At step440, the game server (e.g., esports management system112) modifies the virtual esports environment based on an interaction from the user device with the third-party content. In some embodiments, modifying the virtual esports environment includes providing the user device with a virtual asset. In some embodiments, the virtual asset can be a digital reward, badge, virtual t-shirt, etc. For example, the user associated with the user device may be chosen as a winner for a third-party contest, game, and/or mini-game and be awarded with a virtual trophy.

FIG.5is a block diagram of an exemplary electronic entertainment system500. The entertainment system500ofFIG.5includes a main memory505, a central processing unit (CPU)510, vector unit515, a graphics processing unit520, an input/output (I/O) processor525, an I/O processor memory530, a controller interface535, a memory card540, a Universal Serial Bus (USB) interface545, and an IEEE interface550. The entertainment system500further includes an operating system read-only memory (OS ROM)555, a sound processing unit560, an optical disc control unit570, and a hard disc drive565, which are connected via a bus575to the I/O processor525.

Entertainment system500may be an electronic game console. Alternatively, the entertainment system500may be implemented as a general-purpose computer, a set-top box, a hand-held game device, a tablet computing device, or a mobile computing device or phone. Entertainment systems may contain more or less operating components depending on a particular form factor, purpose, or design.

The CPU510, the vector unit515, the graphics processing unit520, and the I/O processor525ofFIG.5communicate via a system bus585. Further, the CPU510ofFIG.5communicates with the main memory505via a dedicated bus580, while the vector unit515and the graphics processing unit520may communicate through a dedicated bus590. The CPU510ofFIG.5executes programs stored in the OS ROM555and the main memory505. The main memory505ofFIG.5may contain pre-stored programs and programs transferred through the I/O Processor525from a CD-ROM, DVD-ROM, or other optical disc (not shown) using the optical disc control unit570. I/O Processor525ofFIG.5may also allow for the introduction of content transferred over a wireless or other communications network (e.g., 4$, LTE, 3G, and so forth). The I/O processor525ofFIG.5primarily controls data exchanges between the various devices of the entertainment system500including the CPU510, the vector unit515, the graphics processing unit520, and the controller interface535.

The graphics processing unit520ofFIG.5executes graphics instructions received from the CPU510and the vector unit515to produce images for display on a display device (not shown). For example, the vector unit515ofFIG.5may transform objects from three-dimensional coordinates to two-dimensional coordinates, and send the two-dimensional coordinates to the graphics processing unit520. Furthermore, the sound processing unit560executes instructions to produce sound signals that are outputted to an audio device such as speakers (not shown). Other devices may be connected to the entertainment system500via the USB interface545, and the IEEE 1394 interface550such as wireless transceivers, which may also be embedded in the system500or as a part of some other component such as a processor.

A user of the entertainment system500ofFIG.5provides instructions via the controller interface535to the CPU510. For example, the user may instruct the CPU510to store certain game information on the memory card540or other non-transitory computer-readable storage media or instruct a character in a game to perform some specified action.

The present invention may be implemented in an application that may be operable by a variety of end user devices. For example, an end user device may be a personal computer, a home entertainment system (e.g., Sony PlayStation2® or Sony PlayStation3® or Sony PlayStation4®), a portable gaming device (e.g., Sony PSP® or Sony Vita®), or a home entertainment system of a different albeit inferior manufacturer. The present methodologies described herein are fully intended to be operable on a variety of devices. The present invention may also be implemented with cross-title neutrality wherein an embodiment of the present system may be utilized across a variety of titles from various publishers.

The present invention may be implemented in an application that may be operable using a variety of devices. Non-transitory computer-readable storage media refer to any medium or media that participate in providing instructions to a central processing unit (CPU) for execution. Such media can take many forms, including, but not limited to, non-volatile and volatile media such as optical or magnetic disks and dynamic memory, respectively. Common forms of non-transitory computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM disk, digital video disk (DVD), any other optical medium, RAM, PROM, EPROM, a FLASHEPROM, and any other memory chip or cartridge.

Various forms of transmission media may be involved in carrying one or more sequences of one or more instructions to a CPU for execution. A bus carries the data to system RAM, from which a CPU retrieves and executes the instructions. The instructions received by system RAM can optionally be stored on a fixed disk either before or after execution by a CPU. Various forms of storage may likewise be implemented as well as the necessary network interfaces and network topologies to implement the same.

The foregoing detailed description of the technology has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The described embodiments were chosen in order to best explain the principles of the technology, its practical application, and to enable others skilled in the art to utilize the technology in various embodiments and with various modifications as are suited to the particular use contemplated. It is intended that the scope of the technology be defined by the claim.