U.S. Pat. No. 10,625,163

A better understanding of the disclosed technology will be obtained from the following detailed description of the preferred embodiments taken in conjunction with the drawings and the attached claims.

DETAILED DESCRIPTION

Generally, the invention herein improves a videogame streaming content by dynamic gameplay determination, accessing supplemental data, and then supplementing the videogame stream with this supplemental content. The stream viewer can interact with the streaming content to better view and personalize receipt of this supplemental data. Therein, the invention improves the viewability of videogame streaming content in the fantasy card videogame space.

FIG. 1illustrates a block diagram of a system100for supplementing a video stream of a fantasy card videogame. The system100includes a client device102, streaming engine104, and a plurality of streaming applications106communicating across a network108.

The client device102can be any suitable gaming device or devices, such as a laptop or personal computer, a video gaming system, a mobile device, etc. The streaming engine104can be any suitable engine or engines providing video streaming or any similar type of video or content distribution. In one embodiment, the streaming engine104may be the Twitch® server accessible using known techniques for streaming content.

The streaming applications106are executable applications disposed on viewer devices. Consistent with known technology, viewers download and run streaming applications to view streaming content. In one embodiment, the streaming application may be from the Twitch® platform allowing for receipt and viewing of streaming content.

In the system100, the client device102executes a videogame110, such as a fantasy card videogame. The videogame110may be a locally-stored executable program, such as being run from local memory. The videogame110may also be a game stored on a disc or cartridge loaded into a gaming slot (not shown). In another embodiment, the videogame110may be a cloud-based or network-based executable game with an interface for player interaction.

As noted herein, a fantasy card videogame is an electronic game using a deck of electronic game cards, the cards having characters or gameplay elements disposed thereon. The game advances by players playing specific cards and the characteristics associated with the characters or gameplay elements advance the game. A fantasy card videogame expressly excludes any card-based game using a standard 52 or 54 card deck, having four defined suits of hearts, diamonds, spades, and clubs. By way of example, a fantasy card videogame can be a version of games using Magic the Gathering® cards but would not include games of poker, blackjack, cribbage, or other playing-card games using the standard 52/54 card deck.

The device102further includes a streaming client112. Consistent with known technologies, the streaming client112may be a locally-executed application facilitating communication with the streaming engine104and distribution of the streaming content. In one example, a user can enable the streaming client112to duplicate a video output feed for distribution to the streaming engine104and subsequent distribution to viewers (via the streaming applications106). The streaming client112may include additional user information for streaming, such as user id, security information, contact information, content distribution list, etc.

Within the client device102, a game log114collects gameplay data generated by the videogame. Gameplay data includes any information relating to gameplay within the videogame110. For example, gameplay data includes the names and identifiers of players, the type of game being played, decks of electronic cards for each player, general game cards, placement (or play) of cards within the game, including sequence of card play. The gameplay data is a log of activities occurring within the game. For example, if the game includes multiple players, the streaming engine104pulls the data from the player running the game and streaming engine, also recording the plays and game log data of player two.

The client device102further includes a log-reader client program (LRCP)116. This program, in one embodiment, is a downloadable executable program running on the client device102. Whereas, in other embodiments, the LRCP does not require placement within the client device, but rather accesses the game log for retrieving and distributing gameplay data as described herein.

The streaming engine104includes an additional processing element, an extension back-end service (EBS) module118. The EBS118is a software module executing either on the same server as the streaming engine104or in another cloud-based execution environment. As described in greater detail below, the EBS118supplements the livestream feed.

In the system100, the streaming applications106further include additional executables, extensions120. The extensions120can operate as a plug-in with the streaming application, providing a viewer application for displaying supplemental content.

Digital collectible card games, such as videogame110, typically contain 1500-2000 unique cards in each games' card pool. Of those, about 50 are active and in-play during any typical game. The system100enables supplementing the game play video stream with specific card information.

Therefore, based on the large amount of electronic gaming cards, a prepared dataset122includes supplemental data. InFIG. 1, the dataset122is illustrated as a stand-alone database, such as accessible via a networked connection. In another embodiment, the dataset122may be locally disposed with the client device102or in any other suitable location.

The prepared dataset122includes electronic card data, from public and/or private data sources (not shown). For example, one source may be image databases, for example images available from scryfall.com, the card image search engine. Another exemplary source may be Gatherer, another Magic Card database. Other sources may include proprietary text or image databases for specific cards, or for example user-generated information for cards.

In one embodiment, the prepared dataset122is hosted on a third party service platform for direct access by LRCPs116executed on client devices102. The third party service can generate specific electronic gaming card content for supplementing the streaming content as described herein.

In one embodiment, a user installs or loads the fantasy card videogame110on his or her local computing/gaming device102. As the user plays the videogame, the execution of the game generates the gameplay data stored in the game log114. The user additionally installs the streaming client112. For example, the streaming client112may be applications available from StreamLabs® OBS.

In one embodiment, the LRCP116is installed by the user on the client device102. The LRCP116runs independent of the videogame110, reading and interpreting the game log. The LRCP116then retrieves the supplemental data from the prepared dataset122based on interpreting the game log. The LRCP116transmits the gameplay information and supplemental content to the EBS118running in conjunction with the streaming engine104.

In one embodiment, the game log, also referred to as a game file, is a simple text file fulfilled with the “event object” in JSON format. The LRCP116can read the log file in a loop, detecting when content changes. Upon change detection, the LRCP116processes the new information by decoding data into a program-readable format, processes the decoded data, and updates a game state representation variable within the LRCP116. The system then uses this updated game-state representation to communicate with the EBS118via an API. As described in greater detail herein, the LRCP116further includes the supplemental data to compliment the updated game-state representations in communicating with the EBS118.

A viewer viewing the streaming content installs and runs the extension120on his or her viewing device. For example, the viewer viewing the streaming content on a personal computing device running the streaming application106can run the extension120as a plug-in or extension to the application106. In one embodiment, the extension120is a web plug-in, enabled by the viewer via account settings associated with the streaming application106. As noted further below, the extension120is a viewer application that in one embodiment uses an overlay interface for providing the supplemental content to the viewer.

FIG. 2illustrates an architecture diagram140with multiple computing resources. The resources include a stream extension142and application portal144. These are user facing elements facilitating engagement and interaction.

The stream extension receives incoming data from the streaming engine104. The extension104can further include a core processing module, a mobile processing module, and a panel module. The application portal144adds further information and user output to supplement the streaming content.

The architecture140includes three backend processing elements, including the streaming extension backend146, the main backend148, and the card image repository150. The stream extension backend146generates output to the stream engine104based on the output from the extension142. Additionally, the extension backend146transmits a signal to the main backend148.

Both backend processing elements146and148include an application program interface for communication and data exchange. In this exemplary embodiment, the backend processing element with API processing modules146and148can utilize a representative state transfers directed towards web services.

The card image repository150can be any suitable memory device or devices have image data stored therein, images of fantasy playing cards. For example, the repository150may be the dataset122ofFIG. 1above.

Further within the system140are databases160,162and164. The stream extension backend146accesses the key-value store160. In one embodiment, the store160is an in-memory database that persists on disk. The data it stores is a key-value format structure but much different kind of values are supported such as Strings, Lists, Sets, Sorted Sets, and Hashes. In one embodiment, the SQL database162may be an opensource database allowing for standard query logic data requests relating to electronic gaming card information. The database164may be a search engine database allowing for data search operations relating to the gaming card information.

Where the system100ofFIG. 1reads the game log,FIG. 3illustrates one embodiment for generating the supplemental content.FIG. 3illustrates general data interaction, where it is recognized these operations can be performed across one or more networked connections.

The system200ofFIG. 3includes the LRCP116receiving the gameplay data from the game log114.

In one embodiment, the LRCP116reads the gameplay data from the game log to extract searchable information therefrom. For example, the gameplay data includes the name or other form of identifier for the cards active or being played within the game. In one embodiment, the gameplay data may include a field identifier indicating the card name. Upon recognition of the field identifier, the LRCP116then extracts the card name.

The LRCP116can therein access one or more databases having supplemental data therein. In one embodiment, the LRCP116accesses the prepared dataset122. Further embodiments provide for accessing other databases(s). By way of example, the system200illustrates 5 exemplary databases, a card data database210A, a marketplace data database210B, an image database210C, a gameplay data database210D, and a miscellaneous data database210N, where N represents any suitable integer. It is recognized that any number of databases210can be accessed. Moreover, the databases210can be local or network accessible, including public and/or proprietary information.

The LRCP116accesses these databases210to retrieve supplemental data associated with gaming cards being played, as indicated by the game log. Varying sources of data can be based on any number of different embodiments. For example, permissions or subscription services may include access to a proprietary image database210. Another embodiment can be a viewer preference or setting for receiving particular types of information.

The LRCP116transmits the gameplay data and supplemental content to the EBS118. In some embodiments, the supplemental content may have different formatting and the LRCP116or the EBS118may perform processing operations to modify the data for presentation. By way of example, card image data may have varying dimensions and all images may be resized to a determined pixel window setting. In another example, card data may be in a different language and requiring translation prior to display.

The EBS118can therein access and retrieve data from the databases210. The EBS118can perform processing operations for formatting or adjusting the supplemental content for the viewer application. In one embodiment, the supplemental content118sent from the EBS module118can be a raw data source with the viewer application filtering content. For example, the EBS module118may send all supplemental data available for a particular card, the viewer application having a setting to display only images and card data, but not market data.

WhereFIG. 3illustrates the LRCP116generating the supplemental content, it is recognized that processing operations may be disposed within or supplemented by the EBS118. For example, the supplemental content retrieval operations can be performed by the EBS118using game log data determined by the LRCP116. Therefore,FIG. 3further includes the dashed-lines showing the alternative embodiment of connecting the EBS118to the servers210and prepared dataset122

WhereFIGS. 1-3illustrate the processing environment and architecture,FIG. 4is a flowchart of the steps of one embodiment of a method for supplementing a video stream of a fantasy card videogame.

Step220is executing a fantasy card videogame including playing electronic gaming cards using a gaming input device. As noted above, the videogame can be executed on a local processing device, with user interactions via a gaming controller or keyboard.

Step222is distributing an output of the fantasy card videogame to a streaming engine for distributing the video stream to at least one viewer application. In a typical embodiment, the streaming engine is network-based with a locally-executed application to process and distribute the game content to the streaming network. The viewer application is typically run by a third party looking to watch the livestream content across the network connection.

Step224is generating gameplay data representing the playing of the electronic gaming cards and storing the gameplay data within the game log. For example, gameplay data may include the name of a card (e.g. the name of the character shown on the card) being played. The gameplay data can also include playing sequence, such as if it is the first, second, third, etc. card being played.

Step226is analyzing the gameplay data to retrieve supplemental content associated with the playing of the electronic gaming cards as indicated by the gameplay data. In one embodiment,FIG. 3illustrates this step.

Step228is transmitting the supplemental content to the backend server. Step228may be performed by the LRCP116ofFIG. 1, the transmission being across the networked connection.

Step230is transmitting the supplemental content to at least one viewer application. In one embodiment, the supplemental content can be translated or otherwise modified for insertion into a presentable format for the viewer application. For example, for image data, images can be resized to fit a preset display window. In another example, for card data, information may be translated into a different language.

Step232is displaying the supplemental content within the viewer application within the viewer in conjunction with the video stream. The videogame itself is streamed via the streaming engine and the viewer application is run, in one embodiment, as an extension or plug-in to the locally-executed streaming player.

The viewer application receives the supplemental data and monitors user interfacing with the streaming application. When the user places a cursor near or over a card, or any other suitable interface operation to signify requesting supplemental content, the viewer application then displays the supplemental content.

FIG. 5illustrates a sample screenshot of a fantasy card videogame including a playing screen240with multiple fantasy cards. In this screenshot, four cards242-248are played with a fifth card250being played. The cards242-250include images, typically an image of the character or a feature of the card. The card being played250additionally includes a title field and card text.

Further visible, the playing screen240includes additional playing cards with images, such as cards260,262,264,266. The screen240further has a deck and a discard pile268for the player, among other features. The screenshot ofFIG. 5illustrates recognized gameplay playing the fantasy card videogame. This screenshot represents not only what the player sees in playing the game, but can also represent the streaming content visible to viewers.

Quickly recognizing and seeing the details of the electronic gaming cards is complicated by the limited real estate of the display screen240. In the streaming viewer,FIG. 6illustrates a screenshot of viewer application in response to viewer interactions. The screenshot280is similar to the screenshot242ofFIG. 5, but includes the visual overlay282including supplemental content.

Based on tracking user interactions, the viewer application notes the placement of a cursor284, or any other suitable positional marker, and displays the supplemental content associated therewith. In this case, a viewer can hover the cursor284near the card being played, causing the display of the supplemental content window282.

In this example, the window282includes one or more images286, a card data field288, and gameplay statistics290. For example, the images can be images of the card image not readily visible in the in-game display. The card data288can be the background story of the card, the powers, strengths and weakness of the character, and the statistics290can be data showing often this card is played and the strength or value of the current play.

In one embodiment, the determining of cards being played in on-screen locations is based on predetermined card-play locations or zones. In these videogames, the gaming board typically includes zones or areas for playing specific types of cards. Thus, detecting a viewer overlay and which supplemental information to display can be zone-based.

Similarly, based on available data from the game log, the game log may not include a specific playing sequence cards. Thus, the overlay application may not directly know which cards in specific zones are viewable and which are occluded. Therefore, one embodiment can include displaying all supplemental data from the zone and allowing the user to select or de-select supplemental content window(s) or section. Another embodiment can include gameplay intelligence to determine or estimate the sequence of card play based on known card plays, for example estimating play Card B after Card A based on the likely relation between Cards A and B.

In one embodiment, the viewer application maps the card images on the display screen using known scanning or image processing techniques. The cards images can be assigned x,y screen coordinates. When the cursor position is detected within this screen coordinates, the viewer application uses a look-up table or other reference to activate the supplemental content window282, retrieve the appropriate fantasy card supplement content, and populate the data fields in the display.

In one exemplary embodiment, player 1 is playing and streaming a collectible trading card game. In this embodiment, player 1 has the LRCP installed and executing concurrent with the game. On a first turn, player 1 draws three cards: (1) water bucket; (2) sanctuary; and (3) bird sanctuary paradise. The LRCP reads the game log, which captures the state of player 1's hand including these three specific cards.

The LRCP broadcasts the card information to the EBS. In one embodiment, the EBS can therein access image data, market price data, and other information on the cards. This information is passed to the streaming application, via the extension. A pub/sub interface of the stream takes the information, renders it visually in the stream. Therein, viewers watching the stream of player 1 can see this supplemental card data.

The viewers, via the streaming app and extension, now see an overlay resting on top of player 1's streaming gameplay. The overlay display illustrates player 1's hand including: water bucket card plus image plus market price data; sanctuary card plus image plus market price data; and bird supreme paradise card plus image plus market price data.

In the game itself, player 1 can play the water bucket card on to the battlefield. The battlefield has a distinct zone for this card play. Player 1 now has water bucket card on the battlefield and sanctuary and bird supreme paradise cards in her hand. The LRCP reads the changes in game state (e.g. one card is now on the battlefield and not in player 1's hand. The process noted above repeats including the LRCP transmitting the updated information to the EBS and stream visual updates send to the extension application.

Back within game, player 1 can now pass. Opponent, player 2, has a hand of three cards, all unknown to player 1 and the stream viewers. Player 2 plays a card, Despicable Mage, on player 2's side of the battlefield.

Again, the LRCP recognizes the change in the game log and submits this information to the EBS. Based thereon, the visual stream is updated based on the gameplay as detected from the game log.

FIG. 7illustrates another viewer display screen, here using a secondary window display. In this embodiment, the streaming display is in a first screen300and the supplemental content is in a second screen302. Therefore, in this second screen, the content does not obscure any of the videogame streaming content. It is recognized that one embodiment may include these two screens300and302on the same output, but the supplemental content302can be directed to a second-screen environment, such as content displayed on a mobile phone and the streaming content viewed on a computer browser.

It is recognized that any number of viewing variations are within the scope of the invention herein. Varying embodiments can include differing windows, but can also include differing content within the windows. For example, one viewing window may include predictive analysis based on knowledge of. the player's decklist and based on what is already known about the game, including cards drawn and played. For example, one embodiment may include artificial intelligence or gameplay heuristics to determine estimated likelihood (percentages) of gameplay sequences as well as player odds. Thus a display window may include not only card information, but gameplay information including a calculated percentage for the player to win the game.

Another example of viewing information can be marketplace data. Players collect different cards and accumulate decks, including by purchasing collections of cards and individual cards. It is recognized marketplaces exist for purchasing these electronic cards. For example,FIG. 3, market data210B database includes market data and can include historical data. Therefore, viewing data may include market price information such as current price on various markets, price last week, 52-week price high and low, percentage change in price over period of time, etc.

Further viewing windows can include augmenting card image with alternative images, such as from public and/or proprietary databases. For example, alternative images may include images different from the actual card image within the game. Further viewing customizations can include stylized frames, alternative colors, larger text formatting for accessibility and readability, etc.

The viewing window can also be a horizontally oriented window, whereFIGS. 6-7illustrate vertically oriented windows. For example, the window can include a moving ticker showing market data for card during gameplay.

FIG. 8illustrates steps of another embodiment of a method for supplementing a video stream of a fantasy card videogame. The above embodiments describe operation in a three processing system environment: the gaming device, the streaming server and backend processing system, and the viewer processing device. Other embodiments allow for different executable locations. For example, the gaming operations can be performed in a network or cloud-based environment with local or client-based execution being simply a user interface. In another example, the game log may be stored on the network and not client-based.

Step320is receiving gameplay data from a game log. The gameplay data is received from videogame execution, whether it be local or network-based execution.

Step322is analyzing the gameplay data to retrieve supplemental content associated with the electronic gaming cards. This analysis may be performed in either a network processing environment and can be locally-executed. In further embodiments, this step can be performed in different environments, such as one retrieval of supplemental information can be via a public content database and further retrieval is from a proprietary database.

As noted above, varying embodiments provide for different analysis and supplemental content. For example, the method and system having gameplay knowledge can engage in predictive analysis, estimating likelihood of card plays and player success. Without gameplay data, it would be statistically improbable to accurately predict the likelihood of any particular card play within a general gaming sequence and subsequent likelihood of a player's victory. The method and system having the gameplay data can then conduct predictive analysis based not on the universe of gamecards, but rather on the known subset of cards left in the player's deck. Therein, the processing system can perform statistical analysis on gameplay activities, including probability or estimations of specific card play using the known data set. The location and execution of these predictive operations can be performed within the processing environment, such as within the LRCP116or the EBS118ofFIG. 1. In another embodiment, the predictive analysis may be performed on a third-party processing device or networked server having greater readily-available processing power.

Step324is transmitting the supplemental content to a viewer application associated with a streaming application and receiving the video stream of the fantasy card videogame. This transmitting step can be sent from a streaming server. In another embodiment, if the user generates and transmits the streaming content from a local device, the distribution can be generated directly from the user's computer.

Step326is displaying the supplemental content in conjunction with the video stream using an overlay function based on interaction with the video stream. In one embodiment, a viewer application is executed on the viewer device, such as a plug-in or extension to the streaming viewer. In another embodiment, viewer application can run on a server or network connection with a cursor position tracking application locally-executing on the viewer device.

In further embodiments, the game log may include additional information usable for improving the supplemental content and/or viewer application. For example, the game log may include card playing sequence information, usable for determining which card(s) are fully viewable on the electronic display.

In another embodiment, the game log may include a comprehensive list of a player's starting deck of cards. This starting deck information allows for a much more compact supplemental data or prepared dataset. For example, if each player has 50 cards, that means the dataset covers 100 cards instead of covering the thousands of cards. This can improve the data search and retrieval time components. Similarly, with smaller datasets, the supplemental data may be loaded at one or more different locations in the streaming and viewer application pipeline.

In one embodiment, the start deck list for a player can be determined from the game log listing the cards within the deck of the player. For example, one technique may include a user registering his or her deck with the game, that registration of a particular deck indicates a subset of cards that can be played within the game. During gameplay, the player pulls cards from the stack of cards and thus this pre-registered deck indicates the defined subset of available cards. In the example of a game having 1500 unique cards, the player can register a deck with 75 of those 1500 cards. Gameplay is one card as a time. Therefore, the processing described herein can be performed relative to the exemplary 75 cards of the registered deck instead of being relative to the full exemplary 1500 cards.

With multiple player games, one embodiment can include security protocols for limiting or prohibiting data sharing. For example, if player A registers his deck of cards, player B should be prohibited from acquiring access to knowledge of this deck. Otherwise, the player B can gain an unfair advantage knowing player A's cards. One exemplary technique for limiting or prohibiting data sharing can be to encapsulate or encrypt pre-registered deck information on a processor only local to the player or on a secured third-party server.

These fantasy card videogames are typically two-player games, against a computer or another player. In streaming content, the stream is initiated based on viewing one specific player's viewpoint. Therefore, where there are two players, it is conceivable to view the same game in two different streams. The system operates to process the game log data available from the player enabling the streaming function. In this embodiment, the game log, acting as the basis of the gameplay data, is generated by the streaming player's game execution.

With multiple players, one embodiment may include a network-based recognition of both players with shared access to the game logs. Access to this data improves supplemental data by having the prepared dataset more readily available. A central repository having knowledge of the game decks for both players can therefore better estimate and anticipate outcome likelihoods and contribute these data points for streaming viewers through the extension. For example, having knowledge of the decks allows estimating a percent likelihood of victory for a particular player based on remaining cards available not only in a player's deck but also with his or her hand.

The present method and system improves videogame streaming technology for fantasy card videogames by supplementing video stream outputs. Prior techniques limited viewer interactivity and the accessibility of in-game information. Prior techniques failed to account for the large number of different cards and variances in each of these cards within these fantasy card videogames. By running accompanying software, the method and system uses existing game log information, uses this information to determine supplemental content, and enhance the streaming viewer experience. The method and system makes card-specific information now readily available without disruption or interruption of the streaming experience.

FIGS. 1 through 8are conceptual illustrations allowing for an explanation of the present invention. Notably, the figures and examples above are not meant to limit the scope of the present invention to a single embodiment, as other embodiments are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the present invention can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present invention are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the invention. In the present specification, an embodiment showing a singular component should not necessarily be limited to other embodiments including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, Applicant does not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such. Further, the present invention encompasses present and future known equivalents to the known components referred to herein by way of illustration.

The foregoing description of the specific embodiments so fully reveals the general nature of the invention that others can, by applying knowledge within the skill of the relevant art(s) (including the contents of the documents cited and incorporated by reference herein), readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present invention. Such adaptations and modifications are therefore intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein.