U.S. Pat. No. 11,014,004

A better understanding of the disclosed technology will be obtained from the following detailed description of the preferred embodiments taken in conjunction with the drawings and the attached claims.

DETAILED DESCRIPTION

Generally, the invention herein improves a streaming content by dynamic gameplay determination, accessing supplemental data, including marketplace data for collectible versions of items being played within the streaming game, and then supplementing the videogame stream with this supplemental content. The stream viewer can interact with the streaming content to facilitate viewers purchasing collectible items and streamers receiving credit for purchases concurrent with the streaming.

FIG. 1illustrates a block diagram of a system100for supplementing a video stream of a fantasy card videogame. The system100includes a client device102, streaming engine104, and a plurality of streaming applications106communicating across a network108.

The client device102can be any suitable gaming device or devices, such as a laptop or personal computer, a video gaming system, a mobile device, etc. The streaming engine104can be any suitable engine or engines providing video streaming or any similar type of video or content distribution. In one embodiment, the streaming engine104may be the Twitch® server accessible using known techniques for streaming content.

The streaming applications106are executable applications disposed on viewer devices. Consistent with known technology, viewers download and run streaming applications to view streaming content. In one embodiment, the streaming application may be from the Twitch® platform allowing for receipt and viewing of streaming content.

In the system100, the client device102executes a videogame110, such as a fantasy card videogame. The videogame110may be a locally-stored executable program, such as being run from local memory. The videogame110may also be a game stored on a disc or cartridge loaded into a gaming slot (not shown). In another embodiment, the videogame110may be a cloud-based or network-based executable game with an interface for player interaction.

As noted herein, a fantasy card videogame is an electronic game using a deck of electronic game cards, the cards having characters or gameplay elements disposed thereon. The game advances by players playing specific cards and the characteristics associated with the characters or gameplay elements advance the game. A fantasy card videogame expressly excludes any card-based game using a standard 52 or 54 card deck, having four defined suits of hearts, diamonds, spades, and clubs. By way of example, a fantasy card videogame can be a version of games using Magic the Gathering® cards but would not include games of poker, blackjack, cribbage, or other playing-card games using the standard 52/54 card deck.

The device102further includes a streaming client112. Consistent with known technologies, the streaming client112may be a locally-executed application facilitating communication with the streaming engine104and distribution of the streaming content. In one example, a user can enable the streaming client112to duplicate a video output feed for distribution to the streaming engine104and subsequent distribution to viewers (via the streaming applications106). The streaming client112may include additional user information for streaming, such as user id, security information, contact information, content distribution list, etc.

Within the client device102, a game log114collects gameplay data generated by the videogame. Gameplay data includes any information relating to gameplay within the videogame110. For example, gameplay data includes the names and identifiers of players, the type of game being played, decks of electronic cards for each player, general game cards, placement (or play) of cards within the game, including sequence of card play. The gameplay data is a log of activities occurring within the game. For example, if the game includes multiple players, the streaming engine104pulls the data from the player running the game and streaming engine, also recording the plays and game log data of player two.

The client device102further includes a log-reader client program (LRCP)116. This program, in one embodiment, is a downloadable executable program running on the client device102. Whereas, in other embodiments, the LRCP does not require placement within the client device, but rather accesses the game log for retrieving and distributing gameplay data as described herein.

The streaming engine104includes an additional processing element, an extension back-end service (EBS) module118. The EBS118is a software module executing either on the same server as the streaming engine104or in another cloud-based execution environment. As described in greater detail below, the EBS118supplements the livestream feed.

In the system100, the streaming applications106further include additional executables, extensions120. The extensions120can operate as a plug-in with the streaming application, providing a viewer application for displaying supplemental content.

Digital collectible card games, such as videogame110, typically contain 1500-2000 unique cards in each games' card pool. Of those, about 50 are active and in-play during any typical game. The system100enables supplementing the game play video stream with specific card information.

Therefore, based on the large amount of electronic gaming cards, a prepared dataset122includes supplemental data. InFIG. 1, the dataset122is illustrated as a stand-alone database, such as accessible via a networked connection. In another embodiment, the dataset122may be locally disposed with the client device102or in any other suitable location.

The prepared dataset122includes electronic card data, from public and/or private data sources (not shown). For example, one source may be image databases, for example images available from scryfall.com, the card image search engine. Another exemplary source may be Gatherer, another Magic Card database. Other sources may include proprietary text or image databases for specific cards, or for example user-generated information for cards.

In one embodiment, the prepared dataset122is hosted on a third party service platform for direct access by LRCPs116executed on client devices102. The third party service can generate specific electronic gaming card content for supplementing the streaming content as described herein.

In one embodiment, a user installs or loads the fantasy card videogame110on his or her local computing/gaming device102. As the user plays the videogame, the execution of the game generates the gameplay data stored in the game log114. The user additionally installs the streaming client112. For example, the streaming client112may be applications available from StreamLabs® OBS.

In one embodiment, the LRCP116is installed by the user on the client device102. The LRCP116runs independent of the videogame110, reading and interpreting the game log. The LRCP116then retrieves the supplemental data from the prepared dataset122based on interpreting the game log. The LRCP116transmits the gameplay information and supplemental content to the EBS118running in conjunction with the streaming engine104.

In one embodiment, the game log, also referred to as a game file, is a simple text file fulfilled with the “event object” in JSON format. The LRCP116can read the log file in a loop, detecting when content changes. Upon change detection, the LRCP116processes the new information by decoding data into a program-readable format, processes the decoded data, and updates a game state representation variable within the LRCP116. The system then uses this updated game-state representation to communicate with the EBS118via an API. As described in greater detail herein, the LRCP116further includes the supplemental data to compliment the updated game-state representations in communicating with the EBS118.

A viewer viewing the streaming content installs and runs the extension120on his or her viewing device. For example, the viewer viewing the streaming content on a personal computing device running the streaming application106can run the extension120as a plug-in or extension to the application106. In one embodiment, the extension120is a web plug-in, enabled by the viewer via account settings associated with the streaming application106. As noted further below, the extension120is a viewer application that in one embodiment uses an overlay interface for providing the supplemental content to the viewer.

FIG. 2illustrates an architecture diagram140with multiple computing resources. The resources include a stream extension142and application portal144. These are user facing elements facilitating engagement and interaction.

The stream extension receives incoming data from the streaming engine104. The extension104can further include a core processing module, a mobile processing module, and a panel module. The application portal144adds further information and user output to supplement the streaming content.

The architecture140includes three backend processing elements, including the streaming extension backend146, the main backend148, and the card image repository150. The stream extension backend146generates output to the stream engine104based on the output from the extension142. Additionally, the extension backend146transmits a signal to the main backend148.

Both backend processing elements146and148include an application program interface for communication and data exchange. In this exemplary embodiment, the backend processing element with API processing modules146and148can utilize a representative state transfers directed towards web services.

The card image repository150can be any suitable memory device or devices have image data stored therein, images of fantasy playing cards. For example, the repository150may be the dataset122ofFIG. 1above.

Further within the system140are databases160,162and164. The stream extension backend146accesses the key-value store160. In one embodiment, the store160is an in-memory database that persists on disk. The data it stores is a key-value format structure but much different kind of values are supported such as Strings, Lists, Sets, Sorted Sets, and Hashes. In one embodiment, the SQL database162may be an opensource database allowing for standard query logic data requests relating to electronic gaming card information. The database164may be a search engine database allowing for data search operations relating to the gaming card information.

Where the system100ofFIG. 1reads the game log,FIG. 3illustrates one embodiment for generating the supplemental content.FIG. 3illustrates general data interaction, where it is recognized these operations can be performed across one or more networked connections.

The system200ofFIG. 3includes the LRCP116receiving the gameplay data from the game log114.

In one embodiment, the LRCP116reads the gameplay data from the game log to extract searchable information therefrom. For example, the gameplay data includes the name or other form of identifier for the cards active or being played within the game. In one embodiment, the gameplay data may include a field identifier indicating the card name. Upon recognition of the field identifier, the LRCP116then extracts the card name.

The LRCP116can therein access one or more databases having supplemental data therein. In one embodiment, the LRCP116accesses the prepared dataset122. Further embodiments provide for accessing other databases(s). By way of example, the system200illustrates5exemplary databases, a card data database210A, a marketplace data database210B, an image database210C, a gameplay data database210D, and a miscellaneous data database210N, where N represents any suitable integer. It is recognized that any number of databases210can be accessed. Moreover, the databases210can be local or network accessible, including public and/or proprietary information.

The LRCP116accesses these databases210to retrieve supplemental data associated with gaming cards being played, as indicated by the game log. Varying sources of data can be based on any number of different embodiments. For example, permissions or subscription services may include access to a proprietary image database210. Another embodiment can be a viewer preference or setting for receiving particular types of information.

The LRCP116transmits the gameplay data and supplemental content to the EBS118. In some embodiments, the supplemental content may have different formatting and the LRCP116or the EBS118may perform processing operations to modify the data for presentation. By way of example, card image data may have varying dimensions and all images may be resized to a determined pixel window setting. In another example, card data may be in a different language and requiring translation prior to display.

The EBS118can therein access and retrieve data from the databases210. The EBS118can perform processing operations for formatting or adjusting the supplemental content for the viewer application. In one embodiment, the supplemental content118sent from the EBS module118can be a raw data source with the viewer application filtering content. For example, the EBS module118may send all supplemental data available for a particular card, the viewer application having a setting to display only images and card data, but not market data.

WhereFIG. 3illustrates the LRCP116generating the supplemental content, it is recognized that processing operations may be disposed within or supplemented by the EBS118. For example, the supplemental content retrieval operations can be performed by the EBS118using game log data determined by the LRCP116. Therefore,FIG. 3further includes the dashed-lines showing the alternative embodiment of connecting the EBS118to the servers210and prepared dataset122

WhereFIGS. 1-3illustrate the processing environment and architecture,FIG. 4is a flowchart of the steps of one embodiment of a method for supplementing a video stream of a fantasy card videogame.

Step220is executing a fantasy card videogame including playing electronic gaming cards using a gaming input device. As noted above, the videogame can be executed on a local processing device, with user interactions via a gaming controller or keyboard.

Step222is distributing an output of the fantasy card videogame to a streaming engine for distributing the video stream to at least one viewer application. In a typical embodiment, the streaming engine is network-based with a locally-executed application to process and distribute the game content to the streaming network. The viewer application is typically run by a third party looking to watch the livestream content across the network connection.

Step224is generating gameplay data representing the playing of the electronic gaming cards and storing the gameplay data within the game log. For example, gameplay data may include the name of a card (e.g. the name of the character shown on the card) being played. The gameplay data can also include playing sequence, such as if it is the first, second, third, etc. card being played.

Step226is analyzing the gameplay data to retrieve supplemental content associated with the playing of the electronic gaming cards as indicated by the gameplay data. In one embodiment,FIG. 3illustrates this step.

Step228is transmitting the supplemental content to the backend server. Step228may be performed by the LRCP116ofFIG. 1, the transmission being across the networked connection.

Step230is transmitting the supplemental content to at least one viewer application. In one embodiment, the supplemental content can be translated or otherwise modified for insertion into a presentable format for the viewer application. For example, for image data, images can be resized to fit a preset display window. In another example, for card data, information may be translated into a different language.

Step232is displaying the supplemental content within the viewer application within the viewer in conjunction with the video stream. The videogame itself is streamed via the streaming engine and the viewer application is run, in one embodiment, as an extension or plug-in to the locally-executed streaming player.

The viewer application receives the supplemental data and monitors user interfacing with the streaming application. When the user places a cursor near or over a card, or any other suitable interface operation to signify requesting supplemental content, the viewer application then displays the supplemental content.

FIG. 5illustrates a sample screenshot of a fantasy card videogame including a playing screen240with multiple fantasy cards. In this screenshot, four cards242-248are played with a fifth card250being played. The cards242-250include images, typically an image of the character or a feature of the card. The card being played250additionally includes a title field and card text.

Further visible, the playing screen240includes additional playing cards with images, such as cards260,262,264,266. The screen240further has a deck and a discard pile268for the player, among other features. The screenshot ofFIG. 5illustrates recognized gameplay playing the fantasy card videogame. This screenshot represents not only what the player sees in playing the game, but can also represent the streaming content visible to viewers.

Quickly recognizing and seeing the details of the electronic gaming cards is complicated by the limited real estate of the display screen240. In the streaming viewer,FIG. 6illustrates a screenshot of viewer application in response to viewer interactions. The screenshot280is similar to the screenshot242ofFIG. 5, but includes the visual overlay282including supplemental content.

Based on tracking user interactions, the viewer application notes the placement of a cursor284, or any other suitable positional marker, and displays the supplemental content associated therewith. In this case, a viewer can hover the cursor284near the card being played, causing the display of the supplemental content window282.

In this example, the window282includes one or more images286, a card data field288, and gameplay statistics290. For example, the images can be images of the card image not readily visible in the in-game display. The card data288can be the background story of the card, the powers, strengths and weakness of the character, and the statistics290can be data showing often this card is played and the strength or value of the current play.

In one embodiment, the determining of cards being played in on-screen locations is based on predetermined card-play locations or zones. In these videogames, the gaming board typically includes zones or areas for playing specific types of cards. Thus, detecting a viewer overlay and which supplemental information to display can be zone-based.

Similarly, based on available data from the game log, the game log may not include a specific playing sequence cards. Thus, the overlay application may not directly know which cards in specific zones are viewable and which are occluded. Therefore, one embodiment can include displaying all supplemental data from the zone and allowing the user to select or de-select supplemental content window(s) or section. Another embodiment can include gameplay intelligence to determine or estimate the sequence of card play based on known card plays, for example estimating play Card B after Card A based on the likely relation between Cards A and B.

In one embodiment, the viewer application maps the card images on the display screen using known scanning or image processing techniques. The cards images can be assigned x,y screen coordinates. When the cursor position is detected within this screen coordinates, the viewer application uses a look-up table or other reference to activate the supplemental content window282, retrieve the appropriate fantasy card supplement content, and populate the data fields in the display.

In one exemplary embodiment, player 1 is playing and streaming a collectible trading card game. In this embodiment, player 1 has the LRCP installed and executing concurrent with the game. On a first turn, player 1 draws three cards: (1) water bucket; (2) sanctuary; and (3) bird sanctuary paradise. The LRCP reads the game log, which captures the state of player 1's hand including these three specific cards.

The LRCP broadcasts the card information to the EBS. In one embodiment, the EBS can therein access image data, market price data, and other information on the cards. This information is passed to the streaming application, via the extension. A pub/sub interface of the stream takes the information, renders it visually in the stream. Therein, viewers watching the stream of player 1 can see this supplemental card data.

The viewers, via the streaming app and extension, now see an overlay resting on top of player 1's streaming gameplay. The overlay display illustrates player 1's hand including: water bucket card plus image plus market price data; sanctuary card plus image plus market price data; and bird supreme paradise card plus image plus market price data.

In the game itself, player 1 can play the water bucket card on to the battlefield. The battlefield has a distinct zone for this card play. Player 1 now has water bucket card on the battlefield and sanctuary and bird supreme paradise cards in her hand. The LRCP reads the changes in game state (e.g. one card is now on the battlefield and not in player 1's hand. The process noted above repeats including the LRCP transmitting the updated information to the EBS and stream visual updates send to the extension application.

Back within game, player 1 can now pass. Opponent, player 2, has a hand of three cards, all unknown to player 1 and the stream viewers. Player 2 plays a card, Despicable Mage, on player 2's side of the battlefield.

Again, the LRCP recognizes the change in the game log and submits this information to the EBS. Based thereon, the visual stream is updated based on the gameplay as detected from the game log.

FIG. 7illustrates another viewer display screen, here using a secondary window display. In this embodiment, the streaming display is in a first screen300and the supplemental content is in a second screen302. Therefore, in this second screen, the content does not obscure any of the videogame streaming content. It is recognized that one embodiment may include these two screens300and302on the same output, but the supplemental content302can be directed to a second-screen environment, such as content displayed on a mobile phone and the streaming content viewed on a computer browser.

It is recognized that any number of viewing variations are within the scope of the invention herein. Varying embodiments can include differing windows, but can also include differing content within the windows. For example, one viewing window may include predictive analysis based on knowledge of the player's decklist and based on what is already known about the game, including cards drawn and played. For example, one embodiment may include artificial intelligence or gameplay heuristics to determine estimated likelihood (percentages) of gameplay sequences as well as player odds. Thus a display window may include not only card information, but gameplay information including a calculated percentage for the player to win the game.

Another example of viewing information can be marketplace data. Players collect different cards and accumulate decks, including by purchasing collections of cards and individual cards. It is recognized marketplaces exist for purchasing these electronic cards. For example,FIG. 3, market data210B database includes market data and can include historical data. Therefore, viewing data may include market price information such as current price on various markets, price last week, 52-week price high and low, percentage change in price over period of time, etc.

Further viewing windows can include augmenting card image with alternative images, such as from public and/or proprietary databases. For example, alternative images may include images different from the actual card image within the game. Further viewing customizations can include stylized frames, alternative colors, larger text formatting for accessibility and readability, etc.

The viewing window can also be a horizontally oriented window, whereFIGS. 6-7illustrate vertically oriented windows. For example, the window can include a moving ticker showing market data for card during gameplay.

FIG. 8illustrates steps of another embodiment of a method for supplementing a video stream of a fantasy card videogame. The above embodiments describe operation in a three processing system environment: the gaming device, the streaming server and backend processing system, and the viewer processing device. Other embodiments allow for different executable locations. For example, the gaming operations can be performed in a network or cloud-based environment with local or client-based execution being simply a user interface. In another example, the game log may be stored on the network and not client-based.

Step320is receiving gameplay data from a game log. The gameplay data is received from videogame execution, whether it be local or network-based execution.

Step322is analyzing the gameplay data to retrieve supplemental content associated with the electronic gaming cards. This analysis may be performed in either a network processing environment and can be locally-executed. In further embodiments, this step can be performed in different environments, such as one retrieval of supplemental information can be via a public content database and further retrieval is from a proprietary database.

As noted above, varying embodiments provide for different analysis and supplemental content. For example, the method and system having gameplay knowledge can engage in predictive analysis, estimating likelihood of card plays and player success. Without gameplay data, it would be statistically improbable to accurately predict the likelihood of any particular card play within a general gaming sequence and subsequent likelihood of a player's victory. The method and system having the gameplay data can then conduct predictive analysis based not on the universe of gamecards, but rather on the known subset of cards left in the player's deck. Therein, the processing system can perform statistical analysis on gameplay activities, including probability or estimations of specific card play using the known data set. The location and execution of these predictive operations can be performed within the processing environment, such as within the LRCP116or the EBS118ofFIG. 1. In another embodiment, the predictive analysis may be performed on a third-party processing device or networked server having greater readily-available processing power.

Step324is transmitting the supplemental content to a viewer application associated with a streaming application and receiving the video stream of the fantasy card videogame. This transmitting step can be sent from a streaming server. In another embodiment, if the user generates and transmits the streaming content from a local device, the distribution can be generated directly from the user's computer.

Step326is displaying the supplemental content in conjunction with the video stream using an overlay function based on interaction with the video stream. In one embodiment, a viewer application is executed on the viewer device, such as a plug-in or extension to the streaming viewer. In another embodiment, viewer application can run on a server or network connection with a cursor position tracking application locally-executing on the viewer device.

In further embodiments, the game log may include additional information usable for improving the supplemental content and/or viewer application. For example, the game log may include card playing sequence information, usable for determining which card(s) are fully viewable on the electronic display.

In another embodiment, the game log may include a comprehensive list of a player's starting deck of cards. This starting deck information allows for a much more compact supplemental data or prepared dataset. For example, if each player has 50 cards, that means the dataset covers 100 cards instead of covering the thousands of cards. This can improve the data search and retrieval time components. Similarly, with smaller datasets, the supplemental data may be loaded at one or more different locations in the streaming and viewer application pipeline.

In one embodiment, the start deck list for a player can be determined from the game log listing the cards within the deck of the player. For example, one technique may include a user registering his or her deck with the game, that registration of a particular deck indicates a subset of cards that can be played within the game. During gameplay, the player pulls cards from the stack of cards and thus this pre-registered deck indicates the defined subset of available cards. In the example of a game having 1500 unique cards, the player can register a deck with75of those 1500 cards. Gameplay is one card as a time. Therefore, the processing described herein can be performed relative to the exemplary 75 cards of the registered deck instead of being relative to the full exemplary 1500 cards.

With multiple player games, one embodiment can include security protocols for limiting or prohibiting data sharing. For example, if player A registers his deck of cards, player B should be prohibited from acquiring access to knowledge of this deck. Otherwise, the player B can gain an unfair advantage knowing player A's cards. One exemplary technique for limiting or prohibiting data sharing can be to encapsulate or encrypt pre-registered deck information on a processor only local to the player or on a secured third-party server.

These fantasy card videogames are typically two-player games, against a computer or another player. In streaming content, the stream is initiated based on viewing one specific player's viewpoint. Therefore, where there are two players, it is conceivable to view the same game in two different streams. The system operates to process the game log data available from the player enabling the streaming function. In this embodiment, the game log, acting as the basis of the gameplay data, is generated by the streaming player's game execution.

With multiple players, one embodiment may include a network-based recognition of both players with shared access to the game logs. Access to this data improves supplemental data by having the prepared dataset more readily available. A central repository having knowledge of the game decks for both players can therefore better estimate and anticipate outcome likelihoods and contribute these data points for streaming viewers through the extension. For example, having knowledge of the decks allows estimating a percent likelihood of victory for a particular player based on remaining cards available not only in a player's deck but also with his or her hand.

FIG. 9illustrates a block diagram of a system400for augmenting a video stream, where the video stream is live or in-person gameplay. The system400is accessible by a user402having a computing device404with a user interface (UI)406thereon. The system includes a processing device408accessing an image database410and a card information database412, as well as receiving input from a video stream distributor414.

The user402may be any suitable user viewing or otherwise interacting with the UI406on the computing device404. The computing device404may be any suitable processing device operative to execute the UI406, engaging connectivity to the processing device408or any other intermediate server or system not expressly shown. By way of example, the computing device404can be a desktop or mobile computer, a set-top box, or a gaming console. In another example, the computing device404can be handheld device, such as a mobile or smart phone, a tablet computer, etc. The computing device404may be any suitable device or combination of devices allowing for the user402to receive content via the user interface406as described herein.

Communication between the processing device408and the device404can be across any suitable communication medium, such as wireless network, wi-fi connection, wired network connection, cable connection, etc.

The UI406may be any suitable application allowing for viewing a video stream. In one embodiment, the UI406can be a stand-alone dedicated viewer application receiving a wide variety of streaming content, for example the Twitch® UI406, available from Amazon, Inc. The UI406may be a proprietary or dedicated viewer for subscriber content. The UI406may include additional functionality beyond static content viewing, allowing for interactivity, for example selecting hyperlinks or accessing externally-referenced content.

The processing device408may be one or more processing devices for performing processing operations as described herein. The processing device408can be a combination of processing devices centrally located or in a distributed environment. The processing devices408may be distributed across a wide area network, such as the Internet in a distributed or cloud-based computing environment, whereby the proximity of the processing device408to the video stream content generation is not expressly limited. Rather, as described in greater detail below, the processing device408performs processing operations on a received video stream and can receive the stream across a networked or any other suitable distribution network.

The databases410and412can be any suitable data storage location or locations having data stored therein. For example, the databases410and412can be unitary data storage units, or can be distributed storage across a plurality of networks, centrally located or distributed. The image database410includes image fingerprint data relating to values representing images of playing cards.

As used herein, game play cards can refer generally to physical cards, such as 4″ by 6″ playing cards. Whereas, the term cards can more generally refer to any playable gaming elements within a fantasy-based card game, such as a coin, an odd-shaped card, a token, a multi-sided die or dice, or any other suitable gaming element or artifact.

The card information database412stores any additional information relating to gameplay cards. By way of example, but not expressly limiting, card data can include commercial information of a marketplace where the card is for sale, card data can include background story about the card, card data can include play data about the card, e.g. frequency of card, uniqueness of play, etc. In another embodiment the totality of gameplay can be a factor for card data, such as expressing the value of a play or move relative to the prior cards having been played, or for example with knowledge of the players' decks.

The video stream distributor414is any suitable device or system generating a video stream. For example, in one embodiment this device414may be a broadcaster directly broadcasting live in-person gameplay. In another example, the device414may be a livestream distribution server allowing for users to access the server and livestream or otherwise view gameplay.

The computerized system400includes the processing device408performing processing operations in response to executable instructions stored in a computer readable medium (not expressly illustrated). The processing device408receives the video stream from the distributor414. As described in greater detail below, the processing device408performs processing operations to detect a card within the video stream, including initially detecting an outline of the card and then performing image recognition operations.

The processing device408accesses the image database410for retrieving the identification of the card and therein uses this identification to retrieve card information from the database412.

The processing device408distributes the card information to the user402viewable on the UI406. The system400augments the video stream of players playing a card-based fantasy game without disrupting the video stream distribution.

WhereFIG. 9is a general system overview,FIG. 10illustrates the processing device408in greater detail. It is recognized that to one skilled in the art, numerous processing elements are omitted for clarity purposes only, such as for example data input reception devices, security and other user registration modules, etc.

As illustrated inFIG. 10, the processing device408includes a plurality of processing modules, including an outline detector module440, an image snapshot engine442, a detection engine444, a card retriever446, a data retriever448, and a publication/subscription module450. These modules may be stand-alone processing modules or may be disposed within more general processing routines for performing processing operations as described herein.

The processing device408receives the video stream, such as from the distribution device414ofFIG. 9. As the video stream proceeds in a sequence of continuous images, the outline detector442analyzes the video stream to detect a card outline.

By way of example,FIG. 11illustrates a sample screenshot of one segment of a card game with multiple cards, here the card with four Xs is being played. For clarity purposes, the hand of the player laying the card on the table is not illustrated). The play of the card is represented by the arcing arrow representing the movement of the card, the arrow not visible within the livestream.

Using the outline detector,FIG. 12illustrates a visual representation detecting edges of the card. Here, the card is detected by having 4 distinct corners. Further discussion of the operations of the outline detector is noted with respect toFIG. 14below.

Upon outline detection, the image snapshot module442creates a snapshot from the video feed. The snapshot is a single frame of the video feed allowing for additional image processing thereon.

Using the snapshot, the detection engine444performs image recognition operations on the card. For example, one embodiment may include using a hashing algorithm to convert the visual image into an image fingerprint, such as using any suitable well-known image to perceptual hashing technique.

Herein, the video stream displays fantasy-based card game gameplay, where these cards have ornate designs, sometimes with minimal variations between different cards. For example, all cards for a particular set may have the same general aesthetics, with common background designs, with variations as to card types, e.g. offensive, defensive, etc., and differences with names, words, and images. The commonalities of these cards complicates performing image recognition on a video frame snapshot. Thus, one embodiment includes the hashing algorithm using a cryptographic hashing technique allowing for higher accuracy in generating the image fingerprint.

Having this image fingerprint, the card retriever446access the database410to retrieve the name of the card. For example, the image fingerprint can identify the card as “Anaba Shaman” from Magic The Gathering®. Having this card name, the data retriever448then accesses database412to retrieve card information. The card information can be any suitable information relating to the card. Using the example of “Anaba Shaman,” the card data can include background or related information on the card. In another example, the card data may include a link to a commercial marketplace where the card is being offered for sale. In another example, the card data can relate to the gameplay relative to the totality of gameplay based on tracking current gameplay activities, including when the card is played within the game or into which zone or segment of the board the gameplay occurs.

The pub/sub module450receives the card data from the retriever448, as well as the video stream. The module450then distributes the card data to a viewing device (not shown) concurrent with the video stream. In one embodiment, the processing device108does not buffer or otherwise slow down the video stream to perform the image recognition. Therefore, any delay between the activity within the video stream and augmentation with the card data is due to processing time for performing the operations of modules440-448.

Various embodiments allow for improving the accuracy and timing of the module440-448operations. For example, in gameplay, each player begins the game with a defined deck of cards. Therefore, the image database410can be pre-populated with just images for cards within the decks. For example, a complete deck may include upwards of 20,000+ cards, all with different images. Thus, if two players each have 20 cards, the database search is not reduced to search relative to 40 entries instead of 20,000.

In another embodiment, the system may include machine learning to improve image recognition techniques. Using a preset knowledge set, e.g. each player's decks, the system may apply machine learning iterative knowledge for improve image recognition operations. For example, one embodiment with machine learning may include applying known machine learning techniques to improve recognition accuracy, including using user feedback or automated feedback. User feed may include manual oversight of card recognition. By contrast, automated feedback may include utilizing pre-existing knowledge of player decks, such as performing a recognition operation and then comparing those results to the known decks for the player, such that recognizing the card being within the deck indicates a correct recognition. Machine learning operations, consistent with known machine learning techniques, can improve card recognition through iterative training operations.

Having preexisting card information can also assist in minimize time for collecting card data. For example, using the above example of knowing of 40 potential cards for a game, the card information database412can pre-populate with commercial marketplace information, e.g. links to online sales or auctions, for these cards instead of conducting any searching for this information while the card is being played in the video feed.

In another example, the card information may include source identifying information relative to the image recognition techniques herein. For example, if the card data is a hyperlink to a commercial website for purchasing the card, pub/sub module450or the card information in the database412can include referral identifier information indicating the sales lead was originated due to the image recognition operations. One example would be adding reference terms to the tail of a URL or utilizing an intermediate URL such as a truncated URL technique.

FIG. 13illustrates a flowchart of the steps of one embodiment of a computerized method for augmenting a video stream. The steps may be performed, in one embodiment, using the processor408ofFIG. 10. A first step, step480, is receiving a video stream including fantasy-based card game gameplay. This video stream can be from any suitable source, including live recording of current gameplay, livestreaming of previously gameplay, etc.

Step482is detecting, from the video stream, an outline of a card being played, such as performed using, fromFIG. 10, outline detector440, snapshot engine442, and detection engine444, and card retriever446with an image fingerprint. Step484is accessing at least one reference database to determine card data associated with the card, such as using the retriever484and the card information database412ofFIG. 10.

Step486is formatting the card data for distribution to at least one viewer, such as may be performed by the pub/sub module450ofFIG. 10. Whereby, step488is distributing the card data to a viewer application for complimenting the video stream of the card-based game gameplay.

In one embodiment, the video stream is received from an external video stream source. For example, the source may be a livestream site or may be a broadcaster. In one embodiment, the video stream may be internal to the image recognition, such as being executed within the same content generation and distribution system.

In one embodiment, the distribution of card data may be to a single viewer or viewing source, such as single UI having a first window for the livestream and a second window the card data. In another embodiment, the distribution of the livestream may be a first viewer and the card data may be to a second viewer, such as being commonly executed, e.g. on the same computer, as the first viewer, or a completely different device, e.g. a second screen.

FIG. 14illustrates a flowchart of a further embodiment of the card detection operations. The flowchart illustrates the iterative nature of the processing technique for active gameplay that includes continuous sequential gameplay.

Step500is to begin or continue the video streaming. As the livestream is fed through a viewing application, video processing techniques seek to improve detecting the card. Live video streaming includes any number of card detection and recognition complications.

One complication is that different camera angles for live video cause cards to be viewed at odd angles. WhereFIG. 11screenshot shows an overhead view and the rectangular nature of the card is readily identifiable, the processing operations account for side views. In a side view, the rectangular shape of the card is now trapezoidal.

Another complication is the complexity of the images on the cards. As noted above, a deck may have upwards of 20,000 cards with common aesthetics. With any degradation in video quality, many of these cards may look similar, if not identical. Image capturing angles, such as the card being trapezoidal in shape, only further compounds image recognition difficulties.

Another complication is the continuous nature of image recognition and the continuous video feed. By not including a delay in the video feed, the image recognition occurs within a real time of the card play.

Therefore, the present method and system includes utilizing one or more image processing techniques. Step502is executing a canny algorithm on the video feed. The Canny algorithm is a known video processing technique for improving edge detection within an image, including processing modules504, such as grayscale, gaussian functions, upper and lower threshold definitions, and object edge detection operations.

Step506is finding contours within the video feed. The contours are image processing markers indicating edges of an elements within the feed. The contours506are general to cards or relative to card game specifics. Therefore, inquiry step508is whether the contours are found. Finding contours is based on image processing techniques continually scanning the images of the video feed.

If the contours are not found in step508, the method reverts back to step500to continue monitoring the video stream. If the contours are found, the method proceeds to step510for determining if the contour may be the edge of the object. For example, a typical game card is 4 inches by 6 inches, thereby based on a scale factor, the processing operations can include determining if the element is likely to be sized 4 inches by 6 inches. In further processing techniques, the processing engine may account for off-center or side camera angles, thus further sizing operations can account for offset camera angles, for example the card being trapezoidal shaped. Similarly, for non-card based images, the determination of edges can be predicated on the predicted shape of the object, such as a coin have a circular or oblong shape, a multi-sided die having an orthogonal shape, etc.

Step510is an inquiry step. If the contours do not determine or estimate an edge of the object, the method reverts back to step500. In the above embodiment of processing the video feed, the contour and dimensions of the edges determined using the Canny algorithm. If the edge is found, step512is a acquiring a snapshot of the video feed. For example, this snapshot may be extracting a single frame from the video feed, allowing for further processing to be performed on this snapshot.

Step514is processing the snapshot for image recognition. Varying image processing modules516can be utilized to improve the accuracy and reliability of the image recognition operations. For example, grayscaling operations can eliminate complications due to coloring variances. In another example, erode operations can soften edges and dilate operations can expand the image. Similarly, one embodiment may include Canny image processing techniques.

In processing the snapshot, step518is determining the hash value, e.g. the image fingerprint. One embodiment includes using cryptographic hashing algorithms for improved accuracy. Step520is accessing a database using the hash value and step522is defining the object as the specific card identified from the database access. Therein, the method ofFIG. 14reverts to step500for continue processing, as after this card play, another player will be playing another card and the method seeks to recognize that next card.

The above system and method is described relative to a single video feed with a single viewer, but the method and system equally applies to any number of feeds and any number of viewers.FIG. 15illustrates one embodiment for augmenting video streams. The system includes a plurality of video stream channels, channel A560, channel B562, channel C564, and channel D566. These streams can be unique streams originating from different sources. Or in another example, e.g. if a large tournament is occurring, the feeds may be individual matches within the tournament.

The video streams are viewable using any suitable type of browser or viewing application. A common example of current video stream viewing is via a Twitch® browser or any other commercially-available browser or application. The content provider, the source of the video channels, includes an API268indicating the various streams.

In the present method and system, an image recognition service (IRS) spawner module570becomes aware of the active channels560-568. In one embodiment, an IRS is spawned for each channel. Therefore, in this example IRS572is spawned for channel A560, IRS574for channel B562, IRS576for channel C564, and IRS578for channel D566. Each IRS may perform operations as noted within processor408ofFIG. 10.

For each channel,560-566, each IRS then processes the stream to detect cards being played. Not expressly illustrated, each IRS may access a common data repository, e.g. databases410and412ofFIG. 10, may have their own database(s), or a combination of dedicated and shared memory.

The pub/sub service module580, in this embodiment, acts as a gateway for content distribution. Where there are multiple channels, there are multiple subscribers viewing one or more channels. Therefore, the pub/sub service module580distributes channel A content582, channel B content584, channel C content586, and channel D content588.

Channel subscription and access may be performed using known techniques for taking multiple incoming feeds and distributing these feeds to different users based on subscription or access request. The pub/sub service module580improves content distribution by augmenting the distribution of the video stream to additional include the card data acquired from each of the IRS572-578.

In one embodiment, the pub/sub service module operates as the heart and soul of the system by including card data. As the IRS determines the card being played, the pub/sub service module580accesses card data for augmenting the video feed. This may include informational data, commercial data, gameplay data, etc.

In one embodiment, the module580may include additional processing techniques for card data. For example, one embodiment may include keeping a list of cards being played relative to a pre-existing deck list (e.g. counting cards) and providing a play estimate for forthcoming play options by different players. In another example, by tracking previous gameplays, one technique may include determining a significance value for a card play. For example, a significant value can include a likelihood of winning based on the particular play, or the strength of a player's hand, etc.

The module580, with additional processing parameters, can thereby augment the gameplay by including dynamic gameplay-related information. Otherwise, the card data can be static data, such more information about the card, where the card is available for purchase, etc.

FIG. 16illustrates one example of a viewer showing gameplay and card data. In this example, the viewer includes gameplay in a main window with card data window to the side. In this example, a bottom window may include user profile information. Here, the card data window can include interactivity, such as if the card data includes an active hyperlink, a user can select the link and view additional information or be directed to another site.

FIG. 17illustrates another example of a viewer with gameplay and card data. In this example, card data may be visible in a screen overlay. For example, the overlay screen may be activated by holding a cursor over the newly played card.

Similarly, the card data is not expressly limited to an actively played card. For example, the card data can be a scroll screen having the previous card data readily accessible. In the example of overlay, an overlay screen activation, e.g. mouse-over for example, can apply to all cards having card data. In this example, a viewer can see all the currently-played cards and access card data for any card in an overlay.

In one embodiment, where the image processing is a continuous scan, image recognition can include recognizing all cards or gaming elements on a board and providing not just new card information, but also the previously-provided card information. In the example of a cursor overlay, placing a cursor or mouse over any card or game piece can then enable a pop-up window or other display formatting showing corresponding card data.

In addition to the recognition of a card or playing element, one embodiment further includes noting the sequence, timing, or location of gameplay. For example, a card played in an early move or a first game board zone may have a different meaning or value from being played later in the game or in a different zone. Therefore, one embodiment includes not only recognizing the cards being played, but also the sequence of gameplays, which can be included as part of card information. For example in this embodiment of a data field noting the sequence of gameplay can represent or provide for prior gameplay events, e.g. player two played “card X” in response to player one playing “card y.”

Additionally, the above embodiments provide for image recognition with disruption of the video feed. Further embodiments may include video feed disruption in a non-live or a non-time sensitive environment. For example, if a tournament occurs in Asia and the video broadcast is to be viewed in Europe or the US, the video feed can be delayed. Therefore, additional embodiments may account for pausing or delaying the video feed for image recognition techniques described herein, such as for example pausing the video feed when a snapshot is taken and resuming the feed once the snapshot is processed. In another example, the interruption of the video feed can include feedback or machine learning iterations for improving recognition accuracy. Whereby, the present method and system is not expressly limited to operating in a real-time environment but can also perform content recognition for augmenting a video stream in a post-processing or a delayed environment.

In a further embodiment, the method and system further enhances the user experience by facilitating a purchase of cards or related collectibles. The method and system supplements the video stream of the fantasy card game to include information allowing a viewer to see and purchase an item via the user interface.

A fantasy game card is available for purchase. Pricing of these cards are often controlled by market conditions, so the present method and system may additionally access one or more pricing resources for determining an offer price. Physical versions of fantasy cards can also include a quality rating, so further information may be processed to verify the quality rating of the card itself and the price adjustments for the specific rating.

Additionally, the present method and system allows for any number of different entities providing cards available for purchase. For example, one seller may be an existing marketplace that provides for commercial transactions outside of streaming content. Another example may be one or more entities pairing up with a marketplace. Here, the marketplace or the seller can make their catalog of cards available for purchase.

Similarly, commercial transactions may be outside of an existing marketplace, such as streamers, players, or other entities offering their inventory for sale. In these embodiments, an existing marketplace may be used for finalizing the commercial aspect of the transaction or the platform that generates the card offer data can facilitate the transaction.

As used herein, gameplay cards relate to physical or virtual cards being actively played in the fantasy card game. An item for purchase is referred to as a collectible card, which may be used in gameplay at a later point in time, kept as part of a collection, or both. Cards for purchase can be individual cards or sealed cards, such as found in sealed packaging of a collection of cards. In addition, the collectible card can also refer to one or more game memorabilia items, such as character figurines, apparel, swords or other physical items. Examples may also be virtual objects associated with the fantasy game, such as in-game content, collectible virtual characters, virtual gaming items. Therefore, the offer for sale can relate to cards as well as other collectibles relating to the fantasy card game.

As described above, there are two primary modes of fantasy card video streams. The first mode is a videogame, with two or more players competing using electronic videogame applications across a shared gaming platform. The second mode is a live or in-person game between two players with a live feed of the active game or a video replay. The present method and system operates with both types of systems.

FIG. 18illustrates one embodiment of system for supplementing a video stream for a fantasy card videogame. The system600operates similar to the system ofFIGS. 1-3, but further facilitates a commercial transaction. The streaming service604is one or more processing platforms for distributing gameplay content, here being multiple players competing in an online fantasy card game. For example, two players can connect via an online gaming platform, that gameplay being sent via a streaming engine associated with the streaming service604.

The game log114captures gameplay data from the streaming engine604, as described above. The game log114operates with the log reader client program116and the card information database210to generate card information consistent with techniques described above. Additionally, the game log114further acquires card name data for the processing device606. This card name data represents the name of a card being played within game from the streaming engine604.

The processing device606receives this card name data. For example, the processing device606can perform a data pull operation to access the game log on a predetermined time interval. In another embodiment, the game log114can push the card name data to the processing device606at defined intervals.

As the price of a particular card can be controlled by market pricing, the processing device606communicates with a marketplace database608using the card name data. The marketplace database608includes market data relating to a card, such as current market prices, exclusivity of the card, recent commercial transactions, ownership authentications, quality authentications, and other data.

The marketplace database608may be any commercial database or commercial trading network location. For example, in one embodiment the commercial database608can be associated with an existing online platform for users to buy and sell gaming cards independent of streaming content. For example, one web-based location may be available from Card Kingdom, Inc. In another embodiment, the database608can be a dedicated network location for a specific fantasy card game, including where users can directly purchase game cards, game content, play videogames, etc.

The system600operates similar to the system100ofFIG. 1above, distributing videogame content to viewers via the game log114populated with gameplay content from the streaming service. The LRCP116processes data from the game log114, including the example of card info from database210. This content is submitted to the extension back-end service118for displaying supplemental content to a viewer viewing the livestream content.

In addition, the system600includes the processing device606generating an electronic offer display with the collectible card being available for purchase. This electronic offer display includes card purchase data allowing a viewer to undertake a commercial transaction. The card purchase data can include price information, card quality or rating information, recent market price trends, card uniqueness or general availability, card gameplay features, shipping or delivery time and costs, as well as other information. Additionally, the electronic offer display can include a link, button or other user interface element providing for a buyer to purchase the card. Moreover, the purchase of the card can be done directly within the processing system600or in another embodiment may be performed by accessing an external commercial transaction network location.

The processing device606generates this electronic offer display in a data format usable by the EBS118. The processing device606transmits the electronic offer display to the EBS118, where the EBS118is part of a viewer application associated with the streaming application viewing the video stream of the fantasy card game. Therefore, inFIG. 18, a viewer application610receives the streaming content and an extension612receives the electronic offer display.

Herein, the display of the electronic offer is viewable concurrent with the streaming content, tying together watching gameplay and an offer for an electronic purchase. The offer display includes a button, link, or other user interface element allowing the viewer of the streaming content to engage in the commercial transaction. As described in greater detail below, the commercial transaction can be conducted within the processing system600using local commercial processing techniques, or via a third party marketplace. In another embodiment, a third-party marketplace system may include an API or other connective application within the processing system600to facilitate the commercial transaction. The commercial transaction can include any number of varying embodiments, such as launching a separate window, separate segment of an existing window, updating the window having the electronic offer therein, by way of example. Moreover, the third-party marketplace can be a dedicated marketplace or can be a marketplace associated with or contained within a videogame platform itself.

FIG. 19illustrates system620for in-person gameplay. The video stream distributor414operates similar as described above inFIG. 9including sending video content to the processing device408with image recognition operations using an image database410and card information database412for detecting fantasy cards during gameplay.

Whereas, the system620further includes a processing device622performing processing operations for supplementing the video stream to facilitate a commercial transaction of collectible card purchases.

In this in-person gameplay video feed system620, the processing device622determines cards being played within the gameplay using image recognition techniques described above. Upon determining a card being played, the processing device622then knows the card name data, such as from the card information database412. Thus using the card name data from the game log, the processing device622accesses the marketplace database608similar to operations described above inFIG. 18. Therein, the processing device622generates the electronic offer display. A streaming application624receives the streaming content and a viewer extension receives the electronic offer display.

A viewer viewing the streaming gameplay content can then view the livestream in a streaming application and view the electronic offer display in an extension626. In one embodiment, the electronic offer display includes an active link and the viewer can electronically purchase the collectible card without disruption of the livestream content. Herein, the electronic offer display for system620is similar to the electronic offer display for system600described above, including user interface and processing operations providing for either a commercial transaction from the system620or via a third-party marketplace.

FIG. 20illustrates a sample screenshot display of fantasy card gameplay. Similar to descriptions above, the fantasy card gameplay relates to fantasy-based card games using various gaming cards representing different gaming elements or characters to compete with another player. One example of a fantasy card game is Magic the Gathering®, where the gameplay can be an online videogame with livestream of active gameplay or a video stream of in-person gameplay with physical cards.

In the sample screenshot ofFIG. 20, window640shows the fantasy card gameplay. Display642, such as in an overlay display generated by the extension612ofFIG. 18, provides the electronic offer display. Display642illustrates an exemplary display window with multiple card images644, each having visible price information. In one embodiment, the card images644can be active displays allowing for not only card information, but up to date pricing information. For example, market prices can vary and the price information can include not only price information, but also price trending details.

A user interacting with the display642can select one or more card images644for actively purchasing the card concurrent with the streaming content visible in display640. The display of card images644can include any number of cards, this example display using a scroll bar allowing the user to navigate the various cards available for purchase. The display of cards, in this example, includes both individual cards and sealed cards. While not expressly displayed, further collectible items can be available for purchase via this display642.

The display642can include a shopping cart icon646and user id648as part of the commercial transaction. The shopping cart, consistent with known shopping cart techniques, can collect multiple cards for a single financial transaction. The user id648allows for coordinating the viewer/purchaser with a financial system to consummate the transaction.

The display642can also include financial transaction fields650, such as a delivery field to indicate subsequent delivery of purchased cards, a payment field to indicate how the financial transaction is processed. One example of a payment field can be a designated credit card, a pre-existing credit account, a third-party payment processing system, e.g. Paypal®, by way of example. The transaction fields650can further include a button or active field to buy and complete the transaction as well as a running total to show the purchase price of selected card(s)644.

The method and system is executable within one or more processing devices across a local or networked computing environment.FIG. 21is a flowchart of the steps of one embodiment of the present method and system supplementing gameplay content by facilitating a commercial transaction of a collectible card. The steps ofFIG. 21can be performed using the processing environments ofFIG. 18 or 19.

Step680is distributing a video stream of a fantasy card game. This step can be performed consistent with the techniques described above. The video game stream can be in-person gameplay or videogame content. Additionally the video feed can be a live feed of active gameplay or can be recorded or time-delayed content.

Step682is electronically receiving card name data indicating one or more fantasy cards being played in the fantasy card game. For example, in one embodiment with a videogame the card name data can be from a game log and with a livestream of in-person gameplay, the card name data can be from a card detection engine.

Step684is electronically accessing a marketplace database using the card name data and electronically retrieving card purchase data therefrom. The card name data may be a data field or string of the name of the character. The card purchase data is data representing or relating a commercial transaction for a collectible card referenced by the card name data. As noted above, this card purchase data can include pricing, card quality or rating, among other information fields.

In one embodiment, determining which card purchase data to retrieve can be on referencing the card name data. This referencing can be based on which card or cards are actively being played in the game. Whereas, one embodiment does not necessarily require a specific one-to-one correlation of gameplay to offer for sale.

Rather, the referencing of a card based on card name data can include varying forms of reference, such as a name of fantasy game being played, a card type within the game, a version of the fantasy game, or other factors.

Step686is generating an electronic offer display indicating the collectible card and the card purchase data. The electronic offer display can include varying amounts of display information, such as the price and other commercial details. The electronic offer display further includes a link selectable by a viewer to directly engage or consummate the commercial transaction without disrupting the video stream content.

For example, one embodiment may include embedded instructions for completing the transaction. In one embodiment, the viewer may have profile data associated with viewing the streaming content. The embedded instructions can access this profile data as part of user verification information or payment information for completing the commercial transaction.

Step688is transmitting the electronic offer display to a viewer application associated with a streaming application providing the video stream of the fantasy card game. As noted inFIGS. 18-20, this display can be provided to a viewer extension, for example viewable in an overlay display. The display can also be in a separate browser or viewer window or a designated section of the viewer window itself.

Therein, step690is displaying the electronic offer display in conjunction with the video stream. As part of the viewing process, the viewer can select to purchase the collectible card. The purchase can be a one-click purchase or can transfer or otherwise engage further purchasing software for finalizing the transactions. In the method and system, different parties may receive referral fees for the commercial transactions, such as a streaming provider, game players, sponsors, a marketplace, etc. Therefore, in processing a payment, the system may extract the referral fee from the purchase price and redirect the referral fee payment.

Further referral fees or bonuses may be deducted, for example for a sponsor sponsoring the fantasy card game or one or more of the game players receiving a fee. Additionally, the processing platform may include a transaction cost. Whereby, the remaining balance for the transaction can then be submitted to the seller to complete the financial transaction. Herein, payments and transfers are performed electronically using known financial transfer protocols and security protocols.

In further embodiments, the streaming entity may select a specific marketplace for selling collectibles. These may be different entities, such as a game manufacturer that creates and distributes the fantasy game working with a trading or consumer marketplace. These entity may also be the same, such as a collectible marketplace providing or sponsoring streaming content to promoting buying and selling of collectibles.

In addition, the item or items for sale may include sealed packs of card products. As used herein, a sealed pack can be any suitable pre-packaged grouping of cards, such as either predetermined packaging or random groupings of cards. One example of a sealed pack can be a package generally known as a booster pack. Booster packs typically contain a random selection of fantasy game cards, the purchaser hoping to expand his or her card deck and leaving the specific cards received to chance. Another type of a sealed pack can be a starter deck designed for the purchaser to begin assembling fantasy game cards. Sealed packs can also include specific cards or cards types, for example a curated packaging of cards. As used herein, a sealed pack generally relates to any pre-packaged collection of cards.

In one embodiment, the processing platform that generates the overlay and content functions may provide its own marketplace information. This can operate similar to the embodiments ofFIGS. 18-19above, with the platform including further operations to better track and refine the market pricing information.

For example, a processing device can actively track market data for collectibles, including processing operations of API operations to tracks user activities, screen scraping techniques, as well as other data collection techniques. The processing system further may utilize actively gameplay data acquired via the game log, including for example the number of active plays of a particular card and number of times the card was part of a winning campaign.

In a further embodiment, additional parties may offer cards for sale, including for example individuals or third-parties. In one example, an individual may seek to sell individual cards using the system ofFIGS. 18-19. This seller can register the cards via a connected commercial marketplace and allow the marketplace to facilitate the commercial transaction. In another embodiment, the seller can submit the card information to the system, such as processing device606ofFIG. 18or processing device622ofFIG. 19. Using techniques described above, the method and system can then complete the commercial transaction where the seller can be charged with ensuring final card delivery to the purchaser.

In one embodiment, a factor for determining a collectible card referenced by the card name data can include user interaction functions with the overlay system showing gameplay.FIGS. 1-17illustrate displaying the supplemental card and game content, as well as allowing for user interactions. The present method and system can track user interactions with various cards or gameplay events. These interactions can determine popularity of cards, which can further influence determining the collectible card referenced by the card name data. For example, if tracking indicates a high user interaction with a specific card, the system may prioritize referencing this card for a commercial transaction based on the card name data being played in the game.

As noted as toFIGS. 1-17above, the overlay display includes viewer interactions. Therefor further data collection can include tracking the number of and type of user interactions with card data overlay displays. For example, if a display of a card garners a high degree of user interaction, that may indicate a greater degree of popularity to be reflected in a market price. Further data can include social media or news topics relating to particular fantasy games or cards.

Whereby, in this embodiment, the processing system can supplement market data with user engagement and interaction data generally relating to fantasy cards.

The market pricing information and updating can be delayed or done in a real time fashion. For example, real time processing can include timed interval access to content database. Using the example of how often a card is played, a general tracking database may be tracking a large number of active games, each with unique card play. After a set number of games, for example 20 matches, the processing device may re-iterate its calculations or analysis of marketplace data for card pricing information, such as increasing a price for a more popular card.

In a further embodiment, the method and system may include additional processing for further facilitating the commercial transaction. One embodiment may include using a local or user-verified wallet for the commercial transaction. The wallet may include known protocols for holding and verifying funds. One embodiment may include using or facilitating cryptocurrency transactions. The processing system may integrate or communicate with a cryptocurrency processing system to allow the viewer to complete the transaction using cryptocurrency.

In one embodiment, the overlay and viewer display information can include purchase information. For example, as viewers are watching the gameplay, the supplementary viewer window can include a pop-up display or secondary display portion that gives a notification of any new or recent purchases. One example can be a push notification to the viewers on the platform that a specific user has just purchased a specific collectible.

In another embodiment, the collectible being purchased can also relate beyond the card being played directly in the game. For example, a collectible can relate to a particular game or event, such as a commemorative card. A particular fantasy card tournament may include one or more commemorative cards that may or may not be specific to gameplay, e.g. a character in the game. Viewers can seek to purchase these items through the supplemental viewer application noted above.

In addition, as the processing platform described herein is a central gateway for the commercial transaction, the system can additionally record and track not only user interactions with card data, but commercial transactions. This tracking data provides a holistic view to user viewing engagements to fantasy card games. As described in one embodiment above, this information can be used to better assess market price for cards.

Similarly, as the platform manages transactions, the platform can additionally provide a blockchain or other type of data recording structure for tracking ownerships of specific cards. In this example, a blockchain recordation tracking ownership can validate provenance of a card, adding a further layer of security on the market price.

Therein, the method and system helps facilitate commercial transactions concurrent with streaming content of a fantasy card game.

FIGS. 1 through 21are conceptual illustrations allowing for an explanation of the present invention. Notably, the figures and examples above are not meant to limit the scope of the present invention to a single embodiment, as other embodiments are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the present invention can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present invention are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the invention. In the present specification, an embodiment showing a singular component should not necessarily be limited to other embodiments including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, Applicant does not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such. Further, the present invention encompasses present and future known equivalents to the known components referred to herein by way of illustration.

The foregoing description of the specific embodiments so fully reveals the general nature of the invention that others can, by applying knowledge within the skill of the relevant art(s) (including the contents of the documents cited and incorporated by reference herein), readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present invention. Such adaptations and modifications are therefore intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. As used herein, executable operations and executable instructions can be performed based on transmission to one or more processing devices via storage in a non-transitory computer readable medium.