U.S. Pat. No. 11,872,495

A better understanding of the disclosed technology will be obtained from the following detailed description of the preferred embodiments taken in conjunction with the drawings and the attached claims.

DETAILED DESCRIPTION

The present method and system operates both for generating hash values for storage, as well as generating hash values for comparison operations. As described herein, as each player engages in gameplay, executable application(s) are capturing screen content, whereby additional processing routines are performing image analysis on one or more sections of the screen content and generating hash values based on the image analysis. These hash values are sent to one or more central repositories.

The hash values can be then centrally stored in one or more central databases, storing hash values from multiple users. At the server or repository level, the server continuously receives and stores various hash values. The values can be both stored as well as used for data comparison/matching routines.

TheFIG.1system illustrates one embodiment, operating at the local processor level, whereby the central repository receives hash values from a large number of feeding systems (e.g., separate gaming computers) running similar operations to the processing operations ofFIG.1.

FIG.1illustrates one embodiment of a computing system100providing a computerized method of electronically detecting users engaged in common videogame gameplay.

As used herein, common videogame gameplay is where two or more users are playing within the same videogame instantiation or event. For example, if a game is played via a network-connected gaming server, there are any number of users actively playing within the same game. These users playing the common videogame gameplay by competing in the same videogame.

The system100includes a local processing device102with multiple processing engines/modules, including a game engine/portal104, a clip engine106, and a hash value engine108. The processing device102further includes a computer readable medium, non-transitory medium, having executable instructions stored thereon, the processing device performing operations in response thereto.

The processing device102can be disposed on a local computing device112, for example but not limited to a laptop computer, desktop computer, tablet or any other suitable gaming or computing device.

The system100includes an input/output component114for user interaction with the gameplay, as well as connectivity to a network116. The network116provides communication to the server120and the server includes a hash value database122.

The processing device102can be any suitable processing device or devices disposed within the computing system112, including for example existing graphical and central processing units performing computer processing operations in response to the executable instructions.

It is recognized by a skilled artisan than many processing elements are omitted from the device112for brevity purposes only and are within known processing operations. For example, the processing device102can include known techniques for data communication and data sharing with the server120via the network116, as well as known techniques for processing input/output operations via the interface elements114.

In one embodiment, the game engine/portal can be an executable application itself or an application running on a secondary platform, for example running on a browser application or a gaming engine application. The engine/portal104allows for user access and execution of videogame games, including network access for multi-player games with other game players across networked connections. In one embodiment, the engine/portal can be a software enabling the gaming applications, by way of example but not limiting in nature being a Steam® application available from Valve Corporation.

The executable operations noted herein are not expressly limited to a game engine/portal or content distribution platform, but can also be executed within any suitable platform having user activities thereon. For example, in one embodiment the processing operations noted, including the image capture, image processing, and/or hash value generations can be performed in a social media platform, or any other type of platform, for determining common gameplay. For example, if users are engaged in gameplay on a social media platform, the present method and system can provide for detecting common gameplay.

Not specifically illustrated, the processing device102can include any number of initialization and verification routines via the input/output114and the processing device102. For example, a user may log into the engine/portal104with a user id and password, the processing device can further facilitate the network communication with one or more gaming servers (not expressly illustrated) across the network116.

In one embodiment, the clip engine106can be one or more executable software applications generating one or more videogame gameplay clips. In one embodiment, the engine106can operate similar to the engine noted in U.S. Publication No. 2019/0262704, the disclosure of which is hereby incorporated by reference. In one embodiment, the engine106can operate similar to the engine for mobile application as noted in U.S. Pat. No. 11,188,760, the disclosure of which is hereby incorporated by reference.

The clip engine106can generate individual frames of gameplay content, based on acquiring gameplay content, such as content made available for livestream or other network-based distribution.

The hash value engine108includes one or more processing elements for processing one or more frames of gameplay and generating a hash value associated therewith. As described in greater detail below, the present method and system takes advantage of static display elements within various games, such that while each individual player views different on-screen displays, these static fields are shared by common players.

The hash value engine108operates by processing a frame image of videogame gameplay, the frame image acquired by the clip engine. The frame image is of gameplay and based on the game itself, the engine108determines at least a portion of the frame image to capture. The engine108may include receiving a game identifier from the clip engine106or from the processing device102and accessing an instruction field indicating where the portion is located. For example, the portion may be an upper right window indicating multiple gaming elements, such as time of gameplay, team health level, etc. Here, the portion of the frame image is the same for all common players.

In one embodiment, the hash value engine108can also include a time factor as part of the hash generation operation. Where computing devices include internal clocks, there exists enough processing variations that many processing device clocks are at different time settings, having variations in milliseconds and larger. Therefore, the processing device102, via the executable instructions110, can acquire a master clock or synchronized time. In one embodiment, the synchronized time is utilized for finding common gameplay via the game engine/portal104and thus the processing device102acquires or resets a master clock time value via the engine/portal104. For example, in an initialization operation, the processing device102can acquire a master clock start time from the portal104, that time value being synchronized across all processing devices engaging the portal104, as described in greater detail inFIG.2below.

The engine108can then extract the portion and perform a processing operation to translate the portion into a hash value. In one embodiment, that can be performed using a machine vision operation to process the portion into a hash value.

In one embodiment, the hash value can include a time component, but it not expressly required. For example, the time component can be added to the front of the hash value in one embodiment and added to the back of the hash value in another embodiment. Wherein the time component is a synchronized time component.

Generation of the hash value can utilize any suitable hash value generation routine as recognized by a skilled artisan. Herein, the end result provides the benefit, the hash value being based on the information acquired from extracted portion(s) of the gameplay output, as well as including time components in various embodiments.

The input/output elements114can be any suitable user interface and output components, for example a keyboard, mouse, joystick, monitor, speakers, headphones, etc. These elements114provide for user gameplay and other functions as noted herein.

The server120can be one or more network-based servers performing matching operations as described herein. The server120can include additional functional elements and is not expressly limited to matching operations, for example the server120can also include gaming functions, social media functions, content distribution functions, etc.

The hash database122is one or more databases disposed within the network environment for storing hash values thereon. The hash values can be any string value of data elements representing a translation or interpretation of the extracted frame portion from the engine108. The database122can be the central repository receiving hash values from multiple users engaging in gameplay.

WhereFIG.1illustrates a single user processing device112, the present method and system operates both with the single device112, but also within a multi-user environment.FIG.2illustrates the broader multi-user embodiments.

InFIG.2, there are any number of individual processing devices, illustrated generally as112_A,112_B,112_C,112_D, and112_N, where N is any suitable integer. For example, the server120may interact with hundreds or thousands of individual computers112providing hash values to the hash database112. For example, the processing devices112can generate unique hash values on regular intervals, for example sending a new hash value every 5 seconds. Thus, the server120is constantly receiving streams of hash values and storing them within the database122.

Visible inFIG.2is a content distribution engine126. This engine126can include one or more engines for distributing gameplay content, for example livestreaming content or clips of gameplay. In one embodiment, the engine126can distribute content via laptop and/or mobile devices, including for example the Medal.tv® engine available via web content distribution and mobile device applications.

In further embodiments using the content distribution engine126, the operations of the processing devices112can also be performed via a networked location and is not expressly limited to desktop or local operations. For example, if the content is being captured and distributed, the image analysis performed by the clip engine (FIG.1element106) and the hash value generation (FIG.1element108) can be done in a network location.

Further illustrated inFIG.2, the server120communicates with a match database124. This database124stores matched content. As noted in further detail below, when hash values match, this indicates common players in gameplay. The match database124can therein store the matched data, for example indicating that player A was playing a specific game with player B as the same time.

The matching operations can be individual matches as well as continues matching. For example, a single match can indicate a high likelihood of common gameplay, but one embodiment may include verification of common gameplay by seeking additional matching hash values. Whereas, if the local systems are generating updated hash values at defined intervals, e.g., every 1 second, every 5 seconds, or any other suitable interval, then these users will have additional matching hash values. Therefore, common gameplay verification can include determining at least a minimum number of matching hash values, for example over a set period of time.

FIG.3illustrates a sample screen grab image200of a user engaged in gameplay. In this screengrab, there are multiple users engaged in an online multi-player game. For example, this game may be an immersive world fantasy game with multiple players teaming up together to join in a quest or compete against other users.

The game display includes various general user interface features. For example, a game statistic bar202can be displayed in the upper right hand corner. The screen200can include avatars or blocks of other players204, a map feature206and a user's statistic bar208at the bottom corner.

The screen image200is the viewable screen of the first user playing the game. The other players playing the same game, engaged in common gameplay, will see a different viewable screen, but will likely have one or more common display elements.

For example, the game statistic bar202can be common to all gameplayers. Therefore, based on the screen grab and knowledge of the game being played, the executable application determines the game being played and the screen portion to examine. Here, the screen portion is the bar202.

In one embodiment, using computer vision or other processing routines, the data of the portion is analyzed and processed into a hash value. As noted above, the hash value generation can use any suitable processing routine as recognized by a skilled artisan.

In one embodiment, the hash value can include image data as well as other common data to gameplay. For example, the data can include clock data, such as clock data generated from a local processing unit, clock data generated from within the game itself, identifier data associated with the instantiation of the game, or any other suitable data as recognized by a skilled artisan. As noted above, one embodiment includes a master clock synchronization factor ensuring all instances of gameplay on disparate computing devices are generating time-inclusive hash values based on the same clock.

FIG.4illustrates one of a sample hash value. In this embodiment, the hash value230is a multi-digit unique number generated based on the image data as well as any other data fields. The length of the hash value can be any suitable length, long enough to be unique and minimize false matches.

As noted inFIGS.1and2above, the hash values are generated at regular intervals during gameplay. In the example of hash values being generated every 5 seconds, the local processor grabs a frame image at every interval and generates a unique hash value. Each hash value is the transmitted to the central server, either continuously sent upon being generated or stored and sent in a batch upload at defined intervals, e.g., sending a new hash value every minute or every 5 minutes.

In one embodiment, the intervals for generating separate hash values can be based on processing load, as well as volume for storing and comparing hash values. With unlimited processing power and storage, hash values can be generated in shorter intervals, but one embodiment includes designating hash value generation at once every 1 minute or even once every 5 minutes. In this embodiment, the hash value generation operations do not deter from processing load associated with the gaming operations and do not generate an unwieldy volume of hash values, which would require further processing load and inherent delay for reviewing the cache values to find matches.

As noted herein, the method and system can provide for authentication of matches, as well as eliminating false positives, by finding multiple matches between users. A single match may be deemed unreliable, thus if users are engaged in common gameplay over a period of 10 minutes and hash values are generated every 1 minutes, common gameplay can be validated by finding multiple matches generated over this 10 minute common gameplay interval.

The hash value can be based on one element within a display or can include multiple display elements.FIG.5is a sample gameplay screenshot250. This display includes a time bar252, player fields254, avatars256and element blocks258. Similar toFIG.3, each user engaged in common gameplay with see different view screens but are likely to have one or more portions displaying identical images.

In the processing of the screenshot250, the processor can detect in this example 2 portions of the screen250. Here, the time bar252and the player fields254are going to be identical by all common players.

Using the hash value generation algorithm or processing routine,FIG.6illustrates a sample hash value270representing at least the two portions of the display250, portion252and portion254. Similar toFIG.4, the hash value270can also include additional data fields associated with the game, as recognized by a skilled artisan.

FIG.7illustrates a flowchart of the steps of one embodiment of a method for electronically detecting users engaged in common gameplay. The steps ofFIG.7can be performed using the operating systems noted above, for exampleFIGS.1and2.

Step300is processing a first image frame by a first user engaged in gameplay. Step302is generating a first hash value based on a first portion of the first frame image, the frame including at least one shared element for the various users engaged in common gameplay. The method may include uploading prior hash values to a central repository, along with various other users additionally uploading their hash values.

Step304is comparing the hash values. In comparing, the method can either not find a match, whereby there may not be any common gameplay users or can find one or more matches. Step306is determining matched between the first user and at least a second user, where the match is the hash values are identical.

With knowledge of the second hash value, step308is determining the identity or other identifier of the second user and step310is designating the first user being engaged in common gameplay with at least the second user.

The method and system can operate continuously for on-going gameplay. For example, one gameplay environment may include six different players competing with and/or against each other. Each of the six players includes the local processor application for capturing gameplay and generating hash values. In this embodiment, the hash values can be based off a frame captured once every five seconds. Therefore, each individual player generates 12 hash values per minute, with six players that is 72 total hash values per minute. If the game lasts 10 minutes, the group generates approximately 720 hash values.

These hash values are uploaded or otherwise transmitted to the server, including a comparison server performing the comparing operations. The method and system can use any suitable comparison or filtering operation, detecting common hash values.

In one embodiment, the hash values are stored for comparison, but are additionally logged or references to the original source. For example, one embodiment may include a look-up table or other reference for a player identified by any suitable identifying information, e.g., a user id, an avatar, a screen name, an IP address, etc. Using the reference database in addition to the comparison provides for not only detecting identical hash values, but also determining the users.

Upon detection of common gameplay, the server120or any other suitable network or local processing device, can perform additional processing operations based thereon. For example, one embodiment may include notifying users of the matched users and transmitting an identifier on a shared platform, for example within the gaming platform.

Another example can be notifying the user that a common gameplay user may be streaming or otherwise distributing gameplay content. For example, a message can indicate that the user may wish to see another user's videogame feed which was determined to include common gameplay. Another example can be providing shared gameplay user identifications as part of a recommendation or other suggestion engine, for example recommending a connection with one or more platforms.

Another example can be generating tracking details for users engaging in gameplay, including types of game, common players, and other data. For example, one embodiment may track common or overlapping gameplay and determining relationships between various users, games, platforms, etc.

For further illustration of the common gameplay,FIGS.8A and8Billustrate two users engaged in common gameplay.FIG.8Aillustrates a first screen capture320from the first user's perspective.FIG.8Billustrates a second screen capture330from the second user's perspective. Both users are playing the same game, as different characters.

In prior systems, the only way to determine that users are playing together is if both users know each other and somehow notify the system they are engaged in common gameplay. For example, if each player was livestreaming his or her gameplay, they may reference that they are actively playing with each other.

In the present invention, the method and system operates by using the screen grabs/image capture running at the local level. The processor, knowing the game being played, determines which portion or portions of the screen to analyze.

As visible inFIGS.8A and8B, this exemplary embodiment includes 5 common portions. Portion340in screen320is identical to portion350in screen330, showing a time bar. Portion342in screen320is identical to portion352in screen330, a life bar. Portions344,346, and348in screen320are identical to portions354,356, and358in screen330, in-game character identifiers.

Whereby, the method and system generating hash values based on image analysis of one or more of these portions, will generating identical hash values for screen320and screen330. With these hash values send to a central location and a comparison operation being performed, the comparison server therein determines that the user that generated screen320is engaged in common gameplay with the user that generated screen330.

In an alternative embodiment, the method and system may utilize port lookup functionality instead of machine vision or other image processing routines.

In another embodiment for detecting common gameplay, the processing operations may utilize port scanning operations and detect common or overlapping port assignments. This embodiment can include a processing operation performing a local port scan, allowing for seeing who is connected to or assigned various ports.

In one embodiment, the port look up operations can detect common gameplay where there are limitations on finding matches via the computer vision and hash value generation routines. This embodiment can be located in a secondary processing layer and looks up the game the user is playing. Knowledge of the game being played can be acquired using any known processing techniques as recognized by a skilled artisan. In another the game lookup can be performed by a doing a port lookup. This processing can then determine via a scanning operation, the IP address the user is connect to on that port.

This port lookup operation can be based on known port values games, for example Minecraft® servers all run on port 25565. This port lookup operation can also be based on a secondary comparison or reference operation to first determine the port value and the determine the game associated with the port value.

Herein, the determination of common gameplay can use the port and IP address features to detect multiple users engaging in the same gameplay. Upon detection, the method and system can use this common gameplay determination consistent with techniques noted above.

In another embodiment, the processing operations can seek utilization of additional processing information for detecting common gameplay. For example, the processing operations can examine process-related details to find server information, such as a Match ID, server IP address, or other suitable information. The matching of players for detecting common gameplay can be based off this server information.

The above embodiments include locally executable processors for performing content recognition and hash value generation. Further embodiments can include non-local based processor(s) to perform this operation and the present invention is not expressly limited to local-based processors.

For example, where the user is distributing gameplay content, for example generating a livestream or other content feed, one or more edge servers may be utilized to perform the processing the frame image and generating hash value based on the content feed. Herein, the processing operations can grab an image frame within the stream and perform the operations at one or more network levels.

FIGS.1through8are conceptual illustrations allowing for an explanation of the present invention. Notably, the figures and examples above are not meant to limit the scope of the present invention to a single embodiment, as other embodiments are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the present invention can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present invention are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the invention. In the present specification, an embodiment showing a singular component should not necessarily be limited to other embodiments including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, Applicant does not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such. Further, the present invention encompasses present and future known equivalents to the known components referred to herein by way of illustration.

The foregoing description of the specific embodiments so fully reveals the general nature of the invention that others can, by applying knowledge within the skill of the relevant art(s) (including the contents of the documents cited and incorporated by reference herein), readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present invention. Such adaptations and modifications are therefore intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. As used herein, executable operations and executable instructions can be performed based on transmission to one or more processing devices via storage in a non-transitory computer readable medium.