U.S. Pat. No. 8,968,093

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings. The same reference numbers may be used in different drawings to identify the same or similar elements. In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular structures, architectures, interfaces, techniques, etc. in order to provide a thorough understanding of the various aspects of the claimed invention. However, it will be apparent to those skilled in the art, having the benefit of the present disclosure, that the various aspects of the invention claimed may be practiced in other examples that depart from these specific details. In certain instances, descriptions of well known devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

FIG. 1illustrates an example system100according to one implementation of the invention. System100may include a media center102, a game server104, a network106and storage108. Media center102may include a game engine110and a content engine112. System100may also include a coordination server114. Although the following description of system100may be in the context of an online gaming system utilizing MPEG4-compatible streaming media object distribution and/or coding schemes and MPEG4-compatible encoding and/or decoding apparatus, the invention is not limited in this regard and other existing and/or to-be-developed media distribution and/or coding schemes consistent with the present invention are contemplated by the present invention.

Media center102may be communicatively coupled through network106to game server104and may provide data and/or game metadata to game server104over network106. Although media center102may be embodied in a single device, such as a single integrated circuit (IC) system on a chip (SOC), in some implementations certain components of media center102may be remote and/or physically separated from other components of media center102. For example, in other implementations, media center102may comprise two or more physically distinct ICs or two or more ICs interconnected and packaged together. Furthermore, although media center102may comprise discrete components, some components may be implemented in hardware, software/firmware, or some combination thereof.

Game server104may include and/or may be communicatively coupled to a source of game content (not shown). Although, as illustrated, game server104may comprise a single content server, such as a web server, in other implementations game server104may comprise two or more content serving devices arranged to provide game content to other devices, such as media center102, coupled to network106. Game server104may provide game content in the form of MPEG4-compatible encoded media objects to other devices, such as media center102, coupled to network106, although the invention is not limited in this respect. The encoded media objects provided and/or streamed by game server104may, for example, comprise encoded still or video image objects and/or encoded audio objects although the invention is not limited in this regard. Game server104may provide such encoded game content and/or media objects in response to data and/or game metadata received over network106from other devices coupled to network106, such as media center102.

In some implementations, game server104may provide static media objects to media center102. Static media objects may be media objects that are designated and/or specified to be replaced by dynamic media objects also provided by game server104. In particular, although the invention is not limited in this regard, game server104may provide static media objects in addition to other media objects to media center102when a game-user/player using media center102first downloads and/or otherwise obtains game software from game server104and/or from another server (not shown).

Network106may be any content delivery network capable of routing data and/or game metadata from and/or game content and/or encoded media objects between game server104and media center102. For example, although the invention is not limited in this regard, network106may comprise the internet or world-wide-web and all local sub-networks required to couple game server104and media center102to the internet. Although media center102and/or game server104may be coupled to network106through wired communications infrastructure, the invention is not limited in this regard and in some implementations media center102and/or game server104may be coupled to network106through wireless communications infrastructure.

Storage108may be communicatively and/or operatively coupled to media center102to provide storage for encoded and/or decoded media objects received from game server104. Storage108may comprise one or more suitable integrated drive electronics (IDE) drives, such as a hard disk drive (HDD) or optical disc drive (e.g., CD-ROM, CD-R/W, DVD-R, DVD-R/W, etc.), to name a few possibilities.

Content engine112may comprise hardware, software, firmware or any combination thereof capable of interacting with game server104, game engine110and storage108in the manner to be described below. Content engine112may decode streams of encoded media objects received from game server104, compose the decoded media objects into composite game scenes and supply the resulting composite game scenes to game engine110. Content engine112may also facilitate interaction with both storage108to store encoded and/or decoded media objects and/or retrieve encoded and/or decoded media objects from storage and game server104to communicate game data to game server104. All of these functions will be discussed in more detail below.

In some implementations, content engine112may provide game data to game server104when media center102and/or content engine112communicates with game server104over network106. For example, although the invention is not limited in this regard, a game-user/player using media center102and wishing to play an online game may establish communication with game server104and, in so doing, content engine112may provide game data to game server104. The game data may include personalized data or attributes such as the game-user/player's game profile, the game-user/player's game playing habits, the game-user/player's local time, etc. In response to the received game data, game server104may provide one or more dynamic media objects to content engine112and, in turn, content engine112may replace one or more static media objects with the one or more dynamic media objects.

Game engine110may comprise hardware, software, firmware or any combination thereof capable of implementing game play. Game engine110may perform several functions including, but not limited to, running local game software facilitating user/player interaction and/or game play with a game environment and/or rendering composite game scenes received from content engine112in response to game control parameters retrieved from storage108.

Coordination server114may include and/or may be communicatively coupled to a source of game coordination data (not shown). Although, as illustrated, coordination server114may comprise a single server, such as a web server, in other implementations coordination server114may comprise two or more server devices, such as web servers, arranged to provide coordinating data and/or control data to other devices, such as media center102and/or game server104, coupled to network106. Although illustrated as physically distinct from game server104, in other implementations, coordination server114and game server104may be embodied within a single server, although the invention is not limited in this respect.

Coordination server114may provide control data to game server104to specify which content and/or media objects game server104should provide to media center102, although the invention is not limited in this respect. Coordination server114may provide such control data in response to game data received over network106from other devices coupled to network106, such as media center102. In addition, coordination server114may be coupled, via network106, to one or more additional media centers (not shown) and may provide control data to game server104specifying what media objects game server104should send to those additional media centers. Coordination server114may provide control data specifying content for media center102and/or additional media centers in response to game data received from media center102and/or in response to game data received from those additional media centers.

FIG. 2illustrates media center102and some other components of the system100ofFIG. 1in more detail. In addition to game engine110and content engine112, media center102may include snooping logic204. As illustrated, content engine112may be compatible with the MPEG4 media coding scheme and/or protocol although the invention is not limited in this regard.

In addition, media center102may implement and/or support several layers and/or portions of layers of MPEG4-compatible delivery multimedia integration framework (DMIF) software to provide a uniform transport interface between content engine112and other components of system100. MPEG4 DMIFs are well known in the art and a more detailed description of their function and/or structure is omitted herein in order not to unnecessarily obscure implementations of the present invention.

Media center102may include a game engine DMIF206to enable the communication and/or streaming of, for example, game scenes including decoded media objects from content engine112to game engine110. Media center102may also include a storage DMIF210to enable the communication of, for example, I/O interface212and/or encoded and/or decoded media objects and/or game data from content engine112and/or game engine110to and/or from storage108.

Media center102may further include a server DMIF208to enable the communication and/or streaming of, for example, encoded media objects from network106to content engine112and/or the communication of game data from content engine112to network106. Media center102may include signal mapping functionality214to enable proper routing of signals including, for example, game data from server DMIF208over network106to game server104and/or coordination server114. Signal mapping214may be provided by hardware, software, firmware or any combination thereof.

Snooping logic204may be provided by hardware, software, firmware or any combination thereof capable of permitting game engine110to extract game control parameters from storage108, using I/O interface212, as needed to allow game engine110to update game data such as the game-user's profile although the invention is not limited in this regard. Although illustrated as being external to game engine110, in some implementations snooping logic204may be internal to game engine110although the invention is not limited in this regard. The functionality of snooping logic204will also be discussed in more detail below.

FIG. 3illustrates some components of media center102and content engine112ofFIGS. 1 and 2in more detail. In one implementation, content engine112may include a synchronization layer (SL) manager302, timing and synchronization logic (TSL)304and an elementary stream (ES) manager306. Although illustrated as separate components, in other implementations SL manager302and TSL304may be implemented within one component, although the invention is not limited in this regard. Similarly, although SL manager302and ES manager306are illustrated as separate components inFIG. 3, in other implementations SL manager302and ES manager306may be implemented as a single component although, again, the invention is not limited in this regard. In addition, SL manager302, TSL304and/or ES manager306may be provided by hardware, software, firmware or any combination thereof.

FIG. 4illustrates an example synchronization layer (SL) packet402that may include header information specifying timing parameters such as MPEG4-compatible time stamps, although the invention is not limited in this regard. In some implementations, SL packet402may include three time stamps: an object reference clock (OCR)404time stamp that may represent a master time base for system100and that may be used to reconstruct a time base for an associated object: a decoding time stamp (DTS)406that may specify a time, referenced to the OCR time stamp, at which to decode an associated encoded media object; and a composition time stamp (CTS)408, also referenced to the OCR time stamp, that may specify a time at which the ES manager306should render a decoded media object.

Referring now toFIGS. 1-4, SL packet402may be associated with an encoded media object such as an encoded media object streamed by game server104. Game server104may provide and/or stream SL packet402to media center102along with one or more associated encoded media objects.

In some implementations, SL manager302may implement and/or support one or more layers and/or portions of layers software compatible with the MPEG4 flextime protocol and/or scheme. Such flextime-compatible software may permit SL manager302to utilize, process and/or modify OCR, DTS and CTS time stamps associated with one or more media objects as will be described in more detail below. The MPEG4 flextime protocol and/or scheme is well known in the art and more detailed description of flextime function and/or structure is omitted herein in order not to unnecessarily obscure implementations of the present invention.

In some implementations, SL manager302may implement and/or support one or more layers and/or portions of layers software compatible with the MPEG4 flexmux protocol and/or scheme. Such flexmux-compatible software may permit SL manager302to de-multiplex streams of encoded media received from game server104via network106and server DMIF208to yield de-multiplexed media objects as will be described in more detail below. In addition, SL manager302may use the flexmux-compatible software to de-multiplex streams of encoded media objects provided by storage DMIF210as will be described in more detail below. The MPEG4 flexmux protocol and/or scheme is well known in the art and more detailed description of flexmux function and/or structure is omitted herein in order not to unnecessarily obscure implementations of the present invention.

In some implementations, SL manager302may utilize storage DMIF210to pull static objects from storage108and may provide those static objects and/or DTS and CTS time stamps associated with those static objects to ES manager306as will be described in more detail below. In addition, SL manager302may, in response to dynamic objects provided by game server104and pushed by server DMIF208, provide DTS and CTS time stamps associated with those dynamic objects to ES manager306as will be described in more detail below. SL manager302may pull the static objects in response to information and/or metadata provided by and/or associated with a dynamic object pushed by game server104.

In some implementations, TSL304may utilize OCR time stamps provided by SL manager and associated with one or more media objects to generate synchronization data such as presentation metadata. In some implementations, TSL304may modify flextime parameters to compensate for small temporal drifts due to delay and/or jitter associated with network106as determined by temporal fluctuations in the OCR time stamps with respect to a local time base. TSL304may provide the presentation metadata to ES manager306to enable ES manager306to compose game scenes. In addition, TSL304may compare OCR time stamps received from SL manager302to a local time base, generate a modified OCR′ time stamp in response to that comparison, and provide the modified OCR′ time stamp to ES manager306. In some implementations, TSL304may compare the OCR time stamps to a timing signal T provided, for example, by an internal clock signal generator (not shown).

In some implementations, TSL304may generate pull primitives and/or pull indicators in response to comparing the OCR time stamps to the local time base and may provide those pull primitives and/or pull indicators to SL manager302along with the modified OCR′ time stamps. In response to the pull primitives and modified OCR′ time stamps received from TSL304, SL manager302may indicate to storage DMIF that one or more static media objects should be pulled from storage108and provided to SL manager302.

In some implementations, ES manager306may implement and/or support one or more layers and/or portions of layers of flextime-compatible software. Such flextime software may permit ES manager306to utilize, process and/or modify the OCR, DTS and CTS time stamps associated with one or more media objects as will also be described in more detail below. For example, ES manager may modify the DTS and/or CTS time stamps associated with static media objects to temporally match the DTS and/or CTS time stamps associated with dynamic media objects.

In response to the OCR′ time stamps provided by TSL304and the DTS/CTS time stamps provided by SL manager302as well as the presentation metadata provided by TSL304, ES manager306may compose and/or multiplex a stream of game scenes and supply that media stream to game engine DMIF206for delivery to game engine110. The composed game scenes may incorporate dynamic media objects swapped for static media objects supplied through storage DMIF210. The composed game scenes may also incorporate additional media objects supplied through server DMIF208. In particular, in response to the time stamps provided by both SL manager302and TSL304, ES manager306may replace one or more static objects provided by ES manager306with one or more dynamic objects as will be described in further detail below.

In addition to streaming encoded media objects associated with one or more SL packets402, game server104may also, in some implementations, stream one or more SL packet specifying MPEG4-compatible binary information for scenes (BIFS) data. The streamed BIFS may inform content engine112of the spatial and temporal location of each media object in a game scene. MPEG4 BIFS are well known in the art and more detailed description of their function and/or structure is omitted herein in order not to unnecessarily obscure implementations of the present invention.

In response to one or more BIFS, content engine112, in particular ES manager306, may compose an associated game scene and provide that scene to game engine110through game engine DMIF206. In particular, in some implementations, a BIFS associated with a dynamic object may specify that ES manager306should replace one or more static objects with one or more dynamic objects when composing a particular game scene as will be described in more detail below.

FIG. 5is a flow chart illustrating a process500of replacing static media objects with dynamic media objects. Although process500, and associated processes, may be described with regard to system100ofFIG. 1and components of system100as illustrated inFIGS. 2 and 3for ease of explanation, the claimed invention is not limited in this regard. In addition, althoughFIG. 5may be described in the context of a game-user/player playing a free online game where an online gaming service provides streaming game content to the user/player through game server104and system100, the invention is not limited in this regard.

Processing may begin by generating game data to reflect game-user/player specific data such as the player's local time and/or some aspect of the player's profile such as the player's age or game level [act502]. One way to do this is by having media center102generate the game data. Specifically, game engine110may generate the game data and supply that game data to content engine112. In some implementations, a game-user/player may first download and/or obtain free online game software including static media objects from a game provider. One way to do this would be to download the game software using media center102, although the invention is not limited in this respect. In some implementations, the downloaded static media objects may be stored in storage108. In some implementations, the static objects stored in storage108may be associated with time stamps, for example, static DTS and/or CTS time stamps. The game-user/player may then establish communication via a network with a game server that may be distinct from the game provider. One way to do this may be to establish communication over network106with game server104using media center102and/or content engine112.

Process500may continue by sending the game data to a game server [act504]. This may be accomplished by content engine112sending the game data to game server104over network106. In other implementations, however, game engine110may send the game data to game server104. In some implementations, content engine112may obtain game data stored in storage108using storage DMIF210and provide that game data to game server104using server DMIF208. In some implementations, the game data stored in storage108may be updated and/or modified using game parameters obtained from game engine110using snooping logic204. For example, when a game-user/player is winning a game, snooping logic204may update and/or modify the game data stored in storage108to reflect this.

Processing may continue with game server104, or a game content generating component (not shown) coupled to game server104, generating and/or providing a dynamic media object in response to the game data [act506-A]. In some implementations, such a dynamic object may comprise personalized advertising generated in response to the game data, although the invention is not limited in this regard. Game server104, or a game content generating component coupled to game server104, may also generate a modified BIF [act506-B] associated with the dynamic media object generated in act506A. Game server104may send the dynamic media object and modified BIFS to media center102[acts508-A and508-B respectively].

In some implementations, coordination server114may, in response to the game data, provide control data to game server104specifying which dynamic media objects it should supply to content engine112. Coordination server114may do so in response not only to game data received from media center102but also in response to game data received from one or more additional media centers (not shown) that are also coupled to game server104through network106. For example, although the invention is not limited in this regard, multiple game-users/players using media center102and one or more additional media centers may use game server104to play a multi-player online game. Coordination server114may then provide coordination for the multi-player online game by, for example, incorporating and/or tracking the actions of the multiple game-users/players as provided by multiple game-users/players' respective game data and, in response, providing control data to game server104.

By communicating with multiple game-users/players and coordinating the content provided by game server104, coordination server114may enable system100to provide dynamic content personalized to each one of the multiple game-users/players. For example, a game-user/player that is losing the multi-player online game may receive different dynamic content than a game-user/player that is winning the multi-player online game.

In some implementations, processing may continue by replacing a static object with the dynamic object as specified by and in response to the modified BIFS [act510]. One way to do this is by having ES manager306of content engine112replace a static object specified by the modified BIFS with a dynamic object associated with and/or specified by the modified BIFS. For example, the modified BIFS may specify that SL manager302provide a specific static object associated with a particular OCR time to ES manager306. More specifically, although the invention is not limited in this regard, the OCR time stamp of the SL packet402associated with the dynamic object may be modified by TSL304and the modified OCR′ supplied to SL manager302. SL manager302may then instruct storage DMIF210to pull one or more static objects in response to the OCR′ time stamp. SL manager302may then provide the static object and its associated DTS and CTS time stamps to ES manager306. In some implementations, ES manager306may then modify the static objects' DTS and CTS time stamps in response to the OCR′ time stamps provided by TSL304to match the DTS and CTS time stamps of the dynamic object. ES manager306may then swap and/or replace the static media object with the dynamic media object.

Processing may proceed with the incorporation of the dynamic media object along with other objects into a stream of objects representing a game scene [act512]. In some implementations, this may be done by ES manager306incorporating and/or composing the dynamic media object with other media objects obtained from either storage108and/or game server104into a single multimedia object stream. In some implementations the multimedia object stream produced by ES manager306may be an MPEG4 media object stream, although the invention is not limited in this regard.

The acts shown inFIG. 5need not be implemented in the order shown; nor do all of the acts necessarily need to be performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. In addition, process500may be repeated one or more times during the course of playing an online game thereby permitting personalized content to be inserted into multiple online game scenes. Further, at least some of the acts in this figure may be implemented as instructions, or groups of instructions, implemented in a machine-readable medium.

The foregoing description of one or more implementations consistent with the principles of the invention provides illustration and description, but is not intended to be exhaustive or to limit the scope of the invention to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practice of various implementations of the invention.

For example, the system, apparatus and methods for dynamically inserting personalized content in online gaming scenes described herein are not limited to systems or apparatus which use MPEG4-compatible media objects. Rather, the claimed invention also contemplates other protocols capable of supporting the dynamic insertion of personalized content in online gaming scenes. Also, although described in terms of using external storage108to store static objects, in some implementations static objects may be stored in storage and/or memory internal to media center102. For example, static object storage may be provided by memory embedded within content engine112and/or game engine110. Clearly, many other implementations may be employed to provide for the dynamic insertion of personalized content in online gaming scenes Consistent with the claimed invention.

No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Variations and modifications may be made to the above-described implementation(s) of the claimed invention without departing substantially from the spirit and principles of the invention. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.