U.S. Pat. No. 10,987,584

DETAILED DESCRIPTION

FIG. 1illustrates a system10configured to provide targeted recommendations to segments of users in a virtual space. System10may be configured to provide the virtual space to the users over a network. Providing the virtual space may include hosting the virtual space over a network. System10may be configured to perform A/B testing on users in order to determine targeted recommendations for segments of users. A targeted recommendation may enhance a user's enjoyment of the virtual space and/or increase the chances of success for the user in the virtual space. The A/B testing may be performed on specific segments of the users, where such segments are determined on user parameters. The user parameters used to segment the users may include one or more of demographic parameters, social parameters, game parameters, activity parameters, and/or other parameters. In some implementations, system10may include a server12. The server12may be configured to communicate with one or more client computing platforms14according to a client/server architecture. The users may access system10and/or the virtual space via client computing platforms14.

Server12may be configured to execute one or more computer program modules. The computer program modules may include one or more of a space module16, a user module18, a parameter module20, a segmentation module22, a recommendation generation module24, a transmission module26, a recommendation redemption module28, a tracking module30, an analysis module32, and/or other modules.

Space module16may be configured to implement the instance of the virtual space executed by the computer modules to determine view information defining views of the virtual space. The view information may then be communicated (e.g., via streaming, via object/position data, and/or other information) from server12to client computing platforms14for presentation to users. The view information determined and transmitted to a given client computing platform14may correspond to an entity being controlled by a user via the given client computing platform14. The view information determined and transmitted to a given client computing platform14may correspond to a location in the virtual space (e.g., the location from which the view is taken, the location the view depicts, and/or other locations), a zoom ratio, a dimensionality of objects, a point-of-view, and/or view parameters. One or more of the view parameters may be selectable by the user.

The instance of the virtual space may comprise a simulated space that is accessible by users via clients (e.g., client computing platforms14) that present the views of the virtual space to a user. The simulated space may have a topography, express ongoing real-time interaction by one or more users, and/or include one or more objects positioned within the topography that are capable of locomotion within the topography. In some instances, the topography may be a 2-dimensional topography. In other instances, the topography may be a 3-dimensional topography. The topography may include dimensions of the space, and/or surface features of a surface or objects that are “native” to the space. In some instances, the topography may describe a surface (e.g., a ground surface) that runs through at least a substantial portion of the space. In some instances, the topography may describe a volume with one or more bodies positioned therein (e.g., a simulation of gravity-deprived space with one or more celestial bodies positioned therein). The instance executed by the computer modules may be synchronous, asynchronous, and/or semi-synchronous.

The above description of the manner in which views of the virtual space are determined by space module16is not intended to be limiting. Space module16may be configured to express the virtual space in a more limited, or more rich, manner. For example, views determined for the virtual space may be selected from a limited set of graphics depicting an event in a given place within the virtual space. The views may include additional content (e.g., text, audio, pre-stored video content, and/or other content) that describes particulars of the current state of the place, beyond the relatively generic graphics. For example, a view may include a generic battle graphic with a textual description of the opponents to be confronted. Other expressions of individual places within the virtual space are contemplated.

Within the instance(s) of the virtual space executed by space module16, users may control entities to interact with the virtual space and/or each other. The entities may include one or more of characters, objects, simulated physical phenomena (e.g., wind, rain, earthquakes, and/or other phenomena), and/or other elements within the virtual space. User-controlled characters may include avatars. As used herein, an entity may refer to an object (or group of objects) present in the virtual space that represents an individual user. The entity may be controlled by the user with which it is associated. The user controlled element(s) may move through and interact with the virtual space (e.g., non-user characters in the virtual space, other objects in the virtual space). The user controlled elements controlled by and/or associated with a given user may be created and/or customized by the given user. The user may have an “inventory” of virtual goods and/or currency that the user can use (e.g., by manipulation of a user character or other user controlled element, and/or other items) within the virtual space.

Control over the entities may be exercised by the users through control inputs and/or commands input through client computing platforms14. The users may interact with each other through communications exchanged within the virtual space. Such communications may include one or more of textual chat, instant messages, private messages, voice communications, and/or other communications. Communications may be received and entered by the users via their respective client computing platforms14. Communications may be routed to and from the appropriate users through server12(e.g., through space module16).

Within the virtual space, users may participate in a game. The game may include various tasks, levels, quests, and/or other challenges or activities for users to participate in. The game may include activities in which users (or their entities) are adversaries, and/or activities in which users (or their entities) are allies. The game may include activities in which users (or their entities) are adversaries of non-player characters, and/or activities in which users (or their entities) are allies of non-player characters. In the game, entities controlled by the user may obtain points, virtual currency or other virtual items, experience points, levels, and/or other demarcations indicating experience and/or success. Space module16may be configured to perform the functions associated with the game in executing the instance of the virtual space.

User module18may be configured to access and/or manage one or more user accounts associated with users of system10. The user accounts may include user information. The one or more user accounts and/or user information may include information stored by server12, one or more of the client computing platforms14, and/or other storage locations. The user accounts may include, for example, information identifying users (e.g., a username or handle, a number, an identifier, and/or other identifying information) within the virtual space, security login information (e.g., a login code or password), virtual space account information, subscription information, virtual currency account information (e.g., related to currency held in credit for a user), relationship information (e.g., information related to relationships between users in the virtual space), virtual space usage information, demographic information associated with users, interaction history among users in the virtual space, information stated by users, activity information of users, browsing history of users, a client computing platform identification associated with a user, a phone number associated with a user, user settings, and/or other information related to users. The user information may include and/or indicate an activity level of the user. The activity level may include previous login time(s), previous logout time(s), login frequency, time spent logged in, and/or other activity information. The user information may also include information related to purchases in or for the virtual space. Such information may include, for example, activity information for individual transactions, a spend rate, a total spend amount, and/or other information related to user purchases.

As will be discussed further below, users may participate in the virtual space by controlling entities within the virtual space. The user information in the user accounts may include information related to the entities controlled by the users in the virtual space. Such information may include, for example, an entity type, an entity class, an entity identification, a level, inventory information, status information, and/or other information related to entities controlled by users in the virtual space.

Parameter module20may be configured to obtain values for one or more user parameters. The values may be stored to the user profiles managed by user module18. The user parameters may include one or more of a demographic parameter, a social parameter, a game parameter, an activity parameter, and/or other parameters.

A demographic parameter may include a parameter related to the real world demographics of the users. For example, a demographic parameter may include one or more of age, gender, geographic location, language, income, education level, career, marital status, and/or other demographic parameters. Parameter module20may be configured to obtain values of demographic parameters based on registration information provided by the user to system10(e.g., upon opening an account to participate in the virtual space and/or the game), information received from an online platform from which the virtual space is accessed (e.g., via a social network website, a microblogging service, and/or other online platforms), information derived or deduced from one or more other user parameters (e.g., determined from a usage parameter, from a social parameter, and/or other parameter), and/or from other information or sources.

A social parameter may include a parameter related to a social network and/or the manner and/or individuals with which a user socializes in and/or out of the virtual space. By way of non-limiting example, a social parameter may include one or more of a parameter derived from a social graph in a social network service, an in-game relationship, a social platform from which the virtual space is accessed, and/or other social parameters. Parameter module20may be configured to obtain values of social parameters from relationships established within the virtual space (e.g., friendships, alliances, and/or other relationships), information received from an online platform from which the virtual space is accessed, information related to the usage of the virtual space (e.g., other users with whom a first user participates in the virtual space and/or the game, other users with whom a first user communicates in the virtual space, and/or other information related to usage), and/or from other information or sources.

A game parameter may include a parameter related to participation and/or usage of the user in the game and/or the virtual space. By way of non-limiting example, a game parameter may include one or more of an entity class, an entity faction, a usage amount, one or more usage times, a level, inventory in the virtual space, a score, and/or other parameters. Parameter module20may be configured to obtain values of the game parameters by monitoring user interaction with the virtual space. This may include monitoring interactions of users and/or the entities they control in the instance of the virtual space executed space module16, and/or monitoring other interactions.

An activity parameter may be related to an activity history of the user in the virtual space. By way of non-limiting example, an activity history may include one or more of activity level of the user, virtual currency account information of the user, relationship information of the user, virtual space usage information of the user, interaction history of the user, browsing history of the user, purchase history and/or other activity history. An activity parameter may also include one or more of an average participation time in a session of the user, an average time in a session after the user experiences an event, an average time between sessions of the game, a value metric representing the value of the user, a spend velocity, and/or other activity parameters. An event may include, for example, joining an alliance, losing status, losing money, getting lost in the virtual game, losing a friendship, gaining a friendship, losing an alliance, receiving an offer for a virtual item, purchasing a virtual item, changing a user entity, changing a view of the game, engaging in activity with another user, entering a new level of the game, engaging in a training session, engaging in an in-game experience separate from the game, and/or other events. A purchase history may include information relating to, for example, user purchases, user sales, user exchanges, user browsing for potential virtual items for purchase, and/or other purchase events. A purchase may include a transaction in which real world currency is exchanged for one or more virtual items, a transaction in which one or more virtual items (e.g., virtual currency) is exchanged for another one or more virtual items, and/or other transactions in which one or more virtual items are received or given by a user.

Segmentation module22may be configured to form segments of the users on one or more of the parameters for which values are obtained by parameter module20. For example, a first segment of users may be formed based on values of a first parameter. The first segment may include some portion of the overall users having values of the first parameter that correspond to each other. This may include values that are the same, values that are equal, values that fall within a specified range, values that are at least as adjacent as a threshold of some adjacency metric, and/or other values that correspond to each other. The use of an exemplary segment formed based on a single parameter is not intended to be limiting. The scope of this disclosure extends to implementations in which the first segment is formed on the first parameter, and on one or more other parameters.

In some implementations, segmentation module22may be configured to identify segments of users that have similar tendencies. For example, a segment of users may have similar activity tendencies, similar usage tendencies, similar responses to recommendations, and/or other tendencies in common. Segmentation module22may identify segments of users having similar tendencies through analysis of the parameters obtained by parameter module20, and/or from other sources.

In some implementations, segmentation module22may be configured to receive administrator selection of one or more values for one or more parameters. Segmentation module22may be configured to define a graphical user interface that is provided to an administrator user to receive entry and/or selection of such values.

By way of illustration,FIG. 2depicts a view40of a graphical user interface configured to receive entry and/or selection of a definition of a segment of users of the virtual space. View40may include one or more of a name field42, a demographic parameter field44, a social parameter field46, a game parameter field48, an activity parameter field50, and/or other fields. Name field42may be configured to receive entry and/or selection of a name for a user segment being defined by the administrator. Demographic parameter field44may be configured to receive entry and/or selection of one or more demographic parameters that should be used to form the user segment. Through demographic parameter field44, the administrator may enter and/or select parameter value(s) for the entered and/or selected one or more demographic parameters. The entered and/or selected parameter value(s) may be implemented to determine whether the individual users should be included in the user segment. The entered and/or selected parameter value(s) may specify a value that users should have to included and/or excluded from the segment, a range to determine whether users should be included and/or excluded, and/or may specify inclusion or exclusion in other ways. Social parameter field46may be configured to receive entry and/or selection of one or more social parameters that should be used to form the user segment. Through social parameter field46, the administrator may enter and/or select parameter value(s) for the entered and/or selected one or more social parameters. Game parameter field48may be configured to receive entry and/or selection of one or more game parameters that should be used to form the user segment. Through game parameter field48, the administrator may enter and/or select parameter value(s) for the entered and/or selected one or more game parameters. Activity parameter field50may be configured to receive entry and/or selection of one or more activity parameters that should be used to form the user segment. Through activity parameter field50, the administrator may enter and/or select parameter value(s) for the entered and/or selected one or more activity parameters.

Returning toFIG. 1, recommendation generation module24may be configured to generate one or more sets of alternative recommendations for respective one or more segments of the users. This may include a set of recommendations for the first user segment that includes at least a first recommendation and a second recommendation. A recommendation may prompt a user to take one or more recommended actions in the game. The recommendation generation module24may be configured such that the first recommendation recommends a first action, and the second recommendation recommends a second action that is different from the first action. The recommendation generation module24may be configured such that the first recommendation recommends a first set of actions, and the second recommendation recommends a second set of actions that is different from the first set of actions. The recommendation generation module24may also be configured to generate one or more recommendations for an individual user based on user parameters. The individual user parameters may be user parameters accessed by segmentation module22, user parameters associated with the individual user, and/or other individual user parameters.

The recommendations within a set of recommendations may differ by one or more variables. The variables of a recommendation may include one or more of the action(s) recommended, the types of actions recommended, the format in which the recommendation is arranged, the method of transmission, one or more incentives related to the one or more recommended actions, and/or other variables. A type of action may include action related to a user entity, to a user relationship, to a view, to a user's activity level, to a user's purchasing ability, to a virtual item, and/or other type of action. An incentive to undertake one or more of the recommended action(s) may include a discount on a price of a virtual item, an ability to undertake another recommended action, and/or other incentives. Non-limiting examples of recommended actions may include, among other things, an upgrade to be obtained for a virtual building, virtual object, virtual troop, or other virtual entities; a membership in a group to be sought (e.g., membership in an alliance, guild, or other group); a virtual building, virtual object, virtual troop, or other virtual entity to be obtained; a quest or other task to be undertaken; a battle or conflict to be engaged in; and/or other actions.

A recommended action may provide a functional advantage in the virtual space (e.g., an advantage in the game). A recommended action may be expressed aesthetically within views of the instance of the virtual space. A recommended action may have value in the virtual space that facilitates increasing a bargaining position of the user. For example, an ability to undertake the recommended action may be transferred by the user to another user for some other consideration within the virtual space (e.g., for an ability to undertake another action, for a relationship, for an alliance, for a virtual item, for a service, for an advantage, and/or other consideration). The transfer may involve, for example, a sale, a sharing, a gift, and/or other transfer. Consideration may also include, for example, virtual items, virtual currency, agreement to join in an alliance, establishment of a relationship, ceding of territory, agreeing to a treaty and/or peace terms in a battle, increased status in the game, increased power, sharing of resources, and/or other consideration.

Recommendation generation module24may be configured to vary the variables in between recommendations in the sets of recommendations to perform A/B testing on the segments of users. The determination of the values of the variables in the individual recommendations (e.g., one or more of the variables to be varied between the recommendations) may be performed manually (e.g., through a graphical user interface defined by recommendation generation module24) by one or more administrators. The determination of the values of the variables in the individual recommendations (e.g., one or more of the variables to be varied between the recommendation) may be made automatically. Any technique for determining differences between recommendations as part of A/B testing may be implemented without departing from the scope of this disclosure. Recommendation generation module24may also be configured to vary the variables in between recommendations in the sets of recommendations to an individual user based on user parameters for the individual user, user information for the individual user, and/or other user data.

Transmission module26may be configured to transmit the recommendations to the users. Transmission module26may transmit a recommendation to a user via one or more of in-game notification, a text message (e.g., via SMS and/or other text message types), an email, a chat message, and instant messenger message, a mobile device alert, and/or other communication media. The media for transmitting the recommendation may be dictated by the recommendation (e.g. as one of the variables), based on a user preference or setting, based on a determination as to whether the user is logged in to the virtual space and/or the game, and/or other information.

Transmission module26may be configured to transmit the recommendations responsive to corresponding triggering events. Such events may include, for example, a game event, a specified real world timing, a user participation in the game, a lack of user participation in the game, and/or other events. A trigger for a triggering event may include, for example, a decrease in an amount of time that the user has spent in a session, an amount of time the user has spent without logging into the game, a dropoff in a ratio of time playing the game in a single session versus time to return to the game in the virtual space, a decrease in participation while in the game, a decrease in participation with other users, an unexpected ending of a game session by a user, a selling of an unexpected amount user's virtual assets, a breaking and/or weakening of an alliance, a rejection of a request for a relationship (e.g., friendship, alliance, and/or other relationship in the game) and/or other triggers. The trigger may be spurred by statistically significant change or may rely on exceeding a threshold value. The threshold amount may be, for example, based on an average of all user activity, based on average activity of an individual user, set by an administrator, and/or otherwise determined.

Recommendation redemption module28may be configured to redeem recommendations transmitted by recommendation transmission module26. This may include undertaking a recommended action, treating the recommendation like a virtual item, and/or other redemptions. For example, recommendation redemption module28may facilitate the sharing or transfer, by the user, of the recommendation. As mentioned above, a recommended action may have value in the virtual space that facilitates increasing a bargaining position of the user.

Tracking module30may be configured to track the responses of users to the individual recommendations generated by recommendation generation module24. This may include determining a time spent viewing a recommendation, determining whether a user viewed a recommendation, determining whether a user redeemed a recommendation (e.g., undertaking the recommended action, transferring the recommendation, and/or other redemption activity), and/or other tracking activities. By way of example, responsive to transmission of the first recommendation to a first subset of users in the first segment, and to transmission of the second recommendation to a second subset of users in the first segment, tracking module30may track the responses of the first subset of users to the first recommendation and/or may track the responses of the second subset of users to the second recommendation.

Analysis module32may be configured to assess the relative effectiveness of recommendations transmitted to the segments of users based on the responses of the users. This may include determining, for individual sets of recommendations, a most effective recommendation in a set of recommendations, a least effective recommendation in a set of recommendations, a relative ranking of recommendations in a set of recommendations, and/or other determinations related to the relative effectiveness of the recommendations in an individual set of recommendations. Such determinations may be made based on the responses of the users as tracked by tracking module30. The determination that the first recommendation is more effective than the second recommendation may be made responsive to some threshold number of transmissions of the first recommendation and/or the second recommendation (alone or combined) being made, a level of difference in effectiveness between the first recommendation and the second recommendation breaching a threshold, a level of effectiveness of the second recommendation falling below a threshold level, and/or other events.

Analysis module32may be configured to determine whether the first recommendation or the second recommendation is more effective for the first segment of users based on the responses of the users in the first segment to the first recommendation and the second recommendation. For example, analysis module32may be configured to assess the relative effectiveness of a given recommendation as a function of a number of users that perform a recommended action included in the given recommendation. Analysis module34may also be configured to assess the relative effectiveness of a given recommendation as a function of retention of users that perform a recommended action included in the given recommendation versus users that do not perform the recommended action. Retention of users may be quantified, for example, as an average retention time of users that perform the recommended activity, as a number of users that are retained for a threshold amount of time, a number of users that spend a threshold amount of money in the game, an average revenue per user, and/or other quantifiers. Analysis module32may also be configured to assess the relative effectiveness of a given recommendation as a function of an average amount of participation by users in the game within a given amount of time after transmission of the recommendation. In another example, analysis module32may also be configured to assess the relative effectiveness of a given recommendation as a function of an average amount of time from the transmission of the recommendation to participation by users in the game. Time may be time in the game or real world time.

Recommendation generation module24may be configured to replace recommendations determined by analysis module32to be relatively ineffective for the individual segments. For example, responsive to a determination that the first recommendation is more effective for the first segment than the second recommendation for the first segment, recommendation generation module24may replace the second recommendation with a third recommendation. This may include replacing the second recommendation with a third recommendation in the first set of recommendations, and/or providing the third recommendation as an alternative to the first recommendation for the first segment. The third recommendation may have been previously generated or defined, of may be generated and/or defined responsive to the determination that the first recommendation is more effective for the first segment than the second recommendation.

Server12and/or client computing platforms14may be operatively linked via one or more electronic communication links. For example, such electronic communication links may be established, at least in part, via a network such as the Internet and/or other networks. It will be appreciated that this is not intended to be limiting, and that the scope of this disclosure includes implementations in which server12and/or client computing platforms14may be operatively linked via some other communication media.

A given client computing platform14may include one or more processors configured to execute computer program modules. The computer program modules may be configured to enable an expert or user associated with the given client computing platform14to interface with server12, and/or provide other functionality attributed herein to client computing platforms14. By way of non-limiting example, the given client computing platform14may include one or more of a desktop computer, a laptop computer, a handheld computer, a tablet computing platform, a NetBook, a Smartphone, a gaming console, and/or other computing platforms.

Server12may include electronic storage38, one or more processors36, and/or other components. Server12may include communication lines, or ports to enable the exchange of information with a network and/or other computing platforms. Illustration of server12inFIG. 1is not intended to be limiting. Server12may include a plurality of hardware, software, and/or firmware components operating together to provide the functionality attributed herein to server12. For example, server12may be implemented by a cloud of computing platforms operating together as server12.

Electronic storage38may comprise electronic storage media that electronically stores information. The electronic storage media of electronic storage38may include one or both of system storage that is provided integrally (i.e., substantially non-removable) with server12and/or removable storage that is removably connectable to server12via, for example, a port (e.g., a USB port, a firewire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage38may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. Electronic storage38may include one or more virtual storage resources (e.g., cloud storage, a virtual private network, and/or other virtual storage resources). Electronic storage38may store software algorithms, information determined by processor36, information received from server12, information received from client computing platforms14, and/or other information that enables server12to function as described herein.

Processor(s)36is configured to provide information processing capabilities in server12. As such, processor36may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information. Although processor36is shown inFIG. 1as a single entity, this is for illustrative purposes only. In some implementations, processor36may include a plurality of processing units. These processing units may be physically located within the same device, or processor36may represent processing functionality of a plurality of devices operating in coordination. The processor36may be configured to execute modules16,18,20,22,24,26,28,30and/or32. Processor36may be configured to execute modules16,18,20,22,24,26,28,30and/or32by software; hardware; firmware; some combination of software, hardware, and/or firmware; and/or other mechanisms for configuring processing capabilities on processor36.

It should be appreciated that although modules16,18,20,22,24,26,28,30and/or32are illustrated inFIG. 1as being co-located within a single processing unit, in implementations in which processor36includes multiple processing units, one or more of modules16,18,20,22,24,26,28,30and/or32may be located remotely from the other modules. The description of the functionality provided by the different modules16,18,20,22,24,26,28,30and/or32described below is for illustrative purposes, and is not intended to be limiting, as any of modules16,18,20,22,24,26,28,30and/or32may provide more or less functionality than is described. For example, one or more of modules16,18,20,22,24,26,28,30and/or32may be eliminated, and some or all of its functionality may be provided by other ones of modules16,18,20,22,24,26,28,30and/or32. As another example, processor36may be configured to execute one or more additional modules that may perform some or all of the functionality attributed below to one of modules16,18,20,22,24,26,28,30and/or32.

FIG. 3illustrates a method50of hosting a virtual space to client computing devices for interaction by users. The operations of method50presented below are intended to be illustrative. In some embodiments, method50may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. Additionally, the order in which the operations of method50are illustrated inFIG. 3and described below is not intended to be limiting.

In some embodiments, method50may be implemented in one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information). The one or more processing devices may include one or more devices executing some or all of the operations of method50in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured through hardware, firmware, and/or software to be specifically designed for execution of one or more of the operations of method50.

At an operation52, an instance of a virtual space may be executed. In some implementations, operation52may be performed by a space module the same as or similar to space module16(shown inFIG. 1and described above).

At an operation54, the executed instance of the virtual space may be implemented to determine view information. The view information may define views to be presented to users via client computing platforms. In some implementations, operation54may be performed by a space module the same as or similar to space module16(shown inFIG. 1and described above).

At an operation56, values of user parameters for users of the virtual space may be obtained. In some implementations, operation56may be performed by a parameter module the same as or similar to parameter module20(shown inFIG. 1and described above).

At an operation58, a first segment of users may be formed on the values of one or more of the parameters as obtained at operation56. The first segment of users may be formed on one or more values of the first parameter. In some implementations, operation58may be performed by a segmentation module the same as or similar to segmentation module22(shown inFIG. 1and described above).

At an operation60, a set of recommendations for the first segment of users may be determined. The set of recommendations may include a first recommendation and a second recommendation. The first recommendation may include a first action different from a second action included in the second recommendation. The first recommendation may different from the second recommendation by one or more variables. In some implementations, operation60may be performed by a recommendation generation module the same as or similar to recommendation generation module24(shown inFIG. 1and described above).

At an operation62, the recommendations in the one or more sets of recommendations may be transmitted to the users in the first segment. This may include transmitting the first recommendation to a first subset of the users in the first segment, and transmitting the second recommendation to a second subset of the users in the first segment. In some implementations, operation62may be performed by a transmission module the same as or similar to transmission module26(shown inFIG. 1and described above).

At an operation64, responses of the users in the first segment to the recommendations in the one or more sets of recommendations may be tracked. In some implementations, operation64may be performed by a tracking module the same as or similar to tracking module30(shown inFIG. 1and described above).

At an operation66, the effectiveness of the recommendations in the one or more sets of recommendations for the first segment may be determined. This may include determining the relative effectiveness for the first segment of the first recommendation, the second recommendation, and/or other recommendations in the one or more sets of recommendations. In some implementations, operation66may be performed by an analysis module the same as or similar to analysis module32(shown inFIG. 1and described above).

For example, analysis module32may be configured to assess the relative effectiveness of a given recommendation as a function of a number of users that perform a recommended action included in the given recommendation. Analysis module32may also be configured to assess the relative effectiveness of a given recommendation as a function of retention of users that perform a recommended action included in the given recommendation versus users that do not perform the recommended action. Retention of users may be quantified, for example, as an average retention time of users that perform the recommended activity, as a number of users that are retained for a threshold amount of time, a number of users that spend a threshold amount of money in the game, an average revenue per user, and/or other quantifiers. Analysis module32may also be configured to assess the relative effectiveness of a given recommendation as a function of an average amount of participation by users in the game within a given amount of time after transmission of the recommendation. In another example, analysis module32may also be configured to assess the relative effectiveness of a given recommendation as a function of an average amount of time from the transmission of the recommendation to participation by users in the game.

Although the system(s) and/or method(s) of this disclosure have been described in detail for the purpose of illustration based on what is currently considered to be the most practical and preferred implementations, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed implementations, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any implementation can be combined with one or more features of any other implementation.