U.S. Pat. No. 11,513,863

DETAILED DESCRIPTION

The implementations of the present disclosure are described below by specific embodiments, and a person skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this description. Although the description of the invention is presented with reference to the preferred embodiment, this does not mean that the features of the invention are limited to this embodiment. On the contrary, an objective of introducing the invention with reference to the implementations is to cover other options or modifications that may be extended based on the claims of the present disclosure. The following description contains numerous specific details in order to provide a thorough understanding of the present disclosure. The present disclosure may also be implemented without these details. In addition, some specific details are omitted from the description in order to avoid obscuring or obscuring the gist of the present disclosure. It should be noted that the embodiments of the present disclosure and the features of the embodiments may be combined with each other under the condition of no conflict.

It should be noted that in this description, similar numerals and letters refer to similar items in the following drawings. Therefore, once an item is defined in one figure, there is no need to define the item in subsequent drawings for further definitions and explanations.

The technical solutions of the present disclosure are clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

In the description of the present disclosure, it should be noted that the orientation or positional relationship indicated by the terms “central”, “upper”, “lower”, “left”, “right”, “vertical”, “horizontal”, “inner”, “outer”, etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the pointed device or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present disclosure. Furthermore, the terms “first”, “second”, and “third” are used for descriptive purposes only and should not be construed to indicate or imply relative importance.

In the description of the present disclosure, unless otherwise specified and limited, it should be noted that the terms “installed”, “attached”, and “connected” should be understood in a broad sense. For example, the connection may be a fixed connection or a detachable connection, or an integral connection; or may be a mechanical connection or an electrical connection; or may be a direct connection or an indirect connection through an intermediate medium, or may be internal communication between two elements. For the person of ordinary skill in the art, the specific meaning of the above terms can be understood according to specific conditions.

In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the embodiments of the present disclosure are further described in detail below with reference to the accompanying drawings.

A distributed cluster usually includes a plurality of node devices in communication connection. A distributed cluster architecture is used for current game servers.

Most games on the market are using the strategy of opening different services in different areas. Game players from different areas are physically isolated from each other. If there is a cross-regional gameplay, data of the players needs to be copied in real time, and a large network delay exists, and stability is poor. In addition, multiple players can correspondingly log into the game servers, and a large amount of data is also correspondingly generated. The amount of data generated by the players is generally stored in a database in the form of accessing the database. When the number of players is relatively large, the players frequently go online and go offline, and the amount of player data is too large, the limited capacity of the database cannot support the amount of player data, and consequently all the processes are caused to slow down. At the same time, each data access is done through the database, which causes a huge pressure on database IO.

A game server architecture disclosed by an embodiment of the present disclosure is described below with reference toFIGS. 1, 2, 3, 4, and 5.FIG. 1is a schematic structural diagram of a game server architecture disclosed by an embodiment of the present disclosure.FIG. 2is a schematic diagram of a correspondence between a logic server group and a game server disclosed in an embodiment of the present disclosure.FIG. 3is a schematic flowchart of a method for controlling a target user by a logic server group disclosed in an embodiment of the present disclosure.FIG. 4is a schematic flowchart of a method for balancing load of a game server by a gateway server disclosed in an embodiment of the present disclosure.FIG. 5is a schematic flowchart of a method for monitoring each server by a center server disclosed in an embodiment of the present disclosure.

As shown inFIG. 1, the game server architecture includes: game servers10, a database cluster20, a consistent-hash-based cache server30, a load balancing cluster40, a logic server group50, a center server60, and a gateway server70.

The gateway server70is respectively connected to the load balancing cluster40and the center server60, to recommend a target game server and service data corresponding to the target game server for a target user according to a predefined rule when the target user accesses to the game servers10for the first time, and enable the target user to access to the target game server by using the center server60. The gateway server70pulls operation data of the game servers10and writes the operation data to the load balancing cluster40to balance loads of the game servers10.

The logic server group50is respectively connected to the game servers10, the database cluster20, the consistent-hash-based cache server30, and the center server60to control a process of the target user, and data generated by the logic server group50is updated to the database cluster20in real time. The database cluster20uses data identification segments to balance the amount of data by weight to dynamically expand the database as the amount of data increases.

The center server60is further used to monitor the game servers10, the database cluster20, the consistent-hash-based cache server30, the load balancing cluster40, and the logic server group50.

The game servers10are connected to the consistent-hash-based cache server30, and the data generated by the game servers10and the logic server group50is synchronously updated to the consistent-hash-based cache server30.

When the data stored in the database cluster20and the consistent-hash-based cache server30is accessed, the data is preferentially accessed from the consistent-hash-based cache server30.

In some embodiments of the present disclosure, the game server10is a node where the player actually logs in, and the consistent-hash-based cache server30can store any type of in-game data. A data structure is a Key/Value structure (key-value pair data structure). Value can be a structure or binary. The consistent-hash-based cache server30uses consistent hash for uniform distribution, and when a node of the cache server is added or deleted, it will only affect a small amount of data to be migrated.

The database cluster20could alternatively be 5 Mysql physical servers, to support 30 million+ players. For the database cluster20, the database cluster20using data identification segments to balance the amount of data by weight means that users accessing to the database cluster20are uniform distributed. For example, 10 Mysql physical servers are connected to the database cluster20, the number of imported players is 1 million, and the 1 million players are uniform distributed by weight 1 on the 10 Mysql physical servers. As the number of players increases, the database cluster20is dynamically expanded to 20 Mysql physical servers as the amount of data increases, and the number of players increases to 2 million, so as to avoid a problem that players are unable to log in when the database cluster cannot be expanded to carry player data in the prior art. For the original 1 to 1 million players, the data thereof is still distributed in the original 10 Mysql physical servers, and relative routing information and data do not change. 1 million to 2 million players have a ⅓ probability of distributing in the original 10 Mysql physical servers, and have a ⅔ probability of distributing in the 10 Mysql physical servers added later. The weights that players are assigned to the different Mysql physical servers can be customized, and it is not limited in this embodiment of the present disclosure. In addition, the data structure in the database cluster20may also be a Key/Value structure, and Value may be a structure or binary.

In some embodiments of the present disclosure, as shown inFIG. 2, the game servers10correspond to the player servers, which correspond to the player process. The logic server group50corresponds to the logic physical server, the logic process group and the logic management server.FIG. 2represents a correspondence among the player processes, the logic process group, the logic server group, and the logic management server. The player process is mainly responsible for processing the player's business logic, such as building upgrades and military training in the game “Art of Conquest”. The logic process group is mainly responsible for the logical processes within the logical server group that need to be publicly locked, such as the public data modified when players entering and leaving the game server10, the plundering of public resources on the game map, and battles, etc. The logic management server is responsible for scheduling the logic process group of a certain logic server group50on a certain physical logic server group. Player's game servers10use a stateless parallel distribution strategy. The player can log in to any game server10each time the player logs in. The business logic related to the player's personal data can be completed in the game server10. After the player is offline, the player logs out by using a lightweight Erlang process. The software in the game server10is developed by using Erlang platform. Compared with the operating system, the Erlang process has an advantage of being lightweight.

In some embodiments of the present disclosure, when the player logs in to the game for the first time, the gateway server70infers the country where the player is located according to an IP of the player, and recommend to the player the game camp where the player is located in the same or similar area according to the country and the language category used by the player client, so as to facilitate communication and common development of players subsequently. Inferring a country where the player is located according to an IP of the player specifically means that the country where the player is located is directly found through the global IP positioning of the open-source IP address database.

In some embodiments of the present disclosure, the logic server group50where the player is located uses a distributed parallel cluster. In some embodiments of the present disclosure, the logic server group includes: a logic service process group, logic physical servers, and a logic management server. The logic service process group is formed on the logic management server. The logic management server is used to monitor the logic physical servers, and dynamically switch an abnormal logic physical server to another idle logic physical server when a logic physical server is abnormal. The logic management server is further used to update dynamic routing information of all logic physical servers in real time. The logic process group of each logic server group50is arranged at a logic physical server. The logic service process group includes a control process for players entering and exiting the game server10, a vision broadcasting process, a player city defense data management process, and a ranking process. Each logic service process group is monitored by the center server60or the logic management server. The center server60is responsible for restarting the logic server when the logic server corresponding to the logic service process group fails. A logic physical server may correspond to a plurality of logic service process groups. A logic physical server can accommodate a plurality of logic service process groups. When a new server is opened, if the hardware resources are idle, a new server on the same logic server group can be opened without adding hardware configuration, which avoids increasing hardware costs. An 8-core, 16G logic physical server can accommodate more than a dozen logic service process groups. In addition, the public data in the logic server group50is modified by the public process, and each player queues up to request, which ensures the consistency of the data.

If the logic server group50is abnormal, the logic management server can monitor the abnormality in real time, and switch to another logic server group50. Dynamic routing information of another logic server group can be updated in real time by the logic management server, and new request information will be routed to the new switched logic server group50, The switching time is short, and it is only within a few hundred milliseconds that the player cannot receive a response. After the switching is successful, the player receives a response immediately, so that the player can normally use the new logic server group50. For some centrally scheduled single-point services, such as logic management servers and guild war servers, globally unique data is modified by using a single process. In order to facilitate other nodes to quickly read the latest data, the embodiment of the present disclosure uses two methods of pushing and pulling to update data, the pushing method means to push, after global core data is modified, the global core data to each node (corresponding to each server) immediately. The pulling method means that when each node uses data, if there is inconsistency in the version number or digest ciphertext of the data, each node actively pulls the latest data. Generally, the data within 100 bytes is verified by the digest ciphertext, and the data exceeding 100 bytes is verified by the version number of the data. In this way, CPU consumption of the two-way calculation of the digest ciphertext is reduced.

In some embodiments of the present disclosure, the logic server group50is connected to the consistent-hash-based cache server30and the database cluster20. The center server60controls the public process to modify the public data in the logic server group50.

In some embodiments of the present disclosure, except that the gateway server70is connected to the load balancing cluster, the remaining servers are centrally deployed in a local area network (such as the Alibaba Cloud computer room in the east of the United States). When players in mainland China and some remote areas log in, they are connected unperceptively through a dedicated line, so as to achieve the purpose of stable low-latency data transmission.

In some embodiments of the present disclosure, the data generated by the player during the game is not only stored in a local cache and the database cluster20, but also updated in real time to the consistent-hash-based cache server30. In this way, the data can be read from the consistent-hash-based cache server30, avoiding the need to go through the database cluster20every time the data is read, thereby reducing the pressure on the database cluster20. In addition, the data of the player can only be modified by the player process of the certain player to maintain the consistency in the data. When there is data interaction between the player and other players, interaction is performed by using an asynchronous inter-process message. During the player is offline, the player process is directly stored in the database cluster. After the player is online, the player process is executed in the order of the storage time of the player process.

In some embodiments of the present disclosure, a battle game server performs dynamic scheduling in the form of a lobby and rooms. Each battle game opens a dynamic room process, and then the players are connected to the specific battle game server by using User Datagram Protocol (UDP), and perform the battle in the way of server frame synchronization.

In some embodiments of the present disclosure, an embodiment of the present disclosure discloses a performance profiling tool that monitors the bottlenecks of a game server by parameters, such as the number of calls for any function at a predetermined time (every 5 minutes), the call time, and the total number of calls, the total call time, and the average call time after the opening of the service, and then performs targeted optimization.

In some embodiments of the present disclosure, the game server architecture further includes: an online player routing server for monitoring the game server where each target user is located.

The online player routing server is respectively connected to the center server and the game servers. When the target user is online, the center server controls the online player routing server to modify knapsack data. When the target user is offline, the center server is switched to the logic server group to modify the knapsack data.

Specifically, message routing in the game is obtained by the online player routing server, and the specific routing information of the players can be requested in batches or individually.

In some embodiments of the present disclosure, the game server architecture further includes: a notification information push server.

The notification information push server is respectively connected to the game server and the logic server, and is used to push target messages corresponding to the game server and the logic server for the target user.

In some embodiments of the present disclosure, the game server architecture further includes: a mailbox server.

The mailbox server is respectively connected to the consistent-hash-based cache server and the database cluster.

When an email corresponding to the target user is received, only first information is updated in the consistent-hash-based cache server memory, and the first information includes a brief information list.

When the target user opens the email, second information is updated in the database cluster, and the second information is detailed information in the email, and the email is marked as read.

Specifically, the email of the player is updated and maintained by using brief information (information in the memory, corresponding to the first information) and detailed information (information in the database cluster, corresponding to the second information). When the player receives a new email, only the brief information in the memory is updated. The detailed information in the database cluster is updated only when the player opens the mailbox to actually read the content of the email. Only one email template is saved for the global email by default, and the email is sent to the mailbox of the player only after the player logs in, so that redundancy of player data is reduced. At the same time, a global mailbox domain name is updated, specifically by using optimistic lock to compare the version number of the mailbox to update it, which greatly improves concurrency while maintaining consistency.

In some embodiments of the present disclosure, when the player logs in by using a global login state lock, players need to obtain a login lock before logging in each time, avoiding inconsistent modification of data of the same player by multiple processes. When the login lock is used by more processes, there is a need to wait in line. In order to give each process a chance to log in, a kick-out signal may be sent to a previous process. After the previous process releases the login lock, the new process can obtain the login lock. If multiple processes compete for a login lock at the same time, only one waiting process is allowed in the queue and the other processes simply refused, avoiding login blocking caused by large number of processes kicking each other. Processes can include player processes and logic processes. As shown inFIG. 3, the logic server group50controlling the target user includes the following steps:

S30: A target process of a target user issues a login request.

S31: Determine whether there is another process in a waiting queue and if there is no another process, it proceeds to S32.

S32: Send a kick-out signal to the process using a login lock, and assign the login lock to the target process after the process using the login lock releases the login lock.

In some embodiments of the present disclosure, as shown inFIG. 4, a script can periodically obtain data, such as a current CPU, memory, and load of each game server10, and then periodically write the data into a load balancing cluster (such as Amazon S3 servers). The gateway server70continuously pulls the latest load files of the game servers10by using a timer, and then recommends the player data to different game servers according to the load weights, so as to implement the load balancing of the game servers.

The gateway server70pulls operation data (the current CPU, memory and load) of the game servers10and writes the operation data to the load balancing cluster to balance the loads of the game servers10, which includes the following steps:

S40: Pull current CPU load data and current memory load data of each game server by using a script.

S41: Write the current CPU load data and current memory load data into a load balancing cluster by using a gateway server.

S42: Assign the memory load data to different game servers according to load weights based on the load balancing cluster, to balance the memory load.

In some embodiments of the present disclosure, the gateway server70is mainly deployed in four major regions (Shanghai, Frankfurt, Singapore and Virginia), and is mainly responsible for functions such as recommending a game server and downloading a game version number, a patch package, and Language packafter a player device is accessed to game server for the first time.

In some embodiments of the present disclosure, as shown inFIG. 5, the center server60is a center server for all servers, and is responsible for functions such as registration of all servers (nodes), periodic state update, and switching between an active node and a standby node. Each node reports a state of itself by using a timer, including data such as a CPU, a memory, and load. Each server is provided with a standby server. After each server is successfully registered, the central server60starts a monitoring process to monitor whether each server node is alive. In this way, double guarantees can be used to ensure that when the server node fails, the active/standby switchover can be performed to ensure that each function can continue to execute normally. In addition, when the server node fails, a short message alarm can be performed.

The game server10, the database cluster20, the consistent-hash-based cache server30, the load balancing cluster40, and the logic server group50all have standby servers, and the center server60monitoring the game server10, the database cluster20and the consistent-hash-based cache server30, the load balancing cluster40, and the logic server group50includes the following steps:

S50: Respectively monitor, by a monitoring process, whether a game server, a database cluster, a consistent-hash-based cache server, a load balancing cluster, and a logic server group are alive; if there is non-alive server in the game servers, the database cluster, the consistent-hash-based cache server, the load balancing cluster, and the logic server group, it proceeds to S51.

S51: Switch the non-alive server to the corresponding standby server.

The embodiment of the present disclosure discloses a game server architecture. A gateway server is connected to a load balancing cluster and a center server. When a target user accesses to the game server for the first time, the gateway server can recommend to the target user a target game server and service data corresponding to the target game server according to a predefined rule, and enable the target user to access to the target game server by the center server. After the target user accesses to the target game server, the gateway server pulls operation data of the game server and writes the operation data into the load balancing cluster to balance load of the game server, avoiding the crash of the game servers caused by excessive load of a certain game server. The database cluster uses data identification segmentation to balance the amount of data by weight. When the amount of data generated by the game server or logical server group is too large, the database cluster can be dynamically expanded to meet the storage requirements of larger amount of data. In addition, when data in the database cluster and a consistent-hash-based cache server is accessed, the data is preferentially accessed from the consistent-hash-based cache server, avoiding the problem in prior art that each data access is done through the database, which causes a huge pressure on database IO. In addition, because the amount of player data generated is synchronously stored in the consistent-hash-based cache server, the data is accessed from the consistent-hash-based cache server, so that the data is accessed at high speed and the network latency is relatively short.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present disclosure, but not to limit the technical solution of the present disclosure. Although the present disclosure has been described in detail with reference to the foregoing embodiments, the person of ordinary skill in the art should understand that: The technical solution described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solution of the embodiments of the present disclosure.