U.S. Pat. No. 10,569,163

DETAILED DESCRIPTION

It will be appreciated that for simplicity and clarity of illustration, where appropriate, reference numerals have been repeated among the different figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein can be practiced without these specific details. In other instances, methods, procedures, and components have not been described in detail so as not to obscure the related relevant feature being described. Also, the description is not to be considered as limiting the scope of the embodiments described herein. The drawings are not necessarily to scale and the proportions of certain parts may be exaggerated to better illustrate details and features of the present disclosure.

The present disclosure, referencing the accompanying drawings, is illustrated by way of examples and not by way of limitation. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one.”

Furthermore, the term “module”, as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language, such as Java, C, or assembly. One or more software instructions in the modules can be embedded in firmware, such as in an EPROM. The modules described herein can be implemented as either software and/or hardware modules and can be stored in any type of non-transitory computer-readable medium or other storage device. Some non-limiting examples of non-transitory computer-readable media include CDs, DVDs, BLU-RAY, flash memory, and hard disk drives.

FIG. 1illustrates an embodiment of a server1that can communicate with at least one mobile terminal2. The mobile terminal2can be a smart phone, a tablet computer, or a personal data assistant (PDA).

FIG. 2illustrates that the server1comprises a storage device11and a processor12. The storage device11stores a system100providing interaction in a virtual reality (VR) multiplayer board game. The system100comprises a number of modules, which are a collection of software instructions executable by the processor12to perform the functions of the system100. In at least one embodiment, the storage device11can be an internal storage device built inside the server1. In other embodiments, the storage device11can be an external storage device removably connected to the server1. For example, the storage device11can be a smart media card, a secure digital card, or a flash card. The processor12can be a central processing unit, a microprocessor, or any other suitable chip having data processing function. In yet other embodiments, the storage device11can be located in a cloud or land-based server (not shown) accessible to the server1through any type of wireless communication systems.

FIG. 3illustrates that the system100comprises an interface providing module101, a matching module102, a VR module103, a communication controlling module104, a position analyzing module105, and an vocal analyzing module106.

Referring toFIG. 4, when the mobile terminal2connects and logins to the server1, the interface providing module101provides a game selection interface1010on a display screen21(shown inFIG. 8) of the mobile terminal2. The game selection interface1010comprises at least two board game device options1011for the player to select. Each board game device option1010corresponds to one board game and to an appropriate board game device3. The board game device3can be a physical board or other physical device appropriate for the game of mahjong, the game of chess, or games requiring playing cards. That is, the game selection interface1010can comprise a mahjong board game device option1011a, a chess board game device option1011b, and a card board game device1011c. When a player wants to play mahjong, the player can use the smart phone to login to the server1, and select the mahjong board game device option1011aof the game selection interface1010. The player can also make use of a board game device3to physically represent the state of play in the VR game. The player can sit beside his own board game device3, and operate or play the cards or small objects (player pieces32, shown inFIG. 7) appropriate for the board game device3. For a mahjong board game device, the player pieces32are mahjong tiles, for a chess board game device the player pieces32are chess pieces, and for a card board game device the player pieces32are cards. Taking the mahjong games for example, operating or playing the player pieces32can include, but is not limited to, discarding a tile, claiming a tile to match a triplet (Pong), claiming a tile for a quadruplet (Kong), claiming a tile to match a sequence (Chow), or claiming a tile to win. The server1comprises a first communication unit13. The mobile terminal2comprises a second communication unit22(seeFIG. 8). Thus, the server1and the mobile terminal2can communicate with each other through the first communication unit13and the second communication unit22. The first communication unit13and the second communication unit22each can be a wireless communication unit, for example, an RFID unit, an infrared unit, a BLUETOOTH unit, or a WI-FI unit.

When one board game device option1010of the game selection interface1010is selected, the matching unit102controls the server1to establish a connection with the board game device3. The board game device3comprises a third communication unit31(seeFIG. 7). Thus, the server1and the board game device3can communicate with each other through the first communication unit13and the third communication unit31. The third communication unit31can also be a wireless communication unit.

Referring to5, after the server1establishes the connection with the board game device3, the VR module103generates a virtual board game scene1030corresponding to the selected board game device option1010. The virtual board game scene1030can show a three-dimensional virtual environment for the corresponding board game to the player. The three-dimensional virtual environment comprises a virtual board game device1031, a plurality of virtual player pieces1032, and other virtual players (opponents). For example, when the mahjong board game device option1010ais selected, the three-dimensional virtual environment of the virtual board game scene1030comprises a virtual mahjong table, a plurality of virtual mahjong tiles, and three other virtual players. Referring toFIG. 6, in at least an embodiment, after the server1establishes the connection with the board game device3, the matching module102provides a player-number selection interface1020on the display screen21of the mobile terminal2. The player-number selection interface1020comprises at least two options (hereinafter, “player-number option”)1021for number of players in the board game selectable by the player. Each player-number option1021corresponds to the number of players. For example, the player-number options1021can comprise a two-player option1021a, a three-player option1021b, and a four-player option1021c. The VR module103generates the virtual board game scene1030only when the total number of players who connect and login to the server1through their own mobile terminals2matches the selected player-number option1021.

The communication controlling module104sends the virtual board game scene1030to the mobile terminal2through the first communication unit13, thereby allowing the mobile terminal2to display the virtual board game scene1030on the display screen21. In actual use, the player can place his mobile terminal2on a VR headset (not shown) and wear the VR headset containing the mobile terminal2. Then, the player can start to operate the player pieces32and participate and take part in the board game. Referring toFIG. 7, the communication controlling module104further sends a moving command to the board game device3through the first communication unit13. The moving command is configured to control a mechanical module33of the board game device3to move the player pieces32to a preset position for the player, thereby facilitating the operation of the player pieces32.

The mobile terminal2further comprises an image capturing unit23and a sound collecting unit24.

The image capturing unit23captures a plurality of successive images within its field of view and generates a first signal accordingly. Each image comprises a sub-image of the player piece32being operated by the player. Thus, when the player operates or moves the player piece32, the image capturing unit23can capture the images of the player piece32being operated, and the images can show an actual moving trajectory of the player piece32being operated. In at least one embodiment, the image capturing unit23is a camera.

The sound collecting unit24collects sound from ambient environment and generates a second signal accordingly. The second signal comprises a vocal instruction of the player for triggering a preset movement or event. For example, when the current state of the player pieces32(that is, the remaining mahjong tiles) matches a condition for enabling at least one preset event (in mahjong games for example, the preset event can include, but is not limited to, the aforementioned Pong, Kong, or Chow), the player can audibly express one preset event, and such an audible expression by the player can be collected by the sound collecting unit24. In at least one embodiment, the sound collecting unit24is a microphone.

The second communication unit22sends the first signal and the second signal to the server1.

The communication controlling module14receives the first signal and the second signal from the mobile terminal2through the first communication unit13.

The position analyzing module105analyzes the first signal to determine a virtual moving trajectory of a corresponding virtual player piece1032in the virtual board game scene1030. In at least an embodiment, the position analyzing module105determines a same reference or anchor point in each image (for example, a corner of the board game device3), determines an initial position of the player piece32being operated relative to the reference point in each image, determines a position of the reference point in the virtual board game scene1030, and determines a finishing position of the corresponding virtual player piece1032in the virtual board game scene1030accordingly. The finishing positions of the virtual player piece1032then form the virtual moving trajectory in the virtual board game scene1030. In another embodiment, the board game device3itself can detect the positions of player pieces32being operated by identification technologies, and transmit such positions to the server1through the third communication unit31. Thus, the position analyzing module105can determine the virtual moving trajectory of the virtual player pieces1032in the virtual board game scene1030.

The VR module103further moves the virtual player piece1032in the virtual board game scene1030according to the virtual moving trajectory, thereby simulating the actual moving trajectory of the player piece32being operated. For example, when the player discards a second mahjong tile, the VR module103moves the second virtual mahjong tile in the virtual board game scene1030accordingly, to simulate the actual process of discarding the tile.

The vocal analyzing module106analyzes the second signal to obtain the vocal instruction of the player, and generates a control command according to the vocal instruction. Since noises may exist in the sound signals, the vocal analyzing module106processes the second signal by noise reduction, thus obtaining the correct vocal expression.

The VR module103further determines at least one virtual player piece1032in the virtual board game scene1030that matches the condition for triggering the preset event included in control command, and controls the determined virtual player piece1032to perform corresponding actions in the virtual board game scene1030, thereby simulating the triggering process of the preset event. For example, when the player shouts “claiming a tile to match a triplet!”, the VR module103controls the virtual mahjong tile discarded by the opponent and a pair of virtual mahjong tiles of the player to form a triplet. In at least an embodiment, each time the virtual player piece1032in the virtual board game scene1030is operated or the preset event is triggered, the VR module103determines the operation of the virtual player piece1032or the triggering of the preset event as a past event, and stores the past event in the storage device11with a time stamp.

In at least an embodiment, the system100further comprises an artificial intelligent module107.

When it is the player's turn to operate the player pieces32, the artificial intelligent module107can analyze the past events stored in the storage device11to determine at least one preset event that can currently be triggered, and generates an event selection command according to the determined preset event. The communication control module104further sends the event selection command to the mobile terminal2through the first communication unit13. Referring toFIG. 9, the event selection command is configured to control the mobile terminal2to display an event selection interface1040on the display screen21. The event selection interface1040comprises at least one preset event option1041for the player to select. Each preset event option1041corresponds to one preset event that can currently be triggered. For example, the operation selection interface1040can comprise a “match a sequence” option1041a, a “match a triplet” option1041b, a “match a quadruplet” option1041c, and a “win” option1041d. The player can audibly select one preset event option1041, and the audible expression can be collected by the sound collecting unit24.

When it is the player's turn to operate the player pieces32, the artificial intelligent module107further analyzes the past events stored in the storage device11to generate a suggestion comprising a suggested operation. Referring toFIG. 10, in at least an embodiment, the VR module103adds the suggestion to the virtual board game scene1030. The artificial intelligent module107comprises a number of artificial intelligent models, and each artificial intelligent model is for one type of board game. Thus, the artificial intelligent module107can generate the suggestion corresponding to each type of board game. The artificial intelligent module107further stores the suggestion in the storage device with a time stamp.

In at least an embodiment, the artificial intelligent module107further analyzes the past events stored in the storage device11by Q-learning to generate a scoring table. The scoring table comprises the current state of the player pieces32, a plurality of executable operations, an expected state for each executable operation, and a reward value for each expected state. The artificial intelligent module107further compares different reward values in the scoring table to determine an executable operation with a highest reward value, and treats the determined executable operation as the suggested operation. The suggestion is then generated by the suggested operation. The scoring table is shown in table 1.

TABLE 1ExecutableCurrent state of the player piecesoperationsTile 1Tile 2. . .Tile 17DrawExpectedForm a. . .OpponentandstatemeldwindiscardReward0+10−500valueChowExpectedWaiting for the. . .Opponentandstateone necessarydrawdiscardtile to winReward+1000−20valuePongExpectedForm allForm all of. . .Form allandstatetripletsone suittripletsdiscardReward+50+30+50valueKongExpectedOpponentGrand 3. . .OpponentandstatewinchiefsdrawdiscardReward−500+100−20value

In at least an embodiment, the artificial intelligent module107further determines whether the actual operation or movement made by the player is the same as the suggested operation in the suggestion. If the actual operation is different from the suggested operation, the artificial intelligent module107determines the expected state for the actual operation, and determines the reward value for the expected state. When the reward value is less than a certain preset reward value, the artificial intelligent module107determines that the actual operation is an erroneous operation, adds the erroneous operation to a learning report that is in a form of audio, text, or image, and outputs the learning report to the mobile terminal2when the current game ends. The player can thus be reminded of a serious mistake. The artificial intelligent module107further stores the learning report in the storage device11.

Furthermore, when a next game begins, the artificial intelligent module107can further continue to determine whether an actual operation made by the player is an erroneous operation. When the actual operation is an erroneous operation, the artificial intelligent module107determines whether the inaccurate operation already exists in the learning report stored in the storage device11(that is, whether such error is repeated). When such error is found to be repeated, by reference to the learning report, the VR module103directly generates a tip on the virtual board game scene1030, reminding the player that the same mistake has occurred before.

Furthermore, when the current game ends, the artificial intelligent module107establishes a learning model according to the past events and the suggestions stored in the storage device11, to simulate an operation process during a current or next game. Thus, the player can repeatedly practice until each actual operation of the player approaches the suggested operations in the suggestions.

FIG. 11illustrates an embodiment of a method for providing interactions in a VR multiplayer board game. The method is provided by way of example, as there are a variety of ways to carry out the method. The method described below can be carried out using the configurations illustrated inFIGS. 1-10, for example, and various elements of these figures are referenced in explaining example method. Each block shown inFIG. 3represents one or more processes, methods, or subroutines, carried out in the example method. Furthermore, the illustrated order of blocks is illustrative only and the order of the blocks can change. Additional blocks can be added or fewer blocks may be utilized or the order of the blocks may be changed, without departing from this disclosure. The example method can begin at block111.

At block111, the interface providing module101provides the game selection interface1010on the display screen21of the mobile terminal2when the mobile terminal2connects and logins to the server1. The game selection interface1010comprises at least two board game device options1011for the player to select. Each board game device option1010corresponds to one board game and to an appropriate board game device3.

At block112, the matching unit102controls the server1to establish the connection with the board game device3when one board game device option1010of the game selection interface1010is selected.

At block113, the matching module102provides the player-number selection interface1020on the display screen21of the mobile terminal2. The player-number selection interface1020comprises at least two player-number options1021for number of players in the board game selectable by the player. Each player-number option1021corresponds to the number of players.

At block114, the VR module103generates the virtual board game scene1030corresponding to the selected board game device option1010when the total number of players who connect and login to the server1through their own mobile terminals2matches the selected player-number option1021.

At block115, the communication controlling module104sends the virtual board game scene1030to the mobile terminal2through the first communication unit13, thereby allowing the mobile terminal2to display the virtual board game scene1030on the display screen21.

At block116, the communication controlling module104further sends the moving command to the board game device3through the first communication unit13. The moving command is configured to control the mechanical module33of the board game device3to move the player pieces32to the preset position for the player, thereby facilitating the operation of the player pieces32.

At block117, the communication controlling module14receives the first signal and the second signal from the mobile terminal2through the first communication unit13. The first signal is generated by a plurality of successive images captured by the image capturing unit23of the mobile terminal2within its field of view. Each image comprises a sub-image of the player piece32being operated by the player. The second signal is generated by sound from ambient environment collected by the sound collecting unit24. The second signal comprises the vocal instruction of the player for triggering a preset movement or event.

At block118, the position analyzing module105analyzes the first signal to determine the virtual moving trajectory of a corresponding virtual player piece1032in the virtual board game scene1030.

At block119, the vocal analyzing module106analyzes the second signal to obtain the vocal instruction of the player, and generates the control command according to the vocal instruction.

At block120, the VR module103moves the virtual player piece1032in the virtual board game scene1030according to the virtual moving trajectory, thereby simulating the actual moving trajectory of the player piece32being operated. The VR module103further determines at least one virtual player piece1032in the virtual board game scene1030that matches the condition for triggering the preset event included in control command, and controls the determined virtual player piece1032to perform corresponding actions in the virtual board game scene1030, thereby simulating the triggering process of the preset event. Each time the virtual player piece1032in the virtual board game scene1030is operated or the preset event is triggered, the VR module103can determine the operation of the virtual player piece1032or the triggering of the preset event as a pass event, and store the pass event in the storage device11with a time stamp.

Referring toFIG. 12, in at least an embodiment, the method further comprises additional blocks after the block120.

At block121, the artificial intelligent module107analyzes the pass events stored in the storage device11to generate a suggestion when it is the player's turn to operate the player pieces32, and the suggestion comprising a suggested operation.

At block122, the artificial intelligent module107determines whether the actual operation of the player is different from the suggested operation and the actual operation is an erroneous operation. If yes, the procedure goes to block123; otherwise, block121is repeated.

At block123, the artificial intelligent module107adds the erroneous operation to a learning report, and outputs the learning report to the mobile terminal2when the current game ends. The player can thus be reminded of a serious mistake. The artificial intelligent module107further stores the learning report in the storage device11.

At block124, the artificial intelligent module107continues to generate a suggestion when a next game begins.

At block125, the artificial intelligent module107determines whether the actual operation of the player is different from the suggested operation in the suggestion and the actual operation is an erroneous operation. If yes, the procedure goes to block126; otherwise, block124is repeated.

At block126, the artificial intelligent module107determines whether the erroneous operation already exists in the learning report stored in the storage device11. If yes, the procedure goes to block127; otherwise, the procedure goes to block128.

At block127, the VR module103directly generates a tip on the virtual board game scene1030, reminding the player that the same mistake has occurred before.

At block128, the artificial intelligent module107adds the erroneous operation to the learning report to update the learning report, and outputs the updated learning report to the mobile terminal2when the current game ends.

With the above configuration, the user can enjoy the virtual board game scene1030and the tactile feedback provided by the board game device3during gaming. Thus, a new level of tactile experience can be brought to the player. Furthermore, the user can learn more skills for playing the board game better by the suggestions and the learning report provided by the server1.

It should be emphasized that the above-described embodiments of the present disclosure, including any particular embodiments, are merely possible examples of implementations, set forth for a clear understanding of the principles of the disclosure. Many variations and modifications can be made to the above-described embodiment(s) of the disclosure without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.