U.S. Pat. No. 9,616,333

DETAIL DESCRIPTIONS OF THE INVENTION

All illustrations of the drawings are for the purpose of describing selected versions of the present invention and are not intended to limit the scope of the present invention.

The present invention is a method for capturing and implementing body movement data through a video game engine. The overall process that is followed by the present invention is shown inFIG. 1while secondary processes that are followed by the present invention are shown inFIGS. 2-20.

With reference toFIG. 1, the present invention utilizes a plurality of inertial measurement units (IMUs) and a plurality of slave microcontrollers. The plurality of IMUs is worn on the user's body and captures data based on the user's movements that corresponds to movements within a video game engine. In the preferred embodiment of the present invention, each of the plurality of IMUs is electronically connected to a corresponding microcontroller from the plurality of slave microcontrollers as shown inFIG. 13. This allows the plurality of IMUs to be worn on multiple user body locations to more accurately capture data based on the user's movements. As shown inFIG. 16, the plurality of IMUs includes a first wrist IMU, a second wrist IMU, a first ankle IMU, and a second ankle IMU, allowing data to be captured from these user body locations. However, the plurality of IMUs may include additional IMUs for capturing data from other user body locations. Alternatively, there may be fewer IMUs as well. The plurality of IMUs may additionally be attached to a tennis racket, a step machine, or a similar sports accessory or item. This allows the plurality of IMUs to capture the user's movements when the user is using various accessories or items while playing. In the preferred embodiment of the present invention, each of the plurality of IMUs comprises at least one accelerometer and at least one gyroscope as shown inFIG. 17. The at least one accelerometer is able to detect the magnitude and direction of proper acceleration experienced by the user's body locations during movement. The at least one gyroscope functions in conjunction with the at least one accelerometer and is able to account for orientation as well as rotation of the user body locations as the user moves.

With continued reference toFIG. 1, the present invention further utilizes a master microcontroller and a computing device. As shown inFIG. 14, the video game engine is installed onto a storage device of the computing device, enabling the computing device to be utilized when playing a video game. The master microcontroller is communicatively coupled to the computing device and the plurality of slave microcontrollers, enabling feedback and two-way communication between the video game engine and the plurality of slave microcontrollers. The connection between the master microcontroller and the computing device may be wired via a connection wire as shown inFIG. 18. In this case, the master microcontroller is electronically connected to the computing device through the connection wire, forming a wired connection. Alternatively, the connection between the master microcontroller and the computing device may be wireless via a wireless communication module as shown inFIG. 19. In this case, the master microcontroller is electronically connected to the computing device through the wireless communication module, forming a wireless connection. In alternative embodiments of the present invention, the plurality of slave microcontrollers may be communicatively coupled directly to the computing device without a need for the master microcontroller.

With further reference toFIG. 1, live movement time series data is captured through the plurality of IMUs. The live movement time series data corresponds to the user's body movement that is captured in real-time while the user is playing. After the live movement time series data is captured, the live movement time series data is received through the plurality of slave microcontrollers. Because each of the plurality of IMUs is electronically connected to a corresponding microcontroller from the plurality of slave microcontrollers, the plurality of slave microcontrollers is able to receive the live movement data time series data from one or more user body locations on which the plurality of IMUs is worn. The live movement time series data is then continuously transmitted from the plurality of slave microcontrollers to the master microcontroller. The master microcontroller is thus able to aggregate the live movement time series data from multiple user body locations.

The live movement time series data is continuously compared to prerecorded movement time series data through the master microcontroller in order to calculate a cost distance between each of a plurality of live points for the live movement time series data and each of a plurality of prerecorded points for the prerecorded movement time series data. The cost distance is a metric corresponding to the similarity between the live movement time series data and the prerecorded time series data and is indicative of the distance between the live movement time series data and the prerecorded time series data. If sufficiently small, the cost distance enables the user's live body movements in the live movement time series data to be “recognized” in the prerecorded time series data and implemented accordingly through the video game engine. After the cost distance is calculated, an in-game movement is implemented through the video game engine according to the cost distance between a selected point from the plurality of live points and a corresponding point from the plurality of prerecorded points. As a result, the user's body movements captured through the plurality of IMUs may thus be implemented through the video game engine. As shown inFIG. 4, the prerecorded movement time series data is retrieved from the computing device through the master microcontroller, enabling the master microcontroller to compare the prerecorded movement time series data that is stored on the computing device to the live movement time series data that is captured in real-time by the plurality of IMUs.

With reference toFIG. 2, in the preferred embodiment of the present invention, a specified movement is implemented as the in-game movement through the video game engine if the cost distance between the selected point and the corresponding point is less than a designated implementation threshold. The designated implementation threshold is a specified value for the cost distance at which a user's body movement in the live movement time series data is recognizable or unrecognizable as a corresponding movement in the prerecorded movement time series data. As a result, if the cost distance is sufficiently small, the specified movement is recognized and implemented through the video game engine. However, if the cost distance between the selected point and the corresponding point is greater than the designated implementation threshold, an alternative movement is implemented as the in-game movement through the video game engine as shown inFIG. 3. In this case, the cost distance between the selected point and the corresponding point is considered too large, rendering the user's body movement in the live movement time series data unrecognizable as the specified movement. As a result, the alternative movement is recognized as the movement that most closely correlates to the selected point in the live movement time series data and is implemented through the video game engine.

The present invention is utilized to control the user's gameplay experience through difficulty adjustment and sensory-enhancement. In the preferred embodiment of the present invention, difficulty adjustment is primarily regulated through biometrics. With reference toFIG. 5, the present invention utilizes a plurality of biometric sensors that is integrated with the plurality of IMUs. Biometric data is continuously captured through the plurality of biometric sensors while the user is playing. The biometric data is utilized to manage the user's gameplay experience. The biometric data is received through the video game engine for processing. A video game difficulty is managed through the video game engine by analyzing the biometric data. For example, if the plurality of biometric sensors detects that the user is struggling, the video game difficulty may be lowered. The video game difficulty is managed in order to prevent the user from overexerting while playing. An algorithm will be utilized to learn the user's threshold and increase or decrease the video game difficulty accordingly.

With reference toFIG. 6, the present invention may additionally utilize at least one sensory-enhancing device that is managed by the video game engine in order to deliver a personalized user gameplay experience. The at least one sensory-enhancing device is activated through the video game engine and may produce various effects. Effects that may be produced by the at least one sensory-enhancing device include, but are not limited to, fragrance and mist.

With reference toFIG. 7, a heartrate monitor is provided as one of the plurality of biometric sensors. The heartrate monitor detects the user's heartrate while the user is playing. A difficulty threshold serves as the point at which the video game engine increases or decreases the video game difficulty. A heartrate reading is continuously captured through the heartrate monitor, enabling the heartrate monitor to continuously monitor the user's heartrate while the user is playing. The video game difficulty is decreased if the heartrate reading is higher than the difficulty threshold as the heartrate reading being higher than the difficulty threshold is indicative of the user struggling.

As shown inFIG. 8, a body thermometer is provided as one of the plurality of biometric sensors. The body thermometer detects the user's body temperature while the user is playing. A body temperature reading is continuously captured through the body thermometer, enabling the body thermometer to continuously monitor the user's body temperature while the user is playing. The video game difficulty is decreased if the body temperature reading is higher than the difficulty threshold as a high body temperature reading is indicative of the user exerting considerable effort.

With reference toFIG. 9, a moisture sensor is provided as one of the plurality of biometric sensors. The moisture sensor detects moisture on the user's body while the user is playing. A body sweat reading is continuously captured through the moisture sensor in order to gauge the user's efforts while playing. The video game difficulty is decreased if the body sweat reading is higher than the difficulty threshold as a high body sweat reading is indicative of the user exerting significant physical effort.

With reference toFIG. 10, a timer for the video game engine is utilized to measure the amount of time that the user has played. A game usage time is continuously monitored through the timer to manage the video game difficulty based on the amount of time that the user has played. The video game difficulty is decreased if the game usage timer is higher than the difficulty threshold, indicating that the user has played for an extensive amount of time.

In the preferred embodiment of the present invention, the biometric data includes user-registered data as well as collected data as shown inFIG. 11. The user-registered data includes various user data such as body fat percentage, heartrate, age, weight, height, chest size, waist size, and hip size. The video game difficulty is managed through the video game engine by comparing the collected data against the user-registered data. As a result, the collected data that is gathered in real-time as the user plays is analyzed with respect to the user-registered data in order to provide personalized management of the video game difficulty. The present invention may utilize additional biometric hardware such as a fingerprint reader and a retina scanner in order to allow the user to unlock, authenticate, and use the computing device.

The video game difficulty may also be managed based on the user's input. With reference toFIG. 12, at least one user input is provided to enable the user to physically manage the video game difficulty. The user is prompted to input at least one user input through the video game engine. The prompt may be periodic or initiated by satisfaction of a condition such as the collected data (e.g. heartrate reading, body temperature, body sweat reading, game usage time) exceeding the difficulty threshold. The at least one user input is received through the video game engine for processing. The video game difficulty is managed through the video game engine by analyzing the at least one user input. The at least one user input may be manual (provided through a switch) or audible (provided through a microphone).

As seen inFIG. 15, each of the plurality of IMUs is able to utilize a wireless transceiver for wireless communication between the plurality of slave microcontrollers and the master microcontroller. The live movement time series data is wirelessly transmitted from each of the plurality of slave microcontrollers to the master microcontroller through the wireless transceiver. The use of the wireless transceiver eliminates any potential hindrances to the user's movement while the user is playing.

With reference toFIG. 20, a wireless sync transmitter for the master microcontroller and a wireless sync receiver for the computing device may be utilized to associate the master microcontroller, the plurality of slave microcontrollers, and the plurality of IMUs to the computing device, eliminating the need for a physical connection between the components. The master microcontroller, the plurality of slave microcontrollers, and the plurality of IMUs are wirelessly synced to the computing device through the wireless sync transmitter and the wireless sync receiver. The plurality of IMUs may then begin to capture and wirelessly transmit the live movement time series data to the master microcontroller through the plurality of slave microcontrollers. The master microcontroller is then able to wireless transmit the cost distance to the video game engine before the in-game movement is implemented.

Although the present invention has been explained in relation to its preferred embodiment, it is understood that many other possible modifications and variations can be made without departing from the spirit and scope of the present invention as hereinafter claimed.