U.S. Pat. No. 7,625,285

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Reference will now be made in detail to various embodiments of the invention, various examples of which are illustrated in the accompanying drawings, wherein like numerals indicate corresponding elements throughout the views.

One embodiment of the present invention is schematically illustrated inFIG. 1which depicts a game controller20for collection and distributing biofeedback information of a human participant of the video gaming system in accordance with one aspect of the present invention. In an exemplary embodiment, the video game controller20comprises a conventional motion controller22, one or more conventional selection buttons24, and a communication link30to the video game system. In addition, the video game controller20(as depicted inFIG. 2), further comprises one or more components selected from the group consisting of a photoelectric plethysmography35, a galvanometer40and a thermocouple50. In one embodiment, the video game controller20, further comprises a processor60. The processor60, for example, may comprise a set of executable instructions such as in the form of software, routines, programs, algorithms, code and the like, which would inter alia, measure a users biofeedback at any given time.

The video game controller20is preferably provided in communication, such as via the token ring, ethernet, telephone modem connection, radio or microwave connection, parallel cables, serial cables, telephone lines, universal serial bus “USB”, Firewire, Bluetooth, fiber optics, infrared “IR”, radio frequency “RF”, and the like, or combinations thereof, with a video gaming system through the communication link30.

In another exemplary embodiment depicted inFIG. 2, the video game controller20further comprises at least one finger pressure cuff55. The finger pressure cuff55is configured to allow insertion of a digit (human finger) into at least a portion of the finger pressure cuff55. An exemplary view of the finger pressure cuff is depicted inFIG. 3. Once a digit is inserted into the finger pressure cuff55, compressed air inflates an inflatable bladder to create a pressure change on the inserted digit in the finger pressure cuff55. As depicted inFIG. 4, the inflated bladder55presses the tissue of the digit against the photo-diode57and LED58allowing the two to function as a photoelectric plethysmography by capturing reflected infrared light from the LED at the photo-diode.

In an alternative embodiment, an LED and photodiode are present without a pressure cuff. In this embodiment, blood volume pulse may be obtained by low pass filtering of the signal (to eliminate noise) and high pass filtering (eliminate continuous component). A LED/photo-diode can also be used to measure heart rate. The signal is first high pass filtered to remove high frequency elements and then amplified by a three stage operational amplifier. Respiration can also be obtained from fluctuations of the signal from baseline due to pulse paradoxes or baseline changes in diastolic and systolic blood pressure due to respiration.

In another exemplary embodiment, the video game controller further comprises at least one digit retention ring80located adjacent to a galvanometer40and thermocouple50as depicted inFIG. 2. The digit retention ring80is configured to allow the insertion of a digit through at least a portion of the digit retention ring80and it places the digit in communication with the thermocouple50and galvanometer40. The galvanometer40is configured to measure a change in resistance across two electrodes in physical communication with a digit. The galvanometer40electrodes measure general skin conductivity of the digit. The skin conductivity is altered by perspiration of the participant. Thus, changes in the conductivity of an electrode can be interpreted as changes in perspiration by the processor60. These changes in perspiration can then be communicated to the video gaming system through the communication link30. In one exemplary embodiment, utilizing four silver electrodes, along two wires, the voltage can be measured to monitor for changes in skin conductivity and also allow a one lead ECG (FIG. 2B). One skilled in the art will appreciate that any conventional galvanometer is suitable for the present invention.

In another exemplary embodiment, the game controller comprises multiple pulse-pressure detectors (photoelectric plethysmography). The participant, after grasping the controller, may be instructed to insert one of their digits, usually their middle finger, into at least one of the pulse-pressure-detectors finger pressure cuffs. In one exemplary embodiment at the proximal end of the finger pressure cuff are two wires which run lengthwise along the cuff into the controller. The cuff is attached to the controller by flexible plastic that is reinforced and continuous with the bladder inside the cuff. One skilled in the art will appreciate that a multitude of flexible materials may be utilized to form the inflatable bladder and the finger pressure cuff. At the end of one of the wires running lengthwise along the cuff is an LED probe that emits infrared light into the tissue of the digit. Part of the light is reflected back to a photo diode connected to the other wire running lengthwise along the cuff, and the reflected light may be substantially filtered and converted to an electrical signal which is transmitted back to the processor60and ultimately to the video gaming system through the communication link30. The finger pressure cuff can be adjusted to maintain a continuous blood volume in the tissue of the digit through a servo-control mechanism. The pressure needed to maintain a transmural pressure of approximately “0” is known as the pulse pressure. Such a device is manufactured by Finapress as well as Ohmeda Monitoring Systems and known to one skilled in the art. This information may be useful to programmers who add custom features to the software application.

In yet another exemplary embodiment, the game controller comprises a thermocouple50. The thermocouple50is configured to measure changes in temperature of the tissue of the digits. Conventional thermocouples known to one skilled in the art may be utilized for measuring the temperature of the digits. Exemplary thermocouples include K type thermocouple “alumuel-chromel” by Omega Engineering and J, K, T. E thermocouples from Io Tech. Thermistors typically give a more imprecise measurement of temperature but can also be utilized on the game controllers. Yellow Springs manufactures a thermistor that would be suitable.

FIG. 5is an exemplary side view of the video game controller in which the finger pressure cuff55and digit retention ring80are shown adjacent each other in an exemplary layout.

The game controller body and other conventional aspects may be constructed from materials known to one skilled in the art. For example, the game controller body may be constructed from polycarbonate, polystyrene, polyvinyl chloride, and the like.

Another embodiment of the present invention depicted inFIG. 6, is a biofeedback ear piece100for a video game system. The biofeedback ear piece100comprises a speaker105, a communication link108and a respiratory voice-sensor110. The speaker105and the respiratory voice sensor110are in electrical communication with the communication link108. As shown in this exemplary embodiment, the respiratory voice sensor110comprises a polyvinylidine fluoride thermocouple120. It measures voltage changes between the two layers of the thermocouple which arises from the temperature differences between inhaled and exhaled air. This signal can then be processed with the processor60into waveform to give a snapshot of the participants inspiration/expiration rate. The ear piece further comprises a microphone sensor125located adjacent the thermocouple120. Thermocouple120and microphone sensor125are located at the distal end of the ear piece100. In an exemplary embodiment, the ear piece100is made of material such as a bendable metal or polymer so the microphone sensor125and thermocouple may be placed directly in front of the mouth of a human in order to better detect respirations and communication from the participant. In an alternative embodiment, capnography or a non-invasive infrared CO2measuring and recording apparatus could be used to reveal information about respiratory rate. The proximal end of the ear piece100comprises a speaker105, through which the video game system may play audio or instructions for the participant.

Another aspect of the present invention is a video gaming system comprising of a video game processor, a computer readable medium containing executable instructions for providing a video game and the game controller of the present invention.

Often computers telecommunication with each other and share information, application and/or services. Sometimes in this setting, the various computers are referred to as nodes, which is a generic term referring to access points in an interconnected system. One type of computer network employs a client-server architecture. The portions of network applications that interact with human users are typically separated from the portions of network applications that process requests and information. Often, the portions of an application that interact with users or access network resources are called client application or client software and, portions of an application that processes requests are called server applications or server software. Client machines tend to run client software and server machines tend to run server software, however, a server can be a client as well.

In an exemplary embodiment, the video game system will typically be provided on a client machine, while the software containing the computer instruction which comprises the instructions to collect and measure the biofeedback from the human could be located on the client computer or the server computer, separate of the client machine.FIG. 8schematically illustrates a sample client-server network235which might be employed to implement an embodiment of the present invention. As one with ordinary skill in the art will readily appreciate, a client-server network is only one type of network and a variety of other configurations, such as peer-peer connections are also considered networks. In a client-server network, a plurality of nodes are interconnected to various nodes send and receive information to/from one another. As shown here, a server node (238) is interconnected with a plurality of client nodes (240) using a connection (239) such as a token ring, ethernet, telephone modem connection, radio or microwave connection, parallel cable, serial cables, telephone lines, universal serial bus “USB”, Firewire, Bluetooth, fiber optics, infrared “IR”, radio frequency “RF”, and the like or combinations thereof. As one skilled in the art can appreciate, the video game system may be connected to other video game systems or servers which further process and/or distribute the biofeedback information and similarly send instructions back to the video game system in order to respond to the biofeedback measurements with the participant. Likewise, one skilled in the art will appreciate, the video game controller of the present invention could act as a client computer itself and be connected through a communication link to a server node.

The general structure of the video game hardware (910) or the control deck can be observed inFIG. 9. As one skilled in the art will appreciate, the structure may comprise a multitude of various arrangements. On the front panel, facing the viewer are two controller adapters (920&922) that can interface With controllers. A game port (940) can be visualized on the top, back of the control deck that allows the interface of the hardware and the software. Also on the top of the control deck is an on/off switch (930) and a reset button (932) that interrupts the running of the software and returns to the beginning of the software program. On the back view of the control deck is an AC power supply port (950) that connects an outside power source to the circuit board. Also, located on the back of the system is a connector (952) that allows the processed data from the software to be transferred and displayed on a monitor via an audio/video cable.

The inside view of the hardware system is depicted inFIG. 10. The main component inside the hardware is a circuit board (960). Attached to the circuit board on the front face are adapters1and2(920&922) that interface with the controllers. Two distinct data types are sent to the circuit board via the controllers. The first is data resulting from the manual manipulation of the controller, while the second is the unprocessed physiological data that is also relayed back to the circuit board. Both undergo multi-level micro-processing, separately, on the circuit board and eventually that processed data is relayed to the software. One skilled in the art will appreciate that the physiological data and data resulting from manual manipulation may be processed together. The port (940) for interface between the software and the circuit board can be visualized on the back of the circuit board. The data is stored and accessed by the software, incorporated into the program in real time and the output is sent back to the circuit board where it is further processed and then delivered to a video monitor via the connector port (952). The a/c current is delivered to the circuit board via the A/C power supply port (950) located on the back, right of the circuit board.

FIG. 11depicts an exemplary embodiment of the present invention.FIG. 11details an exemplary progression of signals as they are processed by the present invention. The game controller has the ability to monitor various phenomenon including GSR (electrodes), temperature (thermocouple), heart rate (LED/photoreceptor) of the participant as well as mechanical manipulation of the controller through internal circuitry and electrical impulses. These raw data streams undergo a primary processing. In the case of heart rate, the signal received back from the photodiode is first high pass filtered to remove high frequency elements from the signal and then amplified in one embodiment using a three stage operational amplifier. In the case of GSR and temperature, the initial signal across the electrodes is amplified using a GSR and temperature signal amplifier, respectively. These transformations are commonplace and known to one skilled in the art. In this schematic diagram, only three physiological variable are addressed. As alluded to before, a multitude of physiological variables can be obtained through the controller including respiration rate (LED/photoreceptor or polyvinylidine fluoride thermocouple), blood volume pulse (LED/photoreceptor), digital arterial pressure (LED/photoreceptor with cuff and servo control mechanism) and P-R interval (four silver electrode system) among others. These signals undergo unique and separate primary signal processing but subsequent transformations are identical to the processing modalities discussed above. Data streams from the manual controls do not undergo primary processing. Primary signal processing may occur in the controller, in the hardware of the system, or even elsewhere depending on the particular embodiment of the invention. All modalities of signals undergo secondary signal processing at a serial port controller. The function of which is to sync the data streams together and to transform the streams into a form that can be easily recognized by a computer. The signals after primary processing are converted from analog to digital form. The resulting signals are then delivered to shift registers which pass the separate signals to a serial to parallel converter. Once in digital, parallel form the data streams have completed secondary signal processing and travel along a data bus for primary micro-processing. In one exemplary embodiment, the microprocessor relays the data to a ROM unit that stores the processed data. Then, a RAM unit accesses the data stored in the ROM unit and calculates a mean, standard deviation, running mean, running standard deviation, and lastly calculates the z value. The z value represents the number of standard deviations the physiological data xi, is away from the running mean on a second to second basis. The z values from all the separate physiological variables is delivered to shift registers, and the streams of data are converted from serial to parallel form. The parallel data is then relayed to a data bus that delivers the data to the software to be stored and read. The results is a multi-level micro-processing system that collects the stream of physiological data and sends it in a form that can be easily utilized by the software.

The information from the manual controls is initially processed in a similar manner. At a serial port controller, the data is first converted from analog to digital data, then converted into serial form by shift registers and finally converted to parallel form. This procedure is well documented in the current art of video game manufacturing and well known to one skilled in the art.

Next, the data is delivered to a distinct multi-level micro-processing unit and then relayed to the software (ROM) to be stored and read. The data from the controls is processed separately from the physiological data.

In one exemplary embodiment of the present invention, two types of information are delivered to the software: the processed data from the manual manipulation of the controller and the processed physiological data. The software is able to respond to both in the actual progression of the game/program. The software, after integrating these two signals into the running of its program, sends output back to the hardware where it undergoes multi-level processing before it is sent to a video monitor to be viewed by the player.

FIG. 12is a schematic diagram of an exemplary set of executable instructions for the primary micro-processing. The micro-processing steps process physiological data and deliver it to the software so that the software can react to the ability of the player to alter his physiology. In one embodiment, the primary microprocessor has RAM and ROM capability so that it can not only store incoming physiological data streams but also perform mathematical transformations on them.FIG. 12is an example of one such transformation. In this particular embodiment, data from all monitored physiological data streams (in digital/parallel form) are sampled approximately every second. This data is initially stored in memory. The first 120 seconds serve as an initialization period. While the data streams are being collected and stored, no calculations are made for each physiological variable. After the first two minutes pass, a mean and standard deviation are calculated. In the subsequent two minutes, these values will represent the running means and running standard deviations and will be utilized to calculate Z values for their respective physiological variable. So each physiological modality such as heart rate will have a unique running mean and standard deviation. The Z value calculated for heart rate will be delivered to the software as a stream along with Z values for other variables being monitored.

After the second two minute time period, a new mean and standard deviation will be calculated for the second time span. The result will be averaged into the running mean and running standard deviation to update these values and the new running mean and running standard deviation will be used to calculate Z values for the next 120 seconds. The running mean and running standard deviation is constantly updated every two minutes as long as game play continues uninterrupted. The Z values for all the physiological variables are continuously being relayed to the software which is referred to as the signal/software interface inFIG. 11.

The components of the videogame system (i.e., circuit boards, power supply, RAM and ROM, display adapter, etc.) are easily obtainable by one of ordinary skill in the art. The circuit board in one exemplary embodiment contains a microprocessor which contains executable instructions.

An example of how a software programmer could utilize the incoming streams of Z-values is depicted inFIG. 13. The programmer is able to utilize this processed data to incorporate outcomes within the game that depend on the player's ability to change certain measured physiological variables. The programmer can incorporate the data into the program using the degree of success so that the outcome in the game depends on how well the player is able to alter his physiology or alternatively, the programmer can us arbitrary cutoff points in all-or-nothing outcomes.

One exemplary embodiment is depicted inFIG. 13.FIG. 13depicts specific examples of how a software programmer can utilize the information from this new computer system to enhance game play. A hypothetical case will be illustrated to better explain the capabilities and features of the system. In one embodiment, a player turns on a display device and the video game machine equipped with the present invention. The player inserts a fantasy game into the control deck and as instructed, the player places his hands around the controller and inserts his second and third digits into the retention rings. The controller is equipped with a thermocouple, silver electrode system and a LED/photoreceptor to measure peripheral skin temperature (PST), galvanic skin resistance (GSR) and heart rate respectively. The PST, GSR and heart rate are measured and processed according to the present invention as described above and illustrated inFIGS. 11 & 12. After two minutes of “initialization”, Z values are calculated for each physiological variable on a second to second basis and relayed to the software.

The introduction ends and the player starts off on his journey through a fantasy world of dragons and sorcerers. Since, the Z values are continuously being updated and relayed to the software, the software programmer can utilize the ability of the player to alter their physiology to change outcomes of the game. For example, as illustrated inFIG. 13, if the Z values for PST becomes greater than one (temperature change in the hand greater than one standard deviation over running mean), the software programmer may allow the player the ability to shoot fireballs. If the values for Z are −1<Z<1, the programmer may allow the player to have no change in game play. If Z0.5, there is no change in game play. If −0.5<Z<0.5 for HR, the software programmer may allow a force field strength of 25%. Whereas if −1<Z<−0.5, the software programmer may allow a 50% force field strength. If −1.5<Z<−1, the software programmer may allow a 75% force field strength; and if Z<−1.5, the software programmer may allow a 100% force field strength. One skilled in the art could conceive of a program or routine designed specifically to create a continuum of effects over a range of Z values.

Another exemplary embodiment of the present invention comprises a hand-held video gaming system as depicted inFIGS. 14 and 15.FIG. 14depicts a front view of the hand-held video gaming system. This configuration is well known to one of ordinary skill in the art. A directional keypad (520) is located adjacent function buttons A&B (522&524). In addition, a select game button (526) and a start button (528) are depicted inFIG. 14.FIG. 15depicts the back view of the hand-held video gaming system. A game port (530) can be visualized on the back of the hand-held gaming system. Located below the game port are three finger wells (550,552and554) that are configured to ensure contact between the 2nd, 3rdand 4thdigits of the game player's hand and the sensors during play. The three basic sensors for monitoring physiological variables are located within the wells. An LED/photodiode (540) unit is depicted in the first well. The LED/photodiode (540) is able to monitor blood volume pulse, peripheral pulse and respiration as previously cited in the art. A silver four electrode unit (542) is illustrated in the second well. This unit is able to monitor GSR and a one-lead ECG. The third well comprises a thermocouple (544). The thermocouple is able to monitor peripheral skin temperature. A compartment (560) allows utilization of a portable power source such as batteries.

An exemplary embodiment of the circuit board is illustrated inFIGS. 16 and 17. A video display (510) is attached to the top of the circuit board as shown inFIG. 16. The physiological data monitored by the sensors located in the finger wells is relayed unprocessed back to the circuit board (590). At this point the physiological data undergoes multi-level processing before being sent, in processed form, to the software via the software-circuit board interface (570). In one exemplary embodiment, the multilevel processing is the same as described above for the video gaming system of the present invention. Once delivered to the software, the data is stored so that it can later be accessed. The data resulting from the manual manipulation of the controller is also relayed back to the circuit board (610). This data undergoes its own unique multi-level processing before it is relayed to the software-circuit board interface (570). Once delivered to the software, this data is also stored for later use. Both types of processed data, physiological and manual, are stored and accessed by the software and incorporated into the video game program in real time and the output is sent back to the circuit board where it is further processed and then delivered to a video display.

The examples of specific embodiments set forth herein are for illustrative purposes only and are not intended to limit the scope of the methods and fabrics of the invention. Additional methods and fabric within the scope of the claimed invention will be apparent of one skilled in the art in view of the teachings set forth herein.