U.S. Pat. No. 9,101,831

DETAILED DESCRIPTION

FIG. 1is an example of a video game system in accordance with aspects of the invention. The video game system includes a game console111with a processor and associated circuitry for executing instructions controlling game play. The instructions controlling game play are generally stored on removable media, for example, an optical disk. Accordingly, the game console may include an optical drive, for example, a DVD-ROM drive, for reading the instructions for game play. In some embodiments, the game console may be a handheld gaming device, including similar internal circuitry as herein described, as well as, for example, a built in display or displays and various different user input devices, and in some embodiments a personal computer or the like may be used instead of a game console.

A video game controller113is also included in the video game system. The video game controller, in one embodiment, has a board-like shape and resembles a skateboard or, in some embodiment, a snowboard. The video game controller generally includes processing circuitry, which may be a processor119or other processing circuitry. The processor may execute instructions according to firmware that is also stored in the video game controller. A wireless transmitter121, or in some embodiments a wired transmitter, included in the video game controller sends controller information to the game console. The wireless transmitter may send information as controlled by the processor. The video game controller includes position related sensors. The position related sensors may provide position information, or information related to position or change in position, such as information related to motion or acceleration. In some embodiments, and as inFIG. 1, the position related sensors are accelerometers117a,117b. The accelerometers may be three-axis accelerometers. The processor may process and combine signals from the accelerometers to determine at least some controller information to be sent to the game console. In some embodiments, other types of position related sensors, for example, geomagnetic sensors, are included in the video game controller.

Proximity sensors115a-dare also included in the video game controller. The embodiment illustrated inFIG. 1includes four proximity sensors, a front sensor, a rear sensor, a left sensor, and a right sensor. Each of the proximity sensors senses when an object is near the sensor. The object may be, for example, a game player's hand or foot. The proximity sensors may be, for example, infrared sensors, ultrasound sensors, or sonar sensors. The processor may process signals from the proximity sensors to determine at least some controller information to be sent to the game console. In some embodiments, the proximity sensors distinguish when an object is sufficiently close to the proximity sensor to be considered in contact with the sensor, near the sensor, or more distant from the sensor. In some embodiments the proximity sensors are infrared sensors, in other embodiments the proximity sensors are ultrasonic sensors, and in other embodiments the proximity sensors are other proximity sensors.

A display device123is also included in the video game system. The display device is generally coupled to the video game console by a cable, although in some embodiments a wireless connection may be used. In many embodiments, the display device is a television. The display device, in the illustrated embodiment, includes a speaker133and commonly two or more speakers. A display screen131of the display device displays video images of game play. In the embodiment ofFIG. 1, the display screen shows a screen shot of video game play of a skateboarding video game. As illustrated, the screen shot shows a front view of a skateboard ridden by a skateboarder in a cityscape.

The game player generally stands on a flat portion of the upper surface of the video game controller during game play. At some times during game play, the game player may stand or place weight on the upturned ends of the video game controller, or place or move feet about sides of the video game controller. Motion (and/or position) of the video game controller, as indicated for example by the accelerometers, is used, for example by the processor of the game console, to determine a game character's actions and/or movement (and/or position) of a board associated with the game character in a game, and the actions and/or movement (and/or position) of the board are generally displayed on the display, for example as commanded by the processor of the game console. Thus the processor may be considered as commanding display of the game character operating the board. In some instances the character may be navigating the board in a virtual game environment by making turns with the board, riding the board forward (or backwards), stopping the board, or otherwise generally performing board navigation operations by translating the board through space of the virtual game environment. In other instances, the character may be performing tricks with the board, namely performing board trick operations, for example by rotating, flipping, ollieing, performing manuals, or grabbing the board, while also navigating the board in various embodiments. Thus, the game character may perform board navigation operations or board trick operations, or both.

For example, in a skateboarding video game when the video game controller is tilted to the left as indicated by the accelerometers, a skateboard in the video game may be displayed as turning, or navigating, to the left in a virtual game environment. Additionally, game actions may be triggered when a game player touches or moves near the video game controller, as indicated by the proximity sensors. In some embodiments the game character performs different tricks based upon a combination of current proximity sensor output values and previous or current accelerometer output values, and the tricks may also depend on a range of proximity as indicated by the proximity sensor. Further, depending on game play state, different combinations of proximity sensor output values and accelerometer output values may result in different game character tricks. For example, a proximity sensor output value indicating an object close to a front of a board while an accelerometer output value indicates a raised front of the board may result in a game character first trick in a first game state and a second trick in a second game state. And a proximity sensor output value indicating an object further from the front of the board in otherwise similar circumstances may result in a third trick and a fourth trick, respectively.

FIG. 2is an example of a block diagram of a game console processing unit in accordance with aspects of the invention. The game console includes a processor211that is connected to other components of the game console via a bus. The other components include a main memory213and a removable memory device215. The processor may execute instructions from the removable memory device215to control game play and store game state information in the main memory. For example, the game play may be for a skateboarder with the instructions controlling possible movements, positions, and locations of a game character and/or skateboard associated with or operated by the game character.

The processor is coupled to an audio driver221and a video driver223. The audio driver produces sound signals and the video driver produces image signals. The sound signals and image signals are transmitter from the game console via a display I/O device225. The display I/O device generally supplies the sound and image signals to a display device external to the game console.

The game console may also include a user I/O device217, a wireless transceiver219, an Internet I/O device227, and other circuitry229. The wireless transceiver may receive control signals from a video game controller that signals actions of a game player. Alternatively or additionally, the game console may receive game player actions via the user I/O device. The Internet I/O device provides a communication channel that may be used, for example, for multiple player games. The other circuitry may include status indicators, such as colored lights.

FIG. 3is a perspective view of a peripheral in accordance with aspects of the invention. The peripheral serves as a video game controller. The peripheral is generally in the shape of a deck of a skateboard. In other embodiments the peripheral is generally in the shape of a snowboard. In many embodiments, the peripheral is somewhat shorter longitudinally than many skateboards, and the peripheral may have a length, for example, of 70 centimeters. The peripheral includes an upper surface711and a lower surface713coupled by sidewalls723. The peripheral includes electronic circuitry, for example, a processor and a transceiver for communication with a video game console. The upper surface includes a substantially flat central portion longitudinally bounded by slightly upturned ends. In operation, feet of a game player are generally located on or about the upper surface, including on or about the flat central portion, on or about either or both upturned ends, or at times to either side of the peripheral.

In some embodiments four proximity sensors, in many embodiments infrared sensors, are located at edges of the peripheral, with a front proximity sensor715at a forward edge of the peripheral, a rear proximity sensor717at a rear edge of the peripheral, a left proximity sensor717at a midpoint of a left side of the peripheral, and a right proximity sensor719at a midpoint of a right side of the peripheral. As illustrated inFIG. 3, covers are provided for the sensors, with the covers generally increasing smoothness of the surface of the peripheral, however, in other embodiments no covers may be provided. The proximity sensor may indicate presence of an object within defined distances of the sensor, with the distances depending on the particular sensor, mounting of the sensor, and related considerations.

FIG. 4is a diagram illustrating an example of proximity sensor ranges of distances with respect to a board in accordance with aspects of the invention. A proximity sensor405is located on an edge of a video game controller403. The proximity sensor, or circuitry associated with the proximity sensor (which in many cases may be considered to be part of the proximity sensor), measures whether an object is near the sensor and when an object is near, measures the distance from the proximity sensor to the object. The object may be a game player's hand or foot. In some embodiments, the proximity sensor, or additional circuitry, quantizes the measured distance into a limited number of discrete ranges of distances, with the limited number of discrete ranges of distances generally being less than the number of potential ranges determinable by the proximity sensor. For example, as illustrated inFIG. 4, the proximity sensor measures whether an object is within a first range R1, within a second range R2, within a third range R3, or more distant. The first range may extend approximately five centimeters from the proximity sensor. This range may indicate, for example, that a game player's hand is contacting the video game controller. The second range may extend approximately from five to twenty centimeters from the proximity sensor. This range may indicate, for example, that a game player's foot is passing the video game controller in a pushing motion. The third range may extend approximately from twenty-five to fifty centimeters from the proximity sensor. This range may indicate, for example, that a game player's hand is grabbing or extending toward the video game controller.

In one embodiment, the proximity sensor utilizes infrared light. An infrared transmitter sends light from the proximity sensor, and an infrared receiver senses light reflected from an object back to the proximity sensor. The light sensed by the infrared receiver varies with the location of the reflecting object. The proximity sensor may use, for example, the angle of the light, the intensity of the light, or the time from transmission to reception to measure the distance from the proximity sensor to the reflecting object. The infrared receiver supplies a signal with a strength that varies with the distance to the reflecting object. In an embodiment where the proximity sensor quantizes the measured distance, thresholds are applied to the signal from the infrared receiver.

FIG. 5is a flowchart of a process for measuring proximity in accordance with aspects of the invention. In some embodiments, the process is performed by a video game controller, by a video game console, or partially by a video game controller and partially by video game console. Additionally, the process may be performed by hardware, software, or a combination of hardware and software. The process utilizes a proximity sensor, for example one of the proximity sensors of the embodiment ofFIG. 1. In some embodiments the proximity sensors are infrared sensors.

In block511, the process activates the proximity sensor. In some embodiments, activating the sensor causes continuous emission of infrared light from the proximity sensor. In other embodiments, an activated sensor may emit pulses of infrared light. In various embodiments pulses of infrared light may be timed with respect to pulses of light of other proximity sensors, for example to reduce the possibility of interference or cross-talk between sensors. Activation of the proximity sensor may include supplying electrical power to the proximity sensor.

In block513, the process receives a return signal. The return signal is a reflection, when an object is near the proximity sensor, of the light emitted when the proximity sensor was activated in block511. In one embodiment, the process receives the return signal through a lens and an array of photo sensors to produce a signal indicative of the angle of the return. When the light is reflected from a more distant object, a more acute angle of return results. In another embodiment, the process receives the return signal with a photo sensor to produce a signal indicative of the intensity of the return. When the light is reflected from a more distant object, less intense light is returned.

In block515, the process determines if an object is near the proximity sensor and when an object is near, a distance to the object. The process utilizes the return signal received in block513to determine an indication of the distance from the proximity sensor to an object. In one embodiment, the process applies thresholds to the signal from block513and the determined distance is quantized into a few ranges, for example, fewer than three ranges, three ranges, or four ranges. The process thereafter returns.

FIG. 6is a flowchart of a process for processing proximity sensor inputs and position information in accordance with aspects of the invention. In some embodiments, the process is performed by a video game controller, for example circuitry of the video game controller, by a video game console, for example a processor of the video game console, or partially by a video game controller and partially by a video game console. In some embodiments, the process utilizes a video game controller, such as the board-like controller of the embodiment ofFIG. 1.

In block613, the process determines position of the video game controller. The process may acquire accelerometer data to utilize in determining the position of the video game controller. In one embodiment, the process reads data from accelerometers in the video game controller. Preferably, the accelerometers are three-axis accelerometers and each accelerometer provides three measurements. Each measurement indicates the acceleration of the video game controller at the location of the accelerometer. The process determines the position of the video game controller from the accelerometer measurements via calculations using Newtonian physics equations. In some embodiments, the process may optionally or additionally receive measurements from a geomagnetic sensor and utilize geomagnetic measurements to determine the position of the video game controller.

In block615, the process processes proximity sensor data. In some embodiments, the proximity sensor data is processed using the process ofFIG. 5. Additionally, the process may process proximity sensor data for a plurality of sensors, for example, the four proximity sensors of the embodiment ofFIG. 1.

In block617, the process determines a trick. A trick triggers a game action, for example, flip move by a skateboarder. The process determines a trick from the position determined in block613and the proximity data processed in block615. For example, a Benihana trick may be determined when the processed proximity sensor data includes near proximity to a rear sensor and the determined position includes that the rear of the game controller is lifted to a high position. In some embodiments, determining a trick may additionally include the current state of the video game. For example, a Fingerflip Left trick may be determined when the processed proximity sensor data includes close proximity to a front sensor, the determined position includes that the game controller is tilted to the left, and the previous state of the game was a Nose move. A trick may additionally be determined by a sequence of measurements. For example, a momentary detection of proximity to a left sensor at a middle range may be determined to be a Pushing trick.

In block619, the process determines whether a trick was determined in block617. If a trick was determined, the process continues to block621; otherwise, the process returns.

In block621, the process commands display of the trick determined in block617. For example, when a Benihana trick is detected the process commands a display showing a skateboarder performing a corresponding trick. The trick may be displayed on video display, for example, the display unit ofFIG. 1. The process thereafter returns.

FIG. 7is a further flowchart of a process for processing proximity sensor inputs and motion information in accordance with aspects of the invention. In some embodiments, the process is performed by a video game controller, by a video game console, or partially by a video game controller and partially by a video game console. In some embodiments, the process utilizes a video game controller, such as the board-like controller of the embodiment ofFIG. 1.

In block713, the process determines motion of the video game controller. The process may acquire accelerometer data to utilize in determining the motion of the video game controller. In one embodiment, the process reads data from accelerometers in the video game controller. Preferably, the accelerometers are three-axis accelerometers and each accelerometer provides three measurements. Each measurement indicates the acceleration of the video game controller at the location of the video game controller. The process determines the motion of the video game controller from the accelerometer measurements via Newtonian physics equations. In some embodiments, the process may optionally or additionally receive measurements from a geomagnetic sensor and utilize geomagnetic measurements to determine the motion of the video game controller. The determined motion may include pitch, roll, and yaw data. Additionally, the determined motion may include time derivatives of the pitch, roll, and yaw data.

In block715, the process processes proximity sensor data. In some embodiments, block715is similar to or the same as block615ofFIG. 6.

In block717, the process determines a trick. A trick triggers a game action in a manner similar to block617ofFIG. 6. The process determines a trick from the motion determined in block713and the proximity data processed in block715. For example, a 360 Kickflip trick may be determined when the processed proximity sensor data includes near proximity to a left sensor and the determined motion includes yaw. In some embodiments, determining a trick may additionally include the current state of the video game. A trick may additionally be determined by a sequence of measurements.

In some embodiments different tricks are determined for different combinations of motion and proximity data. For example, as illustrated in Table I below, a yaw board movement may result in a Shove It trick if a rear proximity sensor indicates an object very close to the rear proximity sensor, for example indicating a foot on a portion of the video game controller about the rear proximity sensor, while a yaw board movement may result in a board slide navigation operation (which may be used to stop motion of the board in a game virtual environment, for example) if the rear proximity sensor indicates no object within range of the rear proximity sensor. Similarly, a roll board movement may result in a steering navigation operation, a flip trick, or a grab trick depending on whether the rear proximity sensor indicates no object within range, an object very close, or a distant object respectively.

TABLE IBoard MovementSensorResultYawNoneSlide (navigation)YawRear Very CloseShove It TrickRoll (Tilt)NoneSteer (navigation)Roll (Tilt)Rear Very CloseFlip TrickRoll (Tilt)Rear DistantGrab Trick

In block719, the process determines whether a trick was determined in block717. If a trick was determined, the process continues to block721; otherwise, the process returns.

In block721, the process commands display of the trick determined in block717. The trick may be displayed on video display, for example, the display unit ofFIG. 1. The process thereafter returns.

Although the invention has been described with respect to certain specific embodiments, it should be recognized that the invention comprises the novel and unobvious claims supported by this disclosure, along with their insubstantial variations.