U.S. Pat. No. 9,539,511

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS

First Embodiment

With reference toFIG. 1, an information processing apparatus for executing an information processing program according to a first embodiment and an information processing system including the information processing apparatus will be described. Hereinafter, in order to provide a specific description, a stationary game apparatus body5is used as an example of the information processing apparatus, and a game system including the game apparatus body5will be described.FIG. 1is an external view illustrating an example of the game system1including the stationary game apparatus body5.FIG. 2is a block diagram illustrating an example of the game apparatus body5. Hereinafter, the game system1will be described.

As shown inFIG. 1, the game system1includes a household television receiver (hereinafter, referred to as a monitor)2which is an example of display means, and the stationary game apparatus3connected to the monitor2via a connection cord. The monitor2includes loudspeakers2afor outputting, in the form of sound, a sound signal outputted from the game apparatus3. Further, the game apparatus3includes: an optical disc4having stored therein a program (e.g., a game program); the game apparatus body5having a computer for executing the program stored in the optical disc4to display a game screen on the monitor2; a terminal device6; a controller7for providing the game apparatus body5with operation information required for operating, for example, objects displayed on the display screen; and a board type controller9. The game system1executes a game process on the game apparatus body5in accordance with a game operation using at least one of the terminal device6, the controller7, and the board type controller9, and displays a game image obtained by the game process on the monitor2and/or the terminal device6. The game apparatus body5is wirelessly connected to the terminal device6, the controller7, and the board type controller9so as to enable wireless communication therebetween. For example, the wireless communication is executed according to the Bluetooth (registered trademark) standard or the IEEE802.11n standard. However, the wireless communication may be executed in accordance with other standards such as standards for infrared communication.

The optical disc4, typifying an information storage medium used for the game apparatus body5in an exchangeable manner, is detachably inserted in the game apparatus body5. The optical disc4has stored therein an information processing program (typically, a game program) to be executed by the game apparatus body5. The game apparatus body5has, on a front surface thereof, an insertion opening for the optical disc4. The game apparatus body5reads and executes the information processing program stored in the optical disc4inserted into the insertion opening to execute the game process.

The monitor2is connected to the game apparatus body5via a connection cord. The monitor2displays a game image obtained by the game process executed by the game apparatus body5. The monitor2includes the loudspeakers2a. The loudspeakers2aeach output a game sound obtained as a result of the game process. In another embodiment, the game apparatus body5and a stationary display unit may be integrated with each other. The communication between the game apparatus body5and the monitor2may be wireless communication.

The game apparatus body5has mounted thereto a flash memory17(seeFIG. 2) which functions as a backup memory for fixedly storing data such as saved data. The game apparatus body5executes the game program or the like stored in the optical disc4, and displays a result thereof as a game image on the monitor2and/or the terminal device6. The game program or the like to be executed may be previously stored in the flash memory17as well as in the optical disc4. Further, the game apparatus body5may reproduce a state of a game played in the past, by using the saved data stored in the flash memory17, and display an image of the game state on the monitor2and/or the terminal device6. A user of the game apparatus3can enjoy the game progress by operating at least one of the terminal device6, the controller7, and the board type controller9while viewing the game image displayed on the monitor2and/or the terminal device6.

The controller7and the board type controller9each wirelessly transmit transmission data such as operation information, by using, for example, the Bluetooth technology, to the game apparatus body5having a controller communication module19. The controller7is operation means for performing, for example, selection of options displayed on the display screen of the monitor2. The controller7includes a housing which is small enough to be held by one hand, and a plurality of operation buttons (including a cross key and the like) which are exposed at the surface of the housing. In addition, as is described later, the controller7includes an imaging information calculation section for taking an image viewed from the controller7. As exemplary imaging targets of the imaging information calculation section, two LED modules (hereinafter, referred to as “markers”)8L and8R are provided in the vicinity of the display screen of the monitor2(above the screen inFIG. 1). Although details will be described later, a user (player) is allowed to perform a game operation while moving the controller7, and the game apparatus body5uses a marker8to calculate the movement, position, attitude and the like of the controller7. The marker8has two markers8L and8R at both ends thereof. Specifically, the marker8L (as well as the marker8R) includes one or more infrared LEDs (Light Emitting Diodes), and emits infrared light forward from the monitor2. The marker8is connected to the game apparatus body5, so that the game apparatus body5can control the infrared LEDs included in the marker8to be lit on or off. The marker8is a portable unit, so that the user is allowed to place the marker8in any position. AlthoughFIG. 1shows a case where the marker8is placed on the monitor2, the location and direction of the marker8may be optionally selected. Further, the controller7is capable of receiving, at a communication section75, transmission data wirelessly transmitted from the controller communication module19of the game apparatus body5, to generate a sound or vibration based on the transmission data.

In another embodiment, the controller7and/or the board type controller9may be connected to the game apparatus body5via a cable. Further, in the exemplary embodiment, the game system1includes a controller7and a board type controller9. However, the game apparatus body5is communicable with a plurality of controllers7and a plurality of board type controllers9. Therefore, a plurality of players can play a game by using a predetermined number of controllers7and board type controller9simultaneously.

The controller7includes a housing which is formed by, for example, plastic molding, and has a plurality of operation sections in the housing. The controller7transmits, to the game apparatus body5, operation data representing an input state to any of the operation sections (i.e., whether or not any of the operation buttons is pressed).

The controller7includes an imaging information calculation section for analyzing image data of an image taken by image pickup means to identify an area having a high luminance in the image, and calculates the position of the center of gravity, the size, and the like of the area. For example, the imaging information calculation section includes image pickup means fixed to the housing of the controller7. A marker emitting infrared light, such as a marker section65of the terminal device6and/or the marker8, is a target to be imaged by the image pickup means. The imaging information calculation section calculates the position of the imaging target in the image taken by the image pickup means, and transmits, to the game apparatus body5, marker coordinate data representing the calculated position. Since the marker coordinate data varies depending on the direction (angle of tilt) or the position of the controller7, the game apparatus body5can calculate the direction and the position of the controller7by using the marker coordinate data.

The controller7includes an acceleration sensor and/or a gyro sensor. The acceleration sensor detects an acceleration (including gravitational acceleration) of the controller7, and transmits acceleration data representing the detected acceleration to the game apparatus body5. Since the acceleration detected by the acceleration sensor varies depending on the direction (angle of tilt) and the movement of the controller7, the game apparatus body5can calculate the direction and the movement of the controller7by using the obtained acceleration data. The gyro sensor detects angular velocities around three axes which are set in the controller7, and transmits angular velocity data representing the detected angular velocities to the game apparatus body5. Since the angular velocities detected by the gyro sensor vary depending on the direction (angle of tilt) and the movement of the controller7, the game apparatus body5can calculate the direction and the movement of the controller7by using the obtained angular velocity data. In this way, the user is allowed to perform a game operation by pressing the operation sections provided on the controller7, or by moving the controller7to change the position and the attitude (angle of tilt) of the controller7.

The controller7is provided with a loudspeaker and a vibrator. The controller7processes sound data transmitted from the game apparatus body5, and causes the loudspeaker to output a sound based on the sound data. Further, the controller7processes vibration data transmitted from the game apparatus body5, and causes the vibrator to vibrate in accordance with the vibration data. In the exemplary embodiment described later, a user is allowed to play a game without using the controller7. The board type controller9will be described in detail later.

The terminal device6is a portable device that is small enough to be held by a user, and the user is allowed to move the terminal device6with hands, or place the terminal device6at any location. Although a specific structure of the terminal device6will be described later, the terminal device6includes an LCD (Liquid Crystal Display)61as display means, and input means (a touch panel62, a gyro sensor604, and the like described later). The terminal device6and the game apparatus body5(a terminal communication module28(seeFIG. 2)) are communicable with each other wirelessly or via a cable. The terminal device6receives, from the game apparatus body5, data of an image (e.g., a game image) generated in the game apparatus body5, and displays the image represented by the data on an LCD61. Although in the exemplary embodiment the LCD61is used as a display device, the terminal device6may include any other display device, such as a display device utilizing EL (Electro Luminescence), for example. Further, the terminal device6transmits, to the game apparatus body5having the terminal communication module28, operation data representing the content of an operation performed on the terminal device6.

Next, with reference toFIG. 2, an internal structure of the game apparatus body5will be described.FIG. 2is a block diagram illustrating an example of an internal structure of the game apparatus body5. The game apparatus body5includes a CPU (Central Processing Unit)10, a system LSI (Large Scale Integration)11, an external main memory12, a ROM/RTC (Read Only Memory/Real Time Clock)13, a disc drive14, an AV-IC (Audio Video-Integrated Circuit)15and the like.

The CPU10, serving as a game processor, executes a program stored in the optical disc4to perform a process. The CPU10is connected to the system LSI11. In addition to the CPU10, the external main memory12, the ROM/RTC13, the disc drive14, and the AV-IC15are connected to the system LSI11. The system LSI11performs processes such as control of data transmission between the respective components connected thereto, generation of an image to be displayed, and acquisition of data from an external apparatus. An internal structure of the system LSI11will be described later. The external main memory12, which is a volatile memory, stores programs loaded from the optical disc4or the flash memory17, and stores various data. The external main memory12is used as a work area and a buffer area for the CPU10. The ROM/RTC13includes a ROM (so-called boot ROM) incorporating a program for booting the game apparatus body5, and a clock circuit (RTC) for counting time. The disc drive14reads, from the optical disc4, program data, texture data and the like, and writes the read data into an internal main memory35described below or the external main memory12.

The system LSI11includes an input/output processor (I/O processor)31, a GPU (Graphics Processor Unit)32, a DSP (Digital Signal Processor)33, a VRAM (Video RAM)34, and the internal main memory35. These components31to35are connected to each other via an internal bus (not shown).

The GPU32, which is a part of rendering means, generates an image in accordance with a graphics command (draw command) supplied from the CPU10. The VRAM34stores data (such as polygon data and texture data) required for the GPU32to execute the graphics command. When an image is generated, the GPU32generates image data by using the data stored in the VRAM3. In the exemplary embodiment, the game apparatus body5may generate both a game image to be displayed on the monitor2and a game image to be displayed on the terminal device6. Hereinafter, the game image to be displayed on the monitor2may be referred to as a “monitor game image”, and the game image to be displayed on the terminal device6may be referred to as a “terminal game image”.

The DSP33, serving as an audio processor, generates sound data by using sound data and sound waveform (tone quality) data stored in the internal main memory35and the external main memory12. In the exemplary embodiment, similarly to the game images, both a game sound to be output from the loudspeakers2aof the monitor2and a game sound to be output from the loudspeakers of the terminal device6may be generated. Hereinafter, the game sound to be output from the monitor2may be referred to as a “monitor game sound”, and the game sound to be output from the terminal device6may be referred to as a “terminal game sound”.

Among the image data and sound data generated by the game apparatus body5, the image data and sound data to be output to the monitor2are read by the AV-IC15. The AV-IC15outputs the read image data to the monitor2via an AV connector16, and outputs the read sound data to the loudspeakers2aincluded in the monitor2. Thereby, an image is displayed on the monitor2, and a sound is output from the loudspeakers2a.

Further, among the image data and sound data generated by the game apparatus body5, the image data and sound data to be output to the terminal device6are transmitted to the terminal device6by the I/O processor31or the like. Data transmission to the terminal device6by the I/O processor31or the like will be described later.

The I/O processor31executes data reception and transmission with the components connected thereto, and download of data from an external apparatus. The I/O processor31is connected to the flash memory17, the network communication module18, the controller communication module19, an extension connector20, a memory card connector21, and a codec LSI27. An antenna23is connected to the controller communication module19. The codec LSI27is connected to the terminal communication module28, and an antenna29is connected to the terminal communication module28.

The game apparatus body5is connected to a network such as the Internet so as to communicate with external information processing apparatuses (for example, other game apparatuses or various servers). That is, the I/O processor31is connected to a network via the network communication module18and the antenna22so as to communicate with external information processing apparatuses connected to the network. The I/O processor31accesses the flash memory17at regular intervals so as to detect for data to be transmitted to the network. When data to be transmitted is detected, the data is transmitted to the network via the network communication module18and the antenna22. Further, the I/O processor31receives, via the network, the antenna22and the network communication module18, data transmitted from the external information processing apparatuses or data downloaded from a download server, and stores the received data in the flash memory17. The CPU10executes a program, and reads the data stored in the flash memory17to use the data for execution of the program. The flash memory17may store not only the data transmitted and received between the game apparatus body5and the external information processing apparatuses, but also saved data (result data or progress data of the process) of the game played with the game apparatus body5. Further, the flash memory17may store programs such as a game program.

The game apparatus body5can receive operation data from the controller7and/or the board type controller9. That is, the I/O processor31receives, via the antenna23and the controller communication module19, operation data or the like transmitted from the controller7and/or the board type controller9, and stores (temporarily) the data in a buffer region of the internal main memory35or the external main memory12. Similarly to the external main memory12, the internal main memory35may store a program loaded from the optical disc4or a program loaded from the flash memory17, and various data. The internal main memory35may be used as a work region or buffer region of the CPU10.

The game apparatus body5is capable of transmitting/receiving image data, sound data and the like to/from the terminal device6. When transmitting a game image (terminal game image) to the terminal device6, the I/O processor31outputs data of a game image generated by the GPU32to the codec LSI27. The codec LSI27performs a predetermined compression process on the image data supplied from the I/O processor31. The terminal communication module28performs wireless communication with the terminal device6. Accordingly, the image data compressed by the codec LSI27is transmitted by the terminal communication module28to the terminal device6via the antenna29. In the exemplary embodiment, the codec LSI27compresses the image data by using a highly efficient compression technique, for example, the H.264 standard. The codec LSI27may adopt other compression techniques. When the communication rate is sufficiently high, uncompressed image data may be transmitted. The terminal communication module28is, for example, a Wi-Fi certified communication module. The terminal communication module28may perform wireless communication with the terminal device6at a high speed by using, for example, the technique of MIMO (Multiple Input Multiple Output) adopted in the IEEE802.11n standard, or may use other communication techniques.

The game apparatus body5transmits, to the terminal device6, sound data as well as the image data. That is, the I/O processor31outputs sound data generated by the DSP33to the terminal communication module28via the codec LSI27. The codec LSI27performs a compression process on the sound data in a similar manner to that for the image data. Any compression technique may be adopted for the sound data. In another embodiment, uncompressed sound data may be transmitted. The terminal communication module28transmits the compressed image data and sound data to the terminal device6via the antenna29.

The game apparatus body5transmits, in addition to the image data and sound data, various control data to the terminal device6, according to need. The control data represent control instructions for the components included in the terminal device6, such as an instruction to control on/off of a marker section (a marker section65shown inFIG. 5), and an instruction to control image taking of a camera (a camera66shown inFIG. 5). The I/O processor31transmits the control data to the terminal device6in response to an instruction from the CPU10. In the exemplary embodiment, the codec LSI27does not perform a data compression process on the control data. In another embodiment, however, the codec LSI27may perform a compression process on the control data. The above-described data transmitted from the game apparatus body5to the terminal device6may be encrypted according to need, or may not be encrypted.

The game apparatus body5can receive various data from the terminal device6. Although details will be described later, in the exemplary embodiment, the terminal device6transmits operation data, image data, and sound data. The respective data transmitted from the terminal device6are received by the terminal communication module28via the antenna29. The image data and sound data transmitted from the terminal device6have been subjected to a similar compression process to that for the image data and sound data transmitted from the game apparatus body5to the terminal device6. Accordingly, these image data and sound data are transmitted from the terminal communication module28to the codec LSI27, and subjected to a decompression process by the codec LSI27. The decompressed data are output to the I/O processor31. On the other hand, since the operation data transmitted from the terminal device6is smaller in amount than the image data and sound data, the operation data need not be compressed. The operation data may be encrypted according to need, or may not be encrypted. Accordingly, the operation data, which has been received by the terminal communication module28, is output to the I/O processor31via the codec LSI27. The I/O processor31stores (temporarily) the data received from the terminal device6in the buffer region of the internal main memory35or the external main memory12.

The game apparatus body5is connectable to other devices and external storage media. That is, an extension connector20and a memory card connector21are connected to the I/O processor31. The expansion connector20is an interface connector as typified by a USB and an SCSI, and is capable of performing communication with the network, instead of the network communication module18, by connecting thereto a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. The memory card connector21is a connector for connecting thereto an external storage medium such as a memory card. For example, the I/O processor31accesses the external storage medium via the expansion connector20or the memory card connector21to save or read data.

The game apparatus body5includes (on the front main surface thereof, for example) a power button24, a reset button25, an insertion slot in which the optical disc4is inserted, an eject button26for ejecting the optical disc4from the insertion slot of the game apparatus body5, and the like. The power button24and the reset button25are connected to the system LSI11. When the power button24is turned on, the respective components of the game apparatus body5are supplied with power. When the reset button25is pressed, the system LSI11re-executes the boot program of the game apparatus body5. The eject button26is connected to the disc drive14. When the eject button26is pressed, the optical disc4is ejected from the disc drive14.

In another embodiment, some of the components of the game apparatus body5may be constituted as an extension device separated from the game apparatus body5. At this time, the extension device may be connected to the game apparatus body5via the extension connector20. Specifically, the extension device may include, for example, the codec LSI27, the terminal communication module28, and the antenna29, and may be detachably connected to the extension connector20. Thus, by connecting the extension device to the game apparatus body which does not have the above-mentioned, the game apparatus body can be made communicable with the terminal device6.

Next, a structure of the terminal device6will be described with reference toFIGS. 3 to 5.FIG. 3is a diagram illustrating an example of an external structure of the terminal device6. More specifically, (a) ofFIG. 3is a front view of the terminal device6, (b) ofFIG. 3is a top view, (c) ofFIG. 3is a right side view, and (d) ofFIG. 3is a bottom view.FIG. 4shows an example of a state in which a user holds the terminal device6with both hands.

As shown inFIG. 3, the terminal device6includes a housing60which generally has a horizontally long plate-like rectangular shape. The housing60is small enough to be held by the user. Therefore, the user is allowed to move the terminal device6with hands, and change the location of the terminal device6.

The terminal device6includes an LCD61on a front surface of the housing60. The LCD61is provided near the center of the front surface of the housing60. Therefore, as shown inFIG. 4, the user, holding the housing60at portions to the right and left of the LCD61, is allowed to move the terminal device6while viewing a screen of the LCD61.FIG. 4shows an example in which the user holds the terminal device6horizontally (i.e., with the longer sides of the terminal device6being oriented horizontally) by holding the housing60at portions to the right and left of the LCD61. However, the user may hold the terminal device6vertically (i.e., with the longer sides of the terminal device6being oriented vertically).

As shown in (a) ofFIG. 3, the terminal device6includes, as operation means, a touch panel62on the screen of the LCD61. In the exemplary embodiment, the touch panel62is, but is not limited to, a resistive film type touch panel. However, a touch panel of any type, such as electrostatic capacitance type, may be used. The touch panel62may be of single touch type or multiple touch type. In the exemplary embodiment, the touch panel62has the same resolution (detection accuracy) as that of the LCD61. However, the resolution of the touch panel62and the resolution of the LCD61need not be the same. Although an input onto the touch panel62is usually performed by using a touch pen, in addition to the touch pen, a finger of the user may be used for performing an input onto the touch panel62. The housing60may have an opening for accommodating the touch pen used for performing an operation to the touch panel62. Since the terminal device6has the touch panel62, the user is allowed to operate the touch panel62while moving the terminal device6. That is, the user is allowed to directly (by using the touch panel62) perform an input onto the screen of the LCD61while moving the LCD61.

As shown inFIG. 3, the terminal device6has, as operation means, two analog sticks63A and63B, and a plurality of operation buttons64A to64L. The analog sticks63A and63B are each a device for designating a direction. The analog sticks63A and63B are each configured such that a stick part thereof to be operated by a finger of the user is slidable or tiltable in any direction (at any angle in any direction such as the upward, the downward, the rightward, the leftward, or the diagonal direction) with respect to the front surface of the housing60. The left analog stick63A is provided to the left of the screen of the LCD61, and the right analog stick63B is provided to the right of the screen of the LCD61. Therefore, the user is allowed to perform an input for designating a direction by using the analog stick63A or63B with either the left or right hand. Further, as shown inFIG. 4, the analog sticks63A and63B are positioned so as to be operated by the user holding the right and left portions of the terminal device6. Therefore, the user is allowed to easily operate the analog sticks63A and63B when the user holds and moves the terminal device6.

The operation buttons64A to64L are each operation means for performing a predetermined input. As described below, the operation buttons64A to64L are positioned so as to be operated by the user holding the right and left portions of the terminal device6(seeFIG. 4). Accordingly, the user is allowed to easily operate the operation means when the user holds and moves the terminal device6.

As shown in (a) ofFIG. 3, among the operation buttons64A to64L, the cross button (direction input button)64A and the operation buttons64B to64H are provided on the front surface of the housing60. The operation buttons64A to64H are positioned so as to be operated by a thumb of the user (seeFIG. 4).

The cross button64A is provided to the left of the LCD61and beneath the left analog stick63A. That is, the cross button64A is positioned so as to be operated by the left hand of the user. The cross button64A is cross-shaped, and is capable of indicating an upward, a downward, a leftward, or a rightward direction. The operation buttons64B to64D are provided beneath the LCD61. The three operation buttons64B to64D are positioned so as to be operated by the right and left hands of the user. The four operation buttons64E to64H are provided to the right of the LCD61and beneath the right analog stick63B. That is, the four operation buttons64E to64H are positioned so as to be operated by the right hand of the user. Further, the four operation buttons64E,64H,64F, and64G are positioned upward, downward, leftward, and rightward, respectively, with respect to a center position of the four operation buttons. Accordingly, the terminal device6may cause the four operation buttons64E to64H to function as buttons which allow the user to designate an upward, a downward, a leftward, or a rightward direction.

As shown in (a), (b), and (c) ofFIG. 3, a first L button64I and a first R button64J are provided on diagonal upper portions (an upper left portion and an upper right portion) of the housing60. Specifically, the first L button64I is provided on the left end of the upper side surface of the plate-shaped housing60so as to protrude from the upper and left side surfaces. The first R button64J is provided on the right end of the upper side surface of the housing60so as to protrude from the upper and right side surfaces. In this way, the first L button64I is positioned so as to be operated by the index finger of the left hand of the user, and the first R button64J is positioned so as to be operated by the index finger of the right hand of the user (seeFIG. 4).

As shown in (b) and (c) ofFIG. 3, leg parts68A and68B are provided so as to protrude from a rear surface (i.e., a surface reverse of the front surface on which the LCD61is provided) of the plate-shaped housing60, and a second L button64K and a second R button64L are provided so as to protrude from the leg parts68A and68B, respectively. Specifically, the second L button64K is provided at a slightly upper position on the left side (the left side as viewed from the front surface side) of the rear surface of the housing60, and the second R button64L is provided at a slightly upper position on the right side (the right side as viewed from the front-surface side) of the rear surface of the housing60. In other words, the second L button64K is provided at a position substantially opposite to the left analog stick63A provided on the front surface, and the second R button64L is provided at a position substantially opposite to the right analog stick63B provided on the front surface. The second L button64K is positioned so as to be operated by the middle finger of the left hand of the user, and the second R button64L is positioned so as to be operated by the middle finger of the right hand of the user (seeFIG. 4). Further, as shown in (c) ofFIG. 3, the leg parts68A and68B each have a surface facing obliquely upward, and the second L button64K and the second R button64L are provided on the oblique surfaces of the leg parts68A and68B, respectively. Thus, the second L button64K and the second R button64L have button surfaces facing obliquely upward. Since it is supposed that the middle finger of the user moves vertically when the user holds the terminal device6, the upward facing button surfaces allow the user to easily press the second L button64K and the second R button64L. Further, the leg parts68A and68B provided on the rear surface of the housing60allow the user to easily hold the housing60. Moreover, the operation buttons provided on the leg parts68A and68B allow the user to easily perform operation while holding the housing60.

In the terminal device6shown inFIG. 3, the second L button64K and the second R button64L are provided on the rear surface of the housing60. Therefore, if the terminal device6is placed with the screen of the LCD61(the front surface of the housing60) facing upward, the screen of the LCD61may not be perfectly horizontal. Accordingly, in another embodiment, three or more leg parts may be provided on the rear surface of the housing60. In this case, if the terminal device6is placed on a floor with the screen of the LCD61facing upward, the three or more leg parts contact the floor. Thus, the terminal device6can be placed with the screen of the LCD61being horizontal. Such a horizontal placement of the terminal device6may be achieved by providing detachable leg parts on the rear surface of the housing60.

The respective operation buttons64A to64L are assigned functions, according to need, in accordance with a game program. For example, the cross button64A may be used for direction designation operation, selection operation, and the like, and the operation buttons64E to64H may be used for determination operation, cancellation operation, and the like.

The terminal device6includes a power button (not shown) for turning on/off the power of the terminal device6. The terminal device6may include an operation button for turning on/off screen display of the LCD61, an operation button for performing connection setting (pairing) with the game apparatus body5, and an operation button for adjusting the volume of loudspeakers (loudspeakers607shown inFIG. 5).

As shown in (a) ofFIG. 3, the terminal device6includes a marker section (a marker section65shown inFIG. 5) including a marker65A and a marker65B, on the front surface of the housing60. For example, the marker section65is provided above the LCD61. The markers65A and65B are each constituted by one or more infrared LEDs, like the markers8L and8R of the marker8. The marker section65is used, like the marker8, for causing the game apparatus body5to calculate a movement or the like of the controller7with respect to the marker section65. The game apparatus body5is capable of controlling the infrared LEDs of the marker section65to be on or off.

The terminal device6includes a camera66as imaging means. The camera66includes an image pickup element (e.g., a CCD image sensor or a CMOS image sensor) having a predetermined resolution, and a lens. For example, the camera66is provided on the front surface of the housing60. Accordingly, the camera66is capable of taking an image of the face of the user holding the terminal device6. For example, the camera66is capable of taking an image of the user playing a game while viewing the LCD61.

The terminal device6has a microphone (a microphone609shown inFIG. 5) as sound input means. A microphone hole60bis provided in the front surface of the housing60. The microphone609is embedded in the housing60at a position inside the microphone hole60b. The microphone609detects for a sound, such as user's voice, around the terminal device6.

The terminal device6has loudspeakers (loudspeakers607shown inFIG. 5) as sound output means. As shown in (d) ofFIG. 3, speaker holes60aare provided in the lower side surface of the housing60. A sound is output through the speaker holes60afrom the loudspeakers607. In the exemplary embodiment, the terminal device6has two loudspeakers, and the speaker holes60aare provided at positions corresponding to a left loudspeaker and a right loudspeaker, respectively.

The terminal device6includes an extension connector67for connecting another device to the terminal device6. In the exemplary embodiment, as shown in (d) ofFIG. 3, the extension connector67is provided in the lower side surface of the housing60. Any device may be connected to the extension connection67. For example, a controller (a gun-shaped controller or the like) used for a specific game or an input device such as a keyboard may be connected to the extension connector67. If another device need not be connected, the extension connector67need not be provided.

In the terminal device6shown inFIG. 3, the shapes of the operation buttons and the housing60, the number of the respective components, and the positions in which the components are provided are merely examples. The shapes, numbers, and positions may be different from those described above.

Next, an internal structure of the terminal device6will be described with reference toFIG. 5.FIG. 5is a block diagram illustrating an example of an internal structure of the terminal device6. As shown inFIG. 5, the terminal device6includes, in addition to the components shown inFIG. 3, a touch panel controller601, a magnetic sensor602, a gyro sensor604, a user interface controller (UI controller)605, a codec LSI606, loudspeakers607, a sound IC608, a microphone609, a wireless module610, an antenna611, an infrared communication module612, a flash memory613, a power supply IC614, a battery615, and a vibrator619. These electronic components are mounted on an electronic circuit board and accommodated in the housing60.

The UI controller605is a circuit for controlling data input to various input/output sections and data output from various input/output sections. The UI controller605is connected to the touch panel controller601, the analog stick63(the analog sticks63A and63B), the operation button64(the operation buttons64A to64L), the marker section65, the magnetic sensor602, the acceleration sensor603, the gyro sensor604, and the vibrator619. Further, the UI controller605is connected to the codec LSI606and the extension connector67. The power supply IC614is connected to the UI controller605, so that power is supplied to the respective components through the UI controller605. The internal battery615is connected to the power supply IC614, so that power is supplied from the battery615. Further, a battery charger616or a cable, which is supplied with power from an external power supply, may be connected to the power supply IC614via a connector or the like. In this case, the terminal device6can be supplied with power and charged from the external power supply by using the battery charger616or the cable. Charging of the terminal device6may be performed by setting the terminal device6on a cradle (not shown) having a charging function.

The touch panel controller601is a circuit which is connected to the touch panel62and controls the touch panel62. The touch panel controller601generates a predetermined form of touch position data, based on a signal from the touch panel62, and outputs the touch position data to the UI controller605. The touch position data represents coordinates of a position at which an input is performed on an input surface of the touch panel62. The touch panel controller601reads a signal from the touch panel62and generates touch position data every predetermined period of time. Further, various control instructions on the touch panel62are output from the UI controller605to the touch panel controller601.

The analog stick63outputs, to the UI controller605, stick data representing a direction in which the stick part operated by a finger of the user slides (or tilts), and an amount of the sliding (tilting). The operation button64outputs, to the UI controller605, operation button data representing an input state of each of the operation buttons64A to64L (whether or not the operation button is pressed).

The magnetic sensor602detects the magnitude and direction of a magnetic field to detect an orientation. Orientation data representing the detected orientation is output to the UI controller605. The UI controller605outputs, to the magnetic sensor602, a control instruction for the magnetic sensor602. Examples of the magnetic sensor602include: an MI (Magnetic Impedance) sensor, a fluxgate sensor, a hall sensor, a GMR (Giant Magneto Resistance) sensor, a TMR (Tunneling Magneto Resistance) sensor, and an AMR (Anisotropic Magneto Resistance) sensor. However, any sensor may be adopted as long as the sensor can detect an orientation. Strictly speaking, the obtained orientation data does not represent an orientation in a place where a magnetic field is generated in addition to the geomagnetism. Even in such a case, it is possible to calculate a change in the attitude of the terminal device6because the orientation data changes when the terminal device6moves.

The acceleration sensor603is provided inside the housing60. The acceleration sensor603detects the magnitudes of linear accelerations along three axial directions (x-axis, y-axis, and z-axis directions shown in (a) ofFIG. 3), respectively. Specifically, in the acceleration sensor603, the long side direction of the housing60is defined as the x-axial direction, the short side direction of the housing60is defined as the y-axial direction, and the direction orthogonal to the front surface of the housing60is defined as the z-axial direction, thereby detecting the magnitudes of the linear accelerations in the respective axial directions. Acceleration data representing the detected accelerations is output to the UI controller605. The UI controller605outputs, to the acceleration sensor603, a control instruction for the acceleration sensor603. In the exemplary embodiment, the acceleration sensor603is, for example, an electrostatic capacitance type MEMS acceleration sensor. However, in another embodiment, another type of acceleration sensor may be used. Further, the acceleration sensor603may be an acceleration sensor for detecting the magnitude of acceleration in one axial direction or two axial directions.

The gyro sensor604is provided inside the housing60. The gyro sensor604detects the angular velocities around the three axes (the above-described x, y, and z axes), respectively. Angular velocity data representing the detected angular velocities is output to the UI controller605. The UI controller605outputs, to the gyro sensor604, a control instruction for the gyro sensor604. Any number and any combination of gyro sensors may be used as long as the angular velocities around three axes are detected. The gyro sensor604may be constituted by a two-axis gyro sensor and a one-axis gyro sensor, like the gyro sensor703. Alternatively, the gyro sensor604may be a gyro sensor for detecting the angular velocity around one axis or two axes.

The vibrator619is, for example, a vibration motor or a solenoid. The vibrator619is connected to the UI controller605. The terminal device6is vibrated by actuating the vibrator619in accordance with a control instruction outputted from the UI controller605to the vibrator619. The vibration of the terminal device6is transmitted to the user's hand holding the terminal device6. Thus, a so-called vibration-feedback game is realized.

The UI controller605outputs, to the codec LSI606, the operation data including the touch position data, the stick data, the operation button data, the orientation data, the acceleration data, and the angular velocity data, which have been received from the respective components. If another device is connected to the terminal device6through the extension connector67, data representing operation to the other device may be included in the operation data.

The codec LSI606is a circuit for performing a compression process on data to be transmitted to the game apparatus body5, and a decompression process on data transmitted from the game apparatus body5. The LCD61, the camera66, the sound IC608, the wireless module610, the flash memory613, and the infrared communication module612are connected to the codec LSI606. The codec LSI606includes a CPU617and an internal memory618. Although the terminal device6is configured not to perform a game process, the terminal device6may execute a program for managing the terminal device6or a program for communication. For example, a program stored in the flash memory613is loaded into the internal memory618and executed by the CPU617when the terminal device6is powered on, thereby starting up the terminal device6. A part of the area of the internal memory618is used as a VRAM for the LCD61.

The camera66takes an image in accordance with an instruction from the game apparatus body5, and outputs data of the taken image to the codec LSI606. The codec LSI606outputs, to the camera66, a control instruction for the camera66, such as an instruction to take an image. The camera66is also capable of taking a moving picture. That is, the camera66is capable of repeatedly performing image taking, and repeatedly outputting image data to the codec LSI606.

The sound IC608is connected to the loudspeakers607and the microphone609. The sound IC608is a circuit for controlling input of sound data from the microphone609to the codec LSI606and output of sound data from the codec LSI606to the loudspeakers607. Specifically, when the sound IC608receives sound data from the codec LSI606, the sound IC608performs D/A conversion on the sound data, and outputs a resultant sound signal to the loudspeakers607to cause the loudspeakers607to output a sound. The microphone609detects sound (such as user's voice) propagated to the terminal device6, and outputs a sound signal representing the sound to the sound IC608. The sound IC608performs A/D conversion on the sound signal from the microphone609, and outputs a predetermined form of sound data to the codec LSI606.

The codec LSI606transmits the image data from the camera66, the sound data from the microphone609, and the operation data from the UI controller605(terminal operation data), to the game apparatus body5through the wireless module610. In the exemplary embodiment, the codec LSI606subjects the image data and the sound data to a compression process similar to that performed by the codec LSI27. The compressed image data and sound data, and the terminal operation data are output to the wireless module610as transmission data. The antenna611is connected to the wireless module610, and the wireless module610transmits the transmission data to the game apparatus body5through the antenna611. The wireless module610has the same function as the terminal communication module28of the game apparatus body5. That is, the wireless module610has a function of connecting to a wireless LAN by a method based on, for example, the IEEE802.11n standard. The data transmitted from the wireless module610may be encrypted according to need, or may not be encrypted

As described above, the transmission data transmitted from the terminal device6to the game apparatus body5includes the operation data (terminal operation data), the image data, and the sound data. If another device is connected to the terminal device6through the extension connector67, data received from the other device may be included in the transmission data. The infrared communication module612performs, with another device, infrared communication based on, for example, the IRDA standard. The codec LSI606may include, in the transmission data, data received by the infrared communication, and transmit the transmission data to the game apparatus body5, according to need.

As described above, the compressed image data and sound data are transmitted from the game apparatus body5to the terminal device6. These data are received by the codec LSI606through the antenna611and the wireless module610. The codec LSI606decompresses the received image data and sound data. The decompressed image data is output to the LCD61, and an image according to the image data is displayed on the LCD61. On the other hand, the decompressed sound data is output to the sound IC608, and a sound based on the sound data is output from the loudspeakers607.

When control data is included in the data received from the game apparatus body5, the codec LSI606and the UI controller605make control instructions for the respective components, according to the control data. As described above, the control data represents control instructions for the respective components (in the exemplary embodiment, the camera66, the touch panel controller601, the marker section65, the sensors602to604, the vibrator619, and the infrared communication module612) included in the terminal device6. In the exemplary embodiment, the control instructions represented by the control data are considered to be instructions to start and halt (stop) the operations of the above-mentioned components. That is, some components which are not used for a game may be halted to reduce power consumption. In this case, data from the halted components are not included in the transmission data transmitted from the terminal device6to the game apparatus body5. Since the marker section65is constituted by infrared LEDs, the marker section65is controlled by simply turning on/off the supply of power thereto.

As described above, the terminal device6includes the operation means such as the touch panel62, the analog sticks63, and the operation buttons64. In another embodiment, however, the terminal device6may include other operation means instead of or in addition to these operation means.

The terminal device6includes the magnetic sensor602, the acceleration sensor603, and the gyro sensor604as sensors for calculating the movement (including the position and the attitude, or a change in the position or the attitude) of the terminal device6. In another embodiment, however, the terminal device6may include one or two of these sensors. In still another embodiment, the terminal device6may include other sensors instead of or in addition to these sensors.

The terminal device6includes the camera66and the microphone609. In another embodiment, however, the terminal device6may not include the camera66and the microphone609, or may include either of the camera66and the microphone609.

The terminal device6includes the marker section65as a component for calculating the positional relation between the terminal device6and the controller7(such as the position and/or the attitude of the terminal device6as viewed from the controller7). In another embodiment, however, the terminal device6may not include the marker section65. In still another embodiment, the terminal device6may include other means as a component for calculating the above-mentioned positional relation. For example, the controller7may include a marker section, and the terminal device6may include an image pickup element. In this case, the marker8may include an image pickup element instead of an infrared LED.

Next, a structure of the board type controller9will be described with reference toFIGS. 6 to 8.FIG. 11is a perspective view illustrating an example of an external appearance of the board type controller9shown inFIG. 1. As shown inFIG. 6, the board type controller9includes a platform9aon which a user stands (on which the user puts his/her feet), and at least four load sensors94ato94dfor detecting a load applied to the platform9a. Each of the load sensors94ato94dis embedded in the platform9a(seeFIG. 7), and the positions where the load sensors94ato94dare provided are indicated by dotted lines inFIG. 6. In the following description, the four load sensors94ato94dmay be collectively referred to as a load sensor94.

The platform9ais formed in the shape of substantially a rectangular parallelepiped, and is in the shape of substantially a rectangle as viewed from the top. For example, the short side of the rectangular shape of the platform9ais approximately 30 cm, and the long side thereof is approximately 50 cm. The upper surface of the platform9ais flat, and has a pair of planes on which the user stands with the bottoms of his/her feet contacting thereto. Specifically, the upper surface of the platform9ahas a plane (a back-left region enclosed with a double line inFIG. 6) on which the user's left foot is put, and a plane (a front-right region enclosed with a double line inFIG. 6) on which the user's right foot is put. The platform9ahas, at four corners thereof, side surfaces each partially projecting outward in a cylindrical shape.

In the platform9a, the four load sensors94ato94dare arranged at predetermined intervals. In the exemplary embodiment, the four load sensors94ato94dare arranged on the periphery of the platform9a, more specifically, at the four corners of the platform9a, respectively. The intervals of the load sensors94ato94dare appropriately set such that the load sensors94ato94dcan accurately detect the intention of a game operation which is expressed by a manner of applying a load onto the platform9aby the user.

FIG. 7shows an example of a cross-sectional view of the board type controller9, taken along a line A-A inFIG. 6, and an example of an enlarged view of a corner part where a load sensor94is arranged. InFIG. 7, the platform9aincludes a support plate90on which the user stands, and legs92. The load sensors94ato94dare provided in positions where the legs92are provided. In the exemplary embodiment, since the four legs92are provided at the four corners, the four load sensors94ato94dare also provided at the corresponding four corners. Each leg92is formed by plastic molding in the shape of substantially a cylinder with a base. Each load sensor94is located on a spherical part92aprovided on the base of the corresponding leg92. The support plate90is supported by the legs92via the load sensors94.

The support plate90includes an upper plate90aforming an upper surface and an upper side surface portion, a lower plate90bforming a lower surface and a lower side surface portion, and an intermediate plate90cprovided between the upper plate90aand the lower plate90b. The upper plate90aand the lower plate90bare formed by, for example, plastic molding, and are integrated using an adhesive or the like. The intermediate plate90cis, for example, formed of a single metal plate by press forming. The intermediate plate90cis fixed onto the four load sensors94ato94d. The upper plate90ahas, on a lower surface thereof, a grid-patterned rib (not shown), and is supported by the intermediate plate90cvia the rib. Therefore, when the user stands on the platform9a, the load is transferred to the four legs92via the support plate90and the load sensors94ato94d. As indicated by arrows inFIG. 7, a reaction from a floor, which is generated by the input load, is transferred from the legs92through the spherical parts92a, the load sensors94ato94dand the intermediate plate90cto the upper plate90a.

Each load sensor94is, for example, a strain gauge (strain sensor) load cell, which is a load converter for converting an input load to an electrical signal. In the load sensor94, a strain-generating body95is deformed according to an input load, resulting in a strain. The strain is converted into a change of electrical resistance and then converted into a change of voltage by a strain sensor96attached to the strain-generating body.95Therefore, the load sensor94outputs, from an output terminal thereof, a voltage signal indicating the input load.

The load sensor94may be of other types, such as a tuning fork type, a string vibration type, an electrostatic capacitance type, a piezoelectric type, a magnetostrictive type, and a gyroscopic type.

Referring back toFIG. 6, the board type controller9further includes a power button9c. When the power button9cis operated (e.g., when the power button9cis pressed) in the state where the board type controller9is not activated, power is supplied to each of circuit components (seeFIG. 8) of the board type controller9. However, there are cases in which the board type controller9is powered on in accordance with an instruction from the game apparatus body5and thereby supply of power to the circuit components is started. The board type controller9may be automatically powered off when a state where the user does not stand thereon continues for a predetermined period of time (e.g., 30 sec) or more. Further, when the power button9cis again operated in the state where the board type controller9is in the active state, the board type controller9may be powered off to stop supply of power to the circuit components.

FIG. 8is a block diagram illustrating an example of an electrical configuration of the board type controller9. InFIG. 8, flows of signals and data are indicated by solid arrows, and supply of power is indicated by dotted arrows.

As shown inFIG. 8, the board type controller9includes a microcomputer100for controlling the operation thereof. The microcomputer100includes a CPU, a ROM, a RAM, and the like, which are not shown. The CPU controls the operation of the board type controller9in accordance with a program stored in the ROM.

The power button9c, an AD converter102, a DC-DC converter104, and a wireless module106are connected to the microcomputer100. An antenna106ais connected to the wireless module106. The four load sensors94ato94bare connected to the AD converter102via amplifiers108, respectively.

Further, the board type controller9includes a battery110for supplying power to the circuit components. In another embodiment, an AC adapter may be connected to the board type controller9instead of the battery110so that commercial power is supplied to the circuit components. In this case, instead of the DC-DC converter104, a power circuit, which converts alternating current into direct current and lowers and rectifies a direct-current voltage, may be provided in the board type controller9. In the exemplary embodiment, power is supplied directly from the battery110to the microcomputer100and the wireless module106. In other words, power is constantly supplied from the battery110to the wireless module106and some components (such as the CPU) in the microcomputer100to detect whether or not the power button9cis turned on and whether or not a command that instructs power-on is transmitted from the game apparatus body5. On the other hand, power is supplied from the battery110through the DC-DC converter104to the load sensors94ato94d, the AD converter102, and the amplifiers108. The DC-DC converter104converts a voltage value of direct current supplied from the battery110into a different voltage value, and supplies the resultant direct current to the load sensors94ato94d, the AD converter102, and the amplifiers108.

Supply of power to the load sensors94ato94d, the A/D converter102and the amplifiers108may be performed according to need by the microcomputer100that controls the DC-DC converter104. Specifically, when the microcomputer100determines that it is necessary to operate the load sensors94ato94dto detect a load, the microcomputer100may control the DC-DC converter104to supply power to the load sensors94ato94d, the A/D converter102and the amplifiers108.

When power is supplied to the load sensors94ato94d, the load sensors94ato94deach output a signal indicating a load inputted thereto. These signals are amplified by the respective amplifiers108, and converted from analog signals into digital data by the A/D converter102. The digital data is input to the microcomputer100. The detected values of the load sensors94ato94dare given identification information of the load sensors94ato94d, so that the load sensors94ato94dcan be identified from the corresponding detected values. Thus, the microcomputer100can obtain the data representing the detected load values of the four load sensors94ato94dat the same time.

On the other hand, when the microcomputer100determines that it is not necessary to operate the load sensors94ato94d, i.e., when it is not the time for load detection, the microcomputer100controls the DC-DC converter104to stop supply of power to the load sensors94ato94d, the A/D converter102, and the amplifiers108. Thus, the board type controller9can operate the load sensors94ato94dto detect a load or a distance only when it is required, resulting in a reduction in power consumption for load detection.

Load detection is typically required when the game apparatus body5(FIG. 1) needs to obtain load data. For example, when game apparatus body5requires load information, the game apparatus body5transmits an information acquisition command to the board type controller9. When the microcomputer100receives the information acquisition command from the game apparatus body5, the microcomputer100controls the DC-DC converter104to supply power to the load sensors94ato94dand the like, thereby detecting a load. On the other hand, when the microcomputer100does not receive a load acquisition command from the game apparatus body5, the microcomputer100controls the DC-DC converter104to stop supply of power to the load sensors94ato94dand the like.

The microcomputer100may control the DC-DC converter104based on a determination that load detection timing occurs at predetermined intervals. When such periodic load detection is performed, information regarding the constant time period may be supplied and stored from the game apparatus body5to the microcomputer100of the board type controller9when the game is started, or it may be preinstalled in the microcomputer100.

The data representing the detected values from the load sensors94ato94dare transmitted as board operation data (input data) for the board type controller9from the microcomputer100via the radio module106and an antenna106bto the game apparatus body5. For example, when the microcomputer100has performed load detection according to a command from the game apparatus body5, the microcomputer100transmits the detected value data of the load sensors94ato94dto the game apparatus body5on receipt of the detected value data from the A/D converter102. The microcomputer100may transmit the detected value data to the game apparatus body5at predetermined intervals. If the interval of the data transmission is longer than the interval of the load detection, data containing load values which have been detected at a plurality of detection timings up to the next transmission timing may be transmitted.

The wireless module106is set so as to perform communication according to the same wireless standard (the Bluetooth, wireless LAN, and the like) as that for the controller communication module19of the game apparatus body5. Accordingly, the CPU10of the game apparatus body5is allowed to transmit an information acquisition command to the board type controller9through the controller communication module19and the like. Thus, the board type controller9is allowed to receive the command from the game apparatus body5through the wireless module106and the antenna106a. Further, the board type controller9is allowed to transmit the board operation data including the load detection values (or load calculation values) of the load sensors94ato94dto the game apparatus body5.

For example, in a game which is executed based on a simple sum of four load values detected by the four load sensors94ato94d, the user is allowed to stand at any position with respect to the four load sensors94ato94dof the board type controller9. That is, the user is allowed to stand on the platform9aat any position and in any direction to play a game. In some kinds of games, however, the direction of a load value detected by each of the four load sensors94viewed from the user needs to be identified. That is, a positional relation between the four load sensors94of the board type controller9and the user needs to be recognized. In this case, for example, the positional relation between the four load sensors94and the user may be previously defined, and the user may be supposed to stand on the platform9ain a manner which allows the predetermined positional relation. Typically, a positional relation in which two of the load sensors94ato94dare present in front of, behind, to the right of, and to the left of the user standing in the center of the platform9a, i.e., a positional relation in which the user stands in the center of the platform9aof the board type controller9, is defined. In this case, the platform9aof the board type controller9is rectangular in shape as viewed from the top, and the power button9cis provided at one side (long side) of the rectangle. Therefore, it is previously ruled that the user, using the power button9cas a guide, stands on the platform9asuch that the long side at which the power button9cis provided is located in a predetermined direction (front, rear, left or right). In this case, each of the load values detected by the load sensors94ato94dis a load value of a predetermined direction (front right, front left, rear right, or rear left) as viewed from the user. Therefore, the board type controller9and the game apparatus body5can find out a direction to which each detected load value corresponds as viewed from the user, based on the identification information of the load sensors94contained in the detected load value data and arrangement data representing previously set (stored) positions or directions of the load sensors94with respect to the user. As a result, it is possible to recognize what is intended by a game operation performed by the user, such as an operation direction of forward, backward, rightward or leftward, or which foot of the user is lifted.

Next, an outline of information processing performed on the game apparatus body5will be described with reference toFIGS. 9, 10A, and 10Bbefore a specific description of processes performed by the game apparatus body5is given. In the following description of the outline of information processing, a first exemplary game is used as an example of information processing.FIG. 9is a diagram illustrating an example of a user who plays the first exemplary game by using the terminal device6and the board type controller9.FIG. 10Ais a diagram illustrating an example of an image of the first exemplary game, which is displayed on the LCD61of the terminal device6.FIG. 10Bis a diagram illustrating an example of an image of the first exemplary game, which is displayed on the monitor2.

As shown inFIG. 9, in the first exemplary game, the user performs an operation by using the terminal device6and the board type controller9. Specifically, the user stands on the board type controller9with the terminal device6in his/her hands, and performs an operation. Then, the user performs an action (e.g., stepping or knee bending) on the board type controller9and simultaneously performs an operation to move the terminal device6, while viewing the image displayed on the monitor2and the image displayed on the LCD61of the terminal device6, thereby playing the game. On each of the LCD61of the terminal device6and the monitor2, a game image is displayed, in which a player object Po performs an action (e.g., an action to change its attitude or an action to change its movement direction) in a virtual world in accordance with the direction and the attitude of the terminal device6held by the user as well as the user's action on the board type controller9, and the position and the attitude of a virtual camera set in the virtual world are changed in accordance with the position and the attitude of the player object Po.

As shown inFIG. 10A, on the LCD61of the terminal device6, the player object Po moving in the virtual world is displayed. In the example shown inFIG. 10A, a virtual camera is located near the back of the player object Po swimming in the sea, and a state of the virtual world viewed from the virtual camera is displayed together with player object Po. Displaying, on the LCD61, the virtual world viewed at close range from behind the player object Po allows the user holding the terminal device6to experience a sense of presence in the virtual world, and allows the user to intuitively know the movement direction and the movement speed of the player object Po. Further, the attitude and the movement direction of the player object Po are changed when the attitude of the terminal device6is changed or the terminal device6is rotated to change its direction, and the position and the attitude of the virtual camera are changed in accordance with the attitude change of the terminal device6. For example, by associating the attitude and the direction of the terminal device6with the attitude and the direction of the virtual camera, the user is allowed to experience a sense of looking into the virtual world through the LCD61of the terminal device6.

Further, as shown inFIG. 10B, the same virtual world as that displayed on the LCD61is displayed on the monitor2. In the example shown inFIG. 10B, a state of the virtual world in which the player object Po swimming in the sea is viewed from a distance in a manner of bird's eye viewing is displayed together with the player object Po. Further, in the example shown inFIG. 10B, a movement trajectory Lp of the player object Po in the sea is displayed in the virtual world. Displaying, on the monitor2, the virtual world in which the player object Po is viewed from a distance in a manner of bird's eye viewing allows the user to easily know the surroundings of the player object Po, and allows another person who watches the user's playing the first exemplary game to enjoy a view in which the player object Po moves in the virtual world.

For example, when the user performs stepping on the board type controller9, the player object Po swims in the sea with flutter kicks at a speed based on the user's stepping action. When the user performs knee bending on the board type controller9, the player object Po swims in the sea with dolphin kicks at a speed based on the user's knee bending action. In this way, the user is allowed to change the swimming style or the movement speed of the player object Po in accordance with his/her action on the board type controller.

For example, as described above, detected load values based on the user's action on the board type controller9are output from the board type controller9. Using the detected load values allows calculation of a total load applied to the board type controller9, and a center-of-gravity position of the load applied to the board type controller9. Further, using a change in the total load or a change in the center-of-gravity position allows estimation of the action of the user on the board type controller9. Based on the estimated user's action on the board type controller9, the swimming style and the movement speed of the player object Po are set.

Further, the attitude and the movement direction of the player object Po swimming in the sea change in accordance with the direction and the attitude of the terminal device6held by the user. For example, when the user turns the terminal device6to the upward, downward, leftward, or rightward direction, the attitude of the player object Po swimming in the sea (the direction to which the player object Po faces in the virtual world) changes in accordance with the direction change of the terminal device6, and the direction in which the player object Po swims also changes. Specifically, when the user changes the direction of the terminal device6such that the back surface of the terminal device6faces upward, the attitude of the player object Po changes such that the player object Po faces the sea surface, and the swimming direction of the player object Po changes such that the player object Po swims upward to the sea surface. When the user changes the direction of the terminal device6such that the back surface of the terminal device6faces leftward, the attitude of the player object Po changes to the left as viewed from the player object Po, and the swimming direction of the player object Po changes to the left. In this way, the user is allowed to change the attitude and the movement direction of the player object Po in accordance with the direction and the attitude of the terminal device6held by the user. For example, by associating the attitude and the direction of the terminal device6with the attitude and the movement direction of the player object Po, the user is allowed to perform, by using the terminal device6, a realistic operation as if he/she becomes the player object Po. Further, since the virtual camera is located near the back of the player object Po swimming in the sea, when the attitude and the movement direction of the player object Po change, the attitude and the position of the virtual camera also change. For example, by associating the attitude and the direction of the terminal device6with the attitude and the direction of the virtual camera, the user is allowed to experience a sense of looking into the virtual world through the LCD61of the terminal device6.

For example, acceleration data or angular velocity data based on a change in the attitude of the terminal device6is output from the terminal device6. Since the direction of the gravitational acceleration applied to the terminal device6can be calculated by using the acceleration represented by the acceleration data, it is possible to estimate the attitude of the terminal device6with respect to the vertical direction in the real space. Further, since the angular velocity and/or the dynamic acceleration applied to the terminal device6are known by using the angular velocity represented by the angular velocity data and/or the acceleration represented by the acceleration data, it is possible to estimate a change in the attitude (a change in the direction) of the terminal device from its initial attitude in the real space by using the angular velocity and/or the dynamic acceleration. Based on the estimated attitude change (direction change) of the terminal device, the attitude and the movement direction of the player object Po as well as the attitude and the position of the virtual camera are set.

Next, a process of the first embodiment, which is performed in the game system1, will be described in detail. First, main data used in the process will be described with reference toFIG. 11.FIG. 11is a diagram illustrating an example of main data and programs stored in the external main memory12and/or the internal main memory35(hereinafter, these two main memories are collectively referred to as a main memory) of the game apparatus body5of the first embodiment.

As shown inFIG. 11, in a data storage area of the main memory, board operation data Daa, terminal operation data Dab, load value data Dac, center-of-gravity position data Dad, terminal device direction/attitude data Dae, operation direction data Daf, action/attitude data Dag, movement vector data Dah, position data Dai, virtual camera data Daj, movement trajectory data Dak, action mode flag data Dam, image data Dan, and the like are stored. In addition to the data shown inFIG. 11, data required for a game process, such as image data of various objects displayed on the monitor2and the LCD61, and sound data used for a game, are optionally stored in the main memory. In a program storage area of the main memory, various programs Pa constituting an information processing program of the first embodiment are stored.

As the board operation data Daa, a series of operation information (board operation data) transmitted as transmission data from the board type controller9is stored, and updated to the latest board operation data. For example, the board operation data Daa includes load data Daa1and the like. The load data Daa1is data representing load values detected by the load sensors94ato94dof the board type controller9.

As the terminal operation data Dab, a series of operation information (terminal operation data) transmitted as transmission data from the terminal device6is stored, and updated to the latest terminal operation data. For example, the terminal operation data Dab includes acceleration data Dab1, angular velocity data Dab2, and the like. The acceleration data Dab1is data representing an acceleration (an acceleration vector) detected by the acceleration sensor603. For example, the acceleration data Dab1represents a three-dimensional acceleration vector whose components are accelerations in the three-axis (x-axis, y-axis, and z-axis) directions shown inFIG. 3. In another embodiment, the acceleration data Dab1may represent an acceleration in any one or more directions. The angular velocity data Dab2is data representing an angular velocity detected by the gyro sensor604. For example, the angular velocity data Dab2represents angular velocities around the three axes (x-axis, y-axis, and z-axis) shown inFIG. 3. In another example, the angular velocity data Dab2may represent an angular velocity around any one or more axes.

The game apparatus body5sequentially receives the data (e.g., the data representing the detected load value, the acceleration, and the angular velocity) included in the operation information transmitted from the controller7, the board type controller9, and the terminal device6at predetermined intervals (e.g., at intervals of 1/200 sec.). For example, the received data is successively stored in the main memory by the I/O processor31. In a process flow described later, the CPU10reads, every frame period (e.g., 1/60 sec.), the latest board operation data and the latest terminal operation data from the main memory, to update the board operation data Daa and the terminal operation data Dab, respectively.

The operation information transmitted at predetermined intervals from the controller7, the board type controller9, and the terminal device6may be temporarily stored in the buffer (not shown) included in the controller communication module19or the terminal communication module28. In this case, the data stored in the buffer is read every frame period, and thus the board operation data Daa (e.g., the load data Daa1) or the terminal operation data Dab (e.g., the acceleration data Dab1and the angular velocity data Dab2) in the main memory is updated to be used. At this time, since the cycle of receiving the operation information is different from the processing cycle, a plurality of pieces of information received at a plurality of timings are stored in the buffer. The process is executed by using only the latest operation information among the plurality of pieces of operation information received at the plurality of timings.

The load value data Dac is an aggregate of data each representing a load value detected by the board type controller9. For example, the load value data Dac is an aggregate of data each representing a sum of load values (total load value) detected by the load sensors94ato94d. Specifically, the load value data Dac is an array of data each representing the above-mentioned total load value within a predetermined period, which is calculated time-sequentially, and the data representing the total load value is time-sequentially stored in each element of the array.

The center-of-gravity position data Dad is an aggregate of data each representing a center-of-gravity position of a load applied to the board type controller9. For example, the center-of-gravity position data Dad is an aggregate of data each representing a position of center of gravity which is calculated based on the load values detected by the respective load sensors94ato94dby using a predetermined numerical formula. Specifically, the center-of-gravity position data Dad is an array of data each representing the above-mentioned center-of-gravity position within a predetermined period, which is calculated time-sequentially, and the data representing the center-of-gravity position is time-sequentially stored in each element of the array.

The terminal device direction/attitude data Dae includes real space reference direction data Dae1, current direction data Dae2, and the like. The real space reference direction data Dae1is data representing a reference direction (attitude; real space reference direction) of the terminal device6in the real space. The current direction data Dae2is data representing a direction and an attitude (current direction) of the terminal device6at the current time in the real space. For example, a real space reference direction data Dae1and a current direction data Dae2are calculated based on the acceleration data Dab1and the angular velocity data Dab2included in the terminal operation data Dab. Methods for calculating a real space reference direction and a current direction will be described later.

The operation direction data Daf includes virtual world reference direction data Daf1, operation instruction direction data Daf2, and the like. The virtual world reference direction data Daf1is data representing a reference direction set in the virtual world. The operation instruction direction data Daf2is data representing an operation instruction direction which is currently instructed in the virtual world by the user. Methods for calculating a virtual world reference direction and an operation instruction direction will be described later.

The action/attitude data Dag is data representing an action and an attitude of the player object Po in the virtual world. The movement vector data Dah is data representing a movement speed and a movement direction of the player object Po in the virtual world. For example, the movement vector data Dah represents a movement vector in the virtual world. The position data Dai is data representing a position of the player object Po in the virtual world. The virtual camera data Daj is data relating to a virtual camera set in the virtual world. For example, the virtual camera data Daj includes data relating to a first virtual camera for generating a game image to be displayed on the LCD61of the terminal device6, and data relating to a second virtual camera for generating a game image to be displayed on the monitor2. The movement trajectory data Dak is data representing a trajectory along which the player object Po has moved in the virtual world.

The action mode flag data Dam is data representing an action mode flag which is set in accordance with an action (e.g., a swimming style) set on the player object Po. For example, when the user performs knee bending on the board type controller9, the action of the player object Po is set to an action of swimming with dolphin kicks in the sea, and the action mode flag is turned on.

The image data Dan includes player object data Dan1, background image data Dan2, and the like. The player object data Dan1is data for arranging the player object Po of the first exemplary game in the virtual world to generate a game image. The background image data Dan2is data for arranging a background of the first exemplary game in the virtual world to generate a game image.

Next, the process of the first embodiment performed by the game apparatus body5will be described in detail with reference toFIGS. 12 to 17.FIG. 12is a flowchart illustrating an example of a process of the first embodiment executed by the game apparatus body5.FIG. 13is a sub-routine illustrating an example of a game control process in step44inFIG. 12.FIG. 14is a sub-routine illustrating an example of a player object setting process in step83inFIG. 13.FIG. 15is a sub-routine illustrating an example of a player object action setting process in step126inFIG. 14.FIG. 16is a diagram illustrating an example of a relationship between a real space reference direction and a current direction which are projected on a horizontal plane in a real space, and a virtual world reference direction and an operation instruction direction which are projected on a horizontal plane in a virtual world.FIG. 17is a diagram illustrating an example of a relationship between a current direction projected on a vertical plane in the real space, and an operation instruction direction projected on a vertical plane in the virtual world. In the flowcharts shown inFIGS. 12 to 15, process steps for causing the player object Po to move and displaying the same, in accordance with an operation performed by a user using the terminal device6and the board type controller9will be mainly described among the whole process, while detailed description of other process steps is not given. Further, inFIGS. 12 to 15, each of the steps executed by the CPU10is abbreviated as “S”.

When the game apparatus body5is powered on, the CPU10of the game apparatus body5executes a boot program stored in the ROM/RTC13to initialize each unit such as the main memory. Then, the information processing program stored in the optical disc4is loaded to the main memory, and the CPU10starts to execute the program. The flowcharts shown inFIGS. 12 to 15show process steps to be performed after the above-mentioned process steps are completed.

As shown inFIG. 12, the CPU10performs an initialization process (step40), and proceeds to the next step. For example, in the initialization process at step40, the CPU10constructs a virtual world, arranges a player object Po and other objects, which appear in a game space, in their initial positions, and sets the initial values of various parameters used for the game process.

Next, the CPU10sets a reference direction based on data transmitted from the terminal device6(step41), and proceeds to the next step. Hereinafter, an example of setting of a reference direction by the CPU10will be described.

The terminal device6repeatedly transmits the above-mentioned data to the game apparatus body5. In the game apparatus body5, the terminal communication module28sequentially receives the above data, and the I/O processor31sequentially stores the terminal operation data, the camera image data, and the microphone sound data in the main memory. In step41, the CPU10reads the latest terminal operation data from the main memory, and updates the acceleration data Dab1and the angular velocity data Dab2.

Next, the CPU10calculates a direction and an attitude of the terminal device6in the real space. For example, the CPU10calculates, as a reference direction (initial attitude) in the real space, a direction and an attitude of the terminal device6at the current time, based on the acceleration represented by the acceleration data Dab1and the angular velocity represented by the angular velocity data Dab2, and updates the real space reference direction data Dae1by using data representing the calculated reference direction of the terminal device6. For example, the CPU10is capable of calculating an amount of rotation (an amount of direction change) of the terminal device6in the real space, per unit time, based on the angular velocity represented by the angular velocity data Dab2. Further, in a state where the terminal device6is substantially stationary (in a static state) in the real space, the acceleration applied to the terminal device6is the gravitational acceleration. Therefore, the CPU10is capable of calculating a direction of gravity applied to the terminal device6(i.e., an attitude of the terminal device6with respect to the vertical direction in the real space), based on the acceleration represented by the acceleration data Dab1. Accordingly, the CPU10is capable of calculating an initial attitude of the terminal device6, based on the acceleration represented by the acceleration data Dab1and the angular velocity represented by the angular velocity data Dab2. In the following description, when step41is executed, a direction in which the back surface of the terminal device6faces in the real space (the z-axis positive direction shown inFIG. 3) is the real space reference direction.

The initial attitude of the terminal device6may be calculated based on the acceleration represented by the acceleration data Dab1, or on the direction of a magnetism detected by the magnetic sensor602. Alternatively, the user is caused to perform a predetermined operation with the terminal device6being set in a specific attitude, and the specific attitude at the time when the predetermined operation is performed may be used as an initial attitude. As for the timing at which the initial attitude is set, i.e., the timing at which step41is executed, step41may be automatically executed when the game is started, or may be executed in accordance with that the user performs a predetermined operation (e.g., an operation to press a predetermined operation button64) by using the terminal device6.

In step41, the CPU10calculates a current direction in which the player object Po faces in the virtual world (e.g., a direction in which the player object Po moves if there is no direction change) as a reference direction (initial attitude) in the virtual world, and updates the virtual world reference direction data Daf1by using data representing the calculated reference direction in the virtual world.

The above-mentioned step41is followed by a process at step42. Thereafter, a process loop of a series of process steps42to51is repeatedly executed every predetermined period (one frame period).

In step42, the CPU10obtains board operation data transmitted from the board type controller9, and proceeds to the next step. The board type controller9repeatedly transmits the board operation data to the game apparatus body5. Accordingly, in the game apparatus body5, the controller communication module19sequentially receives the board operation data, and the I/O processor31sequentially stores the received board operation data in the main memory. The interval of transmission of the board operation data from the board type controller9may be shorter than the game processing period (one frame period), and for example, it is 1/200 sec. In step42, the CPU10reads the latest board operation data from the main memory to update the board operation data Daa1. The board operation data includes data representing identification information of the load sensors94ato94d, and data representing the load values detected by the load sensors94ato94d. The load data Daa1is updated by using each of the data identified by the identification information.

Next, the CPU10obtains various data transmitted from the terminal device6(step43), and proceeds to the next step. The terminal device6repeatedly transmits the data to the game apparatus body5. Accordingly, in the game apparatus body5, the terminal communication module28sequentially receives the data, and the codec LSI27sequentially performs a decompression process on the camera image data and the microphone sound data. Then, the I/O processor31sequentially stores the terminal operation data, the camera image data, and the microphone sound data in the main memory. In step43, the CPU10reads the latest terminal operation data from the main memory to update the acceleration data Dab1and the angular velocity data Dab2.

Next, the CPU10performs a game control process (step44), and proceeds to the next step. The game control process is a process in which, for example, the player object Po in the virtual world is caused to move in accordance with a game operation performed by the user, thereby progressing the game. In the first exemplary game, the user is allowed to play various games by using the terminal device6and the board type controller9. Hereinafter, the game control process at step44will be described with reference toFIG. 13.

InFIG. 13, the CPU10calculates a load value and a center-of-gravity position (step81), and proceeds to the next step. For example, the CPU10calculates a total load value by summing up the detected load values represented by the load data Daa1, and updates the latest data in the time-sequence data array of the load value data Dac by using the data representing the total load value. Specifically, since the load data Daa1represents the latest load values detected by the load sensors94ato94d, the total load value is obtained by summing up the detected load values. Further, the CPU10calculates a center-of-gravity position by using the detected load values represented by the load data Daa1, and updates the latest data in the time-sequence data array of the center-of-gravity position data Dad by using the data representing the center-of-gravity position. Hereinafter, an example of a center-of-gravity position calculation method will be described.

The center-of-gravity position is a position of the center of gravity of a load applied to the platform9aof the board type controller9, and is determined based on the load values detected by the load sensors94ato94d(seeFIG. 6). For example, the center-of-gravity position is represented by coordinate values based on a predetermined coordinate system corresponding to a position on the platform9aof the board type controller9(e.g., a X1Y1coordinate system in which the center of the platform9acorresponds to an origin thereof, the long side direction of the platform9acorresponds to the X1axis direction, and the short side direction corresponds to the Y1axis direction). Assuming that a load value detected by the load sensor94ais defined as “a”, a load value detected by the load sensor94bis defined as “b”, a load value detected by the load sensor94cis defined as “c”, and a load value detected by the load sensor94dis defined as “d”, an X1-axis coordinate value (X1) and a Y-axis coordinate value (Y1) of the center of gravity are calculated in accordance with the following equations:
X1=((a+c)−(b+d))×m
Y1=((c+d)−(a+b))×n

where m and n are predetermined constants.

The total load value and the center-of-gravity position, which are thus calculated, vary in accordance with an action or a weight shift (posture) of the user on the board type controller9. For example, when the user repeats stepping on the board type controller9, the total load value varies in accordance with the stepping cycle, and the center-of-gravity position shifts as if reciprocating, according to the stepping cycle, between the position on which the user puts his/her left foot and the position on which the user puts his/her right foot. In another example, when the user repeats knee bending on the board type controller9, the total load value varies in accordance with the cycle of the knee bending, but the variation of the center-of-gravity position is relatively reduced.

Next, the CPU10calculates a direction change and an attitude of the terminal device6(step82), and proceeds to the next step. For example, the CPU10calculates a direction change and an attitude of the terminal device6, based on the acceleration represented by the acceleration data Dab1and the angular velocity represented by the angular velocity data Dab2, and updates the current direction data Dae2by using data representing the current direction which corresponds to the calculated direction change and attitude of the terminal device6. The CPU10is capable of calculating an amount of rotation (an amount of direction change) of the terminal device6in the real space, per unit time, based on the angular velocity represented by the angular velocity data Dab2. Further, in a state where the terminal device6is substantially stationary (in a static state) in the real space, the acceleration applied to the terminal device6is the gravitational acceleration. Therefore, the CPU10is capable of calculating a direction of gravity applied to the terminal device6(i.e., the attitude of the terminal device6with respect to the vertical direction in the real space), based on the acceleration represented by the acceleration data Dab1. Accordingly, the CPU10is capable of calculating a direction change and an attitude of the terminal device6, based on the acceleration represented by the acceleration data Dab1and the angular velocity represented by the angular velocity data Dab2. In the following description, a direction (the z-axis positive direction shown inFIG. 3) at the current time, in which the back surface of the terminal device6faces in the real space, is regarded as the current direction of the terminal device6.

In the exemplary embodiment, a direction change and an attitude of the terminal device6are calculated based on the data representing the acceleration and the angular velocity detected by the terminal device6. In another embodiment, however, a direction change and an attitude of the terminal device6may be calculated by using any one piece data or three or more pieces of data. For example, the magnetic sensor602included in the terminal device6detects a geomagnetism applied to the terminal device6. Therefore, based on the direction of the geomagnetism applied to the terminal device6, a predetermined orientation with respect to the terminal device6(i.e., the attitude of the terminal device6with respect to the predetermined orientation) can be calculated. Even when a magnetic field is generated in addition to the geomagnetism in the real space where the terminal device6is located, an amount of rotation of the terminal device6can be calculated. Accordingly, the CPU10is capable of calculating a direction change and an attitude of the terminal device6by using at least one of the data representing the acceleration, the data representing the angular velocity, and the data representing the magnetism, which are detected by the terminal device6.

Any calculation method may be used for calculating an attitude of the terminal device6. For example, a calculation method is considered, in which the attitude of the terminal device6, which is calculated based on the angular velocity represented by the angular velocity data Dab2, is corrected by using the acceleration represented by the acceleration data Dab1and the direction of the magnetism detected by the magnetic sensor602.

Specifically, the CPU10firstly calculates an attitude of the terminal device6based on the angular velocity represented by the angular velocity data Dab2. Any method may be used for calculating an attitude of the terminal device6based on the angular velocity. For example, an attitude of the terminal device6may be calculated by using the immediately preceding attitude (most-recently calculated attitude) and the current angular velocity (the angular velocity currently obtained at step42in the process loop). The CPU10causes the immediately preceding attitude to rotate at the current angular velocity for a unit time to obtain a new attitude. The immediately preceding attitude is represented by the current direction data Dae2, and the current angular velocity is represented by the angular velocity data Dab2. Accordingly, the CPU10reads the current direction data Dae2and the angular velocity data Dab2, and calculates an attitude of the terminal device6. As described above, the initial attitude of the terminal device6is determined in step41. Therefore, when calculating an attitude of the terminal device6based on the angular velocity, the CPU10can calculate an attitude of the terminal device6at the current time, based on the initial attitude of the terminal device6.

Next, the CPU10corrects the attitude of the terminal device6calculated based on the angular velocity, by using the acceleration represented by the acceleration data Dab1. Specifically, the CPU10calculates an attitude of the terminal device6based on the acceleration represented by the acceleration data Dab1. In a state where the terminal device6is substantially stationary, the acceleration applied to the terminal device6is the gravitational acceleration. Accordingly, in this state, the CPU10is capable of calculating a direction of the gravitational acceleration (direction of gravity) by using the direction of the acceleration represented by the acceleration data Dab1, and therefore, is capable of calculating a direction (attitude) of the terminal device6with respect to the direction of gravity.

After the calculation of the attitude of the terminal device6based on the acceleration, the CPU10corrects the attitude based on the angular velocity, by using the attitude based on the acceleration. Specifically, the CPU10performs a correction to approximate, at a predetermined rate, the attitude of the terminal device6calculated based on the angular velocity to the attitude of the terminal device6calculated based on the acceleration. The predetermined rate may be a preset fixed value, or may be set according to, for example, the acceleration represented by the acceleration data Dab1. Regarding the attitude of the terminal device6calculated based on the acceleration, since an attitude cannot be calculated with respect to the direction of rotation around the direction of gravity, the CPU10may not perform correction of the attitude with respect to this rotation direction. When correcting the attitude of the terminal device6calculated based on the angular velocity, based on the direction of magnetism detected by the magnetic sensor602, the CPU10approximates, at a predetermined rate, the attitude of the terminal device6calculated based on the angular velocity to the attitude of the terminal device6calculated based on the direction of magnetism. Thus, the CPU10is capable of accurately calculating an attitude of the terminal device6.

Next, the CPU10sets a player object Po (step83), and proceeds to the next step. Hereinafter, a player object setting process in step83will be described with reference toFIG. 14.

InFIG. 14, the CPU10calculates a horizontal angular difference between the real space reference direction and the current direction (step120), and proceeds to the next step. The horizontal angular difference is an angular difference (e.g., an angle A shown inFIG. 16) which is obtained by projecting an angular difference between the real space reference direction and the current direction in the real space on a horizontal plane. The horizontal angular difference indicates an angle at which the direction of the terminal device6(the direction in which the back surface of the terminal device6faces (the z-axis positive direction shown inFIG. 3)) is changed from the initial attitude of the terminal device6around the vertical direction in the real space. For example, the CPU10calculates the horizontal angular difference by using the real space reference direction represented by the real space reference direction data Dae1, and the current direction represented by the current direction data Dae2.

Next, the CPU10calculates an operation instruction direction with respect to the virtual world reference direction in accordance with the horizontal angular difference and the attitude (up-down angle) of the terminal device6in the up-down direction (step121), and proceeds to the next step. For example, as shown inFIG. 16, the CPU10calculates, based on the virtual world reference direction represented by the virtual world reference direction data Daf1, an operation instruction direction in the virtual world such that an angular difference which occurs when the virtual world reference direction and the operation instruction direction are projected on the horizontal plane in the virtual world is equal to the horizontal angular difference, and the same positional relation is achieved between the real space and the virtual world (that is, if the current direction rotates to the left with respect to the real space reference direction, the operation instruction direction also rotates to the left with respect to the virtual world reference direction). Please refer to angle A shown inFIG. 16. Further, as shown inFIG. 17, the CPU10calculates, based on the current direction represented by the current direction data Dae2, an operation instruction direction in the virtual world such that an angular difference between the horizontal direction in the virtual world and the operation instruction direction is equal to an angular difference between the horizontal direction in the real space and the current direction, and the same positional relation is achieved between the real space and the virtual world (that is, if the current direction is downward with respect to the horizontal direction in the real space, the operation instruction direction is also downward with respect to the horizontal direction in virtual world). Please refer to angle C inFIG. 17. Then, the CPU10updates the operation instruction direction data Daft by using the calculated operation instruction directions.

Next, the CPU10determines whether or not the player object Po moves (step122). For example, when a movement vector is set in the movement vector data Dah, the CPU10determines that the player object Po moves. When the player object Po moves, the CPU10proceeds to step123. On the other hand, when the player object Po does not move, the CPU10proceeds to step129.

In step123, the CPU10calculates a rotation movement angle in accordance with the horizontal angular difference calculated at step120, and proceeds to the next step. For example, the CPU10calculates a rotation movement angle B (angle B shown inFIG. 16) in accordance with the following equation:
B=A×a1
where A is the horizontal angular difference, and a1 is a positive constant (e.g., 0.001) smaller than 1.

Next, the CPU10changes the virtual world reference direction in accordance with the rotation movement angle calculated at step123(step124), and proceeds to the next step. For example, when the virtual world reference direction represented by the virtual world reference direction data Daf1is projected on the horizontal plane in the virtual world, the CPU10changes the virtual world reference direction such that the virtual world reference direction approaches the operation instruction direction by the rotation movement angle, and updates the virtual world reference direction data Daf1by using the changed virtual world reference direction.

Next, the CPU10sets a direction of the player object Po in the virtual world, based on the operation instruction direction represented by the operation instruction direction data Daf2(step125), and proceeds to the next step. For example, the CPU10updates the action/attitude data Dag by using the operation instruction direction represented by the operation instruction direction data Daft as the direction of the player object Po. Accordingly, the direction of the player object Po in the virtual world is set based on the attitude of the terminal device6in the real space. For example, when the user turns the back surface of the terminal device6upward, the direction of the player object Po is set at an elevation angle according to an angle at which the back surface faces upward with respect to the vertical direction. When the user turns the back surface of the terminal device6downward, the direction of the player object Po is set at a depression angle according to an angle at which the back surface faces downward with respect to the vertical direction. Further, a left-right direction of the player object Po is set based on a left-right direction change of the terminal device6with respect to the real space reference direction. For example, when the terminal device6rotates in the yaw direction (seeFIG. 3) which is a rotation direction around the y axis, the direction of the player object Po also changes in accordance with the amount of the rotation of the terminal device6. Specifically, when the terminal device6rotates around the y axis to the left as viewed from the y-axis negative direction, the direction of the player object Po also changes to the left as viewed from the player object Po in accordance with the amount of this rotation. When the user maintains the state where the terminal device6is rotated to the left or to the right with respective to the real space reference direction, the virtual world reference direction continuously changes to the left or to the right, and therefore, the operation instruction direction continuously changes to the left or to the right in accordance with the change in the virtual world reference direction when step121is executed. Accordingly, when the user maintains the state where the terminal device6is rotated to the left or to the right with respect to the real space reference direction, the direction of the player object Po continuously changes to the left or to the right. That is, even when the user desires to greatly change the direction of the player object Po (e.g., to change the direction of the player object Po to the opposite direction), the user need not greatly change the direction of the terminal device6to the opposite direction or the like, and thus the operability of the user is enhanced.

Next, the CPU10performs a player object action setting process (step126), and proceeds to the next step. Hereinafter, the player object action setting process at step126will be described with reference toFIG. 15.

InFIG. 15, the CPU10determines whether or not the action mode flag represented by the action mode flag data Dam is set to OFF (step140). When the action mode flag is set to OFF, the CPU10proceeds to step141. When the action mode flag is set to ON, the CPU10proceeds to step148.

In step141, the CPU10determines whether or not the total load value is equal to or greater than a predetermined value. For example, the CPU10determines whether or not the latest total load value represented by the load value data Dac is equal to or greater than the predetermined value. The predetermined value is a threshold value for determining that the user performs as action such as knee banding on the board type controller9. For example, the threshold value is set to a value obtained by adding a predetermined rate (e.g., 20%) to the weight of the user on the board type controller9. The weight of the user on the board type controller9may be calculated based on the total load value (e.g., the weight may be an average of total load values obtained within a predetermined period), or may be previously inputted by the user. When the total load value is equal to or greater than the predetermined value, the CPU10proceeds to step142. When the total load value is smaller than the predetermined value, the CPU10proceeds to step144.

In step142, the CPU10sets the object action to an action of doing a dolphin kick, and proceeds to the next step. For example, when the total load value is equal to or greater than the predetermined value, the CPU10determines that the user performs knee bending on the board type controller9. In this case, the CPU10sets the action of the player object Po to an action of swimming in the dolphin kick style, and to an action of doing a dolphin kick, and then updates the action/attitude data Dag by using the set action.

Next, the CPU10sets the action mode flag to ON (step143), and ends the process of this sub-routine. For example, the CPU10sets the action mode flag represented by the action mode flag data Dam to ON.

On the other hand, when it is determined at step141that the total load value is smaller than the predetermined value, the CPU10determines whether or not the center-of-gravity position represented by the center-of-gravity position data Dad has shifted from outside the right-foot region to inside the right-foot region (step144). When the result of the determination is that the center-of-gravity position has shifted from outside the right-foot region to inside the right-foot region, the CPU10proceeds to step145. On the other hand, when the result of the determination is that the center-of-gravity position has not shifted from outside the right-foot region to inside the right-foot region, the CPU10proceeds to step146.

In the predetermined coordinate system (e.g., an X1Y1coordinate system in which the center of the platform9acorresponds to an origin thereof, the long side direction of the platform9acorresponds to the X1axis direction, and the short side direction corresponds to the Y1axis direction) corresponding to a position on the platform9aof the board type controller9, a region (right-foot region) to which the center-of-gravity position shifts when the user lifts the left foot and puts down the right foot in the stepping action, and a region (left-foot region) to which the center-of-gravity position shifts when the user lifts the right foot and puts down the left foot in the stepping action, are set. For example, the left-foot region and the right-foot region are provided on the left-half part and the right-half part of the platform9a(with the power button9cfacing downward, for example), respectively, and a neutral region having a predetermined width and extending in the front-rear direction is set between the left-foot region and the right-foot region. In the process at step144, the determination result is positive when the latest center-of-gravity position represented by the center-of-gravity position data Dad shifts from outside the right-foot region to inside the right-foot region, such as when the latest center-of-gravity position shifts from inside the neutral region to inside the right-foot region.

It is considered that the user may perform stepping while turning on the board type controller9. In this case, if the positions of the left-foot region, the right-foot region, and the neutral region are fixed, accurate determination of the stepping action cannot be performed. In such a case, the left-foot region, the right-foot region, and the neutral region in the coordinate system may be shifted based on the shift of the center-of-gravity position in the coordinate system. For example, if the distance of shift of the center-of-gravity position within a predetermined period is equal to or longer than a predetermined length, the neutral region is shifted such that, in the center position of the movement trajectory of the center-of-gravity position within the predetermined period, the neutral region extends orthogonally to the movement trajectory, and the left-foot region and the right-foot region are shifted in accordance with the shift of the neutral region. Shifting the left-foot region, the right-foot region, and the neutral region in this way enables accurate determination of the stepping action even when the user performs stepping while turning on the board type controller9.

In the above-described determination of the stepping action, the CPU10determines whether the foot with which the user takes a step on the board type controller9is the left foot or the right foot. However, such distinction between the left foot and the right foot may not be performed. For example, in the case where the process is performed such that the above-mentioned “left foot” is regarded as “one foot” and the “right foot” as “the other foot”, it is possible to make an instruction of an action in which the user's stepping action is synchronized with the flutter kick action of the player object Po although there is a possibility that the foot with which the user takes a step and the foot with which the player object Po does a flutter kick may be reversed to those described above.

In step145, the CPU10sets the object action to an action of doing a flutter kick with its right foot, and ends the process of this sub-routine. For example, when the center-of-gravity position shifts from outside the right-foot region to inside the right-foot region, the CPU10determines that the user performs stepping such that he/she lifts the left foot and puts down the right foot on the board type controller9. In this case, the CPU10sets the action of the player object Po to an action of swimming in the flutter kick style, and to an action of doing a flutter kick with its right foot, and then updates the action/attitude data Dag by using the set action.

In step146, the CPU10determines whether or not the center-of-gravity position represented by the center-of-gravity position data Dad shifts from outside the left-foot region to inside the left-foot region. When the result of the determination is that the center-of-gravity position shifts from outside the left-foot region to inside the left-foot region, the CPU10proceeds to step147. When the result of the determination is that the center-of-gravity position does not shift from outside the left-foot region to inside the left-foot region, the CPU10proceeds to step151

In step147, the CPU10sets the object action to an action of doing a flutter kick with its left foot, and ends the process of this sub-routine. For example, when the center-of-gravity position shifts from outside the left-foot region to inside the left-foot region, the CPU10determines that the user performs stepping such that he/she lifts the right foot and puts down the left foot on the board type controller9. In this case, the CPU10sets the action of the player object Po to an action of swimming in the flutter kick style, and to an action of doing a flutter kick with its left foot, and then updates the action/attitude data Dag by using the set action.

When it is determined at step140that the action mode flag is ON, the CPU10determines whether or not a predetermined time has passed from when the action mode flag was set to ON (step148). When the result of the determination is that the predetermined time has passed, the CPU10proceeds to step149. When the predetermined time has not passed, the CPU10proceeds to step150. The predetermined time is a time from when a dolphin kick is done to when a next dolphin kick operation input or flutter kick operation input is enabled. Until the predetermined time has passed, determination of the user's action by using the board type controller9is not performed. Thereby, it is possible to prevent an erroneous determination of the user's action, which might occur immediately after a dolphin kick.

In step149, the CPU10sets the action mode flag to OFF, and ends the process of this sub-routine. For example, the CPU10sets the action mode flag represented by the action mode flag data Dam to OFF.

On the other hand, in step150, the CPU10sets the object action to “during dolphin kick”, and ends the process of this sub-routine. For example, the CPU10continuously sets the action of the player object Po to an action of swimming in the dolphin kick style, and to “during dolphin kick”, and then updates the action/attitude data Dag by using the set action.

In the case where the action mode flag is OFF (negative determination at step140), the total load value is smaller than the predetermined value (negative determination at step141), the center-of-gravity position does not shift from outside the right-foot region to inside the right-foot region (negative determination at step144), and the center-of-gravity position does not shift from outside the left-foot region to inside the left-foot region (negative determination at step146), the CPU10sets the object action to an action of continuing the currently set action (step151), and ends the process of this sub-routine. For example, when an action of doing a kick (e.g., a flutter kick) is set, the CPU10cancels the action, and maintains the swimming style of the player object Po, which is set as the object action, and then updates the action/attitude data Dag by using the set action.

Referring back toFIG. 14, after the player object action setting process at step126, the CPU10sets a movement vector of the player object Po (step127), and proceeds to the next step. For example, the CPU10calculates a movement speed of the player object Po, based on the time-sequential array of the total load values (the history of variation of the total load value) represented by the load value data Dac, and the action (swimming style) of the player object Po which is set at step126. For example, the shorter the cycle of variation of the total load value is, the higher the movement speed of the player object Po the CPU10calculates. The greater the maximum value or the variation width of the total load value is, the higher the movement speed of the player object Po the CPU10calculates. That is, the shorter the pitch of the user's stepping or knee bending on the board type controller9is, the higher the movement speed of the player object Po is. The greater the force of the user's stepping or knee bending on the board type controller9is, the higher the movement speed of the player object Po is. Then, the CPU10corrects the calculated movement speed by using a coefficient according to the set swimming style, thereby determining a movement speed of the player object Po. The movement speed of the player object Po is set so as to be relatively high at the timing of doing a kick and be gradually lowered until a next kick.

Further, the CPU10sets, as a movement direction of the player object Po, the operation instruction direction in the virtual world, which is represented by the operation instruction direction data Daf2. Then, the CPU10calculates a movement vector in the virtual world by using the calculated movement speed and movement direction of the player object Po, and updates the movement vector data Dah by using the calculated movement vector. For example, the CPU10sets a length of the movement vector by using the calculated movement speed of the player object Po, and sets a direction of the movement vector by using the operation instruction direction represented by the operation instruction direction data Daf2.

Next, the CPU10calculates a position of the player object Po (step128), and ends the process of this sub-routine. For example, the CPU10causes the position of the player object Po represented by the position data Dai to shift in the virtual world, based on the movement vector represented by the movement vector data Dah, and updates the position data Dai by using the position after the shifting.

On the other hand, when it is determined at step122that the player object Po does not move, the CPU10sets a direction of the player object Po in the virtual world, based on the operation instruction direction represented by the operation instruction direction data Daf2(step129), and proceeds to the next step. Since the process at step129is identical to the process at step125, repeated description is not necessary.

Next, the CPU10sets the object action in a static state (step130), and ends the process of this sub-routine. For example, the CPU10sets the action of the player object Po in a static state (e.g., a state in which the player object Po is not swimming but drifting in the sea), and updates the action/attitude data Dag by using the set action.

Referring back toFIG. 13, after the player object setting process at step83, the CPU10sets parameters relating to virtual cameras (step84), and proceeds to the next step. For example, a terminal game image and a monitor game image are each generated as, for example, a three-dimensional CG image which is obtained by calculating a scene of a game space viewed from a virtual camera arranged in a virtual world (virtual space). Specifically, a first virtual camera for generating a terminal game image is set such that a state of the virtual world viewed at close range from behind the player object Po arranged in the virtual world is included in the terminal game image. A second virtual camera for generating a monitor game image is set in the same virtual world in which the first virtual camera is set, such that a state of the virtual world in which the player object Po arranged in the virtual world is viewed from a distance in a manner of bird's eye viewing is included in the monitor game image. The CPU10sets positions of the first virtual camera and the second virtual camera in the virtual world (parameters relating to the first virtual camera and the second virtual camera), based on the position of the player object Po represented by the position data Dai, and the operation instruction direction (the direction of the player object Po) in accordance with the operation instruction direction data Da12. In this way, the terminal game image and the monitor game image are game images of the same virtual world viewed from different viewpoints. Therefore, the game images of the virtual world viewed from the different viewpoints are displayed on the LCD61and the monitor2, respectively. Although the direction of the first virtual camera is set based on the operation instruction direction (the direction of the player object Po), since the operation instruction direction (the direction of the player object Po) is set based on the attitude of the terminal device6as described above, the direction of the first virtual camera is eventually controlled based on the attitude of the terminal device6. That is, the user is allowed to control the first virtual camera for generating the virtual world image displayed on the LCD61of the terminal device6, by moving the body of the terminal device6. Since the attitude and the direction of the terminal device6are associated with the attitude and the direction of the first virtual camera, the user is allowed to experience a sense of looking into the virtual world through the LCD61of the terminal device6.

Next, the CPU10sets a movement trajectory of the player object Po (step85), and ends the process of the sub-routine. For example, the CPU10adds the current position of the player object Po, which is represented by the position data Dai, to the movement trajectory represented by the movement trajectory data Dak, and updates the movement trajectory data Dak by using the movement trajectory to which the current position is added.

Referring back toFIG. 12, after the game control process at step44, the CPU10and the GPU32generate a monitor game image to be displayed on the monitor2(step45), and proceed to the next step. For example, the CPU10and the GPU32read, from the main memory, the data representing the result of the game control process performed at step44, and read, from the VRAM34, the data required for generating a monitor game image. Then, the CPU10and the GPU32generate a game image by using the read data, and store the generated monitor game image in the VRAM34. Any monitor game image may be generated by any method so long as the monitor game image represents the result of the game control process performed at step44. For example, the monitor game image may be a three-dimensional CG image which is generated through the steps of: arranging the second virtual camera in the virtual world based on the parameters relating to the second virtual camera, which are represented by the virtual camera data Daj; arranging the player object Po in the virtual world based on the action/attitude data Dag and the position data Dai; arranging the movement trajectory Lp in the virtual world based on the movement trajectory data Dak; and calculating the scene of the virtual world viewed from the second virtual camera.

Next, the CPU10and the GPU32generate a terminal game image to be displayed on the terminal device6(step46), and proceed to the next step. For example, the CPU10and the GPU32read, from the main memory, the data representing the result of the game control process performed at step44, and read, from the VRAM34, the data required for generating a terminal game image. Then, the CPU10and the GPU32generate a terminal game image by using the read data, and store the generated terminal game image in the VRAM34. Like the monitor game image, any terminal game image may be generated by any method so long as the terminal game image represents the result of the game control process performed at step44. The terminal game image may be generated by the same method as, or a different method from, that for the monitor game image. For example, the terminal game image may be a three-dimensional CG image which is generated through the steps of: arranging the first virtual camera in the virtual world based on the parameters relating to the first virtual camera, which are represented by the virtual camera data Daj; arranging the player object Po in the virtual world based on the action/attitude data Dag and the position data Dai; and calculating the scene of the virtual world viewed from the first virtual camera.

When the action of the player object Po represented by the action/attitude data Dag is an action of doing a kick (action of doing a dolphin kick or a flutter kick), the player object Po may be arranged in the virtual world so as to perform the action of doing a kick. Thereby, it is possible to cause the player object Po to move in synchronization with the action (stepping or knee bending) the user performs on the board type controller9.

Next, the CPU10generates a monitor game sound to be output to the loudspeakers2aof the monitor2(step47), and proceeds to the next step. For example, the CPU10causes the DSP33to generate a monitor game sound to be output from the loudspeakers2a, in accordance with the result of the game control process performed at step44. In an example, the CPU10causes the DSP33to generate a monitor game sound in which BGM or the like to be output from the monitor2is added to voices and action sounds of the objects, sound effects and the like, which are supposed to be heard on the basis of the position of the second virtual camera in the virtual world which is set based on the result of the game control process at step44.

Next, the CPU10generates a terminal game sound to be output to the loudspeakers607of the terminal device6(step48), and proceeds to the next step. For example, the CPU10causes the DSP33to generate a terminal game sound to be output from the loudspeakers607, in accordance with the result of the game control process performed at step44. In an example, the CPU10causes the DSP33to generate a terminal game sound in which BGM or the like to be output from the terminal device6is added to voices and action sounds of the objects, sound effects and the like, which are supposed to be heard on the basis of the position of the first virtual camera in the virtual world which is set based on the result of the game control process at step44. The terminal game sound may be the same as or different from the monitor game sound. Further, the terminal game sound may be partially different from the monitor game sound. For example, the terminal game sound and the monitor game sound may include the same BGM and different sound effects. When the monitor game sound and the terminal game sound are the same, the terminal game sound generation step at step48need not be performed.

Next, the CPU10outputs the monitor game image and the monitor game sound to the monitor2(step49), and proceeds to the next step. For example, the CPU10transmits, to the AV-IC15, the data of the monitor game image stored in the VRAM34and the data of the monitor game sound generated by the DSP33. The AV-IC15transmits the data of the monitor game image and the data of the monitor game sound through the AV connector16to the monitor2. Thereby, the monitor game image is displayed on the monitor2, and the monitor game sound is output from the loudspeakers2a.

Next, the CPU10transmits the terminal game image and the terminal game sound to the terminal device6(step50), and proceeds to the next step. For example, the CPU10transmits, to the codec LSI27, the data of the terminal game image stored in the VRAM34and the data of the terminal game sound generated by the DSP33. The codec LSI27performs a predetermined compression process on these data. The compressed data of the terminal game image and the compressed data of the terminal game sound are transmitted from the codec LSI27to the terminal communication module28, and then transmitted from the terminal communication module28to the terminal device6via the antenna29. The data of the terminal game image and the data of the terminal game sound transmitted from the game apparatus body5are received by the wireless module610of the terminal device6, and are subjected to a predetermined decompression process by the codec LSI606. The decompressed data of the terminal game image is output to the LCD61, and the decompressed data of the terminal game sound is output to the sound IC608. Thereby, the terminal game image is displayed on the LCD61, and the terminal game sound is output from the loudspeakers607.

Next, the CPU10determines whether to end the game (step51). The game is to be ended, for example, when conditions for game over or game clear are satisfied, or when the user has performed an operation to end the game. When the game is not to be ended, the CPU10returns to step42and repeats the processing, whereas when the game is to be ended, the CPU10ends the processing of the flowchart. Thereafter, the series of process steps42to51are repeatedly executed until the CPU10determines to end the game at step51.

According to the above-mentioned processing, when the user performs an operation based on his/her action on the board type controller9and thereby a process based on the user's action is performed, the user is allowed to view the result of the process on the LCD61of the portable terminal device6held by the user. Therefore, the user, who performs an action with the terminal device6in his/her hands, is allowed to view the result of the process in a favorable situation. Further, the user is allowed to control the virtual camera, and the attitude and the movement direction of the player object Po by moving the portable terminal device6. Therefore, it is possible to provide the user with a sense of presence in the virtual world, and with an operation sense of becoming the player object Po.

Second Embodiment

The game played by using the terminal device6and the board type controller9is not limited to the above-mentioned first exemplary game, but various exemplary games are considered. In the following description of the outline of information processing according to a second embodiment, a second exemplary game is adopted as an example of the information processing. Since an information processing apparatus for executing an information processing program according to the second embodiment, and an information processing system including the information processing apparatus are identical to those of the first embodiment, the same components as those of the first embodiment are designated by the same reference numerals, and the detailed description thereof is omitted.

Firstly, an outline of information processing performed on the game apparatus body5will be described with reference toFIGS. 18A, 18B, 19, and 20before a specific description of process steps performed by the game apparatus body5is given.FIG. 18Ashows an example of an image displayed on the LCD61of the terminal device6in the second exemplary game.FIG. 18Bshows an example of an image displayed on the monitor2in the second exemplary game.FIG. 19shows another example of an image displayed on the LCD61of the terminal device6in the second exemplary game.FIG. 20is a schematic diagram illustrating an example in which the position and the attitude of a virtual camera vary depending on the attitude of the terminal device6.

In the second exemplary game, as in the first exemplary game, a user performs an operation by using the terminal device6and the board type controller9. Specifically, as in the first exemplary game, the user stands on the board type controller9with the terminal device6in his/her hands, and performs an operation. Then, the user performs an action on the board type controller9(e.g., the user performs knee bending on the board type controller9, or adjusts the center-of-gravity position of a load applied to the board type controller9), and simultaneously performs an operation to move the terminal device6, while viewing the image displayed on the monitor2and the image displayed on the LCD61of the terminal device6, thereby playing the game. On each of the LCD61of the terminal device6and the monitor2, a game image is displayed, in which a player object Po and a virtual camera in a virtual world each perform an action (e.g., an action of the player object Po to change its attitude and movement direction, and an action of the virtual camera to change its position and attitude) in accordance with the direction and the attitude of the terminal device held by the user, and the action of the user on the board type controller9, and furthermore, the position and the attitude of the virtual camera set in the virtual world are changed in accordance with the position and the attitude of the player object Po and/or the action of the user.

As shown inFIG. 18A, on the LCD61of the terminal device6, a player object Po moving in a virtual world is displayed. In the example shown inFIG. 18A, a virtual camera is located near the back of the player object Po which rides on a pogo stick on the ground in the virtual world, and a state of the virtual world viewed from the virtual camera is displayed together with the player object Po. Displaying, on the LCD61, the virtual world viewed at close range from behind the player object Po allows the user holding the terminal device6to experience a sense of presence in the virtual world, and to intuitively know the movement direction and the movement speed of the player object Po.

Further, as shown inFIG. 18B, the same virtual world as that displayed on the LCD61is displayed on the monitor2. In the example shown inFIG. 18B, a state of the virtual world in which the player object Po on the pogo stick is viewed from a distance in a manner of bird's eye viewing, is displayed together with the player object Po. Displaying, on the monitor2, the state of the virtual world in which the player object Po is viewed from a distance allows the user to easily know the surroundings of the player object Po, and allows another person who watches the user's playing the second exemplary game to enjoy a view in which the player object Po moves in the virtual world.

For example, when the user performs an action such as knee bending (jumping) on the board type controller9, the player object Po jumps with the pogo stick at a speed according to the knee bending. When the user performs an action to shift the center-of-gravity position of the load applied to the board type controller9while the player object Po is in the air in the virtual world, the movement direction of the player object Po in the air is changed to a direction according to the shift of the center-of-gravity position. In this way, the user is allowed to change the jumping power (movement speed) and the movement direction of the player object Po in accordance with his/her action on the board type controller9.

When the user changes the attitude of the terminal device6or rotates the terminal device6to change its direction, the movement direction in which the player object Po jumps out, or the position and the attitude of the virtual camera is changed. For example, when the user turns the terminal device6to the left or to the right in the real space, the direction in which the player object Po moves in the virtual world changes in accordance with the direction change of the terminal device6. Further, in accordance with the change in the movement direction, the position and the attitude of the virtual camera change such that a horizontal line-of-sight direction obtained by projecting the line-of-sight direction of the virtual camera on the horizontal plane in the virtual world becomes equal to a horizontal movement direction obtained by projecting the movement direction on the horizontal plane in the virtual world. Specifically, when the user changes the direction of the terminal device6such that the back surface thereof turns to the left, the movement direction of the player object Po changes to the leftward direction in the virtual world. Then, the line of sight of the virtual camera turns to the movement direction, and the position and the attitude of the virtual camera change such that the virtual camera is located near and behind the player object Po.

When the user turns the terminal device6upward or downward in the real space, the position and the attitude of the virtual camera change in accordance with the direction change of the terminal device6. For example, as shown inFIG. 20, when the user holds the terminal device6in such an attitude that the back surface of the terminal device6faces the horizontal direction in the real space (i.e., the back surface of the terminal device6or the display surface of the LCD61is orthogonal to the horizontal direction), the virtual camera is located in a subjective position near the back of the player object Po in such an attitude that the line-of-sight direction of the virtual camera is equal to the horizontal direction in the virtual world. In this case, a virtual world as shown inFIG. 18Ais displayed on the LCD61of the terminal device6. On the other hand, when the user holds the terminal device6in such an attitude that the back surface of the terminal device6faces downward in the real space (i.e., the back surface of the terminal device6or the display surface of the LCD61is orthogonal to the vertical direction, and the back surface faces downward), the virtual camera is located in a position (overhead position) in which the line of sight of the virtual camera is equal to the vertical direction in the virtual world and the virtual camera views the player object Po from above, and the point of view is in front of the player object Po in the movement direction. In this case, a virtual world as shown inFIG. 19is displayed on the LCD61of the terminal device6. When the user changes the direction of the terminal device6from the horizontal direction to the downward direction in the real space, the attitude and the position of the virtual camera also change from the subjective position to the overhead position in accordance with the attitude of the terminal device6.

In this way, by associating the attitude and the direction of the terminal device6with the attitude and the direction of the virtual camera, the user is allowed to experience a sense of looking into the virtual world through the LCD61of the terminal device6. Further, when the user holds the terminal device6in such an attitude that the back surface of the terminal device6faces downward in the real space, the feet of the player object Po are displayed on the LCD61of the terminal device6, which allows the user to check the landing point of the player object Po or surroundings of the feet of the player object Po. That is, when the user changes the up-down direction of the terminal device6, the user is allowed to secure a field of view in the movement direction of the player object Po, or a field of view toward the feet of the player object Po, and furthermore, allowed to selectively display his/her desired image on the LCD61in accordance with the game situation.

Next, the process of the second embodiment, which is performed by the game system1, will be described in detail. Firstly, main data used in the process according to the second embodiment will be described with reference toFIG. 21.FIG. 21is a diagram illustrating an example of main data and programs stored in the main memory of the game apparatus body5of the second embodiment.

As shown inFIG. 21, in a data storage area of the main memory, board operation data Dba, terminal operation data Dbb, load value data Dbc, center-of-gravity position data Dbd, terminal device direction/attitude data Dbe, operation direction data Dbf, action/attitude data Dbg, movement vector data Dbh, position data Dbi, virtual camera data Dbj, jumping power data Dbk, jump flag data Dbm, squat flag data Dbn, image data Dbo, and the like are stored. In addition to the data shown inFIG. 21, data required for a game process, such as image data of various objects displayed on the monitor2and the LCD61, and sound data used for a game, are optionally stored in the main memory. In a program storage area of the main memory, various programs Pb constituting an information processing program of the second embodiment are stored.

The board operation data Dba includes load data Dba1and the like. Since the load data Dba1is identical to the load data Daa1of the first embodiment, repeated description is not necessary.

The terminal operation data Dbb includes acceleration data Dbb1, angular velocity data Dbb2, and the like. Since the acceleration data Dbb1and the angular velocity data Dbb2are identical to the acceleration data Dab1and the angular velocity data Dab2of the first embodiment, respectively, repeated description is not necessary.

Since the load value data Dbc and the center-of-gravity position data Dbd are identical to the load value data Dac and the center-of-gravity position data Dad of the first embodiment, respectively, repeated description is not necessary.

The terminal device direction/attitude data Dbe includes real space reference direction data Dbe1, current direction data Dbe2, and the like. Since the real space reference direction data Dbe1and the current direction data Dbe2are identical to the real space reference direction data Dae1and the current direction data Dae2of the first embodiment, respectively, repeated description is not necessary.

The operation direction data Dbf includes virtual world reference direction data Dbf1, operation instruction direction data Dbf2, and the like. The virtual world reference direction data Dbf1is data representing a virtual world reference direction set in the virtual world. The operation instruction direction data Dbf2is data representing an operation instruction direction which is currently instructed in the virtual world by the user. Methods for calculating the virtual world reference direction and the operation instruction direction of the second embodiment will be described later.

The action/attitude data Dbg, the movement vector data Dbh, the position data Dbi, and the virtual camera data Dbj are, as in the first embodiment, data representing the action and the attitude of the player object Po, data representing the movement speed and the movement direction of the player object Po, data representing the position of the player object Po, and data relating to the virtual camera, respectively, although the calculation methods thereof are different from those of the first embodiment as described later.

The jumping power data Dbk is data representing a jumping power (movement speed) of the player object Po, which is calculated based on the user's action on the board type controller9.

The jump flag data Dbm is data representing a jump flag which is turned on while the player object Po is jumping. The squat flag data Dbn is data representing a squat flag which is turned on when it is determined that the user squats on the board type controller9.

The image data Dbo includes player object data Dbo1, background image data Dbo2, and the like. The player object data Dbo1is data for arranging the player object Po of the second exemplary game in the virtual world to generate a game image. The background image data Dbo2is data for arranging a background of the second exemplary game in the virtual world to generate a game image.

Next, the process of the second embodiment, which is performed by the game apparatus body5, will be described in detail with reference toFIGS. 22 to 24.FIG. 22is a sub-routine illustrating an example of a player object setting process of the second embodiment executed by the game apparatus body5.FIG. 23is a sub-routine illustrating an example of a jumping operation setting process at step165inFIG. 22.FIG. 24is a diagram illustrating an example of “squat” determination and “stretch” determination with respect to variation in the total load value. In the flowcharts ofFIGS. 22 and 23, process steps for causing the player object Po to move and displaying the same, in accordance with an operation performed by a user using the terminal device6and the board type controller9, will be mainly described among the whole process, while detailed description of other process steps is not given. Further, inFIGS. 22 to 23, each of the steps executed by the CPU10is abbreviated as “S”.

When the game apparatus body5is powered on, the CPU10of the game apparatus body5executes a boot program stored in the ROM/RTC13to initialize each unit such as the main memory. Then, the information processing program stored in the optical disc4is loaded to the main memory, and the CPU10starts to execute the program. The process of the second embodiment to be performed after completion of the above-mentioned process steps is identical to the process of the first embodiment except the game control process at step44in the main flow shown inFIG. 12. Therefore, the identical part of the process is not repeatedly described. Further, the game control process at step44according to the second embodiment is identical to the sub-routine ofFIG. 13of the first embodiment except the player object setting process at step83, and except that the movement trajectory setting process at step85is not performed. Therefore, the identical part of the process is not repeatedly described. Hereinafter, the player object setting process at step83according to the second embodiment will be described with reference toFIG. 22.

InFIG. 22, the CPU10calculates a horizontal angular difference between the real space reference direction and the current direction (step161), and proceeds to the next step. The horizontal angular difference is an angular difference which is obtained by projecting an angular difference between the real space reference direction and the current direction in the real space on the horizontal plane, as in the first embodiment. The horizontal angular difference indicates an angle at which the direction of the terminal device6(the direction in which the back surface of the terminal device6faces (the z-axis positive direction shown inFIG. 3)) is changed from the initial attitude of the terminal device6around vertical direction in the real space. For example, the CPU10calculates a horizontal angular difference by using the real space reference direction represented by the real space reference direction data Dbe1, and the current direction represented by the current direction data Dbe2.

Next, the CPU10calculates an operation instruction direction with respect to the virtual world reference direction, in accordance with the horizontal angular difference and the attitude (up/down angle) of the terminal device6in the up-down direction (step162), and proceeds to the next step. For example, the CPU10calculates, based on the virtual world reference direction represented by the virtual world reference direction data Dbf1, an operation instruction direction in the virtual world such that an angular difference which occurs when the virtual world reference direction and the operation instruction direction are projected on the horizontal plane in the virtual world is equal to the horizontal angular difference, and the same positional relation is achieved between the real space and the virtual world (i.e., if the current direction rotates to the left with respect to the real space reference direction, the operation instruction direction also rotates to the left with respect to the virtual world reference direction). Further, the CPU10calculates, based on the current direction represented by the current direction data Dbe2, an operation instruction direction in the virtual world such that an angular difference between the horizontal direction and the operation instruction direction in the virtual world is equal to an angular difference between the horizontal direction and the current direction in the real space, and the same positional relation is achieved between the real space and the virtual world (that is, if the current direction is the downward direction with respect to the horizontal direction in the real space, the operation instruction direction is also the downward direction with respect to the horizontal direction in virtual world). Then, the CPU10updates the operation instruction direction data Dbf2by using the calculated operation instruction direction.

In the setting of the operation instruction direction at step162, a range in which the operation instruction direction can be set may be limited. For example, the operation instruction direction may be set within a predetermined angular range on the left side and the right side of the virtual world reference direction with respect to the horizontal direction in the virtual world, and within a predetermined angular range on the upper side and the lower side of the horizontal direction with respect to the vertical direction in the virtual world. In this case, if the calculated operation instruction direction is outside the above-mentioned angular range, the operation instruction direction is shifted into the angular range closest to the calculated operation instruction direction. In another example, a current direction may be set within a predetermined angular range on the left side and the right side of the real space reference direction with respect to the horizontal direction in the real space, and within a predetermined angular range on the upper side and the lower side of the horizontal direction with respect to the vertical direction in the real space. If the calculated current direction is outside the above-mentioned angular range, the current direction is shifted into the angular range closest to the calculated current direction. Also in this case, since the operation instruction direction is calculated based on the current direction limited within the angular range, the operation instruction direction is also set within the angular ranges set in the upward, downward, leftward, or rightward direction.

Next, the CPU10determines whether or not the jump flag is set to OFF (step163). For example, when the jump flag represented by the jump flag data Dbm is set to OFF, the CPU10determines that the player object Po is not jumping, and proceeds to step164. On the other hand, when the jump flag represented by the jump flag data Dbm is set to ON, the CPU10determines that the player object Po is jumping, and proceeds to step166.

In step164, the CPU10sets the object action in a pre jumping state, and proceeds to next step. For example, the CPU10sets the action of the player object Po in a pre jumping state (e.g., a state in which the player object Po lands on the ground in the virtual world and tries to take a next jump), and updates the action/attitude data Dbg by using the set action.

Next, the CPU10performs a jumping action setting process (step165), and ends the process of this sub-routine. Hereinafter, the jumping action setting process performed at step165will be described with reference toFIG. 23.

InFIG. 23, the CPU10determines whether or not the squat flag represented by the squat flag data Dbn is set to OFF (step181). When the squat flag is set to OFF, the CPU10proceeds to step182. On the other hand, when the squat flag is set to ON, the CPU10proceeds to step185.

In step182, the CPU10determines whether or not the total load value is equal to or greater than a first threshold. For example, the CPU10determines whether or not the latest total load value represented by the load value data Dbc is equal to or greater than the first threshold. The first threshold is a threshold value for determining that the user performs an action of squatting down on the board type controller9. For example, the first threshold is set at a value obtained by adding a predetermined rate (e.g., 20%) to the weight of the user on the board type controller9. The weight of the user on the board type controller9may be calculated based on the total load value (e.g., an average of total load values obtained in a predetermined period), or may be previously inputted by the user. When the total load value is equal to or greater than the first threshold, the CPU10proceeds to step183. On the other hand, when the total load value is smaller than the first threshold, the CPU10proceeds to step185.

When the user performs an action of bending and stretching (jumping) on the board type controller9, it is general that the user firstly takes an action of squatting on the board type controller9and then takes an action of stretching upward. As shown inFIG. 24, when the user takes an action of squatting on the board type controller9, the total load value once falls below the weight of the user and thereafter exceeds the weight. That is, the CPU10can detect that the user has performed an action of squatting, by detecting that the total load value exceeds the weight of the user (the total load value becomes equal to or greater than the first threshold).

In step183, the CPU10sets the object action to the squatting action, and proceeds to the next step. For example, the CPU10sets the action of the player object Po to the squatting action (e.g., a state in which the player object Po squats on the pogo stick for a next jump, with the pogo stick landing on the ground in the virtual world), and updates the action/attitude data Dbg by using the set action.

Next, the CPU10sets the squat flag to ON (step184), and proceeds to step185. For example, the CPU10sets the squat flag represented by the squat flag data Dbn to ON.

In step185, the CPU10determines whether or not the squat flag represented by the squat flag data Dbn is set to ON. When the squat flag is set to ON, the CPU10proceeds to step186. On the other hand, when the squat flag is set to OFF, the CPU10ends the process of this sub-routine.

In step186, the CPU10determines whether or not the total load value is smaller than a second threshold. For example, the CPU10determines whether or not the latest total load value represented by the load value data Dbc is smaller than the second threshold. The second threshold is a threshold value for determining that the user performs an action of stretching on the board type controller9. For example, the second threshold is set at a value obtained by subtracting a predetermined rate (e.g., 20%) from the weight of the user on the board type controller9. When the total load value is equal to or greater than the second threshold, the CPU10proceeds to step187. On the other hand, when the total load value is smaller than the second threshold, the CPU10proceeds to step188.

When the user performs an action of bending and stretching (jumping) on the board type controller9, it is general that the user firstly takes an action of squatting on the board type controller9and then takes an action of stretching upward. As shown inFIG. 24, when the user takes an action of stretching on the board type controller9, the total load value once exceeds the weight of the user and thereafter falls below the weight. That is, the CPU10can detect that the user has performed an action of stretching, by detecting that the total load value falls below the weight of the user (the total load value becomes smaller than the second threshold) after the detection of the squatting action.

In step187, the CPU10adds a jumping power, and ends the process of this sub-routine. For example, the CPU10adds a predetermined amount to the parameter of the jumping power represented by the jumping power data Dbk, and updates the jumping power data Dbk by using the parameter of the jumping power after the addition. The process at step187is performed when the determination result is positive (i.e., the squat flag is ON) at step185, and the determination result is negative (i.e., the total load value is equal to or greater than the second threshold) at step186. That is, the process at step187is performed during a period from when the user squats to when it is determined that the user stretches. The longer the period (“squat” period) is, the more the jumping power is increased gradually.

On the other hand, in step188, the CPU10sets the object action to the stretching action, and proceeds to the next step. For example, the CPU10sets the action of the player object Po to the stretching action (e.g., an action of jumping into the air with the pogo stick from the state in which the pogo stick lands on the ground in the virtual world), and updates the action/attitude data Dbg by using the set action.

Next, the CPU10sets a movement vector of the player object Po, based on the operation instruction direction and the jumping power (step189), and proceeds to the next step. For example, the CPU10calculates a movement speed of the player object Po, based on the parameter of the jumping power represented by the jumping power data Dbk. In an example, the CPU10calculates a movement speed of the player object Po so as to be proportional to the parameter of the jumping power. Further, the CPU10sets a movement direction of the player object Po, based on the operation instruction direction represented by the operation instruction direction data Dbf2. For example, a movement direction is set such that a horizontal movement direction obtained by projecting the movement direction on the horizontal plane in the virtual world is equal to a horizontal operation instruction direction obtained by projecting the operation instruction direction on the horizontal plane in the virtual world. Further, a climb angle in the movement direction, at which the player object Po moves upward in the virtual world, is set to a specific angle. Then, the CPU10calculates a movement vector in the virtual world by using the movement speed and the movement direction of the player object Po, and updates the movement vector data Dbh by using the calculated movement vector. For example, the CPU10sets a length of the movement vector by using the calculated movement speed of the player object Po, and sets a direction of the movement vector based on the operation instruction direction represented by the operation instruction direction data Dbf2.

Next, the CPU10initializes the jumping power (step190), and proceeds to the next step. For example, the CPU10initializes the parameter of the jumping power represented by the jumping power data Dbk to a predetermined initial value, thereby to update the jumping power data Dbk. The initial value at step190may be a value indicating that the jumping power is 0, or may be a value indicating a relatively low jumping power.

Next, the CPU10sets the squat flag to OFF (step191), and proceeds to the next step. For example, the CPU10sets the squat flag represented by the squat flag data Dbn to OFF.

Next, the CPU10sets the jump flag to ON (step192), and ends the process of this sub-routine. For example, the CPU10sets the jump flag represented by the jump flag data Dbm to ON.

Referring back toFIG. 22, when it is determined at step163that the jump flag is ON, the CPU10sets the object action to “during jumping” (step166), and proceeds to the next step. For example, the CPU10sets the action of the player object Po to “during jumping” (e.g., a state in which the player object Po on the pogo stick moves in the air in the virtual world), and updates the action/attitude data Dbg by using the set action.

Next, the CPU10corrects the movement vector of the player object Po based on the center-of-gravity position (step167), and proceeds to the next step. For example, the CPU10corrects the movement vector represented by the movement vector data Dbh by using the latest center-of-gravity position represented by the center-of-gravity position data Dbd, and updates the movement vector data Dbh by using the corrected movement vector. In an example, the CPU10calculates a correction vector extending from the center of the platform9ato the center-of-gravity position, and adds the correction vector to the movement vector that is currently set to correct the movement vector. The length of the correction vector may be a value obtained by multiplexing the distance between the center of the platform9aand the center-of-gravity position by a predetermined constant, or may be a fixed value. For example, if the center-of-gravity position is present on a position shifted by a distance D from the center of the platform9ato the left, a correction vector having a length according to the distance D in the leftward direction in the virtual world is calculated, and added to the currently set movement vector. Thereby, the movement vector is corrected toward the position where the center-of-gravity position is present by an amount according to the distance between the center of the platform9aand the center-of-gravity position.

Next, the CPU10corrects the movement vector of the player object Po based on the environment set in the virtual world (step168), and proceeds to the next step. For example, the CPU10corrects the movement vector represented by the movement vector data Dbh, based on the environment (gravity, wind, influence of other objects, and the like) set in the virtual world, and updates the movement vector data Dbh by using the corrected movement vector.

Next, the CPU10calculates a position of the player object Po (step169), and proceeds to the next step. For example, the CPU10causes the position of the player object Po represented by the position data Dbi to shift in the virtual world, based on the movement vector represented by the movement vector data Dbh, and updates the position data Dbi by using the position after the shifting.

Next, the CPU10determines whether or not the player object Po has landed from the air in the virtual world (step170). For example, the CPU10determines whether or not the player object Po has landed (e.g., the position of the player object Po is on a topography object or another object in the virtual world) based on the position of the player object Po represented by the position data Dbi. When the result of the determination is that the player object Po has landed, the CPU10proceeds to step171. On the other hand, when the result of the determination is that the player object Po has not landed, the CPU10ends the process of this sub-routine.

In step171, the CPU10initializes the movement vector of the player object Po, and proceeds to the next step. For example, the CPU10initializes the movement vector represented by the movement vector data Dbh to update the movement vector data Dbh. In step171, the movement vector may be initialized to a parameter indicating that the player object Po does not move, or may be initialized to a predetermined movement vector.

Next, the CPU10sets the jump flag to OFF (step172), and ends the process of this sub-routine. For example, the CPU10sets the jump flag represented by the jump flag data Dbm to OFF.

In the virtual camera setting process (step84) of the second embodiment, a first virtual camera for generating a terminal game image is set such that a first player object Po1arranged in the virtual world is included in the display range, based on the operation instruction direction represented by the operation instruction direction data Dbf2. Specifically, as described with reference toFIG. 20, when the operation instruction direction is the horizontal direction in the virtual world, the operation instruction direction is regarded as the line-of-sight direction, and the player object Po is arranged substantially in the center of the view volume, and the first virtual camera is set at the subjective position near the player object Po. When the operation instruction direction is the vertical direction in the virtual world, the operation instruction direction is regarded as the line-of-sight direction, and a position in front of the player object Po in the movement direction of the player object Po is regarded as a point of view, and the first virtual camera is set at the overhead position at which the first virtual camera views the player object Po from above. When the operation instruction direction is between the horizontal direction and the vertical direction, the attitude and the position of the first virtual camera are also changed from the subjective position to the overhead position in accordance with the direction of the operation instruction direction. On the other hand, a second virtual camera for generating a monitor game image is set in the same virtual world as the first virtual camera such that a state of the virtual world in which the player object Po is viewed from a distance in a manner of bird's eye viewing is included in the monitor game image. The CPU10sets the positions of the first virtual camera and the second virtual camera in the virtual world (the parameters of the first virtual camera and the second virtual camera), based on the position of the player object Po represented by the position data Dbi and the operation instruction direction represented by the operation instruction direction data Dbf2.

In the monitor game image generation process (step45) and the terminal game image generation process (step46) of the second embodiment, images representing the result of the game control process are generated, respectively, as in the first embodiment. For example, the monitor game image is generated as a three-dimensional CG image which is generated through the steps of: arranging the second virtual camera in the virtual world based on the parameters relating to the second virtual camera, which are represented by the virtual camera data Dbj; arranging the player object Po and the like in the virtual world based on the action/attitude data Dbg and the position data Dbi; and calculating a scene of the virtual world viewed from the second virtual camera. The terminal game image is generated as a three-dimensional CG image which is generated through the steps of: arranging the first virtual camera in the virtual world based on the parameters relating to the first virtual camera, which are represented by the virtual camera data Dbj; arranging the player object Po and the like in the virtual world based on the action/attitude data Dbg and the position data Dbi; and calculating a scene of the virtual world viewed from the first virtual camera.

In this way, according to the above-mentioned process, when the user performs an operation based on his/her action on the board type controller9and thereby a process based on the user's action is performed, the user is allowed to view the result of the process on the LCD61of the portable terminal device6held by the user. Therefore, the user, who performs an action with the terminal device6in his/her hands, is allowed to view the result of the process in a favorable situation for the user. Further, since the user is allowed to control the position and the attitude of the virtual camera or the attitude and the movement direction of the player object Po by moving the portable terminal device6, it is possible to provide the user with a sense of presence in the virtual world, and with an operation sense of becoming the player object Po. Furthermore, in the second exemplary game, the movement direction of the player object Po is determined based on the attitude of the portable terminal device6, but the movement direction after the player object Po has jumped is adjusted in accordance with the action of the user on the board type controller9. Accordingly, the movement direction of the player object Po can be changed not only by an operation using the terminal device6but also by an operation using the board type controller9, which provides the user with a new operation sense.

Third Embodiment

Hereinafter, information processing according to a third embodiment will be described based on another exemplary game which is played by using the terminal device6and the board type controller9. In the following description of the outline of the information processing according to the third embodiment, a third exemplary game is adopted as an example of the information processing. Since an information processing apparatus for executing an information processing program according to the third embodiment, and an information processing system including the information processing apparatus are identical to those of the first embodiment, the same components as those of the first embodiment are designated by the same reference numerals, and the detailed description thereof is omitted.

Firstly, an outline of information processing performed on the game apparatus body5will be described with reference toFIGS. 25A, 25B, and 26before a specific description of processes of the third embodiment performed by the game apparatus body5is given. In the following description of the outline of the information processing according to the third embodiment, a third exemplary game is adopted as an example of the information processing.FIG. 25Ashows an example of an image displayed on the LCD61of the terminal device6in the third exemplary game.FIG. 25Bshows an example of an image displayed on the monitor2in the third exemplary game.FIG. 26is a schematic diagram illustrating an example of actions of a first player object Po1and a second player object Po2in accordance with the attitude of the terminal device6.

In the third exemplary game, as in the first exemplary game shown inFIG. 9, a user performs an operation by using the terminal device6and the board type controller9. Specifically, as in the first exemplary game, the user stands on the board type controller9with the terminal device6in his/her hands, and performs an operation. Then, the user performs an action on the board type controller9(e.g., an action of placing his/her weight forward and backward on the board type controller9), and performs an operation to move the terminal device6, while viewing the image displayed on the monitor2and the image displayed on the LCD61of the terminal device6, thereby playing the game. On the LCD61of the terminal device6and on the monitor2, a game image is displayed, in which the first player object Po1, the second player object Po2, and a target object To each perform an action in a virtual world (e.g., an action of the first player object Po1to change its attitude and movement direction, an action of the second player object Po2to change its direction, or an action of the target object To to move) in accordance with the direction and the attitude of the terminal device held by the user, and the action of the user on the board type controller9, and furthermore, the position and the attitude of a virtual camera set in the virtual world are changed in accordance with the position and the attitude of the first player object Po1.

As shown inFIG. 25A, on the LCD61of the terminal device6, a player object Po moving in a virtual world is displayed. In the example shown inFIG. 25A, the player object Po includes two player objects (a first player object Po1and a second player object Po2). As an example of the player object Po, a tank moving on the ground in the virtual world is displayed. Specifically, the first player object Po1corresponds to the body of the tank, and the second player object Po2corresponds to a turret of the tank. A virtual camera is located near the back of the first player object Po, and a state of the virtual world viewed from the virtual camera is displayed together with the player object Po. Further, in the example shown inFIG. 25A, the target object To, which is a target of a shell shot by the tank, is arranged in the virtual world, and an expected trajectory S of the shell from a front end of the turret (second player object Po2) of the tank to the target object to is displayed. An enemy object Eo, which is an enemy of the player object Po, is arranged in the virtual world. Displaying, on the LCD61, the virtual world viewed from near the back of the player object Po (specifically, the first player object Po1) allows the user holding the terminal device6to experience a sense of presence in the virtual world, and to intuitively know the movement direction and the movement speed of the player object Po.

Further, as shown inFIG. 25B, the same virtual world as that displayed on the LCD61is displayed on the monitor2. In the example shown inFIG. 25B, a state of the virtual world in which the player object Po is viewed from a distance in a manner of bird's eye viewing, is displayed together with the player object Po, the target object To, the enemy object Eo, and the expected trajectory S. Displaying, on the monitor2, the state of the virtual world in which the player object Po is viewed from a distance allows the user to easily know the surroundings of the player object Po, and allows another person who watches the user's playing the third exemplary game to enjoy a view in which the player object Po moves in the virtual world.

For example, when the user performs an action to shift forward the center-of-gravity position of a load the user applies to the board type controller9, the player object Po moves forward in the direction of the first player object Po1at a speed according to a forward length from the center of the platform9ato the center-of-gravity position. When the user performs an action to shift backward the center-of-gravity position of the load the user applies to the board type controller9, the player object Po moves backward in the direction of the first player object Po1at a speed according to a backward length from the center of the platform9ato the center-of-gravity position. In this way, the user is allowed to move the player object Po forward and backward in accordance with his/her action on the board type controller9.

When the user changes the attitude of the terminal device6or rotates the terminal device6to change the direction of the terminal device6, the direction of the player object Po and the position of the target object To change. Specifically, as shown inFIG. 26, when the direction of the terminal device6rotates within a turret/target direction operation range corresponding to a turret/target movement angle range, the second player object Po2(turret) rotates to the left or to the right in the virtual world, and the position of the target object To, which is a target of a shell shot from the turret, also moves in the virtual world. When the direction of the terminal device6rotates into a body direction operation range corresponding to a range outside the turret/target movement angle range, the first player object Pal (the tank body) as well as the second player object Po2rotate to the left or to the right in the virtual world in accordance with the change in the direction of the terminal device6, and further, position of the target object To also shifts in the virtual world in accordance with the rotation of the second player object Po2. Since the virtual camera is set near the back of the first player object Po1as described above, the attitude and the position of the virtual camera change with the change in the direction of the first player object Po1. Accordingly, when the user changes the left-right direction of the terminal device6within the turret/target direction operation range in the real space, the user is allowed to change the position of the target object To without changing the position and the attitude of the virtual camera, and therefore, the user is allowed to adjust the target position with the game screen displayed on the LCD61being fixed. That is, the operation using the terminal device6within the turret/target direction operation range provides the user with favorable operation environment for adjusting the position of the target object To to attack the enemy object Eo. Further, when the user changes the direction of the terminal device6to the left or to the right in the body direction operation range in the real space, the user is allowed to change the direction of the first player object Po1and the position and the direction of the virtual camera. Therefore, the user is allowed to change the movement direction of the player object Po, and adjusts the target position to the outside of the game screen displayed on the LCD61. In this way, setting at least two operation ranges allows the user to perform various operations by using the attitude of a single device.

When the direction of the terminal device6changes upward or downward, the direction of the turret (second player object Po2) changes upward or downward in the virtual world with the change in the direction of the terminal device6, and the position of the target object To, which is a target of a shell shot from the turret, also changes in the virtual world. That is, the user is allowed to change the direction of the turret (the position of the target object To) upward, downward, leftward, and rightward in accordance with the direction of the terminal device6. Thus, the user is allowed to direct the turret (the arrange the target object To) to his/her desired direction (position) by only changing the direction of the terminal device6.

Next, the process of the third embodiment, which is performed by the game system1, will be described in detail. First, main data used in the process according to the third embodiment will be described with reference toFIG. 27.FIG. 27is a diagram illustrating an example of main data and programs stored in the main memory of the game apparatus body5of the third embodiment.

As shown inFIG. 27, in a data storage area of the main memory, board operation data Dca, terminal operation data Dcb, load value data Dcc, center-of-gravity position data Dcd, terminal device direction/attitude data Dee, operation direction data Dcf, action/attitude data Dcg, movement vector data Dch, position data Dci, virtual camera data Dcj, image data Dck, and the like are stored. In addition to the data shown inFIG. 27, data required for a game process, such as image data of various objects displayed on the monitor2and the LCD61, and sound data used for a game, are optionally stored in the main memory. In a program storage area of the main memory, various programs Pc constituting an information processing program of the third embodiment are stored.

The board operation data Dca includes load data Dca1and the like. Since the load data Dca1is identical to the load data Daa1of the first embodiment, repeated description is not necessary.

The terminal operation data Dcb includes acceleration data Dcb1, angular velocity data Dcb2, and the like. Since the acceleration data Dcb1and the angular velocity data Dcb2are identical to the acceleration data Dab1and the angular velocity data Dab2of the first embodiment, respectively, repeated description is not necessary.

Since the load value data Dcc and the center-of-gravity position data Dcd are identical to the load value data Dac and the center-of-gravity position data Dad of the first embodiment, respectively, repeated description is not necessary.

The terminal device direction/attitude data Dce includes real space reference direction data Dce1, current direction data Dce2, and the like. As in the first embodiment, the real space reference direction data Dce1is data representing the reference direction of the terminal device6in the real space, and the current direction data Dce2is data representing the direction and the attitude of the terminal device6at the current time in the real space. However, in the third embodiment, the respective direction data are subjected to various corrections. Methods of calculating the real space reference direction and the current direction according to the third embodiment will be described later.

The operation direction data Dcf includes virtual world reference direction data Dcf1, operation instruction direction data Dcf2, and the like. The virtual world reference direction data Dcf1is data representing a virtual world reference direction set in the virtual world. The operation instruction direction data Dcf2is data representing an operation instruction direction which is currently instructed in the virtual world by the user. Methods for calculating the virtual world reference direction and the operation instruction direction of the third embodiment will be described later.

The action/attitude data Dcg includes body attitude data Dcg1and turret attitude data Dcg2, and the like. The body attitude data Dcg is data representing the action and the attitude of the first player object Po1(tank body) in the virtual world. The turret attitude data Dcg2is data representing the action and the attitude of the second player object Po1(turret) in the virtual world.

The movement vector data Dch is data representing the movement speeds and the movement directions of the respective player objects (the first player object Po1and the second player object Po2) and the shell object as in the first embodiment, although the calculation method thereof is different from that of the first embodiment as described later.

The position data Dci is data representing the positions of the player object Po, the target object To, the shell object and the like in the virtual world. The virtual camera data Dcj is data relating to the virtual camera as in the first embodiment although the calculation method thereof is different from that of the first embodiment as described later.

The image data Dck includes player object data Dck1, background image data Dck2, and the like. The player object data Dck1is data for arranging the player object Po (the first player object Po1and the second player object Po2), the target object To, and the shell object of the third exemplary game in the virtual world to generate a game image. The background image data Dck2is data for arranging a background of the third exemplary game in the virtual world to generate a game image.

Next, the process of the third embodiment, which is performed by the game apparatus body5, will be described in detail with reference toFIGS. 28 and 29.FIG. 28is a sub-routine illustrating an example of a player object setting process of the third embodiment executed by the game apparatus body5.FIG. 29is a sub-routine illustrating an example of an operation instruction direction calculation process at step201inFIG. 28. In the flowcharts ofFIGS. 28 and 29, process steps for causing the player object Po to move and displaying the same, in accordance with an operation performed by a user using the terminal device6and the board type controller9, will be mainly described among the whole process, while detailed description of other process steps is not given. Further, inFIGS. 28 to 29, each of the steps executed by the CPU10is abbreviated as “S”.

When the game apparatus body5is powered on, the CPU10of the game apparatus body5executes a boot program stored in the ROM/RTC13to initialize each unit such as the main memory. Then, the information processing program stored in the optical disc4is loaded to the main memory, and the CPU10starts to execute the program. The process of the third embodiment to be performed after completion of the above-mentioned process steps is identical to the process of the main flow of the first embodiment shown inFIG. 12except the reference direction setting process at step44and the game control process at step45. Therefore, the identical part of the process is not repeatedly described. The game control process at step44according to the third embodiment is identical to the sub-routine of the first embodiment shown inFIG. 13except the player object setting process at step83, and except that the movement trajectory setting process at step85is not performed. Therefore, the identical part of the process is not repeatedly described.

In the main flow of the third embodiment, in the reference direction setting process at step41, a reference direction is set after the attitude and the direction of the terminal device6are subjected to various corrections, and the real space reference direction data Dce1is updated by using the reference direction after the corrections. Further, the real space reference direction after the corrections is converted into a model coordinate system in the virtual world, and the virtual world reference direction data Dcf1is updated by using the direction after the conversion, as the reference direction in the virtual world. The methods for correcting the attitude and the direction of the terminal device6will be described later. Hereinafter, the player object setting process at step83according to the third embodiment will be described with reference toFIG. 28.

InFIG. 28, the CPU10performs an operation instruction direction calculation process (step201), and proceeds to the next step. Hereinafter, the operation instruction direction calculation process at step201will be described with reference toFIG. 29.

InFIG. 29, the CPU10corrects the forward direction of the terminal device6with respect to the upward and downward directions (step221), and proceeds to the next step. For example, the CPU10corrects the direction (attitude) of the terminal device6such that a state in which the terminal device6tilts downward at a predetermined angle (e.g., 42°) with respect to the horizontal direction in the real space becomes the operation instruction direction in the horizontal direction. Specifically, in step82, the CPU10calculates X-axis, Y-axis, and Z-axis directions of the terminal device6in the real space, based on the acceleration represented by the acceleration data Dcb1and the angular velocity represented by the angular velocity data Dcb2, and updates the current direction data Dce2by using the calculated X-axis, Y-axis, and Z-axis directions as current directions. Then, in step221, the CPU10corrects the Y-axis direction and the Z-axis direction such that the Y-axis direction and the Z-axis direction tilt upward around the X axis by the above-mentioned predetermined angle (i.e., the Y-axis direction and the Z-axis direction are rotated to the right by the predetermined angle around the X axis as viewed in the X-axis direction), by using the X-axis, Y-axis, and Z-axis directions represented by the current direction data Dce2.

Next, the CPU10corrects the tilt of the terminal device6around the Z axis (step222), and proceeds to the next step. For example, the CPU10corrects the direction (attitude) of the terminal device6such that the X axis of the terminal device6becomes the horizontal direction in the real space. Specifically, the CPU10forcibly corrects the X-axis direction to the horizontal direction in the real space by using the X-axis, Y-axis, and Z-axis directions corrected in step221, and newly calculates a Z-axis direction, based on a cross product of the X-axis direction and the Y-axis direction. Then, the CPU10newly calculates a Y-axis direction, based on a cross product of the newly calculated Z-axis direction and the X-axis direction corrected to the horizontal direction.

Next, the CPU10performs a correction to limit the tilt of the terminal device6in the up-down direction of the terminal device6(step223), and proceeds to the next step. For example, the CPU10limits the Z-axis direction newly calculated at step222into a predetermined range with respect to the up-down direction in the real space. If the Z-axis direction is outside the range, the CPU10resets the Z-axis direction in the closest range. Then, the CPU10newly calculates a Y-axis direction, based on a cross product of the reset Z-axis direction and the X-axis direction corrected to the horizontal direction at step222. Further, the CPU10newly calculates an X-axis direction, based on a cross product of the newly calculated Y-axis direction and the reset Z-axis direction. Then, the CPU10updates the current direction data Dce2by using the newly calculated X-axis direction and Y-axis direction, and the reset Z-axis direction.

Also in the reference direction setting process at step41, the X-axis, Y-axis, and Z-axis directions of the terminal device6are corrected as in steps221to223, and the real space reference direction data Dce1is updated by using the corrected Z-axis direction as the real space reference direction.

Next, the CPU10calculates a horizontal angular difference between the real space reference direction and the current direction (step224), and proceeds to the next step. The horizontal angular difference is an angular difference obtained by projecting an angular difference between the real space reference direction in the real space and the Z-axis direction represented by the current direction data Dce2on the horizontal plane. The horizontal angular difference indicates an angle at which the direction of the terminal device6(the direction in which the back surface of the terminal device6faces (the z-axis positive direction shown inFIG. 3)) is changed from the initial attitude of the terminal device6around the vertical direction in the real space. For example, the CPU10calculates the horizontal angular difference by using the real space reference direction represented by the real space reference direction data Dce1, and the Z-axis direction represented by the current direction data Dce2.

Next, the CPU10calculates an up-down angular difference between the horizontal direction in the real space and the current direction (step225), and proceeds to the next step. For example, the CPU10calculates, as an up-down angular difference, an angular difference between the horizontal direction in the real space and the Z-axis direction, by using the Z-axis direction represented by the current direction data Dce2.

Next, the CPU10calculates an operation instruction direction corresponding to the virtual world reference direction, in accordance with the horizontal angular difference calculated at step224and the up-down angular difference calculated at step225(step226), and ends the process of this sub-routine. For example, based on the virtual world reference direction represented by the virtual world reference direction data Dcf1, the CPU10calculates an operation instruction direction in the virtual world such that an angular difference that occurs when the virtual world reference direction and the operation instruction direction are projected on the horizontal plane in the virtual world is equal to the horizontal angular difference, and the same positional relation is achieved between the real space and the virtual world (i.e., if the Z-axis direction rotates to the left with respect to the real space reference direction, the operation instruction direction also rotates to the left with respect to the virtual world reference direction). Further, the CPU10calculates an operation instruction direction in the virtual world such that an angular difference between the horizontal direction in the virtual world and the operation instruction direction is equal to the up-down angular difference, and the same positional relation is achieved between the real space and the virtual world (i.e., if the Z-axis direction is downward with respect to the horizontal direction of the real space, the operation instruction direction is also downward with respect to the horizontal direction in the virtual world). Then, the CPU10updates the operation instruction direction data Dcf2by using the calculated operation instruction direction.

Referring back toFIG. 28, after the operation instruction direction calculation process at step201, the CPU10determines whether or not the operation instruction direction is within a first operation range (step202). When the operation instruction direction is within the first operation range, the CPU10proceeds to step203. On the other hand, when the operation instruction direction is outside the first operation range, the CPU10proceeds to step204. The first operation range is a rotation angle range in the horizontal direction, in which the direction of the second player object Po2and the position of the target object To are changed in accordance with the operation instruction direction. For example, the first operation range is set to a range from −5° to +5° centering around the virtual world reference direction. Then, in step202, the CPU10determines whether or not an angular difference which occurs when the virtual world reference direction and the operation instruction direction are projected on the horizontal plane in the virtual world is within the first operation range, by using the virtual world reference direction represented by the virtual world reference direction data Dcf1and the operation instruction direction represented by the operation instruction direction data Dcf2. The first operation range corresponds an example of the turret/target direction operation range shown inFIG. 26.

In step203, the CPU10sets the direction (turret angle) of the second player object Po2and the position (target position) of the target object To in accordance with the operation instruction direction, and proceeds to step209. For example, the CPU10sets, as the turret direction of the second player object Po2, the operation instruction direction represented by the operation instruction direction data Dcf2, and updates the turret attitude data Dcg2by using the set turret direction. Based on the turret direction of the second player object Po2and the position of the player object Po represented by the position data Dci, the CPU10calculates, as a position of the target object To, a position at which a shell object shot from the second player object Po2at a predetermined shell speed lands in the virtual world. Then, the CPU10updates, by using the calculated position, the position of the target object To represented by the position data Dci.

On the other hand, in step204, the CPU10determines whether or not the operation instruction direction is within a second operation range. When the operation instruction direction is within the second operation range, the CPU10proceeds to step205. On the other hand, when the operation instruction direction is outside the second operation range, the CPU10proceeds to step207. The second operation range is a rotation angle range in the horizontal direction, in which the direction of the second player object Po2and the position of the target object To are changed in accordance with the operation instruction direction, and further, the forward direction (body direction) of the first player object Po1is changed so as to approach the operation instruction direction by a predetermined angle. For example, the second operation range includes a range from −87° to −5° and a range from +5° to +87° centering around the vertical world reference direction. Then, in step204, the CPU10determines whether or not an angular difference which occurs when the virtual world reference direction and the operation instruction direction are projected on the horizontal plane in the virtual world is within the second operation range, by using the virtual world reference direction represented by the virtual world reference direction data Dcf1and the operation instruction direction represented by the operation instruction direction data Dcf2. The second operation range corresponds an example of the body direction operation range shown inFIG. 26.

In step205, the CPU10sets the direction (turret angle) of the second player object Po2and the position (target position) of the target object To in accordance with the operation instruction direction, and proceeds to the next step. For example, the CPU10sets the operation instruction direction represented by the operation instruction direction data Dcf2, as the turret direction of the second player object Po2, and updates the turret attitude data Dcg2by using the set turret. Based on the turret direction of the second player object Po2and the position of the player object Po represented by the position data Dci, the CPU10calculates, as a position of the target object To, a position at which a shell object shot from the second player object Po2at a predetermined shell speed lands in the virtual world. Then, the CPU10updates, by using the calculated position, the position of the target object To represented by the position data Dci.

Next, the CPU10sets the forward direction (body direction) of the first player object Po1in accordance with the operation instruction direction (step206), and proceeds to step209. For example, based on the forward direction of the first player object Po1represented by the body attitude data Dcg1and the operation instruction direction represented by the operation instruction direction data Dcf2, the CPU10calculates an angular difference Od which occurs when the forward direction of the first player object Po1and the operation instruction direction are projected on the horizontal plane in the virtual world. Then, CPU10calculates an angle θc2at which the forward direction of the first player object Po1is changed, in accordance with the following equation:
θc2=1.0−cos θd

Then, the CPU10changes the forward direction of the first player object Po1so as to approach the operation instruction direction by the calculated angle θc2, and updates the body attitude data Dcg1by using the changed forward direction. Further, since the line-of-sight direction of the virtual camera for taking a game image to be displayed on the LCD61is set to a direction in which the first player object Po1faces, the CPU10changes the virtual world reference direction represented by the virtual world reference direction data Dcf1, to the same direction and by the same angle θc2as the forward direction of the first player object Po1, and updates the virtual world reference direction data Dcf1by using the changed virtual world reference direction.

On the other hand, when it is determined at step204that the operation instruction direction is outside the second operation range, the CPU10determines that the operation instruction direction is within a third operation range. Then, the CPU10sets the direction (body direction) of the first player object Po1in accordance with the operation instruction direction (step207), and proceeds to the next step. The third operation range is a rotation angle range in the horizontal direction, in which the forward direction (body direction) of the first player object Po1is changed at an angle relatively greater than that in the second operation range. For example, the third operation range includes a range from −180° to −87° and a range from +87° to +180° centering around the virtual world reference direction. The third operation range corresponds an example of the body direction operation range shown inFIG. 26, and a combination of the second operation range and the third operation range corresponds to an example of the body direction operation range shown inFIG. 26.

In step207, the CPU10changes the forward direction of the first player object Po1so as to approach the operation instruction direction by a predetermined angle θc3(e.g., 1.0°), based on the forward direction of the first player object Po1represented by the body attitude data Dcg1and the operation instruction direction represented by the operation instruction direction data Dcf2, and updates the body attitude data Dcg1by using the changed forward direction. Since the line-of-sight direction of the virtual camera for taking a game image to be displayed on the LCD61is set to the direction in which the first player object Po1faces, the CPU10changes the virtual world reference direction represented by the virtual world reference direction data Dcf1, to the same direction and by the same angle θc3as the forward direction of the first player object Po1, and updates the virtual world reference direction data Dcf1by using the changed virtual world reference direction.

Next, the CPU10sets the direction (turret angle) of the second player object Po2and the position (target position) of the target object To in accordance with the body direction (step208), and proceeds to step209. For example, in the third operation range, the direction of the second player object Po2(turret) arranged on the first player object Po1changes with the direction change of the first player object Po1while maintaining the positional relation with the first player object Po1, and further, the position of the target object To also changes with the direction change of the first player object Po1. Accordingly, the CPU10changes the turret direction of the second player object Po2represented by the turret attitude data Dcg2, to the same direction and by the same angle θc3as the forward direction of the first player object Po1, and updates the turret attitude data Dcg2by using the changed turret direction. Then, based on the turret direction of the second player object Po2and the position of the player object Po represented by the position data Dci, the CPU10calculates, as a position of the target object To, a position at which a shell object shot from the second player object Po2at a predetermined shell speed lands in the virtual world. Then, the CPU10updates, by using the calculated position, the position of the target object To represented by the position data Dci. That is, in the process at step208, the turret direction of the second player object Po2rotates in the virtual world together with the first player object Po1while maintaining an angle (e.g., −87° or +87°) corresponding to the boundary between the second operation range and the third operation range with respect to the first player object Po1.

In step209, the CPU10sets a movement vector based on the center-of-gravity position and the body direction, and proceeds to the next step. For example, the CPU10calculates a movement speed of the player object Po based on the latest center-of-gravity position represented by the center-of-gravity position data Dcd. For example, the CPU10calculates a movement speed of the player object Po, based on a Y1axis value (a coordinate value in a short-side direction) of the center-of-gravity position in a predetermined coordinate system corresponding to a position on the platform9aof the board type controller9. For example, when the center of the platform9ais set at the origin of the coordinate system, if the Y1axis value of the center-of-gravity position is greater than a predetermined numerical value (i.e., if the center-of-gravity position is provided at a position apart from the center of the platform9a, forward or backward, by a predetermined distance or more), the CPU10sets a forward or backward movement speed of the player object Po in accordance with the numerical value. Specifically, when the front side of the board type controller9(the side on which the load sensors94cand94dare provided as shown inFIG. 6) is set in the Y1axis direction, if the Y1axis direction of the center-of-gravity position is a positive value and its absolute value is greater than the predetermined numerical value, the CPU10sets a forward movement speed (e.g., a positive movement speed) of the player object Po in accordance with the numerical value. On the other hand, if the Y1axis value of the center-of-gravity position is a negative value and its absolute value is greater than the predetermined numerical value, the CPU10sets a backward movement speed (e.g., a negative movement speed) of the player object Po in accordance with the numerical value.

When the forward movement speed (e.g., the positive movement speed) of the player object Po is set, the CPU10regards the forward direction of the first player object Po1represented by the body attitude data Dcg1, as the movement direction of the player object Po. When the backward movement speed (e.g., the negative movement speed) of the player object Po is set, the CPU10regards a direction opposite to the forward direction of the first player object Po1represented by the body attitude data Dcg1, as the movement direction of the player object Po. Then, the CPU10calculates a movement vector in the virtual world, based on the calculated movement speed and movement direction of the player object Po, and updates the movement vector data Dch by using the calculated movement vector. For example, the CPU10sets a length of the movement vector by using the calculated movement speed of the player object Po, and sets a direction of the movement vector by using the forward direction of the first player object Po1represented by the body attitude data Dcg1or a direction opposite to the forward direction.

Next, the CPU10calculates a position of the player object Po (step210), and proceeds to the next step. For example, the CPU10causes the position of the player object Po represented by the position data Dci to shift in the virtual world, based on the movement vector represented by the movement vector data Dch, and updates the position data Dci by using the position after the shifting.

Next, the CPU10determines whether or not the player object Po is during shelling (step211). When the player object Po is during shelling, the CPU10proceeds to step212. On the other hand, if the player object Po is not during shelling, the CPU10ends the process of this sub-routine. The player object Po is capable of shooting a shell object from the second player object Po2toward the target object To, and a shell is shot from the turret toward the set turret direction in accordance with an operation by the user (e.g., an operation to press an attack button (operation button64E) of the terminal device6). The “during shelling” means a period from when the operation to cause the player object Po to shoot a shell object toward the target object To is performed to when the shell object collides with another object or the like and disappears.

In step212, the CPU10calculates a position and a movement vector of the shell object, and ends the process of this sub-routine. For example, when the above-mentioned operation to shoot the shell object has been performed, the CPU10sets the position of the shell object in the turret as the second player object Po2, and sets a movement vector having a predetermined shell speed in the turret direction, thereby updating the movement vector of the shell object represented by the movement vector data Dch, and the position of the shell object represented by the position data Dci. After the shell object was shot, the CPU10moves the shell object in the virtual world, based on the set movement vector, to set a new position of the shell object, and updates the position of the shell object represented by the position data Dci by using the set position. Further, the CPU10corrects the movement vector, based on the environment (gravity, wind, influence of other objects, and the like) set in the virtual world, and updates the movement vector of the shell object represented by the movement vector data Dch by using the corrected movement vector.

In the virtual camera setting process (step84) of the third embodiment, the first virtual camera for generating a terminal game image is set such that a state of the virtual world viewed from near the back of the first player object Po1arranged in the virtual world is included in the terminal game image. On the other hand, the second virtual camera for generating a monitor game image is set in the same virtual world as the first virtual camera, such that a state of the virtual world in which the player object Po arranged in the virtual world is viewed from a distance, in the forward direction of the first player object Po1, in a manner of bird's eye viewing is included in the monitor game image. The CPU10set the positions of the first virtual camera and the second virtual camera in the virtual world (the parameters relating to the first virtual camera and the second virtual camera), based on the position of the player object Po represented by the position data Dbi and the forward direction of the first player object Po1represented by the body attitude data Dcg1.

In the monitor game image generation process (step45) and the terminal game image generation process (step46) of the third embodiment, images representing the result of the game control process are generated, respectively, as in the first embodiment. For example, the monitor game image is generated as a three-dimensional CG image which is generated through the steps of: arranging the second virtual camera in the virtual world based on the parameters relating to the second virtual camera, which are represented by the virtual camera data Dcj; arranging the player object Po (the first player object Po1and the second player object Po2), the target object To, the shell object, and the like in the virtual world based on the action/attitude data Dcg and the position data Dci; and calculating a scene of the virtual world viewed from the second virtual camera. The terminal game image is generated as a three-dimensional CG image which is generated through the steps of: arranging the first virtual camera in the virtual world based on the parameters relating to the first virtual camera, which are represented by the virtual camera data Dcj; arranging the player object Po, the target object To, the shell object, and the like in the virtual world based on the action/attitude data Dcg and the position data Dci; and calculating a scene of the virtual world viewed from the first virtual camera.

In this way, according to the above-mentioned process, in the operation using the terminal device6within the turret/target direction operation range (the first operation range), only the direction of the second player object Po2and the position of the target object To change, while the direction of the first player object Po1and the display range of the virtual world displayed on the LCD61do not change. Accordingly, in the operation using the terminal device6within the turret/target direction operation range (the first operation range), the objects to be moved by this operation are limited, and favorable operation environment is provided for adjusting the position of the target object To to shoot a shell toward the enemy object Eo or the like. Further, in the operation using the terminal device6within the body direction operation range (the second operation range or the third operation range) in the real space, not only the direction of the second player object Po2and the position of the target object To but also the direction of the first player object Po1and the position and the direction of the virtual camera can be changed. Accordingly, the operation using the terminal device6within the body direction operation range enables an operation of the objects other than the objects operable in the turret/target direction operation range, and is favorable for changing the movement direction of the player object Po, or changing the display range on the LCD61. Thus, setting at least two operation ranges allows the user to perform various operations by using the attitude of a single device.

In the above-mentioned first to third exemplary games, control (control of the position and the direction) of the virtual camera (the first virtual camera) for generating an image to be displayed on the LCD61is performed based on the attitude of the terminal device6. Such control provides the user with an image that makes the user feel like looking into the virtual world through the LCD61, and thereby allows the user to experience a sense of presence in the virtual world. Further, the operation based on the attitude of the terminal device6enables rotation operations in three directions, i.e., the left-right swing motion (yaw) around the vertical direction (e.g., around the Y-axis direction), the up-down swing motion (pitch) around the left-right horizontal direction (e.g., around the X-axis direction), and the left-right rotation (roll) around the front-rear horizontal direction (e.g., around the Z-axis direction). Therefore, the operation based on the attitude of the terminal device6is suitable for control of the virtual camera that is similarly movable in the virtual world. By controlling the attitude of the virtual camera in the virtual world so as to be equal to the attitude of the terminal device6in the real space, it is possible to provide the user with an image that makes the user feel like looking into the virtual world along his/her desired direction. Further, in the first to third exemplary games, the player object performs an action (e.g., moves) in accordance with an action of the user on the board type controller9. That is, the image displayed on the LCD61allows the user to experience a sense of presence in the virtual world, and moreover, allows the user to experience an operation sense of becoming the player character in the real space, which enhances the sense of presence in the virtual world.

In the first to third exemplary games, the attitude of the player object Po displayed on the LCD61is controlled based on the attitude of the terminal device6. Such control provides the user with an operation environment as if the terminal device6is the player object Po, and allows the user to experience a sense of becoming the player object Po in the virtual world, and a sense of directly operating the player object Po. Further, the operation based on the attitude of the terminal device6enables rotation operations in three directions, i.e., the left-right swing motion (yaw) around the vertical direction (e.g., around the Y-axis direction), the up-down swing motion (pitch) around the left-right horizontal direction (e.g., around the X-axis direction), and the left-right rotation (roll) around the front-rear horizontal direction (e.g., around the Z-axis direction). Therefore, the operation based on the attitude of the terminal device6is suitable for control of the player object Po that is similarly movable in the virtual world. For example, in the first exemplary game, the left-right swing motion (yaw) around the vertical direction (Y-axis direction) along the LCD61of the terminal device6is related to an attitude change (yaw) of the player object Po in the left-right direction, the up-down swing motion (pitch) around the left-right direction (X-axis direction) along the LCD61is related to an attitude change (pitch) of the player object Po in the up-down direction, and the left-right rotation (roll) around the depth direction (Z-axis direction) of the LCD61is related to a rotational attitude change (roll) around the forward direction of the player object Po, thereby providing the user with an image of an object whose attitude changes as desired by the user in the virtual world. Further, in the first to third exemplary games, the player object performs an action (e.g., moves) in accordance with an action of the user on the board type controller9. That is, the user operates one player object by using a plurality of devices (the terminal device6and the board type controller9), which realizes a new operation. Further, control of the attitude of the player object based on the attitude of the terminal device6allows the user to experience a sense of becoming the player object in the virtual world, and a sense of directly operating the player object. Moreover, user's operation on the board type controller9allows the user to experience a sense of becoming the player character in the real space, which enhances the sense of presence in the virtual world. Further, in the first to third exemplary games, it is possible to move the player object in the direction forward from the player object. Accordingly, the user is allowed to set, based on the attitude of the terminal device6, the movement direction of the player object as well as the attitude (forward direction) of the player object, and thus the user is allowed to easily and intuitively set the movement direction of the player object to his/her desired direction. Further, as well as the attitude of the player object, it is possible to set the movement direction of the player object in accordance with the rotation operations in the three directions of the terminal device6. Therefore, the user is allowed to set his/her desired movement direction as it is in the virtual world.

In the first to third exemplary games, the movement direction of the player object Po is set based on the attitude of the terminal device6. Such setting of the movement direction provides the user with an operation environment as if the terminal device6becomes the player object Po, and allows the user to experience a sense of presence in the virtual world as the player object Po, and a sense of directly operating the player object Po. Further, the operation using the attitude of the terminal device6enables rotation operations in three directions, i.e., the left-right swing motion (yaw) around the vertical direction (e.g., around the Y-axis direction), the up-down swing motion (pitch) around the left-right horizontal direction (e.g., around the X-axis direction), and the left-right rotation (roll) around the front-rear horizontal direction (e.g., around the Z-axis direction). Therefore, the operation based on the attitude of the terminal device6is suitable for control of the player object Po whose movement direction can be set in at least one direction among the three directions in the virtual world. For example, in the case of the first exemplary game, the left-right swing motion (yaw) around the vertical direction (Y-axis direction) along the LCD61of the terminal device6is related to a movement direction change (yaw) of the player object Po in the left-right direction, and the up-down swing motion (pitch) around the left-right direction (X-axis direction) along the LCD61is related to a movement direction change (pitch) of the player object Po in the up-down direction, thereby providing an object control for causing the player object Po to move in a movement direction desired by the user in the virtual world. Further, in the first to third exemplary games, setting of the movement (e.g., setting of the movement speed) of the player object is performed in accordance with an action of the user on the board type controller9. That is, the user controls the movement direction and the movement speed of one player object by using a plurality of devices (the terminal device6and the board type controller9), thereby realizing a new operation. Further, setting of the movement direction of the player object based on the attitude of the terminal device6allows the user to experience a sense of becoming the player object in the virtual world, and a sense of directly operating the player object. Moreover, the user's operation on the board type controller9allows the user to experience an operation sense of becoming the player object in the real space, which enhances the sense of presence in the virtual world.

In the first to third exemplary games, the direction of the virtual world displayed on the LCD61of the terminal device6is the movement direction of the player object. Accordingly, the user is allowed to set, based on the attitude of the terminal device6, the movement direction of the player object. Further, since the virtual world viewed in the movement direction is displayed on the LCD61, the user is allowed to easily and intuitively set the movement direction of the player object to his/her desired direction. Further, as well as the direction of the virtual camera, it is possible to set the movement direction of the player object in accordance with the rotation operations in the three directions of the terminal device6, the user is allowed to set his/her desired movement direction as it is in the virtual world.

Further, in the first to third exemplary games, an image of a virtual world including at least the player object Po is displayed on the LCD61of the terminal device6. However, an image of another type of a virtual world may be displayed on the LCD61. For example, an image of a virtual world viewed from a subjective point of the player object Po may be displayed on the LCD61without displaying the player object Po. In this case, the direction of the virtual world displayed on the LCD61from the subjective point may be the same as the movement direction of the player object Po, or may be different from the movement direction of the player object Po. When the direction of the virtual world displayed on the LCD61from the subjective point is the same as the movement direction of the player object Po, the virtual world viewed in the movement direction is displayed on the LCD61. Therefore, the user is allowed to easily and intuitively set the movement direction of the player object to his/her desired direction.

The above-mentioned first to third exemplary games adopts an example of a process in which control of the position and the attitude of the virtual camera is immediately associated with the operation instruction direction determined based on the attitude of the terminal device6. However, control of the position and the attitude of the virtual camera may be delayed for a predetermined time in accordance with a change in the operation instruction direction. In this case, after the direction of the player object has changed, the position and the direction of the virtual camera follows the direction of the player object after the predetermined time.

Further, in the first to third exemplary games, the game image displayed on the LCD61of the terminal device6and the game image displayed on the monitor2are images showing the same virtual world (virtual space), but the images are viewed from different viewpoints or in different ranges of the virtual world. Accordingly, the user is allowed to view different views or ranges of the virtual world (virtual space) displayed on the two display screens, and thereby allowed to view appropriate game images according to the game situation or the like. Further, in the first to third exemplary games, the user holds and operates the terminal device6to change the position and the attitude of the player object or the virtual camera in accordance with the attitude and the position of the terminal device6in the real space, and change the image displayed on the LCD61in accordance with the position and the attitude of the player object or the virtual camera. Accordingly, the user, who views the image displayed on the LCD61with the terminal device6in his/her hands, is given a sense of presence in the virtual world (virtual space). On the other hand, it is difficult for the user, who views only the image displayed on the LCD61, to know the position of the player object in the entire virtual world (virtual space) and the surroundings of the player object. Such a problem is resolved by displaying, on the monitor2, a relatively wide range of the virtual world (virtual space).

In the first to third exemplary games, the user is allowed to perform an operation by changing a load applied to the board type controller9by the user, and further allowed to perform an operation (an operation according to the attitude or the position of the terminal device6, a touch operation, a button operation, or the like) with holding the terminal device6on which a game image including a player object is displayed. The player object displayed on the terminal device6is caused to move in the virtual world (virtual space) in accordance with the operation of changing the load applied to the board type controller9and/or the operation using the terminal device6. Accordingly, the user is allowed to experience a sense of acting as the player object and viewing the virtual world (virtual space), and a sense of operating the player object in the real world.

In the operation using the board type controller9, presence/absence of a load applied to the board type controller9, a change in a load, and a center-of-gravity position of a load are used for various operation controls for the player objects. For example, in the first to third exemplary games, the action, the movement speed, the movement speed and the like of the player object are changed in accordance with a change in the load applied to the board type controller9, and the center-of-gravity position of the load. In the exemplary games, although the direction (e.g., the movement direction or the forward direction of the player object) in the virtual world (virtual space) is set in accordance with the direction of the user's operation, the direction is set based on the attitude (direction) of the terminal device6. Further, correction of the movement direction in the air in the virtual world in the second exemplary game, and switching between the forward movement and the backward movement in the third exemplary game, are performed based on the center-of-gravity position of the load applied to the board type controller9. That is, between the operation based on the attitude of the terminal device6and the operation using the board type controller9, an appropriate operation, which facilitates setting of a direction or enhances a sense of presence in the virtual world, is selected. Thus, an appropriate one, between the attitude of the terminal device6and the center-of-gravity position of the load applied to the board type controller9, can be selected according to the game content, as an operation to instruct a direction. In this way, in the case where the terminal device6and the board type controller9are used as operation means, when setting a direction in the virtual world (virtual space) in accordance with an operation of the user, the user is allowed to select an appropriate operation among a plurality of operations as options.

The game system1allows the user to play various games by using the terminal device6and the board type controller9as operation means. The terminal device6serves as a controller which allows the user to perform an input by an operation based on its motion, a touch operation, a button operation or the like, while it also serves as a portable display or a second display. Therefore, the game system1realizes a wide range of games. That is, since the terminal device6functions as a display device, there may be a game system in which the terminal device6is used as display means and the board type controller9is used as operation means while the monitor2and the controller7are not used. Further, since the terminal device6functions as an operation device as well as a display device, there may be a game system in which the terminal device6is used as display means and the terminal device6and the board type controller9are used as operation means while the monitor2and the controller7are not used. Furthermore, since the terminal device6functions as an operation device as well as a display device, there may be a game system in which the terminal device6is used as display means and operation means while the monitor2, the controller7, and the board type controller9are not used.

In the above-mentioned embodiments, the terminal device6functions as a so-called thin client terminal which does not execute a game process. However, at least a part of the sequential game processes executed by the game apparatus body5may be executed by the terminal device6. For example, the terminal game image generating process may be executed by the terminal device6. In another example, all the sequential game processes to be executed by the game apparatus body5may be executed by the terminal device6. In this case, since the terminal device6functions as a processing device for executing the game processes as well as a display device, there may be a game system in which the terminal device6is used as display means, the board type controller9is used as operation means, and the terminal device6is used as processing means while the monitor2, the game apparatus body5, and the controller7are not used. In this game system, only the terminal device6and the board type controller9are connected wirelessly or via a cable, and board operation data is transmitted from the board type controller9to the terminal device6, thereby realizing various kinds of games. Further, when the board type controller9is not used, there may be a game system in which the terminal device6is used as display means, operation means, and processing means.

In the above-mentioned embodiments, the attitude data (e.g., data outputted from at least one of the magnetic sensor602, the acceleration sensor603, and the gyro sensor604) used for calculating the movement (including the position and the attitude, or a change in the position or the attitude) of the terminal device6is transmitted from the terminal device6to the game apparatus body5, and the movement of the terminal device6is calculated by the information processing in the game apparatus body5. However, the movement of the terminal device6to be calculated in the game apparatus body5may be calculated in the terminal device6. In this case, data representing the movement of the terminal device6, which is calculated in the terminal device6(i.e., data representing the position and the attitude of the termination device, or a change in the position or the attitude, which is calculated based on the attitude data) is transmitted from the terminal device6to the game apparatus body5, and used in the information processing in the game apparatus body5.

In the above description, the terminal device6and the game apparatus body5are connected by wireless communication, and the board type controller9and the game apparatus body5are connected by wireless communication. However, the inter-device wireless communication may be achieved in a manner other than mentioned above. In a first example, the terminal device6serves as a relay device for another wireless communication. In this case, the board operation data of the board type controller9is wirelessly transmitted to the terminal device6, and the terminal device6wirelessly transmits, to the game apparatus body5, the terminal operation data of the terminal device6together with the received board operation data. In this case, while the terminal device6and the game apparatus body5are directly connected by wireless communication, the board type controller9is connected via the terminal device6to the game apparatus body5by wireless communication. In a second example, the board type controller9serves as a relay device for another wireless communication. In this case, the terminal operation data of the terminal device6is wirelessly transmitted to the board type controller9, and the board type controller9wirelessly transmits, to the game apparatus body5, the board operation data of the board type controller9together with the received terminal operation data. In this case, the board type controller9and the game apparatus body5are directly connected by wireless communication, and the terminal device6is connected via the board type controller9to the game apparatus body5by wireless communication. When another device serves as a relay device to transmit operation data to the game apparatus body5, a device which generates the operation data and the relay device which relays the control data may be electrically connected via a cable.

Further, the terminal device6and/or the board type controller9may be electrically connected to the game apparatus body5via cables. In this case, the cables connected to the terminal device6and/or the board type controller9are connected to a connection terminal of the game apparatus body5. In a first example, the terminal device6and the game apparatus body5are electrically connected via a first cable, and the board type controller9and the game apparatus body5are electrically connected via a second cable. In a second example, the terminal device6is electrically connected to the game apparatus body5via a cable. In this case, the board operation data of the board type controller9may be wirelessly transmitted to the terminal device6and then transmitted to the game apparatus body5via the cable. Alternatively, the board operation data of the board type controller9may be wirelessly transmitted from the board type controller9directly to the game apparatus body5. In a third example, the board type controller9is electrically connected to the game apparatus body5via a cable. In this case, the terminal operation data of the terminal device6may be wirelessly transmitted to the board type controller9and then transmitted to the game apparatus body5via the cable. Alternatively, the terminal operation data of the terminal device6may be wirelessly transmitted from the terminal device6directly to the game apparatus body5.

Further, in the above-mentioned embodiments, the game system1includes one terminal device6and one board type controller9. However, the game system1may be configured to include a plurality of terminal devices6and a plurality of board type controllers9. In this case, the game apparatus body5may be wirelessly communicable with the respective terminal devices6and the respective type controllers9, and may transmit game image data, game sound data, and control data to the respective terminal devices, and receive terminal operation data, camera image data, microphone sound data, and board operation data from the respective terminal devices6and the respective board type controllers9. When the game apparatus body5wirelessly communicates with the plurality of terminal devices6and the plurality of board type controllers9, the game apparatus body5may perform the wireless communications in a time division manner or a frequency division manner.

When the game system1includes a plurality of terminal devices6and a plurality of board type controllers9as described above, a plurality of users are allowed to play more types of games. For example, when the game system1includes two pairs of terminal devices6and board type controllers9, two users are allowed to play a game simultaneously. Further, when the game system1includes two pairs of terminal devices6and board type controllers9, since the game system1includes three display units, game images for three users can be generated to be displayed on the respective display units.

In the above description, the board type controller9is provided with a plurality of load sensors94. However, if information of the center-of-gravity position of a load applied to the board type controller9is not required in the above-mentioned processing, the board type controller9may be provided with at least one load sensor94.

The exemplary embodiments have been described by using the stationary game apparatus3. However, the information processing program may be executed by an information processing apparatus such as a hand-held game apparatus or a general personal computer. Further, the exemplary embodiments may be applied to, as well as the game apparatus, any hand-held electronic devices, e.g., a PDA (Personal Digital Assistant), a mobile telephone, a personal computer, a camera, and the like. Any device may be connected to the terminal device6and the board type controller9wirelessly or via cables.

In the above description, the information processing is executed by the game apparatus body5. However, at least a part of the process steps in the information processing may be performed by another apparatus provided outside the game system1. For example, when the game apparatus body5is configured to be communicable with another apparatus (e.g., a server or another game apparatus), the process steps in the information processing may be performed by the game apparatus body5in combination with the other apparatus. For example, the other apparatus performs a process of setting a player object, a virtual world and the like, and data relating to an action or an attitude of the player object is transmitted from the game apparatus body5to the other apparatus, and thereby the information processing is performed. Then, image data representing a virtual world generated in the other apparatus is transmitted to the game apparatus body5, and the virtual world is displayed on the monitor2and the LCD61. Thus, when at least a part of the process steps in the information processing is performed by another apparatus, the same processing as the information processing is achieved. At least a part of the process steps in the information processing may be performed by the board type controller9(microcomputer100). Further, the above-mentioned information processing can be executed by one processor or by a cooperation of a plurality of processors which is/are included in an information processing system constituted by at least one information processing apparatus. Further, in the exemplary embodiment, the processes shown in the above-mentioned flowcharts are performed by the CPU10of the game apparatus body5executing the predetermined program. However, a part or the entirety of the processes may be performed by a dedicated circuit included in the game apparatus body5.

Further, the shape of the game apparatus body5, the shapes of the terminal device6, the controller7, and the board type controller9, and the shapes, numbers, and mounting positions of the operation buttons and the sensors, are merely examples. It should be understood that other shapes, numbers, and mounting positions may be adopted. Further, the processing sequences, set values, display modes, and values used in determination which are used in the above-mentioned game process are merely examples. It is understood that other sequences, display modes, and values may be adopted.

Further, the information processing program (game program) may be supplied to the game apparatus body5via a wired or wireless communication circuit, as well as to the game apparatus body5via an external storage medium such as the optical disc4. Further, the information processing program may be previously stored in a nonvolatile storage device inside the game apparatus body5. Examples of an information storage medium storing the information processing program therein include: a CD-ROM, a DVD, any other optical disc-shaped storage medium similar to these media, a flexible disk, a hard disk, a magnetic optical disk, and a magnetic tape. Further, the information storage medium storing the information processing program therein may be a nonvolatile semiconductor memory or a volatile memory. Such a storage medium is a storage medium readable by a computer or the like. For example, it is possible to provide the above-mentioned various functions by causing a computer or the like to read and execute the programs stored in these storage media.

The systems, devices and apparatuses described herein may include one or more processors, which may be located in one place or distributed in a variety of places communicating via one or more networks. Such processor(s) can, for example, use conventional 3D graphics transformations, virtual camera and other techniques to provide appropriate images for display. By way of example and without limitation, the processors can be any of: a processor that is part of or is a separate component co-located with the stationary display and which communicates remotely (e.g., wirelessly) with the movable display; or a processor that is part of or is a separate component co-located with the movable display and communicates remotely (e.g., wirelessly) with the stationary display or associated equipment; or a distributed processing arrangement some of which is contained within the movable display housing and some of which is co-located with the stationary display, the distributed portions communicating together via a connection such as a wireless or wired network; or a processor(s) located remotely (e.g., in the cloud) from both the stationary and movable displays and communicating with each of them via one or more network connections; or any combination or variation of the above.

The processors can be implemented using one or more general-purpose processors, one or more specialized graphics processors, or combinations of these. These may be supplemented by specifically-designed ASICs (application specific integrated circuits) and/or logic circuitry. In the case of a distributed processor architecture or arrangement, appropriate data exchange and transmission protocols are used to provide low latency and maintain interactivity, as will be understood by those skilled in the art.

Similarly, program instructions, data and other information for implementing the systems and methods described herein may be stored in one or more on-board and/or removable memory devices. Multiple memory devices may be part of the same device or different devices, which are co-located or remotely located with respect to each other.

While the exemplary embodiments have been described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is understood that numerous other modifications and variations can be devised. It is also understood that, from the description of specific embodiments, the one skilled in the art can implement the exemplary embodiments in the equivalent range based on the description and on the common technological knowledge. Further, it should be understood that terms used in the present specification have meanings generally used in the art concerned unless otherwise specified. Therefore, unless otherwise defined, all the jargon and technical terms have the same meanings as those generally understood by one skilled in the art of the exemplary embodiments. In the event of any conflict, the present specification (including meanings defined herein) has priority.