U.S. Pat. No. 8,177,643

DETAILED DESCRIPTION

The following detailed description is directed to technologies for providing out-of-band voice communication with interactive voice response services from a game console during gameplay. Utilizing the technologies described herein, a user of a game console, personal computer, or other device may access media features of the device and online service functionality without exiting from or pausing an ongoing game. The user may access these features and services via out-of-band voice communication through a headset or other audio input/output device with a voice recognition engine. By accessing and controlling these features and services with voice commands, the user may continue to utilize the game console's primary controls for gameplay as well as remain focused on the action of the ongoing game.

While the subject matter described herein is presented in the general context of program modules that execute in conjunction with the execution of an operating system and application programs on a computer system, those skilled in the art will recognize that other implementations may be performed in combination with other types of program modules. Generally, program modules include routine, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the subject matter described herein may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.

In the following detailed description, references are made to the accompanying drawings that form a part hereof, and that show, by way of illustration, specific embodiments or examples. Referring now to the drawings, in which like numerals represent like elements through the several figures, aspects of a methodology for providing out-of-band voice communication with interactive voice response services during gameplay will be presented.

Turning now toFIG. 1, details will be provided regarding an illustrative operating environment and several software components provided by the embodiments presented herein.FIG. 1illustrates an exemplary system100for providing out-of-band voice communication with interactive voice response services during gameplay. The system100includes a game console102. The game console102may be a special purposed computing device, such as the XBOX 360® video game and entertainment system from MICROSOFT CORPORATION of Redmond, the PLAYSTATION®3 game console from SONY COMPUTER ENTERTAINMENT INC. of Tokyo, Japan, or the NINTENDO® WII™ game console from NINTENDO COMPANY LTD. of Kyoto, Japan. It will be appreciated that the game console102may represent any computing device executing a game program104, including, but not limited to a personal computer, a set-top box, a cell phone, or a hand-held gaming system.

A user106may interface with the game console102through a game controller108. The game controller108may provide a number of buttons, keys, triggers, joysticks, trackballs, touchpads, accelerometers, and other input devices that the user106utilizes to interact with the game program104or any other software executing on the game console102. The user106may further interface with the game console102using a headset110. The headset110may allow the user to chat with other participants in online, multi-player games, for example. The online, multi-player games may be hosted on a multi-player gaming server112that is connected to the game console102via a network114, such as the Internet. The multi-player gaming server112may be provided by an online game service, such as the XBOX LIVE® online game service from MICROSOFT CORPORATION.

It will be appreciated that the headset110depicted inFIG. 1may be any audio input/output (“I/O”) device known in the art, including, but not limited to, a microphone and a speaker attached to the game console102. It will be further appreciated that the headset110may be utilized by the user106to access other audio features of a game program104executing on the game console102beyond the multi-player chat described above, including, but not limited to, voice commands to control aspect of gameplay or simulated communication within the context of the game scenario, such as communication with air traffic control in a flight simulator.

According to embodiments, when the user106is participating in a game program104executing on the game console102, the console normally routes I/O from the headset110to the audio feature or features of the game program. For example, if the user is participating in a multi-player game hosted on the multi-player gaming server112, the game console102will establish an in-band communication channel116between the headset110and the multi-player gaming server112, as illustrated inFIG. 1, to facilitate the chat feature amongst the participants. The in-band communication channel116may utilize a voice over IP (“VoIP”) connection established over the network114or some other protocol known in the art for two-way voice communication.

If the user106presses a designated button, such as the button118, on the game controller108during gameplay, the game console102uses an internal switching mechanism120to switch connectivity of the headset110from the in-band communication channel116with the multi-player gaming server112or other audio feature of the game program104to an out-of-band communication channel124with a voice recognition engine126A,126B (collectively referred to herein as voice recognition engine126). The internal switching mechanism may be implemented in the hardware of the game console102or may be implemented in the operating system122controlling game console. In one embodiment, the switching mechanism is provided by the game program104executing on the game console102.

The voice recognition engine126is a hardware device or software module configured to receive voice commands from the headset110or other audio I/O device, utilize speech recognition technology to parse the command text from the voice command, and execute a function or functions associated with the command text, as will be described in more detail below in regard toFIG. 2. In one embodiment, the voice recognition engine126consists of an on-system voice recognition engine126A implemented onboard the game console102. In another embodiment, the voice recognition engine126consists of an off-system voice recognition engine126B implemented in an external server or device and connected to the game console102via the network114.

The choice of implementation may depend on a number of factors, including, but not limited to, the scope of the functions available via associated voice commands and the overall processing power of the game console102. For example, functions related to media playback on the game console102itself may be better suited for the on-system voice recognition engine126A. For voice commands requiring more processing power for recognition, such as searching online media by name or artist, the off-system voice recognition engine126B may be the more appropriate choice. In a further embodiment, the voice recognition engine126may consist of a combination of both on-system and off-system components.

Once out-of-band connectivity between the headset110and the voice recognition engine126is established through the switching mechanism120, the user may issue voice commands using the headset to the voice recognition engine. The voice recognition engine is able to receive the voice commands from the headset110and send aural responses to the headset without interrupting gameplay on the game console102. In addition, any in-band communication channel116established between the headset110and the multi-player gaming server112or other audio feature may remain active. Once the processing of voice commands from the user is complete, the game console102can utilize the switching mechanism120to switch connectivity of the headset110back to the in-band communication channel116in order to continue using multi-player chat or other audio feature of the active game. Processing of voice commands may be determined to be complete when the user106issues a “quit” voice command, when the execution of the function associated with a particular voice command finishes, when enough silence is detected in the user's utterance to be considered an end of speech marker, or when the user releases the designated button118on the game controller108, for example.

In order to execute the function or functions associated with each recognized voice command, the voice recognition engine126may have access to a variety of interactive voice response (“IVR”) systems, communication networks, and other external services. In one embodiment, to facilitate out-of-band voice communication with a third party128during gameplay, the voice recognition engine126has access to a telephone network containing a telephone130of the third party. Upon receiving a voice command associated with establishing a third party telephone call, the voice recognition engine126may establish a communication link between the headset110on the game console102and the telephone130via the public switched telephone network (“PSTN”), or via the Internet utilizing VoIP technology, for example.

In another embodiment, the voice recognition engine126is connected to a voice portal service132, such as that provided by TELLME NETWORKS INC. of Mountain View, Calif., in order to facilitate voice commands from the user106to retrieve information from the Internet. The voice portal services may allow the user106to search and download music or other media; access game hints; retrieve news, sports, weather or stock quotes; or find movies, showtimes and theaters, all using voice commands through the out-of-band communication channel124established by the game console102without interrupting gameplay. The voice recognition engine126may provide user specific information to the voice portal service132, such as the user's profile, statistics, or information about the current game being played on the game console102, in order to provide context for processing the voice commands of the user106.

In other embodiments, the voice recognition engine126may be further connected to IVR systems, such as the IVR system134, provided by individual vendors, allowing the user106to order pizza, access social networking functionality, or perform other functions using voice commands sent through the out-of-band communication channel124, without interrupting gameplay on the game console102. It will be appreciated that many other IVR systems and services could be imagined by one skilled in the art than those described herein with which the voice recognition engine126could have access in order to provide voice command-based services to the user106. It is intended that this application include all such IVR systems and services.

In addition, the voice recognition engine126may have access to local functions provided by the game console102, like media playback functions, in order to allow the user106to select the current music track which is being played in combination with the game audio, for example, without interrupting gameplay or the execution of the game program104on the console.

Referring now toFIG. 2, additional details will be provided regarding the embodiments presented herein. In particular,FIG. 2illustrates a routine200for providing out-of-band voice communication with interactive voice response services from a game console during gameplay. It should be appreciated that the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations described herein are referred to variously as states operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. It should also be appreciated that more or fewer operations may be performed than shown in the figures and described herein. These operations may also be performed in a different order than those described herein.

The routine200begins at operation202, where the game console102detects a button press on the game controller108or other signal from the user106indicating that the user wishes to issue a voice command. As described briefly above, any button, such as the button118, or combination of buttons on the game controller108may be configured to invoke the out-of-band connection with the voice recognition engine. It will be appreciated that other methods beyond a button press on the game controller108could be used to indicate the user106wishes to issue a voice command, including, but not limited to, pressing a button on the headset110or uttering an attention word into the headset, such as “Computer.” It is intended that this application include all such methods.

Once the button press or other signal is detected, the routine200moves from operation202to operation204, where the game console102switches the headset connection from in-band game communication to out-of-band communication with the voice recognition engine126. According to embodiments, this is accomplished using an internal switching mechanism120of the game console102without interrupting gameplay.

For example, the user106may be actively engaged in an in-band game communication through the headset110, such as a multi-player chat hosted on the multi-player gaming server112, when the user presses the button118on the game controller108. The game console102detects the button press and establishes an out-of-band communication channel124with the voice recognition engine, either an on-system voice recognition engine126A or an off-system voice recognition engine126B, depending on the chosen implementation. The game console102then switches the headset I/O from the in-band communication channel116with the multi-player gaming server112to the out-of-band communication channel124with the voice recognition engine126using the switching mechanism120.

In one embodiment, the in-band communication channel116is “kept alive” while communication between the headset and the voice recognition engine126is active, allowing the user106to return to the multi-player chat or other in-band game communication when the voice command processing is complete. In a further embodiment, once the headset connection is switched to out-of-band communication with the voice recognition engine126, the voice recognition engine responds to the user106with a ready signal indicating that the voice recognition engine is ready to receive a voice command. For example, the voice recognition engine126may play a tone or speech such as “ready” to the headset110to indicate that the voice command can be spoken by the user106. Alternatively, the voice recognition engine126may cause the game console102to overlay a status message on the game display indicating that the voice recognition engine is ready to receive a voice command.

From operation204, the routine200proceeds to operation206, where the voice recognition engine126receives the voice command from the user106. For example, the user106may say “call Joe” into the headset110connected to the game console102. Once the voice command is received, the routine200moves to operation208, where the voice recognition engine126utilizes speech recognition technology to parse the command details from the voice command received. It will be appreciated that many speech recognition technologies commonly known in the art may be utilized to parse the voice command. According to one embodiment, the voice recognition engine126converts the voice command to a textual representation of the command before processing the command. In another embodiment, the voice command may be matched with pre-recorded voice commands for the current user106in order to identify the command desired.

In a further embodiment, a hybrid method may be utilized, where the voice recognition engine126uses speech-to-text conversion to recognize a root command in the voice command, and then matches the remaining portions of the voice command to pre-recorded elements. For example, if the user106issues the voice command “call Joe,” the voice recognition engine126may parse the root command “call” from the voice command indicating that the user wishes to place a telephone call to a third party. The voice recognition engine126may then search a list of pre-recorded names of third parties to find the best match for the remaining element of the voice command, in other words “Joe.” Similarly, if the user106issues the command “Play Smashing Pumpkins,” the voice recognition engine126may recognize the root command “play” in the voice command indicating that the user106wishes to play media content on the game console102. The voice recognition engine126may then search through the titles and artists of all media content accessible to the game console102in order to find content best matching the remaining element of the voice command “Smashing Pumpkins.”

According to another embodiment, after recognizing the voice command, the voice recognition engine126may issue a confirmation to the user in order to verify the command parsed from the voice command. The voice recognition engine126may then wait for approval from the user106before proceeding to execute any function associated with the command. For example, if the users says “Call Joe,” the voice recognition engine126may respond by playing the speech “Would you like to call Joe Thompson at home?” over the headset110before placing the call. The voice recognition engine126may wait for the user106to say “Yes” or “Ok” before establishing the telephone call to the identified third party. In a further example, the voice recognition engine126may cause the game console102to overlay a verification message on the game display and wait for an aural response from the headset or additional button press on the game controller108before proceeding.

The routine200then proceeds from operation208to operation210where the voice recognition engine126executes any function associated with the recognized voice command. For example, upon recognizing the command “call Joe,” the voice recognition engine126may issue a command to a call control server over the network114to establish a telephone connection between the game console102and a telephone130at Joe's location. As discussed above, this may be accomplished using VoIP technology over the network114, a traditional PSTN, a combination of the two, or some other communication technology or protocol known in the art. In order to establish the telephone connection, the voice recognition engine126must know the location or telephone number assigned to Joe. This may have been previously entered by the user106or may appear in an address book or some other data file associated with the user and available to the voice recognition engine126.

In another embodiment, the voice recognition engine126simply connects the headset110through the out-of-band communication channel124to an external IVR system134to allow the user106to continue issuing voice commands to complete the function. For example, upon recognizing the command “what's the weather outside,” the voice recognition engine126may establish an out-of-band communication channel between the headset110and a voice portal service132, as described above in regard toFIG. 1. The voice recognition engine126may provide context information regarding the user106to the voice portal service132to allow the voice portal service determine the weather information for the user's location and respond to the user through the out-of-band communication channel and headset110. However, if the voice portal service132requires more information, such as the user's location, then the voice portal service may allow the user106to further interact with the service using voice commands.

In a further example, upon recognizing the command “order pizza,” the voice recognition engine126may connect the headset110through the out-of-band communication channel to an IVR system134operated by a pizza delivery company. The IVR system134may then use voice prompts and internal speech-recognition technologies to walk the user106through the process of ordering the pizza.

According to another embodiment, the voice recognition engine126may perform all the speech recognition required to execute the function associated with the command. For example, upon recognizing the command “get game hints,” the voice recognition engine126may prompt the user106for additional information required for selecting game hints and then perform a query at a website containing the desired game hint data. Upon retrieving the data, the voice recognition engine126may convert the game hint data to speech and play the speech to the user106through the headset110.

From operation210, the routine200ends. As described briefly above, upon completion of a voice command, the game console102may restore the connection between the headset110and the in-band communication channel, allowing the user106to return to the audio feature of the game previously being utilized through the headset. In addition, the voice recognition engine126or a connected voice portal service132or IVR system134may monitor the progress of the execution of the function associated with the voice command and provide status messages to the user106. For example, if a pizza is ordered by a voice command to the voice recognition engine126through an IVR system134, the IVR system may cause the game console102to provide a status message as an overlay on the game display upon delivery of the pizza.

FIG. 3shows an example computer architecture for computers300capable of executing the software components described herein for providing out-of-band voice communication with interactive voice response services during gameplay, in the manner presented above. The computer architecture shown inFIG. 3illustrates a conventional game console, desktop computer, laptop, or server computer and may be utilized to execute any aspects of the software components presented herein described as executing on the game console102or other computing platform.

The computer architecture shown inFIG. 3includes a central processing unit302(CPU), a system memory308, including a random access memory314(RAM) and a read-only memory316(ROM), and a system bus304that couples the memory to the CPU302. A basic input/output system containing the basic routines that help to transfer information between elements within the computer300, such as during startup, is stored in the ROM316. The computer300also includes a mass storage device310for storing an operating system318, application programs, and other program modules, which are described in greater detail herein.

The mass storage device310is connected to the CPU302through a mass storage controller (not shown) connected to the bus304. The mass storage device310and its associated computer-readable media provide non-volatile storage for the computer300. Although the description of computer-readable media contained herein refers to a mass storage device, such as a hard disk or CD-ROM drive, it should be appreciated by those skilled in the art that computer-readable media can be any available computer storage media that can be accessed by the computer300.

By way of example, and not limitation, computer-readable media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. For example, computer-readable media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROM, digital versatile disks (DVD), HD-DVD, BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computer300.

According to various embodiments, the computer300may operate in a networked environment using logical connections to remote computers through a network such as the network114. The computer300may connect to the network114through a network interface unit306connected to the bus304. It should be appreciated that the network interface unit306may also be utilized to connect to other types of networks and remote computer systems. The computer300may also include an input/output controller312for receiving and processing input from a number of other devices, including the game controller108, the headset110, a keyboard, a mouse, or an electronic stylus (not shown inFIG. 3). Similarly, an input/output controller may provide output to a display, a printer, or other type of output device.

As mentioned briefly above, a number of program modules and data files may be stored in the mass storage device310and RAM314of the computer300, including an operating system122suitable for controlling the operation of a game console, desktop, laptop, or server computer. The mass storage device310and RAM314may also store one or more program modules. In particular, the mass storage device310and the RAM314may store the on-system voice recognition engine126A or a game program104, each of which was described in detail above in regard toFIG. 1. The mass storage device310and the RAM314may also store other types of program modules or data.

Based on the foregoing, it should be appreciated that technologies for providing out-of-band voice communication with interactive voice response services during gameplay are provided herein. Although the subject matter presented herein has been described in language specific to computer structural features, methodological acts, and computer readable media, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features, acts, or media described herein. Rather, the specific features, acts, and mediums are disclosed as example forms of implementing the claims.

The subject matter described above is provided by way of illustration only and should not be construed as limiting. Various modifications and changes may be made to the subject matter described herein without following the example embodiments and applications illustrated and described, and without departing from the true spirit and scope of the present invention, which is set forth in the following claims.