U.S. Pat. No. 10,630,773

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments of the claimed subject matter, a system and method for network coupled cloud gaming, examples of which are illustrated in the accompanying drawings. While the claimed subject matter will be described in conjunction with the preferred embodiments, it will be understood that they are not intended to limit these embodiments. On the contrary, the claimed subject matter is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope as defined by the appended claims.

Furthermore, in the following detailed descriptions of embodiments of the claimed subject matter, numerous specific details are set forth in order to provide a thorough understanding of the claimed subject matter. However, it will be recognized by one of ordinary skill in the art that the claimed subject matter may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits have not been described in detail as not to obscure unnecessarily aspects of the claimed subject matter.

Some portions of the detailed descriptions which follow are presented in terms of procedures, steps, logic blocks, processing, and other symbolic representations of operations on data bits that can be performed on computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. A procedure, computer generated step, logic block, process, etc., is here, and generally, conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present claimed subject matter, discussions utilizing terms such as “storing,” “creating,” “generating,” “protecting,” “receiving,” “sending,” “determining,” “selecting,” “initiating,” “mapping,” “translating,” “accessing,” “writing,” or the like, refer to the action and processes of a computer system or integrated circuit, or similar electronic computing device, including an embedded system, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

System and Method for Network Coupled Gaming

Embodiments of the present invention reduce the cost of accessing game streaming services by removing the need for separate hardware for running a game streaming client. Embodiments include an internet connectable controller that is connected to a user's local network and thereby connected to the Internet and game streaming service. The game can then be streamed independently to an Internet connected display logged into the game streaming service without the need for a game console. For instance, the game can be independently streamed to a television, a tablet, a phone, or other display device that is capable of executing a game streaming client.

Embodiments of the present invention lower the latency between the user and the gaming service. Having the controller directly connected to the user's network eliminates the extra step of a processing by a local gaming console thereby reducing user input latency, audio input/output latency, and physical controller feedback latency.

Further, problems with audio routing and the limits of audio capabilities of the Operating System (OS) of the game console are eliminated. With a controller directly connected to the game streaming service, the handling of audio routing can be performed by the more capable game streaming service systems. Further, the user controller can be tied together with the audio streams so the input and output audio streams can be tailored to each individual user.

Embodiments are described with respect to a game streaming service but this is exemplary only and embodiments may be used with any gaming system, cloud based gaming, or type of application or content.

FIG. 1shows an exemplary operating environment of a cloud based gaming system.FIG. 1depicts an exemplary operating environment100with game controllers that communicate with local gaming console which sends information from the game controllers to a cloud based gaming system150. The exemplary operating environment100includes game controllers102a-b, a gaming tablet104, a gaming console110, a display device112, a router120, a wireless access point122, and the cloud based gaming system150.

The game controller102ais a wireless game controller that is wirelessly connected to the gaming tablet104. The tablet104is wirelessly connected to the wireless access point122. The wireless access point122is connected to the router120. The wireless access point122may be integrated into the router120or vice versa.

The game controller102bis a wireless game controller that is wirelessly connected to the gaming console110. The game controller102bmay be connected to the gaming console110by a wired connection. The gaming console110is connected to the router120. The gaming console110is further connected to a display device112. The display device112may be a television, monitor, etc.

The router120is connected to the cloud based gaming system150via the Internet. The tablet104and the gaming console110execute a game client which communicates with the cloud based gaming system150. The router120routes communications between respective game streaming clients executing on the table104and the gaming console110and the game streaming server150.

A user can start up a cloud based game through the tablet104. The game executes on the cloud based gaming system150and output of the game is sent and displayed on a display screen of the tablet104. The tablet104receives input from the user via the game controller102a.

Alternatively, a user can start up the game client through the gaming console110. The game executes in the cloud based gaming system150and the output of the game is sent and displayed on a display screen of the display device112. The gaming console110receives input from the user via the game controller102b.

The controllers102a-bsend button presses to the tablet104or the gaming console110, both of which are executing respective game clients. Each game client then repackages the respective button press information and sends to the respective repackaged information to the cloud based gaming system150. The game client also receives audio and other feedback (e.g., rumble information for vibrating the controller as directed by the game), repackages the information and sends it to the appropriate gaming controller. In both directions, the data is repackaged by the game client and then sent to the cloud based gaming system150or a controller thereby adding latency to the communication. Thus, the tablet104and the game console110add latency to communications to and from the game steaming service150.

The tablet104and the gaming console110can have a simple audio input and audio output device, for instance, supporting only a single input and output audio stream. This may create problems when multiple audio-enabled controllers are used. For example, if multiple audio streams should be directed to/from the multiple controllers, the game client needs to be capable of routing the audio to/from the correct controller. Such capability and the associate complexity are beyond the abilities of the hardware and software of the game client.

FIGS. 2-6illustrate example components used by various embodiments of the present invention. Although specific components are disclosed inFIGS. 2-6, it should be appreciated that such components are exemplary. That is, embodiments are well suited to having various other components or variations of the components recited in2-6. It is appreciated that the components inFIGS. 2-6may operate with other components than those presented, and that not all of the components ofFIGS. 2-6are required to practice embodiments of the present invention as described herein.

FIG. 2shows an exemplary operating environment of a game streaming service with controllers and a display device coupled thereto, in accordance with various embodiments of the present invention.FIG. 2depicts an exemplary operating environment200with game controllers that communicate with a wireless access point and a game streaming service250. The exemplary operating environment200includes game controllers202a-b, a display device212, a router220, a wireless access point222, the game streaming service250, and a game streaming server260.

The game controller202ais a wireless game controller that is wirelessly connected to the wireless access point222. In some embodiments, the game controller202amay be coupled to the wireless access point222or the router220by a wired connection (e.g., Ethernet). The wireless access point222is communicatively coupled to the router220. In some embodiments, the wireless access point222is integrated into the router220or vice versa.

The game controller202bis a wireless game controller that is wirelessly connected to the wireless access point222. The game controller202bmay be coupled to the wireless access point222or the router220by a wired connection (e.g., Ethernet). The controllers202a-bmay further have respective audio interfaces (e.g., audio input jacks, Bluetooth modules, etc.) configured for coupling of respective microphones, headsets, headphones, etc.

The game controllers202a-bare configured for communicating with the game streaming service250without the need for repackaging of respective button information, audio information, and feedback information. The game controllers202a-bare thus able to communicate controller information (e.g., button information, audio information, and feedback information) to the game streaming service250independent of a tablet (e.g., the tablet104) or a gaming console (e.g., the gaming console110). The game controllers202a-bhave improved responsiveness because the controller202a-bcommunicate with the game streaming service250without the latency added by an intermediate device (e.g., the tablet104or the gaming console110).

The display device212is coupled to the router220. The display device212may be a television (TV), projector, monitor, etc. The display device212may include one or more speakers and/or audio jacks for outputting audio. The display device212may also have a microphone and/or one or more audio jacks for audio input.

The display device212is configured for accessing and communicating with one or more types of networks (e.g., wired and/or wireless) and is configured for executing a game streaming client for the displaying and decoding of video and audio from the game streaming service250. The display device212may execute a game streaming client which allows users to login to the game streaming service250to view the video output and/or hear audio output of a game. The display device212is further configured for allowing one or more users to log into the game streaming service250. After the one or more users have logged into the game streaming service250, the appropriate video and audio streams are sent to the display device212. For example, if two users are logged into the game streaming service250via the display device212, the game streaming service250sends split screen video and associated audio to the display device212. The game streaming client executing on the display device212thus can display visual and audio feedback of the commands issued by the controllers202a-b. The responsiveness of the video and audio is thus enhanced because the latency of having the video processed by an intermediate device (e.g., the game console110) is removed. The game streaming server260may handle execution of the game for the game streaming service250. The game streaming server260and the game streaming client executing on the display device212may communicate through a customized protocol designed for the low latency communication between the display device212and the game streaming server260.

In some embodiments, the display device212may be a mobile device, e.g., a smartphone or a tablet, thereby allowing a user to play a game with the controller202aat any location of the user's choosing, e.g., a coffee shop, library, etc.

The router220is coupled to the game streaming service250by one or more networks, including the Internet. The router220routes communications between the game streaming client executing on the display device212and the game streaming server250. The router120further routes communications to and from the controllers202a-band the game streaming server250.

Respective users of the controllers202a-blogin the controllers202a-binto the game streaming service250, as described below. The users further log into their respective accounts on the game streaming server250via the display device212. The controllers202a-bcan then be used to select a game to be played via the display device212. During the game play, button information and/or audio information from the controllers202a-bis sent to the game streaming service250where the game is executing and the video and/or audio (e.g., encoded video and/or audio) output of the game is sent the display device212for output. The game streaming service250further determines any feedback, for instance tactile feedback like rumble or vibration feedback, based on button information and/or audio information which is sent to the controllers202a-b, e.g., when a player's car crashes in the game.

The game streaming service250includes one or more of a game streaming server260. Embodiments of the present invention include the game streaming server260configured to communicate directly with the controllers202a-band the display device212, e.g., without communicating through a gaming console (e.g., the gaming console110). For example, each of the controllers202a-bsend individual packets to the game streaming server206instead of the communications from the game controllers202a-bgoing to a gaming console (e.g., the gaming console110) and then to the game streaming server260. The game streaming server260and the game controllers202a-bmay communicate through a customized protocol designed for the low latency communication between the controllers202a-band the game streaming server260.

FIG. 3shows an exemplary operating environment with multiple locations, in accordance with various embodiments of the present invention.FIG. 3depicts a game being played using a game streaming service350at locations370-380and associated audio routing.

Controllers302a-b, a headset324, a wireless access point322a display device312, and a router320are located at location370. A controller392, a headset390, a wireless access point394, a display device396, and a router398are located at location380. The wireless access points322and394, the routers320and398, the display devices312and396, and the controllers302a-band392can operate in a substantially similar manner as similar elements as described above.

The headset324is coupled to the controller302a. The headset390is coupled to the controller392. The headsets324and392may be coupled to the controllers302aand392via a wireless connection (e.g., Bluetooth) or wired connection (e.g., a Universal Serial Bus (USB) connection). The headsets324and392allow respective users to hear audio output from a game running on the game streaming service350. The headsets324and392further allow respective users to talk to each other or other players via respective microphones and give voice commands to the game running on the game streaming service350.

The game streaming service350supports the playing of a game among multiple users that may be at one or more locations. The locations370and380may be different houses, places of business, dormitories, convention centers, etc. Locations307and380may be geographically close (e.g., on the same block or street) or distant (e.g., across country or across the planet).

The respective users of the controllers302a-blogin to the game streaming service350with the display device312(e.g., using a television remote control) and configure their respective controllers302a-bfor communicating with the game streaming service350as described with respect toFIG. 4. For example, the user of the controller392logs into the game streaming service350with display device396and configures the controller392for communicating with the game streaming service350as described with respect toFIG. 4. The respective users of the controllers302a-band390further log into their game streaming accounts of the game streaming service350with their respective controllers.

The game streaming service350further supports individual audio streams for each player. The controllers302a-band392support sending one or more respective indicators and/or information to the game streaming service350that a respective audio device, e.g., a headset, head phones, a microphone, etc., is coupled to the respective controller. For example, the indicator may indicate whether headphones only are coupled to the controller, whether microphone only is coupled to the controller, or both headphones and microphone of a headset are coupled to the controller.

A game streaming server360of the game streaming service350receives the indicators of audio devices coupled to the controller. The game streaming server360determines, based on the respective indicators of the audio devices coupled to the controllers, how and where audio streams will be routed.

Some operating systems, e.g., the Android operating system, available from Google, Inc., of Mountain View, Calif., are designed around having a single audio input and audio output. For example, if four users are playing a racing game that is displayed on a single television and in different parts of the tracks, the television will be sent an audio mashup or combination of the four respective audio tracks. In contrast, having one controller for each of the four players, each with respective audio jacks and respective communication with the game streaming service350, allows for a personalized audio experience. For example, if player one is coming up behind player three, player one will hear the audio of player three coming up behind player one. Similarly, player three can hear the audio of player one being in front of player three instead of the audio streams being combined together. As such, if a gaming console (e.g., the game console110) runs the Android operating system, an associated game service will combine all of the audio streams into a single stream and the gaming console will not have the opportunity or ability to split up the audio streams.

The game streaming server360(and thereby the game streaming service350) is configured to send individual audio streams for each respective controller and each display device. Each of the controllers302a-band392and the display devices312and396may support audio input and output.

As shown inFIG. 3, the controllers302aand392have coupled headsets324and394, respectively, and transmit indicators that headsets are coupled thereto to the game streaming service350. The game streaming server360can then determine that individual respective audio streams are to be sent to the controllers302aand394. If both controllers302a-bdid not have coupled headsets, the audio sent to the display device330would be a combined audio output of each of the respective audio streams.

In some embodiments, due to the headset390being coupled to the controller392, the game streaming server360determines that an audio stream does not need to be sent to the display device396. The video stream of the game may thus be sent to the display device396without an audio stream. This allows the player using the controller392to play without disturbing other people at the location380because the audio stream for the controller392is through the headset390and not out of the speakers of the display device396.

The controller302btransmits an indicator that there is not a headset or other audio device coupled to the controller302b. Based on the indicator from the controller302b, the game streaming server360determines that the audio stream associated with the controller302bshould be sent to the display device312. The display device312then outputs the audio stream as audio330, e.g., out of one or more speakers. In some embodiments, the display device312may be sent a combined audio stream.

It is appreciated that if there was an audio device coupled to the controller302b, the game streaming server360could have the audio stream for the controller302bsent to the controller302binstead of the display device312. This would thereby allow the players of the game at the location370to play without disturbing others because the audio streams for each of the controllers302a-bwould be going through respective coupled headsets and there could be no audio output of speakers of the display device312.

Based on the audio device indicators, the game streaming service350can further support voice input from the controllers302a-band392. For example, the game streaming service350can listen to the respective microphone audio streams from the controllers320aand392for audio or voice commands. If the display device312has a microphone coupled thereto, the game streaming service350can receive audio or voice commands from the user of the controller302b. Based on the voice commands, the game streaming server360can determine to change the display output sent to the display devices312and396. For example, if the user of the controller302asays the command “map,” the game streaming server360may analyze and perform voice recognition and in response update the portion of the video displayed on the display device312for the controller302ato display a map while the portion of video displayed for the controller302bcontinues to show the game. As another example, the user of controller302acould say the command “infrared mode,” and the portion of the video displayed on the display device312for the controller302awould be updated to show heat signatures of the characters of the game.

In some embodiments, the audio jacks of a controller may be used to send a voice based password to the game streaming service350. For example, the user of the controller302amay provide a voice print password which is authenticated by the game streaming service350prior to accepting commands to enter a game from the controller302a. This can add a layer of security to the controller and prevent unauthorized access to the account associated with the controller.

During game play, the game streaming server360may also determine tactile feedback for the controllers302a-band392. For example, based on a car hitting a wall that is being controlled with the controllers302a, the streaming gaming server360can determine and send a rumble signal which is sent to the controller302ato vibrate the controller302a.

FIG. 4shows exemplary communications for configuring a controller, in accordance with various embodiments of the present invention.FIG. 4depicts an exemplary communication sequence for configuring a controller402to communicate with a game streaming service450. The controller402may be configured using a device410, e.g., a smartphone, tablet, computer, etc., or via a voice commands from a user420.

In some embodiments, the controller402is configured (e.g., by default) to search for a default controller configuration Wi-Fi network, e.g., with a Service Set Identifier (SSID) of “Controller Network Setup,” under certain conditions. The controller402may search for the default controller configuration Wi-Fi network in response to a user pressing a network configuration button, a reset button, or other button combination. In some embodiments, the controller402may search for the default controller configuration Wi-Fi network when no known network if is found. A network configuration module404configures the controller402to connect to the default network or a local network based on received network information, as described herein.

A user downloads an application414to a data store412of the device410(e.g., cellphone, tablet, computer, etc.) which when executed configures the device410to act as a wireless access point for a Wi-Fi network with a default network name450, e.g., with an SSID of “Controller Network Setup.” The device410broadcasts the default network name450and associated information for allowing a controller to connect to the wireless network setup by the device410.

The controller402recognizes the default network name450and sends information to connect to default network452. In response to a connection with the controller402, the application414presents the user with one or more screens and input fields for configuring the controller402. In some embodiments, the controller402is coupled to the device via a wired connection, e.g., USB. The application414allows the user to enter network configuration information (e.g., network name or SSID), wireless key or password, Dynamic Host Configuration Protocol (DHCP) or static Internet Protocol (IP) address information, subnet, gateway, etc., and game account or game streaming service account information (e.g., login and password). In some embodiments, the application414presents the user with a list of networks visible to the device410and prompts the user for a password to be used by the controller402to access a selected network. If the user is at a business (e.g., a coffee shop or hotel), the application414may be used to accept any license agreements that are necessary for the controller402to access the local network of the business.

The network configuration and game streaming service account information454is then sent to the controller402. The network configuration module404configures the controller402based on the received the network configuration and game streaming service account information. The network configuration module404stores the network configuration and login information454in the configuration storage module408. The configuration storage module408may be used by the controller402to access to networks that have previously been accessed by the controller402.

If connected to the default configuration network, upon configuring itself the controller402can disconnect from the default configuration network and connect to network associated with the network configuration information. In response to connecting the local network based on the network configuration information, the controller402connects to and logs into the game streaming service associated with the game streaming account information (e.g., the game streaming service250). For example, a user may use his or her cell phone, when at a friend's house, to configure his or her controller to access the friend's local wireless network.

In some embodiments, the network configuration information can include a Media Access Control (MAC) address (e.g., the MAC address of the device being used to configure the device) that the controller402is to use for communicating with the network specified in the network configuration information. This use of the provided MAC address by the controller402can handle situations, e.g., in a hotel, where a user is limited in the number of devices he or she can connect to a local network.

In some embodiments, the network configuration information and/or game streaming account information can be input into the controller402with voice commands using voice recognition. For example, a user could plug in a headset into the controller402, push a configuration button (e.g., or reset configuration button) and be provided voice prompts460from the voice prompt configuration module410. The user can respond by providing voice responses including a wireless network SSID, wireless password, and game streaming account information. Based on the voice responses462, the voice prompt configuration module410performs voice recognition and provides the output to the network configuration module404which then configures the controller4021. Voice recognition may thus be used in place of the application414executing on the device410coupled to the controller402.

FIG. 5shows exemplary components of a controller in accordance with various embodiments of the present invention.FIG. 5depicts exemplary components of a controller supporting audio input and output, tactile feedback, button input, and direct communication with a network and a game streaming service (e.g., the game streaming service250). The controller500is thus able to take advantage of the lower latencies available through the direct network and Internet connection.

FIG. 5further depicts the components of an exemplary controller in accordance with embodiments of the present invention providing the execution platform for certain hardware-based and software-based functionality. Although specific components are disclosed in controller500, it should be appreciated that such components are examples. That is, embodiments of the present invention are well suited to having various other components or variations of the components recited in controller500. It is appreciated that the components in controller500may operate with other components other than those presented, and that not all of the components of controller500may be required to achieve the goals of controller500.

The controller500includes a storage502, a network controller (NC)504, a memory506, an input module508, a processing unit510, an audio interface512, a communications bus530, and a power source520. The power source520provides power to controller500and may be a DC or AC power source. The communications bus530communicatively couples the components of controller500.

The processing unit510and the NC504can be integrated into a single integrated circuit die and processing unit510and NC504may share various resources, such as instruction logic, buffers, functional units and so on, or separate resources may be provided for processing and general-purpose operations. In some embodiments, the processing unit510and/or the NC504may be programmable circuits or processors, e.g., a field-programmable gate array (FPGA). Some embodiments may comprise a firmware update that is configured to allow the controller500to communicate with a game streaming service via the NC504(e.g., via a wireless access point and the Internet) instead of with a gaming console (e.g., the gaming console110). The firmware update would thus enable a controller to communicate with a wireless network instead of a gaming console. The firmware may allow the controller to search for the aforementioned default controller network configuration Wi-Fi network.

The storage502is configured for storing game streaming service login information, network information, and/or software for execution on the controller500. The storage502may include non-volatile memory and further be configured to download firmware or microcode for updating the processing unit510and/or the NC504. The storage502can include storage for storing a list of known or previously configured communication networks. For example, the storage502may include a list of SSID and key pairs and network settings (e.g., DHCP settings) of wireless networks that the controller has been configured to access. This thereby allows the controller500to access the communication networks with associated information in the storage502whenever the controller is in range of the communication networks.

The network controller (NC)504is coupled to communications bus530and is configured for communication with one or more pieces of network equipment including a wireless access point (e.g., the wireless access point222and a game streaming service (e.g., the game streaming service250). It is appreciated that the network controller504in combination with other components are configured for communicating with a game streaming service (e.g., the game streaming service250) without the need for a gaming console (e.g., the game console110). The network controller504may use the memory506in order to send and receive communications and/or provide received data to other components of the controller500.

The input module508allows entry of user commands into controller500which may include, but are not limited to, one or more buttons, one or more joysticks, one or more navigation pads, one or more directional pads, keyboards (e.g., QWERTY), up/down buttons, left/right buttons, trigger buttons, select buttons, mode buttons, and the like.

The processing unit510receives commands via input module508and may receive audio via the audio interface512. In some embodiments, the processing unit510is a Central Processing Unit (CPU). The processing unit510can control a variety of operations including, but not limited to, voice recognition, voice processing, and sending voice commands for processing (e.g., by the game streaming service250), receiving and sending commands from the input module508, handling communications with a game streaming service via the network controller504, configuration of the network controller504for communication with the game streaming service, and logging into a game streaming service. The processing unit510can use the memory506in order to perform or carry out the aforementioned functions and/or operations.

The audio interface512is configured for coupling audio equipment including, but not limited to, a headset, headphones, a microphone, etc. The audio interface512may provide received voice commands or voice input to processing unit510and output game audio to a headset or headphones. It is appreciated that audio interface512may output to audio equipment integrated within controller500, e.g., an integrated speaker.

FIG. 6shows exemplary components of a display device in accordance with various embodiments of the present invention. Although specific components are disclosed in a display device600it should be appreciated that such components are examples. That is, embodiments of the present invention are well suited to having various other components or variations of the components recited in display device600. It is appreciated that the components in display device600may operate with other components than other those presented, and that not all of the components of display device600may be required to achieve the goals of display device600. It is appreciated that embodiments of display device600include a variety of internet capable display devices, including but not limited to televisions, Cathode Ray Tube (CRT) displays, computer monitors, Liquid Crystal Displays (LCDs), plasma displays, projection displays, projectors, etc.

FIG. 6depicts an exemplary display device and associated components. The display device600includes a content receiver602, a display controller604, a memory606, a command processor608, a processing unit610, an audio controller612, a display screen614, and a network controller (NC)616. The communications bus630communicatively couples the components of the display device600.

The content receiver602receives content for display by the device600. The content receiver602may receive signals including content from a variety of sources including, but not limited to, computers, computer networks, portable devices, set top boxes, over the air broadcasts, cable broadcasts, satellite broadcasts, Digital versatile Discs (DVDs), Blu-ray discs, Digital Video Broadcasting—Handheld (DVB-H), Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting Satellite services to Handhelds (DVB-SH), Digital Audio Broadcasting (DAB), Digital Video Broadcasting IP Datacasting (DVB-IPDC), and Internet Protocol Television (IPTV), etc.

The display controller604controls the display screen614of the display device600. The display screen614can be a CRT, a LCD, a plasma display, projection based display, and a Digital Light Processing (DLP) display, etc., configured for output of video and/or images (e.g., in conjunction with audio controller612).

The memory606can be volatile or non-volatile memory used by various components of the display device600. The memory606can be used by the network controller616in order to send and receive communications and/or provide received data to other components of the controller500. The memory606can be used by the processing unit510in order to perform or carry out the aforementioned functions and/or operations, and execute a game streaming client for displaying game output and outputting game audio.

The command module608receives commands (e.g., via a remote control). The command module608may receive commands via a variety of receivers including, but not limited to, infrared receivers and radio frequency receivers. The commands may have been issued via remote control (e.g., infrared) and can be used to configure the display device600for access to one or more networks, including the Internet, and a game streaming service (e.g., account information associated with the game streaming service250).

The processing unit610is configured for executing an application for configuring access to a game streaming service including logging into the game streaming service and outputting audio and video associated with a game being played by the game streaming service. In some embodiments, the application executed by the processing unit610is configured to decode and output audio and video streamed from a game streaming service (e.g., the game streaming service250).

Audio controller612controls audio output for display device600including a variety of outputs including, but not limited to, 2, 2.1, 3.1, 5.1, 6.1, 7.1, and 8.1 channel audio. The audio content may be received via content receiver602and/or the network controller616from a game streaming service (e.g., the game streaming service250). It is appreciated that audio controller612may output to audio equipment integrated within display device600.

The storage618is configured for storing game streaming service account (e.g., login and password) information, network information, and/or software for execution on the display device600. The storage618is configured for storing an application for logging into the game streaming service and outputting audio and video associated with a game being played by the game streaming service. The storage618may be non-volatile memory and further be configured to download firmware or microcode for updating the processing unit610and/or the NC616. The storage618can include storage for storing a list of known or previously configured communication networks which can be used to automate connection thereto.

The network controller (NC)616is coupled to communications bus530and is configured for communication with one or more networks (e.g., a wireless network or wired network) and a game streaming service (e.g., the game streaming service250). It is appreciated that the network controller616in combination with other components are configured for receiving and outputting audio and video associated with a game streaming service (e.g., the game streaming service250) without the need for a gaming console (e.g., the gaming console110). The network controller616may use the memory606in order to send and receive communications and/or provide received data to other components of the display device600.

The processing unit610and the NC616can be integrated into a single integrated circuit die and processing unit610and NC616may share various resources, such as instruction logic, buffers, functional units and so on, or separate resources may be provided for video and/or audio processing and general-purpose operations. In some embodiments, the processing unit610and/or the NC616may be programmable circuits or processors, e.g., a field-programmable gate array (FPGA).

With reference toFIGS. 7-9, flowcharts700-900illustrate example functions used by various embodiments for a controller directly communicating with a game streaming service, as described herein. Although specific function blocks (“blocks”) are disclosed in flowcharts700-900, such steps are exemplary. That is, embodiments are well suited to performing various other blocks or variations of the blocks recited in flowcharts700-900. It is appreciated that the blocks in flowcharts700-900can be performed in an order different than presented, and that not all of the blocks in flowcharts700-900need be performed.

FIG. 7shows an exemplary controller implemented process for configuring a controller and controlling a game, in accordance with various embodiments.FIG. 7depicts a process700during which a controller is configured, connected to a game streaming service, and a game is launched and played.

At block702, a wireless network is connected to having an identifier matching a default network name, by a controller. As described herein, the controller may automatically connect to a wireless network setup by a device, e.g., smartphone, tablet, or computer, configured to function as a wireless access point.

At block704, network information associated with a wireless access point is received. The network information can be received from the device after a user is prompted for the network information, as described herein.

At block706, account information is received. The account information can be game streaming account information received from the device after a user was prompted for account information, as described herein.

At block708, the wireless network having the identifier matching the default network name is disconnected from by the controller. The controller can disconnect from the wireless network having the identifier matching the default network name in order to reconfigure its network settings based on the received network information, as described herein.

At block710, the controller connects to the wireless access point (e.g., the wireless access point222). The wireless account point is communicatively coupled to one or more networks, including the Internet, and the game streaming service (e.g., the game streaming service250), as described herein.

At block712, the game streaming service account information is sent from the controller to the game streaming service. As described herein, the game streaming service account information is sent from the controller to the game streaming service without a gaming console (e.g., the gaming console110).

At block714, a game selection is sent to the game streaming service. As described herein, the controller can be used to select a game from the game streaming service to be launched, via the controller and a display device (e.g., the display device212).

At block716, whether an audio device is coupled to the controller is determined. As described herein, a controller can detect whether an audio device, e.g., a headset or headphones, is coupled to an audio interface or jack of the controller. If an audio device is detected, block718can be performed. If an audio device is not detected, blocks724or728may be performed.

At block718, an indicator indicating that an audio device is coupled the controller is sent. The indicator can further indicate the type of audio device, e.g., headset or headphones, and the capabilities of the audio device, e.g., recording ability using a microphone, as described herein.

At block720, audio information is received in response to the indicator, as described herein. The audio information can be an individualized audio stream for the game character associated with the controller. For example, the individualized audio stream may include a screeching sounds of a car that is being controlled with the controller.

At block722, the audio information is output to the audio device coupled to the controller. For example, the audio information can be output to headphones of a headset.

At block724, an input is received at the controller. The input may a button press, joystick information, etc., from a user, as described herein.

At block726, the input received at the controller is sent via the wireless access point to the game streaming service. As described herein, the input from a user is sent to the game streaming service to control a character or other aspect of the game executing that is on the game streaming service.

At block728, a voice command is received. The voice command may be received via a microphone, e.g., of a headset, coupled to the controller, as described herein. Blocks724and/or728may then be performed as further input is received during the playing of the game.

At block730, the voice command is sent to the game streaming service. The game streaming service can then respond to the voice command by manipulating a character and/or updating images and/or video that are sent to the display device. Blocks724and/or728may then be performed as further input is received during the playing of the game.

FIG. 8shows an exemplary computer implemented process for streaming a game, in accordance with various embodiments.FIG. 8depicts a process800of streaming a game to a display device and receiving input from a controller connected to a network, as described herein. The display device and the controller are each individually logged into the game streaming service. The process800may be performed by one or more computing systems of a game streaming service (e.g., the game streaming service250).

At block802, account information for a game streaming service is received from a display device via network connection. The display device may be an internet connected and capable television or projector, as described herein. The account information may be received at a game streaming server (e.g., game streaming server260).

At block804, game streaming account information is received from a controller. The game streaming account information can be received from the controller after the controller has received account information and network information, as described herein.

At block806, one or more a game selection screens are sent. The display device may be sent video and/or images to allow a user to navigate using the controller to select a game, as described herein.

At block808, a selection of a game of the game streaming service is received from the controller. The selection can be based on input from the controller being used to select a game displayed on the display device, as described herein.

At block810, the game is executed. In response to the selection of the game, the game is executed or launched on the game streaming service, as described herein.

At block812, output the game is sent to the display device via the network connection. The output of the game is streamed from the game streaming service to the display device, as described herein.

At block814, whether an audio device is coupled to a controller is determined. The determination can be based on whether the controller sends an indicator that an audio device, e.g., headset or headphones, are coupled to the controller. If an audio device is coupled to the controller, block816is performed. If an audio device is not coupled to the controller, block818is performed.

At block816, an individualized audio stream is sent to the controller. The individualized audio stream can be sent including audio only for the user using the controller and be for output through the headset or headphones coupled to the controller.

At block818, an audio stream is sent to the display device. If there are more than two or more players playing the game, the audio stream may be a combined stream, as described herein. It is appreciated that combined audio and individualized audio can be sent to one or more locations, as described herein.

At block820, an input is received from the controller. The input can be from a button, joystick, voice command, etc., from a controller, as described herein, and received at a game streaming server.

At block822, a result of the input is sent to the display device. The game streaming server can send the result of the input, e.g., a game character moving, to the display device for display to a user.

FIG. 9shows an exemplary process for configuring a controller, in accordance with various embodiments.FIG. 9depicts of process900of configuring network and game streaming service account information on a controller with another device (e.g., smartphone, tablet, computer, etc.). In some embodiments, a first portion of process900may be performed by a smartphone and a second portion performed by a controller.

At block902, an application is executed. The application can be downloaded or loaded onto a smartphone, tablet, computer, or the like and executed, as described herein.

At block904, a local Wi-Fi network is configured. The local Wi-Fi network can be configured for a smartphone, tablet, computer, or the like to act as a wireless access point, as described herein.

At block906, Wi-Fi network information is broadcast. The Wi-Fi information that is broadcast can include a default SSID of “Controller Configuration Network,” as described herein.

At block908, the controller recognizes the Wi-Fi network. In some embodiments, the controller can recognize the Wi-Fi network based on the Wi-Fi network having a default SSID of “Controller Configuration Network,” as described herein.

At block910, the controller is connected. The controller can connect to the Wi-Fi network with the default SSID of “Controller Configuration Network,” as described herein. In some embodiments, blocks910and914may be performed over a wired connection (e.g., USB).

At block912, prompts for account information and network information are presented. For example, a screen of a smartphone, a tablet, a computer, or the like may display a form with one or more fields for a user to enter or select account information associated with a game streaming service and network information associated with a local network communicatively coupled with a game streaming service.

At block914, the account information and the network information are sent to the controller. The account information and the network information can be sent to the controller over a wireless or wired connection, as described herein.

At block916, the controller disconnects from the local Wi-Fi network. The controller can disconnect from the default controller configuration network in order to configure itself for coupling to a local network communicatively coupled to the game streaming service, as described herein.

At block918, the controller configures its network controller. The controller configures itself for coupling to a local network communicatively coupled to the game streaming service, as described herein.

At block920, the controller connects to a local network based on the network information. This allows the controller to then connect to the game streaming service, send the account information to the game streaming service, and launch a game.

FIG. 10shows exemplary computer system1000in accordance with one embodiment of the present invention. Computer system1000depicts the components of a generic computer system in accordance with embodiments of the present invention providing the execution platform for certain hardware-based and software-based functionality. In some embodiments, a game streaming service (e.g., the game streaming service250) includes one or more of exemplary computer system1000. In general, computer system1000comprises at least one CPU1001, a system memory1015, and at least one graphics processor unit (GPU)1010. The CPU1001can be coupled to the system memory1015via a bridge component/memory controller (not shown) or can be directly coupled to the system memory1015via a memory controller (not shown) internal to the CPU1001. The GPU1010may be coupled to a display1012. One or more additional GPUs can optionally be coupled to system1000to further increase its computational power. The GPU(s)1010is coupled to the CPU1001and the system memory1015. The GPU1010can be implemented as a discrete component, a discrete graphics card designed to couple to the computer system1000via a connector (e.g., AGP slot, PCI-Express slot, etc.), a discrete integrated circuit die (e.g., mounted directly on a motherboard), or as an integrated GPU included within the integrated circuit die of a computer system chipset component (not shown). Additionally, a local graphics memory1014can be included for the GPU1010for high bandwidth graphics data storage.

The CPU1001and the GPU1010can also be integrated into a single integrated circuit die and the CPU and GPU may share various resources, such as instruction logic, buffers, functional units and so on, or separate resources may be provided for graphics and general-purpose operations. The GPU may further be integrated into a core logic component. Accordingly, any or all the circuits and/or functionality described herein as being associated with the GPU1010can also be implemented in, and performed by, a suitably equipped CPU1001. Additionally, while embodiments herein may make reference to a GPU, it should be noted that the described circuits and/or functionality can also be implemented and other types of processors (e.g., general purpose or other special-purpose coprocessors) or within a CPU.

In one exemplary embodiment, GPU1010is operable for General-purpose computing on graphics processing units (GPGPU) computing. GPU1010may execute Compute Unified Device Architecture (CUDA) programs and Open Computing Language (OpenCL) programs. GPU1010may thus be used for a variety of computing applications including simulations of molecular dynamics, computational fluid dynamics, reservoir simulations, and finite structural problems. It is appreciated that the parallel architecture of GPU1010may have significant performance advantages over CPU1001.

System1000can be implemented as, for example, a desktop computer system or server computer system having a powerful general-purpose CPU1001coupled to a dedicated graphics rendering GPU1010. In such an embodiment, components can be included that add peripheral buses, specialized audio/video components, IO devices, and the like. Similarly, system1000can be implemented as a handheld device (e.g., cellphone, etc.), direct broadcast satellite (DBS)/terrestrial set-top box or a set-top video game console device such as, for example, the Xbox®, available from Microsoft Corporation of Redmond, Wash., or the PlayStation3®, available from Sony Computer Entertainment Corporation of Tokyo, Japan. System1000can also be implemented as a “system on a chip” (SoC), where the electronics (e.g., the components1001,1015,1010,1014, and the like) of a computing device are wholly contained within a single integrated circuit die. Examples include a hand-held instrument with a display, a car navigation system, a portable entertainment system, and the like.

In the foregoing specification, embodiments have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicant to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Hence, no limitation, element, property, feature, advantage, or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.