U.S. Pat. No. 10,143,928

DETAILED DESCRIPTION

This description relates to providing live streaming broadcasting of active gameplay of a video game application from a computer gaming device.

FIG. 1schematically shows a computer gaming device100connected to a network122. Computer gaming device100is shown in simplified form. Computer gaming device100may take any suitable form, including a game console, entertainment computing device, portable gaming device, tablet, laptop, desktop, virtual- or augmented-reality device, or other computing device that may be used to execute a video game, live broadcast, application, and/or other content.

Computer gaming device100includes an operating system102configured to manage system resources of the computer gaming device. Further, operating system102may be configured to monitor, oversee, and/or manage operation of various software programs, services, and applications executed by computer gaming device100. For example, operating system102establishes the software ecosystem wherein various system and third party software applications may be executed with defined access to system resources, interactions with other software applications, and permissions. Additionally, operating system102may present a user interface defining the mechanism of interaction of a user with computer gaming device100. As a non-limiting example, sensors116may collect sensor data which may be interpreted through a natural user interface (NUI) as commands to execute specific functionalities of computer gaming device100. For example, microphone118may detect speech commands from a user instructing operating system102of computer gaming device100to execute video game104. Visual data captured by camera120may be processed within operating system102and likewise interpreted as executable gesture commands.

Additionally, operating system102may facilitate the interface between one or more peripherals and computer gaming device100, thus allowing a user to provide input to computer gaming device100via a headset microphone, peripheral camera, companion computing device, and/or other peripheral.

Operating system102may also control communications between computing gaming device100and other computing devices via a network122(e.g., the Internet). The communications may include upload and download of content, streaming broadcasts, and/or shared computing.

Operating system102provides the software environment in which video game104and a platform-level live streaming broadcast application may be simultaneously executed. A platform-level live streaming broadcast application106B may be executed as an integral part of operating system102or as a separate platform-level live streaming broadcast application106A. In either configuration, platform-level live streaming broadcast application106A/106B may oversee video game104and provide a live streaming broadcast of gameplay to network122. The live streaming broadcast of the active gameplay may be hosted locally or by a remote server allowing others to access the broadcast on-demand via a remote client108or mobile client110. Furthermore, the broadcast optionally may be integrated into an electronic marketplace114and/or a social network112.

Platform-level live streaming broadcast application106A/106B may provide a live streaming broadcast of active gameplay of the user/broadcaster to network122in a plurality of broadcast configurations. Each broadcast configuration may include any of a plurality of configurable elements such as a video and audio stream of the active gameplay, a live video stream of the broadcaster performing the active gameplay, a live audio stream of the broadcaster during active gameplay, and an interactive text log. Platform-level live streaming broadcast application106A/106B may allow a user to define the configurable elements included in the broadcast, the format of the included elements within the broadcast, and other features of the broadcast.

Additionally, operating system102and/or platform-level live streaming broadcast application106A/106B may be configured to change the broadcast configuration of the broadcast of active gameplay responsive to a received command. Furthermore, operating system102and/or platform-level live streaming broadcast application106A/106B may execute the change in broadcast configuration without interruption of the active gameplay and the live broadcast. Thus, the broadcaster may actively control the content, format, features, and other aspects of the broadcast without interrupting the broadcast experience for broadcast viewers.

Operating system102may receive a plurality of commands from a user or broadcaster, such as to initiate a broadcast, change the broadcast configuration, change the configuration of the platform-level live streaming broadcast application, and/or terminate broadcasting. Furthermore, operating system102and/or platform-level live streaming broadcast application106A/106B may receive commands while providing active gameplay of the video game application. A user may indicate a command to computer gaming device100by voice command, gestures, game controller input, input on a connected mobile computing device and/or peripheral device, or any other suitable means.

Additionally, operating system102and/or platform-level live streaming broadcast application106A/106B may receive a command responsive to detecting a trigger event within the gaming environment of the broadcaster. For example, operating system102may be configured to change the broadcast configuration of the broadcast from a first broadcast configuration including a live audio stream of the broadcaster to a second broadcast configuration without a live audio stream when computer gaming device100detects the broadcaster activating a communication functionality of a portable communication device. Other trigger events may include an observer entering or exiting the broadcaster's gaming environment, the broadcaster entering or exiting the broadcaster's gaming environment, or any other recognizable event. A specific trigger event and the responsive change to the broadcast configuration may be defined by the user through a graphical user interface, defined within by an OS developer, broadcast application developer, game developer, or defined by any other suitable party.

FIGS. 2A, 2B, and 2Cillustrate example commands200,204, and206from a broadcaster or user202. InFIG. 2A, user202is engaged in active gameplay of a video game application executed by a computer gaming device and displayed on display device210. While the video game application is providing active gameplay, the computer gaming device may receive a live image stream and/or a live audio stream of user202and the gaming environment of user202. In this manner, the computer gaming device may monitor the received video and audio streams for commands from user202.

As a non-limiting example, user202may provide a command in the form of a voice command200to broadcast the active gameplay. The computer gaming device may receive an audio stream containing voice command200from a directional microphone in sensor212. The computer gaming device may then process the received audio stream and may detect voice command200. Responsive to voice command200, the computer gaming device to may execute the platform-level live streaming broadcast application to broadcast the active gameplay without interrupting the active gameplay.

Furthermore, upon receipt of voice command200, the computer gaming device may identify user202based upon the received audio stream and/or other information. Identification of user202by voice identification may be performed locally at the computer gaming device or remotely at a connected server. User202may also be identified based upon an image of user202within an image stream received by the computer gaming device. In this case, user202may be identified by facial recognition performed locally by the computer gaming device or remotely at a connected server. Other methods of identification of user202may include position data received from a directional microphone, skeletal tracking, and any other suitable identification method. Alternately, user202may be identified by an active network login profile or by identifying a device (e.g., mobile phone) that is detected by the computer gaming device. Upon identification of user202, the computer gaming device may associate the broadcast of active gameplay with a broadcasting and/or social network profile of user202.

FIG. 2Billustrates an example command to change the broadcast configuration while broadcasting. In this case, user202provides a voice command204to stop broadcasting a live video feed of the user (as captured by sensor212). Upon receipt of voice command204, the computer gaming device may suspend the broadcasting of the live video of the user while continuing to broadcast the game visuals themselves. The user need not pause or otherwise interrupt active gameplay to execute a command.

Other example broadcast commands may be to start video broadcast, stop audio broadcast, start audio broadcast, open or close a chat window, change the volume of the microphone output, change the arrangement and inclusion of elements of the platform-level live streaming broadcast application on the display device or within the broadcast, and/or change any other aspect of the broadcast.

FIG. 2Cillustrates user202providing a voice command206to terminate broadcasting the active gameplay. Responsive to receiving voice command206, the computer gaming device may terminate the broadcast of active gameplay without interrupting the active gameplay. That is, the user may continue to play the game without interruption although others will no longer be able to view a broadcast of the gameplay.

FIG. 3illustrates an alternative multiplayer example where the computer gaming device is providing active gameplay to a plurality of users. In this example, a first user302and a second user304are playing the video game application together on display device306. In this case, second user304is logged in as the active network login profile. The computer gaming device receives a voice command300to broadcast the active gameplay of the video game application from a first user302. Upon receiving voice command300via sensor308, the computer gaming device may identify the first user302as the user giving the command. The identification of first user302may be based upon sensor data received from sensor308as discussed above. Additionally, the computer gaming device may use a combination of sensors to identify first user302. For example, the computer gaming device may receive an audio stream including voice command300and data indicating a direction to the source of voice command300from a directional microphone in sensor308. The computer gaming device may process the data indicating the direction to the source and analyze a video image of the area from which the sound originated (e.g., first user302). First user302may be identified based upon the image from the received image stream. Upon identification of first user302, the broadcast may be associated with the social network profile of first user302. If first user302is not identified by the computer gaming device, the broadcast may be associated with a social network profile linked with the active network login profile of second user304. Alternatively, if first user302is not identified the broadcast may not be initiated or may be linked to a generic broadcasting profile.

As discussed above a user may provide the computer gaming device with a network login profile. In addition to association with a broadcast, the network login profile may be associated with a social network. Through the social network association a user may be provided with a notification400that a social-network friend is broadcasting as illustrated inFIG. 4. Notification400may also include a direct link to the broadcast, broadcast hosting service, and/or any other suitable information that will allow a user receiving notification400to locate and view the broadcast. It will be appreciated that notifications of broadcasts may also be provided to login profiles that a user has selected to follow and/or designated as a favorite in addition to profiles designated as friends. It will be further appreciated that in addition to notification400delivered during active gameplay, notifications of broadcasts of interest may be delivered while the user is watching television, personal computing, or using the computer gaming device for any other purpose. Furthermore, notifications can be sent to any device associated with the user, such as a tablet, phone, companion computing device, and/or other mobile device. Furthermore, individuals in a user's social network may likewise receive notification of the user's broadcasts. The computer gaming device and/or a companion service may include instructions executable at the initiation of a broadcast to review the social network profile associated with the broadcast and send a notification400to each individual within the user's social network. It will be appreciated that notification400may be received on another user's computer gaming device, tablet, and/or any other suitable mobile computing device.

FIG. 5illustrates an example menu500for a platform-level live streaming broadcast application. Upon launching the platform-level live streaming broadcast application, the player may be presented with application menu504. Application menu504may include search functionality, social network elements, broadcasting, or any other suitable functionality. Also included is a game and channel menu. The game and channel menu may include video game application tiles502representing video game applications and/or channel tiles506representing broadcast channels. The content of game and channel menu may include player configured elements, elements based upon video games owned by the player, analysis of player preference, and/or elements representing new and/or upcoming video games. It will be appreciated that the player may navigate either menu by any suitable means including selecting a menu option via a controller, performing a gesture, issuing a voice command, and/or controlling the interface by pupil movements.

FIG. 6illustrates an example configuration of a broadcast600of active gameplay including a window displaying the gameplay602, an interactive text log/chat window604, channel identifier608, follow object610, and favorite object612. As shown, chat window604includes comments from other users watching the broadcast and provides the capability of the player to interact with those observing the gameplay broadcast. Channel identifier608may also be displayed indicating the profile of the broadcaster. Channel identifier608, follow object610, and favorite object612may be rendered as executable menu options. Broadcast600may include a number of executables allowing a user watching the broadcast to follow the channel, view a social network profile linked to the broadcast, and/or report the channel for inappropriate content. It will be appreciated that the broadcaster may configure the broadcast channel to not include chat window604, channel identifier608, follow object610, and/or favorite object612. It will be further appreciated that the user watching the gameplay broadcast may also enable or disable chat window604.

Optionally, the broadcaster may configure the broadcast channel to include executable links that will allow broadcast viewers to join the video gaming session. In this manner, a broadcast viewer may easily transition from viewing the broadcast to playing along with the broadcaster in an on-line multiplayer video gaming experience. Another option may include the broadcaster providing direct links within the broadcast to a party text or video chat. It will also be appreciated that links to other community functionalities may be included in the broadcast by the broadcaster.

FIG. 7illustrates a broadcast700including an additional window706in which a live video stream of a player during the active gameplay is provided. The computer gaming device may be configured to capture the video data using both camera and microphone arrays. The computer gaming device may include instructions to use the camera to track the user in the event the player physically moves in three-dimensional space. Furthermore, the computer gaming device may include instructions to perform image processing on the captured video data prior to broadcasting the video stream. As a non-limiting example, the computer gaming device may include background removal and/or subject framing to only broadcast video of the player. Additionally, the player may initiate or cease the live video feed during gameplay as discussed with regard toFIGS. 2A-Cand/or through a companion computing device. It will be appreciated that video and/or audio commentary optionally may be recorded after the active gameplay for archived video.

As described above, live streaming broadcasts of active gameplay may be integrated into an electronic marketplace. The electronic marketplace may be operated from a remote server. Further, the electronic marketplace may be configured to be accessible by any compatible computing device including personal computers, game consoles, home entertainment computers, set top boxes, mobile phones, tablets, and web browsing devices, for example.FIG. 8Aillustrates an example electronic marketplace800where a plurality of video game applications are available for purchase. Each of the video game applications available for purchase may be represented by a graphical tile810A/B identifying the video game application. As used herein, a tile may be any textural, graphical, video, or other representation of a game. When included, an image within tile810A/B may be of cover art, a main character, or other identifying image. A payment object812A/B may also be displayed for each video game application which may be executable to initiate the purchase of the respective video game application.

Integrated into electronic marketplace800are preview objects802A/B,804A/B, and806A/B which may be executable to initiate viewing of a live broadcast of another player's active gameplay of the video game application. Preview object802A/B may be configured to initiate viewing of the live broadcast that currently has the most viewers. Preview object804A/B may be configured to initiate viewing of the live broadcast that currently has a second most viewers. For each preview object802A/B and804A/B, the electronic marketplace may identify and display a number of viewers currently viewing each live broadcast. A third preview link806A/B may be configured to initiate a display of a listing of all live broadcasts of the video game application. This integration allows a customer to execute preview objects802A/B,804A/B,806A/B and observe the active gameplay of the video game application. Preview objects802A/B,804A/B, and806A/B may be presented in the form of a hyperlink, graphical tile, live broadcast window, or other suitable format. In implementations where the preview object does not always provide live images of the associated broadcast, electronic marketplace800may be configured such that a mouseover, hover, or other partial selection of a preview object may result in the display of preview images from the respective live broadcast and/or a playback of a short segment of active gameplay. This integration provides the advantage of providing a customer with additional information to make a more informed decision about purchasing the game.

Each preview object may only be linked to video game platforms configured to purchase from the electronic marketplace. For example, a user visiting the electronic marketplace using a personal gaming computer would be provided preview objects which initiate viewing of broadcasts of active gameplay of the video game application from other users using a same type or class of personal gaming computer. Likewise, a user visiting the electronic marketplace from a video gaming console may be provided preview objects which initiate viewing active gameplay from other video gaming console users of the same type or class.

FIG. 8Billustrates an example social network profile814including a preview object816providing a direct link to a current active gameplay broadcast. Social network profile814may also include executables to link to or follow the social network profile. Thus, another user viewing social network profile814will be informed of any current broadcasts associated with the social network profile.

Social network profile814may be configured with a preview object816providing a direct link to a current broadcast from the player associated with the network profile814. Additionally, social network profile may include executables linked to archived broadcasts from that player.

As an additional example, a computer gaming device may login and connect directly to a specific network which facilitates software updates, hosts media, on-line multi- and single player video games, and/or electronic marketplaces. Further, the network may include social network functionality providing a user with access to a community of other subscribers to the network. In this case, the network may employ a gamer profile as a specific type of a social network profile. The gamer profile may then be updated with activities of the associated user within the specific network ecosystem. Direct links to broadcasts and notifications of broadcasts may be provided to other members of the network community as discussed above with reference toFIGS. 4 and 8B. The activity updates, notifications of broadcasts, links to broadcasts, and access to the gamer profile may be serviced by and within the specific network in a closed manner. In other words, users that are not part of the network community may not have access to these features. Alternatively, such features may be offered across platforms and network ecosystems.

It will be further appreciated that a user of the computer gaming device may configure the platform-level live streaming broadcast application to avoid integrating preview object816into their social network profile. Further, a user may choose not to allow broadcasts of their active gameplay to be integrated into the electronic marketplace. Additionally, the remote server may contain instructions to anonymize the direct links to broadcasts of live gameplay to protect the privacy of the player.

FIG. 9illustrates a method900for broadcasting live gameplay from a computer gaming device. Method900may be executed by a computer gaming device, for example.

At902, method900includes executing a video game application. The video game application may provide active video gameplay to the user. While the video game application is providing active gameplay the user may provide a command to the computer gaming device to broadcast active gameplay of the video game application.

At904, method900includes determining if a command to broadcast active gameplay of the video game application is received by the computer gaming device. The broadcast command may be provided to the computer gaming device by selecting a menu option via a controller, performing a gesture, voice command, gaze tracking, or any other suitable means. Upon receiving a command to broadcast the active gameplay, method900continues to906. If not, the system can continue to wait for such a command and/or return to other operations.

At906, method900includes identifying the user providing the broadcast command. The user may be identified based upon received sensor information as discussed above forFIG. 2A, an active network login profile, or any other suitable method. If the user is identified, method900continues to908and the active gameplay broadcast may be associated with a broadcast and/or social network profile of the recognized user. If the user is not recognized, method900continues to910and the active gameplay broadcast may be associated with the currently active network login profile and/or a generic broadcast profile.

At910, method900includes the option to archive the broadcast. If the platform-level live streaming broadcast application is configured to archive the broadcast or a command is received from the user, method900proceeds to912.

At912, method900includes storing the broadcast of the active gameplay. The broadcast of the active gameplay may be stored locally by the computer gaming device, remotely at a remote server, and/or a combination of both local and remote storage. It will be appreciated that a user may provide the command to archive the broadcast while streaming. It will be further appreciated that in the event that the command to archive the broadcast is given during a broadcast of active gameplay, the computer gaming device may include any buffered data from the broadcast prior to receiving the command to archive the broadcast. The archived broadcast may include any combination of gameplay video, in-game chat logs, chat audio, and/or player video or voice commentary. The user optionally may access a previously archived gameplay broadcast and may annotate the archived broadcast by recording voice and/or video commentary.

FIG. 10illustrates a method1000of broadcasting live gameplay from a computer gaming device. Method1000may be executed by a computer gaming device, for example.

At1002, method1000includes executing a video game application.

At1004, method1000includes broadcasting active gameplay of the video game application with a first broadcast configuration. For example, the first broadcast configuration may include broadcasting with a live video stream of the broadcaster's environment. Alternately, the first broadcast configuration may include any combination of the configurable elements (live video stream and/or live audio stream of the broadcaster's environment, interactive text log, etc.).

At1006, method1000includes determining if a command to change the broadcast configuration is received by the computer gaming device. As an example, a command to change the broadcast configuration may include a command to broadcast without the live video feed of the broadcaster's gaming environment. As previously discussed, the command may include receiving a command from a game controller, receiving a voice command, receiving a gesture command, receiving a command from a portable communications device, detecting a trigger event, or any other suitable method. Responsive to a command to change the broadcast configuration, method1000continues to1008. If a command is not received, the computer gaming device may wait until a command is received and/or return to other operations.

At1008, method1000includes broadcasting the active gameplay of the video game application with a second broadcast configuration without interrupting the active gameplay or the broadcast of active gameplay. Thus, in this example, the computer gaming device may broadcast the active gameplay in the second broadcast configuration without the live video feed of the broadcaster's gaming environment. Furthermore, the computer gaming device may accomplish the broadcast configuration change without interruption of the active gameplay or the broadcast. In this way, a user may dynamically toggle aspects of the broadcast (e.g., video on/off, audio on/off, etc.) without having to stop playing the game. This can be particularly advantageous in multi-player games that do not allow for pausing the game.

FIG. 11illustrates a method1100of operating an electronic marketplace. Method1100may be executed by a computer gaming device, for example.

At1102, method1100includes for each of a plurality of video game applications, displaying a tile identifying a video game application.

At1104, method1100includes display a payment object configured to initiate the purchase of the video game application.

At1106, method1100includes displaying a preview object configured to initiate the viewing of a live broadcast of another's active gameplay of the video game application. As discussed above forFIG. 8A, the preview object optionally may be configured to initiate the viewing of a live broadcast with the most active viewers.

Optionally, at1108, method1100includes displaying a second preview object configured to initiate the viewing of a different live broadcast.

Optionally at1110, method1100includes displaying a third preview object configured to initiate a display of a listing of all live broadcasts of another's active gameplay of the video game application. Other preview objects can additionally or alternatively be displayed in different implementations.

FIG. 12illustrates a companion computing device1206usable to initiate and/or control the broadcast of gameplay from a currently playing video game and/or to record other types of content. As shown, computer gaming device1202displays rendered gameplay1204of a basketball video game, and companion computing device1206has a live streaming broadcast companion user interface1208. In such an implementation, companion computing device1206may be used to control the broadcast, display, configuration, etc. of a live streaming broadcast via the live streaming broadcast companion user interface1208. The companion computing device1206may be used in conjunction with computer gaming device1202to provide a secondary gaming experience that enhances gameplay of a video game. Non-limiting examples of companion computing device1206include a portable gaming device, a mobile computing device, a smartphone, etc.

In one example, the user provides input to the companion computing device that indicates a broadcast command. The live streaming broadcast command is sent from the companion computing device1206to the platform-level live streaming broadcast application on the computer gaming device1202(e.g., directly via wired or wireless communication, or via a network accessible service in communication with the companion computing device1206and computer gaming device1202). In response to receiving the command selecting a broadcast channel, the platform-level live streaming broadcast application accesses the broadcast of the active gameplay or disconnects from the broadcast of the active gameplay. It will be noted that while the companion computing device1206is shown being used in conjunction with a computer gaming device playing a video game, the companion computing device may be used with other computing systems and may be used with types of video content other than video games. It will be appreciated that in addition to sending commands to access and disconnect from a live streaming broadcast, companion computing device1206may be used to send commands to change the configuration of the broadcast and the platform-level live streaming broadcast application.

In some implementations, the methods and processes described herein may be tied to a computing system of one or more computing devices. In particular, such methods and processes may be implemented as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product.

FIG. 13schematically shows a non-limiting implementation of a computing system1300that can enact one or more of the methods and processes described above. Computing system1300is shown in simplified form. Computing system1300may take the form of one or more console game systems, personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices. For example, computing system1300may take the form of computer gaming device100shown inFIG. 1or companion computing device1206shown inFIG. 12.

Computing system1300includes a logic machine1302and a storage machine1304. Computing system1300may optionally include a display subsystem1306, a communication subsystem1308, and/or other components not shown inFIG. 13.

Logic machine1302includes one or more physical devices configured to execute instructions. For example, the logic machine may be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.

The logic machine may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic machine may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic machine may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic machine optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic machine may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.

Storage machine1304includes one or more physical devices configured to hold instructions executable by the logic machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage machine1304may be transformed—e.g., to hold different data.

Storage machine1304may include removable and/or built-in devices. Storage machine1304may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage machine1304may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.

It will be appreciated that storage machine1304includes one or more physical devices. However, aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a finite duration.

Aspects of logic machine1302and storage machine1304may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP/ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.

The terms “module,” “program,” and “engine” may be used to describe an aspect of computing system1300implemented to perform a particular function. In some cases, a module, program, or engine may be instantiated via logic machine1302executing instructions held by storage machine1304. For example, the platform-level in-game recording companion may be instantiated via logic machine1302executing instructions held by storage machine1304. It will be understood that different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms “module,” “program,” and “engine” may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.

When included, display subsystem1306may be used to present a visual representation of data held by storage machine1304. This visual representation may take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the storage machine, and thus transform the state of the storage machine, the state of display subsystem1306may likewise be transformed to visually represent changes in the underlying data. Display subsystem1306may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic machine1302and/or storage machine1304in a shared enclosure, or such display devices may be peripheral display devices.

When included, communication subsystem1308may be configured to communicatively couple computing system1300with one or more other computing devices. Communication subsystem1308may include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network. In some implementations, the communication subsystem may allow computing system1300to send and/or receive messages to and/or from other devices via a network such as the Internet.

As noted above, an NUI interface system1301may be configured to provide user input to computing system1300. To this end, the NUI interface system includes a logic machine1310and a storage machine1312. To detect the user input, the NUI interface system receives low-level input (i.e., signal) from an array of sensory components, which may include one or more visible light cameras1314, depth cameras1316, and microphones1318. Other example NUI componentry may include one or more infrared or stereoscopic cameras; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity. In some implementations, the NUI interface system may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller.

The NUI interface system processes the low-level input from the sensory components to yield an actionable, high-level input to computing system1300. Such action may generate corresponding text-based user input or other high-level commands, which are received in computing system1300. In some implementations, NUI interface system and sensory componentry may be integrated together, at least in part. In other implementations, the NUI interface system may be integrated with the computing system and receive low-level input from peripheral sensory components.

It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific implementations or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.

The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.