U.S. Pat. No. 12,357,909

DETAILED DESCRIPTION

FIG.1illustrates a cloud gaming system100employing latency-aware interactivity delays to mitigate unintended player interaction with player interactive windows in accordance with at least one embodiment. The cloud gaming system100includes a data center102having a gaming server104connected to a client device106via one or more packet-switched networks108. The gaming server104operates to execute a video game software application110(hereinafter, “video game110”) that results in the real-time rendering of a sequence of video frames, which the gaming server104encodes and otherwise formats for inclusion in a video stream112transmitted to the client device106. The client device106operates to receive the video stream112, decode the encoded video frames contained therein, and display the decoded video frames in sequence so as to provide a video presentation of rendered graphical content to a player. Audio content is generated by the gaming server104through execution of the video game110and provided to the client device106for output to the player in a similar manner.

The client device106includes a network interface (not shown) connected to the network108to receive the video data and audio data transmitted from the gaming server104via the network108, and compute, storage, and display resources (not shown) for decoding and displaying the video frames of the transmitted video stream112and for outputting the corresponding audio content. The client device106can include, for example, a desktop computer, a notebook computer, a tablet computer, a compute-enabled cellular phone (that is, a “smart phone”), a compute-enabled television (that is, a “smart TV”), and the like. In some embodiments, the processing and display of video and audio content from the video game110executed at the gaming server104is facilitated by a software application executed at the client device106(and represented inFIG.1by graphical user interface (GUI)114). This software application can include, for example, a cloud-gaming-specific software application. In other embodiments, a more general software application is utilized, and the video stream112and audio stream (not shown) are formatted and transmitted by the gaming server104to be compatible with this software application. For example, in one embodiment the client device106utilizes a web browser that utilizes a hypertext markup language (HTML) format for display of the video content represented by the video stream112and output of the audio content represented by the associated audio stream, and the gaming server104provides the video stream112in a format, such as .mp4 or .MOV, compatible with the HTML format for display via the web browser.

Being interactive, the video game110utilizes player input to at least partially direct the game play experience of a gaming session with the player. This player input is received at least in part through a game controller116having one or more user-manipulable components118that, in response to being manipulated by the player, result in a processing component of the game controller116, or otherwise associated with the game controller116, generating a corresponding player feedback data and providing that player feedback data for transmission to the gaming server104via the packet-switched network108as part of a player input stream120. Although illustrated inFIG.1as having a “game pad” or “joystick” form factor, the game controller116can include any of a variety or combination of player input devices, such as a game pad, joystick, keyboard, mouse, touchpad, trackball, steering wheel or yoke, pedal, dance pad, simulated gun, optical motion tracker, inertial sensor, light/laser source and detector, and the like. As such, these user-manipulable components118can include buttons, paddles, keys, rotary detectors, motion detectors for corresponding motions, pressure sensors, touch sensors, light sensors, microphones or other audio sensors, and the like. For ease of reference and in recognition of common colloquial use, the user-manipulable components118are generally referred to herein as “buttons118” or more generally “buttons” and “pressing” a button or “press” of a button will be understood to be the manipulation of a user-manipulable component118in a manner results in the game controller116generating and transmitting a corresponding player input data. That is, reference to pressing of a button should not be limited to a player literally pressing a physical button, but rather is understood to refer to a player manipulating a user-manipulable component118of any form in a manner that results in generation of corresponding input. For example, a player making a particular motion in view of an optical motion tracker and a player speaking a specified word into a microphone both constitute “pressing a button118” of the game controller116.

In some embodiments, the game controller116is tethered to the client device106such that the player input data is transmitted to the packet-switched network108via the client device106. To illustrate, the game controller116can be wired or wirelessly connected to the client device106such that all data output by the game controller116is routed through a network interface between the client device106and the packet-switched network108. In other embodiments, the game controller116has a separate, independent network connection with the gaming server104via the packet-switched network108. For example, the cloud gaming system100could employ a wireless local area network (WLAN) access point (not shown) local to the client device106and the game controller116and connected to the network108, and wherein each of the client device106and the game controller116establishes a separate wireless connection with the WLAN access point to independently communicate with the gaming server104via the network108and the WLAN access point.

As the gaming server104is “remote” relative to the client device106due to their connection via the one or more networks108, there is a relatively large latency between when a video frame is rendered by the video game110and when the data representing that video frame is received at the client device106and made ready for display. Likewise, as the gaming server104is also “remote” relative to the game controller116, there is also a relatively large latency between when the player pushes a button118on the game controller116and when the player input data representative of that button push is finally received by the video game software application110. The combination of these two latencies, as well as various other latencies, such as the player's reaction time and processing latencies at the gaming server104and the client device106(the sum of which is referred to herein as the “cumulative latency”), in the cloud gaming system100can lead to the aforementioned issue in which player input provided during the course of normal game play is temporally generated at the same time or slightly after the video game110instantiates a player interactive window (hereinafter, “PIW” for brevity), such as the PIW122illustrated inFIG.1, and by virtue of the latency in transmitting and displaying the first frame containing this instantiated PIW122and the latency in transmitting the player input data representing the player input from the game controller116to the gaming server104, the player input could be misinterpreted as an interaction with the PIW122.

Accordingly, in at least one embodiment, the gaming server104employs a latency-aware player input filter124that operates to filter out player input from the player input stream120from reaching the video game110or otherwise being acted upon by the video game110as an interaction with the PIW122during a filter window following detection of instantiation of the PIW122by the video game110. The duration of the filter window is based on an estimation or other determination of the cumulative latency currently exhibited by the cloud gaming system100, and in general is selected to be of sufficient duration to mitigate most or all of the cumulative latency while not being so long as to present an increased risk of inadvertently filtering out player input that is intended by the player to be an interaction with the PIW122. This filtering process is described in greater detail below with reference toFIGS.2-4.

FIG.2illustrates a general hardware configuration200and software configuration202of the gaming server104in accordance with some embodiments of the present disclosure. As depicted by the hardware configuration200, the gaming server104includes one or more processors, such as one or more central processing units (CPUs)204, one or more graphics processing units (GPUs206), one or more application specific integrated circuits (ASICs)(not shown), and the like. The hardware configuration200further includes a network interface208coupled to the network108, one or more input/output (I/O) devices210, (such as a keyboard, mouse, or display), one or more mass storage devices212(such as a hard drive, optical disc drive, or network-attached storage), and the like. The hardware configuration200further includes at least one random access memory (RAM) or other system memory214accessible by the one or more processors. The system memory214stores one or more software applications that include program code representing executable instructions that manipulate the one or more processors to perform various operations described herein. These software applications include the aforementioned video game110as well as a set of one or more software applications collectively referred to as platform software216.

As a general overview, the platform software216operates to facilitate execution of the video game110at the gaming server104. In some implementations, the game developer of the video game110specifically configures the video game110for execution by a gaming server, in which case the platform software216provides application programming interfaces (APIs) and other “hooks” that provide a more transparent interaction between the hardware configuration200and the video game110. In other embodiments, the video game110is developed for a specific local hardware platform, such as a personal computer (PC) platform or a game console platform, in which case the platform software216emulates the typical interfaces between the video game110and its expected local hardware platform, and thus renders the actual underlying hardware configuration200of the gaming server104less transparent to the video game110. An example configuration of the platform software216and its interaction with the video game110are illustrated by the software configuration202.

In the depicted software configuration202, the platform software216includes an operating system (OS)218, a gaming server module220, a media server module222, an encoder module224, a player interactivity module226, a PIW detection module228, an input filter module230, and a latency monitor module232, as well as other various modules for supporting cloud gaming executing as known in the art. The OS218operates to manage the overall operation of the gaming server104and to act as an interface between the hardware components and the higher-level software layers. The gaming server module220operates to provide server-side functionalities associated with the video game110, such as setting up game sessions, storing session state data and other game-related data, processing gameplay inputs, and rendering gameplay outputs in response to the gameplay inputs. The media server module222operates to host a media streaming site, receive concurrent ancillary or supplemental media streams associated with an online gaming session, and provide the concurrent media streams (e.g., video stream112,FIG.1) to the client device106for concurrent display with a gaming session that is being supported during execution of the video game110. The encoder module224operates to encode the media streams for transmission to the client device106. Although illustrated as a software module executed by the processor resources of the gaming server104, in other embodiments the encoder module224can be implemented in whole or in part as a hardcoded or programmable logic integrated circuit, such as an ASIC. The player interactivity module226operates to extract player input data in the player input stream120and present the player input data as player inputs to the video game110in an appropriate format. To illustrate, in implementations in which the platform software216emulates a PC, game console or other local gaming environment, the player interactivity module226can operate to emulate inputs to one or more instances of the video game110as though the inputs were player inputs coming from a local game controller for each instance, rather than the game controller116remote to the gaming server104and connected via the network108.

The PIW detection module228, the latency monitor module232, and the input filter module230together operate to implement the latency-aware player input filter124(FIG.1). The latency monitor module232operates to monitor one or more of: current network conditions, processor loading conditions, and player interactions to determine the various latencies that contribute to the delay between instantiation of a PIW in a rendered frame and display of that frame at a display of the client device106and the delay between player manipulation of the game controller116and receipt of the corresponding player input data at the gaming server104(or, more specifically, the player interactivity module226), and from these latencies determine a current cumulative latency value234representative of a summation of some or all of these individual latencies. The PIW detection module228operates to monitor the execution of the video game110to detect instantiation of a PIW in the video stream being rendered by the PIW detection module228, and in response to such a detection, issue a PIW instantiation signal236to the input filter module230in the form of a value written to a register or other storage location, triggering of an interrupt or exception, and the like. The input filter module230in turn operates to implement an input filtering window in response to the PIW instantiation signal23indicating that a PIW has been instantiated, and for a duration that is based on the current cumulative latency value234provided by the latency monitor module232. While in the input filtering window, the input filter module230operates to filter out some or all of the player inputs being provided by the player interactivity module226so as to prevent these player inputs from being acted upon by the video game110and thus mitigating the risk that a player input provided in a period between instantiation of a PIW in a frame and display of that frame at the client device106, and thus presumably intended as input for normal game play, is not interpreted by the video game110as player input associated with the PIW.

In the illustrated embodiment, both the PIW detection module228and the input filter module230are implemented in the platform software216; that is, “external” to the video game110. In such embodiments, the PIW detection module228and the input filter module230can be implemented as part of, or in association with, one or more APIs provided as interfaces between the video game110and the platform software216. To illustrate, software configuration202can employ a display API238used by the video game110to provide frame rendering calls and other instructions for execution by the GPU206, as well as metadata associated with such frame rendering instructions, and the PIW detection module228can monitor the data and control flows through the display API238to detect instantiation of a PIW. As another example, the software configuration202can utilize a player input API240to facilitate the provision of player inputs received in the player input stream120to the video game110, and the input filter module230can be implemented as part of, or otherwise connected to, this player input API240. In other embodiments, one or both of the PIW detection module228and the input filter module230are implemented as part of the video game110itself. For example, the current cumulative latency value234can be provided to the video game110, and the video game110itself detects that it is instantiating a PIW and in response renders the PIW non-interactive for a corresponding input filtering window or otherwise disregards some or all player inputs received from the platform software216via the player input API240for a specified duration following instantiation of the PIW and which is based on the current cumulative latency value234.

FIG.3illustrates a method300describing the operations of the platform software216and video game110, and more particularly, the latency monitor module232, PIW detection module228, and input filter module230, in greater detail in accordance with some embodiments. The method300implements two separate subprocesses that operate concurrently at the gaming server104: a cumulative latency monitoring subprocess302; and a latency-aware selective player input filtering subprocess304. Turning first to the cumulative latency monitoring subprocess302, this subprocess serves to determine an initial value for the current cumulative latency value234and periodically update the current cumulative latency value234over time as conditions change. As previously noted, the cumulative latency value234is representative of a sum of various latencies in the cloud gaming system which reflect the net or sum duration between when a PIW is first instantiated in a frame rendered by the video game110and when the gaming server104can first expect to receive a player input that was intended by the player to be associated with the PIW. The latencies contributing to this duration include the encoding and other processing of the first frame within the gaming server104, the latency involved in transmitting the first frame having the PIW from the gaming server104to the client device106via the network108, the decoding and other processing of the first frame at the client device106, the scan out and display of the first frame at a display of the client device106, the player's reaction time in detecting the display of the PIW and the earliest opportunity to press a button of the game controller116in response, the latency in transmitting player input data representative of the press of the button from the game controller116to the gaming server104via the network108, and the processing and provision of the player input data to the video game110. For the following, it is assumed that the processing latencies are either minimal compared to the transmission latencies and the player reaction time, or can be incorporated using a fixed or proportional margin to represent the sum of these particular delays. However, in other embodiments, the latency monitor module232can estimate or otherwise detect these latencies in real time along with the other latency monitoring operations described below.

At block306, the latency monitor module232determines a player reaction time value representing the player's reaction time in interacting with the video game110. This can be an estimated value based on observed characteristics of the player in particular. To illustrate, the latency monitor module232can monitor the video stream112for particular events represented in the video content and measure how long it takes for the player to react based on received player input (and accounting for network latencies, as described below). Alternatively, the latency monitor module232may assume a fixed or standard player reaction time that is either general to all game play, particular to the type of game play provided by the video game110(e.g., a standard player reaction time for all first person shooter-type games), or particular to the specific title of the video game110being executed. This player reaction time value thus may be stored in configuration information for the platform software216, or if specific to the video game title, may be provided as configuration data or metadata by the video game110during its initialization.

At block308, the latency monitor module232determines the applicable network latencies that contribute to the instantiation/interaction delay. As noted above, these include the downstream latency in transmitting a rendered frame from the gaming server to the client device106via the packet-switched network and an upstream latency in transmitting a player input data from the game controller116to the gaming server104via the network108. These latencies can be determined using any of a variety of well-known or proprietary network latency measurement techniques, such as measuring path latency using a “tracert” function, measuring the joint upstream and downstream latency using a Round Trip Time (RTT) measurement technique, measuring the upstream latency and the downstream latency separately using a Time to First Byte (TTFB) latency measurement technique, and the like.

At block310, the latency monitor module232determines the current cumulative latency value234based on a sum of the player reaction time value and the upstream and downstream network latencies. To illustrate, for a fixed player reaction time of 300 ms, a measured upstream network latency of 60 ms, and a measured downstream network latency of 40 ms, the current cumulative latency value234could be determined as a straight sum of these three values, or 400 ms. In other embodiments, this sum could be increased by a fixed value (e.g., 20 ms) or a proportional value (e.g., 10%) to provide a margin of safety to account for additional latencies, such as processing latencies at the client device106and the gaming server104, to arrive at the current cumulative latency value234. In the illustrated embodiment, it is assumed that the player reaction time is relatively fixed, and thus the player reaction time value is determined at initialization and does not need to be updated or refreshed. However, in many networks, changes in congestion, utilization, and available routing paths can result in changes in network latencies, and thus the process of blocks308and310is iterated periodically to update the current cumulative latency value234so as to reflect current network conditions.

In parallel with the cumulative latency monitoring subprocess302, the latency-aware selective player input filtering subprocess304operates to mitigate the risk of player input being inadvertently interpreted as an interaction with a PIW due to the latencies introduced by the remote relationship between the gaming server104, the client device106, and the game controller116in the cloud game system100. Accordingly, during an iteration of the subprocess304, at block312the PIW detection module228monitors the frame rendering process dictated by the video game110to detect initial instantiation of a PIW. In some embodiments, the PIW detection module228detects instantiation of a PIW through an explicit notification process313by video game110. In this approach, the video game110is configured to identify when a PIW is going to be utilized and then send a notification of instantiation of the PIW to the PIW detection module228via an API or other interface, such as the display API238when the PIW detection module228is implemented as part of the platform software216or via internal signaling when the PIW detection module228is implemented in the video game110itself. It will be appreciated that this approach necessitates that the game developer develop or modify the video game title to expressly cooperate with the platform software216in this manner.

However, as noted above, in some implementations the platform software216emulates being a local gaming platform so that the video game110operates as though it is executing on a PC, game console, or other local platform, rather than on a cloud gaming server. In such situations, the PIW detection module228operates to detect instantiation of a PIW through an implicit detection using an PIW object recognition process315in which an object recognition model is used to detect the likely first instance of a suspected PIW in a frame being rendered for the video stream112. To illustrate, if in frame N−1 no object indicative of a PIW is found but in the next frame N an object indicative of a PIW is detected, the PIW detection module228can assert the PIW instantiation signal236in association with frame N. The detection of an object representative of a PIW can rely on any of a variety of parameters and aspects. To illustrate, it may be understood that for a given video game title, or for a class of video games in general, that generated PIWs typically have specific sets or ranges of sizes or dimensions, particular shapes (elongated rectangles, for example), particular visual characteristics, such as color, shading, presence of rounded corners, presence of alphanumeric characters or other text, presence of specific text or other alphanumeric characters (“YES”, “NO”, “OK”, for example), presence of button objects, presence in a particular location or area of the frame, and the like. The object detection model employed by the PIW detection module228thus can be used to search for the presence of such parameters and aspects, determine a confidence value based on the number and strength of the match of such parameters and aspects to components of the frame, and then identify the frame as containing an instantiated PIW via thresholding of the confidence value.

When a likely instantiation of a PIW in the video stream112is detected, the PIW detection module228provides the PIW instantiation signal236. In response, at block314the input filtering module230triggers a filter window for filtering out player input at the gaming server104. As explained above, the duration of the filter window typically is set based on various latencies in the cloud gaming system100so as to prevent most, if not all, possible player inputs that were triggered before the player is likely aware of an instantiated PIW via display of the initial frame or frames containing the PIW at the client device106. As this duration is represented by the current cumulative latency value234determined by the latency monitor module232from the most recent iteration of the cumulative latency monitoring subprocess302, triggering the filter window can, include for example, setting and starting a filter timer (e.g., a countdown timer) based on the cumulative latency value234. Thereafter, while the filter window is active, at block316the input filter module230monitors the player input stream120received at the gaming server104from the game controller116for the presence of player input data representing a player input. When a player input is detected, at block318the input filter module230applies a filter to the player input. As represented by block319, in some embodiments this filter is a total or complete filter in which all player inputs from game controller116are blocked from being provided to the video game110or otherwise being interpreted by the video game110as player interaction with the instantiated PIW. For example, explicit notification from the video game110of an upcoming instantiation of the PIW, or PIW instantiation detection via object recognition, may not indicate with sufficient certainty the buttons118that are relevant to the PIW (that is, the buttons that would provide player input that would initiate some action with respect to the PIW). In such cases, the total filtering often would be appropriate.

However, in other instances, the explicit notification process313involves explicit notification by the video game110of the player inputs that are considered to be interactive with the PIW, or the implicit detection of instantiation of a PIW may involve detecting a particular set of options associated with a particular subset of buttons of the game controller116, in which case the input filter module230can instead implement targeted filtering (represented by block321) in which only the subset of player inputs identified as pertinent to the interactivity of the PIW are filtered out, while other player inputs are permitted to be provided to the video game110or otherwise allowed to be acted upon by the video game110during the filtering window.

At block320, the input filter module230monitors for the end of the filter window set at block314, such as through observation of the lapse of a countdown timer initiated at the start of the filter window. In response, the input filter module230terminates or otherwise deactivates the filter window and ceases filtering of player input at block322, such that all player input received thereafter in the player input stream120is permitted to be provided to, or otherwise acted upon by, the video game110, subject to the player input being filtered by another process in the platform software216or in the video game110itself.

FIG.4illustrates a simple example of the operation of method300ofFIG.3in the cloud gaming system100ofFIGS.1and2in accordance with some embodiments. In the illustrated diagram, a simplified state representation of the cloud gaming system100at three different points in time, time T=0 ms (view401), T=200 ms (view402), T=700 ms (view403). Further, assume for this example that the player reaction time is determined to be 300 ms, the upstream and downstream latencies each currently are measured at 50 ms apiece, and an error margin of 10% is applied, resulting in a current cumulative latency of 440 ms (=(300+50+50)*1.10), represented in the current cumulative latency value234.

As represented in view401, at time T=0 ms a frame411of the video stream112is currently displayed at the client device106, with frame N already having been rendered by the video game110, transmitted to the client device106via the network108, and processed for display by the client device106. Also at this time, the video game110is rendering a subsequent frame412, for which transmission has not yet initiated. Further, at time T=0 ms, the player has interacted with the game controller116to press an “X” button, in response to which the game controller116has generated a player input data404that is in flight to the gaming server104.

As represented in view402, at time T=200 ms the frame412is currently on display at the client device106and the X button press represented by the player input data405has already been received and implemented by the video game110so as to impart some effect on the gameplay. Moreover, at this time the video game110has rendered a frame413that contains the first instance of a PIW421, which in this example is a dialog or tutorial window that prompts the player to press the “Y” button on the game controller116to close the window. Concurrently, the player, reacting to the game play represented by previously-displayed frames (e.g., which may include frame411displayed at time T=0 ms), has pressed button Y of the game controller116, which results in player input data405representing the Y button press being in flight at time T=200 ms.

As view402illustrates, the frame413containing the first instance of the PIW421has not yet been displayed to the player, much less even reached the client device106, and thus the player is unaware that the PIW421exits at time T=200 ms. As such, the player has pressed the Y button of the game controller116in response to normal game play as represented by frame412, frame411and preceding frames displayed at the client device106. However, from the perspective of the video game110, the PIW421has been instantiated and, without a mechanism to transparently or opaquely render the PIW421briefly non-interactive, the gaming server104and the video game110could incorrectly interpret the Y button press represented by player input data405as a player interaction indicating that the PIW421should be closed (that is, removed from any further frames the video stream112).

Accordingly, to mitigate the inadvertent interpretation of normal gameplay input as PIW interaction input, a filter window is implemented for a 440 ms duration (e.g., from T=200 ms to 640 ms, assuming the filter window is in place at T=200 ms) in response to instantiation of the PIW421, such as through detecting the visual objects representative of the PIW421in the frame413or through explicit notification from the video game110that the PIW421has been instantiation. In this example, the input filter module230filters out all player inputs in the filter window. Accordingly, as represented by filter exclusion block408, the Y button press represented by the player input data405is filtered out of the player input stream120when it is received and processed by the gaming server104. This filtering out of the Y button press can include, for example, preventing the input representing the Y button press from reaching the video game110(e.g., by filtering it out at the platform software216) or by external or internal direction at the video game110itself that instructs the video game110to refrain from interpreting the Y button press as an interaction with the PIW421.

As represented in view403, at time T=700 ms the frame413has already been displayed at the client device106, a frame414is currently on display at the client device106(with frame414being rendered at some point after frame413and containing the PIW421) and the video game110has rendered a subsequent frame415that contains another instance of the PIW421. Concurrently, the player, reacting to the PIW421displayed in frame414and previous frames (including frame413) has pressed button Y of the game controller116to close the dialog/tutorial box represented by PIW421, which results in player input data406representing the Y button press being in flight at time T=700 ms.

Further, at this point in time, the filter window has terminated. Accordingly, when the player input data406reaches the gaming server104at a subsequent time, the player input data406is permitted to be provided to, or otherwise acted upon, by the video game110, which in this case the represented Y button press is correctly interpreted as an interaction with the PIW421, and more specifically, the indicated player interaction that causes the video game110to close the PIW421so that it does not appear in frames rendered subsequent to frame415.

In some embodiments, certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software comprises one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

A computer readable storage medium may include any storage medium, or combination of storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer readable storage medium may be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.