U.S. Pat. No. 10,265,627

DETAILED DESCRIPTION

Embodiments will now be described with reference to the accompanying figures. The foregoing may, however, be embodied in many different forms and should not be construed as limited to the illustrated embodiments set forth herein.

All documents mentioned herein are hereby incorporated by reference in their entirety. References to items in the singular should be understood to include items in the plural, and vice versa, unless explicitly stated otherwise or clear from the text. Grammatical conjunctions are intended to express any and all disjunctive and conjunctive combinations of conjoined clauses, sentences, words, and the like, unless otherwise stated or clear from the context. Thus, for example, the term “or” should generally be understood to mean “and/or.”

Recitation of ranges of values herein are not intended to be limiting, referring instead individually to any and all values falling within the range, unless otherwise indicated herein, and each separate value within such a range is incorporated into the specification as if it were individually recited herein. The words “about,” “approximately” or the like, when accompanying a numerical value, are to be construed as indicating a deviation as would be appreciated by one of ordinary skill in the art to operate satisfactorily for an intended purpose. Similarly, words of approximation such as “approximately” or “substantially” when used in reference to physical characteristics, should be understood to contemplate a range of deviations that would be appreciated by one of ordinary skill in the art to operate satisfactorily for a corresponding use, function, purpose, or the like. Ranges of values and/or numeric values are provided herein as examples only, and do not constitute a limitation on the scope of the described embodiments. Where ranges of values are provided, they are also intended to include each value within the range as if set forth individually, unless expressly stated to the contrary. The use of any and all examples, or exemplary language (“e.g.,” “such as,” or the like) provided herein, is intended merely to better illuminate the embodiments and does not pose a limitation on the scope of the embodiments. No language in the specification should be construed as indicating any unclaimed element as essential to the practice of the embodiments.

In the following description, it is understood that terms such as “first,” “second,” “top,” “bottom,” “upper,” “lower,” and the like, are words of convenience and are not to be construed as limiting terms.

Described herein are devices, systems, and methods for virtual reality simulations in which a user may use one or more accessories tracked in a physical space to interact with a virtual reality environment. As used herein, a “virtual reality environment,” shall be understood to include a simulated environment experienced by a user through one or more computer-generated sensory stimuli (e.g., sights, sounds, forces, and combinations thereof) and in which the user's reaction, in a physical space, to such sensory stimuli may result in changes in the simulated environment. In general, unless otherwise specified or made clear from the context, virtual reality environments may include any of various different levels of immersion for a user, ranging from complete immersion in computer-generated sensory stimuli to augmented reality environments including both virtual and real-world objects. As used herein, the terms “real-world,” “physical world,” “physical space,” and variations thereof generally refer to a physical setting separate from computer-generated stimuli. Thus, for example, a physical space may include the three-dimensional space occupied by, or in the vicinity of, a user playing a virtual reality game or, further or instead, may include real-world events that occur in physical reality (e.g., apart from computer-generated stimuli associated with the virtual reality environment).

In general, the devices, systems, and methods of the present disclosure may be used to provide virtual reality simulations associated with a variety of different implementations in which real-world data of a moving object forms a basis of a graphical representation of the moving object in a virtual reality environment, and the user may interact with the graphical representation of the moving object in the virtual reality environment. In the disclosure that follows, these devices, systems, and methods are described with respect to virtual reality simulations of the sport of cricket, which has dynamic aspects that serve as useful contexts for describing challenges addressed by the devices, systems, and methods of the present disclosure. For example, cricket bowling techniques may exhibit greater variation as compared to a sport like baseball and, as described in greater detail below, implementations described herein may facilitate simulating such variations in a virtual reality environment. Thus, as described in greater detail below, certain implementations may be used to simulate a variety of bowlers and bowling techniques implemented in cricket, as well as bowlers of different quality, skill, physical abilities, and so on. Additionally, or alternatively, certain implementations may be used to simulate a variety of settings for playing cricket, where such settings may have an effect on cricket play.

The use of cricket in the description that follows should be understood to be by way of example and not limitation. That is, unless otherwise specified or made clear from the context, it will be understood that the devices, systems, and methods described herein may be applicable to virtual reality simulations of other sports, games, and activities, or more generally to any other type of simulation. Thus, unless a contrary intent is indicated or clear from the context, the devices, systems, and methods of the present disclosure may be used for virtual reality simulation of other sports such as baseball, softball, Wiffle® ball, fencing, tennis, badminton, squash, racquetball, soccer, table tennis, and so on. Further or instead, the devices, systems, and methods of the present disclosure may be used for virtual reality simulation in other gaming aspects such as first-person combat (e.g., fighting or shooting) games. Still further, or instead, the devices, systems, and methods of the present disclosure may be used for virtual reality simulation in any of various different training contexts, such as medical training in which the user may carry out a simulated medical procedure in the virtual reality simulation.

In certain implementations, virtual reality simulations described herein may be based on data from one or more live-action sequences. For example, data from a live-action sequence may be incorporated (e.g., in a raw form or in a manipulated form) into a virtual reality simulation in a virtual reality environment, where a user may experience situations that are based at least in part on (e.g., closely resembling) situations that are occurring, or that have occurred, in the live-action sequence.

As used herein, unless otherwise specified or made clear from the context, the term “live-action sequence” or variations thereof shall refer to any of various different combinations of movements, physical actions, or circumstances occurring in the physical world. In some instances, such live-action sequences may be temporally coupled to a virtual reality simulation, occurring substantially simultaneously (e.g., within a few seconds or minutes) or in near real time (e.g., within less than a few seconds) relative to corresponding activity in a virtual reality simulation. Further, or instead, such live-action sequences may be temporally decoupled from a virtual reality simulation, such as may be useful for providing the virtual reality simulation to a user “on-demand.” In the context of the use of data from a live-action sequence of a sporting event, as described herein, the data may correspond to at least a portion of a non-simulated sporting event that is occurring, or that has occurred, in the physical world. This may include, for example, data recorded from a sporting event (e.g., through video recording, still-frame images, motion sensors, or a combination thereof).

It will thus be understood that the user(s) of devices, systems, and methods disclosed herein may include a human user seeking an immersive simulated experience (e.g., in a virtual reality sports simulation). This may include a user looking to experience a simulated activity without performing the activity in the physical world, e.g., because of lack of access to the activity or a parameter thereof (e.g., lack of a proper setting, lack of equipment, lack of requisite participants, and so on), or to mitigate the risk associated with the activity in question. The user may also or instead include a person interested in training or otherwise practicing or improving their skills for a particular simulated activity. In the context of sports, the user may include a person with varying skill levels or experience, e.g., a child, an adult, an amateur, a professional, and so on.

Referring now toFIG. 1, a system100may include one or more accessories110, a computing device120, a database130, and a content source140in communication with one another (e.g., hardwired to one another, in wireless communication with one another, interconnected with one another over a data network102, or a combination thereof). The content source140may include data142from a live action sequence150. The database130may store the data142and, further or instead, other content useful for forming a virtual reality simulation. In use, the user101may interact with the system100through the accessories110(e.g., by wearing or wielding one or more of the accessories110) to interact with a virtual reality environment provided by the computing device120. For example, and as described in greater detail below, the accessories110may include one or more haptic devices to provide, to the user101, force feedback emulating a real-world sensation that the user101would experience when playing a live-action sport corresponding to the type of simulated event. In this manner, the system100may create a relatively realistic experience for the user101of the system100. For example, in the context of virtual reality sports simulation, the system100may create, for the user101, an experience that more closely corresponds to the experience of playing a sport in the physical world. Further, and as described in greater detail herein, the system100may incorporate data142from the live action sequence150into the virtual reality environment, so that the user101may experience situations based at least in part on sequences from the live-action sequence150.

As described herein, the system100may facilitate virtual reality simulation, and more specifically, in some implementations, virtual reality sports simulation. As discussed above, an example of a sport that may benefit from virtual reality sports simulation facilitated by the system100is the sport of cricket. To this end, one or more of the accessories110described herein may correspond to accessories typically used when playing cricket. For example, the accessories110may include one or more of a bat300(see, e.g.,FIGS. 3A, 3B and 3C), one or more gloves400(see, e.g.,FIGS. 4A and 4B), a helmet500(see, e.g.,FIGS. 5A-5C), and one or more pads600(see, e.g.,FIG. 6). It will be understood, however, that other accessories (e.g., specific to other sports or activities) are also or instead possible. It will be further understood that, unless otherwise indicated or made clear from the context, attributes of a particular instance of the accessories110discussed herein may also or instead be included on another, different instance of the accessories110discussed herein.

Referring now toFIGS. 2A and 2B, the user101in a physical space200may use the system100(FIG. 1) to interact with a virtual reality environment202as part of a virtual reality cricket simulation. For example, the user101may interact with the system100(FIG. 1) in a relatively controlled environment, such as at home, in a training facility (e.g., a sports training facility such as a batting cage), in a gaming facility (e.g., an arcade), and so on. For such a virtual reality simulation to become an immersive, realistic experience for the user101, the user101may interact with one or more of the accessories110described above to receive haptic or other sensory feedback associated with the simulated sequence.

As shown inFIG. 2B, the virtual reality environment202may include a setting204simulated to resemble an environment corresponding to a particular activity. For example, in the context of a virtual reality cricket simulation, the setting204may include one or more of an infield, an outfield, a boundary, a sky, a stadium, a lighting condition (e.g., whether it is daytime or nighttime), and a weather condition. Further, the virtual reality environment202may include virtual representations (e.g., simulations) of participants in a simulated activity. That is, a user101may view simulations of a playing field, a bowler, fielders, as well as other aspects of live play of the sport of cricket. For example, the virtual reality environment202may include one or more virtual players, such as a first virtual player206representing a cricketer that is batting (the batsman) and a second virtual player208representing a cricketer that is bowling to the batsmen (the bowler). It will be understood that more or fewer virtual players may be represented in the virtual reality environment202. Further, or instead, one or more of the virtual players may be a virtual representation (e.g., a first-person virtual representation) of the user101(FIG. 2A). Alternatively, or additionally, the user101may be represented by another component or object of the virtual reality environment202(e.g., human or non-human, animate or inanimate). In certain instances, the user101may not be specifically represented within the virtual reality environment202. In the example shown inFIG. 2B, the user101is represented within the virtual reality environment202as the first virtual player206, and is represented as a batsman.

Referring now toFIGS. 1, 2A, and 2B, the system100may provide the user101with a three-dimensional, panoramic view including, for example, a first-person view of a bowler and a surrounding environment, where the surrounding environment may include one or more elements found in a game setting (e.g., a stadium). As described in greater detail below, the second virtual reality player208may be a simulation of a real-world bowler, with movements of the second virtual reality player208created by at least partially replicating real-life movements of a bowler from stored video of past bowling performances. For example, the second virtual reality player208may be a digitized avatar of a generic bowler, with simulated movements of the second virtual reality player208created by animating body joints of the avatar using techniques such as key-framing, motion capture, or a combination thereof.

To facilitate forming simulations described herein as immersive, realistic experiences for a user101, it may be useful to provide the user101with relatively realistic force feedback corresponding to forces associated with the real-world activity being simulated. As described in greater detail below, such force feedback may be provided through one or more of the accessories110. By way of example, when striking a bowled ball210in the virtual reality environment202, the bat300wielded by the user101in the physical space200may provide the user101with the feeling of impacting the bowled ball210, as if the impact occurred in the physical space200. Continuing with this example, movement of one or more solenoids included in the bat300, as described in further detail below, may transmit forces to hands of the user101gripping the bat300. By way of further or alternative example, the system100may advantageously provide the user101with a physical stimulus in the physical space200when the user101is struck by a bowled ball210in the virtual reality environment202. In certain implementations, it may be advantageous for the user101to experience the potential for being hit with a ball in the virtual reality environment202. Thus, for example, the display112of the system100may represent hands or other portion of the body of the user101as the user101bats in the virtual reality environment202, with the representations of one or more body parts of the user101in the virtual reality environment202providing the user101with a more realistic sensation of the potential for being struck (e.g., a user101may view a representation of one or more of their hands, head or helmet500, hands or gloves400, and so on). To facilitate presenting these and other realistic experiences to the user101, the system100may generally include software, hardware, and accessories110operable in coordination with one another according to any one or more of the various different techniques described herein.

Additionally, or alternatively, the veracity of simulations described herein may benefit from including, in the virtual reality environment202, one or more attributes mimicking the effects of the same attributes in the physical world. For example, in the context of cricket, certain bowlers may release a cricket ball such that the ball curves as the ball moves through the air—this is commonly referred to as “swing” in cricket—and the setting of the cricket match may affect this movement. By way of example, on a humid or cloudy day, a bowled ball210may be more likely to swing. These conditions may be commonly found in cooler-climates, such as the climates of England and New Zealand, and certain implementations described herein may be used to simulate such settings within the virtual reality environment202. Further, certain implementations may to simulate a variety of conditions of a playing surface (referred to as the “pitch” in cricket). For example, a majority of balls in cricket hit the pitch before reaching the batsman, and thus the conditions of the pitch may play a significant role in the result of a bowled ball210. For example, when a bowled ball210hits the pitch, the seam of the ball may react with the ground and create what is commonly referred to as “movement off the seam.” By way of example, greener pitches (playing surfaces between a batsman and a bowler that include grass) or relatively damp pitches may increase the likelihood of creating such movement off the seam. Relatively drier pitches, such as those typically found in India and Pakistan, may be more amenable to creating movement of a bowled ball210using spin, with bowlers using this technique commonly referred to as “spin bowlers.” In general, the system100may simulate one or more of the aforementioned conditions (and one or more other conditions or parameters) in the virtual reality environment202to achieve an immersive, realistic experience for the user101.

Having provided an overall context for the system100and its use for virtual reality simulation, various aspects of the system100and techniques for forming immersive and useful virtual reality simulations using the system100will now be described. The description that follows is divided into sections describing hardware of the system100useful for forming the virtual reality environment202(I. HARDWARE), accessories useful for facilitating interaction between the user101and aspects of the virtual reality environment202(II. ACCESSORIES), and virtual reality simulations formed using the system100(III. SIMULATIONS). In general, it should be appreciated that these sections are presented for the sake of clarity of explanation and, unless otherwise specified or made clear from the context, these sections should not be considered to be limiting.

I. Hardware

As discussed above, the components of the system100shown for example inFIG. 1may be connected to one another over a data network102. The data network102may include any network(s) or internetwork(s) suitable for communicating data and control information among portions of the system100. This may include public networks such as the Internet, private networks, telecommunications networks such as the Public Switched Telephone Network or cellular networks using third generation (e.g., 3G or IMT-2000), fourth generation (e.g., LTE (E-UTRA)) or WiMAX-Advanced (IEEE 802.16m) and/or other technologies, as well as any of a variety of corporate area or local area networks and other switches, routers, hubs, gateways, and the like that might be used to carry data among portions of the system100. The data network102may thus include wired or wireless networks, or any combination thereof. One skilled in the art will also recognize that the components shown in the system100need not be connected by a data network102, and thus may work in conjunction with one another, independently of the data network102.

Communication over the data network102, or other communication between components of the system100, may be facilitated via one or more instances of a communications interface106. The communications interface106may include, or may be connected in a communicating relationship with, a network interface or the like. The communications interface106may include any combination of hardware and software suitable for coupling the components of the system100to a remote device (e.g., a computing device120) in a communicating relationship through a data network102. By way of example and not limitation, this may include electronics for a wired or wireless Ethernet connection operating according to the IEEE 802.11 standard (or any variation thereof), or any other short or long-range wireless networking components. This may include hardware for short-range data communications such as Bluetooth or an infrared transceiver, which may be used to couple into a local area network or the like that is in turn coupled to a data network102such as the Internet. This may also or instead include hardware/software for a WiMAX connection or a cellular network connection (using, e.g., CDMA, GSM, LTE, or any other suitable protocol or combination of protocols). Additionally, or alternatively, a controller150may control participation by the components of the system100in any network to which the communications interface106is connected, such as by autonomously connecting to the data network102to retrieve status updates and the like.

In general, the display112may provide the user101with a visual representation (e.g., using one or more graphical representations or computer-rendered scenes) of the virtual reality environment202. The display112may present to the user101one or more of still images, video data, or a combination thereof. To this end, the display112may include one or more of a two-dimensional display or a three-dimensional display. In some aspects, the display112may present a first-person view of a virtual representation of the user101to the user101. In certain implementations, the display112may be associated with one or more of the accessories110, such as the helmet500(see, e.g.,FIGS. 5A-5C) worn by the user101during a simulation and described in greater detail below. In some implementations, at least a portion of the display112is included on, or forms part of, another component of the system100, such as the computing device120.

The computing device120may include, or otherwise be in communication with, a processor122and a memory124. While the computing device120may be integrally formed in some instances, it should be appreciated that the computing device120may be advantageously distributed (e.g., with the processor122and the memory124supported on different portions of the system100) in some applications. In general, the processor122may process the data142received from the content source140and, additionally or alternatively, the memory124may store the data142in any one or more of various different forms (e.g., raw, processed, or a combination thereof). The computing device120may also or instead be used to control one or more components of the system100, and it will thus be understood that aspects of one or more instances of a controllers150described herein may also or instead apply to the computing device120and vice-versa.

In general, the computing device120may include any devices within the system100to manage, monitor, communicate with, or otherwise interact with other components in the system100. This may include desktop computers, laptop computers, network computers, gaming systems or devices, tablets, smartphones, wearable devices, or any other device that can participate in the system100as contemplated herein. In an implementation, the computing device120(or a component thereof, e.g., the processor122or the memory124) is integral with another component in the system100(e.g., the controller150or the accessories110).

In some aspects, the computing device120may include a user interface. The user interface may include a graphical user interface, a text or command line interface, a voice-controlled interface, and/or a gesture-based interface. In implementations, the user interface may control operation of one or more of the components of the system100, as well as provide access to and communication with one or more of the components of the system100.

The database130may include any one or more of various different types of databases known in the art, including data stores, data repositories, or other memory devices or systems as well as combinations of the foregoing. In some implementations, the memory124of the computing device may act as the database130, or vice-versa. In general, the database130may store the data142in a raw or processed format. In addition to, or instead of, raw or processed forms of the data142from the content source140or a live-action sequence150as described below, the data142may include instructions for controlling one or more components of the system100, such as computer code, external or third-party information processed or manipulated for use in a virtual reality simulation program (e.g., by the computing device120), or combinations thereof.

As stated above, the content source140may include data142received from a live-action sequence150. The live-action sequence150may include circumstances occurring in the physical world, such as a portion of a non-simulated sporting event that is occurring, or that has occurred, in the physical world. In this manner, data142from the live-action sequence150may include projectile data indicative of movement of a projectile launched by a player in the live-action sequence150. Specifically, the data142may include information regarding a cricket ball in a cricket match such as location information, temporal information, spin information, speed information, or any one or more other types of information useful for presenting a trajectory of the cricket ball in the virtual environment202. In some implementations, the data142may be suitable for inclusion in a virtual reality simulation program in a raw form (e.g., without further processing by the computing device106). Alternatively, or additionally, the data142may be processed and/or manipulated before it is used as part of a virtual reality simulation or otherwise used in coordination with one or more components of the system100to carry out a virtual reality simulation. In some implementations, the data142is derived from recorded information from a sporting event, where the information is typically used by umpires/referees, broadcasters, coaches, players, and the like, to track the path or expected path of a ball to aid in making, challenging, or analyzing rulings on the field of play. For example, in the context of cricket, the data142may include information typically used to determine where a cricket ball would have struck if a batsman were not in the path of the ball. This data142may represent, in some instances, a starting-point for manipulation and incorporation into a system100as part of a virtual reality simulation.

The data142may be collected, stored, processed, or otherwise generally included on the content source140. In some instances, the content source140may include a server with a memory storing the data142, where such a server may provide an interface such as a web-based user interface for use of the data142. The content source140may thus include a third-party resource.

The controller150may be electronically coupled (e.g., wired or wirelessly) in a communicating relationship with one or more of the other components of the system100and operable to control one or more of the other components of the system150. In some aspects, the controller150may be part of another component of the system150(e.g., the computing device120or one or more of the accessories110). Further, although one instance of the controller150is shown inFIG. 1, it will be understood that one or more different components of the system100may each include a respective instance of the controller150, which may function independently or in a coordinated manner with one or more other components of the system100(e.g., with other instances of the controller150). In general, the controller150may include, or otherwise be in communication with, an instance of the processor122and an instance of the memory124, such as those shown in the figure as included on the computing device120.

The controller150may include any combination of software and processing circuitry suitable for controlling the various components of the system100described herein including without limitation processors122, microprocessors, microcontrollers, application-specific integrated circuits, programmable gate arrays, and any other digital and/or analog components, as well as combinations of the foregoing, along with inputs and outputs for transceiving control signals, drive signals, power signals, sensor signals, and the like. In certain implementations, the controller150may include processing circuitry with sufficient computational power to provide related functions such as executing an operating system, providing a graphical user interface (e.g., to the display112), to set and provide rules and instructions for operation of a component of the system100, to convert sensed information into instructions, and to operate a web server or otherwise host remote operators and/or activity through a communications interface106or the like. In certain implementations, the controller150may include a printed circuit board, an Arduino controller or similar, a Raspberry Pi controller or the like, a prototyping board, or other computer related components.

The processor122may include an onboard processor for one or more of the computing device120and the controller150. The processor122may be any as described herein or otherwise known in the art. In an implementation, the processor122is included on, or is in communication with, a server that hosts an application for operating and controlling the system100.

The memory124may be any as described herein or otherwise known in the art. The memory124may contain computer code and may store data142such as sequences of actuation or movement of one or more of the accessories110or other hardware of the system100. The memory124may contain computer-executable code stored thereon that provides instructions for the processor122for implementation in the system100, for example, for controlling one or more components in the system100. Thus, the memory124may include a non-transitory computer readable medium having stored thereon computer executable instructions for causing the processor122to carry out any one or more of the methods described herein such as to carry out all or a portion of a virtual simulation.

II. Accessories

Having provided an overall context for a system100for virtual reality simulation, various implementations of the accessories110will now be described. Unless otherwise specified, or made clear from the context, it will be generally understood that each of the accessories110may be used as part of the system100to carry out various different aspects of the virtual reality simulations described herein. As described in greater detail below, the accessories110may be used for improving an experience of virtual reality simulation, particularly in the context of virtual reality sports simulation.

Referring now toFIGS. 3A, 3B, and 3C, as described herein, an accessory for use in a virtual reality simulation system or a virtual reality game may include a bat300. Although this accessory is described herein as a bat300(and more specifically as a cricket bat), it will be understood that this accessory may also or instead include another device for a virtual reality simulation system where features thereof (e.g., components that facilitate haptic feedback) may be advantageous or desired. For example, the features of the bat300described herein may be included as part of an accessory including or representing a weapon (e.g., a sword), a baton, a stick, a club, and so forth. Similarly, although the bat300is described herein as providing haptic feedback to simulate contact with a projectile such as a ball (e.g., a cricket ball), the haptic feedback may also or instead simulate contact with other objects, projectiles, or otherwise, whether static or moving.

The bat300may include a housing310having a handle312and a body314extending from the handle312, a tracking device320, one or more solenoids330disposed within the housing310, and a controller350.

The bat300may be wielded by a user101in the physical world while the user101is participating in a virtual reality simulation. In use, and when held by the user101, the bat300may simulate impact caused by a projectile striking the bat300(and vice-versa) without the bat300in the physical world ever striking such a projectile (e.g., the bat300may simulate the impact of a cricket bat striking a cricket ball during play in the physical world). The simulated impact may be provided through actuation of one or more of the solenoids330, which in turn may cause the bat300to vibrate as a form of haptic feedback for a user101holding the bat300. The haptic feedback provided by the bat300to the user101may vary based on a physical parameter of the bat300(such as the shape of the bat300, the size of the bat300, and the material of the bat300), and/or a parameter of a contact event that occurs within a virtual reality environment202. The contact event may include contact between virtual representations of the bat300and a projectile (e.g., a virtual representation of a cricket ball) within the virtual reality environment202. In this manner, the haptic feedback provided by the bat300to the user101may represent one or more different simulated contact scenarios based on, for example, the location where a virtual representation of the bat300made contact with the virtual representation of a projectile, or other factors related to a contact event such as a speed of the virtual representation of the projectile, spin of the virtual representation of the projectile, speed of the bat300being swung by the user101(or a relationship between the speed of the bat300in physical world to the speed of the bat300in the virtual reality environment202), an angle of the bat300being swung by the user101(or a relationship between the angle of the bat300in physical world to the angle of the bat300in the virtual reality environment202), exit speed or exit angle of the virtual representation of the projectile after contact with the virtual representation of the bat300, and so forth.

The bat300may generally include a size and shape that substantially resembles a typical cricket bat. For example, and as discussed above, the housing310may have a handle312and a body314extending from the handle312, where the handle312is sized and shaped for holding by the user101and where the body314includes one or more surfaces configured for striking a projectile (e.g., a cricket ball). Specifically, the bat300may include a first end301having the handle312, a second end302disposed away from the handle312, a first surface315bounded by a top edge316and a bottom edge317, and a second surface318disposed opposite the first surface318. In some instances, the first surface315may include a face319structurally configured to contact a cricket ball and the second surface318may include one or more of a bulge, a swell, and a spline, where one or more of these features may be found on typical cricket bats.

The housing310may be made of similar materials relative to a typical cricket bat. For example, the housing310may be made of one or more of carbon fiber and fiberglass. The housing310may also or instead include wood or a composite material that resembles wood in one or more of appearance, feel, weight, and so on.

In general, the size of the bat300may resemble that of a typical cricket bat as discussed above. Thus, in certain implementations, the bat300may have a length of no more than about 38 inches (about 965 mm), a width of no more than about 4.25 inches (about 108 mm), an overall depth of no more than about 2.64 inches (about 67 mm), and edges of no more than about 1.56 inches (about 40 mm).

In some implementations, one or more portions of the bat300receive or otherwise cooperate with one or more parts of a third-party system, such as a video game system or a video game console. For example, the handle312of the bat300may define a void for inserting at least a portion of a video game controller or other component of a video game system.

The tracking device320may be operable to track a position of the housing310(e.g., a specific portion of the housing310or the bat300generally). The tracking device320may communicate the position to a virtual reality environment202(see, e.g.,FIG. 2B) in substantially real time (e.g., having a time delay of no more than 25 milliseconds). To this end, the tracking device320may be monitored by one or more sensors322(e.g., external sensors such as one or more lasers that perform predetermined sweeps of a physical space where the bat300is being used). The tracking device320may work in conjunction with the controller350so that simulated movement of the bat300is provided within the virtual reality environment202in substantially real time based on information provided by the tracking device320in the physical world. As discussed herein, the bat300may represent one of the accessories110in the system100described with reference toFIG. 1, and thus the bat300may also or instead work in conjunction with one or more other components of that system100. For example, the bat300may include a communications interface106to communicate with a processor122that is executing a virtual reality cricket game within a virtual reality environment202, where the processor122is configured to receive a position of the housing310and to render the position of the housing310within the virtual reality environment202in substantially real time.

As discussed above, the bat300may include one or more solenoids330that are actuatable to provide force feedback to a user101of the bat300. To that end, the controller350, which may be the same or similar to any of the controllers described herein (e.g., the controller150ofFIG. 1), may be in communication with the plurality of solenoids330for controlling actuation thereof. Specifically, the controller350may receive information related to location-specific contact between virtual representations of the bat300and a projectile (e.g., a virtual representation of a cricket ball) within the virtual reality environment202and to selectively actuate one or more of the plurality of solenoids330to exert a force based on the location-specific contact. The force exerted by one or more of the plurality of solenoids330may be directed on the housing310of the bat300such that a user101holding the handle312(or other portion) of the bat300feels the force as haptic feedback. Thus, in some implementations, the solenoids330may exert a force on the handle312of the bat300in a relative indirect manner (e.g., from the body314that is connected to the handle312). Alternatively, one or more of the solenoids330may be positioned within the handle312of the bat300, or may be coupled to the handle312of the bat300, to directly apply a force to the handle312.

The solenoids330may be positioned within the housing310in a predetermined arrangement, orientation, and general configuration so that one or more of the solenoids330, when actuated, may provide haptic feedback to a user101of the bat300that simulates a predetermined contact scenario or event. Similarly, the number of solenoids330included within the housing310, and the number of solenoids330that are actuated by the controller350, may facilitate simulation of one or more specific, simulated contact scenarios. Each of these simulated contact scenarios may include, for example, a virtual representation of a ball striking a virtual representation of the bat300in a different location on the bat300or at a different simulated force or direction of impact, such that there is different haptic feedback provided to a user101for different location specific and/or force specific contact between virtual representations of the bat300and a ball within the virtual reality environment202.

By way of example, one or more of the simulated contact scenarios may include simulation of a ball striking the bat300on (i) a tip340of the bat300defined by a distal end of the face319disposed substantially adjacent to the second end302of the bat300, (ii) a base342of the bat300defined by a proximal end of the face319disposed substantially adjacent to the handle312, (iii) an upper contact area344on or adjacent to the top edge316of the bat300, (iv) a lower contact area346on or adjacent to the bottom edge317of the bat300, (v) a “sweet spot”348on the face319disposed between a centerline304of the face319and the distal end of the face319, and (vi) a middle-hit area349of the face319disposed between the sweet spot348and the proximal end of the face319. Although these specific examples of predetermined contact scenarios are provided above, it will be understood that other contact scenarios are also or instead possible for simulation.

The solenoids330may be positioned within the housing310and specifically actuated to facilitate haptic feedback for a user101wielding the bat300for the above-identified exemplary simulated contact scenarios. For example, in certain implementations, one or more solenoids330may be disposed adjacent to the second end302of the bat300and are configured to actuate during simulated contact scenario (i) discussed above; one or more solenoids330may be disposed adjacent to the first end301of the bat300and are configured to actuate during simulated contact scenario (ii) discussed above; one or more solenoids330may be disposed adjacent to the top edge316of the bat300and are configured to actuate during simulated contact scenario (iii) discussed above; and one or more solenoids330may be disposed adjacent to the bottom edge317of the bat300and are configured to actuate during simulated contact scenario (iv) discussed above.

Simulated contact scenario (v) discussed above may represent an ideal contact event between a cricket bat and a cricket ball, such as contact made in the sweet spot348of the bat300. Thus, in some aspects, all of the solenoids330in the plurality of solenoids330may be configured to actuate during this simulated contact scenario to alert a user101that they have made contact in the sweet spot348of a virtual representation of the bat300. Also, in some implementations, one or more of the solenoids330may be configured to actuate in a plurality of different power modes, where a power mode corresponds to the force exerted by a solenoid330. Some examples of such power modes may include a low-power mode and a high-power mode, where the low-power mode exerts less force than the high-power mode. To this end, in an implementation, all of the solenoids330may actuate in a low-power mode during simulated contact scenario (v) discussed above to create the feeling of a relatively smooth and desirous impact for the user101. Similarly, because simulated contact scenario (vi) discussed above may represent a slight “mis-hit” contact event between a cricket bat and a cricket ball, in some aspects, all of the solenoids330in the plurality of solenoids330may be configured to actuate during this simulated contact scenario to alert a user101that such an event has occurred, but in a different power mode from simulated contact scenario (v)—e.g., a high-power mode such that the feedback is relatively jarring to a user101indicating the slight mis-hit. Other power modes are also or instead possible for actuation of the solenoids330.

It will be understood that other arrangements for the solenoids330, and other actuation techniques, sequences, and scenarios for the solenoids330are also or instead possible. However, in general, the physical arrangement of the plurality of solenoids330within the housing310may provide a predetermined force distribution for certain location-specific or force-specific contact between virtual representations of the bat300and a projectile within the virtual reality environment202.

An example of a specific arrangement for the solenoids330is shown inFIGS. 3B and 3C. In general, the physical arrangement of the plurality of solenoids330within the housing310may provide a predetermined center of gravity for the bat300(e.g., one that substantially resembles the predetermined center of gravity for a typical cricket bat having a similar size and shape). Similarly, the housing310of the bat300, with the plurality of solenoids330included therein, may be weighted to provide a relatively similar feel to a typical cricket bat having a similar size and shape (e.g., while holding the bat300and during a swing by the user101). For example, a typical cricket bat may have a weight between about 2.0 lbs. and about 3.5 lbs., and thus the housing310and components included therein may be selected to have a cumulative weight between about 2.0 lbs. and about 3.5 lbs.

As discussed herein, each of the plurality of solenoids330may be positioned in a predetermined orientation (e.g., relative to one another or relative to one or more surfaces of the housing310of the bat300). For example, the predetermined orientation of at least one of the plurality of solenoids330may be substantially normal to a plane of the face319of the bat300. Also, or instead, the predetermined orientation of at least one of the plurality of solenoids330may be at a non-ninety-degree angle relative to a plane of the face319of the bat300. Thus, one or more of the plurality of solenoids330may have an axis of linear actuation disposed at an angle between about 1-degree and about 89-degrees relative to a plane of the face319of the bat300. For example, the predetermined orientation of at least one of the plurality of solenoids330may have an axis of linear actuation disposed at an angle of about 35 degrees offset from a plane of the face319of the bat300(or another surface of the bat300). Also, or instead, at least one of the plurality of solenoids330may be disposed substantially level with a center plane of the bat300. By way of further example, the predetermined orientation of the plurality of solenoids330may include at least two of the plurality of solenoids330having respective axes of linear actuation at least about 70 degrees opposed to one another and no more than about 145 degrees offset from a plane of the face319of the bat300(or another surface of the bat300). Other arrangements and orientations are also or instead possible.

In some implementations, the number of the plurality of solenoids330includes at least six solenoids330as shown inFIGS. 3B and 3C. For example, at least two of the solenoids330may be disposed adjacent to the first end301of the bat300and at least another two of the solenoids330may be disposed adjacent to the second end302of the bat300, where two other solenoids330are disposed therebetween. However, it will be understood that more than six solenoids330or less than six solenoids330are possible without departing from the scope of this disclosure, and the number, positioning, and orientation of the solenoids330may vary without departing from the scope of this disclosure.

Thus, in general, one or more of the solenoids330may be disposed within the body314of the housing310as described herein. However, one or more of the solenoids330may also or instead be disposed in another portion of the bat300such as the handle312. Similarly, in some implementations, the handle312may include a protrusion313engaged with a solenoid330for conveying haptic feedback to a user's hands when the user is gripping the handle312during exertion of a force based on a location-specific contact event. Other mechanical or structural features are also or instead possible for inclusion on the bat300for conveying haptic feedback to a user's hands when the user is gripping the handle312during exertion of the force based on a location-specific or force-specific contact event.

Generally, one or more of the solenoids330may include a movable member332, such as a movable arm, where movement of the movable member332facilitates haptic feedback as described herein. Thus, one or more of the solenoids330may include a linear actuator or similar. A movable member332of one or more of the solenoids330may be spring-loaded or otherwise biased such that, upon release, the movable member332extends or otherwise moves to create a force or vibration corresponding to haptic feedback. For example, the movable member332, upon movement thereof, may impact a contact surface334causing vibration in the bat300. The contact surface334may be a surface of the housing310or a separate surface disposed within the housing310. Stated otherwise, in some implementations, the bat300may include a contact surface334disposed adjacent to at least one of the plurality of solenoids330, where at least a portion of this solenoid330(e.g., a movable member332thereof) is structurally configured to contact the contact surface334when actuated. Also, or instead, movement of the movable member332itself may provide the force or vibration corresponding to haptic feedback (e.g., without contacting a surface of the bat300).

Movement of the movable member332of a solenoid330may be facilitated by a motor336included on the solenoid330(e.g., a direct current motor). In certain implementations, one or more of the solenoids330is capable of providing about eight kilograms of force. However, because a typical cricketer may experience about 40-55 kilograms of force when batting, it will be understood that more powerful solenoids330are also or instead possible without departing from the scope of this disclosure.

The bat300may further include one or more power sources360within the housing310that are in electrical communication with one or more powered components of the bat300(e.g., one or more of the plurality of solenoids330and the controller350). The one or more power sources360may include a battery (e.g., a rechargeable battery). For example, a power source360may include a wireless rechargeable battery that can be recharged using a short-range or long-range wireless recharging system. The power source360may also or instead be coupled to a port (e.g., a USB port) for connection to an electrical outlet or similar for charging.

Referring now toFIGS. 4A and 4B, an accessory for use in a virtual reality simulation system or a virtual reality game may include a glove400. As described above, although this accessory is described herein in the context of a cricket simulation, it will be understood that this accessory may also or instead be adapted for use in other contexts.

The glove400may be sized and shaped to receive at least one portion of a hand of a user101. For example, the glove400may be structurally configured to receive the entire hand of a user101, or one or more fingers and the thumb of the user101. The glove400may be adapted for use with a cooperating accessory, for example, another glove such that each of a user's hands are engaged with such accessories. The glove400may resemble a typical glove that is worn for protection, grip, and comfort by a cricket batsman. Thus, the glove400may include padding402disposed on or within at least a portion of the glove400. Similar to other accessories described herein, the glove400may include a tracking device420, one or more haptic feedback actuators430, and a controller450.

The tracking device420may be the same or similar to other tracking devices described herein (e.g., the tracking device320with reference to the bat300shown inFIGS. 3A, 3B, and 3C). In general, the tracking device420may be coupled to the glove400and operable to track a position of the glove400(e.g., to communicate the position to the virtual reality environment202in substantially real time). Thus, similar to the tracking device320of the bat300, the tracking device420may be monitored by one or more sensors422. As discussed herein, the glove400may represent one of the accessories110in the system100described with reference toFIG. 1, and thus the glove400may also or instead work in conjunction with one or more other components of that system100. For example, the glove400may include a communications interface106to communicate with a processor122that is executing a virtual reality cricket game within the virtual reality environment202, where the processor122receives a position of the glove400and renders the position of the glove400within the virtual reality environment202in substantially real time. In this manner, the virtual reality environment202may include a virtual representation of the glove400viewable by the user101.

In some implementations, the glove400is flexible to grasp a bat, such as the bat300described above. To this end, the tracking device430may be configured to detect and communicate finger flexion, or thumb flexion, of the user101wearing the glove400. The tracking device430may also or instead be configured to detect and communicate an orientation of the glove400, or other position and movement information.

The one or more haptic feedback actuators430may be coupled to the glove400in a predetermined arrangement, where one or more of the haptic feedback actuators430are actuatable to transmit forces to at least one portion of the hand of the user101in the glove400. Thus, in use, the haptic feedback actuators430may transmit a force to a wearer of the glove400to simulate a contact scenario that takes place within the virtual reality environment202. The haptic feedback actuators430may be disposed in one or more locations of the glove400. For example, the haptic feedback actuators430may be dispersed throughout different locations within the glove400corresponding to different regions of a user's hand when wearing the glove400. This may include implementations where one or more haptic feedback actuators430are disposed along one or more portions of the glove400sized and shaped to receive a finger of a wearer, a thumb of a wearer, a palm of a wearer, a backside of a wearer's hand, and combinations of the foregoing.

The haptic feedback actuators430may include, or be formed on, an insert432disposed within the glove400. The insert432may be disposed within padding402of the glove400or between layers of padding402. To this end, the padding402may include a top layer and a bottom layer, with one or more haptic feedback actuators430disposed in-between these layers.

The controller450may be the same or similar to other controllers described herein. In general, the controller450may be in communication with the tracking device420, one or more of the haptic feedback actuators430, and a virtual reality environment202(FIG. 2B), for example, e.g., for controlling one or more aspects of one or more of these components. For example, the controller450may be configured to: receive, from the tracking device420, a position of the tracking device420; transmit the position of the tracking device420to the virtual reality environment202; receive, from the virtual reality environment202, an indication of force on a virtual glove (corresponding to the glove400in the physical world) in the virtual reality environment202; and actuate one or more haptic feedback actuators430on the glove400to simulate the force on the virtual glove in the virtual reality environment202.

By way of example, the indication of force on the virtual glove in the virtual reality environment202may correspond to a ball striking a bat being held by the hand of a user101. In this manner, when such contact between a virtual bat and a virtual ball is made within the virtual reality environment202, a user101in the physical world may feel a representative force in the glove400through actuation of one or more of the haptic feedback actuators430. By way of further example, the indication of force on the virtual glove in the virtual reality environment202may also or instead correspond to a ball striking the glove400. In this manner, when such contact between a virtual ball and a virtual glove is made within the virtual reality environment202, a user101in the physical world may feel a representative force in the glove400through actuation of one or more of the haptic feedback actuators430.

In some aspects, the glove400is the only accessory providing such haptic feedback. In other aspects, the glove400works in conjunction with one or more other accessories (e.g., the bat300described above) to provide a more realistic feel for the user101. To this end, one or more of the haptic feedback actuators430may operate in coordination with one or more haptic devices on another accessory wielded by the user101(e.g., the solenoids330included on a bat300held by the user101).

To differentiate between different simulated contact scenarios (e.g., a virtual ball striking a virtual bat or virtual glove), different haptic feedback actuators430may actuate and/or the haptic feedback actuators430may actuate in different power modes to create different feedback. Further, the glove400may facilitate feedback that is location specific or force specific within the glove400itself. For example, the haptic feedback actuators430may be disposed throughout different portions of the glove400, such that certain haptic feedback actuators430in certain designated locations may be actuated depending upon the location of contact in a simulated contact scenario. Thus, one or more of the haptic feedback actuators430may be operable to adjust feedback based on a parameter in the virtual reality environment202, where such a parameter may include one or more of a virtual bat selected by the user101, a location on the virtual bat where a virtual ball makes contact in the virtual reality environment202, a vertical displacement between the virtual bat and the virtual ball in the virtual reality environment202, a location on the virtual glove where a virtual ball makes contact in the virtual reality environment202, a force of impact, and so on. Similarly, one or more of the haptic feedback actuators430may be operable to adjust feedback based on an attribute of one or more of a virtual ball and a virtual bat in the virtual reality environment202, where such an attribute may include one or more of ball speed, ball spin (if any), bat speed, bat angle, an exit speed of a bat held by the user101, and an exit angle of the bat. Such an attribute for the virtual bat may directly correspond to motion and use of a bat300or other accessory in a physical space.

It will be understood that the glove400(and/or another accessory described herein) may also or instead include one or more other sensors470. These sensors470may include one or more of the following: a force sensor, a contact profilometer, a non-contact profilometer, an optical sensor, a laser, a temperature sensor, a motion sensor, an imaging device, a camera, an encoder, an infrared detector, a weight sensor, a sound sensor, a light sensor, a sensor to detect a presence (or absence) of an object, and so on.

Referring now toFIGS. 5A, 5B, and 5C, an accessory for use in a virtual reality simulation system or a virtual reality game may include a helmet500. As described above, although this accessory is described herein in the context of a cricket simulation, it will be understood that this accessory may also or instead be adapted for use in other contexts.

The helmet500may include a shell510that is positionable about at least one portion of a head of a user101, a display530coupled to the shell510, an audio device540coupled to the shell510, and a tracking device520coupled to the shell510.

The shell510may be sized and shaped to substantially mimic, in both look and feel, a cricket batsman's helmet. That is, the shell510may resemble a real-world cricket helmet worn by a typical batsman for safety. For example, to more realistically provide a cricket gaming experience, the shell510may include an actual, real-world cricket helmet adapted to accommodate one or more of the display530, the audio device540, and the tracking device520.

The display530may be the same or similar to any of the displays described herein or otherwise known in the art of virtual reality simulation. For example, the display530may be included on a virtual reality head mounted display (HMD) visor.

As shown inFIGS. 5A and 5B, the display530may be movable between different positions. For example, and as shown inFIG. 5A, the display530may be placed in a first position where the display530is viewable by a user101with the shell510positioned about at least a portion of the head of the user101. And as shown inFIG. 5B, the display530may be placed in a second position where the display530is not obstructing at least part of a user's vision with the shell510positioned about at least a portion of the head of the user101. To accommodate the display530being movable between different positions, the display530may be pivotable, slidable, extendable, and so on, relative to the shell510. For example, and as shown inFIGS. 5A and 5B, the helmet500may include a pivoting joint512that couples the display530to the shell510. More specifically, the helmet500may include a display mount514coupled to the shell510that is sized and shaped to receive the display530therein or thereon, where the display mount514includes or is otherwise coupled to a pivoting joint512or other connection (e.g., a hinge) facilitating movement of the display510relative to the shell510. Thus, at least a portion of the display mount514may be movable relative to the shell510. For example, the display mount514may be movable to place the display510in the first position shown inFIG. 5Aand the second position shown inFIG. 5B, or other positions.

The display530may also or instead be removable and replaceable from the display mount514, such as where the display530is included on a mobile computing device (e.g., a smartphone) and the mobile computing device is removably mountable to the helmet500via the display mount514.

The audio device540may be operable to provide audio output from the virtual reality environment202to the user101with the shell510positioned about a portion of the head of the user101. In some implementations, the audio device540includes headphones or earbuds. The audio device540may be integral with the shell510.

The tracking device520may be the same or similar to any of the tracking devices described herein or otherwise known in the art of virtual reality simulation. In general, the tracking device520may be operable to track a position of the helmet500, and to communicate the position to a virtual reality environment202in substantially real time. As discussed herein, the helmet500may represent one of the accessories110in the system100described with reference toFIG. 1, and thus the helmet500may also or instead work in conjunction with one or more other components of that system100. For example, the helmet500may include a communications interface106to communicate with a processor122that is executing a virtual reality cricket game within the virtual reality environment202, where the processor122is configured to receive a position of the helmet500and to render the position of the helmet500within the virtual reality environment202in substantially real time. In this manner, the virtual reality environment202may include a virtual representation of the helmet500viewable by the user101. The tracking device520may be disposed on or within the shell510of the helmet500.

It will be understood that the helmet500may also or instead include any of the features described above with reference to other accessories110in the system100described with reference toFIG. 1or elsewhere in this disclosure. Thus, the helmet500may include sensors, solenoids or other haptic feedback devices, and so on.

In addition to the accessories described above for use in a virtual reality simulation system or a virtual reality game, which are set forth by way of example and not of limitation, other accessories are also or instead possible. One such accessory includes the pads600shown inFIG. 6. The pads600may include one or more of the features described herein that aid in a virtual reality simulation becoming more of an immersive, realistic experience for a user. For example, the pads600may include one or more haptic feedback actuators630that facilitate a user101to feel relatively realistic force feedback corresponding to forces that may be experienced when partaking in an activity in the real world that is being simulated in a virtual reality environment202(FIG. 2B).

In general, the pads600may include a wearable accessory, and although shown as typical padding that a cricket player might wear during a cricket match, it will be understood that other wearable accessories are contemplated herein. This may include other padding-type or add-on accessories, as well as more typical wearable clothes such as hats, pants, shirts, and so on.

In the context of a cricket simulation, one or more haptic feedback actuators630in the pads600may actuate to simulate a batsman being struck by a bowled ball (e.g., when such an instance occurs in a virtual environment as described herein).

III. Simulation

Having provided an overall context for a system100for virtual reality simulation (see, e.g.,FIG. 1) and various hardware components that may be included in such a system (see, e.g.,FIGS. 2A-6), various simulation techniques will now be described. It will be understood that the following virtual reality simulation techniques may be used for improving an experience of virtual reality simulation, and more particularly, for improving an experience of virtual reality sports simulation. To that end, it will be understood that one or more of the following virtual reality simulation techniques may be used in conjunction with one or more of the hardware accessories or other components described herein.

FIG. 7is a flow chart of an exemplary method700of operating a virtual reality game. Unless otherwise specified or made clear from the context, it should be appreciated that the exemplary method700may be carried out using any one or more of the devices, systems, and methods described herein. Thus, for example, the exemplary method700may be carried out using the system100(see, e.g.,FIG. 1) and, more specifically, may be carried out to create a realistic virtual reality simulation for an end user101incorporating one or more of the accessories110described herein (e.g., the bat300ofFIGS. 3A-3C). It will thus be understood that, while the exemplary method700may emphasize use of a bat300as the accessory110being used, the exemplary method700may be adapted with any of the other accessories110discussed herein.

As shown in step702, the exemplary method700may include receiving projectile data. The projectile data may be related to a virtual projectile within a virtual reality environment, which, as discussed in more detail below, may be directly or indirectly correlated to a real-world projectile in a live-action sequence. The projectile data may include temporal data related to the arrival of a virtual projectile within a predetermined volume adjacent to a virtual player in a virtual reality environment, and spatial data related to a trajectory of the virtual projectile in the virtual reality environment.

As shown in step704, the exemplary method700may include receiving accessory data including movement of an accessory in a physical space (e.g., an accessory held by, worn by, or otherwise wielded by a user of a virtual reality simulation). The accessory data may correspond to movement of a virtual accessory of a virtual player within the virtual reality environment.

As shown in step706, the exemplary method700may include displaying the virtual accessory and the virtual projectile on a display of a virtual reality simulation system (e.g., a display that is viewable by a user). The exemplary method700may also or instead include simulating tracked movement of the accessory within the virtual reality environment.

As shown in step708, the exemplary method700may include adjusting a trajectory of the virtual accessory based on additional tracked movement of the accessory. Thus, for example, if a user adjusts movement of the accessory in the real-world, the virtual accessory of the virtual player within the virtual reality environment may be adjusted in a similar fashion.

As shown in step710, the exemplary method700may include determining, based on a comparison of the accessory data and the projectile data, a contact scenario between the virtual accessory and the virtual projectile within the virtual reality environment. The contact scenario may be any of the simulated or predetermined contact scenarios discussed herein.

As shown in step712, the exemplary method700may include determining whether the virtual accessory will contact the virtual projectile within the virtual reality environment. When it is determined that no contact is made, or will be made, between the virtual accessory and the virtual projectile within the virtual reality environment, the exemplary method700may include repeating one or more of the steps of receiving projectile data, receiving accessory day, and so on, until it is determined that there is or will be contact between the virtual accessory and a virtual projectile within the virtual reality environment. It will also be understood that other steps in the exemplary method700may still be performed, however, even when it is determined that no contact is made, or will be made, between the virtual accessory and the virtual projectile within the virtual reality environment, such as transmitting appropriate audio feedback (e.g., a sound of a projectile passing by a virtual player without making contact with the virtual accessory). When it is determined that contact is made, or will be made, between the virtual accessory and a virtual projectile within the virtual reality environment, the exemplary method700may continue to the remaining steps shown in the exemplary method700.

As shown in step714, the exemplary method700may include determining a contact location on the virtual accessory and a contact time based on the accessory data and the projectile data. This may also or instead include determining a contact force based on the accessory data and the projectile data.

As shown in step716, the exemplary method700may include, based on the contact scenario, selectively actuating one or more haptic feedback devices (e.g., solenoids) coupled to the accessory to provide haptic feedback to a user grasping, wearing, or wielding the accessory. This haptic feedback may substantially simulate contact between the accessory and a projectile. Selectively actuating one or more haptic feedback devices may further include defining a number of discrete contact scenarios characterizing contact between the virtual accessory and the virtual projectile within the virtual reality environment at a number of different locations on the virtual accessory and selectively actuating one or more haptic feedback devices according to one of the number of discrete contact scenarios most closely corresponding to a contact location estimated within the virtual reality environment. The contact scenario may also or instead include a characteristic pertaining to a level of force characterizing contact between the virtual accessory and the virtual projectile within the virtual reality environment, and the exemplary method700may also or instead include selectively actuating one or more haptic feedback devices according to this level of force.

As shown in step718, the exemplary method700may include selecting audio feedback based on the contact scenario, determining timing for sending the audio feedback to a speaker to align with timing of the contact scenario, and transmitting the audio feedback to the speaker. In certain implementations, each simulated contact scenario has an accompanying audio feedback selection. For example, a virtual projectile hitting a certain part of the virtual accessory may be accompanied by a different sound than the virtual projectile hitting a different part of the virtual accessory. The speaker may include one or more of the audio devices540discussed herein (e.g., with reference to the helmet500).

As discussed herein, the present teachings may utilize data from a live-action sequence. As further discussed herein, the live-action sequence may be occurring in near real time relative to operation of the virtual reality environment, or the live-action sequence may be a recording (e.g., of a completed sporting event or similar).

In the context of cricket, a common approach for batsmen in real-world environments is to practice against live bowling or to use a mechanical machine to practice form and timing. However, these real-world approaches may be limited by the fact that every bowler's delivery has its own subtle characteristics that are typically not replicated by conventional tools and methods. While some simulations (e.g., video games) may replicate a general delivery type as described above and may use a generic avatar to simulate a bowler, the computer-generated deliveries of these bowlers may vary significantly from actual real-world bowlers. Also, the release point of bowler avatars may become relatively easy for a batsman to predict, thus not being reflective of randomness of different bowlers' real-world release points. So, while a user may become proficient at hitting a cricket ball in a typical video game environment, such practice does not often translate into success in real-world situations because the timing and release point recognition may be vastly different. Another approach typically used includes reviewing film (e.g., reviewing still shots or video footage of a particular bowler). However, it may be difficult to capture still shots and video footage from a batsman's perspective, and, as a result, traditional still shots and video footage may fail to provide a batsman with the immersive experience of facing a real-world bowler. Implementations described herein may improve upon the aforementioned deficiencies by facilitating more immersive, realistic experiences for users.

For example, certain implementations discussed herein may facilitate a first-person perspective of cricket balls that incorporate the actual delivery and trajectory of balls from real-world bowlers. In some aspects, a user may watch a digitized avatar of a real-world bowler perform their own bowling sequence with a simulated cricket ball delivered from the bowler's tracked release point that leaves the bowler's hand. Depending on the ball, data may be used to simulate a flight path (and spin rate, spin direction, and so on, as applicable) associated with a bowled ball in the real-world.

FIG. 8is a flow chart of an exemplary method800of virtual reality simulation (e.g., using data from a live-action sequence). Unless otherwise specified or made clear from the context, it should be appreciated that the exemplary method800may be carried out using any one or more of the devices, systems, and methods described herein. In general, the exemplary method800may include using data from a live-action sequence to select an appropriate representation of a virtual player based on the data. For example, if the data includes information regarding a certain bowled ball in a live-action cricket match, simply placing that data directly into a virtual reality simulation may result in a mis-matched virtual bowler relative to a virtual representation of that bowled ball. Thus, a virtual bowler should have a release point that corresponds to the actual release point of the ball in the live-action cricket match. Also, or instead, a virtual bowler should have an appropriate motion corresponding to the ball in the live-action cricket match—e.g., a slower release for a relatively slow bowled ball.

The exemplary method800may also or instead include altering or adjusting data from a live-action sequence for incorporation into a virtual reality simulation. This may include, for example, adjusting a trajectory of a virtual ball relative to the ball in the live-action cricket match based on a difference in parameters between the virtual reality environment and the physical setting from the live-action cricket match—e.g., different weather or a different type of pitch. This may also or instead include, for example, adjusting a trajectory of a virtual ball relative to the ball in the live-action cricket match based on a difference in parameters or attributes between one or more virtual players in the virtual reality environment and one or more live-action players from the live-action cricket match. For example, if a virtual batsman is batting in a different alignment or is using a different batting stance (e.g., a right-handed or a left-handed stance) that differs from a batsman to which the ball in the live-action cricket match was bowled, this information may be used to adjust the trajectory of the virtual ball. This may be done in the same or similar manner in which a bowler in the physical world would adjust the bowled ball's trajectory based on the differing attribute. By way of further example, a virtual batsman may also or instead have a differing physical attribute relative to a batsman to which the ball in the live-action cricket match was bowled (e.g., is shorter or taller), and this information may be used to adjust the trajectory of the virtual ball. Altering or adjusting data from a live-action sequence for incorporation into a virtual reality simulation may also or instead include reformatting the data, filling in gaps in the data, and/or reverse-engineering or otherwise manipulating the data so that it can be effectively used in the virtual reality simulation.

As shown in step802, the exemplary method800may include generating a virtual reality environment including a virtual player in a setting. As discussed herein, the virtual reality environment may be configured for use in a virtual reality cricket simulation, where a projectile includes a cricket ball and the virtual player is a bowler.

As shown in step804, the exemplary method800may include receiving projectile data indicative of movement of a projectile launched by a player in a live-action sequence. As discussed herein, in the context of cricket, the projectile data may include information regarding a bowled ball from a live-action cricket match, and thus the player in the live-action sequence may include a bowler in the live-action cricket match. The projectile data may include one or more discrete locations of the projectile before, during, or after the release of the projectile launched by the player in the live-action sequence. The projectile data may also or instead include a trajectory of the projectile, a speed of the projectile (e.g., an initial velocity of the projectile when released by the player or a velocity recorded downstream from the player, where the velocity may be provided as vectors in three-dimensional space), a spin of the projectile (if any), a release point of the projectile, a release angle of the projectile, at least one location of the projectile downstream from the player (e.g., at a mid-point between its release and a target), a target location of the projectile downstream from the player (e.g., where it hits a target, or where it would have hit a target if not intervened with), and so on. Also, or instead, some of the aforementioned datapoints may be calculated or estimated from other information included in the projectile data. For example, a spin of the projectile, if present, may be estimated from a trajectory and a speed of the projectile, and/or from contact between the projectile and a playing surface (e.g., by analyzing the resulting directional vectors of the projectile before and after contacting the playing surface).

The projectile data may also or instead include information pertaining to the player that launched the projectile in the live-action sequence. For example, the projectile data may include the distance covered in a player's delivery when bowling a ball in cricket (e.g., the distance of the player's “run-up”), the speed of one or more portions of the delivery (e.g., the speed of the run-up, pre-delivery stride, ball release, or follow though), whether the player has a “side-on” or “front-on” action, and so on.

As shown in step806, the exemplary method800may include, based on the projectile data, identifying a release point of the projectile by the player in the live-action sequence. The release point may be included in the projectile data, such that identifying the release point includes simply reading the projectile data. Also, or instead, identifying the release point may include calculating the release point based on information in the projectile data (e.g., based on a trajectory of the projectile).

As shown in step808, the exemplary method800may include determining a motion of the virtual player in the virtual reality environment based on the projectile data and the release point. For example, if the projectile data shows a relatively slow bowled ball, the motion of the virtual player may be determined to be relatively slow, or have a relatively short run-up in their delivery.

Determining the motion of the virtual player may include selecting one of a plurality of motions stored in a database (e.g., the database130ofFIG. 1). These motions may include motions of avatars or video feeds of live-action sequences as described below in step810. The selected motion from the plurality of motions may be selected to most closely match attributes of the projectile data or the determined motion. In some implementations, the attributes of the projectile data are weighted. For example, while it may be true that a cricket bowler typically bowls a ball slower to achieve greater spin, the projectile data may show that a certain cricket bowler in a live-action sequence may have achieved both relatively high spin and a relatively high velocity. In such circumstances, the velocity attribute may be weighted higher than the spin attribute, so that a selected motion demonstrates that a ball will be released at a relatively high speed. Also, or instead, if the projectile data includes information pertaining to the player that launches the projectile, that information may be weighted more than information pertaining to the projectile itself. In this manner, if the player concealed the pace of the projectile or the spin of the projectile during their delivery, the player's concealment in their delivery may be substantially replicated in the virtual reality environment, thereby providing a more realistic experience to a user.

As shown in step810, the exemplary method800may include, on a display of the virtual reality environment viewable by a user, displaying the virtual player moving according to the motion and a graphical representation of the projectile moving according to a temporal series of locations of the projectile. Displaying the virtual player moving according to the motion may include presenting a first-person view of the virtual player on the display as described herein. Displaying the virtual player moving according to the motion may also or instead include presenting video data of the player from a live-action sequence. In this manner, the virtual player may more directly correspond to the player that launched the projectile in a live-action sequence. Displaying the virtual player moving according to the motion may also or instead include presenting an avatar. The avatar may be based off of a player in the physical world, such as where the avatar is created using one or more of key-framing and motion capture techniques of the player in the physical world.

As shown in step812, the exemplary method800may include, based on the projectile data, identifying a first trajectory of the projectile. The first trajectory may include the actual trajectory of the projectile in the real-world.

As shown in step814, the exemplary method800may include manipulating the first trajectory using one or more parameters to determine a second trajectory. Manipulating the first trajectory may include adding a curvature to the first trajectory. The curvature may be based at least in part on a spin of the projectile (if any), or a reaction of the projectile when contacting a playing surface. Adding curvature to the first trajectory may be accomplished by introducing a constant bi-directional drag force (in the x- and z-directions) on the projectile. This constant force may be based at least in part on one or more of spin, seam angle, velocity in the direction opposite to the drag vector, air density, cross-sectional area of the projectile, and a drag force coefficient. Manipulating the first trajectory may also or instead include interpolating between different paths for the projectile created using one or more projectile motion equations. For example, manipulating the first trajectory may include cubic spline interpolation between three-dimensional data points to generate third-order polynomial equations that simulate the trajectory of the projectile in three-dimensional space.

Manipulating the first trajectory may also or instead include changing a parameter of the projectile data. By way of example, such a parameter may include one or more of a release point of the projectile, a release angle of the projectile, an initial speed of the projectile when released by the player, and a location of the projectile downstream from the player. For example, manipulating the first trajectory may include changing the release angle of the projectile or the rotation/swing of the player, which in turn can change the effect of a drag force on the projectile.

As discussed herein, the parameter may be changed based on a difference between the live-action sequence and the setting of the virtual reality environment. Thus, the exemplary method800may include altering a path of the graphical representation of the projectile in the virtual reality environment from a trajectory included in the projectile data based on one or more predetermined parameters that differ between the live-action sequence and the setting of the virtual reality environment. By way of example, such a difference between the live-action sequence and the setting of the virtual reality environment may include one or more of a playing surface, weather, lighting, time of day or time of year, climate or altitude (e.g., for air density), a physical attribute of a user (e.g., a height of the user for displaying the user as a batsman in a virtual reality cricket simulation, whether the user is right-handed or left-handed, and so on), and a physical attribute of a virtual player (e.g., the height of a bowler in a virtual reality cricket simulation, whether the bowler is right-handed or left-handed, and so on).

As shown in step816, the exemplary method800may include, on a display of the virtual reality environment viewable by a user, displaying a graphical representation of the projectile launched from the virtual player and moving according to the second trajectory.

Thus, using techniques described above, data from a live cricket match with recorded ball data (e.g., trajectory and location data) may be used in a virtual reality environment (e.g., in substantially real time). The virtual reality environment may substantially mimic the real-world setting from the live cricket match, or a different setting, where the user, as the batman, may see virtual representations from a first-person perspective including representations of themselves (e.g., their hands or gloves, their bat, a part of their helmet, and so on). In this manner, the user may view a virtual reality version of a real-world cricket ball that is bowled in the same or similar manner in a live-action cricket match. Furthermore, virtual reality simulation techniques disclosed herein may facilitate the user viewing a playback of recorded data from a live-action sequence. Thus, in implementations, a virtual reality simulation method facilitates game play with professional athletes.

As described above, for a more immersive experience, movements of a bowler from a live-action sequence such as a live cricket match may be recorded and automatically applied to a player within the virtual reality simulation. The player may thus make the same approach as a professional bowler in a live-action match, and may release a ball in the virtual reality simulation that follows the same (or similar) path as the ball in the live-action match. This may facilitate asynchronous play between fans at home and professionals around the world.

While certain implementations have been described, other implementations are additionally or alternatively possible. For example, while a certain configuration of an accessory including a bat300is described above with reference toFIGS. 3A and 3B, other accessory configurations are additionally or alternatively possible for the bat. For example, referring now toFIG. 9, a bat300′ may include an alternate arrangement and orientation for the solenoids330′ disposed therein. For the sake of efficient description, elements with prime (′) element numbers inFIG. 9should be understood to be similar to elements with unprimed element numbers inFIGS. 3A and 3B, and are not described separately herein.

In another alternate implementation, one or more haptic feedback devices or solenoids are disposed on an exterior of an accessory. In this manner, the haptic feedback devices may be structurally configured to strike or otherwise contact an exterior surface of the accessory to provide force feedback (e.g., for replicating a projectile or other object striking the surface of the accessory).

Further, it will be understood that any of the devices, systems, and methods described herein may also or instead include other hardware such as a camera or other sensors, power sources, controls, input devices such as a keyboard, a touchpad, a computer mouse, a switch, a dial, a button, and so on, and output devices such as a display, a speaker or other audio transducer, light-emitting diodes or other lighting or display components, and the like. Other hardware may also or instead include a variety of cable connections and/or hardware adapters for connecting to, for example, external computers, external hardware, external instrumentation or data acquisition systems, and the like.

Moreover, it will be understood that any of the devices, systems, and methods described herein may also or instead include other aspects of virtual reality simulation such as those found in typical virtual reality gaming. By way of example, a virtual reality simulation described herein may score a user's performance and decisions. In the context of cricket, these scores may be based on whether to bat, the quality of the batting, and other game-related factors. In one or more embodiments, a user may repeat a sequence of virtual play or may move on to additional plays. A user's progress (or regression) over time may be tracked and monitored. The user may also or instead be able to access scores, replay scenes, as well as view and review data, for example, via a personalized summary on a webpage, mobile application, or gaming interface.

The above systems, devices, methods, processes, and the like may be realized in hardware, software, or any combination of these suitable for a particular application. The hardware may include a general-purpose computer and/or dedicated computing device. This includes realization in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable devices or processing circuitry, along with internal and/or external memory. This may also, or instead, include one or more application specific integrated circuits, programmable gate arrays, programmable array logic components, or any other device or devices that may be configured to process electronic signals. It will further be appreciated that a realization of the processes or devices described above may include computer-executable code created using a structured programming language such as C, an object oriented programming language such as C++, or any other high-level or low-level programming language (including assembly languages, hardware description languages, and database programming languages and technologies) that may be stored, compiled or interpreted to run on one of the above devices, as well as heterogeneous combinations of processors, processor architectures, or combinations of different hardware and software. In another aspect, the methods may be embodied in systems that perform the steps thereof, and may be distributed across devices in a number of ways. At the same time, processing may be distributed across devices such as the various systems described above, or all of the functionality may be integrated into a dedicated, standalone device or other hardware. In another aspect, means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such permutations and combinations are intended to fall within the scope of the present disclosure.

Embodiments disclosed herein may include computer program products comprising computer-executable code or computer-usable code that, when executing on one or more computing devices, performs any and/or all of the steps thereof. The code may be stored in a non-transitory fashion in a computer memory, which may be a memory from which the program executes (such as random-access memory associated with a processor), or a storage device such as a disk drive, flash memory or any other optical, electromagnetic, magnetic, infrared or other device or combination of devices. In another aspect, any of the systems and methods described above may be embodied in any suitable transmission or propagation medium carrying computer-executable code and/or any inputs or outputs from same.

It will be appreciated that the devices, systems, and methods described above are set forth by way of example and not of limitation. Absent an explicit indication to the contrary, the disclosed steps may be modified, supplemented, omitted, and/or re-ordered without departing from the scope of this disclosure. Numerous variations, additions, omissions, and other modifications will be apparent to one of ordinary skill in the art. In addition, the order or presentation of method steps in the description and drawings above is not intended to require this order of performing the recited steps unless a particular order is expressly required or otherwise clear from the context.

The method steps of the implementations described herein are intended to include any suitable method of causing such method steps to be performed, consistent with the patentability of the following claims, unless a different meaning is expressly provided or otherwise clear from the context. So, for example, performing the step of X includes any suitable method for causing another party such as a remote user, a remote processing resource (e.g., a server or cloud computer) or a machine to perform the step of X. Similarly, performing steps X, Y and Z may include any method of directing or controlling any combination of such other individuals or resources to perform steps X, Y and Z to obtain the benefit of such steps. Thus, method steps of the implementations described herein are intended to include any suitable method of causing one or more other parties or entities to perform the steps, consistent with the patentability of the following claims, unless a different meaning is expressly provided or otherwise clear from the context. Such parties or entities need not be under the direction or control of any other party or entity, and need not be located within a particular jurisdiction.

It should further be appreciated that the methods above are provided by way of example. Absent an explicit indication to the contrary, the disclosed steps may be modified, supplemented, omitted, and/or re-ordered without departing from the scope of this disclosure.

It will be appreciated that the methods and systems described above are set forth by way of example and not of limitation. Numerous variations, additions, omissions, and other modifications will be apparent to one of ordinary skill in the art. In addition, the order or presentation of method steps in the description and drawings above is not intended to require this order of performing the recited steps unless a particular order is expressly required or otherwise clear from the context. Thus, while particular embodiments have been shown and described, it will be apparent to those skilled in the art that various changes and modifications in form and details may be made therein without departing from the spirit and scope of this disclosure and are intended to form a part of the invention as defined by the following claims, which are to be interpreted in the broadest sense allowable by law.