U.S. Pat. No. 10,463,978

DETAILED DESCRIPTION

The present disclosure describes embodiments with reference to the drawing figures listed above. Persons of ordinary skill in the art will appreciate that the description and figures illustrate rather than limit the disclosure and that, in general, the figures are not drawn to scale for clarity of presentation. Such skilled persons will also realize that many more embodiments are possible by applying the inventive principles contained herein and that such embodiments fall within the scope of the disclosure which is not to be limited except by the claims.

Many game controllers, and the individual controls within them, may be challenging to operate for users that have a physical disability, particularly users that are visually-impaired or blind. Some early games, for example, were text heavy, requiring the user to imagine the virtual world in which the game takes place by reading text, which often made the game inaccessible and excluded visually-impaired or blind users. Some of the greatest game improvements have occurred in the use of complex graphics. Games employing these complex graphics are challenging to use for visually-impaired or blind users since the accompanied audible feedback may only describe the game's complex graphics in a limited manner through the use of screen readers. Although the gaming industry has made some progress towards improving a game's general accessibility, a need remains for improved game controllers and accessories that address the very specialized manner in which visually-impaired or blind users interact with video games.

FIGS. 1A-1Care views of an exemplary game controller system100, in accordance with some embodiments.FIG. 3Ais a bottom view of an exemplary game controller system with an attached exemplary braille accessory, in accordance with some embodiments. Referring toFIGS. 1A-1C and 3A, a game controller system100may include a game controller104and a braille accessory102removably attached to game controller104. Braille accessory102may provide a visually-impaired or blind user with braille input and output gaming capability. Game controller104may be sized and shaped to be held by the hands130of a user as is well known to a person of ordinary skill in the art. In an embodiment, game controller104may be shaped or sized to fit an average hand size of a population of users. Game controller104may include controls on a top side for manipulation by a user's thumbs and controls on a bottom side for manipulation by a user's fingers, as the user grips game controller using with two hands. Users may have preferences on the shape, size, position and texture of such thumb-side and finger-side controls that differ from the standard controls that are integral to game controller104. Moreover, some users may prefer to manipulate controls on just the thumb-side of game controller104without having controls located on the finger-side of game controller104.

Game controller104may be configured to translate user input into control signals that are provided to a computing device121, such as a personal computer or gaming console, as is well known to a person of ordinary skill in the art. Computing device121may include a memory device to store instructions and a processing device to execute the instructions stored in the memory device. Computing device121may include any type or size of memory and any type of processing device, as is well known to a person of ordinary skill in the art. Computing device121may be a personal computer, laptop, tablet, mobile device, gaming console, and the like.

The control signals may be mapped to commands to control a video game or perform other operations to control the video game or computing device121. In an embodiment, game controller104may be configured to transmit control signals to a computing device using any wired or wireless transmission means known to a person of ordinary skill in the art.

Game controller104may include a grip101configured to be held using two hands130. As such, grip101includes a left-hand portion103configured to be gripped by a left hand and a right-hand portion105configured to be gripped by a right hand. Right-hand portion105may oppose left-hand portion103. Further, a central portion107may be positioned intermediate the left-hand portion103and the right-hand portion105and may house a plurality of controls, e.g., joysticks109, directional pad111, action buttons113, triggers115, and the like.

In an embodiment, braille accessory102may be sized to be received by handheld-sized game controller104. Braille accessory102may be removably attached to an underside of game controller104as shown inFIG. 1Ausing any attachment means known to a person of ordinary skill in the art e.g., attachment mechanism156, further developed below.

Braille accessory102may include a braille display106to output a braille representation of any game text126or any game audio occurring during the course of game play (and displayed on output section122of game display128) to make game text126accessible to visually-impaired or blind users. In an embodiment, game text126may include text that is displayed in a game chat window or box120in which gaming participants can engage in live discussion with one another. Game chat window120allows a user to see messages he or she has received, as well as messages that he or she has sent. Game chat window120may have any of a variety of designs.

Game chat window120may include a larger output section122on top of a smaller input section124. Output section122may show messages that the user may have already sent or messages that the user may have received from other players. Input section124may display messages created by the user in real time that have not yet been sent. Online video games, especially massively multiplayer online (MMO) games often include chat windows or boxes120for communication between players. Chat windows or boxes120may be customizable in size, color, location, and the like and may be changed to receive only messages from certain players or include multiple tabs to provide different information in discrete layers of a single window display128. In some circumstances, any kind of game audio may also be translated to braille for output to braille display106.

Braille accessory102may receive game text126or game audio using any wired or wireless transmission means known to a person of ordinary skill in the art. In an embodiment, braille accessory102may receive game text126using a wired cable140connected between braille accessory102and computing device121executing the game, e.g., a personal computer or gaming console. In another embodiment, braille accessory102may receive game text126using any wireless means known to a person of ordinary skill in the art, e.g., a wireless circuit or receiver (not shown). In yet another embodiment, braille accessory102may receive game text126from game controller104through a wired or wireless connection between braille accessory102and game controller104. Game controller104may receive game text126from computing device121executing the game through any wired or wireless means known to a person of ordinary skill in the art, e.g., cables, wireless receivers, and the like. Game controller104, in turn, may transmit game text126to braille accessory102for translation from text to braille, for output on braille display106. In an embodiment, translation of game text126to braille may occur at braille accessory102(e.g., using control circuit165). In other embodiments, translation of game text126to braille may occur at game controller104, with the results transmitted to the braille accessory102for output on braille display106.

Braille display106may include a plurality of braille cells108arranged in an array110having a plurality of columns112and rows114. AlthoughFIG. 1Ashows array110having three columns112and three rows114, a person of ordinary skill in the art should recognize that any number of braille cells108arranged in any number of columns112or rows114comes within the scope of the disclosure. Each braille cell108includes six pins or dots116arranged in two columns and three rows that can be read tactually with the fingers of visually-impaired or blind users. Each braille cell108may represent a single letter, number, punctuation, or other character using braille coding as is well known to a person of ordinary skill in the art.FIG. 4illustrates an exemplary braille alphabet.

In an embodiment shown inFIG. 3A, braille cell308may be rotated by ninety degrees such that six pins or dots316are arranged in three columns and two rows (i.e., three pins or dots arranged horizontally and two pins316or dots316arranged vertically). By doing so, a “line” of text may be formed not by horizontally-adjacent cells, but by vertically-adjacent cells to account for the user's finger position on braille accessory302on the back of controller304. Put differently, rotating braille cell308by ninety degrees allows for a user's finger position as it wraps around from front to back of controller304to more naturally read a line of text on braille display306.

To read braille display106, a visually-impaired or blind user may run his or her fingers, e.g., index, middle, ring, or pinky fingers, on hands130through the braille cells108to tactually detect raised pins or dots116as is well known to a person of ordinary skill in the art. In an embodiment, a visually-impaired or blind user may read braille display106vertically from top to bottom and left to right such that the first character is represented in braille cell108located in the first column112first row114, the second character is represented in braille cell108located in the first column112second row114, the third character is represented in braille cell108located in the first column112third row114, the fourth character is represented in braille cell108located in the second column112first row114, the fifth character is represented in braille cell108located in the second column112second row114, and so on.

Braille display106may further include scroll actuator118to scroll game text126. Scroll actuator118may include any actuating means known to a person of ordinary skill in the art, including any electronic or mechanical actuating means, e.g., switch, push button, touch activated electronic switch, and so on. Because braille display106has a predetermined number of braille cells108that may represent a predetermined number of characters in game text126or game audio, it may be necessary for a visually-impaired or blind user to actuate or otherwise indicate to braille accessory102that it is ready to receive a new batch of characters from game text126by actuating scroll actuator118after reading the current batch of characters displayed or output on braille display106. A visually-impaired or blind user may program or set braille accessory102to scroll a single column112or row114or multiple columns112or rows114of new characters per each actuation of scroll actuator118. Conversely, a visually-impaired or blind user may program or set braille accessory102to scroll all new characters per each actuation of scroll actuator118. In an embodiment, scroll actuator118may scroll the translation of game text126or game audio one character at a time in a vertical direction, with an oldest character represented by a top left braille cell being scrolled off braille display106and with a newest character to be represented by a bottom right braille cell being scrolled on to the braille display106.

Braille accessory102may include six braille paddles132A-F to allow a visually-impaired or blind user to input or chord braille characters during the course of game play to send to other players. Braille paddles132A-F may be shaped or curved to follow a contour game controller104as game controller104is gripped by hands130.

In an embodiment, braille accessory102may translate the chorded braille characters to text126that is shown on input section124of game display128after transmission from braille accessory102to computing device121or game controller104. In other embodiments, braille accessory102may transmit chorded braille to game controller104for translation to game text126. Braille paddles132A-F may be located on a finger-side of game controller104, three braille paddles132A-C on a right side and three braille paddles132D-F on a left side of game controller104, when game controller104is gripped thumb-side up by hands130of a visually-impaired or blind user. Braille paddles132A-F may be manipulated by fingers other than the thumbs, e.g., index, middle, ring, or pinky fingers, of a user gripping game controller104with both hands130. Control circuit165on braille accessory102may translate a touch force applied to any of braille paddles132A-F into an individual braille character.

In an embodiment, braille paddles132A-F may be actuated by a touch force from fingers on hands130to generate signals that are mapped or translated from braille to text126or game audio. The text126or game audio is then transmitted from braille accessory102to computing device121or from braille accessory102to game controller104to computing device121for display on input section124before being displayed on output section122of chat window120after being sent. Such a configuration would allow a visually-impaired or blind user to communicate with other players in real time.

The chording of braille on braille paddles132A-F is well known to a person of ordinary skill in the art and may follow any number of braille coding schemes, such as the 6-dot scheme shown inFIG. 4. A person of ordinary skill in the art should recognize that other braille coding schemes exist that come within the scope of the disclosure, e.g., 8-dot braille coding schemes.

Braille accessory102may transmit the braille chords input by a visually-impaired or blind user using braille paddles132A-F to computing device121or game controller104using any wired or wireless transmission means known to a person of ordinary skill in the art, e.g., cable140, wireless circuit or receiver (not shown), and the like. Each of braille paddles132A-F may be shaped for actuation by a user's fingers other than the thumb, as the user grips game controller104using both hands and as braille accessory102is received on an underside of game controller104.

In an embodiment, braille paddles132A-F may be removably affixed or attached to braille accessory102or game controller104via any attachment mechanism known to a person of ordinary skill in the art, e.g., screws or other fasteners. In an embodiment, braille paddles132A-F may be attached to braille accessory102or game controller104using pivots as described in U.S. patent application 2016/0317919 to Gassoway, et al., entitled Paddle Accessory For A Game Controller, which is included herein by reference.

Moreover, game controller104may enable braille paddles132A-F to be quickly and easily swapped on braille accessory102or game controller104without the use of tools. Such a configuration may facilitate the use of differently sized or oriented braille paddles132A-F for a customizable game controller104and/or braille accessory102.

Referring toFIGS. 1B and 1C, game controller104may include a plurality of controls located on a top or thumb-side. These controls may be configured to be manually manipulated by right and/or left thumbs of a user gripping the game controller with two hands130during gameplay. In some embodiments, a user may manipulate one or more of the plurality of controls with an index finger. The plurality of controls may include joysticks109, directional pad111, action buttons113, triggers115, and so on as is well known to a person of ordinary skill in the art. Game controller104may include any suitable number and type of controls.

FIG. 1Dis a bottom view of an exemplary game controller system with an attached exemplary braille accessory, in accordance with some embodiments. Referring toFIGS. 1A-1D, braille accessory102may include two removably attached braille paddles132A and132D that are coupled to operate with four braille paddles132B,132C,132E, and132F removably attached to game controller104. In an embodiment in which braille accessory102is removably attached to game controller104, braille accessory102may receive control signals from all six braille paddles132A-F that braille accessory102then translates from braille to text and transmits to computing device121for display as text126on input section124. In an embodiment in which four braille paddles132B,132C,132E, and132F are removably attached to game controller104, braille accessory102may receive controls signals for braille paddles132B,132C,132E, and132F from game controller104that braille accessory102then synchronizes or otherwise couples to controls signals from braille paddles132A and132D, before translating the braille chording representative of all six braille paddles132A-F to text126.

In another embodiment, game controller104may receive control signals from all six braille paddles132A-F, including braille paddles132A and132D attached to braille accessory102, that game controller104then translates from braille to text and transmits to computing device121for display as text126on input section124.

FIG. 1Eis an exploded view of an exemplary braille accessory, in accordance with some embodiments. Referring toFIGS. 1A-1E, braille accessory102may include a housing150to enclose electronic circuits and mechanisms that control operation of braille display106and/or braille paddles132A-F. Housing150may include a first housing portion152and a second housing portion154that removably attaches to first housing portion152using any removable fastening mechanism known to a person of ordinary skill in the art, e.g., screws, built in snaps, and the like. First housing portion152may attach to second housing portion154using snap features designed into first housing portion152and/or second housing portion154, when first housing portion152and second housing portion are made of molded plastic. Housing150may be made of any material known to a person of ordinary skill in the art, e.g., plastic, metal, and the like.

First housing portion152may include an attachment mechanism156to removably attach braille accessory102to game controller104. In an embodiment, attachment mechanism156may be a substantially u-shaped clamp that extends from an end of first housing portion152and removably attaches to central portion107of game controller104. In an embodiment, attachment mechanism156may removably attach game controller104using any removably attachment means known to a person of ordinary skill in the art, e.g., fasteners, snap features built into housing150, and/or game controller104.

A printed circuit assembly160may be located in an interior cavity162of housing150. Printed circuit assembly160may include a plurality of electronic circuits165to control operation of braille display106and/or braille paddles132A-F. In an embodiment, printed circuit assembly160may include a plurality of actuation sensors164that corresponds to braille paddles132A-F. Each of braille paddles132A-F may be configured to activate a corresponding actuation sensor164in response to a force touch to generate a control signal responsive to being depressed (e.g., via finger manipulation). In an embodiment, control circuit165may detect actuation of sensors164to translate the touch force applied to the at least one of paddles132A-F into individual braille characters.

In some embodiments, printed circuit board160may include other sensors that correspond to the plurality of input controls, e.g., joysticks109, directional pad111, action buttons113, or triggers115.

The plurality of actuation sensors164may be any type of sensor or switch known to a person of ordinary skill in the art capable of detecting actuation or manipulation of braille paddles132A-F, e.g., dome switches, tactile switches, buttons, levers, dials, rockers, Hall Effect sensors, and other electronic sensing components. Printed circuit assembly160may include any number or type of actuation sensors164.

Printed circuit assembly160may include a processing device and a memory device as part of electronic circuits165to control operation of braille display106and/or braille paddles132A-F.

FIG. 2Ais a side view of an exemplary braille display206, in accordance with some embodiments andFIG. 2Bis a top view of a column and row electrode arrangement for driving an exemplary braille display206, in accordance with some embodiments. Referring toFIGS. 1A-1E, 2A, and 2B, braille display106or206may have any construction, design, and be made of any materials known to a person of ordinary skill in the art. Refreshable braille display106or206may raise and lower a large array110of pins or dots116arranged in a closely spaced arrangement, allowing quick tactile perception for rapid reading. Refreshable braille display106or206may raise or lower pins or dots116to approximately 0.5 mm with an actuation force of at least 0.15N. Refreshable braille display106or206may employ pins or dots116using any actuating means known to a person of ordinary skill in the art, e.g., electro-active polymers (EAP) materials, electromagnetic means, piezo-electric means, thermal means, pneumatic means, shape memory alloys, and the like.

In an embodiment, braille display106or206may include the use of EAP materials that may aid in footprint reduction such that the braille accessory102is sized to be received by handheld game controller104. One benefit of EAP materials is the ability to pack a large array110of dots or pins116in a relatively small area without interference between dots or pin116.

Braille display206may include an array210of dots or pins216made of any shape, size, or material to fit the application that is known to a person of ordinary skill in the art. Dots or pins216may be mounted on a film221made of any material known to a person of ordinary skill, e.g., rubber, silicone, plastic, and the like. Rows of electrodes213and columns of electrodes215may be arranged on either side of an EAP film219to activate individual elements in EAP film219. Electrodes213and215may be formed of any conductive material known to a person of ordinary skill in the art, e.g., metals. In an embodiment, electrodes213and215may be formed on a substrate217using known semiconductor and other manufacturing techniques, e.g., screen printing, deposition and patterning techniques, and the like.

Each element in EAP film219is mounted with a corresponding dot or pin216in array210. Each element in EAP film219is deactivated or lowered from a top surface of braille display206by applying a voltage across a thickness of EAP film219that, in turn, may cause a local thickness reduction in EAP film219. Although requiring a high actuation voltage, EAP film219may consume very little power and require no power to keep an actuator at a given position.

Braille accessory102may activate dots or pins216to create tactile patterns or highs and lows representative of text126to be read by a visually-impaired or blind user. In an embodiment, a visually-impaired or blind user may use his fingertips to scan dots or pins216to read text126, while freeing his thumbs to manage control of gameplay. Each of the plurality of pins216may include a corresponding electroactive polymer in EAP film219. A control circuit165included in printed circuit assembly160may deliver a current or voltage to the electroactive polymer to actuate the pin216to an active state based at least in part on the translation of the game text126or the game audio to the individual braille characters

EAP film219may be made of any EAP-type material, ionic or field-activated, known to a person of ordinary skill in the art, e.g., conductive polymers, dielectric elastomers, ferroelectric materials, ionic polymer metal composites, and polyvinylidene difluoride materials. EAP film219may be activated using electrostatic forces between two electrodes that squeeze or compress the polymer. For ionic EAP film219, actuation is caused by the displacement of ions inside the polymer. Only a few volts are needed for actuation, but the ionic flow implies a higher electrical power needed for actuation, and energy is needed to keep the actuator at a given position. Examples of ionic EAP film219are conductive polymers, ionic polymer-metal composites (IPMCs), and responsive gels.

Persons of ordinary skill in the art will appreciate that the present disclosure is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present disclosure includes both combinations and sub-combinations of the various features described hereinabove as well as modifications and variations which would occur to such skilled persons upon reading the foregoing description. Only the claims define the scope of the disclosure.