U.S. Pat. No. 8,161,397

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Reference will now be made in detail to specific embodiments of the invention including the best modes contemplated by the inventors for carrying out the invention. Examples of these specific embodiments are illustrated in the accompanying drawings. While the invention is described in conjunction with these specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. On the contrary, it is intended to cover alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims. In the following description, specific details are set forth in order to provide a thorough understanding of the present invention. The present invention may be practiced without some or all of these specific details. In addition, well known features may not have been described in detail to avoid unnecessarily obscuring the invention.

In accordance with the present invention, the components, process steps, and/or data structures may be implemented using various types of operating systems, programming languages, computing platforms, computer programs, and/or general purpose machines. In addition, those of ordinary skill in the art will recognize that devices of a less general purpose nature, such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or the like, may also be used without departing from the scope and spirit of the inventive concepts disclosed herein. The present invention may also be tangibly embodied as a set of computer instructions stored on a computer readable medium, such as a memory device.

In an embodiment of the present invention, a user's digital environment is mirrored into a virtual world in real-time. The user is then able to use a mirrored virtual world object to affect the corresponding real-world object. Such mirroring can provide intuitive visual cues to the user about the capabilities of their physical environment, and enables easy and intuitive utilization of the capabilities of the resources in the environment.

Embodiments of the present invention provide for the automatic and dynamic sensing of the resources in the environment surrounding a user device, such as, for example, a mobile device.

Embodiments of the present invention also provide for automatic and dynamic code generation for operations on mirrored virtual world objects to control and/or affect real-world objects.

Embodiments of the present invention also bring real-world objects into a virtual world in an automatic and/or assisted way. Specifically, if a real-world object does not have a pre-built virtual presence, some of the functionality can be abstracted to provide a presence for such an object in the virtual world.

An embodiment of the present invention utilizes service and device discovery to automatically discover a user's digital environment. Service discovery may be enabled using a variety of different technologies, such as Universal Plug and Play (UPnP).

FIG. 2is a diagram illustrating a high level overview of an example embodiment of the present invention. This general architecture may run on top of existing virtual world systems, although the invention is not limited by such systems.

A virtual world server200is an external entity that acts as a virtual world host. The entire virtual world environment may be controlled at the server, including the look and feel of the environment, access to multiple virtual spaces, etc.

The local virtual world proxy202is an entity that acts as a mediator between local virtual world clients204and a virtual world server200. The local virtual world proxy202contains a virtual world proxy206that is compatible with a chosen virtual world system. The virtual world proxy206is capable of acting on the interaction between the virtual world client and the virtual world server. The virtual world proxy206can filter information traveling between the client and the virtual world server by taking away some details or by injecting new details. This feature of injection and removal of virtual world entities is used to accomplish some of the aims of the present invention.

In addition to a virtual world proxy206, the local virtual world proxy202is also capable of interacting with the local physical environment. It has the capability to hold the knowledge about the local environment including the presence of local resources. This knowledge can be obtained in multiple ways. The entities208,210within the local environment can publish their information to the local virtual world proxy, or the local virtual world proxy can fetch such information from a dedicated service such as the local DNS, or a service registry, or through using a protocol such as UPnP.

There are multiple devices and services208,210(collectively known as resources) within the environment, which form the knowledge base within the local virtual world proxy. Each resource has one or more features that can be accessed over the network. The information that is registered at the local virtual world proxy is sufficient to use such features.

Along with the above-stated resources, there are multiple devices202carried by various users within the local environment. Each of these devices has information and/or capabilities that can be useful over the network.

A user device202such as a mobile device, for example, may be equipped with a virtual world client212that is capable of displaying the virtual world environment and also allowing users to interact with the virtual world. Along with the virtual world client, the user device also has the capability to sense the local environment through popular mechanism such as Bluetooth, WiFi, UPnP, etc using a local environmental sensor214.

FIG. 3is a component diagram illustrating a local virtual world proxy in accordance with an embodiment of the present invention. This depicts local virtual world proxy204ofFIG. 2in more detail. A mirror manager300is responsible for managing the translation of real-world resource capabilities into their virtual world counterparts.

A protocol library302is a store containing the details necessary for the mirror manager300to convert the capabilities of real world resources (as expressed through their service descriptions) into virtual world scripts that would enable a real-world object to be interacted within the virtual world. A model library304is a store for real-world objects. The resource manager306is responsible for all the interactions with resources308over the network. It collects information about the available resources such as the service descriptions, resource descriptions, etc. The information collected is sent to the mirror manager300.

The primary responsibility of the mirror manager300is to inject the virtual objects into the virtual world client310. The client310sends information about the resources visible to itself (through its local environment sensor) to the local virtual world proxy. A visualization manager312uses this information to inject only those virtual resources into the corresponding client310. The information presented to the client310need not be limited to only those visible to itself. The visualization manager312may inject other virtual resources into the client310, based on the application/implementation.

When the client310performs any operations on the virtual objects, these interactions are reported back to the local virtual world proxy. These operations are then sent to the resource manager, which then relays that back to the corresponding physical resource.

The virtual world proxy also filters out information related to the local resources from going into the virtual world server, keeping the real world integration local.

Additional capabilities may be added to provide security of the real-world resources. For example, in some embodiments, it may be preferable to limit access to certain resources to certain people. The local virtual world proxy can manage such security associations, or may interact with another device, such as an Authentication, Authorization, and Accounting (AAA) server, to provide such services.

Any number of security protocols may be used to ensure the secured access of the local resources. In one example, a user name and password may be required in order to access a resource. In another example, a security certificate may be required in order to access a resource. In another example, a security key may be required in order to access a resource.

Furthermore, additional embodiments may permit different virtual world object representations for the same resource when viewed by different users. For example, embodiments are possible wherein when one user views a local printer the printer is colored red, while when another user views a local printer the printer is colored green. This differentiation may be done for many different reasons. For example, it may be preferable to label certain resources that are “off limits” as red (in lieu of hiding them entirely). Another reason may simply to offer more appealing colors to resources that are “preferred” for the particular user. For example, in an office digital environment, a user may be able to access any printer in the virtual office, but it may be preferably to color the printer that is closest to his physical location as something different than the color of a remote printer, to encourage the user to print to the closest printer.

Another reason to vary colors of resources among users is simply to provide a more appealing virtual world appearance for the user. Some users, for example, may express a preference for certain color schemes. While one user might prefer that printers in his environment be blue, another user might prefer they be yellow. The local virtual world proxy is capable of managing such combinations.

Furthermore, while appearance is described above in terms of color, any number of visual characteristics of the object may be altered from user-to-user. Size, shape, shading, etc. are all possibly alterable visual characteristics.

An additional embodiment is conceived where the invention is integrated into an existing service discovery mechanism. Such an embodiment would not require that the virtual world client contain a local environmental sensor. Users can interact with presented service through their virtual world environments, even though their own devices have no new service discovery features.

In another embodiment of the present invention, a proposed visual representation of a resource may first be sent to the user for approval. The user is able to accept or reject the proposed visual representation. In cases of a rejection, the system may select another possible visual representation to the user. This process may continue until the user finds a virtual representation to his or her liking. Alternatively, the user could upload or create his own virtual representation for the resource.

FIG. 4is a flow diagram illustrating a method for operating a local virtual world proxy in accordance with an embodiment of the present invention. Each step of the method may be performed in hardware, software or any combination thereof. At400, general resource information may be collected and sent to a mirror manager. At402, virtual world scripts may be generated for resources for which general resource information was collected. This may include accessing a protocol library, wherein the protocol library includes information as to how to convert capabilities of real-life resources into virtual world scripts. The capabilities may be expressed through service descriptions. At404, models for representations of resources for which general resource information was collected may be stored in a model library. At406, a virtual world view corresponding to a virtual world client is received. This may be received from a virtual world server. At408, information about resources available to the virtual world client may be obtained. This may be information generated by a local environmental sensor of the virtual world client. At410, virtual representations of one or more resources available to a virtual world client are fetched. At412, virtual representations are injected into the virtual world view. At414, the virtual world view is forwarded to the virtual world client for display to a user of the virtual world client.

FIG. 5is a screen capture of a virtual world in accordance with an embodiment of the present invention. While not pictured, the user's real life office may contain a computer, a printer, a fax machine, and a telephone.FIG. 5depicts the virtual world representation of the office, complete with computer500, printer502, and telephone504. The devices500-504may be depicted using virtual representations stored in the model library. Users of the virtual world may interact with the devices500-504because the capabilities of the real life resources have been converted into virtual world scripts (of course, as described above, there may be some objects that, for one reason or another, do not permit user interaction—these devices may still have virtual world scripts if at some point some user can interact with them via the virtual world).

While the invention has been particularly shown and described with reference to specific embodiments thereof, it will be understood by those skilled in the art that changes in the form and details of the disclosed embodiments may be made without departing from the spirit or scope of the invention. In addition, although various advantages, aspects, and objects of the present invention have been discussed herein with reference to various embodiments, it will be understood that the scope of the invention should not be limited by reference to such advantages, aspects, and objects. Rather, the scope of the invention should be determined with reference to the appended claims.