U.S. Pat. No. 10,867,454

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure will be described in detail with reference to the appended drawings. Note that, in this specification and the appended drawings, structural elements that have substantially the same function and structure are denoted with the same reference numerals, and repeated explanation of these structural elements is omitted.

Moreover, the description will be given in the following order.

1. Configuration of information processing apparatus

2. Configuration of virtual space and small space

3. Method of operation using small space

3-1. Movement of virtual object

3-2. Drawing pseudo virtual object out of small space

3-3. Operation on user virtual object

3-4. Drawing pseudo user virtual object out of small space

4. Processing example of operation using small space in virtual space

5. Other methods of operation using small space

6. Method of displaying pseudo virtual object in small space

7. Method of operation on small space

7-1. Method of operation of changing size of small space

7-2. Operation of changing scale in small space and scroll operation in small space

8. Method of operation using small space for shifting to virtual space in different modes

9. Method of operation using small space in virtual space having plural hierarchies

10. Method of operation for deleting and displaying small space

11. Example of application to augmented reality space (AR space)

12. Supplement

13. Concluding remarks

1. Configuration of Information Processing Apparatus

An information processing apparatus according to an embodiment of the present disclosure is described with reference toFIGS. 1 and 2. The information processing apparatus provided in the present disclosure includes a user information detector102that detects a user100. In addition, the information processing apparatus provided in the present disclosure arranges a virtual object of the user100in a virtual space on the basis of information related to the user100detected by the user information detector102. In addition, the information processing apparatus provided in the present disclosure processes an operation of the user100on the virtual space on the basis of the information related to the user100detected by the user information detector102. Moreover, the technology provided in the present disclosure is preferably applicable to an information processing apparatus using the virtual space, such as game consoles. In addition, the information processing apparatus is preferably a general-purpose computer or a portable terminal having communication means such as smartphones. In addition, the virtual space is an example of a first space including a virtual object that is an example of a first object.

FIG. 1is a diagram illustrating an example of the user information detector102used in the information processing apparatus. InFIG. 1, the user information detector102is an imaging device, and in one example, four user information detectors102are installed to acquire information related to the user100, such as the motion, shape, and position of the user100. In addition, the user information detector102is preferably, in one example, an imaging device that captures the entire circumference, and in this case, one imaging device can be used. Moreover, the user information detector102can be any device that acquires the information related to the user100, and is not limited to an imaging device. In one example, the user information detector102can be a device that irradiates the user100with light such as infrared rays and acquires the information related to the user100by detecting the reflected light.

FIG. 2is a block diagram illustrating the configuration of the information processing apparatus provided in the present disclosure. The information processing apparatus provided in the present disclosure further includes a user model generation unit104, a user viewpoint detector106, a virtual space information generation unit108, a small space information generation unit110, an operation information processing unit112, a storage unit114.

The user model generation unit104generates a user virtual object to be arranged in the virtual space on the basis of the information related to the user100acquired by the user information detector102. The user virtual object generated by the user model generation unit104can be a form imitating the form of the user100in the real space or can have a different form from the user100in the real space. In one example, the user model generation unit104can generate a user virtual object having different clothes from the user100in the real space.

The user viewpoint detector106detects a direction in which the user100is facing. In one example, the user viewpoint detector106can be a device to be worn on the head of the user100(e.g., a device like headgear). In this case, the user viewpoint detector106can be provided with an image sensor, and the viewpoint of the user100can be detected on the basis of an image captured by the image sensor. In addition, the user viewpoint detector106can be provided with a gyro-sensor and/or an acceleration sensor, and the viewpoint of the user100can be detected on the basis of the angular velocity, angular acceleration, and acceleration detected by such a sensor. In addition, the user information detector102described above can include the function of the user viewpoint detector106. In the case where the user information detector102is an imaging device, the viewpoint of the user100can be detected by allowing the imaging device to detect the eye of the user100.

The virtual space information generation unit108generates information related to the virtual space on the basis of information or the like stored in the storage unit114. The virtual space-related information generated by the virtual space information generation unit108includes information related to the virtual object arranged in the virtual space. In addition, the virtual space information generation unit108arranges the user virtual object generated by the user model generation unit104in the virtual space. Moreover, the user virtual object is an example of the virtual object and is included in the virtual object. Moreover, the virtual space information generation unit108is an example of a first space information generation unit that generates information related to the virtual space that is an example of the first space.

The small space information generation unit110generates information related to a small space to be described later, which is arranged in the virtual space. The small space can be a space obtained by copying at least a portion of the virtual space, and in this case, the small space information generation unit110receives the information related to the virtual space from the virtual space information generation unit108. Then, the small space information generation unit110generates information related to the small space on the basis of the information received from the virtual space information generation unit108. In addition, the small space information generation unit110sends the information related to the generated small space to the virtual space information generation unit108, and the virtual space information generation unit108arranges the small space in the virtual space on the basis of the information. Moreover, the small space information generation unit110is an example of a second space information generation unit that generates information related to the small space that is an example of the second space.

As described above, in a case where the small space is obtained by copying at least a portion of the virtual space, the virtual space information generation unit108and the small space information generation unit110exchange information with each other. In addition, the small space information generation unit110can generate the information related to the small space on the basis of the information stored in the storage unit114without copying the virtual space. In addition, the small space information generation unit110can generate the information related to the small space on the basis of the information from the virtual space information generation unit108and the information stored in the storage unit114.

The operation information processing unit112processes information related to an operation on the virtual space on the basis of the operation of the user100on the small space. The operation information processing unit112recognizes that the user100is operating the small space on the basis of the information related to motion of the user100from the user information detector102. Then, the operation information processing unit112processes the information related to an operation on the virtual space on the basis of the recognized operation of the user100on the small space. The virtual space information generation unit108generates information related to the virtual space on the basis of the operation information processed by the operation information processing unit112.

Moreover, inFIG. 1, the user information detector102is shown as an imaging device. However, the user information detector102can be an operation device that is operated with the hand of the user100, or can be an operation device configured to be worn on the hand of the user100. The operation information processing unit112can process the operation information on the basis of the information from the user information detector102configured as described above.

The information processing apparatus provided in the present disclosure further includes a video signal generation unit116and a video display unit118. The video signal generation unit116generates a video signal on the basis of the virtual space-related information generated by the virtual space information generation unit108. Here, the virtual space-related information generated by the virtual space information generation unit108includes the small space-related information generated by the small space information generation unit110.

The video display unit118displays a video image of the virtual space on the basis of the video signal generated by the video signal generation unit116. The video display unit118can be, in one example, a typical monitor, or can be a display device attached to the user100, like head-mounted displays, which gives a sense of immersion in the virtual space to the user100. In addition, the video display unit118can be provided with a liquid crystal display or can be provided with an organic EL (Electro Luminescence) display.

2. Configuration of Virtual Space and Small Space

The above description is given of the configuration of the information processing apparatus provided in the present disclosure. The following description is given of the configuration of the virtual space and the small space arranged in the virtual space, which are displayed in the information processing apparatus provided in the present disclosure. The small space is used for the operation of the user100, as described later.

FIG. 3is a diagram illustrating the configuration of a virtual space200based on the information generated by the virtual space information generation unit108. The virtual space200includes, in one example, a virtual object. Examples of the virtual object include a virtual object202indicating the user100(hereinafter also referred to as a user virtual object), a virtual object204indicating a person different from the user100(hereinafter also referred to as an other person virtual object), and virtual objects206and208indicating things (hereinafter also referred to as a thing virtual object). The user virtual object202is a virtual object operated by the user100. This operation is performed on the basis of motion or the like of the user100detected by the user information detector102.

Further, the virtual space200has a small space300, which is arranged in the virtual space200, based on the information generated by the small space information generation unit110. In one example, the small space300can be configured to be obtained by completely copying the configuration of the virtual space200, by copying only a part of the configuration of the virtual space200, or by including a configuration different from that of the virtual space200. This allows the small space300to include a pseudo virtual object corresponding to the virtual object in the virtual space200. Moreover, a view surrounded by a dotted circle inFIG. 3is a view showing by enlarging the small space300arranged in the virtual space200. Moreover, the pseudo virtual object is an example of a second object included in the small space300that is an example of the second space.

FIG. 3illustrates an example in which the configuration of the virtual space200and the configuration of the small space300are substantially the same. InFIG. 3, the small space300includes therein a pseudo user virtual object302corresponding to the user virtual object202and a pseudo other person virtual object304corresponding to the other person virtual object204. In addition, the small space300includes therein pseudo thing virtual objects306and308respectively corresponding to the thing virtual objects206and208.

Moreover, inFIG. 3, the pseudo virtual objects in the small space300are displayed in a positional relationship similar to that between the virtual objects in the virtual space200. However, the small space300can be configured such that a specific pseudo virtual object like the pseudo user virtual object302is displayed in the center of the small space300. Such configuration of the small space300makes it possible for the user100to recognize more intuitively the positional relationship with a specific virtual object in the virtual space200.

Further, the small space300can be configured to be different in scale (scale ratio) from the virtual space200. In other words, the scale in the small space300can be equal to the scale in the virtual space200, or can be larger or smaller than the scale in the virtual space200.FIG. 3illustrates an example in which the scale in the small space300is smaller than the scale in the virtual space200. Thus, the pseudo virtual object in the small space300is displayed smaller than the virtual object in the virtual space200.

Moreover, the small space300can be configured to perform display for distinguishing the small space300from the periphery of the small space300to indicate the boundary of the small space300. In one example, the small space300can be configured such that the whole or a part of the periphery of the small space300varies periodically. In addition, the small space300can be displayed in a manner that a shining ring moves from one end to the other end of a side of the outer frame of the small space300(e.g., from the upper end to the lower end, or from the left end to the right end) periodically. Alternately, the small space300can be displayed in a manner that the whole of the small space300shines periodically. In addition, the periphery of the small space300can be indicated in a manner that texture thereof varies. In other words, the small space300can be displayed translucently, can be displayed in the form of polygons, can be displayed in the form of wire frame, or can be displayed with increased luminance.

FIG. 3illustrates an example in which the virtual space200is displayed as an elevated view of the virtual space200from above. However, the virtual space200can be displayed across the user's viewpoint.FIG. 4is a diagram illustrating an example of the virtual space200displayed across the user's viewpoint. InFIG. 4, an arm portion202aof the user virtual object202is displayed instead of the entire image of the user virtual object202. However, as illustrated inFIG. 4, in this event, the entire pseudo user virtual object302corresponding to the user virtual object202in the virtual space200can be displayed in the small space300. Moreover, a view surrounded by a dotted circle inFIG. 4is a view showing by enlarging the small space300arranged in the virtual space200.

FIG. 5is a diagram illustrating another example of a mode in which the small space300is displayed. InFIGS. 3 and 4, the small space300is a cube, but the small space300can be a cylindrical small space300aor a triangular pyramidal small space300bas illustrated inFIG. 5. In other words, the shape of the small space300can be any shape. In addition, inFIGS. 3 to 5, the small space300is three-dimensional, but as illustrated inFIG. 6, the small space300can be displayed as a two-dimensional plane300cin which the small space300is projected from the upper surface or a two-dimensional plane300din which the small space300is projected from the side face. The small space300can be represented in a one-dimensional form.

3. Method of Operation Using Small Space

The above description is given of the configuration of the virtual space200and the small space300displayed in the information processing apparatus provided in the present disclosure. The following description is given of various examples of an operation on the virtual space200performed by the user100using the small space300. The method of operation described below is useful, in one example, in performing an operation by the user100in a game using the augmented virtual space200.FIGS. 7 and 8are diagrams illustrating a process of moving the virtual object in the virtual space200by the user100performing an operation of moving the pseudo virtual object of the small space300within the small space300.

The user virtual object202is moved in the virtual space200by movement of the user100in the real space or by operating any operation device. As illustrated inFIG. 7, when the arm of the user virtual object202goes into the small space300by the operation of the user100, a portion202acorresponding to the arm of the user virtual object202that goes into the small space300is displayed at a corresponding position in the virtual space200.

As illustrated inFIG. 8, the pseudo other person virtual object304in the small space300is operated by the operation of the user100. Here, the user100performs an operation so that the virtual user object202grasps the pseudo other person virtual object304in the small space300and moves the pseudo other person virtual object304. Then, when the pseudo other person virtual object304is operated, the user virtual object202in the virtual space200moves as illustrated inFIG. 8in response to movement of the pseudo other person virtual object304in the small space300. InFIG. 8, the user virtual object202before the movement is indicated by two-dot chain line, and the user virtual object202after the movement is indicated by solid line.

The method of an operation using the small space300as described above makes it possible for the user100to operate easily a virtual object that is far from the user virtual object202in the virtual space200. Moreover, the above description is given of the example in which the portion202acorresponding to the arm of the user virtual object202that goes into the small space300is displayed in the virtual space200. However, the portion202acorresponding to the arm of the user virtual object202that goes into the small space300is not necessarily displayed.

The above description is given of the method of operation of moving a virtual object in the virtual space200by allowing the user100to perform the operation of moving the pseudo virtual object in the small space300. The following description is given of a method of operation of moving a virtual object in the virtual space200by allowing the user100to perform an operation of drawing the pseudo virtual object in the small space300out of the small space300.

As illustrated inFIG. 9, when the pseudo other person virtual object304in the small space300is moved upward by the operation of the user100, the other person virtual object204in the virtual space200moves upward in the virtual space200in response to the movement of the pseudo other person virtual object304. Then, when a portion of the pseudo other person virtual object304is drawn out of the small space300, the portion (the upper half of the body inFIG. 9) of the pseudo other person virtual object304drawn out of the small space300is displayed in an enlarged form in conformity with the scale of the virtual space200. In addition, in this event, a portion of the other person virtual object204corresponding to the portion (the upper half of the body inFIG. 9) of the pseudo other person virtual object304drawn out of the small space300is not displayed.

FIG. 10is a diagram illustrating a state in which the pseudo other person virtual object304is completely drawn out of the small space300by the operation of the user100after the state illustrated inFIG. 9. As illustrated inFIG. 10, the pseudo other person virtual object304completely drawn out of the small space300is converted into the other person virtual object204, and is arranged in the virtual space200. Such an operation method makes it possible for the user100to move easily a virtual object that is far from the user virtual object202in the virtual space200to the vicinity of the user virtual object202.

The above description is given of the method of the operation of moving the virtual object in the virtual space200by allowing the user100to perform an operation of drawing the pseudo virtual object in the small space300out of the small space300. The following description is given of a method of operation on the user virtual object202. As described above, the user virtual object202in the virtual space200is a virtual object representing the user100and can have a different property from other virtual objects.FIG. 11is a diagram illustrating a method of an operation of moving the pseudo user virtual object302in the small space300and accordingly of moving the user virtual object202in the virtual space200.

Similar to the operation method described above, when the user100moves the pseudo user virtual object302in the small space300, accordingly the user virtual object202in the virtual space200also moves. However, in this event, when the small space300does not move along with the user virtual object202, the relative positional relationship between the hand of the user virtual object202and the small space300will change as the user virtual object202moves. If the relative positional relationship between the hand of the user virtual object202and the small space300is changed, it is certain that the user100fails to perform an operation using the small space300.

To solve this problem, as illustrated inFIG. 11, in the case where the user virtual object202moves, the small space300also moves along with the user virtual object202in response to the movement of the user virtual object202. Such an operation method makes it possible for the user100to move easily the user virtual object202to a remote place in the virtual space200. In addition, even in the case where the user virtual object202moves, the user100is able to perform continuously the operation using the small space300. Moreover, inFIG. 11, the user virtual object202before the movement is indicated by two-dot chain line, and the user virtual object202after the movement is indicated by solid line.

Further, to solve the above-described problem, the configuration in which the operation is confirmed when the user100performs the processing of confirming the operation without immediately considering the operation on the pseudo user virtual object302in the small space300can be possible. In one example, the confirmation of the operation can be performed by allowing the user100to grasp and move the pseudo user virtual object302and then to release the pseudo user virtual object302.

The above description is given of the method of the operation of moving the pseudo user virtual object302into the small space300. The following description is given of a method of an operation in which the pseudo user virtual object302is drawn out of the small space300with reference toFIG. 12. Similar to the operation method described above, when the pseudo user virtual object302in the small space300is drawn upward, accordingly the user virtual object202in the virtual space200also moves upward.

Then, when the pseudo user virtual object302is completely drawn out of the small space300as illustrated inFIG. 12, the user100logs out from the virtual space200. In one example, in a case where the virtual space200is a virtual space200relating to a game displayed by a game console or the like, the user100logs out and ends the game by performing the operation of drawing the pseudo user virtual object302out of the small space300. Moreover, in the example described above, the user100logs out from the virtual space200by drawing completely the pseudo user virtual object302out of the small space300. However, the user100can log out from the virtual space200by drawing a portion of the pseudo user virtual object302out of the small space300. Such an operation method makes it possible for the user100to log out from the virtual space200more easily.

4. Processing Example of Operation Using Small Space in Virtual Space

The above description is given of the basic operation method using the small space300in the virtual space200. The following description is given of the processing example of the operation using the small space300in the virtual space200as described above with reference toFIG. 13.

In S100, the user information detector102first acquires information related to the user100, such as the motion, position, or the like of the user100in the real space. Then, in S102, the virtual space information generation unit108generates information related to the virtual space200on the basis of information from the user model generation unit104, the storage unit114, and the small space information generation unit110. In addition, the small space information generation unit110generates information related to the small space300on the basis of the information from the virtual space information generation unit108and the storage unit114.

Then, in S104, the operation information processing unit112determines whether or not the pseudo virtual object in the small space300is operated on the basis of the information from the user information detector102and the virtual space information generation unit108. If the operation information processing unit112determines that the pseudo virtual object is not operated in S104, the processing proceeds to S116.

Then, in S116, the video signal generation unit116generates a video signal on the basis of the information generated by the virtual space information generation unit108. Then, in S118, the video display unit118displays the video of the virtual space200on the basis of the video signal generated by the video signal generation unit116.

In S104, if the operation information processing unit112determines that the pseudo virtual object is operated, the processing proceeds to S106. In S106, the operation information processing unit112calculates the physical influence quantity on the basis of the operation of the pseudo virtual object. Then, in S108, the operation information processing unit112calculates the movement amount of the pseudo virtual object that is an operation target and the movement amount of the virtual object corresponding to the operated pseudo virtual object.

Then, in S110, the operation information processing unit112determines whether or not the destination of the operation target is outside the small space300and whether or not the operation target is the pseudo user virtual object302. In S110, if the operation information processing unit112determines that the destination of the operation target is outside the small space300and the operation target is the pseudo user virtual object302, the processing proceeds to S112. Then, in S112, the user100logs out from the virtual space200.

In S110, if the operation information processing unit112determines that the destination of the operation target is not outside the small space300or the operation target is not the pseudo user virtual object302, the processing proceeds to S114. In S114, the operation information processing unit112sends the movement amount calculated in S108to the virtual space information generation unit108and the small space information generation unit110. Then, the virtual space information generation unit108and the small space information generation unit110generate information related to the virtual space200and the small space300after movement of the operation target on the basis of the received movement amount. Then, the processing proceeds to S116and then S118, and the video display unit118displays the virtual space200after the movement of the operation target.

5. Other Methods of Operation Using Small Space

The above description is given of the processing example of the operation using the small space300in the virtual space200. The following description is given of other methods of operation in which the small space300is used.FIG. 14is a diagram illustrating an example in which a pseudo virtual object in the small space300is operated using a tool.

As described above, the small space300can include a pseudo virtual object whose configuration of a virtual object or the like in the virtual space200is reduced. In this case, it is conceivable that the size of the pseudo virtual object included in the small space300is small and so it will be difficult for the user100to operate the pseudo virtual object.

Thus, the user100operates a pseudo virtual object in the small space300using a tool such as tweezers210as illustrated inFIG. 14. The operation of the pseudo virtual object using the tool in this manner makes it possible for the user100to operate the small-sized pseudo virtual object more accurately. Moreover, the tool can be selected using a tool bar displayed in the virtual space200. In addition, the tweezers210are an example of the tool and not limited to this example. In one example, the tool can be a long needle, and so the user100can operate by piercing the pseudo virtual object with the needle.

Further, the user100can specify a region in the small space300and can operate the pseudo virtual objects included in the region together.FIG. 15is a diagram illustrating this operation method in which the user100specifies a region312in the small space300through the user virtual object202. Then, when the user100operates one of three pseudo virtual objects310included in the region312after specifying the region312, the three pseudo thing virtual objects are operated.

In this event, the region312can be specified using the finger of the user virtual object202. In one example, the region312can be specified by tracing the region312with the index finger of the user virtual object202. In addition, the region312can be specified using the above-described tool. In addition, the display mode of a pseudo virtual object that is the operation target can be changed so that the user100can recognize that the pseudo virtual object is the operation target by specifying the region312. In one example, the pseudo virtual object that is the operation target can change in color by specifying the region312.

6. Method of Displaying Pseudo Virtual Object in Small Space

The above description is given of the various operation methods of the pseudo virtual object in the small spaces300. The following description is given of another method of displaying the pseudo virtual object.FIG. 16is a diagram illustrating an example in which the pseudo user virtual object302and the pseudo other person virtual object304are arranged behind a pseudo thing virtual object314. In a case where a pseudo virtual object is arranged behind another pseudo virtual object in this manner, the user100fails to recognize existence of a pseudo virtual object arranged behind the pseudo thing virtual object314. Thus, in the case where a pseudo virtual object is arranged behind another pseudo virtual object and the rearward pseudo virtual object is hidden as illustrated inFIG. 16, a portion of the forward pseudo virtual object can be displayed translucently as shown by an ellipse inFIG. 16. This allows the user100to recognize the existence of a hidden pseudo virtual object.

In the case where the hidden pseudo virtual object is an important pseudo virtual object, a portion of the pseudo virtual object can be displayed translucently as described above. In one example, in the case where the hidden pseudo virtual object is a pseudo virtual object that is operable by the user100, a portion of the hidden pseudo virtual object can be displayed translucently. In addition, only in the case where the pseudo user virtual object302is hidden, a portion of the hidden pseudo virtual object can be displayed translucently.

The above description is given of the example in which a portion of the pseudo virtual object is displayed translucently. The following description is given of an example in which a pseudo virtual object corresponding to an inoperable virtual object is not displayed in the small space300.FIG. 17is a diagram illustrating an example in which the other person virtual object204and the thing virtual object208are inoperable and so a corresponding pseudo virtual object is not displayed in the small space300. In this case, the pseudo user virtual object302and the pseudo thing virtual object306are displayed in the small space300.

In this manner, the pseudo virtual object corresponding to the inoperable virtual object is not displayed in the small space300, so the user100can easily recognize an operable virtual object. Moreover, the display mode of the pseudo virtual object corresponding to the inoperable virtual object can be changed as illustrated inFIG. 16. In other words, in the case where the operable virtual objects are the user virtual object302and the other person virtual object304inFIG. 16, another inoperable virtual object (the thing virtual object314inFIG. 16) can be displayed transparently to allow the user100to recognize easily an operable virtual object. In this event, even if the user100tries to operate the pseudo virtual object corresponding to the inoperable virtual object, the pseudo virtual object can be configured so that it fails to be operated, in one example, by allowing the arm the user virtual object202to pass through it.

Further, the operable pseudo virtual objects can be shown in various other ways. In one example, the operable pseudo virtual object can be indicated by periodic changes of all or part of the pseudo virtual objects. In addition, the operable pseudo virtual object can be indicated by periodically moving the shining ring from one end to the other end of the pseudo virtual object, or can be indicated by shining the entire pseudo virtual object periodically. In addition, the operable pseudo virtual object can be indicated in a manner that texture thereof varies. In other words, the operable pseudo virtual object can be displayed translucently, can be displayed with a polygon, can be displayed in the form of wire frame, or can be displayed with increased luminance.

Further, the operable pseudo virtual object can be indicated by changing the size of the pseudo virtual object. In other words, the operable pseudo virtual object can be indicated by enlarging or reducing the pseudo virtual object. In addition, the operable pseudo virtual object can be indicated by a change in the periphery of the pseudo virtual object. In other words, the operable pseudo virtual object can be indicated by giving an expression like a misty condition around the pseudo virtual object.

7. Method of Operation on Small Space

The above description is given of the various operation methods of the pseudo virtual object in the small spaces300. The following description is given of a method of operation on the small space300.FIGS. 18 and 19are diagrams illustrating an example of a method of an operation in which the user100performs an operation of changing the size of the small space300through the user virtual object202.

As illustrated inFIG. 18, the user virtual object202first touches one edge of the small space300or holds one edge of the small space300. Then, as illustrated inFIG. 19, the user virtual object202performs an action of pulling up one edge of the small space300, so the size of the small space300can be increased. In this event, the size of the small space300increases and the scale in the small space300can also be changed with a change in the size of the small space300.

In other words, the pseudo user virtual object302and the pseudo other person virtual object304in the small space300illustrated inFIG. 19are respectively larger than the pseudo user virtual object302and the pseudo other person virtual object304in the small space300illustrated inFIG. 18. As will be appreciated, the user100can reduce the small space300in a similar manner of the operation. Such an operation method makes it possible for the user100to display the small space300on a scale corresponding to the configuration in the virtual space200. In one example, in a case where many virtual objects exist in the virtual space200, it is possible to operate easily the pseudo virtual object in the small space300by increasing the size of the small space300.

Further, in the example described above, the scale in the small space300is changed with a change in the size of the small space300. However, even in the case where the size of the small space300is changed, the scale in the small space300is not necessarily changed. In one example, in a case where the size of the small space300increases, the range displayed in the small space300can be widened without a change in the scale in the small space300. Such an operation method makes it possible for the user100to perceive the configuration in the wider range of the virtual space200.

Further, as illustrated inFIG. 20, the scale in the small space300can be larger than the scale in the virtual space200depending on a change in the size of the small space300. In other words, as illustrated inFIG. 20, the pseudo user virtual object302in the small space300can be displayed larger than the user virtual object202in the virtual space200. Such an operation method makes it possible for the user100to operate a portion of the virtual object in more detail. In one example, in the example ofFIG. 20, the user100is able to operate a portion of the user virtual object202, for example, clothes or the like, more finely and more accurately.

The above description is given of the method of the operation of changing the size of the small space300. The following description is given of an operation of changing the scale in the small space300and a scroll operation in the small space. As illustrated inFIG. 21, the user100performs a gesture like pinch-out as indicated by an arrow400through the user virtual object202in a predetermined area such as the ground or wall in the small space300, so the scale in the small space300can be enlarged. In addition, the user100performs a gesture like scrolling as indicated by an arrow402through the user virtual object202in the predetermined area such as the ground or wall in the small space300, so the display area in the small space300can be changed.

FIG. 22is a diagram illustrating the state of the small space300after enlargement of the scale in the small space300by an operation like pinch-out indicated by the arrow400inFIG. 21. As can be appreciated fromFIG. 22, the scale in the small space300ofFIG. 22is larger than the scale in the small space300ofFIG. 21. Thus, the pseudo user virtual object302and the pseudo other person virtual object304inFIG. 22are displayed respectively larger than the pseudo user virtual object302and the pseudo other person virtual object304in the small space300inFIG. 21.

FIG. 23is a diagram illustrating the state of the small space300after the change in the display area in the small space300by the operation like scrolling indicated by the arrow402inFIG. 21. As can be appreciated fromFIG. 23, the scroll operation allows the pseudo user virtual object302and the pseudo other person virtual object304that existed at the left end of the small space300inFIG. 21to be located at the right end of the small space300inFIG. 23. Moreover, in the case of such an operation, although the display mode in the small space300is changed, the size of the small space300in the virtual space200is not changed. Such an operation method makes it possible for the user100to change the scale in the small space300using the simple operation method and to change the display area of the small space300.

8. Method of Operation Using Small Space for Shifting to Virtual Space in Different Modes

The above description is given of the method of operation on the small space300. The following description is given of a method of an operation of shifting to the virtual space200in different modes using the small space300.FIG. 24is a diagram illustrating an example in which the virtual space200includes two small spaces300eand300farranged in the virtual space200. These two small spaces have different internal configurations in the small space300.

The first small space300eis the small space300obtained by copying the configuration in the virtual space200used currently and includes the pseudo user virtual object302corresponding to the user virtual object202in the virtual space200. The second small space300fis the small space300having a configuration different from that in the virtual space200used currently, and is based on information stored, in one example, in the storage unit114. The second small space300fincludes a pseudo virtual objects corresponding to a virtual object that is not included in the virtual space200used currently, such as the pseudo other person virtual object304and the pseudo thing virtual object308.

Then, as illustrated inFIG. 25, the user100is able to make transition of the virtual space200to the state of the second small space300fby performing an operation of bringing the first small space300einto contact with the second small space300f. Such an operation method makes it possible for the user100to make transition of the virtual space200so that it may include the configuration of the small space300including any configuration stored in the storage unit114.

Moreover, information, which is stored in the storage unit114, related to the second small space300fcan be information created in advance by the user100. In one example, the user100can create the information related to the second small space300fincluding any pseudo virtual object by the operation of the user100. In addition, the information, which is stored in the storage unit114, related to the second small space300fcan be information related to the state of the virtual space200used previously. In this case, the above-described operation performed by the user100allows the state of the virtual space200used previously to be read from the storage unit114and allows the virtual space200to be made transition to the previous state.

The user100can select the previous time point using a bar representing the time series displayed in the virtual space200as the method of selecting the previous time point. In addition, the user100can select the previous time point, in one example, by rotating the second small space300f. In this event, in the second small space300f, the previous state of the virtual space200can be reproduced in response to the rotation of the second small space300f. In one example, the rotation of the second small space300fallows the state of the virtual space200one hour ago to be reproduced in the second small space300f. Similarly, the rotation of the second small space300fallows the state of the virtual space200two hours ago to be reproduced in the second small space300f.

Further, the state of the virtual space200can be continuously stored, periodically stored, or can be stored on the basis of a predetermined operation by the user100, or can be stored on the basis of a specific event in the virtual space200. Here, the specific event is, in one example, a case where the user virtual object202goes into a specific place in the virtual space200or a case where the user virtual object202touches a specific virtual object in the virtual space200.

9. Method of Operation Using Small Space in Virtual Space Having Plural Hierarchies

The above description is given of the method of operation using the small space300for shifting to the virtual space200in different modes. The following description is given of a method of operation using the small space300in the virtual space200having a plurality of hierarches.

The virtual space200can have a plurality of hierarchies, and the user100can perform an operation such that the user virtual object202goes into the small space300as illustrated inFIG. 26. Such an operation by the user100allows the virtual space200can be made transition to the virtual space200of a different hierarchy.

FIG. 27is a diagram illustrating an example in the case where the user virtual object202exists in the virtual space200of the third hierarchy. When the user100goes into the virtual space200, it is difficult to recognize a hierarchy of the virtual space200. Thus, as illustrated inFIG. 27, a hierarchical display212indicating the current hierarchy of the virtual space200can be displayed in the virtual space200. In addition, the display state of the small space300to be displayed can be changed depending on the hierarchy of the virtual space200. In one example, the small space300can be displayed as a cube in the case of the first hierarchy, the small space300can be displayed as a triangular pyramid in the case of the second hierarchy, and the small space300can be displayed as a cylinder in the case of the third hierarchy. Such a configuration makes it possible for the user100to recognize easily a hierarchy of the virtual space200used currently.

10. Method of Operation for Deleting and Displaying Small Space

The above description is given of the method of the operation using the small space300in the virtual space200having a plurality of hierarches. The following description is given of a method of an operation for deleting the small space300and an operation for displaying the small space300.

As described above, the method of the operation using the small space300is useful for the operation of the virtual object in the virtual space200. However, the small space300can become an obstacle, so the small space300can be temporarily deleted from the virtual space200. In one example, as illustrated inFIG. 28, the small space300can be deleted by placing the small space300on the palm of the user virtual object202and performing an operation by the user100of sweeping off it with the other arm of the user virtual object202. In addition, the small space300can be deleted by allowing the user100to perform an operation of directing the palm downward from the state in which the small space300is placed on the palm of the user virtual object202. In addition, the small space300can be deleted by allowing the user100to perform an operation of stepping on the small space300with the foot of the user virtual object202.

Moreover, the deleted small space300can be displayed again by the operation of the user100, and there are various operation methods therefor. In one example, the small space300can be displayed by allowing the user100to perform an operation of directing the palm of the user virtual object202upward.

Further, as illustrated inFIGS. 29 and 30, the small space300can be displayed by allowing the user100to perform an operation of folding the hand of the user virtual object202and then opening the hand of the user virtual object202.

Such a configuration makes it possible for the user100to delete easily the small space300in the case where the small space300is an obstacle in the virtual space200. In addition, it is possible for the user100to display easily the small space300when the user100performs the operation using the small space300after deletion of the small space300.

11. Example of Application to Augmented Reality Space (AR Space)

The above description is given of the operation method in which the small space300is used in the virtual space200. The following description is given of an example in which the above-described operation method using the small space300is applied to the AR.

The AR space is a virtual space in which information is added to the real environment through a transparent display or the like or is a virtual space in which information is deleted from the real environment. In one example, an AR space to which furniture or the like, which is not in the real space, is added to an image of the room in the real space acquired through an imaging device or the like can be presented to the user100through a display device.

The method of the operation using the small space300provided in the present disclosure is applied to the operation method in the AR space, so the user100is able to operate easily the virtual object in the AR space. Furthermore, the user100is able to operate an object in the real space corresponding to the virtual object in the AR space by operating the virtual object in the AR space using the small space300in the AR space.

In one example, when there is a lighting fixture in the real space and there is a virtual object corresponding to the lighting fixture in the AR space, the user100can operate the lighting fixture in the real space by operating a pseudo virtual object corresponding to the lighting fixture in the small space300. In one example, in a case where the user100performs an operation of turning on the switch for the pseudo virtual object corresponding to the lighting fixture in the small space300, the switch of the lighting fixture in the real space can be turned on. Such an operation makes it possible for the user100to operate easily an object at a position distant from the user100in the real space.

Further, as can be appreciated, the user100is able to operate on information added to the AR space using the small space300although it does not exist in the real space. In one example, in a case where a sofa, which does not exist in the real space, is added to the AR space and the AR space is displayed, the user100can operate the added sofa using the small space300. Such an operation makes it possible for the user100to operate easily an object, which does not exist in the real space, in the AR space.

12. Supplement

The preferred embodiment(s) of the present disclosure has/have been described above with reference to the accompanying drawings, whilst the present disclosure is not limited to the above examples. A person skilled in the art may find various alterations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present disclosure.

In one example, the description provided above with reference to the example ofFIG. 24is given of the example in which a plurality of small spaces300having different configurations are arranged in the virtual space200. However, a plurality of small spaces300having the same configuration can be arranged in the virtual space200. In one example, the user100can rotate one small space300and so perceive the configuration of the virtual space200used currently from an angle different from another small space300. Such a configuration makes it possible for the user100is able to perceive the state of the virtual space200used currently simultaneously from different angles. In addition, the user100can bring a plurality of small spaces300into contact with each other, thereby moving the pseudo virtual object into another small space300via the contacted surface. Such an operation makes it possible for the user100to move easily the pseudo virtual object in the small space300from end to end.

Further, the description provided above with reference to the example ofFIG. 9is given of the method of the operation of drawing the pseudo virtual object out of the small space300. However, conversely, an operation of putting the virtual object in the virtual space200into the small space300can be performed. This operation method makes it possible for the user100to place easily the virtual object at hand to a far place in the virtual space200.

Further,FIGS. 27 and 28illustrate the example in which the virtual space200has a plurality of hierarchies. In this event, an example of presenting the hierarchy of the virtual space200to the user100by displaying a hierarchical display of the virtual space200in the virtual space200is illustrated. However, re-immersion into a predetermined hierarchy or higher can be prohibited to prevent excessive immersion into the virtual space200.

Further, a computer program for causing the virtual space information generation unit108, the small space information generation unit110, and the operation information processing unit112to operate as described above can be provided. In addition, a storage medium in which such a program is stored can be provided.

13. Concluding Remarks

As described above, the information processing apparatus provided in the present disclosure enables the user100to operate easily the virtual object in the virtual space200using the small space300including the pseudo virtual object. Accordingly, the user100is able to easily place the virtual object in the virtual space200far away or place it at hand of the user virtual object202. In addition, the user100is able to operate the user virtual object202in the virtual space200by operating the pseudo user virtual object302in the small space300. This makes it possible for the user100to move the user virtual object202far away in the virtual space200and to log out from the virtual space200.

Further, the information processing apparatus according to the embodiment of the present disclosure allows the pseudo virtual object corresponding to the inoperable virtual object to be displayed in different modes. In addition, the pseudo virtual object corresponding to the inoperable virtual object is not necessarily displayed. Such a configuration makes it possible for the user100to recognize easily an operable virtual object in the virtual space200.

Further, the information processing apparatus according to the embodiment of the present disclosure allows the pseudo virtual object in the small space300to be operated using a tool. Such a configuration makes it possible for the user100to operate easily a small-sized pseudo virtual object in the small space300.

Further, the effects described in this specification are merely illustrative or exemplified effects, and are not limitative. That is, with or in the place of the above effects, the technology according to the present disclosure may achieve other effects that are clear to those skilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing apparatus including:

a first space information generation unit configured to generate information related to a first space including a first object;

a second space information generation unit configured to generate information related to a second space such that the second space is arranged in the first space, the second space being smaller than the first space and including the first object in the first space; and

an operation information processing unit configured to process information regarding an operation on the first space on the basis of an operation on the second space by a user.

(2)

The information processing apparatus according to (1),

in which the first space is a virtual space.

(3)

The information processing apparatus according to (2),

in which the first space information generation unit generates the information related to the first space such that a first object is included in the first space, and

the second space information generation unit generates the information related to the second space such that a second object corresponding to the first object in the first space is included in the second space.

(4)

The information processing apparatus according to (3),

in which the first object moves in response to motion of the second object by the operation of the user.

(5)

The information processing apparatus according to (3) or (4),

in which the operation on the second space by the user includes moving the second object in the second space to an outside of the second space.

(6)

The information processing apparatus according to any one of (3) to (5),

in which the operation on the second space by the user includes moving the first object in the first space into the second space.

(7)

The information processing apparatus according to any one of (3) to (6),

in which the operation on the second space by the user includes operating the second object of the user in the second space.

(8)

The information processing apparatus according to (3),

in which the operation on the second space by the user includes moving the second object of the user in the second space to an outside of the second space, and

the user logs out from the virtual space when the second object of the user in the second space is moved to an outside of the second space.

(9)

The information processing apparatus according to any one of (2) to (8),

in which the second space information generation unit generates the information related to the second space such that a size of the second space is changed in response to the operation of the user.

(10)

The information processing apparatus according to any one of (2) to (9),

in which the second space information generation unit generates the information related to the second space such that a scale in the second space is changed in response to the operation of the user.

(11)

The information processing apparatus according to any one of (2) to (10),

in which the second space information generation unit generates the information related to the second space such that a plurality of the second spaces are arranged in the first space.

(12)

The information processing apparatus according to any one of (2) to (11), further including:

a storage unit configured to store a state of the first space,

in which the second space information generation unit generates the information related to the second space in which the state of the first space stored in the storage unit is reproduced.

(13)

The information processing apparatus according to (12),

in which the first space information generation unit generates the information related to the first space in which the state of the first space stored in the storage unit is reproduced on the basis of a specific operation performed by the user on the second space in which the state of the first space stored in the storage unit is reproduced.

(14)

The information processing apparatus according to (13),

in which a plurality of the second spaces are arranged in the first space, and

the specific operation performed by the user on the second space in which the state of the first space stored in the storage unit is reproduced is an operation of bring one of the plurality of second spaces into contact with another of the plurality of second spaces by the user.

(15)

The information processing apparatus according to any one of (2) to (14),

in which the second space information generation unit generates the information related to the second space such that display for distinguishing the first space from the second space is performed on the second space or on a periphery of the second space.

(16)

The information processing apparatus according to any one of (3) to (15),

in which the second space information generation unit generates the information related to the second space such that a portion of the second object in the second space is displayed in a display mode different from a display mode of another object in the second object.

(17)

The information processing apparatus according to (16),

in which the second space information generation unit generates the information related to the second space such that the first object inoperable by the user in the first space is prevented from being displayed in the second space.

(18)

The information processing apparatus according to any one of (2) to (17),

in which the operation on the second space by the user is performed using a specific tool.

(19)

A method including:

generating, by a first space information generation unit, information related to a first space;

generating, by a second space information generation unit, information related to a second space such that the second space is arranged in the first space, the second space being smaller than the first space; and

processing, by an operation information processing unit, information regarding an operation on the first space by allowing a user to perform an operation on the second space,

in which the second space is arranged in the first space.

(20)

A computer program for

causing a first space information generation unit to generate information related to a first space,

causing a second space information generation unit to generate information related to a second space such that the second space is arranged in the first space, the second space being smaller than the first space, and

causing an operation information processing unit to process information regarding an operation on the first space on the basis of an operation on the second space by a user.

REFERENCE SIGNS LIST

100user102user information detector104user model generation unit106user viewpoint detector108virtual space information generation unit110small space information generation unit112operation information processing unit114storage unit116video signal generation unit118video display unit200virtual space202user virtual object204other person virtual object206,208thing virtual object300small space302pseudo user virtual object304pseudo other person virtual object306,308pseudo thing virtual object