U.S. Pat. No. 10,953,331

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

Exemplary embodiments will be described in detail with reference to the accompanying drawings. These exemplary embodiments will be described in detail for those skilled in the art in order to practice the present disclosure. It should be appreciated that various exemplary embodiments are different but do not have to be exclusive. For example, specific shapes, configurations, and characteristics described in an exemplary embodiment may be implemented in another exemplary embodiment without departing from the spirit and the scope of the present disclosure. In addition, it should be understood that position and arrangement of individual components in each disclosed exemplary embodiment may be changed without departing from the spirit and the scope of the present disclosure. Therefore, a detailed description described below should not be construed as being restrictive. In addition, the scope of the present disclosure is defined only by the accompanying claims and their equivalents if appropriate. Similar reference numerals will be used to describe the same or similar functions throughout the accompanying drawings. It will be understood that for the purposes of this disclosure, “at least one of X, Y, and Z” can be construed as X only, Y only, Z only, or any combination of two or more items X, Y, and Z (e.g., XYZ, XYY, YZ, ZZ).

The terminology used herein is for the purpose of describing exemplary embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising”, when used in this specification, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It will be understood that when an element is referred to as being “connected to” another element, it can be directly connected to the other element, or intervening elements may be present.

Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings.

FIG. 1illustrates an example of a network environment according to exemplary embodiments.

Referring toFIG. 1, the network environment includes clients110, servers, for example, a map server120, an augmented map server130, and a game server140, and a network150.FIG. 1is provided as an example only and thus, the number of terminals and/or the number of servers are not limited thereto.

The client110refers to an electronic device that performs wireless communication with an IoT device acting as a beacon, and may be a fixed terminal or a mobile terminal configured as a computer device. For example, the client110may be a wearable device, a head mounted display (HMD) device, an AR-type device providing an augmented reality function, a smartphone, a mobile phone, a navigation, a computer, a laptop computer, a digital broadcasting terminal, a personal digital assistance (PDA), a portable multimedia player (PMP), a tablet PC, and the like. The client110may communicate with another client110and/or the game server140over the network150in a wired communication manner or a wireless communication manner.

The communication scheme is not particularly limited and may include a communication method that uses a wireless near field communication between devices as well as a communication method using a communication network, for example, a mobile communication network, the wired Internet, the wireless Internet, and a broadcasting network, etc., which may be included in the network150. For example, the network150may include at least one of network topologies that include networks, for example, a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Also, the network150may include at least one of network topologies that include a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, and the like. However, it is only an example and the exemplary embodiments are not limited thereto.

Each of the servers, for example, the map server120, the augmented map server130, and the game server140, may be configured as a computer device or a plurality of computer devices that provides instructions, codes, files, contents, services, etc., through communication with the client110and the network150.

For example, the game server140may provide a file for installing an application to the client110connected through the network150. In this case, the client110may install the application using the file provided from the game server140. Also, the client110may use a service or content provided from the game server140by connecting to the game server140under control of at least one program, for example, a browser or the installed application, and an operating system (OS) included in the client110. For example, in response to a service request message transmitted from the client110to the game server140over the network under control of the application, the game server140may transmit a code corresponding to the service request message to the client110. The client110may provide content for the user by configuring and displaying a screen corresponding to the code under control of the application.

FIG. 2illustrates an example of a client and a game server according to exemplary embodiments.

Referring toFIG. 2, a client210may include, for example, a processor211, an input/output (I/O) interface213, a communication module214, a memory215, and a display216. The client210may include the processor211and one or more applications212that operate on the client210.

The client210may include the memory215, the processor211, the communication module214, and the I/O interface213. A game server250may include a memory253, a processor251, and a communication module252. The memory215,253may include a permanent mass storage device, such as random access memory (RAM), read only memory (ROM), a disk drive, etc., as a computer-readable storage medium. Also, an operating system (OS) and at least one program code, for example, the aforementioned code for browser or the application installed and executed on the client210, may be stored in the memory215,253. Such software constituent elements may be loaded from another computer-readable storage medium separate from the memory215,253using a drive mechanism. The other computer-readable storage medium may include, for example, a floppy drive, a disk, a tape, a DVD/CD-ROM drive, a memory card, etc. According to other exemplary embodiments, software constituent elements may be loaded to the memory215,253through the communication module214,252, instead of, or in addition to, the computer-readable storage medium. For example, at least one program may be loaded to the memory215,253based on a program, for example, the application, installed by files provided over the network220from developers or a file distribution system, for example, the game server140, which provides an installation file of the application.

The processor211,251may be configured to process computer-readable instructions, for example, the aforementioned at least one program code, of a computer program by performing basic arithmetic operations, logic operations, and I/O operations. The computer-readable instructions may be provided from the memory215,253and/or the communication module214,252to the processor211,251. For example, the processor211,251may be configured to execute received instructions in response to the program code stored in the storage device, such as the memory215,253.

The communication module214,252may provide a function for communication between the client210and the game server250over the network220, and may provide a function for communication with another client or another server. For example, a request created at the processor211of the client210based on a program code stored in a storage device such as the memory215may be transferred to the game server250over the network220under control of the communication module214. Inversely, a control signal, an instruction, content, a file, etc., provided under control of the processor251of the game server250may be received at the client210through the communication module214of the client210by going through the communication module252and the network220. For example, a control signal, an instruction, etc., of the game server250received through the communication module214may be transferred to the processor211or the memory215, and content, a file, etc., may be stored in a storage medium further includable in the client210.

The I/O interface213may be a device used for interface with an I/O device. For example, an input device may include a keyboard, a mouse, etc., and an output device may include a device, such as a display for displaying a communication session of an application. As another example, the I/O interface213may be a device for interface with an apparatus in which an input function and an output function are integrated into a single function, such as a touch screen. In detail, when processing instructions of the computer program loaded to the memory215, the processor211of the client210may display a service screen configured using data provided from the game server250or another terminal, or may display content on the display216through the I/O interface213.

According to other exemplary embodiments, the client210and the game server250may include a greater or lesser number of constituent elements than the number of constituent elements ofFIG. 2. However, there is no need to clearly illustrate many constituent elements according to the related art. For example, the client210may include at least a portion of an I/O device, or may further include other constituent elements, for example, a transceiver, a global positioning system (GPS) module, a camera, a variety of sensors, a database, and the like.

FIG. 3illustrates an example of a processor included in a game server according to exemplary embodiments.

Referring toFIG. 3, a processor300included in the game server may include a receiver310, a mapper320, a determiner330, and a provider340as constituent elements. The processor300and the constituent elements of the processor300may be configured to execute instructions according to at least one program code and a code of an OS included in a memory. Here, the constituent elements of the processor300may be representations of different functions performed at the processor300in response to a control command provided from a program code stored in the game server.

The processor300may load, to the memory, a program code stored in a file of a program for a game providing method based on augmented reality. Each of the processor300and the receiver310, the mapper320, the determiner330, and the provider340included in the processor300may execute an instruction corresponding to the program code loaded to the memory.

The receiver310may receive an augmented map that is created in response to performing 3D modeling based on object information about a reality object at an augmented map server. The receiver310may provide map data and object information about the reality object to the augmented map server in response to extracting a location of the reality object from a map provided from a map server.

The mapper320may map an augmented reality object based on a location of the reality object in response to receiving the augmented map.

The determiner330may determine a location of a client based on beacon information of a beacon. The determiner330may determine a location of a user by performing a wireless near field communication between the beacon and a device worn by the user. The determiner330may calculate 3D location coordinates of the client based on a triangulation technique or a trilateration technique using at least three beacons in response to performing wireless near field communication between the beacon and the client.

The provider340may provide an augmented reality game environment by disposing the reality object on the augmented map based on the location of the client. The provider340may provide a battlefield environment and proceed with a game through a predetermined game rule by locating at least three beacons, and by mapping the augmented reality object and the reality object based on beacon information of the at least three beacons and the location of the client. The provider340may determine whether a distance between the reality object and the augmented reality object measured at a distance sensor or the beacon is included in a predetermined distance range, and may provide the augmented reality game environment in response to determining that the distance between the reality object and the augmented reality object is included in the predetermined distance.

FIG. 4illustrates an example of an operation of a game server that embodies augmented reality according to exemplary embodiments.

A map server is configured to provide a map, and may store the map in a map database. Here, the map server may provide the map stored in the map database as a map service. The map server may provide the map itself to a user, and also may process the map and provide the processed map. For example, Internet portable sites, for example, Google, Naver, Daum, etc., may provide the respective map services, for example, Google map, Naver map, Daum map, etc, and may provide an open source, such as API of Google.

An augmented map server may receive the map from the map server, and may perform 3D modeling based on reality object information about a reality object. Here, the augmented map server may create an augmented map by performing 3D modeling based on a location value and location information associated with the reality object. For example, the augmented map server may receive a map API of Google provided from the map server, and may add game information for providing an augmented reality game environment. The augmented map server may create the augmented map by adding a map, items, characters, etc., associated with a game to the map.

In response to the augmented map server creating the augmented map by performing 3D modeling, the game server may map an augmented reality object based on a location of the reality object. The game server may provide an augmented reality game environment430by disposing the reality object on the augmented reality object. For example, assuming a battle game, the game server may map an augmented reality object in which a battle environment created at the augmented map server is constructed at a location of a reality object.

Here, if 3D modeling on the augmented map fails, the game server may dispose the augmented reality object based on the location of the reality object of the map using specific equipment. For example, the game server may provide the augmented reality game environment430by verifying a location of the reality object through an apparatus for tracking a pattern and a location, and by disposing the augmented reality object.

The game server may determine a location of a client using one or more IoT devices that acts as beacons420,421, and422. Also, the game server may determine the location of the client based on location information received from a GPS and beacon information of beacons420,421, and422. Here, the beacon information may include unique identification data of the respective beacons420,421, and422, location information thereof, etc.

The game server may verify the location of the client through wireless communication with the beacons420,421, and422. For example, the user may wear a wearable device410, for example, an HMD device or an AR-type device, which provides an augmented reality function, and may perform Bluetooth communication with the beacons420,421, and422. The user may execute and operate a game in which the augmented reality output through the apparatuses is configured.

The AR-type device may include wearable glasses, a device that combines a virtual reality and a camera and provides location information of a GPS, etc. The HMD device is an image display device configured to display an image of augmented reality and an image of virtual reality through a variety of sensor techniques and wireless communication. The HIVID device or the AR-type device may move a scene displayed on a screen in response to a movement of the user on a 3D space, and may enhance the user's immersion into and the reality of game content by applying a motion of the user to content control.

If a distance between the reality object and the augmented reality object is not secured at a distance sensor and the like configured to measure a distance, the game server may not provide an augmented reality game environment. The game server may determine whether a distance between the reality object and the augmented reality object measured at the beacon or a distance sensor is included in a predetermined distance range, and may provide the augmented reality game environment in response to determining that the distance between the reality object and the augmented reality object is included in the predetermined distance. Here, the game server may use all of the augmented reality object and the reality object as the game object.

The game server may provide a battlefield environment and may proceed with a game through a predetermined game rule by locating at least three beacons, and by mapping the augmented reality object and the reality object based on beacon information of the at least three beacons and the location of the client.

As described above, the game server may provide a game in which a battle takes place using a weapon or a tool at a view of the user to a first person shooter (FPS), for example, an adventure game, an action game, a reality survival game, etc. The game server may provide a game that is created using a 3D scheme and thus, provides a view of a character in the game to be the same as a view of the user, and provides an enhanced reality compared to other games. Here, the game server may provide a view shift of the character to interact with a client, and accordingly, the user may shift a view of the character at the user's intention so that the character may perform an action using the weapon or body.

According to exemplary embodiments, the game server may create an augmented map by performing 3D modeling based on a map of real world, without performing a separate operation on a map or creating a new map, and may configure a battlefield environment by mapping an augmented reality object. Thus, it is possible to provide a battlefield online game based on the augmented reality.

According to exemplary embodiments, the game server may quickly proceed with a game without buffering by performing wireless near field communication with the client using an IoT device acting as a beacon, and may provide a game environment corresponding to a reaction of the client in real time. Thus, it is possible to enhance the user's immersion into the game.

FIG. 5illustrates an example of searching, at a game server, for a location of a client according to exemplary embodiments.

The game server may search for the location of the client using an IoT device acting as a beacon, may provide a game environment by mapping an augmented reality object and a reality object, and may proceed with a game based on a predetermined game rule. For example, the game server may determine the location of the client based on the beacon and information, such as location information of a GPS, a distance sensor, and the like.

For example, with assumption that a first beacon (beacon1)511, a second beacon (beacon2)512, and a third beacon (beacon3)513are present, each beacon may include unique identification data. Each of the first beacon511, the second beacon512, and the third beacon513may be fixed to a place, such as a wall, as a fixed node, and may be used to recognize a mobile node. Each of the first beacon511, the second beacon512, and the third beacon513may track the location of the client.

The game server may calculate 3D location coordinates of the client based on a triangulation technique or a trilateration technique using at least three beacons. Here, the triangulation technique refers to a method of calculating coordinates and a distance of a single point using a feature of a triangle, and may induce location coordinates by, if a point and two reference points are given, measuring angles between a base and other two sides from a triangle configured by the point and the two reference points, by measuring a length of the side, and by performing some calculations. The trilateration technique refers to a method of calculating a relative location of an object using the geometry of a triangle and may use at least two reference points and a distance between an object and each reference point.

The game server may calculate location coordinates using equations of three circles, each in which a distance between each beacon and the location of the client is defined as a radius, based on at least three beacons.

Alternatively, the game server may perform a 3D location recognition calculation by calculating a location of the user using four beacons and a trilateration technique, and may calculate location coordinates by inducing a maximum value and a minimum value of the calculated coordinates of the user, by defining a 3D area in which the coordinates are distributed, by decomposing the 3D area at specific distance intervals, and by applying an average value of four intersection points of the circular circumference configured by a beacons Also, the game server may calculate the location of the client based on a beacon signal strength.

FIG. 6illustrates an example of a method of executing a battlefield online game using augmented reality according to exemplary embodiments.

In operation651, a map server610may store map data and may extract a location of a reality object based on location information of a map. The map server610may transfer the map data and object information about the reality object to an augmented map server620.

The augmented map server620may perform 3D modeling based on the object information about the reality object received from the map server610in operation652, and may create an augmented map in operation653. For example, the augmented map server620may transfer the augmented map created by performing 3D modeling to a game server630.

In operation654, the game server630may receive the augmented map from the augmented map server620and may map an augmented reality object based on a location of the reality object. In operation655, the game server630may determine a location of a client640based on location information of a GPS and beacon information of a beacon. The game server630may calculate 3D location coordinates of the client640based on a triangulation technique or a trilateration technique using at least three beacons, in response to performing wireless near field communication between the beacon and the like.

In operation656, the game server630may provide an augmented reality game environment by disposing the reality object on an augmented map based on the location of the client640. Here, the game server630may determine the location of the client640using an IoT device. In response to interaction between the game server630and the client640through wireless communication, the game server630may provide the augmented reality game environment by disposing the reality object on the augmented map based on the location of the client640. In operation657, the game server630may proceed with a game based on a preset game rule.

The units described herein may be implemented using hardware components, software components, and/or a combination thereof. For example, a processing device may be implemented using one or more general-purpose or special purpose computers, such as, for example, a processor, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a field programmable array, a programmable logic unit, a microprocessor or any other device capable of responding to and executing instructions in a defined manner. The processing device may run an operating system (OS) and one or more software applications that run on the OS. The processing device also may access, store, manipulate, process, and create data in response to execution of the software. For purpose of simplicity, the description of a processing device is used as singular; however, one skilled in the art will be appreciated that a processing device may include multiple processing elements and multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. In addition, different processing configurations are possible, such as parallel processors.

The software may include a computer program, a piece of code, an instruction, or some combination thereof, for independently or collectively instructing or configuring the processing device to operate as desired. Software and data may be embodied permanently or temporarily in any type of machine, component, physical or virtual equipment, computer storage medium or device, or in a propagated signal wave capable of providing instructions or data to or being interpreted by the processing device. The software also may be distributed over network coupled computer systems so that the software is stored and executed in a distributed fashion. In particular, the software and data may be stored by one or more computer readable recording mediums.

The exemplary embodiments may be recorded in non-transitory computer-readable media including program instructions to implement various operations embodied by a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. The media and program instructions may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of non-transitory computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM disks and DVD; magneto-optical media such as floptical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments.

While certain exemplary embodiments and implementations have been described herein, other embodiments and modifications will be apparent from this description. Accordingly, the invention is not limited to such embodiments, but rather to the broader scope of the presented claims and various obvious modifications and equivalent arrangements.