U.S. Pat. No. 10,127,003

DETAILED DESCRIPTION OF FIGURES

FIGS. 1-10illustrates alternate embodiments and various configurations of the invention.

FIG. 1illustrates a first embodiment comprising a system with Global Layer Storage with mapping. In a preferred embodiment of this system, it is comprised of Network Layer Storage,1500, with mapping in a global location. In other words, Global Layer Storage is provided at one location that is on a connected Appliance Data Network of a plurality of appliances, communicating and providing information to the plurality of computing appliances each with their own display. The connected Appliance Data Network in this embodiment is comprised of a connection;1501,111,121,131; of each subsystem;1500,110,120,130respectively; to the Appliance Data Network,101. The Display Computing Appliances110,120and130. These are instances of a Display Computing Appliance2500as described in further detail inFIG. 25. The Network Layer Storage,1500, is described in further detail inFIG. 15. It should be noted that in this embodiment shown inFIG. 1and the embodiments illustrated inFIGS. 2, 3, 4, 5, 6, 7, 8, 9 and 10that three Display Computing Appliances; as embodied inFIG. 24, 25, 26, 27 or 28; are shown but any number more than one is acceptable.

FIG. 2illustrates a system with Distributed Layer Storage with mapping. There are multiple distributed networked layer storage at separate locations;240,250,260; on the connected Appliance Data Network and each having mapping within the storage coupled to a plurality of user display computing appliances;210,220,230; each with their own display. The connected Appliance Data Network in this embodiment is comprised of a connection;241,251,261,211,221,231; of each subsystem;240,250,260,210,220,230respectively; to the Appliance Data Network,201. In this embodiment a specific Data Layer is stored only once in one of the Network Layer Storage locations,240,250,260. The Display Computing Appliance,2500, is the embodiment of blocks210,220and230and is described in further detail inFIG. 25. The Network Layer Storage,1500, is described in further detail inFIG. 15. The number of Network Layer Storage locations and Display Computing Appliances can be the same or different. It should be noted that in this embodiment shown inFIG. 2and the embodiments illustrated in figures;4,6,7,9and10; the physical location of the Network Layer Storage and the user display computing appliances may be in the same physical location or they may be in their own separate physical locations. Some appliances may have Network Layer Storage in the same physical location as a Display Computing Appliance or even integrated within the appliances, or some may not. The choice configuration is quite flexible and compatible with the present invention. Likewise three Network Layer Storage; embodied as shown inFIGS. 15, 18, 19, 20, 21 and 22; instances are shown inFIGS. 2, 4, 6, 7, 9 and 10but any number above one is acceptable.

FIG. 3illustrates a system with Global Layer Storage and distributed mapping as illustrated in this embodiment. Global network storage,1900, is concentrated in one location, communicating via the connected Appliance Data Network to a plurality of display computing appliances each having local mapping in the particular appliances to provide logic to map the global Data Layer storage. The connected Appliance Data Network in this embodiment is comprised of a connection;1901,311,321,331; of each subsystem;1900,310,320,330respectively; to the Appliance Data Network,301. The Display Computing Appliance,2400, is the embodiment of subsystems310,320and330and is described in further detail inFIG. 24. The Network Layer Storage,1900, is described in further detail inFIG. 19.

FIG. 4illustrates a system with Distributed Layer Storage and distributed mapping, as illustrated inFIG. 4, the system here has a plurality of networked layer storage locations;440,450,460; which are in communication via the connected Appliance Data Network with a plurality of user display computing appliances;410,420,430; which each containing a display and mapping for the layer storage. The connected Appliance Data Network in this embodiment is comprised of a connection;441,451,461,411,421,431; of each subsystem;440,450,460,410,420,430respectively; to the Appliance Data Network,401. In this embodiment a specific Data Layer is stored only once in one of the Network Layer Storage locations,440,450,460. The Display Computing Appliance,2400, is the embodiment of subsystems410,420and430and is described in further detail inFIG. 24. The Network Layer Storage,1900, is the embodiment of subsystems440,450and460and is described in further detail inFIG. 19.

FIG. 5illustrates an embodiment of a system with Global Layer Storage with mapping and layer-edit logic. This is similar toFIG. 1with the exception that editing logic is added to both the Network Layer Storage and to each display-computing appliance. Thus, particular layers can be edited by a selected user, and results in causing modifications to the respective layer storage, and in accordance with one aspect of the present invention, can also selectively alter the composite display presentation on one or more selected ones of the user display computing appliances, depending on the mapping provided. The connected Appliance Data Network in this embodiment is comprised of a connection;1801,511,521,531; of each subsystem;1800,510,520,530respectively; to the Appliance Data Network,501. The Display Computing Appliance,2600, is the embodiment of subsystems510,520and530and is described in further detail inFIG. 26. The Network Layer Storage,1800, is described in further detail inFIG. 18.

FIG. 6illustrates an embodiment of a system with Distributed Layer Storage with mapping and Data Layer edits. This is similar toFIG. 2with the exception that editing logic has been added to both the Network Layer Storage. There are a plurality of Network Layer Storage memory;640,650,660; and a plurality of user display computing appliances;610,620,630; each and all of which has editing logic added to facilitate selectively allowing the user to make changes to the respective zero, one or more layers as determined by the Mapping Logic. The connected Appliance Data Network in this embodiment is comprised of a connection;641,651,661,611,621,631; of each subsystem;640,650,660,610,620,630respectively; to the Appliance Data Network,601. In this embodiment a specific Data Layer is stored only once in one of the Network Layer Storage locations,640,650,660. The Display Computing Appliance,2600, is the embodiment of subsystems610,620and630and is described in further detail inFIG. 26. The Network Layer Storage,1800, is the embodiment of subsystems640,650and660and is described in further detail inFIG. 18.

FIG. 7illustrates an embodiment of a system with Distributed Layer Storage with mapping and Data Layer edits and server synchronization. This system is similar toFIG. 6with the exception that each of the Network Layer Storage devices has server synchronization provided for and included in the storage device. The connected Appliance Data Network in this embodiment is comprised of a connection;741,751,761,711,721,731; of each subsystem;740,750,760,710,720,730respectively; to the Appliance Data Network,701. The Display Computing Appliance,2600, is the embodiment of subsystems710,720and730and is described in further detail inFIG. 26. The Network Layer Storage,2200, is the embodiment of subsystems740,750and760and is described in further detail inFIG. 22. This embodiment includes the synchronization logic with the Network Layer Storage,2200. The embodiment illustrated in thisFIG. 7and the embodiments illustrated inFIGS. 9 and 10provide for synchronization that allows for the same updated data for a respective particular layer to be stored in memory devices at multiple locations in each one of the Network Layer Storage devices, and allows the display computing appliances to access that respective layer from any one of the Network Layer Storage devices. The server synchronization provides the logic that makes sure that when a change is made to a layer in one of the Network Layer Storage devices that the change is communicated to all of the other Network Layer Storage devices that have the same respective layer stored in them, so that the respective layer has its Layer Data updated to contain the same information. This allows for a much more flexible storage network and the Network Layer Storage devices can optionally be, or not be, physically co-located with a respective one of the user display appliances. Alternative, or additionally, some of the user display appliances may have Network Layer Storage, while others do not. Also, it is not a requirement of the present invention that all layers are stored in each and all Network Layer Storage devices. The synchronization is only required when at least two of the Network Layer Storage devices contain a same respective layer, i.e., duplicate copies, so that those storage devices can be maintained to have identical Layer Data stored for that respective same layer. In addition, there is no restriction on how many locations a particular layer is duplicated. Obviously, this provides for a very flexible choice of storage options that allows for static or dynamic allocation of storage for layers which can be optimized for tradeoffs in storage requirements, network traffic/bandwidth/latency and the complexity of managing multiple copies storage.

FIG. 8illustrates an embodiment of a system is similar toFIG. 3.FIG. 8illustrates an embodiment of a system of layered data storage with global Data Layer storage, distributed mapping and selectively providing for Data Layer edits. In this illustrated embodiment of the system, there is a centralized global Network Layer Storage coupled for communication with multiple computing display appliances. The Mapping Logic is done at each appliance and, different from the system ofFIG. 3. There is Edit Logic which has been added to each user display computing appliance and to the Network Layer Storage, so that for a selectively enabled user, the enabled user can make changes to a particular respective mapped edit layer. In addition, a Global Control Processor,890, is included which works in conjunction with the Control Processors in the Display Computing Appliances,810,820,830. The connected Appliance Data Network in this embodiment is comprised of a connection;2001,811,821,831,891; of each subsystem;2000,810,820,830,890respectively; to the Appliance Data Network,801. The Display Computing Appliance,2700, is the embodiment of subsystems810,820and830and is described in further detail inFIG. 27. The Network Layer Storage,2000, is described in further detail inFIG. 20. Note that the Global Control Processor, in this embodiment and in the embodiments illustrated inFIGS. 9 and 10, is not required to be present. Some systems do not require centralized coordination of activities and the Global Control Processor would have nothing to do so can be eliminated.

FIG. 9illustrates an embodiment of a system of Distributed Layer Storage, Data Layer edits, distributed mapping and server synchronization. This is similar toFIG. 4with the exception that logic and storage to selectively permit Data Layer edits have been added so that selectively enabled ones of the users may change data stored for a particular layer, and there is server synchronization. The server synchronization logic in the Network Layer Storage,2100, allows change data for a layer to be stored in multiple Network Layer Storage devices and locations and the server synchronization makes sure that once a change is made to Layer Data for a respective layer in one network storage device and location that the respective change to the respective Layer Data is synchronized and that an identical change is to respective Layer Data in all other locations that said respective layer exists. The connected Appliance Data Network in this embodiment is comprised of a connection;941,951,961,911,921,931,991; of each subsystem;940,950,960,910,920,930,990respectively; to the Appliance Data Network,901. The Display Computing Appliance,2700, is the embodiment of subsystems910,920and930and is described in further detail inFIG. 27. The Network Layer Storage,2100, is the embodiment of subsystems940,950and960and is described in further detail inFIG. 21. In the illustrated embodiment ofFIG. 9, the user display-computing appliance may selectively access a respective layer from any of the network storage devices and locations that contains that respective layer. The user display-computing appliance also contains Edit Logic to selectively allow the user to make and communicate changes to the Network Layer Storage.

FIG. 10illustrates an embodiment of a system of Distributed Layer Storage, Data Layer edits, distributed mapping and client synchronization. This is similar toFIG. 9with a difference being that the synchronization is contained in the user display computing appliance and not the Network Layer Storage. Thus, in the embodiment ofFIG. 10, it is the responsibility of each user display-computing appliance that when there is a change made at that respective user display-computing appliance to data for a respective layer in one network storage layer location that the user display-computing appliance synchronizes the change in the data for the respective layer so as to make the change in the data for the respective layer in all the network storage devices and locations so as to maintain that the respective data for the respective layer is identical in all layer storage devices. In this embodiment, the user display-computing appliance can selectively access the respective Data Layer from any one of the network storage devices/locations and the network storage layer device/location can be (or not be) co-located with each user display-computing appliance. The connected Appliance Data Network in this embodiment is comprised of a connection;1041,1051,1061,1011,1021,1031,1091; of each subsystem;1040,1050,1060,1010,1020,1030,1090respectively; to the Appliance Data Network,1001. The Display Computing Appliance,2800, is the embodiment of subsystems1010,1020and1030and is described in further detail inFIG. 28. The Network Layer Storage,2000, is the embodiment of subsystems1040,1050and1060and is described in further detail inFIG. 20.

FIG. 11illustrates an embodiment of the mapping storage logic with Edit Logic and layer partitioning logic for global mapping. This mapping storage logic is used when the Mapping Logic is distributed and therefore each user at each user display-computing appliance is capable of having their own information for their mapping. The Mapping Logic is responsive to the display logic, which determines when to update the display. The display logic selects a particular user and communicates that to user input,1111. The level input,1112, is responsive to the display logic which starts with level 0 to obtain the first Data Layer pointer,1113, and part,1114, for the first layer of the display. The Network Layer Storage is responsive to the Data Layer pointer and part. The display logic in turn is responsive to the Network Layer Storage and receives the first desired portion of Layer Data. The display logic then increments the level to 1 to obtain the second portion of Layer Data for combining with the first layer for the display. The display logic continues to increment the level and combining the resulting portion of Layer Data with previous portions of Layer Data until an invalid Data Layer pointer is obtained indicating that no more layers are to be included in the display. The ordering of the Data Layers for display, since a layer layered hierarchically for display on top of another one may be different in appearance than the layers ordered the other way. The mapping table,1110, provides a mapping (such as a pointer or data layer name) to each of the Data Layers that should be used and a description of the part of the Layer Data to use. For instance, one may want to only display the top part of the Data Layer so display mapping layers for the part would indicate that the Data Layer for only the top part be used for display. This Mapping Logic,1100, also includes an Edit Logic,1120, (such as a table with control logic) which is used to determine which layer should be used for editing for which user at each user display-computing appliance and when editing is to be allowed. The Edit Logic requires that it be indicated which user it is that the information is wanted for on the edit user input,1121, and it provides the edit level on edit level output,1122, which would then be used later in the Mapping Logic (or table) to find the particular pointer to the Data Layer which the changes would then be made to, and stored in. This storage Mapping Logic also includes an input to the Edit Logic (table),1120, to be able to change the edit level in the Edit Logic for a particular user responsive to the mapping control logic. The Mapping Logic (table),1110, also has inputs for the user,1115, level,1116, Data Layer,1117, and part,1118, to allow the mapping control logic to make changes to these mapping tables for the operation of the system. The Mapping Table Data Example,1130, shows a case where there are 3 users and 4 Data Layers. User A is displaying 2 different layers. User B is displaying 2 layers, 1 of which is shared with user A and B. User C is displaying 4 layers, 1 of which is shared with user A and B, another is shared only with B, another is shared only with A, and the last layer is only displayed by C. In addition, user B is displaying a different part of the layer than either A or C. The Edit Table Data Example,1140, shows a case, in conjunction with the Mapping Table Data Example, wherein user A is editing Data Layer2, user B is editing Data Layer3and user C is editing Data Layer4. User Input,1121, is a dual purpose input. It selects a row based upon the user using the system, and gets/sets data value in table based on control input. Inputs and outputs forFIG. 11are shown as coupled inFIGS. 18 and 20. The embodiment inFIG. 11is compatible for distributed or centralized use.

FIG. 12illustrates an embodiment of a system with Mapping Logic with layer partitioning for global mapping. This is similar toFIG. 11with the exception that it does not include the Edit Logic,1120, that is inFIG. 11. Table1210is similar to table1110. Signals1211,1212,1213,1214,1215,1216,1217and1218are similar to signals1111,1112,1113,1114,1115,1116,1117and1118respectively. Other than that, the operation of this is similar to that of the system ofFIG. 11. Example data for table1210is shown in1230. Inputs and outputs forFIG. 12are shown as coupled inFIG. 15. The embodiment inFIG. 12is compatible for distributed or centralized use.

FIG. 13shows the embodiment of a Mapping Logic for distributed mapping. This is similar toFIG. 12except that with distributed mapping, there is a separate one of the Mapping Logic included for each user that is included in the user display-computing appliance. However, there is only the one user information in the Mapping Logic or table for each user, and therefore, there is no user input in this mapping table to select an entry by user, since the Mapping Logic in the user appliance all applies to a particular respective user. Again, the Mapping Logic is responsive to the display logic, which determines when to update the display which increments the level to obtain the Data Layer pointers and part information to find the associated part of the associated Data Layer in the correct order to create a combined display. The mapping control logic has input to the Mapping Logic (table) in order to modify the values of the table during operation. Table1310is similar to table1210. Signals1312,1313,1314,1316,1317and1318are similar to signals1212,1213,1214,1216,1217and1218respectively. Other than that, the operation of this is similar to that of the system ofFIG. 12. Example data for table1310is shown for three users in1330,1331and1332. Inputs and outputs forFIG. 13are shown as coupled inFIG. 24.

FIG. 14shows the embodiment of mapping layer logic with edits and for distributed mapping. Table1410operates exactly as Table1310inFIG. 13. Edit Table1420is similar to operation as Edit Level1120except that with distributed mapping, there is a separate one of the Edit Level included for each user that is included in the user display-computing appliance. However, there is only the one user information in the Edit Level for each user, and therefore, there is no user input to select an entry by user, since the Edit Level in the user appliance all applies to a particular respective user. Signals1122and1123are similar to signals1422and1423respectively. The edit level provides a mapping for a respective user at a respective user display-computing appliance, which mapping determines which Data Layer is a respective assigned level that the particular user should be using for editing. It should be noted that there is also an ability to put a value in the edit level that would indicate that the user is not able to edit any level. As one example, for no edit permission, the edit level could be set to an invalid level value, and if that value was accessed by the edit Mapping Logic (e.g., mapping table), it would provide an invalid Data Layer pointer, and therefore, no Edit Data Layer would be assigned, and thus, nothing could be edited for that respective user. Example data for table1410is shown for three users in1430,1431and1432. Example data for edit level1420is shown for three users in1440,1441and1442. Inputs and outputs forFIG. 14are shown as coupled inFIG. 27 or 28.

FIG. 15illustrates an embodiment of Network Layer Storage with mapping. This embodiment comprises layer storage display logic, which selectively combines for use in display respective selected ones of the layers responsive to the Mapping Logic. Layer data is obtained from layer storage,1600, (further described inFIG. 16) on data path1601which is coupled via1531to signal3901of the display logic,3900, (further described inFIG. 39). The desired layer data is selected by signal layer,1602, and signal part,1603, which specify which data layer and what part of the data layer to retrieve in layer storage1600. Signal1213is coupled to1602via1532and signal1214is coupled to1603via1533from the mapping logic1200(further described inFIG. 12). The mapping logic,1200, is setup through the signals1215,1216,1217, and1218sent from mapping control,1520. The mapping control is responsive to messages sent through the connected Appliance Data Network,1501, translated by the network interface,1510, and coupled via1521to the mapping control,1520, from a plurality of Control Processors2550(seeFIG. 25) to allow the Mapping Logic,1200, to be modified. The Control Processors send messages based on role of the user and the specific embodiment. The modifications of the Mapping Logic,1200, control which Data Layers are combined for display and their order for each user. The display logic,3900, (further described inFIG. 39) generates the signal user,3902and signal level,3903, which is coupled to the mapping logic,1200, to signals1211and1212via1541and1542respectively. The layer data retrieved,3901, previously described herein, is combined into display presentation data for the user on signal3904. Signal3904is coupled to network interface,1510, that sends the display presentation data via the appliance data network,1501, to the appropriate display computing appliance. The display logic,3900, obtains layer data combination parameters via signal3905that control how the layer data is combined. Signal3905is coupled to the network interface,1510, which receives the layer data combination parameters from the appliance data network,1501, from a plurality of Control Processors2550. Mapping Control,1520, sets up all of the values for the edit table and the mapping table, and what layers are being used by which users, and what layers each user can be editing. User inputs, display data and program data are coupled via network interface1510.

FIG. 16illustrates an embodiment of a system showing layer storage, as illustrated and utilized inFIG. 15and other embodiments. In this embodiment, there are multiple Data Layers,1610,1620,1630, storing Layer Data and there is Data Layer Access,1640, to allow the system to access a particular Data Layer. The Data Layers,1610,1620,1630are each illustrated in block7100as shown inFIG. 71. Layer Storage1600does not use the Data In,7101, input of Data Layer7100in elements1610,1620,1630. The system is not limited to three Data Layers, there may be any number of Data Layers at any given time. The Data Layer Access provides a mapping (e.g., a pointer or data layer name) for a respective particular Data Layer responsive to Mapping Logic,1200inFIG. 15in one embodiment. The Data Layer ID,1602, input (responsive to the layer output,1213,FIG. 12, in said embodiment) identifies a particular Data Layer;1610,1620, or1630. The part,1603, is coupled via1533to the part,1214, that defines a portion of said Layer Data which is communicated via connection1613,1623or1633respectively. The Data Layer Access,1640, is coupled with signals1602and1603. Signal7103of1610is coupled to1613. Signal7103of1620is coupled to1623. Signal7103of1630is coupled to1633. Signal7102of1610is coupled to1612. Signal7102of1620is coupled to1622. Signal7102of1630is coupled to1632. The signal1603is sent to one of the signals1613,1623or1633depending on the signal1602. The corresponding data output,1612,1622or1632, respectively provides a portion of the requested Layer Data which is conveyed to the Layer Part Data Out,1601, by means of the Data Layer Access,1640. This provides the means to access a particular Data Layer and provides for output of the respective Layer Part Data Out,1601, of Layer Data.

FIG. 17illustrates an embodiment of a layer storage with Edit Logic for edits. This is used in various embodiments. It is similar toFIG. 16with all the same functions ofFIG. 16, plus with the addition of the ability to accept change data for a portion of a for a specific Data Layer. In this embodiment, there are multiple Data Layers,1710,1720,1730, (similar to1610,1620,1630inFIG. 16) storing Layer Data and there is Data Layer Access,1740, (similar to1640inFIG. 16) to allow the system to access a particular Data Layer. The Data Layers,1710,1720,1730are each an instance of subsystem7100as illustrated inFIG. 71. The system is not limited to three Data Layers, there may be any number of Data Layers at any given time. The Data Layer ID input,1702, and Part input,1703, are responsive to the Mapping Logic,1100, Mapping Control,1820, and Layer Edit Logic,4000, in one embodiment described inFIG. 18. The Layer Part Data Out,1701, provides the current state of the portion of Layer Data described by inputs1702and1703. The Layer Part Data In,1704, is change data responsive to the Layer Edit Logic's,4000, output,4002, (in said embodiment) is used to replace the portion of Layer Data. Layer Part Data In,1704, is result of processing by the Layer Edit Logic,4000, (in said embodiment) to modify the specified contents of the portion of Layer Data with changes responsive to user input. Data Layers may be created at any time by the system by specifying a Data Layer ID that does not exist in Layer Storage1700and providing Layer Part Data In,1704. A Data Layer,1710,1720,1730, will be eliminated when a Data Layer has contains no Layer Data. If the Data Layer ID,1702, identifies a non-existent Data Layer then Layer Part Data Out,1701, contains no data. Data Layer Access,1740, uses the data layer id,1702, to select on of the data layer storage subsystems,1710,1720or1730. The layer part data in,1704, and part,1703are coupled by Data Layer Access,1740, to the aforementioned selected subsystem,1710,1720or1730and coupled to the corresponding data in and part inputs,7101in1710via1711and7103in1710via1713,7101in1720via1721and7103in1720via1723, or7101in1730via1731and7103in1730via1733. The selected subsystem1710,1720or1730modifies a portion of the layer data stored within based on the change data on data in,7101and the part,7103, specifying the portion.

FIG. 18illustrates an alternate embodiment of the present invention, showing a system that is similar toFIG. 15but with the addition of Layer Edit Logic,4000, which provides for the network interface,1810, to receive information defining permission to allow a respective selected Data Layer to be edited and provides the information on which Data Layer to edit and communicates change data from the respective user at the respective user display-computer appliance for storage in the respective Data Layer in the layer storage with edits1700. The change data is received via the appliance data network,1801, from a display computing appliance (seeFIG. 5 or 6) which is coupled to the network interface1810that sends the change data via1811which is coupled to the Layer Edit Logic4000, via4001. The Layer Edit Logic is responsive to the change data provided on input4001. This initiates a series of events. First, the user information is stripped from the change data request and output on4004and connected to the Mapping Logic,1100, via1861to inputs1121and1111. Inputs1121and1111are shared and each can independently trigger a request for the edit level. The Mapping Logic outputs the edit level on1122and is coupled to the level input,1112, again a shared input via1862. Now that the level and part are provided on inputs1111and1112, the Mapping Logic outputs Layer Data pointer and part on outputs1113and1114. These are connected to inputs1702and1703respectively. The layer storage is responsive to these inputs and outputs the edit portion of Layer Data on output1701. This is information is conveyed via1831to input4003of the Layer Edit Logic,4000. The Layer Edit Logic combines the edit portion of Layer Data with the change data and outputs the modified portion of Layer Data on4002which is coupled to1704via1834. Finally, the layer storage with edits replaces the edit portion of Layer Data with the modified portion of Layer Data. The rest of the operation is similar to1500inFIG. 15. The signals1701,1702,1703,1111,1112,1113,1114,1115,1116,1117,1118,1801,1821,1831,1832,1833,1841and1842inFIG. 18are similar in operation to1601,1602,1603,1211,1212,1213,1214,1215,1216,1217,1218,1501,1521,1531,1532,1533,1541and1542inFIG. 15respectively and are described in detail inFIG. 15. In addition, subsystem1700inFIG. 18is similar in operation to subsystem1600inFIG. 15for retrieving layer data to generate the display and is described in detail inFIG. 15. Mapping Control,1820, sets up all of the values for the edit table and the mapping table in Mapping Logic,1100, and what layers are being used by which users, and what layers each user can be editing. User input, display data and program data are coupled via network interface1810.

FIG. 19illustrates an alternate embodiment of the present invention showing Distributed Layer Storage. As illustrated inFIG. 19, layer storage,1600, is illustrated with Data Layer network interface,1910, that allows the layer storage to communicate with the user display computing appliance as shown in two embodiments inFIGS. 3 and 4. These embodiments are similar to the embodiments shown inFIGS. 1 and 2respectively except that the Display Logic,3900; Mapping Logic,1300; and Mapping Control,2420, is now located in the Display Computing Appliance,2400. Layer Part Data Out,1601is coupled via1931to the layer data network interface,1910, and then to network1901that is coupled (inFIG. 3 or 4 and 24) to the Display Logic,3900in one of the specific Display Computing Appliances;310,320,330or410,420,430. The display computing appliance receiving the layer data is the same as the Display Computing Appliance that sends to the data Layer,1313, and Part,1314. Signals1313and1314are coupled through the appliance data network,1901, (seeFIG. 3 or 4 and 24) and coupled via the layer data network interface,1910, to1602via1932and1603via1933. The effective operation illustrated inFIG. 19is the same inFIG. 15except that the display logic, mapping logic and mapping control are moved from the the layer storage and into the display computing appliance.

FIG. 20illustrates an alternate embodiment showing a system having Distributed Layer Storage structure with edit logic. This is similar to that shown inFIG. 19. However, this illustrated embodiment includes edit logic within Layer Storage With Edits,1700, to selectively allow respective Data Layer in the layer storage to be modified for editing, permitting a respective user at a respective user display-computing appliance to make edits. The change data is output on4002in display computing appliance2700or4102in display computing appliance2800and coupled to the appliance data network,2001(see details inFIGS. 8 and 27orFIGS. 10 and 28). The change data is coupled from the layer data network interface2010via2034to1704. Similar to subsystem1900, the signals1702and1703are similarly coupled from the display computing appliance via coupling2032and2033respectively to provide the layer storage with edits,1700, the data layer and part to modify the storage based on the change data. The signals2031,2032,2033and2001are similar in operation to signals1931,1932,1933and1901for retrieving layer data and described in detail inFIG. 19. The effective operation illustrated inFIG. 20is the same inFIG. 18except that the display logic, mapping logic, layer edit logic and mapping control are moved from the the layer storage and into the display computing appliance.

FIG. 21shows another embodiment of the present invention, showing a Distributed Layer Storage structure with edit logic and synchronization logic. This is similar toFIG. 20, with the addition of synchronization logic so that when edits are made to so as to change the Layer Data as stored in a particular layer of the Data Layers, then those changes are selectively communicated to other distributed Network Layer Storage elements for local storage of the respective change in the respective particular layer of the respective local Data Layers, so as to maintain the Layer Data in the respective Data Layer in all the places that that Data Layer is stored. Since in this embodiment, a respective particular layer can be stored in multiple elements, the system needs to have all those elements synchronized and updated so as to contain the same data, and the synchronization logic maintains that data integrity. The signals2131,2132,2133,2134and2101are similar in operation to signals2031,2032,2033,2034and2001for retrieving layer data and editing layer data and described in detail inFIG. 20. The only difference is that the change data,2134, is coupled through the layer edit logic,4100, to4102and then coupled by2194to layer data in,1704. The synchronization logic in4100outputs change data information on4107and is coupled by2137to the layer data network interface,2110, to send the change data information on the appliance data network,2101, to the other layer storage locations that may have duplicate data layers so the layer data remains identical. When2134receives change data from synchronization logic in another layer storage, the synchronization logic,4100, separates out the layer to sync, on4105and part to sync,4106. The current layer data4103is responsive to4105and4106by coupling4105to1702via2192and coupling4106to1703via2193.1701is responsive to1702and1703.1701is coupled to4103via2191. Finally,4102is responsive to both2134and4103and is coupled to1704via2194which replaces the change data in the layer storage,1700. The effective operation illustrated inFIG. 21is the same inFIG. 22except that the display logic, mapping logic and mapping control are moved from the the layer storage and into the display computing appliance but the layer edit logic with synchronization remains in the layer storage.

The synchronization logic can operate in different synchronization modes. There are 3 illustrated modes: one is a change sync mode, which allows all the changes made to the respective Layer Data in a respective Data Layer responsive to a respective user's edits, to be communicated to all layer storage locations that are storing the same layer regardless of user. In the full sync layer mode, all Layer Data in a respective Data Layer is communicated to all layer storage locations that are storing the same layer regardless of user. This mode is usually used only infrequently and at one location. There are times when the synchronization fails due to network failures, some layer storage not being available during operation and other failures. In this case, the best copy of the layer storage needs to be communicated with the layer storage that is not synchronized with the other layer storage, i.e., the layer storage is not an exact duplicate and the synchronization logic has lost track of what to do. This mode restores the synchronization and then the mode can be returned to the normal state of Change Sync Mode. The Full Sync Mode requires significant bandwidth to communicate all Layer Data to all the duplicate copies and is generally impractical to always perform so is only done when necessary. The last mode is the No Sync Mode which turns off the synchronization for a particular layer storage location. This could be used for a graceful way to remove layer storage from the system or to make changes offline and then implement them quickly by making changes to this layer storage then changing the mode to Full Sync Mode, and finally back to Change Sync Mode. Each layer storage location and Data Layer stored in said location can be set to one of these modes independently. However usually all are set to Change Sync Mode.

FIG. 22illustrates an alternate embodiment of the present invention, showing a system similar toFIG. 18, but with the addition of synchronization that involves signals which, similar toFIG. 21, maintains the data integrity across the distributed Network Layer Storage elements as inFIG. 18. Signals2211,2231,2232,2233,2241,2242,2222,2261,2262,2201inFIG. 22are similar to1811,1831,1832,1833,1835,1836,1822,1861,1862,1801respectively inFIG. 18and are discussed in detail there. Signals2291,2292,2293,2294,2234and2237inFIG. 22are similar to signals2191,2192,2193,2194,2134and2137respectively inFIG. 21.

FIG. 23illustrates an embodiment of the present invention wherein simultaneous document collaboration, video, voice and audio communications occur in the same system. Illustrated inFIG. 23are two multimedia collaboration systems2300and2350. Contained within the multimedia collaboration systems is a display computing appliance,2301and2351respectively. The display computing appliances communicate via connection2310. Such communication is detailed elsewhere in this document as described inFIGS. 1 through 10and elsewhere herein. A display,2304and2354respectively, is contained in each multimedia collaboration system. The output of the display computing appliance2301is coupled to the display2304via signal2305. Likewise, the output of the display computing appliance2351is coupled to the display2354via signal2355. The displays provide a visual output for the user of each display computing appliance. A speaker,2302and2352respectively, a microphone,2303and2353respectively, and video camera,2307and2357respectively, are contained in each of the collaboration systems. Microphone,2303, is cross coupled via2306with the speaker,2352. Likewise, microphone,2353, is cross coupled via2356with the speaker,2302. This system of speakers and microphones allows the users of the collaboration systems to carry on an voice conversation while using the display computing appliances2301and2351. Video camera2307is coupled to the display2354via signal2308. Video camera2357is coupled to the display2304via signal2358. The display,2304, will simultaneously provide a video output from signals2305and2358. Likewise, the display,2354, will simultaneously provide a video output from signals2355and2308. The video along with the voice provides for a simultaneous video conferencing ability along with the display computing appliance's document collaboration.

This illustration shows both voice and video used simultaneously with the display computing appliances. However, only the voice or the video could be used with the display computing appliance. The connections2306,2356,2308,2358and2310can be separate physical connections or can be combined in a single, fewer, or networking type physical network connections. The signals2306,2356,2308and2358can be analog or digital in nature. Other multimedia collaboration systems can be added with voice and video capabilities. In cases of 3 or more multimedia collaboration systems, the microphone and camera signals, similar to2306and2308, can be optionally coupled to the display and speakers of one or a plurality of other multimedia collaboration systems providing video conferencing capabilities for more the one user. In addition, the source for the audio could be provided by another source other than a microphone such as a digital audio file, video file, tape recorder, MIDI file or other audio source. Likewise, the video does not need to be supplied by a video camera and could be provided by a digital video file, video tape, or other video source.

FIG. 24illustrates another embodiment of the present invention, showing a system comprising a display computing appliance with Mapping Logic, an appliance network interface, user display logic, Control Processor, and a user display, (also referred to as a user appliance). This user appliance accesses the network storage through the appliance network interface,2410, which is coupled with the appliance data network,2401, that is coupled to network layer storage,1900. (SeeFIGS. 3, 4 and 19) The Mapping Control,2420, is coupled responsive to the Control Processor,2450, via2421, which is used to control the operation of the user appliance by defining the Data Layers and parts used to make up the display to the user. The Mapping Logic,1300, is responsive to and modified by mapping control,2420, coupled via level1316, data layer1317and part1318. The mapping control sets the level and can obtain the value of the data layer and part which would then be reported back to the control processor. Also the mapping control can set the level and set the value of the data layer and part obtained from the control processor. The display logic,3900, is responsive to the Control Processor,2450, for setup. The portion of Layer Data is responsive to a the Mapping Logic,1300, which in turn is responsive to the display logic,3900, via level,3903, coupled to1312. The Mapping Logic,1300, is responsive to level on1312, and outputs a Layer Data pointer,1313, and part,1314, which is coupled to layer storage via the appliance network interface,2410, and the connected Appliance Data Network connector,2401. The display logic is also responsive to portion of Layer Data from layer storage coupled by the connected Appliance Data Network connector,2401, appliance network interface,2410, and data path3901. In this manner the display logic changes the level,3903, and obtains layer data on3901which is used to build a display presentation for output on3904. The display presentation,3904, is coupled via2431to the user display,2430. The user control input,2460, provides input via mouse, keyboard, touchscreen, voice recognition, video recognition and other devices to provide user control of the operation of the system. The user input is coupled to the control processor,2450, via2452.

FIG. 25illustrates another embodiment of the present invention, showing a system comprising a display computing appliance with display,2530, Control Processor,2550, and an interface,2510, (also referred to as a basic user appliance) to access network storage, Mapping Logic, display logic and mapping control via the connected Appliance Data Network. This particular embodiment requires that the network storage also provide logic to provide an output defining a combined display of various Data Layers for a local display on a respective basic user appliance for a particular respective user. The User display receives a display presentation from the display network interface,2510, coupled by2531. The display network interface receives the display presentation from the coupled appliance network2501which is coupled to network layer storage,1500(seeFIGS. 1, 2 and 15.) The user control input,2560, provides input via mouse, keyboard, touchscreen, voice recognition, video recognition and other devices to provide user control of the operation of the system. The user input is coupled to the control processor,2550, via2552. The control processor is coupled with other control processors in the system (seeFIGS. 1 and 2) via signal2551coupled with the display network interface,2510, and the appliance data network,2501. The control processor is also coupled with network layer storage,1500, in the system (seeFIGS. 1, 2 and 15) via signal2551coupled with the display network interface,2510, and the appliance data network,2501.

FIG. 26illustrates an alternate embodiment of the present invention, showing a display-computing appliance with display,2630, Control Processor,2650, and input device,2640. This is similar toFIG. 25with the addition of an user input device at the user display-computing appliance which user input device is used to communicate with the network storage to provide communication of changes to respective Layer Data in a respective Data Layer in the network storage. The signals2652,2651,2631and2601are similar to2552,2551,2531and2501respectively inFIG. 25and are described in detail therein.FIG. 26uses network layer storage,1800or2200(seeFIG. 18 or 22instead of15). display computing appliance,2600, is included in embodiments illustrated inFIGS. 5, 6 and 7instead ofFIGS. 1 and 2. There is an additional coupling from the user control and annotation input device,2640, via2641to the display and input network interface,2610. User supplied changes for the document are sent on this path and then are coupled via the appliance data network,2601, to the layer edit logic in network layer storage,1800or2200(seeFIGS. 5, 6, 7, 18 and 22). The user input device provides an interface between the user and the system to provide data of various types for changes to Layer Data. Textual data can be provided with a physical keyboard, virtual keyboard on a touch screen, virtual keyboard using a mouse, mouse gestures, camera gesture recognition, voice recognition, import from a file or network data transfer and other means. Images are provided by mouse, touchscreen input, stylus and digitizer input, camera, and import of images from a file or network data transfer in the form bitmap images such as JPEG, TIFF, PNG, GIF and many other formats, vector drawings, 2D and 3D models and other means. Video is provided by camera, and import of video from a file or network data transfer in the form of a sequence of images or computer simulations such as AVI, MPG, FLV, M1V, M4P, MOV, MPEG, OGG, VOB, WMV, DirectX, OpenGL, many other formats and other means. Audio is provided by microphone, and import of video from a file, CD, DVD, iPod, MP3 player, cell phone or network data transfer in the forms such as WAV, WMA, AIFF, the audio portion of aforementioned video, many other formats and other means.

FIG. 27illustrates another embodiment of the present inventions, showing a display-computing appliance with Mapping Logic,1400, mapping control,2720, user display logic,3900, Control Processor,2750, user display,2730, user input device,2740, and Layer Edit Logic,4000, for a respective user. This embodiment is similar toFIG. 24with the addition of a user input device and Layer Edit Logic to allow a user of the display-computing appliance to make changes to the Layer Data that is communicated to the layer storage. This embodiment also contains Mapping Logic utilized for that respective user and the display logic for that respective user, and the user display device to provide a presentation responsive to the display logic. The synchronization for the system is performed in the network layer storage with edits and synchronization,2100. SeeFIGS. 9 and 21for the rest of the system. The user control and annotation input device,2740, is coupled with the control processor,2750, via2752to all the user to control various aspects of the system. The user control and annotation input device is also coupled via2741to the layer edit logic,4000, to provide user changes to the document presented on the user display,2730. The layer edit logic uses the current layer data, on4003, which is coupled via2791to the appliance network interface with edits. The current layer data is obtained when accessing layer data for the display presentation, detailed later. The layer edit logic combines the user changes to the document with the current layer data and outputs the change data on4002and sends it via coupling2715to the appliance network interface with edits,2710. The appliance network interface with edits in turn sends the change data to the network layer storage with edits and synchronization,2100(seeFIGS. 9 and 21) to change the layer data that is stored. The control processor is responsive to other control processors in the system and the global control processor,990, via the coupling of the control processor to the appliance network interface with edits,2710, via2751and the appliance data network,2701. The mapping control2720is responsive to setups from the control processor,2750, via2721. The display logic3900is responsive to setups from the control processor via2753. The mapping logic,1400, is responsive to the mapping control,2720, via1416,1417,1418,1423as described inFIG. 14. The mapping logic uses the output from1422coupled to the level,1412, to look up the data layer,1413, and part,1414, when user changes to the document are being processed in layer edit logic,4000, so the current layer data is available on2791for editing. The display logic,3900, also sets the level,3903, for creating the display presentation. The mapping logic uses this output from3903coupled to the level,1412, to look up the data layer,1413, and part,1414. The data layer is coupled by2713to the appliance network interface with edits,2710, to the appliance data network,2701, which is coupled to1702in the network layer storage,1800(seeFIGS. 9 and 18). Likewise, part is coupled by2714to the appliance network interface with edits,2710, to the appliance data network,2701, which is coupled to1703in the network layer storage,1800. If there are multiple network layer storage instances such as940,950,960shown inFIG. 9, then the control processor and appliance network interface with edits determine which of the instances to communicate with on the appliance data network. The requested layer data responsive to data layer and part is output on1701in the selected network layer storage,2100. The layer data is coupled to the appliance data network,2701and the appliance network with edits,2710, couples the layer data to3901via2711for the display or4003via2791for edits. The display logic combines the layer data for each level and outputs the display presentation on3904. The display presentation is coupled to the user display,2730, via2731.

FIG. 28illustrates an alternate embodiment of a display-computing appliance with Mapping Logic,1400, mapping control,2820, user display logic,3900, user display,2830, user Layer Edit Logic with synchronization,4100. This embodiment is similar toFIG. 27, with the addition of synchronization as inFIG. 22. The signals2892,2893and2894inFIG. 28are similar to signals2292,2293and2294are used for synchronization as described inFIG. 22. The signals2841,2852,2851,2853,2862,2831,2811,2813,2814,1416,1417,1418,1423and2801inFIG. 28are similar signals2741,2752,2751,2753,2762,2731,2711,2713,2714,1416,1417,1418,1423and2701respectively inFIG. 27. This embodiment is used when synchronization is not part of the network data storage, but is instead distributed and contained in every user appliance. It is then the responsibility of each user appliance to communicate changes to all other locations (e.g., other user appliances) of Network Layer Storage to make changes to a respective particular layer if that respective particular layer is it stored in multiple locations.

FIG. 29illustrates an example of a setup of the Mapping Logic as embodied in Mapping Logic,1400, as illustrated inFIG. 14for a music team where the Mapping Logic is distributed in each display computing appliance. This example has two display computing appliances, leader1and leader2, in the leader role and two display computing appliances, member3and member4, in the member role. All the Control Processors on the team of display computing appliances are responsive to a Control Processor in display computing appliance that is assigned a leader role. The Control Processors add the Data Layer of the leader display computing appliance that is being edited to their mapping control to be displayed. Thus a leader always has their layer displayed on all appliances in the team. This example has two leaders so both are displayed. Leader1is displaying a common layer, optional2layer, optional1layer, leader2layer and leader1layer. The leader1layer is being edited by leader1because the edit level is set to 4 and the leader1layer is level4in the mapping table. The user of leader1does not see the optional3layer, member3layer or member4layer because they are not in the mapping table. Leader2is displaying a common layer, leader1layer and leader2layer. The leader2layer is being edited by leader2because the edit level is set to 2 and the leader2layer is level2in the mapping table. The user of leader2does not see the optional1, optional2, optional3layer, member3layer or member4layer because they are not in the mapping table. Member3is displaying a common layer, optional3layer, leader2layer, leader1layer and member3layer. The member3layer is being edited by member3because the edit level is set to 4 and the member3layer is level4in the mapping table. The user of member3does not see the optional1layer, optional2layer, or member4layer because they are not in the mapping table. Member4is displaying a common layer, leader2layer, leader1layer and member4layer. The member4layer is being edited by member4because the edit level is set to 3 and the member4layer is level3in the mapping table. The user of member4does not see the optional1layer, optional2layer, optional3layer, or member3layer because they are not in the mapping table. This example show the flexibility of the display options for each display computing appliance in the system. The roles of each display computing appliance define the operations of the Control Processors which can quickly quickly change the Mapping Logic to provide dynamic display options for the users. The example inFIGS. 29, 32 and 37can also be applied to the embodiment of the Mapping Logic1100,1200and1300. The setup for Mapping Logic,1300, is identical to Mapping Logic,1400, except that the edit level is not included. Mapping Logic1100and1200are similar to1400and1300respectively except that all the mapping tables are combined into one table that contains another column for the user. Similarly, the edit levels are combined into one table with another column for the user. Examples of a setup are provided with each of the respective Mapping Logic elements,1100,1200,1300and1400. The synchronize mode for leader data layers is normally the change sync mode. A leader can initiate a synchronize mode of full sync mode for their respective leader layer.

FIG. 30illustrates the music team data flow for a the first embodiment. Some team roles allow a portion of Layer Data to be sent from its original Data Layer to a different Data Layer. This flow diagram illustrates the steps for a system as illustrated inFIG. 9. This embodiment describes the receiving appliance making the determination of which Data Layer to put the portion of Layer Data.

FIG. 31illustrates the music team data flow in a second embodiment of that workflow. This flow diagram illustrates the steps for a system as illustrated inFIG. 9. This embodiment describes the sending appliance making the determination of which Data Layer to put the portion of Layer Data.

FIG. 32illustrates an example of a setup of the Mapping Logic as embodied in Mapping Logic,1400, as illustrated inFIG. 14for an education team where the Mapping Logic is distributed in each display computing appliance. This example has one display computing appliance, teacher, in the teacher role and a plurality of display computing appliances, student1, student2, student3, . . . , in the student role. All the Control Processors on the team of display computing appliances are responsive to a Control Processor in display computing appliance that is assigned a teacher role. The Control Processors add the Data Layer of the teacher that is being edited to their mapping control to be displayed. Thus a teacher always has their layer displayed on all appliances in the team. This example has one teacher but multiple teachers are allowed, however all teachers share the same Teacher Data Layer. This example illustrates the entire class viewing the same part, lesson6but this does not have to be the case. However, the teacher's Control Processor may communicate with the student Control Processors and prohibit the students from changing the part being viewed, or alternatively allow one or all the students change the part, e.g., move on to lesson7. A student always shows 3 Data Layers, the Common Data Layer, the Teacher Data Layer and the student's own Data Layer as illustrated in the example setup “Student Y Mapping Logic”. The student always edits their own Data Layer but the Teacher Data Layer is always included on the display. A teacher has three difference modes for display: teacher mode, multi-mode and teacher-student mode. The teacher mode displays 2 Data Layers, the Teacher Data Layer and Common Data Layer as illustrated in the example setup “Teacher Mapping Logic in Teacher Mode”. The teacher edits the Teacher Data Layer (multiple teacher appliances would edit the same Data Layer) and since the students are viewing this Data Layer they receive the edits as well but can't modify them. The multi-mode displays the Teacher Data Layer and all the Student Data Layers as shown inFIG. 60. The teacher display logic operates in a multi-view mode. The common layer, level=0, is reduced in size and duplicated combined for the display multiple times. Then every other layer is reduced in size and placed over one of the copies of the Common Data Layer. The resulting display provides an overview of everyone in the class. The example of the Mapping Logic setup is shown in “Teacher Mapping Logic in Multi-Mode”. Alternative embodiments would allow the Mapping Logic to include the Common Data Layer with different parts to correspond to the parts that are currently being used by each student. The student layer would then have a part corresponding to the current student's part. This is accomplished with communication between the Control Processors. The teacher-student mode is identical to the mapping for a specific student. This allows the teacher to modify that Student's Data Layer and work one-on-one with the student to correct their mistakes or give private comments as shown in the section “Teacher Mapping Logic in Student Mode for Student X”. Since the teacher is modifying the student's layer the student can make changes to the teacher's comments. An alternative embodiment would create another Data Layer for each student for the teacher to edit and visible only on the respective student's appliance. This would allow the teacher to make private edits on for a student but not allow the student to edit them. The synchronize mode for teacher layer and all student layers is normally the change sync mode. A teacher can initiate a synchronize mode of full sync mode for the teacher layer. A teacher can initiate a synchronize mode of full sync for one or a plurality of student layers.

FIG. 33illustrates the team data flow for an education team, for a student, for a first embodiment. This flow diagram illustrates the steps for a system as illustrated inFIG. 9. This embodiment describes the receiving teacher appliance of student portion of Layer Data making the determination of which Data Layer to put the portion of Layer Data.

FIG. 34illustrates an alternative data flow for an education team, for the student. This flow diagram illustrates the steps for a system as illustrated inFIG. 9. This embodiment describes the sending student appliance of student portion of Layer Data making the determination of which Data Layer to put the portion of Layer Data.

FIG. 35illustrates the team data flow for an education team, for a teacher. This flow diagram illustrates the steps for a system as illustrated inFIG. 9. This embodiment describes the receiving appliance of teacher portion of Layer Data making the determination of which Data Layer to put the portion of Layer Data.

FIG. 36illustrates the education team data flow for student layer edits made by the teacher. This flow diagram illustrates the steps for a system as illustrated inFIG. 9. This embodiment describes the sending teacher appliance of teacher portion of Layer Data making the determination of which Data Layer to put the portion of Layer Data.

FIG. 37illustrates an example of a setup of the Mapping Logic as embodied in Mapping Logic,1400, as illustrated inFIG. 14for an ad hoc team where the Mapping Logic is distributed in each display computing appliance. This example has a plurality of display computing appliances, member1, member2, member3, . . . , in the ad hoc role. The ad hoc role does not put many limits on each appliance. Each appliance is allowed to select whatever Data Layers they choose to display and what ever part to choose. In this example, member1is displaying page7of the Common Data Layer, notes1Data Layer and member1Data Layer. Member1is editing their own Data Layer, member1. Member2is displaying page8of the Common Data Layer, member1Data Layer and member2Data Layer. Member2is editing member1's Data Layer, not their own. Note that member2is editing a portion of the Data Layers that neither member1or3are currently viewing, however, if member1or3changes their part to page8they would obtain the changes being made by member2. Member3is displaying page7of the Common Data Layer and member1Data Layer. It is not editing any layer and just monitoring member1's activities. The Control Processor of each appliance could change the Mapping Logic responsive to user input. Alternative embodiments would allow the Control Processors to communicate and prohibit some settings, e.g., an appliance could prohibit viewing or editing of their Data Layer.

FIG. 38illustrates data flow of user-to-user edits in an Ad Hoc team. The flow chart illustrates to steps that the user and Display Computing Appliances follow to share edits. Initially, the user on the sending appliances selects a portion of the layer data in a data layer. They the user selects appliances that the should receive the selected layer data. The layer data is sent to those appliances. The receiving appliances do not store the received layer data in the same data layer. They store a copy of the layer data in the data layer that the receiving appliances has marked for edit. The sending and receiving appliances thus copy layer data from one data layer to a different data layer. Each appliance then has an unique combination of layer data from other appliances and the layer data generated on their own respective appliance. Once copied into the edit data layer of the appliance, each appliance can selectively send it to another appliance, remove or modify the layer data.

FIG. 39illustrates another embodiment of display logic. This display logic has Layer Data Combined Storage,3920, for combined Layer Data, a Display Logic Controller,3930, and a logic means, Layer Data Combine,3910, to combine layer data from Data Layers for multiple Data Layers, and a connection,3904, to send the result to the display for presentation. The Display Logic Controller,3930, receives a display setup,3905, from the Control Processor, e.g., the user to display. It then outputs the user on3902and successive levels,3903, starting with 0 to the Mapping Logic. The layer storage is responsive to the Mapping Logic and provides a portion of layer data,3901. The Display Logic Controller senses when no portion of layer data is available to stop the process and indicate via connection,3931, to the Layer Data Combined Storage,3920, to output the result on display out,3904. When a valid portion of layer data is present on3901, the Display Logic Controller,3930, indicates via3932to the Layer Data Combine,3910, to combine in a specified way the portion of layer data with the current Layer Data Combined Storage,3920, via3912. The Layer Data Combine,3910, outputs the combination result on3911and the Layer Data Combined Storage,3920, is instructed by the Display Logic Controller,3930, via3931to store this combined result.

FIG. 40illustrates an embodiment of the Layer Edit Logic,4000, which is responsive to Data Layer change information,4001, that is obtained from user input. The information separation,4010, is responsive to the Data Layer change information and provides two outputs: user to be modified,4004, and modifications,4011. The Mapping Logic is responsive to the user to be modified and the Layer Storage is responsive to the Mapping Logic which provides the Current Data Layer coupled via4003. The Data Layer modifier,4020, is responsive to both the Current Data Layer and modifications and applies the modifications to the Current Data Layer and outputs the modified Data Layer,4002. The modified Data Layer is then coupled to the the Layer Storage where it replaces the current layer data.

FIG. 41illustrates an embodiment of Layer Edit Logic with synchronization,4100. This is similar to the embodiment,4000, inFIG. 40but with the addition of synchronization of multiple copies of Data Layers. The elements4101,4160,4161,4120,4103,4102are functionally equivalent to4001,4010,4011,4020,4003,4002respectively. The “send changes to other Layer Storage locations”,4140, otherwise known as “send changes” is responsive to the user modifications,4161. The layer changed,4108, and part changed,4109. Signals4108and4109are active when a display computing appliance makes a change in the layer storage.4108and4109are coupled to the Mapping Logic which is responsive to the user to be modified,4104. Send changes combines these inputs and sends it to other Layer Storage locations that have a duplicate of the layer changed via coupled by Data Layer Sync Out,4107, which is connected Appliance Data Network. Data Layer Sync Out is received from the connected Appliance Data Network from other instances of Layer Edit Logic,4100on Data Layer Sync In,4101. The user information separation,4160, provides three outputs, layer to sync,4105, part to sync,4106, and user modifications,4151.4105and4106are only output when a synchronization change occurs, when initiated by a user the user to be modified,4104is active. Thus the operation varies depending on the source of the change. The synchronization process bypasses the Mapping Logic because it is dependent only on the Layer Storage. The layer to sync and part to sync are sent directly to the Layer Storage which is responsive to those signals and returns the Current Data Layer,4103. Similar to user modifications, a synchronization change is applied to the Current Data Layer,4103, and the Data Layer Modifier,4120, outputs a Modified Data Layer,4102, which is used to replace the Current Data Layer in layer storage.

FIG. 42illustrates an embodiment of a coordinator appliance that provides coordination functions on a team. The illustrated embodiment of the coordinator appliance is comprised of a user input,4220, coordinator logic,4230, database,4400, coordinator network interface,4240, and a display,4210. The user input devices,4220, includes all the inputs used by the user of the appliance such as keyboards, mouse, touch screen, pen tablets, microphones, cameras, etc. These inputs coupled to the coordinator control logic,4230, via4221. The coordinator control logic evaluates this input,4212, and messages from the network,4231,4233,4234, to determine create a presentation for the display. The coordinator control logic is also responsive to the database,4400, and the assigned role for the appliance. Coordinator appliances generally lead the team and determine which appliances are part of the team. The assigned role helps determine how to create the display presentation and what messages should be sent to other appliances via4232. The coordinator network interface,4240, handles the network communications and keeps track of other appliances on the network. Outgoing messages,4241, are sent to other appliances both for identification and controlling other appliances. Incoming messages,4242, contain both identification and control information. The database,4400, contains information about the current state of the appliance but also the Layer Storage. The display presents information to the user. Many times this is visually on a computer screen, but may also take other forms such as sound, music, static images, movies, flash, interactive games, projectors, tactile feedback (resistance to movement in a joy stick, or vibration), motion of objects, etc.

FIG. 43illustrates an embodiment of a non-coordinator appliance. It is similar to the illustrated coordinator appliance as inFIG. 42, having a user input, but instead of having coordinator database logic and coordinator network interface and logic, the illustrated non-coordinator appliance has non-coordinator control logic. The embodiment inFIGS. 42 and 43is that the control logic is different as to what messages are sent and not sent. A non-coordinator appliance is a member of a team but is not a leader. For example, in an educational team, the teacher role is a coordinator appliance and the student appliances are non-coordinators. The facilitator role in a meeting is the leader of a meeting team. Participants in a meeting are non-coordinators. The presenter in a meeting team is also a participant but has different functionality than a participant. However, the presenter could be assigned both the facilitator and presenter roles.

FIG. 44illustrates an embodiment of a database,4400. The database has two basic parts: database storage,4500, and information gathering functions as illustrated by4410,4420,4430,4440,4450and4460. The database storage provides a structured method for storing and retrieving information. A common form of this is a relational database which provides tables of information that can be joined together to provide complex, yet efficient, searches for the stored information. However, other methods can be used which may be similar to a relational database but not meeting the standards for a relational database, such as indexed lists, b-trees, linked-lists and other methods. Some of these can provide the same information but with a significantly different structure than a relational database. In this embodiment, the docid is a unique descriptor of a unique document. The pagid is a unique descriptor of a unique page within a document (identifies part of layer data). The imgid is a unique descriptor of a unique set of images which comprises the common base layer for a page of a document (part of layer data). image index selects on the unique set of images for a imgid. Each image in the set is visually identical to the others in the set except that it is a different resolution. image file name is the file that contains the image data for a given imgid and image index. The appid is a unique descriptor of a specific appliance on the network. The section and pagemark are a user-friendly description of a specific page (pagid) in a document (docid). Doodle file name is the name of a file that contains the layer data for a specific page (pagid). The desired size is a signal responsive to the display that provides the optimum size for the display. “Is document in the library?”,4410, receives an docid on4411, and queries the database storage,4500, and returns a yes (and the docid),4412, or no,4413. “Is document's page in the library?”,4420, receives an docid,4421, and pagid,4422, and queries the database storage,4500, and returns yes and the docid,4424, and the pagid,4425; or no and docid,4423. “Is appliance on my team?”,4430, receives an appid,4431, and queries the database storage,4500, and returns yes,4432, and no,4433. “Get information about page”,4440, receives an docid,4441, and pagid,4442, and queries the database storage,4500, and returns the docid,4443, pagid,4444, section,4445, pagemark,4446. “Is page info in library?”,4450, receives a pagid,4451, and queries the database storage,4500, and returns yes and pagid,4454, no and pagid,4452, section,4455, pagemark,4456, and doodle file name,4457. “Get information about page”,4440, receives an docid,4441, and pagid,4442, and queries the database storage,4500, and returns the docid,4443, pagid,4444, section,4445, pagemark,4446. “Find best image?”,4460, receives an imgid,4461, and desired size,4471, and queries the database storage,4500, and returns image index,4466, and doodle file name,4467.

FIG. 45illustrates an alternate embodiment of Database Storage,4500. The database storage is composed a plurality of tables and a file system,4540, for storing large amounts of data, e.g., layer data. The tables could be implemented in a number of ways included the use of a relational database, group object structure (5300) using properties as column values, b-tree files, etc. The tables have rows and columns. The ordering of the columns is not important, but the ordering of the rows is important in some tables. In this case, some implementations will need to add a column sortable values to keep track of the preferred order. The doc table,4510, and docpage table,4520, need to maintain ordering of the rows and the others do not. Ordering is important in these tables because the order of the rows define the order that the user sees the documents and pages within the document. The tables are: doc table,4510, docpag table,4520, value table,4530, pageimage table,4550, image table,4560, my team table,4570, program table,4580, registry table,4590, and key table,4595.

The tables are coupled together by common columns. When a row in one table and a row in another table have a common column and the values are the same then the other columns in each of the tables are related. This is called a join in a relational database. The doc table is coupled,4515, to the docpage table via a common column docid. The docpage table is coupled,4525, to the value table,4530, and pageimage table,4550, via a common column pagid. The pageimage table is coupled,4555, to the image table,4560, via the common column imgid. The rows of the doc table represent a set of documents in a particular order, much like books carefully placed in a bookshelf. The columns in the doc table,4511, are the docid, title, visible and time. The docid is a unique identifier of the document. The title is the title of the document. Visible determines whether the document will be shown or hidden. Time is the date the document was created. Other document attributes can also be stored in other columns. The rows of the docpage table represent a page in each document and the order of the rows put the pages in order as they are in a book. The columns are docid, pagid, and visible. There are usually many rows with the same docid, one for each page in the document but each row has a different pagid. This allows pages to be quickly reordered in using just this table. The visible column allows an individual page to be shown or hidden. The value table has the following columns: pagid, pageno, split, doodle_dir, doodle_esf, doodle_file, section and pagemark. The pagid is used to couple the rows of the docpage table and the pagid values are unique in each row of the value table. The pageno column provides a page number for the page which is used for the display for the user. The split value is provides a location on the page where it can be split between the top and bottom halves for display. The doodle_dir provides a directory in the file system,4540, where the a plurality of Data Layer store their layer data for this page in a doodle file. The doodle_esf provides for a container file like zip that contains the doodle file. The doodle_file provides the name of the file in the file system or in the doodle_esf container. Section provides a descriptive name of this page and all pages that follow in this document until a new section is provided for the user. Pagemark provides a descriptive name of this page only for the user. The pageimage table,4550, has two columns: pagid, imgid. The pagid values are unique in each row of the pageimage table. The imgid usually is unique but not always. This allows an image to be duplicated by using the same imgid value for two different pagid's. The image table,4560, provides the location and attributes of the base common images used as the common layer data for the page. The columns in the image table are imgid, imgdir, imgname, imgesf, imgwidth, imgpageno, time. The imgid is a unique identifier for a set representations of a visually identical image to the user. There are multiple images stored at different resolutions but visually look the same to the user. This reduces the computations required by the appliance to resize an image to fit on a particular display, the closest best fit can be selected that has been precomputed and stored. The imgdir provides the directory in the file system where the image is located. Imgesf is the file name of a container that can hold many files like a zip file. Imgname is the file name of the image on the file system directly or the name in the imgesf if provided. imgwidth and imgheight provide the width and height of the image so the optimum image can be selected.

Since there can be multiple rows with the same value of imgid, the image index indicates which of the rows contains the selected image for a page and display. Find best image,4460, uses the imgid,4565, matching image table rows,4562, and the desired size,4471, to find the best image location,4467. The location is combined in the full_path_name,4545. The file system,4540, is responsive to the full_path_name coupled by4543and outputs the file data,4541, via4542which is the image data to be sent to the display. The “Is page info in library?”,4450, is responsive to the pagid coupled by4525and4451. The output,4457, provides information about the page including the Layer Storage except base Common Data Layer (doodle) location. The full_path_name,4545is responsive to the page information and creates a description of the location of the Layer Storage for the file system,4540. The file system is responsive to the coupling signal4543, and outputs via4542to the file_data,4541, which is the Layer Storage except the base Common Data Layer for processing by the display logic in the coordinator control logic or non-coordinator control logic.

The my team table has one column which is teammember. This table has a list of team members on their team. This table is only present on a coordinator appliance. The program table has two columns, prog_param and prog_value. Each row of the program table has a prog_param value which defines parameter for the application. The prog_value in the same row is the value for the prog_param is the value for the parameter. These parameters are used to define a number of items for the appliance like its name, the current document and page being displayed and many other items that need to be remembered when the appliance is turned off and then back on. The registry table has two columns, reg_param and reg_value. It is similar to the program table but is used for installation parameters and on some cases may be stored in a different manner than the other tables. The key table has the following columns: docid, appid, time, key. This table is used to store encryption keys that some of the content may be using. The key column contains the key for a document specified by docid and for a specific appliance specified by appid. A group of documents may have been created at the same time and can have the same key so they are identified with the time column.

FIG. 46illustrates an alternate embodiment of the present invention, showing a system wherein multiple teams each are made up of multiple appliances, and wherein the multiple teams can co-exist on the same network, but act as independent teams. All teams communicate over a common network,4650, and connections4611,4621,4631,4641. As illustrated inFIG. 18, there is shown a music team,4610, an education team,4620, a meeting team,4630, and an Ad Hoc team,4640. The music team is shown as optimized for people collaborating using music to rehearse, edit, and/or perform the displayed music. The education team is shown as optimized for a system operating in a classroom setting where there is a teacher and multiple students collaborating. The meeting team is sown as optimized for the use during a meeting for collaboration of multiple people, where there are multiple participants, each having an input device with one or more (some) making presentations and with each member having the ability to make their own separate input onto the team presentation. The Ad Hoc team is shown as optimized for collaboration of informal gatherings of people so that they can each communicate with their own ideas to a common display, and provides a system to facilitate those meetings. Many other teams are possible, four have been shown in this illustration.

FIG. 47illustrates roles within a music team. There are shown multiple roles and the functioning of the appliance. What layers are used by a particular appliance depends on its role. Any edits that may be made will depend on the role that an appliance is functioning in. The team communicates over a common network,4750and connections4711,4721,4731,4741.

In a music team, there can be a leader appliance,4710, which is the coordinator, and the leader appliance can make changes to and communicate these changes (ranging from edits to a display document, to page jumps, to document ordering, to import or export, etc.) with all the other appliances. There may be multiple leader appliances on a team.

A member appliance,4720, is one that generally listens to what the leader appliance is doing but may be able to have some other limited functionality (such as making local-only edits, or local-only page jumps, or local-only document order changes).

A listener appliance,4730and4740, is similar to a member appliance, but it is strictly only able to listen to commands and input from the members and leaders, and cannot make any changes by itself.

FIG. 48illustrates roles in an education team. In the illustrated embodiment, there are two different roles: the teacher and the student. The team communicates over a common network,4850and connections4811,4821,4831,4841. As illustrated, the teacher,4810, (or all teacherPs on a team) has the ability to communicate with all the students,4820,4830and4840, as well as with any other teacher on a same team that may be on the team and can selectively choose for communication with and to students as either to the entire class or to an individual selected student. The teacher appliance can also select to view in real-time all of the students' appliances' displays at once in a multi-screen mode. As illustrated, the student appliance communicates only with the teacher (or teachers, but not with other students) so that the teacher is aware of what the student is doing and themselves. The student is not able to communicate with other students.

FIG. 49shows an embodiment of appliance roles in a meeting team. The team communicates over a common network,4950and connections4911,4921,4931,4941. As illustrated, three are different roles. The roles of a particular appliance can change during the course of a meeting. The three illustrated roles are facilitator, presenter and attendee. The facilitator,4910, is the appliance that determines which appliances are allowed to be at and in the meeting and determines and decides which appliances are permitted to be a presenter and which appliances are an attendee,4930and4940. The presenter,4920, is allowed to have his/her edits sent from his/her appliance to the appliances of other members of the team. The presenter can selectively control which page the respective users at each appliance in meeting is looking at. The facilitator can selectively change which appliance is the presenter. Individual attendees have more limited control. At the discretion of the facilitator appliance, an attendee appliance may or may not be permitted to communicate with other appliances. A meeting can have different presenters at different times. The facilitator an change who (which appliance) is enabled as a presenter. Usually there are multiple attendees and one presenter, however there are cases when multiple presenters are desired. Multiple presenters can be confusing to the attendees in a meeting but there are times when two or more people are mutually presenting. In extreme cases, everyone could be assigned the presenter role. There can be times when there is no presenter, such as times in between different presenters or during breaks. A presenter and facilitator has all the functionality of an attendee.

FIG. 50shows appliance roles in the Ad Hoc team. As illustrated, in an Ad Hoc team, there is only one role. The team communicates over a common network,5050and connections5011,5021,5031,5041. Each peer appliance,5010,5020,5030and5040, has the same role and functionality as the other appliances, and can selectively communicate with any one or multiple or all of the other appliances, as desired. The Ad Hoc of team gives each appliance a great deal of flexibility, and does not require the structure that the other types of teams have.

FIG. 51shows identification messages in a team. A coordinator appliance,5110, receives messages,5121,5131and5241, from all other appliances,5120,5130and5140, indicating what role and what capabilities each appliance has. The coordinator appliance can then send messages,5111, to all the other appliances to indicate what role they have in a particular team and whether they are on the team and what that appliance functionality is. In some teams, there may be multiple coordinator appliances. In some embodiments, the team can be made up of all coordinator appliances (as in an Ad Hoc team). However in the case of a meeting team or education team, there is generally one coordinator appliance and all the rest are non-coordinator appliances.

FIG. 52shows conversational messages in a team. Regardless of whether an appliance is a coordinator appliance,5210, or a non-coordinator appliance,5220,5230and5240, each appliance has an ability to send messages,5211,5221,5231and5241, between one appliance and one or all the other appliances. Whether a message is sent, or not, is dependent on the logic of the role and the team, and is controlled according to the role of the respective appliance on a respective team. For example, in general, the rule is that an appliance will not send a message to a appliance that is on a different team. Therefore, the logic for the roles in respective appliances determine which appliance a message is sent depending on what that other appliances' role is.

FIG. 53illustrates an embodiment of a group object,5300. This is a container object for other group objects, text objects, point objects and properties stored in memory. The subgroup list,5350, is coupled to the list header,5310, via5318and contains an ordered list of pointers to a plurality of group objects as shown by links5351,5352and5353. While 3 links are shown here in the subgroup list, point list and text list any number of links can be in the list including none. The ordering of the subgroup list allows groups to be traversed in a specific order, this is important later when used to define drawing operations and the order that items are drawn. This list defines a group tree of expanding subgroups. The point list,5320, is coupled to the list header,5310, via5312and contains an ordered list of pointers to a plurality of point objects as shown by links5321,5322and5323. The point objects in the list define a vector that can be used to draw lines. The text list,5330, is coupled to the list header,5310, via5314and contains an ordered list of pointers to a plurality of text objects as shown by links5331,5332and5333. Each text object in the list is drawn with the provided text in the object and in the order in the text list. Property storage,5340, provides for a value responsive to a property in property storage which is communicated via5316. Any number of properties are available and subgroups can access the properties in any of their parents. A subgroup can redefine a specific property and then the subgroup will use that value instead of the value from the parent. In this manner, the text font, text color, text style, line width, line color, line style and other properties can be defined to control the appearance of the lines and text drawn. In addition, a visible property can be set which controls whether a group object is to be drawn or skipped. These properties are used later to define Data Layers (seeFIG. 56).

FIG. 54illustrates an embodiment of a Point Object,5400. The point object stores three items, an X location,5420, Y location,5430, and op,5440, or operation to be performed. The operations minimally are move to and draw to but other draw operations are possible. When used in a point list,5320, the point objects allow you to move to an XY location, draw to a plurality of other XY locations, and then start again with another move to as many times as desired to make a vector drawing. Note that properties in the property storage of one of the containing group objects can be used to modify how the lines are drawn (color, width, . . . ).

FIG. 55illustrates an embodiment of a Text Object,5500. The text object stores three items, an X location,5520, Y location,5530, and text,5540. When used in a text list,5330, the text objects allow you to place text at an XY location in the order of the text list. Note that properties in the property storage of one of the containing group objects can be used to modify how the text is drawn (font, style, color, . . . ). As with point objects, text objects in their respective list share the same property values. If you wish some objects to use different property values then place them in their own group object and link to it in the subgroup list.

FIG. 56illustrates an embodiment of a Annotation Wrapper Group,5600. The annotation wrapper group provides an alternative embodiment for the combination of storage of layer data and Mapping Logic. This embodiment stores a part of the layer data except the common image stored elsewhere. The wrapper group,5300, is entered from link5601. The wrapper group has a subgroup list that points to group objects that define the Data Layers. Shown in this example are 3 Data Layers, layer group,5630,5631and5632, linked by5641,5642and5643respectively, however any number of Data Layers can be defined. A version property is defined in each layer group. In this manner the layer data can be selected by traversing only the group tree where the version property matches the desired Data Layer. For instance, Data Layer A would be contained in layer group5630, similarly Data Layer B would be contained in layer group5631and lastly Data Layer C would be contained in layer group5632. Each layer group contains a subgroup list that provides the layer data for the Data Layer. Layer group,5630, contains 3 subgroups: two point groups,5650and5651, and one text group,5652, coupled by5641,5642and5643respectively. Layer group,5631, contains just 1 subgroup: point group5653coupled by5644. Layer group,5632, contains just 1 subgroup: combined group5654coupled by5645. The point groups,5650,5651,5653and5686, and the text groups,5652and5685, each have property values that define how the respective points and text are drawn. The point groups link their respective point lists,5320, to point objects,5670,5671,5672,5673,5674,5676,5677,5678,5696,5697, via links5661,5662,5663,5664,5665,5667,5668,5669,5691,5692respectively. The text groups link their respective text lists,5330, to text objects,5675and5695, via links5666,5690respectively. The combined group,5654, allows for a more complex structure of drawing items by containing a subgroup list with a text group and point group. This allows properties to be set once in the combined group,5654, e.g., color property, and then the text and point groups,5685and5686, do not need to set this property and will use the property in the combined group5654. Also, the combined group,5654, can set its visible property to hidden and then the text group,5685, and point group,5686, will not be drawn but can be easily drawn by changing the visible property in the combined group to show. Quite complex tree structures can be created for the layer data and only some of the simpler structures are shown, however, the structure shown in this example provides a very flexible and useful structure for layer data. For instance, when layer data B needs to be delivered to an element in the system for processing, only the link5622, needs to be passed to access the data.

FIG. 57illustrates an embodiment of a Music Team. A music team is composed of one or more leader appliances,5730,5760and5780, and a number of member appliances,5710,5720,5740,5750and5770. The appliances are connected to a network. Appliances5710,5720,5730,5740,5750and5760are connected to a local area network,5701, via connections5711,5721,5731,5741,5751and5761respectively. Although connected to the same local area network, appliances5710and5720are in a physically different room than appliances5730,5740,5750and5760but the operation is identical. The local area network,5701, connects to the Internet,5790, via connection5702. Appliances5770and5780are in different remote location and connected to the Internet,5790, via connections5771and5781. The Internet connections generally do not allow the connections5702,5771and5781to communicate directly. In some cases this is possible with the use of static IP addresses or VPN connections, and then the appliances5770and5780can communicate with the other appliances just as if they are in the same room. When this is not available, most of the time, then the connections5702,5771and5781communicate with a static connection on the Internet5791which is coupled with a communication server,5792. The communication server relays messages between the connections5702,5771and5781. In this manner, appliances5770and5780can communicate with the other appliances just as if they are in the same room. Some appliances may have special displays such as5720, which has a projector attached and may users can view the display.

FIG. 58illustrates an education team,5801, with one teacher,5810, and three students,5820,5830,5840, displays where the teacher is communicating to the entire classroom. All appliances view the background image,5802, as the Base Data Layer. The teacher can write an annotation,5811, on their appliance in the Teacher Data Layer, (in the example shown, the word “Work Carefully”). Each student is also viewing the Teacher Data Layer so each student immediately sees the annotation,5811, on their appliance also. Each student appliance has a unique Data Layer that they view and edit. Each student sees the Teacher Data Layer and their own Data Layer. Thus, Sarah,5820, views the Sarah Data Layer and the Teacher Data Layer. Jim,5830, views the Jim Data Layer and the Teacher Data Layer. And Bill,5820, views the Bill Data Layer and the Teacher Data Layer. Each student is doing the assignment as well. Student Sarah,5820, wrote “7”,5821, as her answer in the Sarah Data Layer. Student Jim,5830, wrote “9”,5831, as his answer in the Jim Data Layer. Student Bill,5830, wrote “8”,5841, as his answer in the Bill Data Layer. In this mode of operation the teacher does not see the students' work because the teacher only views the Teacher Data Layer.

FIG. 59illustrates an education team,5901, with a teacher,5910, interacting with a student,5920, one-on-one. Illustrated is Sarah which is viewing the Sarah Data Layer and Teacher Data Layer. The teacher is viewing the Teacher Data Layer and a specific student's Data Layer, in this example the same as Sarah,5920. Both the student and teacher edit the Sarah Data Layer. This allows the teacher and the student to make annotations in the specific student's Data Layer which are visible by both, but not the entire classroom. In this illustration, the teacher wrote “Incorrect”,5912, which is also viewed on the Sarah display5920. The student responded with a “?”,6022, which the teacher views on their display,5910. None of the other students are viewing this interaction between Sarah and the teacher so Sarah can be more confident in asking a question. Nor are the other students bothered by the Sarah/teacher interaction.

FIG. 60illustrates an education team,6010, with one teacher,6010, and three students,5820,5830and5840, where the teacher is monitoring the entire classroom's progress. The student displays are the same as inFIG. 58. This illustrates an embodiment of a Teacher multi-view screen. The display screen,6010, is shown with 3 reduced size images of the screens of various appliances. Shown is a 2 by 2 grid of reduced size display screen images but a different grid is possible such as 3 by 3, 4 by 4, 5 by 5, etc. The preferred embodiment is uses the same division vertically as horizontally because the aspect ratio of the reduced images is the same as the original. Alternatively, 3 by 4, 2 by 3 and other divisions are possible. Alternatively, some of the reduced images can overlap on another and when highlighted they move to the top of the display so they can be fully seen. The teacher can view a version of each student on a portion of the teacher display,6010. Each student version uses a scaled version of the background image,5802, as a Data Layer as well as the teacher and specific student layer. In this illustration, student display,5820, is shown in the teacher display,6010as6020. Student display,5830, is shown in the teacher display6010as6030. Student display,5840, is shown in the teacher display6010as6040. The teacher display,5810as shown inFIG. 58, is shown in the teacher display6010as6010. When a student modifies their own Data Layer, the teacher is updated with the annotations made by the student in their part of the display. The teacher can select one of the portions of the teacher display,6020,6030or6040, to enter the mode as shown inFIG. 59for a teacher-student one-on-one session. The teacher can select the portion of the teacher display,6010, to enter the mode as shown inFIG. 58for working with the entire classroom.

FIG. 61illustrates a social team where the owner, Frankie,6103, is viewing all the messages6111,6112,6113,6114,6115,6116,6117,6118,6119,6120,6121,6122,6123,6124,6125,6126,6127and6128on a display6101. These messages are displayed in various combinations inFIGS. 62, 63, 64, 65, 66, 67, 68, 69 and 70, responsive to the user,6103in this figure, and selections by the user. Frankie is a recording artist and does concerts. He is using Chat to keep in touch with his fans and his manager, John. This chat team is composed of three entities: user and owner (Frankie), user (John) and a Groupie subteam. The Groupie subteam is composed of many users, in this example: Mary, Jim, Sally, Jane, David, Fred, Randy, Kim and Bill and FrankS. User Frankie can use two different Data Layers, Frankie and FrankS. As the owner, Frankie can has control of who can join, what Data Layers they can view and what Data Layers they can modify. John is allowed to view all Data Layers but does not have a Data Layer in the Groupie subteam. Frankie can see everything that is going on but only Frankie will be able to see his messages. The Groupie subteam users each have a separate Data Layer but they can only view Data Layers in the Groupie subteam. The Groupie subteam can communicate with each other and can communicate with Frankie only if Frankie uses the FrankS Data Layer. In this figure the user, Frankie,6103, has selected to view all the messages available.

FIG. 62illustrates a social team where the user, Frankie6203, is viewing messages,6111,6114,6120,6125and6128between himself and his manager, John6204, on display6201. Frankie can keep in touch with his manager, John, without the clutter of the fans. His responses to John are not viewed by the fans. The Data Layers for the Groupie subteam are not displayed, including FrankS Data Layer. The John Data Layer is visible because the user has requested that Data Layer. The Frankie Data Layer is visible because that is the user's Data Layer. They are combined by the display logic,3900(seeFIG. 39).

FIG. 63illustrates a social team where the user, Frankie6303, is viewing is messages6112,6113,6115,6116,6117,6118,6119,6121,6122,6123,6124,6126, and6127on a display6301. Frankie can keep in touch with his fans, his responses or messages to the fans are seen by everyone. All the Data Layers in the subteam, Groupies,6301, are shown. The Frankie Data Layer and John Data Layer are not visible. John's messages,6111,6120,6128are in the John Data Layer and are thus not shown. Only the message,6124, from Frankie in the FrankS Data Layer is shown. Messages6114and6125from Frankie are not shown because they are in the Frankie Data Layer. These messages combined by display logic3900.

FIG. 64illustrates a social team where the user, Frankie6403, is viewing threads of messages that he has participated in. Frankie can see the message threads that he has participated in. Here is a thread with his manager and with the fans. Note how this eliminates extraneous messages. A more expanded view of the thread could also be shown (seeFIG. 70). All Data Layers are shown but only messages in all layers that are directly referenced by a Frankie Data Layer or FrankS Data Layer message are shown. One thread of messages is shown because of Frankie's messages6114and6125in the Frankie Data Layer referenced to John. John starts the thread of messages with message6111to Frankie to which Frankie responds with message6114. John responds with message6125and Frankie finishes the thread of messages with message6128. This single thread of messages contains two messages from Frankie but any number could be included. A second thread of messages is shown because of Frankie's message6124in the FrankS Data Layer to Jane's message6119. David's message6118is included because it was responded to by Jane's message6119. Russ' message6117is included because it was responded to by message6118. Jane started the thread of messages with message6116which was responded to by message6116. Only one message,6124, from Frankie is in this thread of messages. Only messages directly responded to Frankie's message6124in the FrankS Data Layer are included by the display logic3900.

FIG. 65illustrates a social team where the user, John6503, is viewing all the messages. John can keep track of everything going on the in the chat room for Frankie. Since John has viewing rights for all Data Layers, all Data Layers are shown. The messages shown are the same as inFIG. 61. Only the user6503is different in display6501from the display6101with user6103.

FIG. 66illustrates a social team where the owner, John6603, is viewing messages between himself and Frankie. John can keep in touch with Frankie without the clutter of the fans. His responses to Frankie are not viewed by the fans. The Frankie Data Layer is visible because the user has requested that Data Layer. The John Data Layer is visible because that is the user's Data Layer. The Data Layers for the Groupie subteam are not displayed, including FrankS Data Layer. They are combined by the display logic,3900.

FIG. 67illustrates a social team where the owner, John6703, is viewing threads of messages that he has participated in. All Data Layers are shown but only messages in all layers that are directly referenced by a John Data Layer are shown. One thread of messages is shown because of Frankie's messages6114and6125in the Frankie Data Layer referenced to John. John starts the thread of messages with message6111to Frankie to which Frankie responds with message6114. John responds with message6125and Frankie finishes the thread of messages with message6128. This single thread of messages contains three messages from Frankie but any number could be included. Not illustrated in this figure is the possibility that John could have responded to a message in the Groupie subteam in the John Data Layer. For instance, John, could have responded to message6127with a message of “We should remove this spammer”. This response would only be visible to Frankie and not to anyone in the Groupie subteam (except Frankie). This would create a thread of messages that John participated in and would be shown in this display,6701, if John had responded to a message as such.

FIG. 68illustrates a social team where the user, Mary6803, is viewing all the messages available to her. Mary is one of the fans. She and the other members of the Groupies subteam can't view the private messages between Frankie and John. Since Mary is only a member of the Groupies subteam, she can only have the Groupies subteam member Data Layers visible. The Frankie Data Layer and John Data Layers are not visible but the FrankS Data Layer is visible. Thus the messages6112,6113,6115,6116,6117,6118,6119,6121,6122,6123,6124,6125,6126and6127on a display6801.

FIG. 69illustrates a social team where the user, Mary6903, is viewing all the threads she has participated in. Since Mary is only a member of the Groupies subteam, she can only have the Groupies subteam member Data Layers visible. The Frankie Data Layer and John Data Layers are not visible but the FrankS Data Layer is visible. Mary only sent one message6112. No one responded to said message and so only said message is displayed. When and if someone does, she'll be able to easily see the response in context.

FIG. 70illustrates a social team where the user, Frankie7003, is viewing threads of messages that he has participated in with a detailed view,7004, on display7001. Frankie can see expanded messages (all branches) of a thread he participated in. This is similar toFIG. 64. The element6404inFIG. 64indicates a normal My Threads view and the element7004inFIG. 70indicates a My Thread detail. There is no difference in the thread of messages that include messages,6111,6114,6120,6125and6128. The second thread has messages6121,6122,6123and6126also included in addition to the messages inFIG. 64. Message6121is included because it is in response to message6116which was directly included in the normal My Threads view. Message6122is included because it is in response to message6119which was directly included in the normal My Threads view. Message6123is included because it is in response to message6122. Message6126is included because it is in response to message6123. These messages are combined by the display logic,3900.

FIG. 71illustrates an individual Data Layer,7100, that contains layer data that is segmented into Layer Data Elements,7170,7171,7172,7179. This illustration shows four (4) Layer Data Elements making up the layer data but any number of Layer Data Elements are possible. Each Layer Data Element contains context and content information. Layer Data Element7170contains an element context,7130and an element content,7140. Layer Data Element7171contains an element context,7131and an element content,7141. Layer Data Element7171contains an element context,7131and an element content,7141. Layer Data Element7172contains an element context,7132and an element content,7142. Layer Data Element7179contains an element context,7139and an element content,7149. The context information,7160,7161,7162,7169, from each Layer Data Element,7170,7171,7172,7179, is coupled to the Layer Data Element selection,7110. In one embodiment, Part,7103, is responsive and coupled to1612,1622,1632inFIG. 16. In another embodiment, Part,7103, is responsive and coupled to1712,1722,1732inFIG. 17. Part provides an input to the Layer Data Element Selection,7110, which selects one or more Layer Data Elements within the layer data,7100. The selection of an element is indicated on signal7120for Layer Data Element7170,7121for Layer Data Element7171,7122for Layer Data Element7172,7129for Layer Data Element7179. Part,7103, can specify more than one Layer Data Element. The specification provided by Part may include “Select all Layer Data Elements”, “Select no Layer Data Elements”, “Create a new Layer Data Element”, or a selection based on comparison with context information from the respective Layer Data Elements. Successful comparisons will cause the layer element selection signals7120,7121,7122,7132to be asserted for the respective Layer Data Element. Element selection signal7120is coupled to element context,7130, and element content,7140. Element selection signal7121is coupled to element context,7131, and element content,7141. Element selection signal7122is coupled to element context,7132, and element content,7142. Element selection signal7129is coupled to element context,7139, and element content,7149. Element contexts,7130,7131,7132,7139, output the context information on context layer data out,7171, and is coupled to Combine Context And Context Data,7170. Element Contents,7140,7141,7142,7149, output the content information on Content Layer Data Out,7172, and is coupled to Combine Context And Content Data,7170. Combine Context And Context Data provides Data Out,7102, that contains all the context and content information of all the Layer Data Elements that are asserted by signals7120,7121,7122,7129. If none of the signals7120,7121,7122,7129, are asserted then Data Out,7102, contains no data. In one embodiment, Data Layer Access,1740via1712,1722or1732inFIG. 17is responsive and coupled to Data Out,7102. In another embodiment, Data Layer Access,1640via1612,1622or1632inFIG. 16is responsive and coupled to Data Out,7102. Data In,7101, provides replacement layer data for a Layer Data Element,7170,7171,7172or7179. Data In is responsive to and coupled to Data Layer Access,1740, via1711,1721or1731inFIG. 17. Data In,7101, is coupled to Separate Context And Content Data,7150. Separate Context And Content Data divides the data into two parts, the context information which is output on Context Layer Data In,7151, and content information which is output on Content Layer Data In,7152. Context Layer Data In,7151, is coupled to the Layer Data Elements to Element Contexts,7130,7131,7132,7139. Content Layer Data In,7152, is coupled to the Layer Data Elements to Element Contents,7140,7141,7142,7149. Part,7103, asserts a single signal,7120,7121,7122or7129, to one of the Layer Data Elements. The selected Layer Data Element,7170,7171,7172or7179, replaces the information in its respective Element Context (7130,7131,7132or7139) and Element Content (7140,7141,7142or7149) with Context Layer Data In and Content Layer Data In. If “Create a new Layer Data Element” is specified by Part,7103, then a new Layer Data Element is created and Context Layer Data In and Content Layer Data In is stored in the said new Layer Data Element. Data Out,7102, contains a duplicate of the Data In,7101, when Data In is used to create or replace Layer Data Element layer data.