This summer the Wonderland team has been lucky enough to host three interns from schools across the country. As we head into the fall, they are all going back to school, taking their projects and accumulated Wonderland knowledge back with them. Over the next few weeks, we’ll have guest blogs from each intern about how they spent their summer.
Hello world, this is
Matt Bonner! I’m a PhD student from Georgia Tech’s Human Centered Computing program, working in Blair MacIntyre’s
Augmented Environments Lab. This summer, I had the opportunity to work with the Wonderland team as a summer intern. Prior to the team’s week of code (check out the previous blog posts!), I developed a metadata system for wonderland.
The Metadata module allows users and other modules to add pieces of information, or metadata, to cells on the fly. This information is then made available to clients and the server for searches or other queries. In addition to this interface, the Metadata module makes it possible to create general types of metadata to be added to any cell. Completely unrelated cells, developed totally apart from the Metadata module can have the same type of metadata added to them, programatically or by the user in-world.
The Metadata module is a great example module, as it uses a wide variety of objects from Wonderland’s toolbox, from cell components to darkstar services to client plugins. It is intended as a backbone or service for other wonderland modules. I’ve developed one such example, the Annotations module, which allows users to attach searchable in-world notes to objects. You can check out the code in the unstable branch of our svn here (Metadata) and here (Annotations).
A general overview of the Metadata and Annotations modules, followed by a brief technical description, is below. For some inspiration on how to use the Metadata system beyond Annotations, check out the future work section!
Metadata Module
A piece of metadata is a collection of fields and values or name-value pairs, such as “author of article” and “matt bonner et. al”. Developers can create different types of metadata, and set which fields are present in that type. Various settings, such as which cells may be tagged with which types of metadata, are also configurable by developers. Users can then tag cells through the cell’s right-click context menu or properties pane with any defined metadata types allowed for that cell, and edit the fields of each piece of metadata. At the moment, developers set whether a field will be user editable or not. Editable fields are by default modifiable by all users.
Users may search the server’s metadata, and go-to cells with metadata that matches their search terms. For a general search, users add blank pieces of metadata of whatever type to their search query, and then edit the fields of these target items. The search then looks for cells with at least one piece of metadata to satisfy each term in the query.
Metadata Module Implementation
At the root of the system are the pieces of metadata attached to cells. Each name-value pair is a string to string mapping. A new type of metadata implements the Metadata interface, and then sets what name-value pairs it will store. The base implementation of this interface, BaseMetadata, should serve for most new types of metadata. A java annotation is added to the type to register it with the system, enabling it in places like the general search and cell context menus. This is accomplished with a ScannedClassLoader, a wonderland utility type. See SimpleMetadata (which extends BaseMetadata) in the Metadata module for an example metadata type.
Tags may be added to any cell that has the metadata cell component; MetadataComponent. This may be added by users in the properties pane after the fact, or included by default via a java annotation (see MetadataSampleCell). The MetadataComponent makes a cell’s metadata available to other processes on the client using a caching system to cut down on messages from the server. A corresponding cell component MO is also automatically created to preserve pieces of metadata across server restarts by storing them in the component’s server state, which is automatically saved by Darkstar. The component MO is also responsible for keeping the MetadataService on the server up to date as pieces of metadata are added, removed and edited.
The MetadataService is an example Darkstar Service, which provides search access to metadata. The general search’s interface/result component on the client is handled by the MetadataPlugin on the client side. The plugin communicates its search filters to the MetadataService, and receives results in return.
The MetadataService is actually a wrapper for a database of metadata implementing the MetadataBackendInterface. My included implementation of this interface uses an embedded ApacheDS LDAP server. If you are curious and a little masochistic, here is a decent introduction to LDAP. This decision replaces the work of creating an extensible search algorithm with the work of translating to and from the backend database’s language. It also makes cross-server searches and distributed metadata backends theoretically possible.
Annotations Module
An example client of the Metadata module, the Annotations Module creates a new type of metadata which is globally visible to users in-world. Since Annotations are a type of metadata, they benefit from a wealth of ‘free’ functionality: they are searchable and you can teleport to them, they are easily attached to cells, they may be edited from the cell properties pane, they are persisted by the Darkstar server and more!
The only thing the Annotations module needs to concern itself with is the client-side display of annotations. There is an interesting side-effect to this. Once the Annotations module is installed (along with the Metadata module), the annotation metadata type is registered with the metadata system: users may add annotations to cells like any other piece of metadata. There will however be no additional visual elements for a cell’s annotations until the AnnotationComponent is added to that cell.
The display of all annotations is a per-client setting, adjustable from the client’s View menu. Individual annotations can be cycled between display modes by shift-clicking them. Each annotation has a right-click context menu that allows users to move annotations by dragging them with the mouse, edit using the HUD or delete them all together.
Each cell starts with a default color scheme for associated annotations, but the colors can be individually adjusted from each annotation’s HUD editing pane.
Annotations Implementation
Thanks to the Metadata module, the Annotation module’s implementation is quite simple. There is an Annotation metadata type, which creates and registers a new type of metadata. There is also a plugin, which communicates with each cell’s annotation component to adjust the global display mode. The Annotation cell component also keeps the graphical side of Annotations up to date with any changes in the metadata. See the AnnotationPane, which offers a HUD-based editing pane for annotations
, for an example of wonderland’s constantly improving HUD system. Finally, there is the in-world graphical component.
Annotations are rendered by working directly with the JME renderer that handles wonderland’s graphics. In JME, ‘Nodes’ represent graphical elements in the world. The AnnotationNode extends the JME BillboardNode, an effectively 2D node type that is always facing the user. AnnotationNodes create a JME Quad type, and then using a regular java BufferedImage that is based on the annotation metadata as the Quad’s texture. The textured quad is then attached as a child of the AnnotationNode, which will cause it to appear when an AnnotationNode is attached to another element/node of the Wonderland scene graph.
An AnnotationEntity is paired with each AnnotationNode. The entity is an intermediary between the Node and an AnnotationComponent, adjusting the Node’s location and adding/removing it from the root Wonderland scene graph when necessary. The AnnotationEntity is also what makes Annotations ‘pickable’, enabling their context menus and drag-to-move capabilities.
The Metadata system is a powerful service, but is only interesting if combined with other projects. That’s where you guys come in! The Annotations module is one example, and I have a few things in mind for our work at Georgia Tech. Here are a few more ideas to get you started. If you have any questions or comments, please post a comment or visit the wonderland forums!
A general purpose ‘aggregator’ cell could be created, to gather all cells that fit a certain metadata profile. For example, all cells tagged with the new metadata type ‘Bug’, with the field ‘assigned’ set to ‘Matt’ and ‘Priority’ set to ‘Urgent’ could appear inside my aggregator cell. This creates a new way to collaboratively manage a project. More abstract uses are also possible – perhaps a cell that increases in size or changes color if a large number of ‘Review’ metadata objects with high numbers in their ‘score’ field are attached. This could be used to call attention to favorite exhibits or locations in a Wonderland world.
A voting and exhibit system using metadata tags could also be fun – have a science fair, and let users tag exhibits with information (“fossils”, “mount vesuvius”) and ratings. Throughout the fair, users could search for exhibits on certain topics, and see which were gaining the most votes.
Metadata objects could be also useful in creating alternative interfaces to Wonderland. The textual, searchable and standard nature of metadata is amenable to a phone interface or an accessible client for visually impaired users. Automated tagging of objects could be undertaken to support this. This could be a very large, very cool project!
One idea we didn’t quite get to for our week of code project was the Audio Annotation. The Annotations module could be extended to work with Wonderland’s audio recording classes, enabling users to attach audio clips to their in-world annotations. Users will then effectively be leaving audio-notes in world. This would be especially cool on the timeline, where users could leave their thoughts on each exhibit!
There are also various extensions or improvements that could be made to the Metadata module itself. One example I plan to do myself is the ability to add a server-side listener that can watch for the addition of specific types of metadata. This would make creating the ‘aggregator’ cell mentioned above very simple.
Another very interesting extension is the ability to tag avatars/users in addition to cells. Imagine attending a virtual conference and tagging people you meet. Your private demo area could be accessible only to people you had given a certain tag. You could search your tags to remember who someone was, or find someone again later.
Like this:
Like Loading...
Posted by wonderlandvw 
