Groovy Wonderland

November 30, 2009

Our guest blog post today comes to us from Douglas Finnigan of the School of IIT at Temasek Polytechnic in Singapore. In addition to his superb introductory tutorials (Wonderland Development, Basic Concepts and Wonderland Development, Intro to Modules), Douglas has been working on ways to simplify Wonderland programming using Groovy. In this article, he also describes a simple particle system that can be used for weather simulation.

Snow in WonderlandFor the past six months, I’ve been part of Sun’s Co-Space project in Singapore, using Wonderland to develop a framework to run and visualise simulations inside a virtual world. An early piece of work was to read the Yahoo! weather RSS feed and visualise the local weather conditions. I was interested in getting dynamics to work in Wonderland, and had already experimented with cloth, so decided to use particle systems for the weather. All we needed was to show basic weather, such as rain and snow (not that it ever snows in Singapore!), so I decided to create a few simple particle systems which could be swapped in and out depending on the weather condition-code number. My first thought was to hardcode each particle system in Java, but this kind of went against the grain. I wanted a way for someone to add, update and load a particle system without having to dig in and modify the module’s source code.

Which is where Groovy came to the rescue.

Groovy has what are called Builder classes, for the (relatively) easy building of hierarchical data models, with built in support for XML, Ant build files, Swing development and more. You can also create your own builders for customised model building. So, after some trial and error in getting Groovy to work with Wonderland (via Netbeans and a fair bit of Ant hacking), most of the module’s code is now written in Groovy. Groovy classes are compiled to byte code, and as long as everything is built in the correct order, Groovy-Java integration is problem free.

The way the Groovy weather module works is that particle systems are defined in a simple, declarative-style markup script. For JMonkeyEngine particles, some attributes are simple primitive values, while others, such as the various influence types, are objects, which must also be defined in the script. So, the basic script structure is:

define Particle System object (String : id, Map : constructor attributes, if required) {
set primitive attributes
set object attributes {
define object (String : id, Map : constructor attributes) {
set primitive attributes
set object attributes {
define object ( ... ) {
...
}
}
}
}
set more object attributes, as required ...
}

A simple example is given at the end.

The classes for reading a script, parsing the data, generating an object tree, making sure the correct methods are called with the right arguments, and then putting this all together to create a particle system object, are all written in Groovy. Scripts are stored on the server, and loaded via url whenever a new weather code arrives from Yahoo!. Scripts can be modified and re-loaded without having to stop and start the Wonderland server. Actually, to speed things up slightly, scripts are pre-processed into a hashmap of particle system objects – this can be improved, but the project was just a simple demo and is now happily retired. (Although since this basic framework can be used to define and create other objects, not just particle systems, it may get resurrected at some point.)

The question is probably: why Groovy? My answer is: speed. Not runtime speed, but development speed. Groovy has an incredible amount of support for IO, list and String processing, meta-programming and reflection (a crucial part of this project), and the removal of great swathes of boilerplate code. For example, reading and extracting data from the Yahoo! weather feed took only five lines of code. In another project, the use of Groovy closures has been invaluable. It would now be very painful to go back to Plain Old Java. The only real drawback is the lack of compile-time error checking, which I can live with. (Scala, which is type-safe, might be even better than Groovy, though I haven’t looked at this yet.)

Rather than extend this entry beyond its allotted shelf-space, I hope to add a tutorial on integrating Groovy with Wonderland soon. (Note, this isn’t about using Groovy as a scripting language for Wonderland, but as part of module development.) I’m constantly amazed at how well designed Wonderland is, especially for extensibility, and the integration of Groovy is yet another example of this. Many thanks to the Wonderland team for their great work, and also for the opportunity to share this blog entry with everyone!

Appendix

Simple example:

Node(prefix: "com.jme.scene", args: "rootNode"){
attachChild {
ParticleMesh (id: "pSystem", args: ["snow", 1500]) {
localTranslation ([0.0, 8.0, 0.0])
startSize 0.075
endSize 0.055
startColor ([1.0, 1.0, 1.0, 0.5])
endColor([1.0, 1.0, 1.0, 0.0])
Texture ("snow.png")
renderState {
BlendState {
blendEnabled true
enabled true
}
}
geometry {
Ring {
outerRadius 10.0
}
}
emitterTransform {
TransformMatrix (args: "quat") {
Quaternion (id: "quat", args: [0.7071f, 0.0f, 0.0f, 0.7071f])
}
}
addController {
ParticleController (id: "contr", args: "pSystem") {
repeatType Controller.RT_CYCLE
controlFlow true
speed 0.10
addInfluence {
WanderInfluence {
wanderDistance 0.75
wanderRadius 0.01
wanderJitter 0.01
}
}
addInfluence {
BasicGravity {
gravityForce ([0.0, -0.01, 0.0])
rotateWithScene true
}
}
}
}
}
}

dot Wonderland: Redux

November 23, 2009

Way back in v0.4, I wrote a blog, dot Wonderland, that described the contents of the "dot" directory that held meta-information, configuration information, and cached assets for Project Wonderland. It was meant to be an under-the-covers peak into some operational details for those who were interested and users, who like me, just love to twiddle with files every once in a while. I’ve been meaning to update that blog post for v0.5, so here we go.

In that earlier blog, I posted an important disclaimer that is worth repeating: modify the .wonderland and .wonderland-server directories at your own risk! Since the information contained in these directories are implementation details and not part of any public API, anything posted here is subject to change without notice.

The .wonderland-server/ directory

In Version 0.5, the files associated with the Project Wonderland server and client are now kept in two separate directories: .wonderland-server/ and .wonderland/, respectively. The structure of the .wonderland/ directory has not changed much since v0.4, so the information described by the dot Wonderland blog is still mostly correct. There are some changes, and I’ll talk about those in another blog.

In this blog, I’d like to focus on the files pertaining to the server, in the .wonderland-server/ directory. First off, Project Wonderland keeps files for each revision of v0.5 in separate directories. If you are working with the main line source code (aka the ‘trunk‘), then files are stored beneath the 0.5-dev/ directory. If you are working with 0.5 User Preview 2, then files are stored beneath the 0.5-preview2/ directory. Once v0.5 is finalized, I’d imagine the files would be stored under the 0.5/ directory.

Keeping the files for each of these sub-versions separate really helps development: we can run and debug each of these different subversions on our development machines at the same time. Users who upgrade from, say 0.5 User Preview 1 to 0.5 User Preview 2, need to manually copy some files if they wish to keep state across these sub-versions. Hopefully the details contained in this blog will make this process a bit easier.

Log files, runtime state, worlds

If you take a look at the ~/.wonderland-server/0.5-dev directory (here, ‘~’ denotes the user’s home directory), you’ll see three subdirectories: log/, run/, and wfs/.

Log Files

The log/ directory is perhaps the easiest to explain, so let’s start there. The directory is only one level deep and contains a bunch of text files (with the .log file extension) — they are the log files for various Project Wonderland server components. The primary one is the web server log, the latest of which is web_server.log.0 (older versions from previous times the Project Wonderland server was run can be found in web_server.log.1, web_server.log.2, etc).

There are three other log files that always appear in this directory: one for the Darkstar server (darkstar_server.log), one for the Voice Bridge (voice_bridge.log), and one for the Shared Application Server (shared_application_server.log). You may also view these web files via the Web-based Administration UI.

A fourth log file (darkstar_server_snapshot.log) is only created if you take snapshots of your world (see WFS: The Wonderland File System in v0.5 for more details).

Worlds and Snapshots

The wfs/ directory contains all of the initial worlds and world snapshots. Each is stored as a collection of XML-formatted files. The structure of the way these XML files are stored on disk is defined by the Wonderland File System (WFS). WFS has evolved in v0.5 since earlier releases, and the WFS: Wonderland File System in v0.5 blog succinctly describes the contents of the wfs/ directory.

Not mentioned in the blog, however, is the recordings/ directory that you may see underneath the wfs/ directory. The recordings/ directory contains the state stored by the Event Recorder, an experimental module that performs 3D recordings inside the virtual world.

The run/ directory

I left the run/ directory as last to describe for a reason: it’s fairly complicated. This directory contains all of the installation information for the Project Wonderland server (including its various server parts, including the Darkstar server, Voice Bridge, and Shared Application Server). It also contains the "live" state of the virtual world (in the form of BerkeleyDB, an embedded database), an up-to-the-minute persistence mechanism that is implemented by the Darkstar middleware layer. The run/ directory also contains the content area for the embedded WebDav server that is bundled with Project Wonderland.

Here’s the directory listing that I current see in ~/.wonderland-server/0.5-dev/run:

content/ docRoot/ webserver/ darkstar_server/
shared_application_server/ wonderland.version deploy/ voice_bridge/
wonderlanddb/ derby.log webdeploy/

Perhaps it’s best to start off by describing which directories you can ignore for now: docRoot/, webserver/, darkstar_server/, shared_application_server/, deploy/, voice_bridge/, wonderlanddb/, and webdeploy/. These directories either contain information regarding the implementation of Project Wonderland’s modular server architecture (i.e. deploy/ and webdeploy/), it’s embedded web and application server (i.e. doctRoot/ and webserver/), or one of the server components that help implement the virtual world (i.e. darkstar_server/, shared_application_server/, voice_bridge/, and wonderlanddb/).

This leaves the following directory: content/. This directory serves as the content root for the WebDav repository. (WebDav, if you are not familiar, is simply an extension to HTTP, and let’s us manage content on the web server remotely by clients). Some of the content stored in this directory is visible to ordinary users via the Project Wonderland client: the Content Browser (Tools -> Content Browser) displays all of the content found beneath the
system/ and users/ directories here. For example, if a user drags-and-drops a model or an image or a PDF file into the Project Wonderland client, it gets uploaded to his/her space beneath the users/ directory.

The content/ directory also contains other information that is not visible to ordinary users, but is nonetheless important. It is information that is needed by the client software, so placing it in the WebDav content area was natural and eased its distribution. In fact, all modules installed into Project Wonderland are stored beneath the modules/ directory, and more specifically, the modules/installed/ directory. Feel free to browse the modules/installed/ directory for a bit: it’s layout is relatively simple. You’ll see a directory per module and the contents of the module JAR file expanded beneath that directory, along with some XML-formatted text files that describe the module. There is no other hidden registry of installed modules: its existence in the modules/installed/ directory is all that is necessary. (Incidentally, if you are ever in a position where you’d like to manually remove all of the installed modules in the system, simple clear out the modules/ directory and restart the server. On the flip side, to install a module, it’s best to go through the normal mechanisms, e.g. the Web Administration UI).

The final directory beneath content/, the checksums/ directory, contains checksums computed for various assets in the system. These checksums are computed when a module is installed and is used by the caching mechanism in the Project Wonderland client to determine when it needs to download a new asset, for example.

Migrating Important Data from One Version to the Next

If you’ve built a world, in say 0.5 User Preview 2 and want to migrate the state of that world to, say the latest trunk, then the information above should provide you with the means. By and large, it’s a simple matter of copying the proper files from one directory to another. I’ve tried this out myself by building a simple world in 0.5 User Preview 2 and then migrating the files so that I have the same world if I want to use the most recent Project Wonderland codebase (the trunk).

Using 0.5 Preview 2, I first installed the Clickable Link module (from the unstable/ section in the wonderland-modules workspace) [1] and created a simple world in 0.5 User Preview 2. I dragged-and-dropped a PDF file, which created a PDF Viewer in-world. I added a Sticky Note. I imported a 3D (.kmz) model using the Insert -> Model menu item and uploaded it to the server, placing it in a module named SatelliteModel. I then took a snapshot of the world (to preserve its state) and exited the client and killed the server.

Next, I ran the server using the trunk codebase. This populated the ~/.wonderland-server/0.5-dev/ directory. I then installed the Clickable Link module [1]. (I’m just using Clickable Link as an example; you will need to re-install any modules you had installed manually when using 0.5 User Preview 2). I then killed the server for the time being.

As a next step, I manually copied over key directories from the 0.5-preview2/ directory into the 0.5-dev/ directory. Here’s what I did:

  1. Copy the SatelliteModel/ directory in the ~/.wonderland-server/0.5-preview2/run/content/modules/installed/ directory to the ~/.wonderland-server/0.5-dev/run/content/modules/installed/ directory. Since I imported the model via the Insert -> Model menu item, it packaged the model into a module and installed it on my server. This first step copied that module from my old installation to my new installation. (You should do this for any model you installed this way). Related to this is Step 2. Note that models that have been dragged-and-dropped into the world are uploaded to WebDav, and migrated in Step 3 below.
  2. Copy the SatelliteModel/ directory in the ~/.wonderland-server/0.5-preview2/run/content/checksums/modules/ directory to the ~/.wonderland-server/0.5-dev/run/content/checksums/modules/ directory. When modules are installed, the system computes checksums for certain of their assets. Since I manually copied the module from the previous installation (Step 1 above), the checksum-generating step did not occur for my new installation. Manually copying the old checksum computation should be sufficient.
  3. Copy the users/ directory in the ~/.wonderland-server/0.5-preview2/run/content/ directory to the ~/.wonderland-server/0.5-dev/run/content/ directory. This copies over all of the content that was uploaded to the server when I dragged-and-dropped content into the world. (If you placed content in the system/ directory, you may wish to copy that over too). Update: Actually, if you’ve created system-wide Placemarks via the Web Admin UI, you will need to copy the system/ directory for sure.
  4. Copy the snapshots/ directory in the ~/.wonderland-server/0.5-preview2/wfs/ directory to the ~/.wonderland-server/0.5-dev/wfs/ directory. This copies over the snapshot I took after I created my world in 0.5 User Preview 2.

After these four steps, I restarted the server running the latest Project Wonderland codebase and restored to the snapshot I took in 0.5 User Preview 2 that I copied over to my new installation. After I restarted the client, I saw the world as I left it in the previous version of the software.

While these manual steps should not take too long to do, a migration tool to perform the migration automatically would be a big help. If you are interested in writing such a tool, I think it would make an excellent contribution from the community.

[1] When I wrote this blog, the Clickable Link module was kept in the unstable/ section of the wonderland-modules workspace. It has since been promoted to the stable/ section and is now bundled with Project Wonderland.  


Reviewing Wonderland Code In-World

November 18, 2009

As the Wonderland core team, we spend a lot of time using Wonderland.  We use it for meetings, we use it for collaborative work, and we even use the world for socializing. But this week we tried something new: a group code review in Wonderland.

Code reviews are an on-going process that we usually work on asynchronously: I email out the code for review, and the reviewers get back to me once they have gone through it. While this is effective for small changes, in this case we wanted to review a large amount of code with the whole group. So we decided to get together in world and discuss the code together, almost like a reading group.

The biggest challenge was getting the code into world. We could have brought a NetBeans editor into world, but that would force everyone to have the same view. Instead, we used Drew Harry’s slide spreader app to display PDFs of the source code. We found it works much better when everyone has their own view of the source.  That way we can assign people to break out of the main discussion and answer specific questions before joining the main group again.  And by using the spatial layout, we can always see at a glance where everyone is.  Here is what it looks like in-world:

Overall, it worked pretty well. We were able to explore the code both as a group and individually. And seeing the code sweeping off into the distance definitely gave a sense of how much there is to review!

There is still plenty of room for improvement.  We had to make the slides very big to make them easier to read, but this makes it hard to align your view with the slides. It would have been great to be able to see the code in the HUD.  Also, there is no syntax highlighting and no ability to jump to other parts of the code, which would have sped things up significantly. All this got me thinking it would be nice to have a lightweight shared code viewer — maybe based on JSyntaxPane – to use in world. Sounds like a new module to write…


Writing a Story, Starting a Movement

November 9, 2009

I am delighted to introduce today’s guest blogger, Nina Nussbaum-Jones from Lockheed Martin. She has been an active participant on the Wonderland forum as well as an enthusiastic advocate for the use of Wonderland technology in her company. Her energy, persistence, and courage are inspirational. Today, Nina is sharing her story with us. Or, more accurately, her storybook.

I have worked at Lockheed Martin for 27 years, starting out as a system programmer in the basement of the computer center (which consisted of 1 DECsystem-20!).  Working closely with other engineers back then meant fighting over the few terminals in the computer room, eavesdropping on arguments nearby, and generally having fun in those big open spaces has led me on a mission to restore that connectedness that has been mercilessly absorbed in the sound-proof cubicle walls that separate the workforce today.

Why Project Wonderland?

After the packaging is tossed, the software installed, and the novelty has dissipated, it’s the heart of the people writing the software that comes through, and the community around it that has forged the open source movement.  And it’s that sense of community that makes open source a movement!** And that’s what we have found in Project Wonderland.

**Remember Arlo Guthrie’s definition in “Alice’s Restaurant”?

Why a story?

The story you are about to read is true.  We have endured for 9 months, persisting in our quest to tell our story, and we’re a bit tired of repeating ourselves.  So I wrote it down, hoping to inspire all of us who want to keep doing what it is that we do and still have fun doing it!  (And also hoping to show that us misunderstood right-brained engineers really do rock!)

Adventures in Wonderland, Starring Nina Nussbaum-Jones and Christopher Stathokostas
(PDF, 2MB)

- Nina Nussbaum-Jones

 

Editor’s Note: For those of you like me who are old enough to remember Arlo Guthrie fondly, but not the details of the song lyrics, and for those too young to have heard them, I thought you might like a pointer to the lyrics of “Alice’s Restaurant.” The part about what makes a movement a movement is almost at the very end…but I hope you’ll read them all the way through and then think about singing along with the rest of us to turn this small movement into something much larger.

 


La Progetto Wunderland

November 2, 2009

One of the indicators of the worldwide reach of a project is when you receive complaints about it only being in English. We found ourselves in this position earlier this year when we were contacted on the forums from Switzerland by Ronny Standke. However, Ronny didn’t contact us to complain–he contacted us to offer to internationalise Project Wonderland! Since then, working with the core team, Ronny has internationalised the majority of the core source code and also provided German (de) and Swiss German (de_CH) localisations. Not to be outdone, our colleague Michel Denis has provided a French (fr) locale, and I’ve produced a British English (en_GB) variant of the default US English. 

We invited Ronny to write a blog entry on his experience: 

‘Here at imedias, the information center for digital media in school and teaching of the University of Applied Sciences Northwestern Switzerland, we were given a task from the canton Solothurn to set up an "active and living environment" for headteachers.
After a long evaluation phase we decided to use Project Wonderland. The reasons for choosing Project Wonderland were:

    • it is available for all major operating systems
    • it can be run without any complex installation (thanks to Java Web Start)
    • it focuses on the more serious aspects of virtual worlds (such asapplication sharing)
    • it is Free Software
    • it is written in Java, a great programming language we can also read and write

However, Project Wonderland obviously has some negative aspects, such as:

    • it is not very mature
    • it is comparatively slow
    • it is not available in the (natural) languages we natively speak in Switzerland (German, French, Italian, Rhaeto-Romanic)

We are confident that Project Wonderland will mature over time and that speed issues will be resolved. The one area where we can help to improve the situation is translating the client to our languages. We downloaded the source code via anonymous svn and performed the localisation (L10n) for German, i.e. we just translated the few resource bundle files we found. Unfortunately, most of Wonderland was still missing internationalisation (i18n). Because we have some experience with i18n/L10n of Java programs we tried adding the missing pieces. After some days of careful source patching, translating, updating from svn and resolving conflicts we had a pretty good i18n coverage.
We did not want to keep this work behind our closed doors so I contacted the Project Wonderland team and asked if they were interested in our work. After sending the initial patch I was welcomed with open arms and a huge amount of appreciation and received developer access almost instantly. Then I tried committing the changes we made piece by piece, checked every change and by doing so noticed several little (mostly cosmetic) things that were crying out for a fix. I became a little too ambitious with it, started breaking things until it raised some eyebrows and was brought back into line. Now I am working in minimal invasive mode, conducting a code review with the owner of the source code before committing my changes and also learned to write bug reports before sending in a patch for review. So, let this be a small lesson for you all future Wonderland contributors. ;-)
Right now we (the whole Project Wonderland team) are still finding pieces of Project Wonderland where i18n is missing or can be improved and fixing those parts. We really look forward to a version where our target group must not jump over a language barrier in addition to the technical challenge they will face when using a virtual world client for the first time in their lives. When working with the development version of Project Wonderland it is great and encouraging to see the improvements that happen to it every day. And it is also great to see that other Project Wonderland team members start taking care of little cosmetic things in the source. ;-)
And (just in case you wondered), we are still calling it "Project Wonderland" here.’

To get some idea of how the user interface changes when using different locales, the screenshots below are of the avatar configuration user interface for the US English, German, French and British English locales, respectively (you may need to widen your browser). Volunteers are needed for other languages, so if you’re able to spend a while translating from English, German, French, or British English to some other language, please get in touch.

US English Avatar Selection UI US English Avatar Configuration UI

German Avatar Selection UI German Avatar Configuration UI

French Avatar Selection UI French Avatar Configuration UI

British English Avatar Selection UI British English Avatar Configuration UI


Follow

Get every new post delivered to your Inbox.

Join 56 other followers

%d bloggers like this: