This is your brain in Wonderland…

July 24, 2008

I wanted to share with you a picture of a recent prototype I put together of brain visualization using a third-party library called MindSeer. MindSeer comes from the folks in the Structural Informatics Group at the University of Washington. It takes a collection of geometry files and MRI data and constructs a Java 3D scene graph of the brain structure. The library itself is fairly extensive and has all sorts of neat features: it lets you color different component overlays (e.g. colors the cortex, veins, and arteries) differently and also allows for "cutaways" so you can see inside the brain.

A special thanks goes out to Eider Moore of the SIG group at Washington who patiently answered my questions about MindSeer! 

To make this happen, all I did was create a new type of cell in Wonderland that, given a list of URLs of the 3D cortex, artery, and vein data, renders the brain in the world using the MindSeer library. Anyone with this new cell type installed on their Wonderland installation and some sample brain data can then incorporate this into their worlds! (Btw — for Wonderland v0.5, I’m working on a module system so that everyone can package their code and art and share it with others).

This is part of a larger effort on my part to create a "Science Playground" in Wonderland — my goal is not only to put together a neat space, but also to exercise and stress Wonderland in different ways. I’m hoping this space will have examples from all different areas of Science: physics, chemistry, medicine, astronomy,  and earth science. Having third-party libraries out there that are written entirely in Java and open-source make my life much, much easier!

 The following is a visualization of MindSeer’s P117 sample data:

 


Automatically launching X11 shared apps

July 22, 2008

In this, my first Wonderblog entry, I will show you how to automatically launch X11 shared apps in Wonderland. In Release 0.5 we plan to provide a Wonderland File System (WFS) feature that you can use to specify apps which are to be automatically launched when the world first starts up. You will be able to do this without modifying the Wonderland code. In the Release 0.4 code (i.e. the current CVS trunk at the time of this writing) it is possible to automatically start X11 shared apps but only with code modifications. I’ve integrated some commented out examples of how to do this. All you need to do is uncomment the code and modify it for the app or apps you wish to automatically launch.

Automatically Launching Server Apps

Usually you want to automatically launch an application when the world is first being initialized. The application will be launched before any normal clients connect to the server. This section demonstrates how to do this.

In lg3d-wonderland/src/classes/org/jdesktop/lg3d/wonderland/appshare/ServerMasterClient.ServerMasterClient() uncomment the following line:

            // Note: Uncomment this to activate SMC automatic shared X11 app launching
            // Here is an example of automatically launching a shared X11 app from the SMC when it starts up
            //autoLaunchSharedX11Apps();

Then modify the following method:

     // Here is an example of automatically launching a shared X11 app from the SMC when it starts up
    private void autoLaunchSharedX11Apps () {

        // Launch an app which is defined on a Shared Apps SMC menu
        Iterator appIterator = AppConfigX11Examples.smcIterator();
        while (appIterator.hasNext()) {
            AppConfigX11Example appConfig = (AppConfigX11Example) appIterator.next();
            if (appConfig.getAppMenu().equals("Team Room") &&
            appConfig.getName().equals("Center Firefox")) {
            System.err.println("Auto-launching the Team Room Center Firefox");
            AppLauncher.startXApp(appConfig);
        }
    }

}

This method will scan the lg3d-wonderland/src/config/apps-{linux,solaris}/apps-smc.xml Shared Apps menu definition file for a particular app (in this case the Center Firefox) in the team room and will launch it after the SMC is finished with its initialization.

Note: during my testing I have observed the following exception occur when a normal Wonderland client first connects but it appears to be innocuous and therefore can be ignored.

      [java] Exception in thread "Escher-DinReader" gnu.x11.Error: #Error
     [java]   code: 10 BAD_ACCESS: operation right denied
     [java]   sequence-number: 42
     [java]   major-opcode: 2 ChangeWindowAttributes
     [java]     at gnu.x11.MessageFactory.build_error(MessageFactory.java:96)
     [java]     at gnu.x11.MessageFactory.build(MessageFactory.java:299)
     [java]     at gnu.x11.DinReader.readMessage(DinReader.java:113)
     [java]     at gnu.x11.DinReader.run(DinReader.java:79)

Automatically Launching Local Apps on Linux and Solaris

Locally launched X11 shared apps are supported on the Linux and Solaris platforms. Sometimes you may wish to launch a particular local X11 client when your client connects to the server. This section demonstrates how to do this.

In lg3d-wonderland/src/classes/org/jdesktop/lg3d/wonderland/Main.java:main uncomment the following code (which is at the end of the method):

     /* Note: Uncomment this to activate local automatic shared X11 app launching

    // Here is an example of automatically locally launching a shared X11 app from a normal Wonderland client
    // when it first starts up.
    //
    // NOTE: Reminder: if initInBestView for the app is set to true in apps.xml this will bring up the
    // in approximately the best view of the initial view location. Note that in MPK20 the initial slide
    // show may therefore obscure the app. To place the locally launched app in a specific location set
    // initInBestView to false and set the cellPosition.x,y,z and cellRotateY parameters.

    if (!ServerMasterClient.iAmTheServerMasterClient()) {

        // Bug workaround: wait 10 seconds. there is a bug which can occur if you launch
        // an app before the Wonderland art assets are fully loaded and initialized
        // NOTE: you may need to tweak this value for your situation
        try { Thread.sleep(10000); } catch (InterruptedException ex) {}

        // Launch an app which is defined on the local menu
        Iterator appIterator = AppConfigX11Examples.iterator();
        while (appIterator.hasNext()) {
            AppConfigX11Example appConfig = (AppConfigX11Example) appIterator.next();
            if (appConfig.getAppMenu().equals(AppConfigX11Example.LOCAL_MENU) &&
                appConfig.getName().equals("Firefox")) {
                System.err.println("Auto-launching a local Firefox");
                AppLauncher.startXApp(appConfig);
            }
        }
    }
    */

This code first checks to make sure it is only run on a client other than the SMC. It then waits a bit for the Wonderland scene graph to be initialized (launching an app before this occurs can fail due to a bug). Note that you may need to adjust this wait for your particular situation.

Finally, please make sure you read the NOTE above about the initInBestView parameter.


JavaOne talk available on Sun Developer Network

July 21, 2008

The Sun Developer Network has posted a recording of the talk that Paul and I gave at JavaOne.  The talk is titled "Project Wonderland: A Toolkit for Building 3D Virtual Worlds."  The talk was a mix of demos and technical content, focusing on how to extend Wonderland by building custom cells.

You can listen to the talk synchronized with the slides, or download the whole talk here:

http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-6125&yr=2008&track=coolstuff

JaveOne talk multimedia view

Note: you will need a free Sun Developer Network account to view the recording.

Unfortunately, there is no video, so you won’t be able to see all our cool demos.  But the technical content is (hopefully) worthwhile. Additionally, since JavaOne, we’ve flushed out the documentation on how to build a custom cell with a set of tutorials on the wiki:

  • Part 1 – Extending Wonderland with a custom cell type
  • Part 2 – Handle the reconfigure of a cell’s properties dynamically.
  • Part 3 – Texture your shape with an image.
  • Part 4 – Handle input events for your cell.
  • Part 5 – Synchronize a cell’s state across multiple Wonderland clients.

The ability to create custom cells with Java code is one of the most important features of Wonderland, so we’re excited to get as many people as possible exploring the possibilities!


Wonderland Network Ports for End Users

July 17, 2008

During the last couple of weeks we’ve had the opportunity to have a wide spectrum of end users join us in a Wonderland world. Perhaps one of the biggest issues our users faced was not having the proper ports opened in their firewalls for the various network traffic between their Wonderland client and the server. Not having the proper ports open in a firewall on the client side could limit a user’s ability to receive and send audio or even connect to the world entirely.

Firewalls can be a tricky and complex issue — sometimes an end user is behind a firewall they configure (perhaps at their home) and other times an end user is behind a firewall maintained by their IT department at work. At the very least, it is necessary to make information about what ports Wonderland uses and why as clear and readily available as possible. That’s the purpose of this blog entry.

Note that we are talking about the firewall on the client side–so we are really addressing end users here. (This information still holds if you are trying to set up a Wonderland server behind a firewall — although typically in that case you are more concerned about opening ports for inbound traffic).

Here’s a table of all the ports Wonderland uses, the type of the port (TCP or UDP), and what part of Wonderland uses the port. These are default values — while we expect most Wonderland servers to use these defaults, keep in mind, each can be configured separately.

Port Protocol Usage
80 TCP Web server to launch Wonderland client
1139 TCP Main communication channel with Wonderland server
5060 UDP Audio signaling communication channel
10000 – 10200 UDP Audio channels
Random TCP Shared application channels

First of all, you will need to be able to establish a connection to the web server on the machine running the Wonderland server to launch the Wonderland client using Java Web Start — typically this web server communication happens over TCP Port 80. If you can surf the web–as is typically the case–then you likely will be able to reach the web server on the Wonderland server machine.

Next, is TCP Port 1139. This is the primary communication channel between your Wonderland client and server. When you click the Log On button, if you are unable to connect, then it is possible that the firewall is preventing you from making an outbound TCP connection over port 1139. Most firewalls should allow outbound TCP connections by default.

The next set of ports are UDP port 5060 and ports 10000 – 10200. These are used by the audio subsystem in Wonderland: if you cannot hear anyone else (or any recorded audio inside the world) and no one else can hear you, these ports may not be properly opened in the firewall. The UDP port 5060 is used for audio ‘signaling’–that is, passing information to establish and tear down audio connections. Each audio source (for example, each avatar counts as an audio source) consumes two UDP ports between the range 10000 and 10200. The client always sends the first message on these ports, so if your firewall allows outbound UDP messages, then you should be fine. Otherwise you may need to open these ports up on your firewall. (The client always sends the first message outbound on the two ports. Typically, your firewall will then allow return inbound messages on the same port by default, subject to a time-out).

Finally, the TCP ports used by the application sharing mechanism in Wonderland. If someone shares a 2D desktop application with the world, and you cannot see it, then it is possible that these ports are not properly opened. By default, the Wonderland server software chooses a TCP port at random and then communicates this choice to the client. The client then uses that port to connect to the server’s application sharing mechanism. If your firewall allows all outbound TCP connections (which is typically the case), then you are fine. If not, then the administrator of the Wonderland server can restrict the application sharing mechanism to choose a port within a specific range (most often, we tend to use the range 44000 – 44500). You can then open up those ports in your firewall.

This blog summarized the network port usage of Wonderland. For more information about configuring Wonderland in the presence of firewalls, NATs, and proxies, take a look at: http://wiki.java.net/bin/view/Javadesktop/ProjectWonderlandFirewall.

If you cannot successfully connect to the Wonderland server or hear audio or see shared applications, there may be other issues besides having the proper ports opened in your firewall. There are some pages on our wiki to help you through some of these other issues:


Postcard from Wonderland: Lecture Hall

July 2, 2008

A while ago, Jon wrote about his first World Builder world, which is a lecture hall. Since then, this has become a standard part of our Wonderland "demo tour". What’s nice about the lecture hall, apart from the fact that it showcases how easy it is to build spaces using our web based World Builder application, is that it’s packed with cool features that make for a really great demo.

Let’s take a quick tour:

Project Wonderland lecture hall

Let’s start with where I’m standing.

First notice the floor mounted microphone, just in front of me and slightly to my right. This is a functional in-world microphone. It’s not just a prop. If I stand on the white square by the microphone, my voice will be amplified so that everyone in the lecture hall hears me at the same volume. Having a microphone in this space provides virtual world users with a familiar real world model for addressing questions to the presenters. The microphone enforces a one-at-a-time mechanism by requiring proximity to the microphone. Of course we could have done away with this completely and given everyone the ability to address the room at full volume, but the virtual microphone allows us to use social mediation to control question asking. Why create a new mechanism when, there’s already an established solution we can bring from the real world?

If you look carefully there’s also a white square on the stage which works in the same way for the presenter.

On the far wall of the lecture hall are two collaborative applications.

On the left is a PDF viewer. This allows a PDF document to be displayed in-world and can be used by the presenter just like they’d use a laptop and a projector to give presentations in the the real world. Unlike the real world, though, audience members can un-sync from the presenter and flip through the slides on their own. Each user can do this independently, and when they’re done, they can re-sync with the presenter to go back to the slide they’re currently on. All without interrupting the presentation. Try doing that in the real world!

To the right is a video application. Presenters can use this to load videos to show during their presentation. The video application supports recorded videos in a wide range of formats courtesy of the fobs video library which provides Java bindings to native video and audio codecs. The video app can also be used to stream live video from Axis webcams, courtesy of the jipcam library.

New in Wonderland 0.4 is a heads up display (HUD). You can see the control panel for the video application in the bottom right. The button that looks like a chain is the sync button. Just like the PDF viewer, individuals can un-sync from the presenter and play the video at their own pace.

Finally, there’s one more interesting component in this feature packed space. On the far left there’s a table with a phone. This phone allows us to dial out from the virtual world to real world phones and have so-called "out worlders" (thanks, Jordan for coining that phrase!) join the meeting in the virtual world. This phone acts like a conference call, people can also dial into the meeting from their phones.

So, that’s Wonderland’s virtual lecture hall. I hope this encourages you to use the features we’re building into Wonderland to create really effective collaborative spaces of your own. We’d love to hear about your projects. Better yet, invite us into your worlds for a show-and-tell. We’d love to meet you!


Follow

Get every new post delivered to your Inbox.

Join 56 other followers

%d bloggers like this: