Printscreen Plugin with a Photo Gallery

August 21, 2013

By Dominik Alessandri & Christian Wietlisbach
Hochschule Luzern Technik & Architektur

As part of our bachelor thesis, along with our Kinect module described last month, we developed a printscreen plugin for Open Wonderland. When the plugin is installed on a server, all connected users can choose if screenshots should be saved locally or on the server. If saved on the server, the pictures can be displayed in an automatically updated photo gallery. The update of the picture gallery is made with a shell script which is running as task in the background on the server. We had to implement it this way, as we did not find a way to save directly in the ‘docRoot’ folder in the ‘run’ directory. The shell script uses rsync to keep these two folders in sync.

When a user is connected to a server with the printscreen plugin installed, he can display the controls of the plugin by selecting ‘Window -> Printscreen Folder':

printscreen_display_window

The Printscreen Folder dialog allows the user to choose where to save the images:

printscreen_settings

When pressing the ‘o’ key, a new screenshot will be taken and either saved locally or on the server. The reason why the key binding was done with ‘o’ is very simple: The printscreen key is not forwarded from the client to the module. The capture of the key event is done on a level before the module gets the event. On this higher level, the “Print Screen” key is filtered and the module doesn’t get any event. Instead the message “Key is not allowed” appears. Version 3 of jMonkey will support taking screenshots by default, making it easier to capture the screen.

On the server-side, the screenshots are saved in /.wonderland-server/0.5/run/content/images/screenshot/. This folder should be created by an administrator when installing the plugin.

To run the photogallery, the content of the file ‘lightbox.zip’ needs to be extracted to /.wonderland-server/0.5/run/docRoot/lightbox/. This photo gallery shows all images stored in /.wonderland-server/0.5/run/docRoot/screenshot/. To update the photo gallery, we need a background task which copies the files from /.wonderland-server/0.5/run/content/images/screenshot/ to /.wonderland-server/0.5/run/docRoot/screenshot/. For example, the shell-script ‘copyScreenshot.sh’ can do the job.

An example of this photo gallery can be seen here: http://147.88.213.71:8080/lightbox/

Photo-gallery

You will need the following files to get the printscreen plugin running:


Avatar Control with Microsoft Kinect

August 1, 2013

By Dominik Alessandri & Christian Wietlisbach
Hochschule Luzern Technik & Architektur

As part of our bachelor thesis, we developed a module called ‘kinect-control’. This module lets you control your avatar by using your body doing gestures.

This module mainly runs on the client but necessary information will be transferred from the server to the client as needed. All the user needs to have is a connected Kinect device and the Kinect SDK Beta 2 installed. This means the module is only available for Windows x86/x64 clients. All other requirements will be shipped from server to client when logging in. This makes it easy for interested parties to use.

To set up a Wonderland server to use the kinect-control module, the administrator needs to change two files on the server:

/.wonderland-server/0.5/run/deploy/wonderland-web-front.war/app/win32/
wonderland_native.jar
/.wonderland-server/0.5/run/deploy/wonderland-web-front.war/app/win64/
wonderland_native.jar

These two files must be replaced. The extended files can be downloaded from

http://147.88.213.71:8080/modules/KinectControl/win32/wonderland_native.jar
http://147.88.213.71:8080/modules/KinectControl/win64/wonderland_native.jar

These new files contain the DLL ‘KinectDLL.dll’ which is necessary for the connection between Open Wonderland and the Kinect.

If you are running this module on the client-side, the first thing you need to do is to connect the Kinect device to your Open Wonderland client. This can be done using the kinect-controller dialog. After installing the kinect-control module, in Wonderland, click on ‘Window -> Kinect Controller':

kinect_display_window

The dialog contains two buttons: ‘Start Kinect’ and ‘Stop Kinect':

kinect_settings
If you have a Kinect device connected to your PC, you can click the button ‘Start Kinect’. After a while, your Kinect Device moves to the initial position and the window displays ‘Connection Works':

kinect_settings_connection_works

You can adjust the angle of your Kinect device by sliding the slider to the desired position:

kinect_settings_angle_slider

Now you are ready to move your avatar by using your body. Place yourself in front of the Kinect device and control your avatar as follows:

  • Walk: Just move your legs up and down
  • Turn right: Hold right arm to the right side
  • Turn left: Hold left arm to the left side
  • Fly up: Hold right arm up
  • Fly down: Hold left arm up

It is possible to extend the gestures recognized by this module. For this, you need to modify the file ‘gesturesBDA.txt’ located in ‘kinect-control-client.jar’ inside ‘kinect-control.jar’ using the software ‘KinectDTW‘. After this file contains your new gesture, you need to map this gesture to a keyboard-input.

The file ‘keyMapping.txt’ contains the allocations from gestures to keyboard-inputs. It is located on server in /.wonderland-server/0.5/run/content/modules/installed/kinect-control/client/. The structure of the file is as follows:

[Name of gesture]=[isMouse[0/1]];[keyCode1[decimal]];[keyCode2[decimal]];[time[millis]]

Example 1:

@Run=0;16;87;2500
Description:
When gesture @Run is recognized, press key 16 (shift) and 87 (w) 2.5 seconds long.

Example 2:

@Walk=0;87;0;3000
Description:
When gesture @Walk is recognized, press key 87 (w) for 3 seconds long.

For a list of all keycodes you can consult http://www.cambiaresearch.com/articles/15/javascript-char-codes-key-codes/.

You will need the following files to get the kinect module running:

A video of the running module can be seen on YouTube:


Building Software Together in Open Wonderland

March 18, 2013

By Nicole Yankelovich

Yesterday I gave a talk at the Northeast GNU/Linux Fest in Cambridge, MA. The theme of my talk was using Open Wonderland as a collaborative software development environment. I used the Subsnapshots project and the Postcards code-a-thon as two case studies to show people how we use Open Wonderland throughout the software development process. One thing that struck me when I put this presentation together was how many collaboration tools we actually have to support this rather complex and multidisciplinary process.

I published my slides on Slideshare in case others are interested.


New Collaborative Text Editor App

December 12, 2012

By Bob Potter

Multi-user text editor iconDisplaying and editing text can be accomplished with several tools in Open Wonderland.  These include Sticky Notes, the PDF Viewer and via the Shared Applications server using Open Office or web-based editors such as Google Docs. Each of these, however, comes with significant limitations.  Sticky Notes are best used for short pieces of text.  The PDF Viewer will show text documents but not permit them to be edited.  Open Office editing is limited to documents managed on the application server, and due to the nature of shared applications, does not support cut and paste from the user’s computer. The cut and paste limitation also applies to web-based apps. In addition to the cut and paste limitation, the usability of the web-based editors is not optimal because of the large overhead of editing text in a web browser on the application server. Additionally, given the nature of shared apps, there can be lags in viewing text updates in Open Office or other Xll apps. Finally, shared applications place a significant overhead on the application server, especially when the user is navigating and editing text.

Frequently in our Wonderland community meetings, we have experienced the need for a text editor that can:

  1. Display a large amount of text,
  2. Be edited concurrently by multiple users,
  3. Support cut and paste from the user’s system,
  4. Import selected file types using drag-and-drop; and
  5. Export the edited text back to the user’s computer.

The new Collaborative Text Editor module recently uploaded to the Module Warehouse meets these needs for routine text editing.

Collaborative Text Editor

Using the Collaborative Text Editor for note-taking in an Open Wonderland community meeting.

Once the module has been installed, a text editor can been inserted into the world using the Insert –> Object dialog.  To edit text, one or more users can take control of the editor, move the cursor to the desired location, and start typing.  The editor is limited to plain text and does not yet support any formatting such as bold and italic. There are two buttons in the lower right of the frame.  The “Sync” button can be used to refresh the display with the master copy of the document stored on the server.  This function is only required if the text appears to get out of sync when multiple people are typing or editing at the same time.  The second button, labelled “Export,” will bring up the Save File dialog, permitting the user to save a copy of the file to their own computer.

The Collaborative Text Editor supports drag and drop of text files. If a user drops in a file with the .txt, .log, .html, .java or .xml extension, the editor will automatically open and display the document. Note that since this is a text editor, it will display the Java, html, or xml source, complete with all the code and markup information. Files dropped into the world can be edited and saved back to the user’s system via the export capability.

This editor is based on the work Jonathan Kaplan (Chief Architect of the Open Wonderland Foundation) had done for the programming demo module called “Sorting Algorithms.”   Within that module, Jon had developed a code editor that permitted concurrent editing by users.  This current collaborative editor was developed by extracting and enhancing this code to create a generic text editor application.

Technical Details

The module contains both client and server code to manage the concurrent insertion and deletion of text.  Since users are actually editing slightly different copies of the same document, both the clients and the server need to adjust the position of each insertion (or deletion) based on the current state of the document in their space.  The server’s responsibility is more complex. It is receiving messages from multiple clients, each possibly being a different version from what is in the server’s master copy.  The server transforms each of these changes into the most recent version and forwards these on to the clients.  The clients have to ensure that any local modifications that have occurred since their last update has been applied are considered when determining the target position of the changes received from the server.

The code for this module is presently hosted on GitHub. (https://github.com/bpotter/textEditor.git)  The standard Open Wonderland license applies to this module. This is still a work in progress. I would appreciate your feedback.


Facebook Group Integration Module

September 24, 2012

By Shaya Pourmirza

This module has been created for communication between a Facebook group and Open Wonderland as part of my masters thesis project at the University of Essex. With this module, users can post comments in a 2D Facebook group and have them dynamically show up within an Open Wonderland world. Additionally, users in the Wonderland world can start new discussions or add comments to existing Facebook discussion threads from in-world.

To support this functionality, each Facebook group integrated with Wonderland has a Facebook Application as an intermediate layer. In this example, I have created a 2D Facebook application (http://shoopi.appspot.com) and deployed it on the Google App Engine. The functionality of this application is described below.

Authentication and Authorization

Before a user can set up a Facebook viewing window in Wonderland, they must use the intermediate Facebook application to generate an Access Token.

Access Token

They can then copy this Access Token from the Facebook application and paste it into a dialog box in the Wonderland world to authorize the Wonderland Facebook code to access Facebook. This step only needs to be done once by an administrator. All other Wonderland users can then see the Facebook discussions in-world.  The 2D Facebook application uses Facebook’s new authentication method which is based on Access Token and OAuth 2.0. (For more information, see Facebook’s Server-Side Authentication documentation.)

Retrieving Data From a Facebook Group

Once the admin user has entered their Access Code in Wonderland, Facebook viewing windows can be opened in-world. These windows can show discussions in private Facebook groups that the authenticated admin user has access to, or they can show discussions in any public Facebook groups.  The Facebook viewing windows are based on the Wonderland HTML Poster module, so all links in these windows are live. There is one Facebook viewing window per Facebook discussion group.

Creating a New Discussion

To open a new Facebook viewing window in Wonderland with an entirely new discussion thread, the user can click on the “Add a New Discussion” link in an existing Facebook viewing window. This launches a New Discussion entry form in an external web browser window.

Create a New Discussion

Once this initial post is created, any in-world or 2D user can add a comment to the discussion.

Adding a New Comment

In-world users add comments to an existing discussion by clicking on the “Click here to add new Comment” link in the appropriate Facebook viewing window. This launches an external browser window. Users are taken to an Authentication Servlet web page. If they have not already logged into Facebook, they will be asked to do so. Once logged in, they are provided with an Access Token.  Users must copy this token and then proceed to the next screen where they can enter their comment and paste in the token. When they click the “New Comment” button to submit their comment, it simultaneously shows up in both the 2D Facebook group and in one of the Facebook viewing windows in the 3D world.

Demo Video

The following video was captured from a user trial session.

It illustrates the procedure of adding two comments from the 3D world into a Facebook group. Since these comments were added at the same time, only one of them will be shown step by step. Finally, participants were asked to fill in a usability questionnaire. The results indicated that users appreciated this new Open Wonderland feature.

Source Code

If you are interested in looking at the code for this module, I have made a zip bundle available on Google Drive:

Facebook Integration Module Source (.zip)

Select “Download” from the File menu to download the complete bundle.


Summary of +Spaces Development in Wonderland

July 10, 2012

By Bernard Horan

One of the goals of the +Spaces project is to prototype three applications in Open Wonderland (OWL) by which citizens can be engaged in policy discussion: polls, debates and role-play simulations. In the earlier blog posts, I focused on the way in which the applications appeared to users and the interaction features they provided. In this blog post, I provide a brief description of the other less visible features of the applications and the way in which, from an architectural standpoint, the applications are implemented in Open Wonderland.

The additional invisible responsibilities of each application are:

  • to receive updates from the +Spaces middleware
  • to communicate user actions to the +Spaces middleware

Each application is implemented as an OWL ‘cell’, packaged in an OWL module. However, there are some additional supporting modules that provide cells and other functionality used by the three applications. I’ll go through each application, beginning with the Polling Booth, and then briefly describe the other functionality before summarising our work.

Polling Booth

The Polling Booth is the simplest of the applications and is implemented as a 2D cell. The client implementation is a 2D Swing application that displays the current users of the Polling Booth, along with metadata describing the poll, and a button by which the user may take the poll. The user interaction of voting in a poll is implemented as a heads-up-display (HUD) component—this is because the activity is private to the user (see Figure 1, below). The description of the poll, such as its title and sequence of questions, is represented as an instance of a Java class that takes the role of a data model, with no behaviour. The Java Poll class is dependent on some other data-model-like classes such as BinaryPollQuestion, and FreeTextAnswer.

Figure 1: +Spaces Polling Booth

The server implementation of the Polling Booth cell acts as a gateway between the client implementation and the +Spaces middleware; thus it has two responsibilities: communication with the client and communication with the +Spaces middleware. The server communication with the client follows the regular OWL architecture, relying on subclasses of App2DCellClientState, App2DCellServerState and CellMessage. However, server communication with the +Spaces middleware follows the example of the connectionsample module that relies on the use of a subclasses of BaseConnection, Message and ResponseMessage and implementations of ServerPlugin and ClientConnectionHandler.

The communication flow is as follows (and is common to all three +Spaces OWL applications):

  • The +Spaces middleware creates a connection to the OWL server, and calls a method on the connection to set the data model of a cell (identified by its cell id) to be an instance of class Poll. The connection sends a message (an instance of a subclass of Message) to the OWL server and waits for a response.
  • On the OWL server, the instance of (an implementation of ClientConnectionHandler) receives the message. It then looks up the instance of CellMO according to the cell id (via the method CellManagerMO.getCell()), casts it to be of the correct class and calls a method to set its poll data model. The CellMO instance, in turn, sends an instance of (a subclass of) CellMessage to its clients, causing the clients to update. Finally, the ClientConnectionHandler returns an instance of (a subclass of) ResponseMessage to indicate if the operation to set the poll was a success.
  • From the client, when the user interacts with the Polling Booth, the answers (and other actions) are sent from the Cell to the CellMO on the server via an instance of (a subclass of) CellMessage. The Polling Booth CellMO then calls a method on the instance of (an implementation of) ClientConnectionHandler to send the user’s actions to the instance of (a subclass of) BaseConnection, where it is received and passed on to the +Spaces middleware via a listener pattern.

Debating Chamber

The Debating Chamber application is also implemented as a cell, taking on two roles: it acts as a container for other elements of the chamber; and it presents a ‘carpet’ to the user on which he or she may express a vote.

Figure 2: +Spaces Debating Chamber

When the CellMO that represents the Debating Chamber is inserted into the world, instances of other subclasses of CellMO are added as its children, as shown above in Figure 2. These are:

  • Two instances of StickyNoteCellMO—these post-its are used to display the ‘statement’ and ‘description’ of the debate.
  • An instance of ModelCellMO—this is used to hold the 3D model of the chamber (the OWL auditorium model).
  • An instance of AudioRecorderCellMO—this is used to produce an audio recording of the debate (it adds the instance of the Debating Chamber CellMO as a listener to be informed when a user terminates a recording).
  • An instance of SharedActionCellMO—this is used to render messages received from other virtual spaces such as Twitter and Facebook (this is based on the twitter viewer module that is in the OWL modules source code repository and rendered as a pigeon on the client).
  • An instance of LinkListCellMO—this is used to render the list of URLs that users may double-click on to open a browser. This cell is part of a new module that is based on the Sticky Note cell: it is rendered similarly, with a list replacing the text area.

The CellMO that represents the Debating Chamber maintains managed references to each of these children (except for the instances of ModelCellMO and AudioRecorderCellMO). For example the Debating Chamber CellMO has a field labelled ‘debateDescriptionCellRef’ which is an instance of ManagedReference<StickyNoteCellMO>. The references are used as follows:

  • When the setDebate() method is called on the Debating Chamber CellMO, it calls the method setText() on each instance of StickyNoteCellMO, with the parameter representing the description or statement, appropriately. It also calls the method setLinks() on the the instance of LinkListCellMO, with a list of links (URL-displayname pairs). The result of these method calls is to update the display of the contents of the cells on the clients.
  • From the client, when the user interacts with the Debating Chamber, the user votes are sent from the Cell to the CellMO on the server via an instance of (a subclass of) CellMessage. From then on, the mechanism follows the same pattern as the Polling Booth.
  • When the +Spaces middleware reports a comment from Twitter or Facebook, it is received as a Message by the implementation of ClientConnectionHandler associated with the Debating Chamber CellMO. This, in turn, calls the method notifySharedMessage() on the Debating Chamber CellMO, which then calls the method addRecord() on the instance of SharedActionCellMO, eventually causing the clients to display the contents of the comment.

In terms of its communication with the +Spaces middleware, the implementation of the Debating Chamber application follows the pattern set by the Polling Booth. That is, it reports user actions and receives comments from other virtual spaces via the OWL server.

Simulation Agora

The ‘Simulation Agora‘ is the name we give to the virtual space that hosts the role-playing simulation. It is the most complex of the applications and is also implemented as a cell, somewhat similar to the Debating Chamber. However, unlike the Debating Chamber it does not provide any direct user interaction features for 3D objects, and instead represents the floor of the agora (rendered as an EU flag) and provides a HUD control panel to a privileged user role by which the stages of the simulation can be managed.

Figure 3: +Spaces Simulation Agora

Figure 3 illustrates some of the elements of the Simulation Agora and the HUD control panel that is accessible to the moderator (or any member of the admin user group).

The HUD control panel is implemented as a Swing tabbed pane. One of the tabs provides the moderator with the means to step forwards and backwards through the stages of a ‘template’ that describes activities of a role-play simulation (more information about the use of role-play templates is provided in an earlier blog posting). Each of the two templates is implemented in Java in the form of a (pseudo) state machine: as the moderator progresses the template from one stage to the next, the state machine calls methods on the CellMO that represents the Simulation Agora to insert and remove child cells and to update the state of those child cells. Without going into too much detail, the sequence of stages of the state machine are as follows:

  1. Meet Participants—this is the starting stage of the simulation
  2. Ice Breaker Vote—this is an optional stage, specified by the policy maker, in which participants may vote on a question related to the policy
  3. Ice Breaker Results—the results of the ice breaker vote are presented
  4. Assign Roles—the moderator assigns the participants to the roles described in the template
  5. Prepare Statements—the participants create a number of statements according to the description of their role (the number of statements is specified by the template)
  6. Read Statements—the participants read out their statements
  7. Mark & Group Statements—the participants and the moderator collaborate to group their statements and to assign votes to statements
  8. Present Results—the participants discuss the results of marking and grouping the statements
  9. Final Vote—if there was an ice-breaker vote, then the participants are given the opportunity to vote again to determine if their views on the policy have changed
  10. End Simulation—the results of the final vote are presented

The children of the cell that represents the Simulation Agora are:

  • An instance of PosterCellMO—this shows the current stage of the simulation by rendering an appropriate image. This cell remains visible at all times.
  • An instance of a simulation carpet cell—this in turn has a child Poster cell that displays the question that the carpet is to be used to answer. The simulation carpet is used to capture the participants’ vote on the ice breaker vote and the final vote. It is added and removed as necessary by the state machine, in response to the moderator’s interaction with the HUD control panel.
  • Two instances of PosterCellMO—to present the results of the ice-breaker vote and final vote. The state machine adds these cells when appropriate and sets their contents to present the results of the preceding votes.
  • A cell that is used by participants to add ‘statements’—this is a cell developed specifically for the project. It is a 2D cell, in which the client implementation is provided by a 2D Swing application that uses an instance of JDesktopPane. The cell is added as a child when the state machine is at the ‘Prepare Statements’ stage.
  • Pairs of cells to represent each user role. Each role is associated with a poster cell describing the role, and next to it a ‘podium’ cell that acts as a place mark. The placemarks are used to enable the participants to jump to a description of their roles. These pairs are added at the ‘Assign Roles’ stage by the state machine.
  • A cell to represent a countdown timer. This is added and removed when necessary by the state machine.
  • A shared action cell, as described in the Debating Chamber application.

The CellMO that represents the Simulation Agora maintains references to the CellIDs of each of these children. For example the Simulation Agora CellMO has a field labelled ‘simulationDescriptionCellID’ which is a reference to the CellID of the instance of PosterCellMO that displays the description of the simulation. The fields are used as follows:

  • When the setSimulation() method is called on the Simulation Agora CellMO, it follows more or less the same pattern as the Debating Chamber, except that it also retrieves the template identifier from the simulation and creates a new instance of the state machine that represents that template.
  • When the moderator steps through the stages of the template, the CellIDs are used to identify the appropriate cells to delete or update, as described in the stages above.
  • From the client, when the user interacts with the Simulation Agora, the user votes and statements are sent from the Cell to the CellMO on the server via an instance of (a subclass of) CellMessage. From then on, the mechanism follows the same pattern as the Polling Booth and Debating Chamber.
  • When the +Spaces middleware reports a comment from Twitter or Facebook, it follows the same pattern as the Debating Chamber.

The other two tabs on the HUD control panel enable the moderator to inform the server when ‘something interesting happens.’ That is, something worth noting by the data analysis service; for example, when a video recording of the session is replayed, viewers are able to jump to appropriate timestamps to view the users’ activities (and listen to their comments). This is implemented using the regular OWL architecture—a message from the Simulation Agora cell to the OWL server which is then forwarded to the +Spaces middleware. All other communication between the +Spaces middleware, the OWL server and the OWL clients is as described in the two applications above.

Other +Spaces Functionality

In addition to the cells that provide the functionality for the Polling Booth, Debating Chamber and Simulation Agora, we also implemented a module with some extra features:

  • A means of reporting user activity to the +Spaces middleware:
    • two users come in range of each other (implemented using a client plugin)
    • a user ‘chats’ (using the text chat)
    • a user is forcibly disconnected
    • a user is forcibly muted
    • a user logs in or logs out
    • a user begins speaking
  • a servlet which provides a means of capturing profile information about a user (which is then forwarded to the +Spaces middleware)

Summary

In summary, we have developed 19 OWL modules for the +Spaces project, the following of which have been (or will be) donated to the OWL source code repository:

  • countdown timer
  • office-converter
  • postercontrol
  • tightvnccontrol
  • twitter
  • webcamcontrol
  • webcaster

However, we could not have completed any of the Wonderland aspects of the +Spaces project without the use of many of the existing modules and the wonderful support of the OWL community. So, on behalf of the project, I’d like to express my thanks to all the member of the community.


Wonderland Fall Showcase

October 16, 2011

Please join us in-world tomorrow for the 2011 Wonderland Fall Showcase. If you haven’t attended a showcase event in the past, this is an opportunity for the open source community to get together and demonstrate new features. This year’s showcase will also serve the dual purpose of testing the first release candidate for the new Wonderland release.

At the showcase, you will see a variety of features demonstrated, starting with general improvements to the user interface and menus. Then we’ll move on to showcase progress on EZMove, the current Wonderland Wednesday project that makes it dramatically easier to reposition objects in the world, and Subsnapshots, the completed Wonderland Wednesday project that makes it possible to export portions of world to reuse in other Wonderland worlds. We also have plans to show off seven or eight modules that have been recently added to the Wonderland Module Warehouse. These include the Office Converter, Styled Sticky Notes, Cardwall, EZClick, and the new poster and video modules.


Follow

Get every new post delivered to your Inbox.

Join 56 other followers

%d bloggers like this: