By Dominik Alessandri & Christian Wietlisbach
Hochschule Luzern Technik & Architektur
As part of our bachelor thesis, we developed a module called ‘kinect-control’. This module lets you control your avatar by using your body doing gestures.
This module mainly runs on the client but necessary information will be transferred from the server to the client as needed. All the user needs to have is a connected Kinect device and the Kinect SDK Beta 2 installed. This means the module is only available for Windows x86/x64 clients. All other requirements will be shipped from server to client when logging in. This makes it easy for interested parties to use.
To set up a Wonderland server to use the kinect-control module, the administrator needs to change two files on the server:
/.wonderland-server/0.5/run/deploy/wonderland-web-front.war/app/win32/ wonderland_native.jar /.wonderland-server/0.5/run/deploy/wonderland-web-front.war/app/win64/ wonderland_native.jar
These two files must be replaced. The extended files can be downloaded from
These new files contain the DLL ‘KinectDLL.dll’ which is necessary for the connection between Open Wonderland and the Kinect.
If you are running this module on the client-side, the first thing you need to do is to connect the Kinect device to your Open Wonderland client. This can be done using the kinect-controller dialog. After installing the kinect-control module, in Wonderland, click on ‘Window -> Kinect Controller’:
The dialog contains two buttons: ‘Start Kinect’ and ‘Stop Kinect’:
You can adjust the angle of your Kinect device by sliding the slider to the desired position:
Now you are ready to move your avatar by using your body. Place yourself in front of the Kinect device and control your avatar as follows:
- Walk: Just move your legs up and down
- Turn right: Hold right arm to the right side
- Turn left: Hold left arm to the left side
- Fly up: Hold right arm up
- Fly down: Hold left arm up
It is possible to extend the gestures recognized by this module. For this, you need to modify the file ‘gesturesBDA.txt’ located in ‘kinect-control-client.jar’ inside ‘kinect-control.jar’ using the software ‘KinectDTW‘. After this file contains your new gesture, you need to map this gesture to a keyboard-input.
The file ‘keyMapping.txt’ contains the allocations from gestures to keyboard-inputs. It is located on server in /.wonderland-server/0.5/run/content/modules/installed/kinect-control/client/. The structure of the file is as follows:
[Name of gesture]=[isMouse[0/1]];[keyCode1[decimal]];[keyCode2[decimal]];[time[millis]]
@Run=0;16;87;2500 Description: When gesture @Run is recognized, press key 16 (shift) and 87 (w) 2.5 seconds long.
@Walk=0;87;0;3000 Description: When gesture @Walk is recognized, press key 87 (w) for 3 seconds long.
You will need the following files to get the kinect module running:
- kinect-control.jar (http://220.127.116.11:8080/modules/KinectControl/kinect-control.jar)
- wonderland_native.jar (x86) (http://18.104.22.168:8080/modules/KinectControl/win32/wonderland_native.jar)
- wonderland_native.jar (x64) (http://22.214.171.124:8080/modules/KinectControl/win64/wonderland_native.jar)
A video of the running module can be seen on YouTube: