Wonderblog

When we created the video player for Wonderland we took what we thought was the most expedient approach by using the Java Media Framework. This turned out to be a real challenge. Not only was JMF no longer being maintained, but a lot of its behavior was bizarre and obscure. So, I was very happy to hear that a team at Ericsson in the Netherlands had built a video player that doesn’t use JMF but uses an open source video library instead. This is just a small part of an incredibly compelling Wonderland project from Ericsson that we’re hoping will be showcased in the "Cool stuff" category at JavaOne later this year.

Here’s a technical overview of the video player by Erik Reitsma, who seems to have a really fun job at Ericsson!

Several people on the mailing list and forum have asked about streaming video in Wonderland. Apparently some people have it working with specific webcams, but generic streaming of video is harder. Since also we liked to stream video from a webcam, we bought an AXIS webcam. Not the 212 PTZ, but a cheaper one, the AXIS 207 . And I could not get it working with the existing video player in Wonderland.

Instead of buying a more expensive webcam, I looked into what was really happening. The current video player uses JMF to decode the incoming video stream, and takes snapshots of the video. These snapshots are then drawn on a surface in the 3D world. So probably there was something going wrong in the JMF part. Since JMF is no longer developed or supported, I started looking for alternatives.

Our AXIS 207 can stream its video in two formats: MPEG-4 and MJPEG (http://en.wikipedia.org/wiki/Motion_JPEG). I would not think of writing an own MPEG-4 decoder in Java. However, MJPEG is basically a multipart stream, containing one JPEG image after another, plus some headers and boundaries. This is quite easy to decode, and it is not hard to put the JPEGs as a texture in a 3D surface.

So what I did was adapt the existing JMF video application to get images from a (pure Java) MJPEG decoder and render the received JPEG images. I used the MJPEG decoder from Cambozola (http://www.charliemouse.com:8080/code/cambozola/) for this, but it would not be too hard to write one from scratch. This simple solution works surprisingly well. The only remaining issue was, that we could only play MJPEG streams, and then even only MJPEG streams that were accessible through HTTP. Now that I had this working video player for our webcam, I wanted to use it to play other videos too.

Fortunately there exists an open source video player that can do just what I was looking for: VLC media player (http://www.videolan.org/vlc/). VLC media player is a highly portable multimedia player for various audio and video formats (MPEG-1, MPEG-2, MPEG-4, DivX, mp3, ogg, …) as well as DVDs, VCDs, and various streaming protocols. But it can also be used as a server to stream audio and video, in many different formats. One of these formats is MJPEG over HTTP, just what I needed for my video application. This means that I can stream any video that VLC can play, as an MJPEG stream, and play it in my video application in Wonderland.

I use the following command to serve a movie xyz.mpg as a stream on port 8050 as (accessible as http://localhost:8050/abc.mjpg):

Parameters such as width, height and framerate can be varied. I repeat the movie 10000 times, so that it runs in a virtually endless loop.

A disadvantage of this approach is that the video is streamed using a quite inefficient codec. I have considered running VLC on the same host as the Wonderland client, and controlling VLC from the Wonderland client. However, the current solution is good enough for my purposes.

Here’s a nice mixed reality example which shows the video player in action:

Erik Reitsma, Ericsson, The Netherlands

This entry was posted on Monday, February 2nd, 2009 at 12:46 pm and is filed under Technical. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.