Video in Wonderland using MJPEG and VLC

When we created the video player for Wonderland we took what we thought was the most expedient approach by using the Java Media Framework. This turned out to be a real challenge. Not only was JMF no longer being maintained, but a lot of its behavior was bizarre and obscure. So, I was very happy to hear that a team at Ericsson in the Netherlands had built a video player that doesn’t use JMF but uses an open source video library instead. This is just a small part of an incredibly compelling Wonderland project from Ericsson that we’re hoping will be showcased in the "Cool stuff" category at JavaOne later this year.

Here’s a technical overview of the video player by Erik Reitsma, who seems to have a really fun job at Ericsson!


Several people on the mailing list and forum have asked about streaming video in Wonderland. Apparently some people have it working with specific webcams, but generic streaming of video is harder. Since also we liked to stream video from a webcam, we bought an AXIS webcam. Not the 212 PTZ, but a cheaper one, the AXIS 207 . And I could not get it working with the existing video player in Wonderland.

Instead of buying a more expensive webcam, I looked into what was really happening. The current video player uses JMF to decode the incoming video stream, and takes snapshots of the video. These snapshots are then drawn on a surface in the 3D world. So probably there was something going wrong in the JMF part. Since JMF is no longer developed or supported, I started looking for alternatives.

Our AXIS 207 can stream its video in two formats: MPEG-4 and MJPEG (http://en.wikipedia.org/wiki/Motion_JPEG). I would not think of writing an own MPEG-4 decoder in Java. However, MJPEG is basically a multipart stream, containing one JPEG image after another, plus some headers and boundaries. This is quite easy to decode, and it is not hard to put the JPEGs as a texture in a 3D surface.

So what I did was adapt the existing JMF video application to get images from a (pure Java) MJPEG decoder and render the received JPEG images. I used the MJPEG decoder from Cambozola (http://www.charliemouse.com:8080/code/cambozola/) for this, but it would not be too hard to write one from scratch. This simple solution works surprisingly well. The only remaining issue was, that we could only play MJPEG streams, and then even only MJPEG streams that were accessible through HTTP. Now that I had this working video player for our webcam, I wanted to use it to play other videos too.

Fortunately there exists an open source video player that can do just what I was looking for: VLC media player (http://www.videolan.org/vlc/). VLC media player is a highly portable multimedia player for various audio and video formats (MPEG-1, MPEG-2, MPEG-4, DivX, mp3, ogg, …) as well as DVDs, VCDs, and various streaming protocols. But it can also be used as a server to stream audio and video, in many different formats. One of these formats is MJPEG over HTTP, just what I needed for my video application. This means that I can stream any video that VLC can play, as an MJPEG stream, and play it in my video application in Wonderland.

I use the following command to serve a movie xyz.mpg as a stream on port 8050 as (accessible as http://localhost:8050/abc.mjpg):

vlc xyz.mpg –no-sout-audio –sout=’#transcode{vcodec=mjpg,vb=512,fps=5,width=640,height=480}:standard{access=http,mux=mpjpeg,dst=:8050/abc.mjpg}’ –input-repeat=10000

Parameters such as width, height and framerate can be varied. I repeat the movie 10000 times, so that it runs in a virtually endless loop.

A disadvantage of this approach is that the video is streamed using a quite inefficient codec. I have considered running VLC on the same host as the Wonderland client, and controlling VLC from the Wonderland client. However, the current solution is good enough for my purposes.

Here’s a nice mixed reality example which shows the video player in action:

Video player

Erik Reitsma, Ericsson, The Netherlands

About these ads

10 Responses to Video in Wonderland using MJPEG and VLC

  1. Ric Moore says:

    Are you currently doing work in version .05 yet? I’d love to see this in action. Thanks for doing the Good Work! Ric

  2. Peter Wyss says:

    I would be also very interested to see this in action in wonderland 0.5! Peter

  3. Erik Reitsma says:

    I would also like to see this in 0.5, of course. However, I have been focusing on other issues for now, which are a bit more independent of the version of WL (audio of shared applications, for example).
    Another thing is, that we are looking for internal funding to continue our work on WL. So, I cannot make promises.
    I have actually not yet looked at what it would take to implement this in 0.5.
    *Erik.

  4. hmmm would also like to see this in 0.5, of course

  5. video says:

    yees this like 0.5

  6. Ric Moore says:

    Can what worked in 0.4 work in 0.5? Video/Audio and PDF viewing would set me straight! Thanks! Ric

  7. Nigel Simpson says:

    Good news! I’ve just ported Erik’s webcam application to Wonderland 0.5. Many thanks to Erik and Jan van der Meer at Ericsson for contributing the code!

  8. Adam Myatt says:

    Nigel, where can we find the module you mentioned you ported (available in the daily trunk builds, unstable modules, etc)?

  9. Nigel Simpson says:

    Hi Adam,
    I’m just waiting on legal approval to release the module. It’ll appear in wonderland-modules as a stable module when it’s released. I’m hoping this will happen soon.
    Nigel

  10. Nigel Simpson says:

    I released the Wonderland 0.5 webcam module today:
    http://forums.java.net/jive/thread.jspa?threadID=68164
    To check out the source:
    svn checkout https://wonderland-modules.dev.java.net/svn/wonderland-modules/trunk/0.5/stable/webcamviewer webcamviewer –username yourusername
    Thanks for your patience!
    Nigel

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 54 other followers

%d bloggers like this: