Wonderland Webcaster, part two

By Bernard Horan

Some time ago, I described work on an Open Wonderland webcaster that had been undertaken by one of our students at the University of Essex. One of the motivations behind the Webcaster was to provide access to those users who merely want to observe (and listen to) the activities in an Open Wonderland (OWL) virtual world instead of actively participating. In particular, we wanted to enable access to a set of users in the +Spaces project, such as policy makers, who wish to watch a session in which a policy is debated or acted out in a role play simulation and not be able to influence the session.

Since the first video-only version of the Webcaster, we’ve experimented with several different approaches to extend the Webcaster to provide audio, one of which is described below. After several unsuccessful attempts, we now have a successful prototype implementation that combines the video broadcast functionality with an existing OWL virtual phone object to provide a unified audio-video webcast.

The video above shows the OWL webcaster being used by two clients.

  • One client is a regular OWL webstart client, into which a Webcaster ‘object’ has been inserted.
  • The other client is a Web Browser that has two standard plugins: Flash and Java. The Browser opens a URL (provided by the Webcaster) hosted on the OWL web server and downloads the HTML content. The content includes a Flash object and a Java applet. (The Java applet is provided by doddlephone–a web-based SIP client.)

The deployment requirements are as follows:

  • The SIP client requires that authentication is enabled on the OWL server
  • The following ports must be open on the server host (and accessible through a firewall): 5080 and 1935.

The Webcaster requires further testing with more concurrent users before release to the OWL module warehouse–we’ll post a message on the forum when it’s ready. However, as ever, the source code is available in the unstable directory of the wonderland-modules SVN repository.

Technical Details

In this section I’ll briefly describe the video-only webcaster, the failed experiment at combining audio and video, and an idea for how the current implementation could be improved.

1. Video-only Webcaster

The video functionality of the webcaster requires a full OWL client to capture the graphics rendered by the in-world webcaster ‘camera’. The captured graphics are transmitted to a Red5 RTMP Server that is likely running on the same host as the OWL server. From there, lightweight clients can connect to receive a Flash ‘stream’ to play in a Flash player (or Web browser plugin). In this case, the firewall has to be configured to enable the OWL client to transmit graphics to the Red5 server, as well as for Flash clients to stream from the Red5 server. This architecture is represented by Figure 1 below.

Architecture for Video-only Webcaster

Figure 1

2. Combining Audio & Video on the OWL Client

The OWL Client receives audio from the voicebridge that it plays through the client’s audio-out drivers, such as its speakers or a headset. The audio it receives is attenuated/spatialised according to the location of the avatar that is associated with the user of the OWL client, but does not include the audio from the microphone connected to the hardware on which the OWL client is running.

Our attempt to combine audio and video on the client first required us to mix the audio from the voicebridge with the audio from the client microphone and then combine that with the captured graphics, before sending it on to the Red5 server. From that point, the Flash clients would be ignorant of the change in architecture and would receive a combined audio/video ‘stream’ to play. This architecture is represented in Figure 2 below.

Figure 2

However, this approach failed due to several problems, including:

  • The audio was received from the perspective of the avatar of the user associated with the client, NOT from the perspective of the webcaster object in the virtual world.
  • The frequency of the received audio could be changed by the user, which made mixing with the audio from the microphone unstable.
  • The extra processing that was required on the OWL client to mix the audio and then combine with the graphics was significant.

Thus, this approach was abandoned as infeasible. An alternative approach: to mix the audio on the voicebridge before sending to the OWL client was also considered. This suffered from the drawback that there was no existing mechanism to send the mixed audio to the OWL client–it would have required an additional connection through a firewall. Again, this was abandoned due the problems of managing ad hoc connections.

3. An Alternative Approach?

The current approach works around the audio problem by using an existing virtual phone object. However, we recognise that it is less than an elegant solution–we would still like to be able to provide a combined audio and video Flash stream to Web Browser clients. Ideally, we’d like the architecture as illustrated in Figure 3. Here the  video is sent from an OWL client to the Red5 server where it is mixed with the spatialised audio that comes from the voicebridge, from the perspective of the webcaster. We have no idea if this would be possible, as it requires expertise in developing Red5 applications. However, if any readers would like to take this on as an experiment, please reply via a comment below. Or, if you have experience of Red5 and think this is a daft idea, let us know via the comments!

Figure 3


2 Responses to Wonderland Webcaster, part two

  1. Nigel says:

    What a really interesting idea. Can see lots of applications for coaching, peer review of performance, role play feedback, marketing, dissemination…. What impact does the webcaster have on performance in the world? Does it limit performance for other world users?

    • Bernard says:

      Nigel, the webcaster adds a performance load to the OWL client that is rendering the image for communication to the red5 server, and I suspect the red5 server incurs a performance load when streaming out to multiple clients. However, other than that I wouldn’t expect any other performance issue (so it should not affect other OWL clients or the OWL server).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: