By Bernard Horan
Some time ago, I described work on an Open Wonderland webcaster that had been undertaken by one of our students at the University of Essex. One of the motivations behind the Webcaster was to provide access to those users who merely want to observe (and listen to) the activities in an Open Wonderland (OWL) virtual world instead of actively participating. In particular, we wanted to enable access to a set of users in the +Spaces project, such as policy makers, who wish to watch a session in which a policy is debated or acted out in a role play simulation and not be able to influence the session.
Since the first video-only version of the Webcaster, we’ve experimented with several different approaches to extend the Webcaster to provide audio, one of which is described below. After several unsuccessful attempts, we now have a successful prototype implementation that combines the video broadcast functionality with an existing OWL virtual phone object to provide a unified audio-video webcast.
The video above shows the OWL webcaster being used by two clients.
- One client is a regular OWL webstart client, into which a Webcaster ‘object’ has been inserted.
- The other client is a Web Browser that has two standard plugins: Flash and Java. The Browser opens a URL (provided by the Webcaster) hosted on the OWL web server and downloads the HTML content. The content includes a Flash object and a Java applet. (The Java applet is provided by doddlephone–a web-based SIP client.)
The deployment requirements are as follows:
- The SIP client requires that authentication is enabled on the OWL server
- The following ports must be open on the server host (and accessible through a firewall): 5080 and 1935.
The Webcaster requires further testing with more concurrent users before release to the OWL module warehouse–we’ll post a message on the forum when it’s ready. However, as ever, the source code is available in the unstable directory of the wonderland-modules SVN repository.
In this section I’ll briefly describe the video-only webcaster, the failed experiment at combining audio and video, and an idea for how the current implementation could be improved.
1. Video-only Webcaster
The video functionality of the webcaster requires a full OWL client to capture the graphics rendered by the in-world webcaster ‘camera’. The captured graphics are transmitted to a Red5 RTMP Server that is likely running on the same host as the OWL server. From there, lightweight clients can connect to receive a Flash ‘stream’ to play in a Flash player (or Web browser plugin). In this case, the firewall has to be configured to enable the OWL client to transmit graphics to the Red5 server, as well as for Flash clients to stream from the Red5 server. This architecture is represented by Figure 1 below.
2. Combining Audio & Video on the OWL Client
The OWL Client receives audio from the voicebridge that it plays through the client’s audio-out drivers, such as its speakers or a headset. The audio it receives is attenuated/spatialised according to the location of the avatar that is associated with the user of the OWL client, but does not include the audio from the microphone connected to the hardware on which the OWL client is running.
Our attempt to combine audio and video on the client first required us to mix the audio from the voicebridge with the audio from the client microphone and then combine that with the captured graphics, before sending it on to the Red5 server. From that point, the Flash clients would be ignorant of the change in architecture and would receive a combined audio/video ‘stream’ to play. This architecture is represented in Figure 2 below.
However, this approach failed due to several problems, including:
- The audio was received from the perspective of the avatar of the user associated with the client, NOT from the perspective of the webcaster object in the virtual world.
- The frequency of the received audio could be changed by the user, which made mixing with the audio from the microphone unstable.
- The extra processing that was required on the OWL client to mix the audio and then combine with the graphics was significant.
Thus, this approach was abandoned as infeasible. An alternative approach: to mix the audio on the voicebridge before sending to the OWL client was also considered. This suffered from the drawback that there was no existing mechanism to send the mixed audio to the OWL client–it would have required an additional connection through a firewall. Again, this was abandoned due the problems of managing ad hoc connections.
3. An Alternative Approach?
The current approach works around the audio problem by using an existing virtual phone object. However, we recognise that it is less than an elegant solution–we would still like to be able to provide a combined audio and video Flash stream to Web Browser clients. Ideally, we’d like the architecture as illustrated in Figure 3. Here the video is sent from an OWL client to the Red5 server where it is mixed with the spatialised audio that comes from the voicebridge, from the perspective of the webcaster. We have no idea if this would be possible, as it requires expertise in developing Red5 applications. However, if any readers would like to take this on as an experiment, please reply via a comment below. Or, if you have experience of Red5 and think this is a daft idea, let us know via the comments!