Jon Kaplan and I attended the 2008 Computer-Supported Cooperative Work conference (CSCW 2008) in San Diego, CA. last week. Sun was a benefactor-level sponsor of the conference, having donated a dozen computers to run a 3-day long virtual worlds event. This event was run and organized by a crack team from PARC made up of Mike Roberts, Greg Wadley, and Nic Ducheneau (not pictured).
Each day they featured a different virtual world – World of Warcraft on day 1, Wonderland on day 2, and Second Life on day 3. Each morning, one of the organizers ran a tutorial on the virtual world of the day, which attendees paid extra for. In the afternoon, anyone attending the conference could drop in and work through the tutorial materials on their own.
Jon loaned a hand getting some small kinks worked out of the internal Wonderland server setup and the two of us acted as TAs during the tutorial session. There were a total of 10 client machines and two participants per machine. The first part of the tutorial involved exploring the world and using shared apps. The second part of the tutorial involved building a world with world builder and trying it out. Mike had an individual server installation set up on each client machine for this second part, so everyone could start with a completely empty world of their own.
I guess I shouldn’t admit to being surprised at how smoothly all this went. Performance was awesome with the local server. The 0.4 software proved to be quite stable. There was not a single crash, or even any noticeably bad software glitches as people stopped, started, and fought for control over every shared app in the world.
This virtual world event was happening in parallel with the rest of the conference. The opening plenary was delivered by Cory Ondrejka, one of the Second Life co-founders. Cory no longer works at Linden Labs, but he talked about how the company got started. His talked focused on how Linden Labs arranged their work environment and he attributed much of the success of the company to the decision to use the software internally for all distributed collaborations. This "eat your dog food" approach is certainly one that the core Wonderland team lives by as well, so hopefully he’s right about that being an important ingredient for success.
The rest of the conference was heavily weighted towards social science research, with a number of presenters talking about ethnographic studies (ones in which researchers observe the "natives") of inhabitants of virtual worlds. If you’re interested in this sort of thing, all the papers are available in the ACM Digital Library.
Rather than describe in detail papers you can read for yourself, let me instead tell you about the virtual worlds workshop, "Virtual Worlds, Collaboration, and Workplace Productivity," that Jon and I helped to run. The position papers for the workshop are available from the workshop site.
The first session of the workshop involved three fairly in-depth "demos." Jon showed clips from 4 or 5 different Wonderland videos and talked through the compelling parts like extreme extensibility, the ability to connect to external data sources, standards-based scripting, and fine-grained security.
Steve Rohall from IBM did a video demo of a virtual world called Bluegrass that he and his colleagues built using the Torque engine. This world has some interesting features, including using streams of floating bubbles to help users see at a glance where there is activity in the space. There is also nice integration with the corporate directory so you can find out more information about others in-world.
The third demo was also from IBM. Wendy Ark showed how IBM is using Second Life as a "rehearsal" space in their Rehearsal Studio project. By rehearsal, they mean preparing to give a talk or practicing for a consulting engagement. The example she showed involved a training environment for consultants where they modeled a fictitious manufacturing company and had the consultants-in-training work through an entire customer engagement over the course of two days, complete with about a dozen actors playing the role of various company employees. This was quite a compelling application of virtual world technology.
After the demos, everyone participating had a chance to present a quick overview of their position papers. The papers divided nicely between system-oriented papers and research studies. I presented some thoughts we’ve been having about how to improve virtual world presentations.
The most interesting part of the workshop from my perspective was the discussion afterward in which we talked about what data collection features those studying virtual world usage would ideally want baked into the virtual world software. Here’s a summary of the list we came up with:
Where’s the activity?
- Number of users over a certain period of time
- Number of simultaneous users over time
- Heat maps showing accurate traffic data
- Record of applications used and for how long
- Avatar locations and proximity to other avatars
- Number of people in "ear shot" of a talking avatar
- Number of times control passes in shared applications
- Number of avatars in proximity to an application being shared
- Record of which interactive objects or applications were used
- Automated video recordings of certain spaces
- Record events to play back later and view from different perspectives
If you’re interested in virtual world evaluation, I have put this list on a data collection wiki page so others can add to it. What logging, data collection, or data visualization features would you like see built into Wonderland? I also highly recommend reading a paper written by Jonathon Cummings (Duke) and Sara Kiesler (CMU) called "Who Collaborates Successfully?" In this paper, the authors study 500 NSF projects that involve collaborations. The paper includes a framework for evaluating collaborations which I believe is ideally suited to helping us evaluate virtual world collaborations.
Let me end by including a fun photo of my former colleague, Bo Begole (left) and his fellow conference organizer David McDonald, who both did a great job in organizing this smooth-running conference.