Assessment in Virtual Worlds

August 12, 2011

By Diego Morillo, Carlos Delgado Kloos, Maria Blanca Ibañez, José Jesús García Rueda

diego morillo

Diego Morillo

I am part of a group that builds Multi-User 3D Virtual Environments as a method of learning using Open Wonderland. Our group belongs to Gradient, the e-learning laboratory inside the GAST group (The Telematics Applications and Services Group), which is part of the Department of Telematic Engineering at the Carlos III University of Madrid (Spain).  To learn more about our work, please visit:  http://wonderland.gast.it.uc3m.es/

World of Questions

3D virtual worlds provide a rich environment where students can not only learn but also be assessed on what they have learned.  3D virtual worlds provide a promising environment where one can assess:

  • Knowledge – the outcome of the assimilation of information through learning
  • Skills – the ability to apply knowledge and put to use the know-how to complete tasks and solve problems, and
  • Competency – the proven ability to use knowledge, skills, and personal, social and/or methodological abilities.

In our Multi-User 3D Virtual Environments for Learning group, we developed an assessment engine for Open Wonderland called “World of Questions” in collaboration with the University Pompeu Fabra.   The overall goal of the assessment engine is to develop an embedded assessment tool where 3D virtual world elements can be used and/or incorporated into any part of a question and the students can either add new content to the virtual world or modify existing content.  To enhance usability, we designed the capability to add content from external resources, like XML files, thereby giving educators the option of creating new tests or modifying existing ones in a quick and easy way.  Another feature of the assessment engine is the ability to create assessment content in a manner that enables educators to adapt their existing assessment tests and assessment items without the need to create new ones. Below is a brief demo of the assessment engine.

Technical Implementation

In defining an assessment mechanism in a 3D virtual world, how questions will be shown in the world and/or how students answer them is very important.  In addition, how the questions will be defined and stored is also vitally important.  We think the best solution to address the latter is the IMS Global Learning Consortium QTI (Question and Test Interoperability) specification because it defines a format for the representation of assessment of content and results.  We employ an XML data binding to define the data model which defines the structure of the questions, the assessment and results.

Because 3D virtual worlds introduce new interactions that are not considered in QTI, the specification cannot be used directly in Open Wonderland.  To enable use of the specification to define questions in a virtual world, we created a new XML file that is related to the QTI XML file, so that we have two files to define a question:

  1. QTI XML file that defines the question following the specification (it could be used in a traditional learning management system); and,
  2.  A new file, the Wonderland-QTI (WL-QTI) file, that includes all the information needed in the virtual world and that which cannot be defined in QTI: the actions that students can do in the world to receive the instructions of a question or to answer it, and the ways in which they receive the information.

New APIS, (Assessment Provision through Interoperable Segments), a QTI engine v2.1, manages the QTI XML file.  This engine also processes the answers of the students and stores their marks.

Among the different types of questions that the QTI specification allows us to define, we have implemented the most frequently used “multiple-choice” questions in World of Questions. In this type of question, students must choose one answer among different possibilities. We defined four different actions that students have to do in the virtual world to give an answer (or to receive the instructions of the questions): select an object; select an NPC (non-player character); access a zone; or talk with an NPC using  AIML (Artificial Intelligence Markup Language).  AIML is an XML-based programming language that relates linguistic patterns with templates that contain the response to user inputs (see Picture 1).

Conversation among an avatar and an NPC

Picture 1: Conversation among an avatar and an NPC

When students initiate action that begins execution of a question, the instructions can appear in several ways: a text or an audio message, or an NPC can prompt them using the AIML language. When students answer, they receive feedback. This feedback can be shown in the same way as the instructions or it can produce a change in the virtual world. For example, an object can appear or disappear, or an image can be inserted (see Picture 2).

Image in the virtual world

Picture 2: Image in the virtual world

This assessment engine will be improved to implement more types of questions and include tools to facilitate teachers in creating worlds and the XML files that define the different questions.

This work was presented at the “Congreso Iberoamericano de Informática Educativa” that took place in Chile in December, 2010. (http://www.ie2010.cl/posters/IE2010-56.pdf).

More about the authors:

To contact the authors and learn about the project, please visit the team page at: http://wonderland.gast.it.uc3m.es/participants.htm .


Follow

Get every new post delivered to your Inbox.

Join 56 other followers

%d bloggers like this: