The first part of our effort is the development of the Mixed-Reality Tabletop (MRT). The MRT is a concept where a mentor and a remote mentee can simultaneously use the 'same' tabletop.  This is accomplished through a camera capturing an image of the tabletop and sending it to the remote table, and a projector displaying the image received of the remote table.  "Real" objects are defined as those that are physically located on the local table, and "virtual" objects are those located on the remote tabletop.

The MRT system requires significantly less infrastructure, compared with a fully immersive virtual reality system.  The users should be unaware of the computer's operations, and remain focused on manipulation of the real and virtual objects on the table.  The MRT does not require prior scripting or complex object modeling for the user prior to a session.  Instead, specific applications are created for the needs of the collaborative session.  Applications are built upon the MRT Toolkit, which provides a basic set of services that allow for connection, video transmission, camera and projector calibration and synchronization, and compositing of application-defined content overlays.  There are two versions of the MRT Toolkit – one is built upon Microsoft Research's ConferenceXP, which allows multiple participants to video conference and add other capabilities to the system, and the other is custom built from the ground up for speed by only allowing a two-point connection and no audio services.

The second part of our effort is an ongoing desire to create and deploy mixed-reality applications using pen-based computing devices. In particular, we have been developing an undergraduate course curriculum that allows computer science students to explore non-traditional human-computer interfaces, to implement complex working systems, and to give them hands-on experience in developing team projects.

Mixed-Reality Tabletop (MRT): A Low-Cost Teleconferencing Framework for Mixed-Reality Applications. D. Bekins, S. Yost, M. Garrett, J. Deutsch, W. Htay, D. Xu, D. Aliaga, IEEE Virtual Reality, short paper, 2006.

A Collaborative Undergraduate Course for Pen-based Computing using Tablet PCs. D. Aliaga, G. Rodriguez-Rivera, D. Xu, Workshop on the Impact of Pen-based Technology, 2006.

Portable Mixed-Reality. Poster presented at Microsoft Faculty Summit, Seattle, WA 2005.

Mixed-Reality Tabletop. Poster presented at Microsoft Faculty Summit , Seattle, WA, 2004.

Mixed-Reality Tabletop. PowerPoint presentation for Microsoft Faculty Summit , Seattle, WA, 2004.

Interactive Classroom Application (Video, 93MB)

Interactive Physics Application (Video, 67MB)

This project is generously supported in part by Microsoft Research and the Learning Experience Team (http://www.conferencexp.net) and the Microsoft Research Tablet PC Curriculum Group.