Tweek: Merging 2D and 3D Interaction in Immersive Environments

Abstract

Developers of virtual environments (VEs) face an oftendifficult problem: users must have some way to interact with the virtual world. The application designers must determine how to map available inputs (button presses, hand gestures, etc.) to actions within the VE. As a result, interaction within a VE is perhaps the most limiting factor for the development of complex virtual reality (VR) applications. For example, interactions with large amounts of data, alphanumeric information, or abstract operations may not map well to current VR interaction methods, which are primarily spatial. Instead, twodimensional (2D) interaction could be more effective. Current practices often involve the development of customized interfaces for each application. The custom interfaces try to match the capabilities of the available input devices. To address these issues, we have developed a middleware tool called Tweek. Tweek presents users with an extensible 2D Java graphical user interface (GUI) that communicates with VR applications. Using this tool, developers are free to create a GUI that provides extended capabilities for interacting with a VE. This paper covers in detail the design of Tweek and its use with VR Juggler, an open source virtual reality development tool.

Keywords

• JavaBeans
• Java
• CORBA
• C++
• Virtual Reality