AVIE: interior view showing T_Visionarium II
AVIEÌýis the world’s first mobile and scalable 360-degree 3D immersive and interactive visualisation system. It measures 10 metres in diameter and 3.5 metres in height. It includes a motion tracking system and a multi-channel audio system. Driven by an experimental framework, it uses a suite of cutting-edge hardware and software resources that enable the development of applications in AI visualisation, immersive environments and networked collaboration. With a number of these systems available, it is deployable on-site with industry and project partners for research development and application.
AVIEÌýhas been installed as core visualisation infrastructure at: ArtLAB ÉPFL (Switzerland); Chengdu Research Institute, Chengdu (China); CityU Hong Kong (Hong Kong); China Coal Technology & Engineering Group, Fushun (China); Experimental Media Performing Arts Centre, Rensselaer University, New York (USA); ZKM Media Museum (Germany); Mines Rescue Pty Ltd; Museums Victoria; Museum of Applied Arts & Sciences; UTS Sydney (all Australia).
It has been commercialised for the mine safety training industry by: Mines Rescue Pty Ltd at its Newcastle, Wollongong, Lithgow and Singleton facilities, Australia; Shenyang Research Institute of China Coal Technology & Engineering Group (CCTEG), Fushun, China.
AVIE: exterior view showing Nebula
AVIEÌýhas been developed though three iterations, with further development on-going. AVIE1 (2004-07) facilitated audio-visual data to respond kinaesthetically to the user's body across the full breadth of the projected space; AVIE2 (2008-2010) has augmented this environment by delivering the ability to recompose digital files in real-time; AVIE3 (2011-2017) has delivered a base-line AI Database system that enables digital agents to interact with users,based on their ability to interpret and respond to the user’s behaviour in space. AVIE provides a platform in which 3D audio-visual experiences are created by a combination of multiple projectors and multiple sound sources, driven by a cluster of computers, running a highly versatile real-time 3D VR engine. It comprises custom designed technologies: a Motion Tracking system, a Mixed Reality system and an AI Reasoning system. The AI system coordinates the actions of the visualisation system by allowing it to autonomously reason about its response to user behaviour. The Motion Tracking system tracks the physical location of users inÌýAVIE, while the Mixed Reality system allows the creation of 'scenarios' that exhibit a feedback loop between the world of the user and of the audio-visual data.
AVIEÌývisualisation forms
High-resolution panoramic stereo images with augmented 3D graphics overlays
High-resolution panoramic videos, including stereo videos
Immersive multimedia data visualisation
Real-time interactive 3D environments supported by artificial intelligence
AVIEÌýinteraction platforms
AVIEÌýutilises two types of interaction:
- remote device: a wireless tablet device allowing real-time selection and navigation within interactive applications
- motion tracking: real time software that detects, captures, tracks and interprets human movement and gestures, allowing capture and audience participation in the direct cinematic experience, interplay between users and digital characters and analysis of user behaviour
Below:ÌýInterior views showing Mining VR
AVIE Components
-
AVIE consists of a cylindrical silvered screen 3.5m high and 10 metres in diameter on which 3D panoramic images sequences and videos are projected. The 6 projectors use active shuttering technology to deliver stereoscopic 3D to end users, and the images from the individual projectors are seamlessly blended by custom software developed in-house at iCinema. 6 high-performance image generators, capable of processing images and videos as well as rendering large virtual environments in real time, are linked to the projectors to deliver the full immersive 3D experience. It accommodates up to 30 people at any one time.
External view
Projector set up -
This provides a multi-purpose sensory environment for unencumbered tracking of viewer movement and gesture, offering a large range of motion capture and interaction design functions. Covering the entireÌýAVIEÌýarea, the system is able to accurately track the spatial position and movements of up to 30 people. When applied to smaller groups of up to five people, it generates high-resolution 3D voxel models of their body movements and gestures, enabling real-time motion tracking and some gesture recognition.
Volumetric voxel tracking -
Directly complementing the unique visualisation and interaction features ofÌýAVIEÌýis its spatialised audio system. This is a 24 channel system with custom surround audio application software. This system enables fully immersive 360-degree placement of sound anywhere around the viewers. As with visual projection, the audio system can be coupled with the Intelligent Vision-Based Interaction and Motion Tracking Systems to allow voices to ‘follow’ projected characters, or to be activated only when viewers move within ‘earshot’ of the virtual audio sources, or correctly perform some task.