Flatworld: Wide Area Mixed Reality System

Flatworld room with people

The FlatWorld project at the University of Southern California Institute for Creative Technologies was developing an innovative immersive virtual reality environment that allows users to freely navigate simulated rooms, buildings, and urban spaces. At its core, FlatWorld utilized a system of "digital flats" - large rear-projection screens that display computer-generated imagery to depict interior spaces, exterior views, or building facades.

By integrating physical props and set pieces with these digital displays, FlatWorld creates a "mixed reality" environment that seamlessly blends virtual and real-world elements. For example, a physical door frame placed in front of a digital flat allows users to open a real door and step through to view a computer-generated exterior scene. This approach draws inspiration from Hollywood set design techniques and theme park attractions to enhance the sense of presence and immersion.

The digital flats are modular and reconfigurable, allowing FlatWorld environments to be rapidly assembled in open spaces to simulate a variety of geographic locations and scenarios. Scenes displayed on the flats can represent real-world settings with high fidelity, such as accurately depicting buildings and urban layouts of Middle Eastern cities.

Moving beyond just visual displays to create a multi-sensory immersive experience, FlatWorld incorporate immersive spatial audio, haptic floor speakers to simulate vibrations, and other "4D" effects like strobe lighting, fans, and even scent generators. These sensory elements are tightly synchronized with the visual displays to create convincing environmental effects like explosions or weather events.

Enabling this complex integration of visual, audio and sensory elements is the FlatWorld Simulation Control Architecture (FSCA). This software framework allows environment designers to orchestrate all aspects of the simulation, including real-time graphics rendering, sound playback, effects triggering, and responses to user interactions with physical props. The FSCA utilizes a network-centric design with a central controller application broadcasting messages to coordinate multiple client nodes handling graphics, audio, tracking, and effects.

Later enhancements to FlatWorld have expanded its display capabilities beyond just rear-projected digital flats. Projector-camera systems are now being employed to enable front-projection onto arbitrary surfaces like walls and furniture. Computer vision techniques compensate for geometric distortion and surface properties, allowing any surface to become an interactive projection screen. This facilitates effects like simulated bullet impacts or destruction on real objects in the environment.

FlatWorld provides a flexible platform for immersive training and simulation that eliminates the need for head-mounted displays. Users can freely walk through large-scale virtual environments, interacting naturally with a mix of physical and virtual elements. The modular design allows rapid reconfiguration to simulate diverse scenarios. By leveraging both cutting-edge display technologies and time-tested set design techniques, FlatWorld creates uniquely compelling and effective immersive experiences.

Video

Two minutes long preshow video, played for visitors before FlatWorld tour.

Jarrell Pair gives a FlatWorld tour in 2006.

Flatworld as in 2008: projection on regular wall (bullet holes), helicopter shadow on the floor, body tracking computer vision system.

Publications