The convergence of sensor networks and virtual worlds is not only a possible solution to their respective limitations, but also the beginning of a new creative medium. In such a “dual reality,” both real and virtual worlds are complete unto themselves, but also enhanced by the ability to mutually reflect, influence, and merge by means of sensor/actuator networks deeply embedded in everyday environments. This paper describes a full implementation of a dual reality system using a popular online virtual world and a human-centric sensor network designed around a common electrical power strip. Example applications (e.g., browsing sensor networks in online virtual worlds), interaction techniques, and design strategies for the dual reality domain are demonstrated and discussed.
Dual Reality: Merging the Real and Virtual
by Joshua Lifton and Joseph A. Paradiso
MIT Media Lab
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.147.9419&rep=rep1&type=pdf&ei=nnx7UNu6GOnNmgXRyIHQAQ&usg=AFQjCNHNhmh-7jGxId3zyc7ryWjVSZZcdA&sig2=czjxMGq8Vzi_u9lAQoJrbw&cad=rja
At the heart of this paper is the concept of “dual reality,” which is defined as an environment resulting from the interplay between the real world and the virtual world, as mediated by networks of sensors and actuators. While both worlds are complete unto themselves, they are also enriched by their ability to mutually reflect, influence, and merge into one another. The dual reality concept, in turn, incorporates two key ideas — that data streams from real-world sensor networks are the raw materials that will fuel creative representations via interactive media that will be commonly experienced, and that online 3D virtual worlds are an ideal venue for the manifestation and interactive browsing of the content generated from such sensor data streams.
Sensor networks will turn the physical world into a palette, virtual worlds will provide the canvas on which the palette is used, and the mappings between the two are what will make their combination, dual reality, an art rather than an exact science. Of course, dual reality media will complement rather than replace other forms of media. Indeed, the end product, that which can be consumed and shared, is unlikely to outwardly resemble current forms of media, even if it is just as varied. Browsing the real world in a metaphorical virtual universe driven by a ubiquitous sensor network and unconstrained by physical boundaries approaches the concept of a digital “omniscience,” where users can fluidly explore phenomena at different locations and scales, perhaps also interacting with reality through distributed displays and actuators. Indeed, a complete consideration of dual reality must also include the possibility of “sensor” data from the virtual world embodied in the real world. Insofar as technically feasible, dual reality is bi-directional — just as sensed data from the real world can be used to enrich the virtual world, so too can sensed data from the virtual world be used to enrich the real world.