Presence and Reality

What is Reality? Is there something out there, or are we all figments of your imagination? Science fiction fans may recognize these questions from movies like the Matrix or books such as Permutation City (G. Egan) and Neuromancer (W. Gibson), which have addressed this issue in a brilliant, thought-provoking and very entertaining way. But this question has now transcended both fancy and philosophy and is an active area of research with a potentially tremendous social impact.

What is becoming clearer to researchers is that what we call reality is simply a mental construct, an educated guess arising from the exchange of information between our minds and the environment. To be sure, science has always been concerned with the phenomenon of reality and the development of means to study it beyond our senses. The microscope, the telescope, radio astronomy, particle accelerators, remote sensing satellites, are examples of artificial means to extend our senses and thus produce an augmented, more thorough picture of what is out there. Moreover, there is a growing trend in physics today to view reality as, well, information. As asked last century by the physicist J. Archibald Wheeler, is It from bit? Or, more recently (Anton Zeilinger), “information is the irreducible kernel from which everything else flows […] the question why nature appears quantized is simply a consequence of the fact that information itself is quantized by necessity.” There are more and more hints today that information plays a crucial role in our notions of reality, even from the physical perspective.

But there is another way to study reality-building in the brain. The term Presence refers to an emerging research field seeking to understand the experience of being and developing technologies to generate and augment it (being someone or something, somewhere, sometime, with others and without physically being there). The central idea is that reality is a product of our brains: we construct reality in our heads from our (actively controlled) sensorial inputs. Ergo, if we “hack” the sensorial data stream, we can substantially alter our subjective perception of reality. Using new input/output technologies (audio, video, tactile displays and actuators, for example), a computer can “immerse” us in a virtual world shared with other real (avatars) or virtual people (agents).

But before we get into the technology, how can we measure such a subjective feeling? Presence is an elusive target because, as at its core, it refers to a qualia, a subjective experience. It is not possible to carry out direct scientific research with qualia. But we can ask experimental subjects about it (“Was it real and, if so, how much?”), observe their behavior, and objectively measure how the body responds to the tricks: are parameters such as heart rate, perspiration, respiration, brain activity (EEG, fMRI) in line with those of a real situation?

How do we hack a human brain in practice? We are still in the early days, and much work needs to be done. Cinema is a quite a crude but effective way to fool a brain, and, at least for a while, we may forget we are in a theater and be “there” in the film in some sense. But more powerful technologies are being developed, such as the CAVE: a room with computer-controlled wall/floor/ceiling displays which, thanks to tracking systems that know where we are and where we are looking, can fool us into seeing and hearing things that are not there—in 3D. Such display devices are still crude today, although it is surprising how effective they can be (a fact which is partially explained by the active side of sensing and the Bayesian nature of perception and cognition). Wearable, comfortable, transparent multimodal “display” technologies with bandwidths matching our own are still to be developed. Moreover, to effectively fool a brain we also have to manage its output (actions). In the CAVE example, one can physically move about or move a hand to “touch” something. Sensory-motor correlations (i.e., an agreement between what you do and how the environment reacts) are crucial for the phenomenon of Presence to occur, and difficult to maintain.

Clearly, one key aspects of this field is its inter-disciplinarity, requiring joint research in human cognition, human-machine interaction and machine intelligence. Machine intelligence is key, as the machine provides the background or even agency as well as the mediator in co-presence (being there with others).

As a technology, Presence is opening the door to study the human brain, partly because it allows for more controlled inputs and better measured outputs of the brain. It is being studied to study the relation of our brain to our bodies, because the body can be treated as part of the brain’s external environment itself. It is also a powerful technology, and as such it can be used in good or bad ways. Blurring the line between what is real and what is not will require responsible use and transparency. Yet, impact on society (e.g., reducing travel, enhancing learning and training and design) will be significant, as it can potentially allow us to radically shift viewpoints and learn more about ourselves, about living with others and about the universe that surrounds us.


What is Reality.pdf164.36 KB