A grand challenge for Presence

UPDATE!

Deadline extended to Dec 31st 2007

In the interest of gathering more ideas, we have decided to extend the deadline one month.

Remember: The winners of the Contest will be selected by the Peach team. The first price will be an iPod touch, and the 2nd and 3rd,  iPod Shuffles.

Although many interesting research lines in Presence are presently being pursued, it is apparent that it would be beneficial for the field today to focus community efforts on a Grand Challenge in core science and technology of Presence. Presence research lies at the intersection of human perception and cognition research, machine intelligence, and human-machine intelligence, and as such it needs contributions from a wide variety of fields. The goal of the field is to develop science and technology to achieve successful replacement/interaction (i.e., presence, being there) and open up a wide range of powerful applications. We ask here contribution of ideas for a Grand Challenge project with a 5 to 10 year horizon addressing the key objectives and questions in Presence while enabling powerful applications.

Join us and provide your ideas here

Such a project should be at core objective of Presence:  producing measurably "real" experiences through sensorial hacks: replacement (or augmentation) and interaction with "bits". Should be ambitious, hard, medium to long term (e.g., ~10 year horizon with 3 year milestones say). It should engage the needed communities in Human Cognition, Human-Machine Interaction and Machine Cognition around a focal problem.

Finally, should a project should lead to major advancement in Presence Theory, Experiment and Technology, and result in major positive social impact. 

A panel session to address the future research lines of Presence was organized as part of Presence 2007 (Oct 26th 2007, Barcelona, Spain). The  summary of  the discussion are being posted here, and the floor is open to the community for comments and new ideas.

Attached Files:

  1. Panel proposal
  2. Martin Buss slides
  3. Giulio Ruffini slides
  4. Panel summary
AttachmentSize
PEACH-GCP-Presence2007.pdf280.26 KB
Presence2007_MartinBuss_PanelPresentation.pdf644.78 KB
GiulioRuffini-PGC-Oct2007.pdf1.92 MB
PGC-20071102.pdf807.23 KB

For completeness, here is a

For completeness, here is a list of the winners of the first grand challenge:

  1. John Waterworth from the Umea University
  2. Giuseppe Riva from the Insituto Auxologico Italiano
  3. Maria Victoria Sanchez - Vives from the ICREA - IDIBAPS

This was originally reported here: http://www.starlab.info/peach/?q=node/164
Giulio Ruffini, Starlab

Grand Challenge

My Self, my Other Selves and the Selves of Others

John Waterworth, Umeå University, Sweden.

[Note: this is a first, very rough, quick and dirty draft, 30 Nov 2007]

The Challenge

To systematically implement and experiment with singular and multiple first, second and third person virtual representations of self, varying factors such as degree of body-virtual image coordination, sensory-motor coupling and visual similarity (amongst other factors) and assess the impact on mediated sense of presence (by means of triangulations of introspective, behavioural and neuro-psychological data). A particular focus for the interpretation of results would be quantum shifts in the quality of presence in response to the manipulation of such independent variables. Theory building would be achieved through progressive model development and hypothesis testing.

First, Second and Third Person Presences

A kind of first person presence is the norm in classic VR, in which we view the virtual world as if we were embodied there ourselves (to some degree). We move our physical head and the virtual view changes accordingly; we move our physical arms and hands and we see a representation of these body parts depicted as if they are collocated with our internal image of our physical body. But just how important is this collocation? Some studies of “dextrous” work in VR suggest that collocation is not a very strong factor in accurate task performance (although of course hand eye coordination is) whether one is working in two or three dimensional space. Why is that? One reason is that we seem to be very adept at dealing with mappings of bodily actions onto the behaviour of tools, as long as the behaviour of the tool is closely coordinated with movements of the body. This is how we can use a computer mouse, drive a car, or fly a remote control model aeroplane. How does body-virtual image collocation affect our sense presence? Is it perhaps the distinction between being present as ourselves versus being present with ourselves?

Increasingly we see ourselves represented in the third person in social virtual spaces, but generally not in a realistic way, and with minimal body-virtual image coordination – as when mouse actions or arrow buttons control gross movements and pre-programmed gestures of our avatar. (There are also examples of first person perspective social spaces (distributed VRs) too, but these tend to be restricted to a limited projected own body (e.g. just an arm or a gun)). In these social spaces we can usually choose the appearance of our virtual persona from a selection of avatars or avatar parts. And these social spaces do give us a degree of co-presence, even though we are looking at ourselves from the outside, as a third person self amongst the third person selves of one or more other people. What happens if our physical body is closely coordinated with that of the avatar? This happens increasingly in animation movies, though obviously not in real time. What happens when a person’s virtual third person avatar (or robot) closely mimics the bodily and facial changes of the physical person in real time. Is there a sudden shift in the quality of presence? How does the realism of the depiction affect my sense of self and of presence? In other words, do I feel more present if my avatar looks and behaves like me, and how does this compare or perhaps interact with degree of body-avatar coordination?

Second person presences of others are also coming along, though usually not corresponding to other real people. Examples include virtual characters that help us do things or entertain us, and more or less personable robots. Would these characters be improved by having a sense of presence (do they perhaps have one already)? Does that require or follow from having a sense of self? And what does or would implementing a sense of presence in virtual personalities tell us about presence in general? Note that having a sense of presence is not the same as having an emotional response system.

As far as I know, there are no second person, interactive and virtual representations of self (I may be wrong). This would be the case where I can interact with a virtual characterization of myself, and which – as with third person self representational avatars – could be more or less like my physical self. If the virtual image (or even robot) is coordinated with my body, it would be somewhat like looking in a more or less distorting mirror. Whether left-right reversed or not could, of course, be a matter of choice, as could many other aspects of body/virtual image mapping. How would this affect the sense of self, and of presence with our selves? How would second-person body mapping compare with first and third person body mapping? This would be of interest to various therapeutic applications, including, for example social phobias and physiotherapeutic approaches to stroke recovery.

When do we want a first, second or third person perspective on our selves and on others

What are the effects on presence? What happens when I multiply my selves – when many avatars respond as me in a variety of social situations? What does all this tell us about the synthesis of presence?

Summarising some Main Questions to be Addressed

  1. How does body-virtual image collocation affect our sense presence?
  2. What would implementing a sense of presence in virtual characters tell us about presence in general?
  3. What happens if our physical body is closely coordinated with that of a third person avatar?
  4. How does second-person body mapping compare with first and third person body mapping in, for example, therapeutic applications?
  5. What happens when I multiply my selves – when many avatars respond as me in a variety of social situations?
  6. When do we want a first, second or third person perspective on our selves and on others
  7. What would a theory of mediated presence that accounted for our findings on the above look like?

 

Building Presence Tools through Competitive Collaboration

Strengthening Presence as a Concept and Field:

A Grand Challenge Idea to Build Useful Tools through Competitive Collaboration

Presence, as a concept and field of inquiry, seems to be largely disjointed and in great need of a cohesive set of codes and contributors.  The Grand Challenge (GC) presents an opportunity to orient Presence researchers from various disciplines toward a common constructive goal, but the GC should be carefully constructed in order to facilitate the greatest advancement of the field that is possible.   Specifically, the Grand Challenge should invite scholars from all related fields not only to compete but also to collaborate on developing two fundamental areas of Presence research: manipulations and measurements. The purpose of this suggestion for a Grand Challenge is not to describe the specific research that should be conducted, but instead to propose how the GC should be structured to induce beneficial collaboration within an environment of competition. 

Relativity of Presence

Presence is not an all-or-nothing, virtual or real phenomenon.  Instead, the intensity of Presence may fluctuate throughout a mediated experience and may be split between numerous mediated experiences.  You may be playing an excellent video game, completely involved in the experience, when something in your physical environment changes, such as a mosquito biting you on the arm, and your Presence in the game is momentarily reduced.  Perhaps it is reduced to the extent that you barely miss beat in the game, or perhaps you accidentally send your avatar to a heinous death.  Regardless, the mosquito drew your attention to the physical world to some extent, thereby decreasing your presence in the game and increasing your presence in the real world.  Building on this argument, if it had been a virtual mosquito, e.g., your computer screen flashing with an instant message, perhaps your presence would have decreased in the video game and increased in the instant-message virtual environment.  The point is that the degree of Presence experienced by an individual is dependant on the stimuli (or lack of stimuli) presented within all salient environments in which the individual is situated.  Hence, the degree of Presence an individual experiences in any given environment can be influenced by manipulations of the stimuli in all salient environments.  By examining how such manipulations influence Presence, it follows that tools can be developed to measure the degree of Presence that is experienced within any given set of environments.

Manipulation and Measurement

While it may seem appealing to focus the Grand Challenge on developing methods to induce the greatest amount of Presence possible, such an endeavor would be inextricably limited by current levels of technological advancement.  This would be a task for engineers, not for social scientists. Instead, the fundamental point of the GC should be to develop both controlled manipulations and broadly applicable measurement tools of Presence.  One way to achieve this goal would be to create an infrastructure for researchers to work on either or both of these areas and collaborate on their projects. 

There is an age-old conundrum associated with this type of measurement-oriented scientific development.  In order to develop a measurement tool, the object of measurement must be divided into discrete and equal amounts.  But, in order to divide something into discrete and equal amounts, it must be measured.  This recursive problem is often overcome by using some older system of measurement to create a basic unit for the new system.  For example, the kilogram is defined as being equal to the mass of one liter of water (with a small margin of error due to modern measurement tools).  Another solution to the problem is to pick a seemingly arbitrary base unit, e.g., the length unit of a foot purportedly was calibrated to the length of King Henry 1’s foot.  Regardless, developing controlled manipulations and a standard measurement system of Presence would be no easy task.  In order to do so, researchers from various disciplines would need to develop such manipulations and measurements independently and then collaborate in order to calibrate their tools.

The Two Areas of the Grand Challenge

The Grand Challenge should be split into two areas of research.  The first area should be comprised of researchers who are developing technologies that facilitate controlled manipulations of Presence.  For example, a virtual environment could be created in which visual or haptic feedback can be easily increased or decreased on a sliding scale.  The second area should be comprised of researchers who are developing technologies that attempt to measure the level of Presence experienced in a given environment.  For example, an EEG cap could be calibrated to record the levels of brain activity that occur when stimuli are presented in one of two different environments (perhaps one virtual and one real), allowing for the relative strength of activity to be interpreted as the relative degree of presence in a given environment. These are simple examples of the types of projects that could be pursued, but the actual projects could span a much wider range of fields and disciplines.

Collaborative Sessions

The heart of the Grand Challenge would be the collaborative sessions that occur at two- or three-year intervals, culminating in a final session at the 10-year milestone.  During these sessions, participants in the Grand Challenge would meet and match their projects with every group from the alternate area.  In other words, each group from the first area (developing controlled manipulations of Presence) would test their technology by using the measurement tools developed by the every group in the second area.  Likewise, each group that is developing a measurement tool would test these tools on the technologies developed by groups in the first area (inducing controlled manipulations of Presence).

The logistics of matching certain groups would most likely be complicated.  For example, some technologies might be mobile while other might be large and fixed, so assembling all groups with their relevant technologies at one place and time would be difficult.  Perhaps the collaborative sessions would not need to occur at the same location, but instead during a particular time frame.  For example, there could be a six-month period in which every group must find some way of meeting and comparing technologies with each group from the alternate area.  These sessions could be timed so that groups could report their results at the ISPR conference.

The most important facet of the GC would be that each team would be able to interact and exchange ideas with every other team in the alternate area.  This type of exchange is the lifeblood of innovation and would most likely lead to the development of an excellent set of technologies that induce controlled levels of Presence and can measure such levels of across various environments. By structuring the GC in this manner, the onus of developing interdisciplinary teams would not fall on the individual teams, but on the GC itself. 

Identifying Winners

Ideally, there would be some sort of prize to motivate participation that would be awarded to the two teams, one from each area, with the most advanced and reliable technologies.  The winners would be the teams that had the strongest association with the largest number of teams from the opposite area.  This assessment would most likely be conducted quantitatively.  For example, a composite factor of the effect sizes for the statistically significant relationships between the manipulations and measurements of Presence could be calculated for all matches between groups.  The group with the highest value for this factor, within each area, would be the winner.  If developing such a statistical tool to determine the winner were not possible, a panel of judges could be assigned to rate the quality of each team’s product.  Of course, the quantitative approach would be preferred because it would be more objective and also because the intent of the GC would be to develop a quantitative system of measurement for Presence.

Outcomes

While the fine details of this suggested Grand Challenge are still somewhat undeveloped, it should be clear that such a GC would greatly advance the field of Presence.  Although the nature of the GC would be competitive, the structure of the competition would facilitate an unparalleled exchange of ideas and research within the field of Presence.  This exchange would facilitate the development of innovative Presence-related tools that could be used by future researchers to establish Presence as a universally applicable concept.  Similarly, this GC would unify the area of inquiry by aligning researchers from various disciplines toward a common goal, thereby ensuring that the field does not disperse and dissolve, but instead coheres and thrives.  Perhaps such speculations overstep the parameters of the present endeavor, but at the very least, this Grand Challenge would be a step in the right direction.

--------------------------------------------------------------------------------------------------------------

Full text of the submission can also be found here.
-----------------------------------
Rabindra (Robby) Ratan
PhD Candidate
Annenberg School for Communication
University of Southern California
3502 Watt Way
Los Angeles, CA 90089

ratan@usc.edu

My proposal for the Grand Challenge: Virtual Embodiment

VIRTUAL EMBODIMENT

 

Synthetic Environments for the Modification of the Bodily Self-Consciousness


In the science-fiction thriller The Matrix, the heroes "plugged in" to a new generation virtual world for training. While their bodies rested in reclining chairs, they learned how to fight martial-arts battles, drive motorcycles and helicopters in an elaborately constructed software program.

Differently from typical virtual environments, that simulates reality, the Matrix environment simulates bodily self consciousness: a subject can acquire a complex behavioral skill – driving a plane – by experiencing through a somatic simulation the behavioral processes related to the skill.

Main objective of this project is to replicate this approach: its goal is the design and development of a method and/or a tool (EMBODIMENT device) able to modify and simulate our most "internal" experience: bodily self consciousness. Specifically, the Embodiment Device will use haptic and virtual reality tools to produce a controlled sensory rearrangement able to simulate a different passive and active body.

In this sense, the project aims, in the short/medium term, to develop a device that can alter in a controlled way the experience of the body (1-3 years). The data and the experience gained could allow, in the long term, both the simulation of the experience of a given body (3-5 years) - e.g., an obese subject could experience a thin body to increase his/her motivation to change – and the development of Matrix-like motor training tools (10-15 years).

Specifically, the goals of the project are:

  • Identifying what kind of experiences (mental and perceptual) can modify and reproduce the somatic experience of the body: In doing this the project will investigate both the characteristics of somatic awareness and the features of actual tool and experiences that can influence these characteristics.
  • Developing an embodiment procedure or device able to modify the actual body experience of the subject.
  • Exploring the possible use of the embodiment procedure/device for real world applications. In particular three possible areas will be investigated: training, psychological rehabilitation, and entertainment.
  • Testing the efficacy of the embodiment device at a scale of operation representing reality.

The full text of the proposal is available here.

------------------------------------
Prof. Giuseppe Riva, Ph.D.
Applied Technology for Neuro-Psychology Lab.
Istituto Auxologico Italiano
Via Pelizza da Volpedo 41
20149 Milan
Italy

web sites:

http://www.ambientintelligence.org
http://www.emergingcommunication.com

Sample Comment

All community members can post comments and ideas here.

Giulio Ruffini, Starlab