Previously, we touched on the affordances of a game controller (and a game’s control scheme) as an extension of the body. And we examined the process of getting into a game proprioceptively, in some depth: this is a mediation of two interfaces, the body–controller interface, and the controller–game interface. Successful proprioception involves compounding the two interfaces into one, such that bodily movements get mapped directly to on-screen outcomes. For example, instead of subconsciously thinking “right index finger presses trigger, trigger causes main weapon to fire”, the mapping becomes “right finger ? main weapon fire”, so that one is soon able to fire the main weapon without grappling consciously with the controller interface.
This calls to mind a sporting analogy: to make the racket/stick/sporting-instrument “an extension of your body”. This certainly is not an overnight process. One starts out getting used to the various sensations, of catching a ball at the wrong part of the swing, of angling the racket in various positions, of reaching with the racket while mid-stride, … then one starts compiling an experiential library of various scenarios—overhead balls, flat shots, smashes, and so on. (These stages often do not separate cleanly, but we can think of them as separate processes.) What follows then is a conscious analysis of one’s technique, thinking about rational responses to each situation, and then training the body to respond to these situations subconsciously. We can say this is a mapping of proprioceptive responses to sensed scenarios.
It readily follows, then, that each game has its own set of strategies, which calls for its own set of mappings. A response–scenario mapping that works for squash would not work for tennis or badminton. And it is a ready leap from this analogy to video gaming: different mappings for different games.
But there is a key difference here. The sportsman has “direct access” to his sporting weapon, while the videogamer’s experience is mediated through his perception of his avatar. How exactly does that work? How does a video gamer come to make their avatar “an extension of their body”?
Here I will focus primarily on two game types, classified based on the kind of camera angle that the player is given. One type has the player inhabiting the avatar’s point of view; the player sees what the avatar would be “seeing”. This is the first-person shooter (FPS). The other type has the camera above the avatar’s head or shoulder, always facing forward. This is the third-person shooter (TPS).
Immediately we are tempted to think that the FPS, being more similar to one’s personal perspective, would lend itself better to proprioception than a TPS. This has been occupying my mind for quite a few afternoons. I may well be wrong, and I welcome any thoughts on this, but I think they are distinctly separate kinds of experience (or qualia, if you prefer).
The first-person virtual experience #
In both cases, is the body capable of identifying with the presented avatars? If Mel Slater’s experiments are anything to go by, I say yes, in different ways. With a VR helmet on, people are able to identify their virtual hands as being under threat when a virtual knife is brought near1. When the camera is moved backward, such that they are able to see the entirety of their virtual avatars, they are able to identify their virtual selves as being under threat when a fan is lowered toward their head2. the latter are generally known as “out-of-body experiences”.
Why is this interesting? Because of experiments involving VR participants inhabiting entities that are dissimilar to themselves. In one experiment that involves:
Using a head-mounted display and body tracking suit, entering into a virtual reality, you can experience yourself as a child of about 4 years old. You look into a mirror, or directly down towards your own body, but you see the child body instead. The brain appears to be remarkably flexible in quickly accepting the proposition that your body is different – especially when you move your body the virtual body is seen to be moving the same as you feel yourself to be moving. The virtual body has substituted your real body.3
Let’s emphasise that again: The virtual body has substituted your real body. Why, that sounds like … possession. In most stories of possession that we hear, the possessor’s body (if there is one) is left behind, and the conscious mind completely inhabits the host body, losing all control over the original body, at least until the possession is undone. To be clear, I am not saying that VR participants are possessing their avatars; I am saying that it sounds very similar to possession in mechanism (but perhaps not in qualia, or quality of experience), and using it as a convenient label.
Now this is even more interesting, because if you possess a virtual avatar, it is not just a matter of you inhabiting a different body but still feeling like your old self. In the experiment, participants inhabited, in separate trials, the body of a child, and of a shrunk-down adult. Here I will just quote directly from the post:
In the two conditions (child or shrunk down adult) both overestimated sizes of objects, as expected. However, the child condition led to much greater size overestimation. It must therefore be not just the size but the form of the body that is having this effect.
And from the study abstract:
We conclude that there are perceptual and probably behavioral correlates of body-ownership illusions that occur as a function of the type of body in which embodiment occurs.
With my limited experience in reading neurology-related papers, I am guessing that means that one who inhabits a child’s body is more likely to behave like a child.
The third-person virtual experience #
From the second study:
There is a more severe contradiction discussed in the literature: you see your body some metres in front of you being tapped on the back, but you feel the tapping (of course) on your back (synchronously with the seen tapping). Here the contradiction produces a strange effect – somehow your body is over ‘there’ (the sensation of touch can shift to the body in front) but of course your visual ego-centre is ‘here’ (where you are really located). There can be a reported sensation of body ownership over that distant virtual body. But is this ‘body ownership’ in the sense of somehow feeling your body to be at the distant location, or is it just a question that you recognise the body as your body, and so in answer to a questionnaire would give a high score to a question about body ownership, but qualitatively this is not the same as when your virtual body is spatially coincident with your real body and seen from first person perspective?
When I read that description, the analogy that comes to mind is that of theatre. The tensions are similar: of wanting to inhabit a character, but not to the extent that one loses all sense of the old/original self. One extends one’s consciousness to include the portrayed character, but at the same time retains awareness of one’s personal state of mind and consciousness, “observing” one’s performance but not being completely immersed in it.
I’ve often wondered about theatrical puppetry. Unlike theatrical acting, where one uses one’s body as the avatar, there is an interface between the self and the avatar: the connected sticks and strings that control the puppet. How do these puppeteers “inhabit” the characters they portray? And is this qualia more similar to the TPS gamer’s qualia than the possessor’s?
The possessor and the puppeteer #
I am cutting things short here, because if I go on any further I will be speaking way beyond any experience (first- or second-hand), and beyond whatever I have read. So I’ll need to end here with this teaser. But please, feel free to chat me up on this, or to set me right on things if this is your area of research.
To sum things up with some clever wordplay:
1. In complete possession, one’s mind inhabits a new body, and takes on its unique qualia, losing almost all awareness (and qualia) of the previous body. FPS games employ this mechanism to a limited extent.
2. In complete puppetry, one’s body “acquires” a new mind (or think of it as a second “personality”, that of the avatar’s character), integrating its qualia without losing awareness of one’s controlling body—an “out-of-body” experience.
… in videogames? #
If you are a gamer, at this point you might be shaking your head slowly. You might even be saying to yourself, “what is this guy on, and can I have some?” I can assure you I am not under the influence of mind-altering drugs at the time of writing, and understandably modern-day games have not brought us to the brink of possession or puppetry yet.
But think about the application of VR technology to gaming, and what it heralds.
For various reasons (which I won’t detail; see the prologue of this series for some examples), today’s first-person shooters are not that immersive yet. Certainly not to this extent, at least:
But once we get there, the question will inevitably arise: how do we make video games more engaging, more immersive? So far, all the VR videogame demos (with the exception of the above experiments) I have seen are FPSes. But I think it quite likely that someday a game studio will publish a VR TPS title, and then we can really try out these arguments in person.
But beyond videogaming, the applications become even more interesting: with reference to Kevin Simler’s excellent Neurons Gone Wild, the potential of both perspectives for rehabilitative therapy, for novel out-of-body experiences (“digital drugs”), and for vacation alternatives, leaves much to think about.
- http://presence-thoughts.blogspot.co.uk/2014/11/what-happens-in-your-brain-when-your.html “What happens in your brain when your virtual body is threatened?” ↩
- http://presence-thoughts.blogspot.co.uk/2014/02/the-presence-of-your-distant-virtual.html “The Presence of Your Distant Virtual Body” ↩
- http://presence-thoughts.blogspot.co.uk/2013/07/presence-through-eyes-of-child.html “Presence Through the Eyes of a Child” ↩