Take My Hand: How We 'Touch' in Virtual Worlds

By Alan Wen on at

In simple terms, Ico is a game about trying to escape a formidable castle. But what makes the Playstation classic so memorable is the image and the feeling of a boy and girl holding hands as they find a way out together. ‘Hand-holding’ may have acquired a more pejorative meaning in games, but the act of holding hands remains a rare sight, a physical and proactive action from the player that creates a bond.

When asked to define the core qualities of Ico in a Glixel interview, one of the few words creator Fumito Ueda responded with was “eroticism.” At first, it seems like a mistranslation — it’s hardly erotic in say the soft-porn way of The Witcher 3 or the cringey manner of Heavy Rain’s QTEs — but perhaps it’s a reminder that public displays of affection like holding hands are sometimes frowned upon in conservative Japan. We may even think decades back to Gumpei Yokoi's Love Tester, a simple electronic machine of no scientific accuracy, the selling point of which was that a couple had to hold hands to activate it. But overlooking such cultural differences, it’s still rare for games to include any playable form of physical interaction that isn’t a violent one.

Physical interactions between people can differ depending on gender, culture and relationships, something that has fascinated Foam Sword’s programmer Moo Yu. “When I was growing up, even hugs were very rare to come by, especially with my father as it just wasn't appropriate in his culture,” he says. Coming from an Asian background myself I can instantly relate to what Yu's saying: some families, my parents’ generation among them, tend not to be so physically affectionate.

Nonetheless, as social animals, physical connection between humans is immensely important and takes a wide range of forms: from the formal handshake, to embracing friends, a peck on the cheek, a hearty backslap or a high-five. If this is so ingrained in us, shouldn’t that also be expressed in the games we play?

Yu is currently working on Knights and Bikes, a co-op action adventure game about two girls (Demelza and Nessa) exploring their home island town. One of the ways their close friendship is expressed is through high-fives, which functions as a healing mechanic. “It was important for us to capture the idea that sometimes to get better, you need someone else to help you along the way, a theme expressed throughout the game,” says Yu. “We liked high-fives in particular because it was such an energetic interaction that was fitting for our two high-energy girls.” And the repetition of such an act, one would guess, becomes a tangible part of the characters' bond.

VR is a medium even more concerned with simulating physical interactions, and of recent note is Polyarc’s debut game Moss. Initially looking like a third-person platformer, what’s integral is the emotional bond that’s created between the player and Quill, the anthropomorphic mouse you adventure with.

Animation director Rick Lico explains that being able to reach in and touch Quill was important from the start. “Because when you’re inside VR, that’s the most natural form of interaction you can have — reaching into the world, and grabbing things, and touching things.”

There is of course a snag in getting the feel of these interactions right: the feedback, or lack of it. There were some tricks Polyarc employed to work around VR’s limitations, such as the subtle haptic feedback you get from the DualShock when you grab onto Quill to heal her, mimicking her heartbeat, while a canned animation also has her taken in the air curled in a ball like you’ve just picked her up. Adding to her personality, she also gets annoyed if you repeatedly grab her when she’s not in need of healing.

A sense of weight is also an issue, so the game's animations were designed to look weighty and have an in-built element of lag. “You’d kind of pull and there’d be this tether onscreen between where your hand really is and where the character is that implies you’re influencing the world, kind of like Patrick Swayze in Ghost,” says Lico. Now there's a VR game waiting to happen.

Quill may exhibit anthropomorphic traits like standing on her hind legs, but she’s still very much a mouse and part of the game's charm is the physical charm of petting her. The heroines of Knights and Bikes also have a pet goose, the winningly named Captain Honkers, whose presence is entirely about allowing players to pet it. In fact, it’s more common for players to want to get touchy-feely when there’s an animal. Is it because it’s easier to express physical affection for animals than it is for humans in games?

The DS and iPhone has of course made touch-controls a part of play. But where petting a Nintendog with the stylus may seem naturally adorable, petting mini-games featured in Japanese games like Hatsune Miku: Project Diva or Fire Emblem Fates — the latter’s feature removed in the Western release entirely — can be not just outright creepy, but also explicitly gamey (and those are just the tamest examples).

“I think it’s challenging to execute because of the uncanny valley,” says Lico, bringing the topic back to VR. “Everyone on this planet interacts with other humans on a daily basis, so what a human does and how they act, and little details in the face, they’re all ingrained in everybody. So to replicate that in a VR environment is immensely challenging.”

If developers are able to overcome the uncanny valley — something Epic is currently working on with its digital actors — Lico sees its powerful potential in VR. “Imagine a story moment, where the character that you’re interacting with turns around to go face their certain death, and you reach out and grab that character’s hand, see the way that they respond to that touch, then you grab stronger onto their hand, and try and convince them not to go, or you let them go and you can see them walk off, and realise the decision you’ve made,” Lico says. “That moment is the pinnacle for me - it’s like the merging of story, or interactivity and touching the human spirit — and I’m looking forward to getting there and being a part of it.”

But if creating bonds with digital creations and NPCs seems too artificial, then what about our interactions with other players? If more people are playing games online to hang out with their buddies in a virtual social space — albeit to shoot, stab and punch one another — shouldn’t these spaces also have a means for social niceties?

Like any action RPG game, Sloclap’s Absolver is focused on combat, but it’s also about practising martial arts, where it’s important to create respectful and positive relationships between players. One important aspect is the ability to revive one another, whether friend or foe. As creative director Pierre Tarno explains, “We needed the revive mechanic for gameplay purposes, but we wanted the actual animation itself to be positive and intimate. We thought that physically reaching out and helping your opponent to their feet was a good way to illustrate that: it is a gesture of goodwill, and creates a bond between players.”

Whereas we’re used to gloating over one another after a PvP duel, getting your arse handed to you by another player only for them to take your hand to restore you to full health, before helping you to dispatch other foes in the world, is something special. Absolver still has the odd dishonourable cur, but has generally fostered a respectful community, where, for example, players use emotes to bow before fights, which is just the mindset the developer envisioned.

The widespread use of emotes just shows how much player expression is valued. Games like Destiny and Overwatch even make many of them rare or available through microtransactions, while it’s not uncommon to see Dark Souls phantoms spamming various gestures in gleeful celebration of jolly co-operation, or perhaps an impromptu conga line while PUBG is loading. But what stops developers from going the extra mile and turning that into something more tangible?

For a small indie studio like Slocap, the answer is fairly simple. “We have to focus on the essentials,” says Tarno, when asked if, instead of just bowing, there could be something like handshakes. “It's true that it would be very cool to be able to shake hands with your opponent, but it is quite complicated to implement. Online players need to be synchronised, their animations have to match, and all this has to be interruptible. Since this isn't core to the experience, we couldn't afford to spend too much time researching this.”

What about the big studios? As with the rest of Polyarc’s founders, Lico previous job was at Bungie, where he was the animation lead for the first Destiny. He did in fact create a number of emotes that did simulate physical interactions, such as secret handshakes and fistbumps. However, they weren’t strictly physical, as players could glitch through each other, while animations were also not in sync.

“Players would have to trigger it at the right time, at the right distance from one another for that to line up properly,” he explains, though it turned out that this shortcoming also made it popular as a metagame. The downside was that once the metagame had been solved, the emotes were used much less, and don’t appear to have carried over to Destiny 2.

Unlike a revive or healing mechanic, there’s also of course no actual gameplay function for these moments of physical contact. They're cosmetics. That may sound superficial but, just as IRL, these kind of gestures are meaningful. But games are also hard to make and when you have a tonne of other critical issues to handle, it’s perhaps understandable that 'shaking hands properly' doesn’t come high in the priority.

There's also the perennial online problem of being careful what you wish for. “Do you need a hug?” is one of Orisa’s voice lines in Overwatch, but she doesn't mean it in a nice way. What if you could hug your fellow teammates? That would require more than just syncing up animation and physicality between character models, but also mutual consent from players, a potential minefield for player toxicity. Not even taking voice chat into account, these are after all environments where the crouch button is used for teabagging or where salty players spam ‘Thanks’ to make it mean anything but. When unwanted attention is already an issue in gaming, especially for women players, adding 'nice' elements to a game may produce outcomes that are anything but.

“We talked about that a lot in Destiny,” says Lico. “That’s one of the reasons you don’t see anything like that, because we never wanted one person to apply will over another person and build it into a community of griefing.” Before leaving Bungie, he did toy with the idea of including passive interactions that would happen automatically if you were standing together with your fireteam, which could still reinforce a sense of camaraderie without getting in the way of gameplay.

We began with Ico, and it's telling that Fumito Ueda’s career began in animation, where creating an emotional story comes through meticulous attention to detail and understanding of movement and body language. You see this quality in all of Team Ico's other games. Lico doesn’t think most developers are yet prioritising these kinds of interactions, but as an animator he believes it's the future for creating meaningful characters in games.

“The further we go down this road of really defining a character, there’s only so much you can do with the way a sword swings. So I think animators are going to need to start looking more at character intent and interaction with the world as opposed to just representing avatar actions.”

“Games like Ico have been an inspiration for us,” finishes Lico. “And I’m pretty sure other developers are going to come to the same conclusions.”