Vivian Shen, a doctoral student at Carnegie Mellon University Robotics Institute, utilizes a virtual reality (VR) headset with a phased array of ultrasound transducers that offer haptic input on the mouth to replicate the sensation of drinking on her lips.
While lips are famously sensual, their combination with the tongue and the gums are surprisingly sensitive – second only to the fingertips in nerve density. Carnegie Mellon University researchers have taken advantage of this to provide a new way for people to experience haptic feedback in virtual worlds.
The VR device
The CMU developed a device that uses airborne ultrasound waves to create sensations on the lips, teeth, and tongue. The device is compact enough to attach to the bottom of virtual reality (VR) goggles. A phased array of 64 small transducers makes up the CMU device. The featured flat, half-moon-shaped array is affixed to the bottom of VR goggles, immediately above the mouth.
Chris Harrison, associate professor of the HCII and director of the FIG Lab, collaborated with Shen and Shultz to create the system.
Shen discussed the team’s findings at the Association for Computing Machinery’s Conference on Human Factors in Computing Systems (CHI 2022) on Monday, May 2, where it earned the Best Paper award.
Shen and Craig Shultz used the device to generate haptic effects like crawling bugs, mud splatter, and raindrops while working with the Future Interfaces Group (FIG). Craig is a postdoctoral fellow at the Human-Computer Interaction Institute (HCII).
Research focus
According to Shen, the mouth is recognized for its responsivity, but scientists have had trouble putting haptic effects on it. VR users dislike putting objects in their mouths or covering their mouths.
Some VR devices are bulky and difficult to maneuver. Shen pointed out that one recent effort used a tiny robotic arm that could spray the lips with water or flick a feather across them but remain infeasible for general use.
On the other hand, researchers have employed ultrasound to send sensations to the hands, allowing them to generate haptic effects like virtual buttons that users can feel themselves pressing.
Ultrasound waves can travel through the air, but only at short distances. Therefore they appeared to be a viable option for mouth-based haptics, according to Shen.
Medical ultrasound imaging and probes are well-known, but they are not renowned for skin simulation or vibration. Focusing on these acoustic waves with frequencies exceeding the threshold of human hearing in a limited region makes it possible to induce sensations.
Multiple ultrasound-generating modules or transducers are used to accomplish this effect. Like any other wave, the ultrasound waves generated by one transducer can interfere with those of other transducers either constructively to amplify the waves or destructively nullify them. “You can get them all to interfere constructively at one point in space if you time the firing of the transducers right,” Shen said. This scenario focused on the lips, teeth, and tongue’s peak amplitude locations. The effect was additionally enhanced by subtly changing the ultrasonic output.
But, according to Shen, the sensations are mostly limited to the hands and mouth. “You can’t feel it anywhere else,” she noted. “Our forearms, our torso—those places lack adequate nerve mechanoreceptors to experience the feeling.”
The haptic effects
The haptic effects comprise swipes, point impulses, swipes, and continuous vibrations that are coordinated with visual pictures and targeted toward the mouth. Sixteen people were used to test various effects, and all the subjects reported that the mouth haptics improved their VR experience.
“It was challenging to tell when things were supposed to touch my face without haptics,” one participant said.
Not all of the impacts were equally effective. The ones that are mouth-specific were the most popular. Brushing one’s teeth, experiencing rainfall from an open window, or having an insect stroll across one’s lips are all examples. Others, such as the sensation of walking through cobwebs, were intriguing, but the effect was weaker since participants anticipated feeling those sensations throughout their bodies, not only in their mouths.
“Consider a VR world with a drinking fountain. Every time you lean down and assume you should be feeling the water, a stream of water comes across your lips,” Shen explained. “It’s fine. It greatly enhances the immersion of the event.”
Shen discovered that even drinking from a mouth-oriented water fountain can be unsettling. “It’s strange because you can feel the water, but it doesn’t feel wet.” Shen explained that “Our phased array finds a balance between being expressive and cheap.”
Scientists might have to do more work to expand the portfolio of haptic effects and make the VR gadget smaller and lighter.
Feature image credit: Vivian Shen, Carnegie Mellon University under CC BY 3.0 US
