Big or Small, It’s All in Your Head: Visuo-Haptic Illusion of Size-Change Using Finger-Repositioning (CHI '24 Full Paper)

Myung Jin Kim, Eyal Ofek, Michel Pahud, Mike Sinclair, and Andrea Bianchi 

"Haptic perception of physical sizes increases the realism and immersion in Virtual Reality (VR). Prior work rendered sizes by exerting pressure on the user’s fingertips or employing tangible, shape-changing devices. These interfaces are constrained by the physical shapes they can assume, making it challenging to simulate objects growing larger or smaller than the perceived size of the interface. Motivated by literature on pseudo-haptics describing the strong influence of visuals over haptic perception, this work investigates modulating the perception of size beyond this range. We developed a fixed-sized VR controller leveraging finger-repositioning to create a visuo-haptic illusion of dynamic size-change of handheld virtual objects. Through two user studies, we found that with an accompanying size-changing visual context, users can perceive virtual object sizes up to 44.2% smaller to 160.4%larger than the perceived size of the device. Without the accompanying visuals, a constant size (141.4% of device size) was perceived."

doi | pdf

SpinOcchietto: A Wearable Skin-Slip Haptic Device for Rendering Width and Motion of Objects Gripped Between the Fingertips (UIST '22 Demo)

Myung Jin Kim and Andrea Bianchi 

"Various haptic feedback techniques have been explored to enable users to interact with their virtual surroundings using their hands. However, investigation on interactions with virtual objects slipping against the skin using skin-slip haptic feedback is still at its early stages. Prior skin-slip virtual reality (VR) haptic display implementations involved bulky actuation mechanisms and were not suitable for multi-finger and bimanual interactions. As a solution to this limitation, we present SpinOcchietto, a wearable skin-slip haptic feedback device using spinning discs for rendering the width and movement of virtual objects gripped between the fingertips. SpinOcchietto was developed to miniaturize and simplify SpinOcchio, a 6-DoF handheld skin-slip haptic display. With its smaller, lighter, and wearable form factor, SpinOcchietto enables users with a wide range of hand sizes to interact with virtual objects with their thumb and index fingers while freeing the rest of the hand. Users can perceive the speed of virtual objects slipping against the fingertips and can use varying grip strengths to grab and release the objects. Three demo applications were developed to showcase the different types of virtual object interactions enabled by the prototype."

doi | pdf

Thumble: One-Handed 3D Object Manipulation Using a Thimble-Shaped Wearable Device in Virtual Reality (UIST '22 Poster)

Changsung Lim*, Jina Kim*, and Myung Jin Kim*
(*All authors have contributed equally)

"Conventional controllers or hand-tracking interactions in VR cause hand fatigue while manipulating 3D objects because repetitive wrist rotation and hand movements are often required. As a solution to this inconvenience, we propose Thumble, a novel wearable input device worn on the thumb for modifying the orientation of 3D objects. Thumble can rotate the 3D objects depending on the orientation of the thumb and using the thumb pad as an input surface on which the index finger rubs to control the direction and degree of rotations. Therefore, it requires minimal motion of the wrist and the arm. Through the informal user study, we collected the subjective feedback of users and found that Thumble has less hand movement than a conventional VR controller."

doi | pdf | poster

SpinOcchio: Understanding Haptic-Visual Congruency of Skin-Slip in VR with a Dynamic Grip Controller (CHI '22 Full Paper)

Myung Jin Kim, Neung Ryu, Wooje Chang, Michel Pahud, Mike Sinclair, and Andrea Bianchi 

"This paper's goal is to understand the haptic-visual congruency perception of skin-slip on the fingertips given visual cues in Virtual Reality (VR). We developed SpinOcchio ('Spin' for the spinning mechanism used, 'Occhio' for the Italian word “eye”), a handheld haptic controller capable of rendering the thickness and slipping of a virtual object pinched between two fingers. This is achieved using a mechanism with spinning and pivoting disks that apply a tangential skin-slip movement to the fingertips. With SpinOcchio, we determined the baseline haptic discrimination threshold for skin-slip, and, using these results, we tested how haptic realism of motion and thickness is perceived with varying visual cues in VR. Surprisingly, the results show that in all cases, visual cues dominate over haptic perception. Based on these results, we suggest applications that leverage skin-slip and grip interaction, contributing further to realistic experiences in VR."

doi | pdf

Exploring Pseudo Hand-Eye Interaction on the Head-Mounted Display (AHs '21)

[Honorable Mention Award]

Myung Jin Kim and Andrea Bianchi 

"Virtual and augmented reality devices and applications have enabled the user to experience a variety of simulated real-life experiences through first-person visual, auditory, and haptic feedback. However, among the numerous everyday interactions that have been emulated, the familiar interaction of touching or rubbing the eyes is yet to be explored and remains to be understood. In this paper, we aim to understand the components of natural hand-eye interaction, propose an interaction technique through a proof-of-concept prototype head-mounted display, and evaluate the user experience of the prototype through a user study. In addition, we share insights emerged from the studies with suggestions for further development of interaction techniques based on combinations of hardware and software. "

doi | pdf

ElaStick: A Handheld Variable Stiffness Display for Rendering Dynamic Haptic Response of Flexible Object (SA ’20 E-Tech) 

Neung Ryu, Myung Jin Kim, and Andrea Bianchi 

"We present ElaStick, a handheld variable stiffness controller capable of simulating the kinesthetic sensation of deformable and flexible objects when swung or shaken. ElaStick is capable of rendering gradual changes of stiffness along two independent axes over a wide continuous range. Two trackers on the controller enable a closed-loop feedback that allows to accurately map the device’s deformations to the visuals of a Virtual Reality application."

doi | pdf

ElaStick: A Handheld Variable Stiffness Display for Rendering Dynamic Haptic Response of Flexible Object (UIST ’20 Full Paper)

[Best Paper Award]

Neung Ryu, Woojin Lee, Myung Jin Kim, and Andrea Bianchi 

"Haptic controllers have an important role in providing rich and immersive Virtual Reality (VR) experiences. While previous works have succeeded in creating handheld devices that simulate dynamic properties of rigid objects, such as weight, shape, and movement, recreating the behavior of flexible objects with different stiffness using ungrounded controllers remains an open challenge. In this paper we present ElaStick, a variable-stiffness controller that simulates the dynamic response resulting from shaking or swinging flexible virtual objects. This is achieved by dynamically changing the stiffness of four custom elastic tendons along a joint that effectively increase and reduce the overall stiffness of a perceived object in 2-DoF. We show that with the proposed mechanism, we can render stiffness with high precision and granularity in a continuous range between 10.8 and 71.5Nmm/degree. We estimate the threshold of the human perception of stiffness with a just-noticeable difference (JND) study and investigate the levels of immersion, realism and enjoyment using a VR application."

doi | pdf

BodyPrinter: Fabricating Circuits Directly on the Skin at Arbitrary Locations Using a Wearable Compact Plotter (UIST ’20 Full Paper) 

Youngkyung Choi, Neung Ryu, Myung Jin Kim, Artem Dementyev, and Andrea Bianchi 

"On-body electronics and sensors offer the opportunity to seamlessly augment the human with computing power. Accordingly, numerous previous work investigated methods that exploit conductive materials and flexible substrates to fabricate circuits in the form of wearable devices, stretchable patches, and stickers that can be attached to the skin. For all these methods, the fabrication process involves several manual steps, such as designing the circuit in software, constructing conductive patches, and manually placing these physical patches on the body. In contrast, in this work, we propose to fabricate electronics directly on the skin. We present BodyPrinter, a wearable conductive-ink deposition machine, that prints flexible electronics directly on the body using skin-safe conductive ink. The paper describes our system in detail and, through a series of examples and a technical evaluation, we show how direct on-body fabrication of electronic circuits and sensors can further enhance the human body."

doi | pdf

Aero-plane: A Handheld Force-Feedback Device that Renders Weight Motion Illusion on a Virtual 2D Plane (UIST ’19 Full Paper & Demo) 

Seungwoo Je, Myung Jin Kim, Woojin Lee, Byungjoo Lee, Xing-Dong Yang, Pedro Lopes, and Andrea Bianchi 

"Force feedback is said to be the next frontier in virtual reality (VR). Recently, with consumers pushing forward with untethered VR, researchers turned away from solutions based on bulky hardware (e.g., exoskeletons and robotic arms) and started exploring smaller portable or wearable devices. However, when it comes to rendering inertial forces, such as when moving a heavy object around or when interacting with objects with unique mass properties, current ungrounded force feedback devices are unable to provide quick weight shifting sensations that can realistically simulate weight changes over 2D surfaces. In this paper we introduce Aero-plane, a force-feedback handheld controller based on two miniature jet propellers that can render shifting weights of up to 14 N within 0.3 seconds. Through two user studies we: (1) characterize the users' ability to perceive and correctly recognize different motion paths on a virtual plane while using our device; and, (2) tested the level of realism and immersion of the controller when used in two VR applications (a rolling ball on a plane, and using kitchen tools of different shapes and sizes). Lastly, we present a set of applications that further explore different usage cases and alternative form-factors for our device."

doi | pdf

Wind-Blaster: a wearable propeller-based prototype that provides ungrounded force-feedback (SIGGRAPH ’18 E-Tech) 

Seungwoo Je, Hyelip Lee, Myung Jin Kim, Andrea Bianchi

"Ungrounded haptic force-feedback is a crucial element for applications that aim to immerse users in virtual environments where also mobility is an important component of the experience, like for example Virtual Reality games. In this paper, we present a novel wearable interface that generates a force-feedback by spinning two drone-propellers mounted on a wrist. The device is interfaced with a game running in Unity, and it is capable to render different haptic stimuli mapped to four weapons. A simple evaluation with users demonstrates the feasibility of the proposed approach."

doi | pdf

An Artistic Provocation to Explore Effects and Opportunities of Virtual Surreal Spaces (DIS '18 Provocations and Work-in-Progress)

Hyelip Lee, Myung Jin Kim, Byungjoo Lee, Andrea Bianchi 

"The concept of surreal virtual space is used in this paper to describe a space which looks realistic but is impossible to exist in reality. For this project, we developed a 3D virtual space using Google Cardboard and an Android mobile device. Referring to the 2D drawing, "Relativity," of M.C. Escher, the virtual space was designed to have multi-directional but connected stairs. This work was exhibited with other artworks at a gallery for a period of three weeks. Despite some minor sensory confusion, all audiences experienced a degree of place illusion, enjoyment and a sense of self-awareness even though the virtual environment did not provide a visual representation of the audience's own body. For future work, we plan to investigate the advantages of these effects and apply them to everyday non-virtual environments. "

doi | pdf