Haptic – Adding a New Dimension to Virtual Reality

Introduction

The word haptic refers to the sense of touch, the sense felt by humans upon touching an object. In technical terms, haptic refers to the sense of touch through feedback technology while interacting with computers. Feedbacks provided by computers are in the form of vibrations, force, and motion. This increases experience of users while dealing with computer-simulated situations.

Haptic technology works by combining the human part and the machine part. The machine part consists of sensors, actuators, computers, real-time algorithms, application program interface, and the end effector, wherein the computer acts as the brain of the entire system and sends appropriate commands to the human part through the end effector.

The Exhibit 1 depicts the basic working principle of the haptic system, wherein when the end effector is activated by the human, sensors in the device measures various parameters, such as angle of movement, force applied, pressure applied, etc. The computer analyzes the data using real-time algorithms and provides the desired output in the form of torque commands to motion devices. Users will receive responses from motion devices placed in the end effector, thus completing a cycle.

Advancements in Haptic Technology

The use of haptic technology is not new for the industry; however, earlier, its adoption was very limited, and the technology was mostly used as tele-manipulation systems. The technology was very costly, and hence, it was utilized for critical applications, such as handling of radioactive materials. The first (known) force-reflecting robotic manipulators were used at the Argonne National Lab, US, in 1954, wherein the haptic technology was based on a master-slave tele-manipulation system.

The Exhibit 2 below showcases the evolution of haptic technology over the years. First-generation haptic devices used electromagnetic technologies, such as vibratory motors, which operate at resonance to provide feedback. The feedback (mostly vibration) provided by such devices was limited, as it was offered to the whole device instead of a touch/point-specific or location on the device.

The limitations in the first generation haptic devices were overcome in the second generation, as these devices provided ‘touch-coordinate specific responses.’ The use of electroactive polymers, piezoelectric, electrostatic, and subsonic audio waves improved the actuation technology in the second generation haptic devices, which enabled the touch/point-specific response. In the third generation, haptic technology delivered both, touch-based specific responses as well as customized haptic effects. Customized haptic effects were necessary for a multi-touch environment, wherein when multiple inputs are provided, the haptic device should be able to recognize and compute, to render the necessary feedback effect. The customization of haptic effect was achieved using multiple actuators connected through low latency network. While the fourth-generation haptic technology enabled pressure-sensitive haptic responses, this technology enabled haptic devices to use and analyze the force exerted by a user into consideration while delivering haptic feedback. The device provided multi-level feedbacks based on the force exerted by the user. This technology was mostly adopted in surface haptics. The future generation of haptic technology will eliminate the need for physical devices to sense virtual objects, and this will be termed as ‘Ultra Haptics.’ The technology will manipulate ultrasound waves, which can be felt by the user.

Continuous advancements in actuation technology, haptic rendering technology, etc., have enabled innovation in haptic devices. The haptic device technology evolved from being a tele-operational haptic system to a graspable haptic system (with VR) to touchable haptic devices, and finally, to the current level of wearable haptic devices.

Emergence of Wearable Haptics

Advancements in haptic devices can be attributed to the continuous sophistication of virtual reality technology. Haptic provides real world-like immersion and interaction features to VR, and hence, complements its features by providing a lifelike experience.

The evolution of haptic technology, discussed in the previous section, is with respect to the actuating technology, feedback effect, computation technology, etc. However, haptic technology is also advancing with respect to its devices (refer Exhibit 3). The convergence of haptic and VR in the form of wearable haptics is set to transform two major industrial processes, namely, product design & development and industrial training.

In the product design & development phase, wearable haptics could play a major role in activities, such as product ergonomics, process ergonomics, and assembly and disassembly (A/D) simulation. The use of wearable haptic technology for such activities provides in-depth functioning abilities to the product. This, in turn, will reduce the need for multiple physical prototyping of the product.

Industrial training processes can also be benefitted by using wearable haptic devices. Such training provides lifelike experience to users in a safe, offline environment. It also increases the safety risks of the user, in addition to eliminating concerns regarding damage caused to a high-value product due to operator error. Thus, haptic technology converged with virtual reality in the form of wearable haptics provides innumerable benefits to the industrial sector. Some of the areas in industrial training processes that wearable haptics can play a major role are listed as follows:

• Safety Training
• Maintenance Training
• Mechanical Training
• Industrial Machining Training
• Material Handling Equipment Training
• Process and Operations Training
• Assembly Line Training

Wearable Haptics in Industrial Sector

The haptics technology is widely used and accepted in the commercial sector, especially in the gaming industry; however, it has gradually started to foray into the industrial sector. This is due to the limitations of haptic technology with respect to industrial use-cases. Current complexities such as virtual modeling and rendering of industrial equipment are major reasons for its slow adoption in the industrial sector. However, research & development activities of wearable haptics for industrial uses are in progress. Some of the major companies currently working on wearable haptics are Valkyrie Industries, Exiii Inc., HaptX Inc., Go Touch VR, etc.

Valkyrie Industries is currently developing a haptic VR suit, which the company is referring to as Iron Man V.1. The suit uses electrical impulses to stimulate muscles, effecting approximate resistance and touch to the user. The product is in its early stages; however, the company has already demonstrated the product in the market. It is expected to find its applications in training and tele-operation purposes to control and manipulate objects from a safe distance.

Exiii Inc., a Japanese manufacturer, has developed Exos, a wearable haptic device that offers tactile feedback of virtual objects. The device lets the user to touch, hold, and grasp virtual objects in the virtual space. It is a wearable exoskeleton device that moves muscles and tendons to produce the sensation of touch and feel. The company’s focus is towards developing haptic devices for the manufacturing sector. It is presently working on creating haptic devices for the automotive and heavy machinery industries.

HaptX, a leading haptic company in the US, has developed HaptX Gloves, a wearable haptic glove that uses microfluidic technology that mimics human touch. Haptic gloves come along with a VR headset, tracker, and a central control box. The company is targeting manufacturing industries for workforce training and product designing applications. HaptX has collaborated with Nissan Design to bring realistic touch to vehicle design.

GoTouch VR, founded in 2016, is also working on developing a wearable haptic platform to interact with virtual 3D objects. The company provided virtual reality haptic training solutions to Schneider Electric, allowing users to get trained in high-risk working scenarios. A real-time control solution of industrial robots was developed for Altran, a global leader in Engineering and R&D services, with the support of GoTouch VR. This helped Altran gain real-time control of robots through touch, thus enabling unsupervised control of robot from a safe environment. The company implemented TOUCHABLE platform through VR touch device to enable touch sensation of various aerospace-specific interaction for Flyinside, a US-based company developing and commercializing VR flying simulators.

The Next Wave of Haptic Technology

In the long-term, haptic technology is expected to move from wearable devices to handheld haptics and mid-air haptics. Handheld haptics supports large-scale body movement and eliminates limitations or challenges of wearable haptic technology, such as weight of the device, complex wiring in the device, constrained hand movement, etc. Handheld devices are slated to be easy and effective and can be used by merely picking it up. However, localizing the diverse requirement of virtual object’s senses such as vibrotactile patterns, texture feedback, thermal feedback, skin-stretch, softness, contact, etc., within a compact handheld device is very challenging. The below example of handheld haptic from Microsoft is a simple handheld device that is aimed at touching, grasping, and manipulating virtual objects. However, it is too early for the device to be used in the industrial set-up.

Researchers at Microsoft Research’s Redmond, WA Labs, have developed a new handheld haptic controller device known as the CLAW. The CLAW is a multi-functional controller that provides VR functionality controls as well as haptic renderings, including touching, grasping, and manipulating virtual objects. The unique functionality of the device is its ability to recognize differences in the user’s grip and correlate it with the virtual content and adapt to a specific haptic rendering.

Mid-air haptic technology could be the successor to the handheld haptic technology. This allows the user to use the virtual object to touch and feel without the need for any physical devices connected to computers.

The technology works on manipulating ultrasound waves, which are condensed into invisible hotspots (these hotspots can be felt by the user). It is still in the research phase; however, a few corporations and research institutions are trying to make it more viable. Challenges such as continuous buzzing sound produced by ultrasound waves and the tactile nature of virtual objects, which lets users hand pass through the virtual object, are currently hindering the progress of this technology. Ultraleap, a company formed by the merger of Ultrahaptics and LEAP Motion in May 2019, is currently working on making progress in this technology.

Conclusion – The future is not around the corner

Though several opportunities exist in the manufacturing industry for the use of wearable haptics, currently, its adoption is very limited. This is mainly due to the high level of precision required for virtual objects in the industrial set-up, which requires complex programming, sophisticated hardware, and significant processing power. Therefore, manufacturers of wearable haptics are largely focusing on gaming and commercial sectors. However, research activities have intensified in the field of wearable haptics over the last 3-5 years, with companies such as Valkyrie, Exiii, GoTouch VR, etc., focusing on innovations and improvements in wearable haptic devices. Experts are optimistic about the technology witnessing good adoption in the industrial sector by 2025.

The current focus is on enhancing the features and functionalities of wearable haptic technology to make it more feasible for industrial uses. Though, there are organizations and research institutions investing in research & development activities with respect to handheld and mid-air haptics, the future of these technologies is still uncertain. However, as witnessed in the past, a small breakthrough in technology can change its future course.