FIP Laboratory @Saitama University

Food texture plays a crucial role in the overall sensory experience and the functional properties of food, evolving dynamically from the first bite to the final chew. Traditional methods for modifying food texture, such as adjusting cooking parameters, often compromise other key attributes, including appearance and nutritional value, whereas existing computational approaches largely treat texture as a static, monolithic attribute. This paper presents a computational method that modulates the internal structure of food to design temporal food texture experiences. Our method generates a silicone mold based on the desired food texture experience. This approach decouples textural properties from visual and amount attributes, enabling independent tuning of sensory factors during different oral processing stages. We characterize the relationship between internal structure and temporal texture perception, specifically finding that infill pattern and shell thickness control the bite and chewing phases, respectively. We present an interactive design interface that allows users to create food items tailored to specific temporal food textures.

3D food printing enables the customization of food shapes and textures, but typically produces uniform taste profiles due to the limited diversity of printable materials. We present TastePrint, a 3D food printing system that achieves layer-wise spatial taste distribution by dynamically applying liquid seasonings with a programmable airbrush during fabrication. The system integrates (1) a graphical user interface (GUI) that allows users to import 3D models, slice them into layers, and specify seasoning channels, spray positions, and intensities, and (2) a customized 3D food printer equipped with a multi-nozzle spray mechanism. We evaluated the system through technical experiments quantifying spray resolution and deposition accuracy, a minimal sensory discrimination study on taste localization, and an exploratory formative user-feedback study involving three home cooks. The spray-resolution model achieved , and the spray-amount model achieved. The filter-paper calibration showed broad consistency with measurements obtained on edible mashed-potato samples. In the sensory discrimination study, participants identified the centralized seasoning pattern as more localized in 27 of 40 trials (67.5%). These findings indicate that TastePrint can provide repeatable hardware-level control over seasoning placement and quantity while offering initial evidence that spatial taste arrangement can remain perceptually meaningful after fabrication.

Human-Food Interaction (HFI) examines how digital integration can enhance dining experiences, with food tagging playing a crucial role in connecting physical dining environments with digital information. Attaching optical tags to the surface or sides of food often detracts from its aesthetics, negatively impacting the perceived taste and overall dining experience. To address this issue, we propose an unobtrusive food tagging approach that embeds tags inside the food, maintaining both its visual appeal and sensory qualities. We first developed a tagging method using a 3D printer and proposed an end-to-end pipeline for embedding and retrieving the tags. We evaluated this method in terms of tag detectability, concealability, and the eating experience. Additionally, we developed tagging methods using molding and stamping to extend acceptability to the traditional cooking environment. Through a workshop with three home chefs, we found that these methods are accessible and easy to adopt for novice users. Our findings demonstrate the potential of embedded food tagging to integrate digital information into the dining experience without compromising culinary integrity. This approach offers new directions for HFI research and practical applications.

Self-healing polymers, which can autonomously repair damage or be repaired through simple treatments, have garnered attention due to their flexibility in addressing damage and enabling long-term use in soft robotics. Many studies on the application of self-healing polymers to soft robotics have focused on realizing healable systems. Such a mechanism makes it possible to construct soft robots. Despite previous research on reconfigurable soft robots, the reconfiguration of actuators—critical components for determining actuation—has not been explored extensively. We propose a method for constructing reconfigurable soft pneumatic actuators using multi-material self-healing polymers, thus enhancing the adaptability and functionality of soft robotic systems. The actuator comprises stretchable and less-stretchable self-healing polymers, which allow for durable and versatile configurations. By adjusting the placement of the less-stretchable polymer on the actuator surface, we can design and alter the deformation, thus achieving various forms of actuation. Our research demonstrates the feasibility of actuators capable of bending, contracting, twisting, and elongating. We provide examples of their application in actuator fabrication, thus showcasing the potential of our study in inspiring the design and fabrication of soft robots utilizing self-healing polymers.

Hand Redirection (HR) techniques have been widely studied in virtual reality (VR) to accommodate the hand position relative to the user’s real hand. While such HR can be applied in a VR since the real hand does not appear to the user, in augmented reality (AR), the user’s real hand is visible alongside the virtual hand, which might reduce the HR illusion using the VR technique. To this challenge, we conducted two exploratory experiments to evaluate whether VR-based HR techniques can be applied in AR without disrupting user performance or whether there are different elements to be used before the HR in AR. Results indicate minimal impact on behavior, supporting HR’s potential as lightweight pseudo-haptic feedback in AR. Our results provided the potential to apply HR in the AR environment.

Providing olfactory stimuli in virtual reality (VR) environments is a promising approach to enhancing users’ sense of presence. In this study, we proposed a system for delivering olfactory stimuli to users using a drone. We implemented a workflow where the drone autonomously moves away when the user approaches the olfactory spot in the different scenarios. Experimental results showed that closer retreat distances increased users’ awareness of airflow, suggesting that physical sensations influence overall experience.

Accessible Virtual Reality (VR) navigation remains challenging, particularly using subtle hand gestures. We propose and evaluate SubtleNav, a system that enhances conventional hand tracking using real-time motion magnification. SubtleNav aims to enable precise and low-effort steering control in a VR environment. A preliminary study compared control success rates between SubtleNav and conventional hand tracking, showing that SubtleNav improves success rates for less dexterous fingers. However, it also introduces a trade-off, negatively impacting stability during fine control and state maintenance. This work demonstrates the feasibility of video magnification for enhancing subtle VR interactions while highlighting the key sensitivity-stability balance that requires further investigation.

Virtual laboratories (VLs) allow for a safe learning environment in the laboratory. However, they have some restrictions on haptic feedback, such as the perceived weight or force feedback, that limit users’ experiences. Our goal is to enhance the experience of VLs solely with a virtual reality (VR) controller, without requiring additional hardware. This paper explores the potential to utilize vibration feedback from a VR controller, combined with visual feedback, to influence the perceived virtual liquid weight in VR. Unlike prior work that focuses on simulating the weight of rigid objects, our work presents a simulation of virtual liquid weight perception using vibration frequency modulation. Through an exploratory study, our results showed that as vibration frequency increased, participants perceived the virtual liquid weight as heavier, and vice versa. As an initial step, we believe that our results provide the potential to enhance the perception in VLs through commercially available haptic devices and offer new opportunities to strengthen perception, realism and interaction in VLs.

Previous work in Human-Food Interaction has investigated how to embed tags into food. One promising approach involves controlling the internal structure using food 3D printers. However, this method is not widely accessible due to the current limitations of food 3D printers. This paper explores alternative fabrication techniques, molding and stamping, for embedding unobtrusive tags inside foods. Our preliminary evaluations showed that the proposed methods can embed tags and have the potential to employ a wide range of materials with lower costs compared with the food 3D printing technique. These findings suggest that low-cost edible tagging technologies could become more accessible and versatile.

Perceived taste and flavor of food are greatly affected by visual information, and manipulating them in projection mapping (PM) has been explored. This paper focuses on another aspect of the gustatory sense‐food texture. We investigate whether perceived food textures can be modified by the perceptually deforming effect in PM. Through a user study, we found that the perceived softness of a pudding could be significantly increased by amplifying the apparent movement of the pudding. Interestingly, other gustatory perceptions unrelated to food textures were not affected. We believe that this result expands the potential of PM-based modulation of gustatory perceptions.

Personalization of eating such that everyone consumes only what they need allows improving our management of food waste. In this paper, we explore the use of food 3D printing to create perceptual illusions for controlling the level of perceived satiety given a defined amount of calories. We present FoodFab, a system that allows users to control their food intake through modifying a food’s internal structure via two 3D printing parameters: infill pattern and infill density. In two experiments with a total of 30 participants, we studied the effect of these parameters on users’ chewing time that is known to affect people’s feeling of satiety. Our results show that we can indeed modify the chewing time by varying infill pattern and density, and thus control perceived satiety. Based on the results, we propose two computational models and integrate them into a user interface that simplifies the creation of personalized food structures.

We propose an approach for the generation of topology-optimized structures with text-guided appearance stylization. This methodology aims to enrich the concurrent design of a structure’s physical functionality and aesthetic appearance. Users can effortlessly input descriptive text to govern the style of the structure. Our system employs a hash-encoded neural network as the implicit structure representation backbone, which serves as the foundation for the co-optimization of structural mechanical performance, style, and connectivity, to ensure full-color, high-quality 3D-printable solutions. We substantiate the effectiveness of our system through extensive comparisons, demonstrations, and a 3D-printing test.

This paper presents a shadowless projection mapping system for interactive applications in which a target surface is frequently occluded from a projector with a user’s body. We propose a delay-free optical solution for this critical problem. Specifically, as the primary technical contribution, we apply a large format retrotransmissive plate to project images onto the target surface from wide viewing angles. We also tackle technical issues unique to the proposed shadowless principle. First, the retrotransmissive optics inevitably suffer from stray light, which leads to significant contrast degradation of the projected result. We propose to block the stray light by covering the retrotransmissive plate with a spatial mask. Because the mask reduces not only the stray light but the achievable luminance of the projected result, we develop a computational algorithm that determines the shape of the mask to balance the image quality. Second, we propose a touch sensing technique by leveraging the optically bidirectional property of the retrotransmissive plate to support interaction between the user and the projected contents on the target object. We implement a proof-of-concept prototype and validate the above-mentioned techniques through experiments.

Accurately simulating and designing the vibro-acoustic performance of 3D printed structures is a challenging task due to the lack of material elastic properties, particularly when designing without specialized measurement instruments. To tackle this issue, we propose a numerical-experimental estimation method based on modal impact sound, which only requires a microphone for measurement. Specifically, we established a modal impact sound simulation pipeline that considers the material anisotropy and estimated the material elastic property by minimizing the residuals between the simulated and recorded modal impact sound features. Our method outperforms the state-of-the-art modal-sound approach in terms of convergence speed and accuracy, and achieves an average modal-frequency residual of 0.66% across two materials and six infills. We also designed and manufactured a xylophone using the estimated material elasticity for further demonstration and verification.

We present a method that utilizes the infill parameter in the 3D printing process to embed information inside the food that is difficult to recognize with the human eye. Our key idea is to utilize the air space to generate a specific pattern inside the food without changing the model geometry. As a result, our method exploits the patterns that appear as hidden edible tags to store the data and simultaneously adds them to a 3D printing pipeline. Our contribution also includes the framework that connects the user with a data-embedding interface through the food 3D printing process, and the decoding system allows the user to decode the information inside the 3D printed food through backlight illumination and a simple image processing technique. We demonstrate our method through the example application scenarios.

In this study, we propose a method to control the color andpattern on the surface of 3D printed objects using bi-stable ther-mochromic materials. We mixed UV-curing resin with bi-stablethermochromic materials and selectively heated the surface of the3D object with a laser (Fig. 1a). Unlike previous attempts that usesimilar bi-stable mechanism , our method allowsboth 3D printed objects and later selectively change the color of3D printed objects. In addition, we have fabricated various itemsusing our proposed method and explored its application scenarios.

An immersive virtual laboratory (VL) could offer flexibility of time and space, as well as safety, for remote students to conduct laboratory activities through online experiential learning. Recording an instructor’s demonstration inside a VL is an approach that allows students to learn directly from a demonstration. However, students have to learn from a recording while controlling the playback, which requires attention spent on additional spatial and temporal cues. This additional cognitive load could lead to errors during the laboratory procedure. To address these challenges, we have identified four design requirements to reduce attention load in VLs; namely, organized learning steps, improved student sense of co-presence, reduction of task-instructor split-attention, and learning independent of interpersonal distance. Based on these requirements, we have designed and implemented spatial-temporal assistive toolsets for laboratories in a virtual environment, namely XR-LIVE, to reduce cognitive load and enhance learning in an asynchronous shared-space demonstration, implemented based on the setup of a standard civil engineering laboratory. We also analyzed students’ behavior in the VL demonstration to design guidelines applicable to generic VLs.

We present interiqr, a method that utilizes the infill parameter in the 3D printing process to embed information inside the food that is difficult to recognize with the human eye. Our key idea is to utilize the air space or secondary materials to generate a specific pattern inside the food without changing the model geometry. As a result, our method exploits the patterns that appear as hidden edible tags to store the data and simultaneously adds them to a 3D printing pipeline. Our contribution also includes the framework that connects the user with a data-embedding interface through the food 3D printing process, and the decoding system allows the user to decode the information inside the 3D printed food through backlight illumination and a simple image processing technique. Finally, we evaluate the usability of our method under different settings and demonstrate our method through the example application scenarios.

Nature evolves structures like honeycombs at optimized performance with limited material. These efficient structures can be artificially created with the collaboration of structural topology optimization and additive manufacturing. However, the extensive computation cost of topology optimization causes low mesh resolution, long solving time, and rough boundaries that fail to match the requirements for meeting the growing personal fabrication demands and printing capability. Therefore, we propose the neural synthesizing topology optimization that leverages a self-supervised coordinate-based network to optimize structures with significantly shorter computation time, where the network encodes the structural material layout as an implicit function of coordinates. Continuous solution space is further generated from optimization tasks under varying boundary conditions or constraints for users’ instant inference of novel solutions. We demonstrate the system’s efficacy for a broad usage scenario through numerical experiments and 3D printing.

Advancing in inkjet fused deposition modeling (FDM) color 3D printing enables to create dedicated aesthetical appeal. However, the complete fabrication of a target color remains limited due to the unusual mismatch in 3D color management systems. In particular, the 3D aspect that makes the 3D color systems different from standard 2D printing, such as ink and substrate characteristics, viewing conditions, and base materials. Therefore, to the best of our knowledge, there is no suitable established method that supports color reproduction for inkjet FDM color 3D printing. In this paper, we analyze the color profile of an inkjet FDM color 3D printer to obtain a color model that could bridge the gap between a digital design (as an input) and the actual 3D printed results (as an output). We then created the color model by reproducing each color mapped to every possible color pair to determine the closest color between the target and printing colors on the basis of the color difference value, which can be rendered in lieu of the original printing colors. We verify our proposed color model by printing the mapped color and conduct a color measurement to compare it with the target colors. From the experimental results, we showed that our mapped colors can represent those desired by the user with an 80% success rate, which can be matched through controlled conditions.

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF. As the core of our technique, we present a practical calibration device that requires a minimal working volume directly in front of the projector lens regardless of the projector’s focusing distance and aperture size. The device consists of a flat-bed scanner and pinhole-array masks. For calibration, a projector projects a series of structured light patterns in the device. The pinholes directionally decompose the structured light, and only the projected rays that pass through the pinholes hit the scanner plane. For each pinhole, we extract a ray passing through the optical center of the projector. Consequently, we regard the projector as a pinhole projector that projects the extracted rays only, and we calibrate the projector by applying the standard camera calibration technique, which assumes a pinhole camera model. Using a proof-of-concept prototype, we demonstrate that our technique can calibrate projectors with different focusing distances and aperture sizes at the same accuracy as a conventional method. Finally, we confirm that our technique can provide intrinsic parameters accurate enough for a dynamic PM application, even when a projector is placed too far from a projection target for a conventional method to calibrate the projector using a fiducial object of reasonable size.

Stereoscopic projection mapping (PM) allows a user to see a three-dimensional (3D) computer-generated (CG) object floating over physical surfaces of arbitrary shapes around us using projected imagery. However, the current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues, which causes a vergence-accommodation conflict (VAC). Therefore, we propose a multifocal approach to mitigate VAC in stereoscopic PM. Our primary technical contribution is to attach electrically focus-tunable lenses (ETLs) to active shutter glasses to control both vergence and accommodation. Specifically, we apply fast and periodical focal sweeps to the ETLs, which causes the “virtual image” (as an optical term) of a scene observed through the ETLs to move back and forth during each sweep period. A 3D CG object is projected from a synchronized high-speed projector only when the virtual image of the projected imagery is located at a desired distance. This provides an observer with the correct focus cues required. In this study, we solve three technical issues that are unique to stereoscopic PM: (1) The 3D CG object is displayed on non-planar and even moving surfaces; (2) the physical surfaces need to be shown without the focus modulation; (3) the shutter glasses additionally need to be synchronized with the ETLs and the projector. We also develop a novel compensation technique to deal with the “lens breathing” artifact that varies the retinal size of the virtual image through focal length modulation. Further, using a proof-of-concept prototype, we demonstrate that our technique can present the virtual image of a target 3D CG object at the correct depth. Finally, we validate the advantage provided by our technique by comparing it with conventional stereoscopic PM using a user study on a depth-matching task.

Advancements in machine learning have recently enabled the hyper-realistic synthesis of prose, images, audio and video data, in what is referred to as artificial intelligence (AI)-generated media. These techniques offer novel opportunities for creating interactions with digital portrayals of individuals that can inspire and intrigue us. AI-generated portrayals of characters can feature synthesized faces, bodies and voices of anyone, from a fictional character to a historical figure, or even a deceased family member. Although negative use cases of this technology have dominated the conversation so far, in this Perspective we highlight emerging positive use cases of AI-generated characters, specifically in supporting learning and well-being. We demonstrate an easy-to-use AI character generation pipeline to enable such outcomes and discuss ethical implications as well as the need for including traceability to help maintain trust in the generated media. As we look towards the future, we foresee generative media as a crucial part of the ever growing landscape of human–AI interaction. Digitally recreating the likeness of a person used to be a costly and complex process. Through the use of generative models, AI-generated characters can now be made with relative ease. Pataranutaporn et al. discuss in this Perspective how this technology can be used for positive applications in education and well-being.

FDM 3D printers allow massive creativity in personal products, but their potential has been limited due to inability to manipulating material properties. Previous work had demonstrated that the desired roughness could be presented simply by controlling the spatial density of tiny pins on a printed surface. This article offers a means of providing the desired softness perception of a printed surface and the desired roughness to expand the haptic dimension over which a user can exert control. Specifically, we control the softness by manipulating the infill structures of a printed surface. However, it is known that a skin contact area affects softness perception. The roughness, which is controlled by pins’ density, may also affect the perceived softness of a printed surface. Therefore, we investigate how the internal structures and the density of the pins affect softness perception. Through psychophysical experiments, we derive a computational model that estimates the perceived softness from the density of the pins and the infill density of a printed surface.

From full-color objects to functional capacitive artifacts, 3D printing multi-materials became essential to broaden the application areas of digital fabrication. We present Programmable Filament, a novel technique that enables multi-material printing using a commodity FDM 3D printer, requiring no hardware upgrades. Our technique builds upon an existing printing technique in which multiple filament segments are printed and spliced into a single threaded filament. We propose an end-to-end pipeline for 3D printing an object in multi-materials, with an introduction of the design systems for end-users. Optimized for low-cost, single-nozzle FDM 3D printers, the system is built upon our computational analysis and experiments to enhance its validity over various printers and materials to design and produce a programmable filament. Finally, we discuss application examples and speculate the future with its potential, such as custom filament manufacturing on-demand.

Personalization of eating such that everyone consumes only what they need allows improving our management of food waste. In this paper, we explore the use of food 3D printing to create perceptual illusions for controlling the level of perceived satiety given a defined amount of calories. We present FoodFab, a system that allows users to control their food intake through modifying a food’s internal structure via two 3D printing parameters: infill pattern and infill density. In two experiments with a total of 30 participants, we studied the effect of these parameters on users’ chewing time that is known to affect people’s feeling of satiety. Our results show that we can indeed modify the chewing time by varying infill pattern and density, and thus control perceived satiety. Based on the results, we propose two computational models and integrate them into a user interface that simplifies the creation of personalized food structures.

The appearance of fabric motion is suggested to affect the human perception of bending stiffness. This study presents a novel spatial augmented reality, or projection mapping, approach that can visually manipulate the perceived bending stiffness of a fabric. Particularly, we proposed a flow enhancement method that can change the apparent fabric motion by using a simple optical flow analysis technique rather than complex physical simulations for interactive applications. Through a psychophysical experiment, we investigated the relationship between the magnification factor of our flow enhancement and the perceived bending stiffness of a fabric. Furthermore, we constructed a prototype application system that allows users to control the stiffness of a fabric without changing the actual physical fabric. By evaluating the prototype, we confirmed that the proposed technique can manipulate the perceived stiffness of various materials (i.e., cotton, polyester, and mixed cotton and linen) at an average accuracy of 90.3 percent.

Camera drones allow exploring remote scenes that are inaccessible or inappropriate to visit in person. However, these exploration experiences are often limited due to insufficient scene information provided by front cameras, where only 2D images or videos are supplied. Combining a camera drone vision with haptic feedback would augment users’ spatial understandings of the remote environment. But such designs are usually difficult for users to learn and apply, due to the complexity of the system and unfluent UAV control. In this paper, we present a new telepresence system for remote environment exploration, with a drone agent controlled by a VR mid-air panel. The drone is capable of generating real-time location and landmark details using integrated Simultaneous Location and Mapping (SLAM). The SLAMs’ point cloud generations are produced using RGB input, and the results are passed to a Generative Adversarial Network (GAN) to reconstruct 3D remote scenes in real-time. The reconstructed objects are taken advantage of by haptic devices which could improve user experience through haptic rendering. Capable of providing both visual and haptic feedback, our system allows users to examine and exploit remote areas without having to be physically present. An experiment has been conducted to verify the usability of 3D reconstruction result in haptic feedback rendering.

In this paper, we present a method to create re-programmable multi-color textures that are made from a single material only. The key idea builds on the use of photochromic inks that can switch their appearance from transparent to colored when exposed to light of a certain wavelength. By mixing cyan, magenta, and yellow (CMY) photochromic dyes into a single solution and leveraging the different absorption spectra of each dye, we can control each color channel in the solution separately. Our approach can transform single-material fabrication techniques, such as coating, into high-resolution multi-color processes. We discuss the material mixing procedure, modifications to the light source, and the algorithm to control each color channel. We then show the results from an experiment in which we evaluated the available color space and the resolution of our textures. Finally, we demonstrate our user interface that allows users to transfer virtual textures onto physical objects and show a range of application examples.

While virtual reality (VR) tools provide an immersive learning experience for students, it is difficult for an instructor to observe the students’ learning activities in a virtual environment (VE). Thus, it hinders interactions that could occur between the instructor and students, which are usually required in a classroom environment to understand how each student learns. Previous work has added virtual awareness cues that can help a small group of students to collaborate in a VE. However, when the number of students increases, such virtual awareness cues can cause visual clutter and confuse the instructor. We propose ObserVAR, a visualization system that allows the instructor to observe students in a VE at scale. ObserVAR uses augmented reality techniques to visualize each student’s gaze in a VE and improves the instructor’s awareness of the entire class. The visualizations are then optimized to reduce visual clutter in the scene using a force-directed graph drawing algorithm. In designing ObserVAR, we first investigated visualizations that can provide the instructor with an overall awareness of the VE that can be scaled up as the number of users increases. Second, we optimized the visualization of students by leveraging a graph drawing algorithm to reduce the visual clutter in the class scene. We compared the performance of our prototype with some commercially available user interfaces for VE classrooms. In our study, ObserVAR has demonstrated improvement and flexibility in several application scenarios.

In this paper, we propose a different perspective on the use of support material: rather than printing support structures for overhangs, our idea is to make use of its transient nature, i.e. the fact that it can be dissolved when placed in a solvent, such as water. This enables a range of new use cases, such as quickly dissolving and replacing parts of a prototype during design iteration, printing temporary assembly labels directly on the object that leave no marks when dissolved, and creating time-dependent mechanisms, such as fading in parts of an image in a shadow art piece or releasing relaxing scents from a 3D printed structure sequentially overnight. Since we use regular support material (PVA), our approach works on consumer 3D printers without any modifications. To facilitate the design of objects that leverage dissolvable support, we built a custom 3D editor plugin that includes a simulation showing how support material dissolves over time. In our evaluation, our simulation predicted geometries that are statistically similar to the example shapes within 10% error across all samples.

This paper presents a Head Mounted Display (HMD) integrated system, that uses a drone and a virtual hand to help the users explore remote environment. The system allows the users to use hand gestures to control the drone and identify the Objects of Interest (OOI) through tactile feedback. The system uses a Convolutional Neural Network to perform object classification with the drone captured image and provides a virtual hand to realize interaction with the object. Accodingly, tactile feedback is also provided to users’ hands to enhance the virtual hand body ownership. The system aims to help users assess space and objects regardless of body limitations, which could not only benefit elderly or handicapped people, but make potential contributions in environment measurement and daily life as well.

Collaboration between a helper and a worker in a 3D collaborative virtual environment usually requires real-time information sharing, since the worker relies on the timely assistance from the helper. In contrast, collaboration between workers requires them to shift their attention between independent tasks and dependent tasks. In worker-worker collaborations, a real-time updating technique could create excess information, which may be a distraction. In this paper, we compare different information sharing techniques and determine an efficient technique for the collaboration between workers. In our user experiment, participants performed a floor plan design task in a designer and engineer pairing on a desktop VR environment. The results showed that the proposed information sharing technique, in which objects are updated based on local users’ actions, is more suitable than real-time updates. In addition, we discuss design implications that can be applied to different collaborative scenarios.

Recent research has shown how to change the color of existing objects using photochromic materials. These materials can switch their appearance from transparent to colored when exposed to light of a certain wavelength. The color remains even when the object is removed from the light source. The process is fully reversible allowing users to recolor the object as many times as they want. So far, these systems have been limited to single color changes, i.e. changes from transparent to colored. In this paper, we present ColorMod, a method to extend this approach to multi-color changes (e.g., red-to-yellow). We accomplish this using a multi-color pattern with one color per voxel across the surface of the object. When recoloring the object, our system locally activates only those voxels that have the desired color and turns all other voxels off. We describe ColorMod’s hardware/software system and its user interface that comes with a conversion tool for 3D printing as well as a painting tool that matches physical voxels with the desired appearance. We also contribute our own material formula for a 3D-printable photochromic ink.

This paper presents the extended work on LazyNav, a head-free, eyes-free and hands-free mid-air ground navigation control model presented at the IEEE 3D User Interfaces (3DUI) 2015, in particular with a new application to the head-mounted display (HMD). Our mid-air interaction metaphor makes use of only a single pair of the remaining tracked body elements to tailor the navigation. Therefore, the user can navigate in the scene while still being able to perform other interactions with her hands and head, e.g., carrying a bag, grasping a cup of coffee, or observing the content by moving her eyes and locally rotating her head. We design several body motions for navigation by considering the use of non-critical body parts and develop assumptions about ground navigation techniques. Through the user studies, we investigate the motions that are easy to discover, easy to control, socially acceptable, accurate and not tiring. Finally, we evaluate the desired ground navigation features with a prototype application in both a large display (LD) and a HMD navigation scenarios. We highlight several recommendations for designing a particular mid-air ground navigation technique for a LD and a HMD.

We present SoftAR, a novel spatial augmented reality (AR) technique based on a pseudo-haptics mechanism that visually manipulates the sense of softness perceived by a user pushing a soft physical object. Considering the limitations of projection-based approaches that change only the surface appearance of a physical object, we propose two projection visual effects, i.e., surface deformation effect (SDE) and body appearance effect (BAE), on the basis of the observations of humans pushing physical objects (Figure 1). The SDE visualizes a two-dimensional deformation of the object surface with a controlled softness parameter, and BAE changes the color of the pushing hand. Through psychophysical experiments, we confirm that the SDE can manipulate softness perception such that the participant perceives significantly greater softness than the actual softness. Furthermore, fBAE, in which BAE is applied only for the finger area, significantly enhances manipulation of the perception of softness. We create a computational model that estimates perceived softness when SDE+fBAE is applied. We construct a prototype SoftAR system in which two application frameworks are implemented. The softness adjustment allows a user to adjust the softness parameter of a physical object, and the softness transfer allows the user to replace the softness with that of another object.

The sense of softness is an essentialhapticcue that significantly af-fects the impressions of soft objects as well as the assessment of theirmaterial qualities. Softness properties should be designed carefullyfor various soft products, particularly those that must be comfortablefor users, such as furniture (e.g., cushions and sofas), clothes (e.g.,hats and shoes), and plush toys (e.g., dolls). For soft products that useimitation materials (e.g., artificial leather and fake fur), manufacturershave pursued not only textures but also softness of products that aresimilar to those of real materials to achieve high-quality products atlow prices. The food industry also attempts to optimize the softness offood products, which can significantly affect the taste of the products.In social human-robot interaction, the softness of a robot’s skin playsan important role because it determines the close physical interactionbetween robots and humans, and the impressions of the robot.

With the growing interest in natural input devices and virtual reality, mid-air ground navigation is becoming a fundamental interaction for a large collection of application scenarios. While classical input devices (e.g., mouse/keyboard, gamepad, touchscreen) have their own ground navigation standards, natural input techniques still lack acknowledged mechanisms for travelling in a 3D scene. In particular, for most applications, navigation is not the primary interaction. Thus, the user should navigate in the scene while still being able to perform other interactions with her hands, and observe the displayed content by moving her eyes and locally rotating her head. Since most ground navigation scenarios require only two degrees of freedom to move forward or backward and rotate the view to the left or to the right, we propose LazyNav a mid-air ground navigation control model which lets the users hands, eyes or local head orientation completely free, making use of a single pair of the remaining tracked body elements to tailor the navigation. To this end, we design several navigation body motions and study their desired properties, such as being easy to discover, easy to control, socially acceptable, accurate and not tiring. We also develop several assumptions about motions design for ground navigation and evaluate them. Finally, we highlight general advices on mid-air ground navigation techniques.