FlyingHand: extending the range of haptic feedback on virtual hand using drone-based object recognition

ACM Special Interest on Computer Graphics and Interactive Techniques Asia, 2018 DOI

Abstract

This paper presents a Head Mounted Display (HMD) integrated system, that uses a drone and a virtual hand to help the users explore remote environment. The system allows the users to use hand gestures to control the drone and identify the Objects of Interest (OOI) through tactile feedback. The system uses a Convolutional Neural Network to perform object classification with the drone captured image and provides a virtual hand to realize interaction with the object. Accodingly, tactile feedback is also provided to users’ hands to enhance the virtual hand body ownership. The system aims to help users assess space and objects regardless of body limitations, which could not only benefit elderly or handicapped people, but make potential contributions in environment measurement and daily life as well.

Bibtex

@inproceedings{2018-flyinghand,
  title = {{FlyingHand: extending the range of haptic feedback on virtual hand using drone-based object recognition}},
  author = {Tinglin Duan AND Parinya Punpongsanon AND Daisuke Iwai AND Kosuke Sato},
  booktitle = {ACM Special Interest on Computer Graphics and Interactive Techniques Asia},
  year = {2018},
  doi = {10.1145/3283254.328325},
  url = {https://www.fip.ics.saitama-u.ac.jp/pubs/flyinghand}
}
© Since 2023 | Fabrication, Interaction and Perception Laboratory is a member of Department of Information and Computer Sciences, Saitama University.