2026 November 15

Assessing Performance Characteristics and Feasibility of Eyes-Free Thumb-to-Finger Gestural Text Entry

(a) John receives a text on his HMD and replies using a Thumb-to-Finger (T2F) Word Gesture Keyboard (WGK). (b) He either looks at his hand for visual guidance (eyes-on) or (c) types eyes-free using proprioception. Consequently, we evaluate in Study 2 eyes-free performance under two hand-visibility conditions: (d) hand above a virtual table, and (e) hand below the table, where it is fully occluded, requiring participants to rely on proprioception.
(a) John receives a text on his HMD and replies using a Thumb-to-Finger (T2F) Word Gesture Keyboard (WGK). (b) He either looks at his hand for visual guidance (eyes-on) or (c) types eyes-free using proprioception. Consequently, we evaluate in Study 2 eyes-free performance under two hand-visibility conditions: (d) hand above a virtual table, and (e) hand below the table, where it is fully occluded, requiring participants to rely on proprioception.
Type:
Conference
Authors:
Rishav Banerjee, Shariff AM Faleel, Omang Baheti, Khalad Hasan, Pourang Irani
Venue:
VRST '26
Date of publication:
2026 November 15
pdf download:
Abstract:
Thumb-to-finger (T2F) word gesture keyboards, where the user swipes their thumb across the index, middle, and ring fingers to type, offer a promising approach for eyes-free text entry. Prior work has not characterized how T2F gesture input degrades without visual guidance. We investigate the performance characteristics of eyes-free T2F text entry through an initial data collection study with N=12 participants. Offline analysis, using a lightweight decoder, measures the true eyes-free penalty without obfuscation. Results show that combining a language-model error correction and per-user layout calibration recovers much of the text entry accuracy lost in the eyes-free condition. A subsequent evaluation with N=16 participants confirms these findings, with users retaining 61.8% of their eyes-on text entry speed when operating eyes-free after minimal onboarding. Our evaluation establishes a conservative lower bound for eyes-free T2F text entry performance, demonstrating that proprioception-driven T2F interactions are viable for eyes-free text-entry when augmented with context and calibration.
Citation:
Rishav Banerjee, Shariff Am Faleel, Omang Baheti, Khalad Hasan, and Pourang Irani. 2026. Assessing Performance Characteristics and Feasibility of Eyes-Free Thumb-to-Finger Gestural Text Entry. In Proceedings of the 32nd ACM Symposium on Virtual Reality Software and Technology (VRST '26). Association for Computing Machinery, New York, NY, USA, Article 50, 1–11. https://doi.org/10.1145/3822517.3848667

@inproceedings{banerjee26_asses_perfor_charac_feasib_eyes,
author = {Banerjee, Rishav and Faleel, Shariff Am and Baheti, Omang and Hasan, Khalad and Irani, Pourang},
title = {Assessing Performance Characteristics and Feasibility of Eyes-Free Thumb-to-Finger Gestural Text Entry},
year = {2026},
isbn = {9798400728112},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3822517.3848667},
doi = {10.1145/3822517.3848667},
abstract = {Thumb-to-finger (T2F) word gesture keyboards, where the user swipes their thumb across the index, middle, and ring fingers to type, offer a promising approach for eyes-free text entry. Prior work has not characterized how T2F gesture input degrades without visual guidance. We investigate the performance characteristics of eyes-free T2F text entry through an initial data collection study with N=12 participants. Offline analysis, using a lightweight decoder, measures the true eyes-free penalty without obfuscation. Results show that combining a language-model error correction and per-user layout calibration recovers much of the text entry accuracy lost in the eyes-free condition. A subsequent evaluation with N=16 participants confirms these findings, with users retaining 61.8% of their eyes-on text entry speed when operating eyes-free after minimal onboarding. Our evaluation establishes a conservative lower bound for eyes-free T2F text entry performance, demonstrating that proprioception-driven T2F interactions are viable for eyes-free text-entry when augmented with context and calibration.},
booktitle = {Proceedings of the 32nd ACM Symposium on Virtual Reality Software and Technology},
articleno = {50},
numpages = {11},
keywords = {Head Mounted Displays, Text Entry, Eyes-Free},
location = {Sendai, Japan},
series = {VRST '26}
}
© 2026 Shariff Faleel