Investigation of keyboard and speech based text entry on mobile devices

Abstract

This work presents four in-depth empirical investigations on the performance and user experience of three popular mainstream mobile text entry methods: Touch Typing on a Software Keyboard (STK), the Gesture Typing on a Software Keyboard (SGK), and Speech Based Text Entry. The first and third studies are lab-based longitudinal text entry experiments. In the second and fourth studies, we use a new text entry evaluation methodology based on the experience sampling method (ESM). In the ESM based studies, participants installed an Android app on their own mobile phones that periodically sampled their text entry performance and user experience amid their everyday activities for four weeks. The studies show that text can be entered at an average speed of 24 to 41 WPM using software keyboards, and 49 to 59 WPM using speech, depending on the method and the user's experience, with 0.9% to 3.6% character error rates remaining for software keyboard and 3.0% to 5.8% for speech. Error rates of SGK and speech based input are a major challenge; and reducing out-of-vocabulary errors is particularly important. Both typing and speech have strengths, weaknesses, and different individual awareness and preferences. Two-thumb touch typing in a focused setting is particularly effective on STK, whereas one-handed SGK typing with the thumb is particularly effective in more mobile situations. Speech is more effective when convenience and constraints take priority, whereas typing is more preferable in public – due to social concerns, network latency issues and background noise. When exposed, users showed a trend to migrate from STK to SGK. We also conclude that studies in the lab and in the wild can both be informative to reveal different aspects of keyboard and speech based text entry, but used in conjunction is more reliable in comprehensively assessing input technologies of current and future generations.