From Mouse and Keyboard to a Greater Set of Physical Interaction Devices and Modalities
An integral part of the desktop interaction experience is the contribution of physical input devices, in particular the mouse as a pointing device and the keyboard as a device for inputting text and evoking commands (e.g., function keys). Almost all software today is designed to rely on these devices. As the personal information cloud model and multiple device forms begin to evolve, the mouse and keyboard can no longer be the only form of physical interaction device. However, the explicit or implicit assumptions of a pointing device and a keyboard are so broadly and deeply adopted in today's software development that even the Windows Tablet PC, which is quite similar to traditional desktop and laptop computers in form and size, is markedly more difficult to use than its predecessors. Developing novel, potent yet practical interaction methods that are suited to non-desktop forms of computing is a rare opportunity for the user-interface research field, a field that in general values novelty, often at the cost of practicality and real-world impact. Developing novel yet practical interaction methods is a difficult challenge, since the novel interaction methods are expected to match the performance of the mouse and keyboard, but without making use of the same long learning curve. Experienced computer users have spent years improving their typing and desktop interaction skills, so that even some artificial conventions have become natural to most users. For non-keyboard-based input methods to gain acceptance by users, deep research and careful design have to be invested in developing them. Leveraging users' existing desktop experience and skill, interaction methods that are "transplants" from the conventional desktop may provide a safe path. Paradoxically, such transplants are often poor replications of the desktop experience, inhibiting the full potential of non-desktop computing devices. For example, when using a pen to interact with a point-and-click style of a desktop graphic user interface, actions that are rather simple for a mouse-based interface, such as a double click, become more awkward, while the dexterity and expressive power of a pen go wasted.
[...]
As devices become more diverse, the interaction modalities may move beyond pointing, typing, or even pen input. Voice and eye-gaze are two modalities that may be taken advantage of in certain situations (Oviatt 2003). Multimodal interfaces could be particularly effective if contextual information can be drawn from sensing and the personal information cloud, so that these modalities are used cooperatively to their respective advantages.
Progress in the dimension of new input methods faces the challenge of overcoming users' existing mental models, skill sets, and habits. (This also holds for dimension 2 and perhaps many others.) Making changes concerning the interlock of user skills acquired under a set of conventions tends to be very difficult. Using the QWERTY keyboard as a prime example, Paul David argues for a “path dependence” or “lock-in” theory, dubbed qwertynomics, in which an accidental sequence of events may lock technology development into a particular irreversible path (David 1985). The opponents of qwertynomics argue that the qwerty keyboard has not been replaced because there is no convincingly superior alternative to the QWERTY layout, citing human factors research (Liebowitz and Margolis 1990). Regardless of the strength of arguments on either side, innovation concerning user interaction clearly has to either tap users' existing skills and behavior or offer dramatic advantages over conventional practice. Today new forms of computer devices clearly demand alternative input and output methods, but they have to be well researched to be successful.
[...]
As devices become more diverse, the interaction modalities may move beyond pointing, typing, or even pen input. Voice and eye-gaze are two modalities that may be taken advantage of in certain situations (Oviatt 2003). Multimodal interfaces could be particularly effective if contextual information can be drawn from sensing and the personal information cloud, so that these modalities are used cooperatively to their respective advantages.
Progress in the dimension of new input methods faces the challenge of overcoming users' existing mental models, skill sets, and habits. (This also holds for dimension 2 and perhaps many others.) Making changes concerning the interlock of user skills acquired under a set of conventions tends to be very difficult. Using the QWERTY keyboard as a prime example, Paul David argues for a “path dependence” or “lock-in” theory, dubbed qwertynomics, in which an accidental sequence of events may lock technology development into a particular irreversible path (David 1985). The opponents of qwertynomics argue that the qwerty keyboard has not been replaced because there is no convincingly superior alternative to the QWERTY layout, citing human factors research (Liebowitz and Margolis 1990). Regardless of the strength of arguments on either side, innovation concerning user interaction clearly has to either tap users' existing skills and behavior or offer dramatic advantages over conventional practice. Today new forms of computer devices clearly demand alternative input and output methods, but they have to be well researched to be successful.