HCI Bibliography Home | HCI Journals | About TIIS | Journal Info | TIIS Journal Volumes | Detailed Records | RefWorks | EndNote | Hide Abstracts
TIIS Tables of Contents: 01020304

ACM Transactions on Interactive Intelligent Systems 1

Editors:Anthony Jameson; John Riedl
Standard No:ISSN 2160-6455, EISSN 2160-6463
Links:Journal Home Page | ACM Digital Library | Table of Contents
  1. TIIS 2011-10 Volume 1 Issue 1
  2. TIIS 2012-01 Volume 1 Issue 2

TIIS 2011-10 Volume 1 Issue 1

Introduction to the Transactions on Interactive Intelligent Systems BIBAFull-Text 1
  Anthony Jameson; John Riedl
This editorial introduction describes the aims and scope of the ACM Transactions on Interactive Intelligent Systems, explains how it aims to constitute a landmark addition to the publication landscape, and shows how the five articles in this inaugural issue fit into the journal's conception.
Why-oriented end-user debugging of naive Bayes text classification BIBAFull-Text 2
  Todd Kulesza; Simone Stumpf; Weng-Keen Wong; Margaret M. Burnett; Stephen Perona; Andrew Ko; Ian Oberst
Machine learning techniques are increasingly used in intelligent assistants, that is, software targeted at and continuously adapting to assist end users with email, shopping, and other tasks. Examples include desktop SPAM filters, recommender systems, and handwriting recognition. Fixing such intelligent assistants when they learn incorrect behavior, however, has received only limited attention. To directly support end-user "debugging" of assistant behaviors learned via statistical machine learning, we present a Why-oriented approach which allows users to ask questions about how the assistant made its predictions, provides answers to these "why" questions, and allows users to interactively change these answers to debug the assistant's current and future predictions. To understand the strengths and weaknesses of this approach, we then conducted an exploratory study to investigate barriers that participants could encounter when debugging an intelligent assistant using our approach, and the information those participants requested to overcome these barriers. To help ensure the inclusiveness of our approach, we also explored how gender differences played a role in understanding barriers and information needs. We then used these results to consider opportunities for Why-oriented approaches to address user barriers and information needs.
Active multiple kernel learning for interactive 3D object retrieval systems BIBAFull-Text 3
  Steven C. H. Hoi; Rong Jin
An effective relevance feedback solution plays a key role in interactive intelligent 3D object retrieval systems. In this work, we investigate the relevance feedback problem for interactive intelligent 3D object retrieval, with the focus on studying effective machine learning algorithms for improving the user's interaction in the retrieval task. One of the key challenges is to learn appropriate kernel similarity measure between 3D objects through the relevance feedback interaction with users. We address this challenge by presenting a novel framework of Active multiple kernel learning (AMKL), which exploits multiple kernel learning techniques for relevance feedback in interactive 3D object retrieval. The proposed framework aims to efficiently identify an optimal combination of multiple kernels by asking the users to label the most informative 3D images. We evaluate the proposed techniques on a dataset of over 10,000 3D models collected from the World Wide Web. Our experimental results show that the proposed AMKL technique is significantly more effective for 3D object retrieval than the regular relevance feedback techniques widely used in interactive content-based image retrieval, and thus is promising for enhancing user's interaction in such interactive intelligent retrieval systems.
Recognizing sketched multistroke primitives BIBAFull-Text 4
  Tracy Hammond; Brandon Paulson
Sketch recognition attempts to interpret the hand-sketched markings made by users on an electronic medium. Through recognition, sketches and diagrams can be interpreted and sent to simulators or other meaningful analyzers. Primitives are the basic building block shapes used by high-level visual grammars to describe the symbols of a given sketch domain. However, one limitation of these primitive recognizers is that they often only support basic shapes drawn with a single stroke. Furthermore, recognizers that do support multistroke primitives place additional constraints on users, such as temporal timeouts or modal button presses to signal shape completion. The goal of this research is twofold. First, we wanted to determine the drawing habits of most users. Our studies found multistroke primitives to be more prevalent than multiple primitives drawn in a single stroke. Additionally, our studies confirmed that threading is less frequent when there are more sides to a figure. Next, we developed an algorithm that is capable of recognizing multistroke primitives without requiring special drawing constraints. The algorithm uses a graph-building and search technique that takes advantage of Tarjan's linear search algorithm, along with principles to determine the goodness of a fit. Our novel, constraint-free recognizer achieves accuracy rates of 96% on freely-drawn primitives.
Multimodal approach to affective human-robot interaction design with children BIBAFull-Text 5
  Sandra Y. Okita; Victor Ng-Thow-Hing; Ravi K. Sarvadevabhatla
Two studies examined the different features of humanoid robots and the influence on children's affective behavior. The first study looked at interaction styles and general features of robots. The second study looked at how the robot's attention influences children's behavior and engagement. Through activities familiar to young children (e.g., table setting, story telling), the first study found that cooperative interaction style elicited more oculesic behavior and social engagement. The second study found that quality of attention, type of attention, and length of interaction influences affective behavior and engagement. In the quality of attention, Wizard-of-Oz (woz) elicited the most affective behavior, but automatic attention worked as well as woz when the interaction was short. The type of attention going from nonverbal to verbal attention increased children's oculesic behavior, utterance, and physiological response. Affective interactions did not seem to depend on a single mechanism, but a well-chosen confluence of technical features.
The SignCom system for data-driven animation of interactive virtual signers: Methodology and Evaluation BIBAFull-Text 6
  Sylvie Gibet; Nicolas Courty; Kyle Duarte; Thibaut Le Naour
In this article we present a multichannel animation system for producing utterances signed in French Sign Language (LSF) by a virtual character. The main challenges of such a system are simultaneously capturing data for the entire body, including the movements of the torso, hands, and face, and developing a data-driven animation engine that takes into account the expressive characteristics of signed languages. Our approach consists of decomposing motion along different channels, representing the body parts that correspond to the linguistic components of signed languages. We show the ability of this animation system to create novel utterances in LSF, and present an evaluation by target users which highlights the importance of the respective body parts in the production of signs. We validate our framework by testing the believability and intelligibility of our virtual signer.

TIIS 2012-01 Volume 1 Issue 2

Introduction to the special issue on eye gaze in intelligent human-machine interaction BIBAFull-Text 7
  Elisabeth André; Joyce Y. Chai
Given the recent advances in eye tracking technology and the availability of nonintrusive and high-performance eye tracking devices, there has never been a better time to explore new opportunities to incorporate eye gaze in intelligent and natural human-machine communication. In this special issue, we present six articles that cover various aspects of eye gaze in human-machine interaction, including applications of gaze tracking in human-machine interaction, techniques that recognize gaze gestures and render gaze behaviors, and the analysis of gaze behaviors in social interactions.
Gaze guidance reduces the number of collisions with pedestrians in a driving simulator BIBAFull-Text 8
  Laura Pomarjanschi; Michael Dorr; Erhardt Barth
Our study explores the potential of gaze guidance in driving and analyzes eye movements and driving behavior in safety-critical situations. We collected eye movements from subjects instructed to drive predetermined routes in a driving simulator. While driving, the subjects performed various cognitive tasks designed to divert their attention away from the road. The 30 subjects were equally divided in two groups, a control and a gaze guidance group. For the latter, potentially dangerous events, such as a pedestrian suddenly crossing the street, were highlighted with temporally transient gaze-contingent cues, which were triggered if the subject did not look at the pedestrian. For the group that drove with gaze guidance, eye movements have a reduced variability after the gaze-capturing event and shorter reaction times to it. More importantly, gaze guidance leads to a safer driving behavior and a significantly reduced number of collisions.
Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond BIBAFull-Text 9
  Georg Buscher; Andreas Dengel; Ralf Biedert; Ludger V. Elst
Reading is one of the most frequent activities of knowledge workers. Eye tracking can provide information on what document parts users read, and how they were read. This article aims at generating implicit relevance feedback from eye movements that can be used for information retrieval personalization and further applications.
   We report the findings from two studies which examine the relation between several eye movement measures and user-perceived relevance of read text passages. The results show that the measures are generally noisy, but after personalizing them we find clear relations between the measures and relevance. In addition, the second study demonstrates the effect of using reading behavior as implicit relevance feedback for personalizing search. The results indicate that gaze-based feedback is very useful and can greatly improve the quality of Web search. The article concludes with an outlook introducing attentive documents keeping track of how users consume them. Based on eye movement feedback, we describe a number of possible applications to make working with documents more effective.
Gliding and saccadic gaze gesture recognition in real time BIBAFull-Text 10
  David Rozado; Javier S. Agustin; Francisco B. Rodriguez; Pablo Varona
Eye movements can be consciously controlled by humans to the extent of performing sequences of predefined movement patterns, or 'gaze gestures'. Gaze gestures can be tracked noninvasively employing a video-based eye tracking system. Gaze gestures hold the potential to become an emerging input paradigm in the context of human-computer interaction (HCI) as low-cost eye trackers become more ubiquitous. The viability of gaze gestures as an innovative way to control a computer rests on how easily they can be assimilated by potential users and also on the ability of machine learning algorithms to discriminate in real time intentional gaze gestures from typical gaze activity performed during standard interaction with electronic devices. In this work, through a set of experiments and user studies, we evaluate the performance of two different gaze gestures modalities, gliding gaze gestures and saccadic gaze gestures, and their corresponding real-time recognition algorithms, Hierarchical Temporal Memory networks and the Needleman-Wunsch algorithm for sequence alignment. Our results show that a specific combination of gaze gesture modality, namely saccadic gaze gestures, and recognition algorithm, Needleman-Wunsch, allows for reliable usage of intentional gaze gestures to interact with a computer with accuracy rates higher than 95% and completion speeds of around 1.5 to 2.5 seconds per gesture. The optimal gaze gesture modality and recognition algorithm do not interfere with otherwise standard human-computer gaze interaction, generating very few false positives during real time recognition and positive feedback from the users. These encouraging results and the low cost eye tracking equipment used, open up a new HCI paradigm for the fields of accessibility and interaction with smartphones, tablets, projected displays and traditional desktop computers.
Taming Mona Lisa: Communicating gaze faithfully in 2D and 3D facial projections BIBAFull-Text 11
  Samer Al Moubayed; Jens Edlund; Jonas Beskow
The perception of gaze plays a crucial role in human-human interaction. Gaze has been shown to matter for a number of aspects of communication and dialogue, especially for managing the flow of the dialogue and participant attention, for deictic referencing, and for the communication of attitude. When developing embodied conversational agents (ECAs) and talking heads, modeling and delivering accurate gaze targets is crucial. Traditionally, systems communicating through talking heads have been displayed to the human conversant using 2D displays, such as flat monitors. This approach introduces severe limitations for an accurate communication of gaze since 2D displays are associated with several powerful effects and illusions, most importantly the Mona Lisa gaze effect, where the gaze of the projected head appears to follow the observer regardless of viewing angle. We describe the Mona Lisa gaze effect and its consequences in the interaction loop, and propose a new approach for displaying talking heads using a 3D projection surface (a physical model of a human head) as an alternative to the traditional flat surface projection. We investigate and compare the accuracy of the perception of gaze direction and the Mona Lisa gaze effect in 2D and 3D projection surfaces in a five subject gaze perception experiment. The experiment confirms that a 3D projection surface completely eliminates the Mona Lisa gaze effect and delivers very accurate gaze direction that is independent of the observer's viewing angle. Based on the data collected in this experiment, we rephrase the formulation of the Mona Lisa gaze effect. The data, when reinterpreted, confirms the predictions of the new model for both 2D and 3D projection surfaces. Finally, we discuss the requirements on different spatially interactive systems in terms of gaze direction, and propose new applications and experiments for interaction in a human-ECA and a human-robot settings made possible by this technology.
Conversational gaze mechanisms for humanlike robots BIBAFull-Text 12
  Bilge Mutlu; Takayuki Kanda; Jodi Forlizzi; Jessica Hodgins; Hiroshi Ishiguro
During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles of interlocutors, managing speaker turns, and signaling discourse structure. If humanlike robots are to have fluent conversations with people, they will need to use these gaze mechanisms effectively. The current work investigates people's use of key conversational gaze mechanisms, how they might be designed for and implemented in humanlike robots, and whether these signals effectively shape human-robot conversations. We focus particularly on whether humanlike gaze mechanisms might help robots signal different participant roles, manage turn-exchanges, and shape how interlocutors perceive the robot and the conversation. The evaluation of these mechanisms involved 36 trials of three-party human-robot conversations. In these trials, the robot used gaze mechanisms to signal to its conversational partners their roles either of two addressees, an addressee and a bystander, or an addressee and a nonparticipant. Results showed that participants conformed to these intended roles 97% of the time. Their conversational roles affected their rapport with the robot, feelings of groupness with their conversational partners, and attention to the task.
Adaptive eye gaze patterns in interactions with human and artificial agents BIBAFull-Text 13
  Chen Yu; Paul Schermerhorn; Matthias Scheutz
Efficient collaborations between interacting agents, be they humans, virtual or embodied agents, require mutual recognition of the goal, appropriate sequencing and coordination of each agent's behavior with others, and making predictions from and about the likely behavior of others. Moment-by-moment eye gaze plays an important role in such interaction and collaboration. In light of this, we used a novel experimental paradigm to systematically investigate gaze patterns in both human-human and human-agent interactions. Participants in the study were asked to interact with either another human or an embodied agent in a joint attention task. Fine-grained multimodal behavioral data were recorded including eye movement data, speech, first-person view video, which were then analyzed to discover various behavioral patterns. Those patterns show that human participants are highly sensitive to momentary multimodal behaviors generated by the social partner (either another human or an artificial agent) and they rapidly adapt their gaze behaviors accordingly. Our results from this data-driven approach provide new findings for understanding micro-behaviors in human-human communication which will be critical for the design of artificial agents that can generate human-like gaze behaviors and engage in multimodal interactions with humans.